In-park dynamic parking space detection and guidance system and method based on video image analysis

Through video image analysis technology, three-dimensional models and electronic maps of parking lots are built, and parking space status is identified in real time and the optimal path is planned, which solves the problems of low efficiency and insufficient real-time detection of traditional parking spaces, and improves parking lot management and user experience.

CN118486191BActive Publication Date: 2024-11-08SUZHOU YOUXIANXIN NETWORK LIFE SERVICE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410947518.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

Traditional parking space detection methods are inefficient and lack real-time and accuracy, and cannot effectively deal with parking lot congestion and insufficient resource utilization.

Method used

A dynamic parking space detection and guidance system for on-site video image analysis is adopted to build a three-dimensional model and electronic map of the parking lot, and use camera data to identify the parking space status in real time, and guide users to find the spare parking space through optimal path planning.

Benefits of technology

It improves the efficiency and accuracy of parking lot management, reduces the time to find parking spaces, optimizes the parking experience, reasonably allocates parking space resources, and alleviates traffic pressure in parking lots.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118486191B_ABST
    Figure CN118486191B_ABST
Patent Text Reader

Abstract

The video image analysis on-site dynamic parking space detection and guidance system and method of the present application relate to the field of parking space detection and positioning technology. By constructing a unique parking space identification code, videos from all cameras in the parking lot are collected, a three-dimensional model of the parking lot is established, the three-dimensional model of the parking lot is projected onto a plane, and an electronic map of the parking lot is determined; video image data of continuous frames in the video are obtained, and the parking space status of each parking space is determined according to the video image data of the continuous frames and the unique parking space identification code; a parking space status recognition model is obtained by training the video image data with the parking space status marked; a polling interval T is set, and real-time video image data is obtained by polling each camera, and the real-time video image data is input into the parking space status recognition model to determine the real-time parking space status of each parking space, and the parking space status in the electronic map is updated; the user's vehicle is guided into the parking space according to the real-time parking space status, and the optimal path to the empty parking space is provided for the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of parking space detection and positioning, and in particular to a system and method for on-site dynamic parking space detection and guidance based on video image analysis. Background Art

[0002] Parking lot management is an important part of urban traffic management. Traditional parking space detection uses ground sensing. When a vehicle is parked on a geomagnetic sensor, the strength of the geomagnetic field changes, which is detected by the sensor. Then, the occupancy of the parking space is determined by the sensor signal.

[0003] The advantage of traditional parking space detection is high accuracy, but the installation and maintenance costs are high, and it may be affected by ground construction and environmental changes. Traditional parking lot management usually relies on methods such as manual patrols and manual counting. However, this method has many defects. First, manual patrols and manual counting are inefficient, prone to omissions and repeated counting, resulting in insufficient utilization of parking lot resources. Secondly, parking lot management lacks real-time and precision, and cannot accurately grasp the real-time status of parking spaces and parking needs, resulting in poor user experience. Traditional methods are also unable to effectively deal with problems such as parking lot congestion and inefficient management, affecting the smoothness of urban traffic and the utilization efficiency of parking lots.

[0004] In order to solve the problems existing in traditional parking lot management, it is necessary to introduce advanced technical means to improve parking lot management and user parking experience. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose a dynamic parking space detection and guidance system and method based on video image analysis, so as to improve the efficiency and accuracy of parking lot management.

[0006] One aspect of the present application provides a method for detecting and guiding dynamic parking spaces in a parking lot by analyzing video images, comprising:

[0007] Step S100: construct a unique parking space identification code, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot;

[0008] The specific method of constructing a unique parking space identification code, collecting videos from all cameras in the parking lot, establishing a three-dimensional model of the parking lot, projecting the three-dimensional model of the parking lot onto a plane, and determining an electronic map of the parking lot is as follows:

[0009] Step S110: using a laser radar to obtain point cloud data of the parking lot and construct a three-dimensional model of the parking lot;

[0010] Step S120: marking semantic information of parking spaces, driving roads, and obstacles on the three-dimensional model of the parking lot;

[0011] Step S130: establishing a projection mapping of the three-dimensional model to the plane according to the internal and external parameters of the camera;

[0012] The specific method of establishing the projection mapping of the three-dimensional model to the plane according to the internal and external parameters of the camera is:

[0013] Step S131: Obtain the intrinsic parameters and extrinsic parameters of the camera to obtain the intrinsic parameter matrix K and the extrinsic parameter matrix ;

[0014] The internal parameters in the internal parameter matrix include: the focal length (fx, fy) and optical center (cx, cy) of the camera. The internal parameter matrix is ;

[0015] The extrinsic parameters in the extrinsic parameter matrix include: the rotation matrix R of the camera and the translation vector t;

[0016] Step S132: converting the point (X, Y, Z) in the three-dimensional model into the camera coordinate system to obtain the coordinates (x, y, z) of the point in the camera coordinate system;

[0017] The point (X, Y, Z) in the three-dimensional model is converted to the camera coordinate system to obtain the calculation formula of the point coordinate (x, y, z) in the camera coordinate system: ;

[0018] Step S133: Project the point in the camera coordinate system onto a two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane. ;

[0019] The point in the camera coordinate system is projected onto a two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane. The calculation formula is: ;

[0020] Step S134: normalize the points in the plane using the intrinsic parameters of the camera Map to pixel coordinates (u, v);

[0021] The points in the normalized plane are The calculation formula mapped to pixel coordinates (u, v) is: ;

[0022] Step S140: Generate a unique parking space identification code for each parking space in the parking lot, and construct an electronic map of the parking lot based on the unique parking space identification code and the projection mapping of the plane;

[0023] Step S200: acquiring video image data of continuous frames in the video, and determining the parking status of each parking space according to the video image data of the continuous frames and the unique parking space identification code;

[0024] The specific method of obtaining the video image data of the continuous frames in the video and determining the parking status of each parking space according to the video image data of the continuous frames and the unique parking space identification code is:

[0025] Step S210: Collecting video image data from all cameras in the parking lot during peak hours and off-peak hours respectively;

[0026] Step S220: Collecting video image data containing different types of vehicles occupying parking spaces, where the different types of vehicles include but are not limited to: sedans, SUVs, and trucks;

[0027] Step S230: manually labeling the video image data according to the parking space status, and labeling the parking space status of each parking space on the video image data, wherein the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving;

[0028] Step S240: for each parking space, integrating different viewing angles of the parking space in different video image data;

[0029] Step S250: for a partially obstructed parking space, searching for video image data of the parking space under the viewing angle of other cameras according to its unique parking space identification code, and unifying and marking the parking space status of the partially obstructed parking space according to the parking space status on the video image;

[0030] Step S300: training a parking space status recognition model using the video image data with the parking space status marked;

[0031] The specific method of training the parking space status recognition model using the video image data with the parking space status marked is:

[0032] Step S310: cropping the image of each parking space to obtain a parking space cropped image, and using the parking space cropped image as input for training a parking space state recognition model;

[0033] Step S320: normalizing the parking space cropped image, and rotating, flipping, and scaling the parking space cropped image to form a data set;

[0034] Step S330: Divide the labeled data set into training samples, verification samples and test samples;

[0035] Preferably, the labeled data set is divided into 70% training samples, 15% validation samples, and 15% test samples;

[0036] Step S340: selecting a convolutional neural network as an initial model for training a parking space state recognition model, using the parking space cropped images as input data for training the parking space state recognition model, using the parking space state annotated corresponding to each parking space cropped image as output data, using accurate identification of the parking space state of each parking space cropped image as a prediction target, using minimization of the total loss of the parking space state recognition model on the training set as a training target, stopping the training when the loss function reaches convergence, and obtaining a parking space state recognition model;

[0037] The calculation formula of the loss function is: , where M represents the total number of parking space status categories, is an indicator variable used to indicate whether the parking space status in the training sample o belongs to category c. represents the probability that the parking space status in the predicted training sample o belongs to category c;

[0038] The total number of categories M of the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving, and M is equal to 4;

[0039] The indicator variable When it is equal to 1, it means that the training sample o belongs to category c. When it is equal to 0, it means that the training sample o does not belong to category c;

[0040] Step S400: setting a polling interval T, obtaining real-time video image data by polling each camera, inputting the real-time video image data into a parking space status recognition model, determining the real-time parking space status of each parking space, and updating the parking space status in the electronic map;

[0041] The specific method for setting the polling interval T is:

[0042] Step S410: defining the result of weighted summation of the system load and data real-time difference of the parking lot camera system as the objective function f(T), defining the value range of the polling interval as a constraint condition, and setting an initial T value;

[0043] The calculation formula of the objective function f(T) is: , where L(T) represents the system load that decreases as the polling interval T increases, α is the trade-off factor of the system load, D(T) represents the data real-time difference that increases as the polling interval T increases, and β is the trade-off factor of the data real-time difference;

[0044] The constraints are: ,in, The minimum value of the polling interval range. The maximum value of the polling interval range;

[0045] The setting of the initial T value refers to randomly generating a group of T values ​​as the initial population;

[0046] Step S420: Use a genetic algorithm to iterate, calculate the objective function f(T) of each T value, set a fitness evaluation threshold, perform a crossover mutation operation on the T value whose objective function is greater than the fitness evaluation threshold, obtain the next generation of T values, and repeat this process. Each time it is repeated, the number of iterations increases by 1;

[0047] Step S430: Set the maximum number of iterations, stop when the number of iterations reaches the maximum number of iterations, and use the T value with the minimum objective function value as the polling interval;

[0048] The specific method of obtaining real-time video image data by polling each camera, inputting the real-time video image data into the parking space status recognition model, determining the real-time parking space status of each parking space, and updating the parking space status in the electronic map is as follows:

[0049] Step S440: Real-time videos of all cameras are acquired once every polling interval T, and the duration of each acquired real-time video is s;

[0050] Step S450: decoding the collected real-time video to obtain a sequence of real-time video image data of continuous frames, using the real-time video image data of continuous frames as input data, and using the parking space state recognition model to recognize the real-time parking space state of each parking space;

[0051] The real-time video image data of continuous frames is used as input data, and the parking space state recognition model is used to identify the parking space state of each parking space. It is necessary to crop the image of each parking space in the real-time video image data to obtain a real-time parking space cropped image, so as to use the parking space state recognition model to identify the real-time parking space state of each parking space;

[0052] Step S460: locating the position of each parking space in the electronic map according to the unique identification code of each parking space, and updating the parking space status of each parking space in the electronic map according to the real-time parking space status identified by the parking space status identification model;

[0053] Step S500: guiding the user's vehicle to a parking space according to the real-time parking space status;

[0054] The specific method of guiding the user's vehicle to a parking space according to the real-time parking space status includes:

[0055] The entrance position m is preset, the channel in the electronic map is used as the edge, the spatial distance between two parking spaces is used as the edge weight, the comprehensive weight of each parking space is calculated, and the parking space with the smallest comprehensive weight is used as the target parking space. Starting from the entrance position, the node with the smallest comprehensive weight among the neighboring nodes of the current node is continuously moved forward until the target parking space is reached;

[0056] The preset entrance position m, the passage in the electronic map as the edge, the spatial distance between two parking spaces as the edge weight, the comprehensive weight of each parking space is calculated, the parking space with the smallest comprehensive weight is taken as the target parking space, and starting from the entrance position, the node with the smallest comprehensive weight among the neighboring nodes of the current node is continuously moved forward until the target parking space is reached. The specific method is:

[0057] Step S510: transmitting the updated parking space status of the electronic map to the user device through the user interface, and obtaining the real-time parking space status of each parking space according to the electronic map;

[0058] Step S520: preset the entry position m as the starting point of the user path planning;

[0059] Step S530: Calculate each parking space Busyness ;

[0060] The busyness The calculation formula is: ,in, A function representing the occupancy of parking spaces, Indicates from parking space The shortest path to the parking lot entrance location m, is the hyperparameter controlling the parking space occupancy term, is the hyperparameter controlling the parking distance term;

[0061] Function of the occupancy of the parking space The function expression is: ;

[0062] Step S540: gridding the electronic map, connecting the passages between parking spaces as edges, with the weight of the edge being the spatial distance between the two parking spaces, and calculating the comprehensive weight of each parking space;

[0063] The specific method for calculating the comprehensive weight of each parking space is:

[0064] The comprehensive weight is calculated by the distance weight and the busyness of the parking space. The distance weight refers to the shortest path from the entrance position m to the parking space. The calculation formula of the comprehensive weight is: , where η is the positive weight of the busyness term;

[0065] Step S550: According to the comprehensive weight, the parking space with the smallest comprehensive weight is selected. As the target parking space, starting from the node at the entrance position, continue to move along the node with the smallest comprehensive weight among the neighboring nodes of the current node until reaching the target parking space, and obtain an optimal path from the entrance position m to the target parking space .

[0066] One aspect of the present application provides a dynamic parking space detection and guidance system for video image analysis, including:

[0067] The electronic map projection module is used to construct a unique identification code for a parking space, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot;

[0068] A data collection and marking module is used to obtain video image data of continuous frames in the video, and determine the parking status of each parking space according to the video image data of the continuous frames and the unique identification code of the parking space;

[0069] A parking space recognition model training module is used to train a parking space status recognition model using video image data with marked parking space status;

[0070] The parking space status real-time update module is used to set the polling interval T, obtain real-time video image data by polling each camera, input the real-time video image data into the parking space status recognition model, determine the real-time parking space status of each parking space, and update the parking space status in the electronic map;

[0071] The optimal path planning module is used to guide the user's vehicle into the parking space according to the real-time parking space status.

[0072] One aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for dynamic parking space detection and guidance in a parking lot by video image analysis are implemented.

[0073] One aspect of the present application provides a readable storage medium storing a computer program suitable for loading by a processor to execute the steps in a method for on-site dynamic parking space detection and guidance based on video image analysis.

[0074] The on-site dynamic parking space detection and guidance system and method based on video image analysis proposed in this application have the following advantages over the prior art:

[0075] Through the real-time updated electronic map, users can easily find vacant parking spaces, thereby reducing the time spent searching for parking spaces in the parking lot, improving parking efficiency and optimizing the parking experience;

[0076] Through parking space status recognition and real-time updated electronic maps, the occupancy and busyness of each parking space in the parking lot can be accurately understood. In this way, managers can adjust parking resources in a timely manner, reasonably allocate parking spaces, and improve the utilization rate of parking lots;

[0077] Through optimal path planning, users can choose the best path to reach an available parking space, avoiding congestion caused by blindly searching for parking spaces in the parking lot, and effectively alleviating traffic pressure in the parking lot;

[0078] The system realizes intelligent management of parking lots through parking space status recognition models and path planning strategies. It can monitor the situation in the parking lot in real time and make intelligent decisions based on real-time data, thereby improving the efficiency and accuracy of parking lot management. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A method flow chart of the on-site dynamic parking space detection and guidance method based on video image analysis provided in this application;

[0080] Figure 2 A timing diagram of the on-site dynamic parking space detection and guidance system for video image analysis provided in this application;

[0081] Figure 3 Functional module diagram of the on-site dynamic parking space detection and guidance system based on video image analysis provided in this application;

[0082] Figure 4 A schematic diagram of the structure of an electronic device provided in this application;

[0083] Figure 5 It is a schematic diagram of the structure of a readable storage medium provided by this application. DETAILED DESCRIPTION

[0084] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0085] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0086] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0087] Unless otherwise defined, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that unless there is a clear explanation in this application, words defined in commonly used dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0088] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0089] Example 1

[0090] like Figure 1 As shown, the method for detecting and guiding dynamic parking spaces in a parking lot by analyzing video images provided by the present application includes:

[0091] Step S100: construct a unique parking space identification code, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot;

[0092] The camera covers the parking area of ​​the entire parking lot and captures the video stream of the parking area in real time;

[0093] The specific method of constructing a unique parking space identification code, collecting videos from all cameras in the parking lot, establishing a three-dimensional model of the parking lot, projecting the three-dimensional model of the parking lot onto a plane, and determining an electronic map of the parking lot is as follows:

[0094] Step S110: using a laser radar to obtain point cloud data of the parking lot and construct a three-dimensional model of the parking lot;

[0095] Step S120: marking semantic information of parking spaces, driving roads, and obstacles on the three-dimensional model of the parking lot;

[0096] The specific method of marking the semantic information of parking spaces, driving roads, and obstacles on the three-dimensional model of the parking lot is: manually marking the position, size, and shape of each parking space in the three-dimensional model, marking the driving roads and obstacles in the parking lot, and the obstacles include pillars, walls, and trees;

[0097] Add semantic information to the marked objects, including marking parking spaces as "empty parking spaces", "occupied parking spaces", "vehicles parking in", "vehicles leaving", marking driving roads as "main roads", and marking obstacles as "pillars", "walls" or "trees";

[0098] Step S130: establishing a projection mapping of the three-dimensional model to the plane according to the internal and external parameters of the camera;

[0099] The specific method of establishing the projection mapping of the three-dimensional model to the plane according to the internal and external parameters of the camera is:

[0100] Step S131: Obtain the intrinsic parameters and extrinsic parameters of the camera to obtain the intrinsic parameter matrix K and the extrinsic parameter matrix ;

[0101] The internal parameters in the internal parameter matrix include: the focal length (fx, fy) and optical center (cx, cy) of the camera. The internal parameter matrix is ;

[0102] The extrinsic parameters in the extrinsic parameter matrix include: the rotation matrix R of the camera and the translation vector t;

[0103] Step S132: converting the point (X, Y, Z) in the three-dimensional model into the camera coordinate system to obtain the coordinates (x, y, z) of the point in the camera coordinate system;

[0104] The point (X, Y, Z) in the three-dimensional model is converted to the camera coordinate system to obtain the calculation formula of the point coordinate (x, y, z) in the camera coordinate system: ;

[0105] Step S133: Project the point in the camera coordinate system onto a two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane. ;

[0106] The point in the camera coordinate system is projected onto a two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane. The calculation formula is: ;

[0107] Step S134: normalize the points in the plane using the intrinsic parameters of the camera Map to pixel coordinates (u, v);

[0108] The points in the normalized plane are The calculation formula mapped to pixel coordinates (u, v) is: ;

[0109] Step S140: Generate a unique parking space identification code for each parking space in the parking lot, and construct an electronic map of the parking lot based on the unique parking space identification code and the projection mapping of the plane;

[0110] Exemplarily, the parking space unique identification code is in the form of numbers, letters or a combination of codes, and each parking space has only one parking space unique identification code;

[0111] Step S200: acquiring video image data of continuous frames in the video, and determining the parking status of each parking space according to the video image data of the continuous frames and the unique parking space identification code;

[0112] The specific method of obtaining the video image data of the continuous frames in the video and determining the parking status of each parking space according to the video image data of the continuous frames and the unique parking space identification code is:

[0113] Step S210: Collecting video image data from all cameras in the parking lot during peak hours and off-peak hours respectively;

[0114] Step S220: Collecting video image data containing different types of vehicles occupying parking spaces, where the different types of vehicles include but are not limited to: sedans, SUVs, and trucks;

[0115] Step S230: manually labeling the video image data according to the parking space status, and labeling the parking space status of each parking space on the video image data, wherein the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving;

[0116] Step S240: for each parking space, integrating different viewing angles of the parking space in different video image data;

[0117] Step S250: for a partially obstructed parking space, searching for video image data of the parking space under the viewing angle of other cameras according to its unique parking space identification code, and unifying and marking the parking space status of the partially obstructed parking space according to the parking space status on the video image;

[0118] Step S300: training a parking space status recognition model using the video image data with the parking space status marked;

[0119] The specific method of training the parking space status recognition model using the video image data with the parking space status marked is:

[0120] Step S310: cropping the image of each parking space to obtain a parking space cropped image, and using the parking space cropped image as input for training a parking space state recognition model;

[0121] Step S320: normalizing the parking space cropped image, and rotating, flipping, and scaling the parking space cropped image to form a data set;

[0122] Step S330: Divide the labeled data set into training samples, verification samples and test samples;

[0123] Preferably, the labeled data set is divided into 70% training samples, 15% validation samples, and 15% test samples;

[0124] Step S340: selecting a convolutional neural network as an initial model for training a parking space state recognition model, using the parking space cropped images as input data for training the parking space state recognition model, using the parking space state annotated corresponding to each parking space cropped image as output data, using accurate identification of the parking space state of each parking space cropped image as a prediction target, using minimization of the total loss of the parking space state recognition model on the training set as a training target, stopping the training when the loss function reaches convergence, and obtaining a parking space state recognition model;

[0125] The calculation formula of the loss function is: , where M represents the total number of parking space status categories, is an indicator variable used to indicate whether the parking space status in the training sample o belongs to category c. represents the probability that the parking space status in the predicted training sample o belongs to category c;

[0126] The total number of categories M of the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving, and M is equal to 4;

[0127] The indicator variable When it is equal to 1, it means that the training sample o belongs to category c. When it is equal to 0, it means that the training sample o does not belong to category c;

[0128] Step S400: setting a polling interval T, obtaining real-time video image data by polling each camera, inputting the real-time video image data into a parking space status recognition model, determining the real-time parking space status of each parking space, and updating the parking space status in the electronic map;

[0129] The polling interval T refers to the interval duration of obtaining the video of the camera each time;

[0130] The specific method for setting the polling interval T is:

[0131] Step S410: defining the result of weighted summation of the system load and data real-time difference of the parking lot camera system as the objective function f(T), defining the value range of the polling interval as a constraint condition, and setting an initial T value;

[0132] The calculation formula of the objective function f(T) is: , where L(T) represents the system load that decreases as the polling interval T increases, α is the trade-off factor of the system load, D(T) represents the data real-time difference that increases as the polling interval T increases, and β is the trade-off factor of the data real-time difference;

[0133] The constraints are: ,in, The minimum value of the polling interval range. The maximum value of the polling interval range;

[0134] The setting of the initial T value refers to randomly generating a group of T values ​​as the initial population;

[0135] The system load weighing factor α and the data real-time difference weighing factor β are set according to actual application requirements;

[0136] Step S420: Use a genetic algorithm to iterate, calculate the objective function f(T) of each T value, set a fitness evaluation threshold, perform a crossover mutation operation on the T value whose objective function is greater than the fitness evaluation threshold, obtain the next generation of T values, and repeat this process. Each time it is repeated, the number of iterations increases by 1;

[0137] Step S430: Set the maximum number of iterations, stop when the number of iterations reaches the maximum number of iterations, and use the T value with the minimum objective function value as the polling interval;

[0138] The maximum number of iterations γ is set according to the actual situation and the experience of professionals;

[0139] The specific method of obtaining real-time video image data by polling each camera, inputting the real-time video image data into the parking space status recognition model, determining the real-time parking space status of each parking space, and updating the parking space status in the electronic map is as follows:

[0140] Step S440: Real-time videos of all cameras are acquired once every polling interval T, and the duration of each acquired real-time video is s;

[0141] Step S450: decoding the collected real-time video to obtain a sequence of real-time video image data of continuous frames, using the real-time video image data of continuous frames as input data, and using the parking space state recognition model to recognize the real-time parking space state of each parking space;

[0142] The real-time video image data of continuous frames is used as input data, and the parking space state recognition model is used to identify the parking space state of each parking space. It is necessary to crop the image of each parking space in the real-time video image data to obtain a real-time parking space cropped image, so as to use the parking space state recognition model to identify the real-time parking space state of each parking space;

[0143] Step S460: locating the position of each parking space in the electronic map according to the parking space unique identification code of each parking space, and updating the parking space status of each parking space in the electronic map according to the real-time parking space status identified by the parking space status identification model;

[0144] Step S500: guiding the user's vehicle to a parking space according to the real-time parking space status;

[0145] The specific method for guiding the user's vehicle to the parking space according to the real-time parking space status is as follows: presetting the entrance position m, taking the channel in the electronic map as the edge, and the spatial distance between two parking spaces as the edge weight, calculating the comprehensive weight of each parking space, taking the parking space with the smallest comprehensive weight as the target parking space, starting from the entrance position, and continuously moving along the node with the smallest comprehensive weight among the neighboring nodes of the current node until reaching the target parking space;

[0146] The preset entrance position m, the passage in the electronic map as the edge, the spatial distance between two parking spaces as the edge weight, the comprehensive weight of each parking space is calculated, the parking space with the smallest comprehensive weight is taken as the target parking space, and starting from the entrance position, the node with the smallest comprehensive weight among the neighboring nodes of the current node is continuously moved forward until the target parking space is reached. The specific method is:

[0147] Step S510: transmitting the updated parking space status of the electronic map to the user device through the user interface, and obtaining the real-time parking space status of each parking space according to the electronic map;

[0148] Step S520: preset the entry position m as the starting point of the user path planning;

[0149] The entrance position refers to the entrance of the parking lot, the node where vehicles enter the parking lot;

[0150] Step S530: Calculate each parking space Busyness ;

[0151] The busyness refers to the demand rate of parking spaces occupied by vehicles heading to the parking lot. The greater the demand rate occupied by vehicles, the busier the parking space is.

[0152] The busyness The calculation formula is: ,in, A function representing the occupancy of parking spaces, Indicates from parking space The shortest path to the parking lot entrance location m, is the hyperparameter controlling the parking space occupancy term, is the hyperparameter controlling the parking distance term;

[0153] Function of the occupancy of the parking space The function expression is: ;

[0154] When the parking space status is occupied, the vehicle is parking, or the vehicle is leaving, the parking space occupancy status is occupied;

[0155] When the parking space status is empty, the parking space occupancy status is unoccupied;

[0156] and is a positive hyperparameter, which can be set according to actual needs;

[0157] The shortest path Refers to the distance from the parking lot entrance to the parking space The shortest path distance between the driving roads;

[0158] The shortest path Use Dijkstra algorithm to obtain;

[0159] Step S540: gridding the electronic map, connecting the passages between parking spaces as edges, with the weight of the edge being the spatial distance between the two parking spaces, and calculating the comprehensive weight of each parking space;

[0160] The specific method for calculating the comprehensive weight of each parking space is:

[0161] The comprehensive weight is obtained by combining the distance weight and the busyness of the parking space. The distance weight refers to the shortest path from the entrance position m to the parking space. The calculation formula of the comprehensive weight is: , where η is the positive weight of the busyness term;

[0162] Step S550: According to the comprehensive weight, the parking space with the smallest comprehensive weight is selected. As the target parking space, starting from the node at the entrance position, continue to move along the node with the smallest comprehensive weight among the neighboring nodes of the current node until reaching the target parking space, and obtain an optimal path from the entrance position m to the target parking space .

[0163] Example 2

[0164] like Figure 3As shown, the on-site dynamic parking space detection and guidance system for video image analysis provided by this application includes:

[0165] The electronic map projection module is used to construct a unique identification code for a parking space, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot;

[0166] A data collection and marking module is used to obtain video image data of continuous frames in the video, and determine the parking status of each parking space according to the video image data of the continuous frames and the unique identification code of the parking space;

[0167] A parking space recognition model training module is used to train a parking space status recognition model using video image data with marked parking space status;

[0168] The parking space status real-time update module is used to set the polling interval T, obtain real-time video image data by polling each camera, input the real-time video image data into the parking space status recognition model, determine the real-time parking space status of each parking space, and update the parking space status in the electronic map;

[0169] The optimal path planning module is used to guide the user's vehicle into the parking space according to the real-time parking space status;

[0170] like Figure 2 As shown, it is a timing diagram of the on-site dynamic parking space detection and guidance system of video image analysis provided by the present application. The on-site dynamic parking space detection and guidance system of video image analysis collects videos from all cameras in the parking lot, and the electronic map projection module processes these videos, and uses laser radar to establish a three-dimensional point cloud model of the parking lot. According to the layout and characteristics of the parking lot presented in the video, the three-dimensional coordinates of the parking spaces and the roads between the parking spaces are projected onto a two-dimensional plane to form an electronic map of the parking lot;

[0171] The data collection and marking module decodes the video collected by the camera to obtain continuous frames of video image data, manually marks the parking status of the parking spaces in the image data, uses the marked video image data as training data, and trains the parking status recognition model by the parking recognition model training module to obtain the parking status recognition model, and deploys it in the parking status real-time update module;

[0172] The parking space status real-time update module acquires real-time video from the camera device in a loop within each polling interval T, uses the parking space status recognition model to recognize the parking space status in the real-time video, and transmits the recognized parking space status to the electronic map for real-time update;

[0173] A user interface is set up in the optimal path planning module to update the electronic map to the user's device in real time. The optimal path planning module generates an optimal path for the user to reach an empty parking space.

[0174] Example 3

[0175] Figure 4 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 As shown, according to another aspect of the present application, an electronic device is also provided. The electronic device may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the method for detecting and guiding dynamic parking spaces in a parking lot by analyzing video images as described above may be executed.

[0176] The method or system according to the embodiment of the present application can also be used by Figure 4 The electronic device architecture shown in FIG. Figure 4 As shown, the electronic device may include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to a network, an input / output component, a hard disk, and the like. A storage device in the electronic device, such as a ROM or a hard disk, may store the on-site dynamic parking space detection and guidance method based on video image analysis provided in the present application. The on-site dynamic parking space detection and guidance method based on video image analysis may, for example, include: constructing a unique parking space identification code, collecting videos from all cameras in the parking lot, establishing a three-dimensional model of the parking lot, projecting the three-dimensional model of the parking lot onto a plane, and determining an electronic map of the parking lot; obtaining video image data of continuous frames in the video, and determining the parking space status of each parking space based on the video image data of continuous frames and the unique parking space identification code; training a parking space status recognition model using video image data marked with parking space status; setting a polling interval T, obtaining real-time video image data by polling each camera, inputting the real-time video image data into the parking space status recognition model, determining the real-time parking space status of each parking space, and updating the parking space status in the electronic map; guiding the user's vehicle into position based on the real-time parking space status. Furthermore, the electronic device may also include a user interface. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of an electronic device are shown.

[0177] Example 4

[0178] Figure 5 Schematic diagram of a readable storage medium structure provided by an embodiment of the present application. Figure 5As shown, it is a readable storage medium according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, the on-site dynamic parking space detection and guidance method for video image analysis according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0179] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium stores machine-readable instructions, the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, for example: construct a parking space unique identification code, collect videos from all cameras in the parking lot, establish a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot; obtain video image data of continuous frames in the video, and determine the parking space status of each parking space according to the video image data of continuous frames and the parking space unique identification code; train a parking space status recognition model using video image data with marked parking space status; set a polling interval T, obtain real-time video image data by polling each camera, input the real-time video image data into the parking space status recognition model, determine the real-time parking space status of each parking space, and update the parking space status in the electronic map; guide the user's vehicle into the parking space according to the real-time parking space status. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0180] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0181] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0182] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting and guiding dynamic parking spaces in a parking lot based on video image analysis, characterized in that: include: Construct a unique parking space identification code, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot; Acquire video image data of continuous frames in the video, and determine the parking space status of each parking space according to the video image data of the continuous frames and the unique parking space identification code; The parking space status recognition model is obtained by training the video image data with the parking space status marked; A polling interval T is set, and real-time video image data is obtained by polling each camera, and the real-time video image data is input into the parking space status recognition model to determine the real-time parking space status of each parking space, and the parking space status in the electronic map is updated; Guide users’ vehicles into parking spaces based on real-time parking space status; The specific method of setting the polling interval T is as follows: defining the result of weighted summation of the system load and data real-time difference of the parking lot camera system as the objective function f(T), defining the value range of the polling interval as a constraint condition, and setting the initial T value; Genetic algorithm is used for iteration to calculate the objective function f(T) of each T value, set the fitness evaluation threshold, perform crossover mutation operation on the T value whose objective function is greater than the fitness evaluation threshold, obtain the next generation of T value, and repeat the process. The number of iterations increases by 1 each time. Set the maximum number of iterations, stop when the number of iterations reaches the maximum number of iterations, and use the T value with the minimum objective function value as the polling interval; The specific method for guiding the user's vehicle to the parking space according to the real-time parking space status is as follows: presetting the entrance position m, taking the channel in the electronic map as the edge, and the spatial distance between two parking spaces as the edge weight, calculating the comprehensive weight of each parking space, taking the parking space with the smallest comprehensive weight as the target parking space, starting from the entrance position, and continuously moving along the node with the smallest comprehensive weight among the neighboring nodes of the current node until reaching the target parking space; The specific method of constructing a unique parking space identification code, collecting videos from all cameras in the parking lot, establishing a three-dimensional model of the parking lot, projecting the three-dimensional model of the parking lot onto a plane, and determining an electronic map of the parking lot is as follows: Use LiDAR to obtain the point cloud data of the parking lot and build a 3D model of the parking lot; Mark the semantic information of parking spaces, driving roads, and obstacles on the 3D model of the parking lot; According to the internal and external parameters of the camera, a projection mapping of the three-dimensional model to the plane is established; Generate a unique parking space identification code for each parking space in the parking lot, and construct an electronic map of the parking lot based on the unique parking space identification code and the projection mapping of the plane; The specific method of establishing the projection mapping of the three-dimensional model to the image plane according to the internal and external parameters of the camera is: Get the intrinsic and extrinsic parameters of the camera, and get the intrinsic parameter matrix K and the extrinsic parameter matrix ; The internal parameters in the internal parameter matrix include: the focal length (fx, fy) and optical center (cx, cy) of the camera. The internal parameter matrix is ; The extrinsic parameters in the extrinsic parameter matrix include: the rotation matrix R of the camera and the translation vector t; Convert the point (X, Y, Z) in the 3D model to the camera coordinate system to obtain the coordinates (x, y, z) of the point in the camera coordinate system; The point (X, Y, Z) in the three-dimensional model is converted to the camera coordinate system to obtain the calculation formula of the point coordinate (x, y, z) in the camera coordinate system: ; Project the point in the camera coordinate system onto the two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane ; The point in the camera coordinate system is projected onto a two-dimensional normalized plane to obtain the coordinates of the point in the normalized plane. The calculation formula is: ; The points in the normalized plane are normalized by the intrinsic parameters of the camera Map to pixel coordinates (u, v); The points in the normalized plane are The calculation formula mapped to pixel coordinates (u, v) is: ; The specific method of obtaining the video image data of the continuous frames in the video and determining the parking status of each parking space according to the video image data of the continuous frames and the unique parking space identification code is: Collect video image data from all cameras in the parking lot during peak hours and non-peak hours; collect video image data containing different types of vehicles occupying parking spaces, including but not limited to: sedans, SUVs, and trucks; Manually annotate the video image data according to the parking space status, and annotate the parking space status of each parking space on the video image data, wherein the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving; For each parking space, different perspectives of the parking space in different video image data are integrated; For partially obstructed parking spaces, the video image data of the parking spaces are searched from the perspectives of other cameras according to the unique identification code of the parking spaces, and the parking status of the partially obstructed parking spaces is unified and marked according to the parking status on the video image; The specific method of training the parking space status recognition model using the video image data with the parking space status marked is: The image of each parking space is cropped to obtain a parking space cropped image, and the parking space cropped image is used as the input for training a parking space state recognition model; Normalize the cropped images of parking spaces, and rotate, flip, and scale them to form a data set; Divide the labeled data set into training samples, validation samples, and test samples; A convolutional neural network is selected as the initial model for training the parking space state recognition model, and the parking space cropped images are used as the input data for training the parking space state recognition model. The parking space state corresponding to each parking space cropped image is used as the output data. Accurately identifying the parking space state of each parking space cropped image is used as the prediction target. Minimizing the total loss of the parking space state recognition model on the training set is used as the training target. When the loss function reaches convergence, the training is stopped to obtain the parking space state recognition model. The calculation formula of the loss function is: , where M represents the total number of parking space status categories, is an indicator variable used to indicate whether the parking space status in the training sample o belongs to category c. represents the probability that the parking space status in the predicted training sample o belongs to category c; The total number of categories M of the parking space status includes: empty parking space, occupied parking space, vehicle parking in, and vehicle leaving, and M is equal to 4.

2. The method for detecting and guiding dynamic parking spaces in a parking lot by video image analysis according to claim 1, characterized in that: The preset entrance position m, the passage in the electronic map as the edge, the spatial distance between two parking spaces as the edge weight, the comprehensive weight of each parking space is calculated, the parking space with the smallest comprehensive weight is taken as the target parking space, and starting from the entrance position, the node with the smallest comprehensive weight among the neighboring nodes of the current node is continuously moved forward until the target parking space is reached. The specific method is: The updated parking space status of the electronic map is transmitted to the user device through the user interface, and the real-time parking space status of each parking space is obtained according to the electronic map; The preset entry position m is used as the starting point of the user path planning; Calculate each parking space Busyness ; The busyness The calculation formula is: ,in, A function representing the occupancy of parking spaces, Indicates from parking space The shortest path to the parking lot entrance location m, is the hyperparameter controlling the parking space occupancy term, is the hyperparameter controlling the parking distance term; Function of the occupancy of the parking space The function expression is: ; The electronic map is gridded, and the passages between parking spaces are connected as edges. The weight of the edge is the spatial distance between the two parking spaces, and the comprehensive weight of each parking space is calculated; The specific method for calculating the comprehensive weight of each parking space is: The comprehensive weight is calculated by the distance weight and the busyness of the parking space. The distance weight refers to the shortest path from the entrance position m to the parking space. The calculation formula of the comprehensive weight is: , where η is the positive weight of the busyness term; According to the comprehensive weight, the parking space with the smallest comprehensive weight As the target parking space, starting from the node at the entrance position, continue to move along the node with the smallest comprehensive weight among the neighboring nodes of the current node until reaching the target parking space, and obtain an optimal path from the entrance position m to the target parking space .

3. A dynamic parking space detection and guidance system based on video image analysis, used to implement the dynamic parking space detection and guidance method based on video image analysis according to any one of claims 1 to 2, characterized in that: include: The electronic map projection module is used to construct a unique identification code for a parking space, collect videos from all cameras in the parking lot, build a three-dimensional model of the parking lot, project the three-dimensional model of the parking lot onto a plane, and determine an electronic map of the parking lot; A data collection and marking module is used to obtain video image data of continuous frames in the video, and determine the parking status of each parking space according to the video image data of the continuous frames and the unique identification code of the parking space; A parking space recognition model training module is used to train a parking space status recognition model using video image data with marked parking space status; The parking space status real-time update module is used to set the polling interval T, obtain real-time video image data by polling each camera, input the real-time video image data into the parking space status recognition model, determine the real-time parking space status of each parking space, and update the parking space status in the electronic map; The optimal path planning module is used to guide the user's vehicle into the parking space according to the real-time parking space status.

4. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the on-site dynamic parking space detection and guidance method based on video image analysis as described in any one of claims 1-2 are implemented.

5. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the steps in the on-site dynamic parking space detection and guidance method based on video image analysis as described in any one of claims 1-2.