Underground parking lot intelligent management system and method based on deep learning

Abstract

The invention relates to an underground parking lot intelligent management system and method based on deep learning, and the system comprises a database module which stores vehicle sensing data, parking space camera information, parking space information and the like; the image capturing module captures image data of the entrance and exit camera and / or the parking space camera; the vehicle detection and tracking module is used for detecting and tracking vehicles in the image data; the vehicle cutting module is used for identifying a vehicle position in the image data and cutting a vehicle image from the image data; the vehicle information extraction module is used for carrying out information extraction on the vehicle image; the feature matching module is used for comparing the extracted feature information with vehicle features stored in a database so as to realize rapid matching and positioning of the vehicle; and the management module is used for storing and updating the real-time information of the vehicles and the parking spaces in the underground parking lot in real time so as to realize various intelligent management. The invention further discloses an underground parking lot intelligent management method based on deep learning.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and intelligent transportation systems, and in particular to an intelligent management system and method for underground parking lots based on deep learning. Background Art

[0002] With the acceleration of urbanization, the number and scale of underground parking lots are constantly expanding. However, the management of underground parking lots has always been a difficult problem in urban traffic management. The traditional parking lot management method is inefficient and prone to problems such as parking space waste and difficulty in finding vehicles. Therefore, how to achieve intelligent management of underground parking lots, improve parking efficiency, and reduce carbon emissions has become an important topic in the current traffic management field.

[0003] The existing technology mainly collects parking space images through cameras, and then uses image processing technology to identify vehicles and parking spaces. This technology can improve the management efficiency of parking lots to a certain extent, but there are still some problems. First, due to the limitations of image processing technology itself, its accuracy and efficiency are subject to certain restrictions. Secondly, this technology requires a lot of computing resources and storage space, which is costly for large-scale parking lots. In addition, the existing technology still has problems such as single function and low intelligence in terms of vehicle search and parking lot lighting control. Therefore, how to realize the intelligent management of underground parking lots, realize real-time intelligent management, improve parking efficiency, and reduce carbon emissions has become an important topic in the current field of traffic management.

[0004] Since the existing underground parking lot management system has a low level of real-time intelligence and is weak in the overall real-time intelligent management of underground parking lots, it is of great practical significance to develop a new real-time underground parking lot management system.

[0005] Deep learning object detection technology has been used in the field of vehicle detection, demonstrating high accuracy and real-time performance. These technologies can process vehicle images and achieve fast and accurate vehicle recognition, providing a technical foundation for real-time vehicle query systems.

[0006] Feature matching technology based on deep learning is widely used in target recognition, target tracking, image stitching, 3D reconstruction and other fields. These technologies improve the accuracy and robustness of matching by learning image features. Image matching technology based on deep learning is an effective way to achieve information recognition and integration, and it connects low-level vision to high-level vision. In recent years, methods based on deep learning have received widespread attention and progress in the field of image matching, providing a new direction for solving image matching problems.

[0007] In summary, with the development of deep learning technology, it is possible to realize a real-time underground parking management system based on deep learning. These systems can use deep learning algorithms to extract and match features of vehicle images to achieve fast and accurate vehicle identification and query, which is particularly suitable for traffic monitoring, vehicle management and safety inspection. However, there are still challenges in existing technologies, such as the real-time and accuracy of data, user privacy and data security protection, service optimization and query efficiency improvement.

[0008] The information included in this background section of the present specification, including any references cited herein and any description or discussion thereof, is included for technical reference purposes only and is not to be construed as subject matter that will limit the scope of the present invention. Summary of the invention

[0009] The present invention is proposed in view of the above and other more concepts.

[0010] According to the present invention, a real-time underground parking lot management system based on deep learning feature matching is disclosed. The system can use deep learning algorithms to extract and match features of underground parking lot vehicle images to achieve real-time vehicle identification and query, and update the data to the system database in real time. The system can perform real-time intelligent management of the underground parking lot based on these real-time data. The system may include a database module, an image capture module, a vehicle detection and tracking module, a feature extraction module, a feature matching module, and a management module.

[0011] More specifically, according to one aspect of the present invention, there is provided an intelligent management system for an underground parking lot based on deep learning, comprising: a database module configured to store vehicle perception data, parking space camera information, parking space information, area information and electronic equipment information; an image capture module configured to capture image data of entrance and exit cameras and / or parking space cameras; a vehicle detection and tracking module configured to detect and track vehicles in image data using a deep learning target detection algorithm; a vehicle cutting module configured to identify vehicle position information in image data and to cut out vehicle images from image data; a vehicle information extraction module configured to extract information from cut vehicle images using a deep learning algorithm; a feature matching module configured to compare the feature information extracted by the vehicle information extraction module with the vehicle features stored in the database, and perform a secondary comparison in combination with the identified vehicle license plate data to achieve rapid matching and positioning of the vehicle; and a management module configured to save and update real-time information of all vehicles and parking spaces in the underground parking lot to the database in real time, and achieve various intelligent management.

[0012] According to one embodiment, the database module further includes the following information: parking space camera information, including the RTSP stream address, camera number, camera name and camera area information of the surveillance camera in the underground parking lot; parking space information, including the parking space number, parking space category, parking space camera number and location data of the parking space in the parking space camera in the underground parking lot; area information, including the area number, area description and camera data under the area in the underground parking lot; and electronic device information, including the number, category and area coding information of various electronic devices in the underground parking lot.

[0013] According to one embodiment, the image capture module is configured to pull the rtsp stream through the video tools ffmpeg and gstream, so as to obtain the real-time image in the parking camera, and send the real-time image information and the parking camera information corresponding to the real-time image to the vehicle detection and tracking module.

[0014] According to one embodiment, the vehicle detection and tracking module further includes: a vehicle detection module, configured to perform size transformation, normalization and standardization processing on the vehicle image, and input it into the deep learning convolutional neural network for inference, and output the vehicle's location data; and a vehicle tracking module, configured to give and update the tracking number to the vehicle based on the position change of the vehicle in multiple vehicle images.

[0015] According to one embodiment, the vehicle information extraction module includes: a vehicle feature extraction module configured to convert a vehicle image into a 512-dimensional feature matrix through a convolutional neural network; and a vehicle license plate recognition result module configured to recognize the license plate number in the vehicle image as characters.

[0016] According to one embodiment, the vehicle license plate number recognition module further includes: a license plate detection module, configured to detect the position information of the license plate in the vehicle image; a license plate number recognition module, configured to pre-process the cut license plate image and send it to the license plate number recognition module, and output the recognized vehicle license plate number.

[0017] According to one embodiment, the feature matching module is configured to judge the relationship between the position of the vehicle and the parking space, and match the vehicle position information obtained by the vehicle detection and tracking module with the parking space information corresponding to the parking space camera in the database, and the matching algorithm uses the intersection and union ratio and long edge comprehensive judgment method for matching; if the parking space is matched, the vehicle and the parking space are associated in the database, and the data on whether the vehicle is parked, the parking space number of the vehicle, and the location data of the underground parking lot where the vehicle is located are updated to the database.

[0018] According to one embodiment, the feature matching module is further configured to determine whether the vehicle in the latest N frames of images is moving, and update the information on whether the vehicle is active into the database, where N is an integer greater than or equal to 1.

[0019] According to one embodiment, the management module is configured to save and update the real-time information of all vehicles and parking spaces in the underground parking lot to a database, and realize at least one of the following intelligent management functions: real-time scheduling of the underground parking lot; displaying the congestion level of the underground parking lot; real-time display of the occupancy status of each parking space; intelligent guidance of parking; enabling car owners to search for their cars in reverse based on the license plate number; dynamic control of the underground parking lot lights based on whether there are active vehicles in different areas; and providing data support for the real-time dynamic situation of the underground parking lot for three-dimensional reconstruction.

[0020] According to another aspect of the present invention, a method for intelligent management of an underground parking lot based on deep learning is also disclosed, which is characterized in that the method is executed by the underground parking lot intelligent management system as described above, and the method includes: a database module establishes a database, and sets video stream information of each parking space camera, location information of the parking space camera, parking space information, association information between the parking space and the parking space camera, location information of the parking space on the camera page, and information of electronic equipment in the underground parking lot in the database; a data capture module performs real-time image capture of each parking space camera; and sends the real-time captured image to a vehicle detection and tracking module, which uses deep learning convolution. The neural network algorithm detects and tracks the vehicles in the image, and crops the detected and tracked vehicles to obtain the image data of the vehicles; the image data of each vehicle obtained by the vehicle detection and tracking module is sent to the vehicle information extraction module, and the vehicle information extraction module uses the deep learning convolutional neural network algorithm to extract the features of the vehicle to obtain the vehicle feature data, and uses the deep learning license plate recognition algorithm to recognize the vehicle license plate to obtain the vehicle license plate data; the extracted vehicle feature data and license plate data are sent to the feature matching module, and the feature matching module matches the vehicle feature data and license plate data in the database using the cosine similarity matching method, wherein, if the vehicle is matched in the database If the match is successful, the latest data of the vehicle including the license plate data and the vehicle feature data is updated to the database; if the database does not match successfully, the vehicle information is added to the database and the time when the vehicle was last detected is updated; the relationship between the vehicle and the parking space position is judged by the vehicle detection and tracking module, wherein the obtained vehicle position information is matched with the parking space information corresponding to the parking space camera in the database using the intersection and union ratio and long edge comprehensive judgment method; if the parking space is matched, the vehicle and the parking space are associated in the database, and the vehicle's parking data, the vehicle parking space number and the vehicle's underground parking lot location data are updated to the database; based on the vehicle detection and tracking module, the latest N frames are judged Whether there is any movement of the vehicle in the image, and updating the information of whether the vehicle is active into the database, where N is an integer greater than or equal to 1; and, saving and updating the real-time information of all vehicles and parking spaces in the underground parking lot into the database through the management module, and performing at least one of the following intelligent management based on the real-time information in the database: real-time scheduling of the underground parking lot; displaying the congestion level of the underground parking lot; real-time display of the occupancy of each parking space; intelligent guidance of parking; enabling car owners to search for the car in reverse according to the license plate number; dynamically controlling the lights of the underground parking lot according to whether there are active vehicles in different areas; and providing data support for the real-time dynamic situation of the underground parking lot for three-dimensional reconstruction.

[0021] Compared with the prior art, the above technical solutions and embodiments of the present invention have at least the following beneficial effects:

[0022] Improve the efficiency of finding cars and parking spaces: through real-time data monitoring and intelligent scheduling, it can reduce the time car owners spend looking for parking spaces, reduce the time car owners spend looking for vehicles after parking, and improve the turnover rate of parking spaces.

[0023] Optimize resource allocation and improve the intelligence level of underground parking lots: provide data support for big data analysis of underground parking lots, predict parking demand, rationally plan parking space layout, avoid waste of resources, conduct intelligent management and intelligent guidance of underground parking lots, and improve the intelligence of underground parking lots.

[0024] Promote green travel: By reducing ineffective driving caused by searching for parking spaces, carbon emissions are reduced and the greening of urban transportation is promoted.

[0025] Reduce costs: The data capture module, vehicle detection and tracking module, vehicle information extraction module, feature matching module and other modules of the present invention can be implemented through deep learning technology. Compared with traditional image processing technology, the required computing resources and storage space are greatly reduced, thereby reducing costs and being more suitable for use in large-scale parking lots.

[0026] Achieve a higher level of intelligent management: The present invention can realize real-time scheduling of underground parking lots, display the congestion level of underground parking lots, display the occupancy status of each parking space in real time, provide intelligent guidance for parking, and enable car owners to search for their cars in reverse based on license plate numbers. Compared with existing technologies, the present invention has a higher level of intelligence and can greatly improve the management efficiency of underground parking lots and reduce carbon emissions.

[0027] Providing data support: The present invention will save and update the real-time information of all vehicles and parking spaces in the underground parking lot in real time, and provide data support for the real-time dynamic situation of the underground parking lot in three-dimensional reconstruction, which is impossible to achieve with the existing technology. In summary, compared with the existing technology, the present invention has the advantages of higher accuracy and efficiency, lower cost, higher intelligence, dynamic control of parking lot lights, and the ability to provide data support, and has significant advantages.

[0028] Improve accuracy and efficiency: The present invention adopts a deep learning convolutional neural network algorithm to detect and track vehicles in images, and uses a deep learning license plate recognition algorithm to identify vehicle license plates. Compared with traditional image processing technology, its accuracy and efficiency are significantly improved, and it can effectively avoid misidentification and missed identification.

[0029] Dynamically control parking lot lights: The present invention can dynamically control underground parking lot lights according to whether there are active vehicles in different areas, so as to achieve the purpose of energy conservation and emission reduction and reduce carbon emissions. This is a function not available in the prior art.

[0030] More embodiments of the present invention can also achieve other advantageous technical effects that are not listed one by one. These other technical effects may be partially described below and are predictable and understandable to those skilled in the art after reading the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above features and advantages of these embodiments and other features and advantages and the manner in which they are achieved will become more apparent, and embodiments of the present invention may be better understood, by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0032] Figure 1 This is a schematic basic configuration and work process flow chart of a real-time underground parking lot management system solution based on deep learning feature matching according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In the following description of the drawings and specific embodiments, the details of one or more embodiments of the present invention will be described. Other features, objects and advantages of the present invention will be apparent from these descriptions, drawings and claims.

[0034] It should be understood that the illustrated and described embodiments are not limited in application to the details of the construction and arrangement of the components set forth in the following description or illustrated in the accompanying drawings. The illustrated embodiments may be other embodiments and may be implemented or executed in various ways. Each example is provided by explaining the disclosed embodiments rather than limiting them. In fact, it will be apparent to those skilled in the art that various modifications and variations may be made to the embodiments of the present invention without departing from the scope or essence of the present invention. For example, a feature illustrated or described as part of one embodiment may be used together with another embodiment to still produce another embodiment. Therefore, the present invention discloses such modifications and variations that fall within the scope of the appended claims and their equivalent elements.

[0035] Likewise, it is understood that the phrases and terms used herein are for descriptive purposes and should not be considered restrictive. The use of "include," "comprise," or "have" and variations thereof herein is intended to be open-ended to include the items listed thereafter and their equivalents and additional items.

[0036] The present invention will be described in more detail below with reference to several specific embodiments and the accompanying drawings.

[0037] like Figure 1 As shown schematically, Figure 1 The following schematically shows a basic configuration and working process flow chart of a real-time underground parking lot management system solution based on deep learning feature matching according to an embodiment of the present invention.

[0038] According to an embodiment of the present invention Figure 1 The deep learning-based underground parking lot intelligent management system shown may include a database module, an image capture module, a vehicle detection and tracking module, a feature extraction module, a feature matching module and a management module, as further discussed in detail below.

[0039] Database Module :This module can be used to store vehicle perception data obtained based on deep learning feature matching, store the camera information of each parking space in the underground parking lot, each parking space information, area information, electronic equipment information, etc., provide logical judgment support for the deep learning feature matching algorithm, and provide data support for the management module. The database module can further include the following information: (1) Parking space camera information: can include the rtsp stream address, camera number, camera name, underground parking area where the camera is located, etc. of the surveillance camera covering the parking spaces in the entire underground parking lot; (2) Parking space information: can include the parking space number, parking space category (traditional parking space, small car, new energy parking space), the camera number where the parking space is located, and the location data of the parking space in the camera (the upper left corner and lower right corner of the parking space rectangular box under the camera's field of view); (3) Area information: can include area number, area description, and camera data under the area; and (4) Electronic equipment information: can include various electronic equipment, numbers, categories, and area coding information in the underground parking lot, so as to facilitate the subsequent management system to intelligently schedule them.

[0040] Image Capture Module : Underground parking lot cameras are divided into import and export cameras and parking space cameras. The import and export cameras are used to identify vehicles entering and exiting the underground parking lot, and the parking space cameras are used to detect the associated information between vehicles and parking spaces and the real-time dynamic information of the underground parking lot. The image data of the import and export cameras and / or parking space cameras can be captured through the image capture module to provide data support for the subsequent deep learning feature matching algorithm.

[0041] The image capture module can further obtain the parking camera information through the database module, and can obtain the rtsp stream from the parking camera information, as well as the parking space information, area information, electronic equipment information, etc. corresponding to the parking camera. The image capture module can be configured to pull the rtsp stream through the video tools ffmpeg and gstream to obtain the real-time image in the parking camera; and the image information and the parking camera information corresponding to the image can be sent to the vehicle detection and tracking module.

[0042] Vehicle detection and tracking module : The deep learning target detection algorithm can be used to detect vehicles in the monitoring image data and obtain the location information of the vehicle in the image; the multi-target tracking algorithm can be used to track the vehicle, and the vehicle image can be cut according to the location information. The further specific operation steps of the vehicle detection and tracking module can be as follows:

[0043] The vehicle detection module can first perform a size transformation process on the image, transforming the size of the image obtained from the image capture module into the input size of the deep learning convolutional neural network; based on the preprocessing rules of the deep learning convolutional neural network, the size-transformed image can be normalized and standardized again; the normalized and standardized image matrix can be used as input data and input into the deep learning convolutional neural network for inference, thereby outputting the position data of multiple vehicles in the image.

[0044] The location information of multiple vehicles obtained by the vehicle detection module can also be input into the vehicle tracking module. The vehicle tracking module can assign and update tracking numbers to the vehicles based on the position changes of the vehicles in multiple images, wherein the vehicles with the same tracking numbers remain unchanged.

[0045] Vehicle cutting module : The vehicle cutting module can identify the location information of the vehicle in the image and cut the vehicle out of the image.

[0046] Vehicle information extraction module : A deep learning algorithm can be used to extract information from the cut vehicle image. The vehicle information extraction module may include a vehicle feature extraction module and a vehicle license plate recognition result module. The vehicle feature extraction module may convert the vehicle image into a 512-dimensional feature matrix through a convolutional neural network, and the vehicle license plate recognition result module may recognize the license plate number in the vehicle image as characters. The vehicle information extraction module may be further divided into a vehicle feature extraction module and a vehicle license plate number recognition module.

[0047] Among them, the vehicle feature extraction module can use a deep learning convolutional neural network to compress vehicle images. First, the vehicle image can be preprocessed by resizing, normalization, and standardization. Then, the preprocessed vehicle image matrix data can be sent to the deep learning convolutional neural network to convert the vehicle image into a 512-dimensional feature matrix.

[0048] The vehicle license plate number recognition module may include a license plate detection module and a license plate number recognition module. The vehicle license plate number recognition module is configured to perform pre-processing such as size transformation, normalization, and standardization on the vehicle image, and then send the pre-processed vehicle image matrix data to its license plate detection module, which detects the position information of the license plate in the vehicle image; then the license plate can be cut out from the vehicle image, and then the cut license plate image is sent to its license plate number recognition module after pre-processing such as size transformation, normalization, and standardization, and the recognized vehicle license plate number can be output.

[0049] Feature matching module : The extracted features can be compared with the vehicle features stored in the database, and a secondary comparison can be performed in combination with the identified vehicle license plate data to achieve rapid matching and positioning of the vehicle; if the match is successful, the latest vehicle data can be updated to the database, including license plate data, vehicle feature data, etc.; if the database does not match successfully, the vehicle information can be added to the database and the last detected time of the vehicle can be updated.

[0050] The feature matching module is configured to also determine the relationship between the vehicle and the parking space. The vehicle position information obtained by the vehicle detection and tracking module is matched with the parking space information corresponding to the camera in the database. The matching algorithm can use the intersection and union ratio and long edge comprehensive judgment method for matching; if the parking space is matched, the vehicle and the parking space can be associated in the database, and the data on whether the vehicle is parked, the parking space number of the vehicle, and the location data of the underground parking lot where the vehicle is located can be updated to the database.

[0051] Preferably, the feature matching module is configured to also determine whether the vehicle in the latest N frames of images is moving based on and together with the vehicle detection and tracking module, and update the information on whether the vehicle is active into the database.

[0052] Management Module :Based on the above modules, the real-time information of all vehicles and parking spaces in the underground parking lot is saved and updated in real time to the database; based on the real-time updated information in the database, the management module can realize a variety of intelligent management functions, including but not limited to real-time scheduling of the underground parking lot; displaying the congestion level of the underground parking lot; real-time display of the occupancy of each parking space; intelligent guidance of parking; enabling car owners to reversely search for the car based on the license plate number and other functions; dynamically controlling the lights in the underground parking lot according to whether there are active vehicles in different areas, so as to achieve energy conservation and emission reduction and reduce carbon emissions; and providing data support for the real-time dynamic situation of the three-dimensional reconstruction of the underground parking lot.

[0053] Combination Figure 1As shown, an exemplary implementation method of a solution of an underground parking lot intelligent management system based on deep learning according to an embodiment of the present invention includes the following steps:

[0054] First, the database module establishes a database and sets the video stream information of each parking space camera, the location information of the camera in the underground parking lot, the information of each parking space, the association information between the parking space and the camera, the location information of each parking space on the camera page, and the information of each electronic device in the underground parking lot.

[0055] For example, multiple parking cameras can be set up in an underground parking lot, each corresponding to a parking space, for real-time capture of the image of the parking space. The parking camera model can be Hikvision DS-2CD2T55FWD-I5, which can have a high-definition resolution of 5 million pixels and can clearly capture the status of the parking space.

[0056] The data capture module is used to capture real-time images of each parking space camera.

[0057] The real-time image capture is sent to the vehicle detection and tracking module, which uses a deep learning convolutional neural network algorithm to detect and track the vehicles in the image and obtain the image data of each vehicle. For example, the YOLOv3 algorithm can be used for vehicle detection and tracking, which has high accuracy and efficiency in this regard. In addition, the detected and tracked vehicles can be cropped to obtain the image data of each vehicle.

[0058] The image data of each vehicle acquired by the vehicle detection and tracking module is sent to the vehicle information extraction module. The vehicle information extraction module can firstly extract the features of the vehicle using the deep learning convolutional neural network algorithm to acquire 512-dimensional vehicle feature data, and then use the deep learning license plate recognition algorithm to recognize the vehicle license plate and acquire the vehicle license plate data. In this regard, for example, the feature extraction algorithm can use the ResNet50 network, and the license plate recognition algorithm can use the EAST algorithm.

[0059] The extracted vehicle feature data and license plate data are sent to the feature matching module. The feature matching module matches the acquired 512-dimensional vehicle feature data and the vehicle license plate data in the database. First, the feature data is matched. The cosine similarity matching method is used to filter out the first 5 data with a cosine similarity less than 0.2 from the database. At the same time, the license plate data is combined to achieve fast matching and positioning of the vehicle; if the match is successful, the latest data of the vehicle is updated to the database, including the license plate data and vehicle feature data; if the database does not match successfully, the vehicle information is added to the database and the last detected time of the vehicle is updated. The database can use a MySQL database, for example, which has high data processing efficiency and stability.

[0060] The relationship between the vehicle and the parking space is judged, and the vehicle position information obtained by the vehicle detection and tracking module is matched with the parking space information corresponding to the camera in the database. The matching algorithm uses the intersection and union ratio and long edge comprehensive judgment method for matching. If a parking space is matched, the vehicle and the parking space are associated in the database, and the vehicle's parking data, the vehicle's parking space number, and the vehicle's underground parking lot location data are updated to the database.

[0061] Based on the vehicle detection and tracking module, it is determined whether the vehicle has moved in the latest N frames of images, and the information on whether the vehicle is active is updated to the database. Specifically, N can be 10, that is, it is determined whether the vehicle has moved in the past 10 frames of images.

[0062] Based on the above modules, the management module will save and update the real-time information of all vehicles and parking spaces in the underground parking lot to the database. Based on the real-time updated information in the database, the management module can realize a variety of intelligent management functions, including but not limited to real-time scheduling of underground parking lots, displaying the congestion level of underground parking lots, real-time display of the occupancy of each parking space, intelligent guidance of parking, reverse search of vehicles by car owners according to license plate numbers, etc., and dynamic control of underground parking lot lights according to whether there are active vehicles in different areas, so as to achieve energy conservation and emission reduction, reduce carbon emissions, and provide data support for the real-time dynamic situation of the underground parking lot three-dimensional reconstruction. The management module can be developed in Python language, for example, with good scalability and maintainability.

[0063] The above are exemplary working steps for the specific implementation of this embodiment. By adopting the deep learning convolutional neural network algorithm, the recognition accuracy and efficiency of vehicles and parking spaces can be improved. At the same time, by optimizing the algorithm and data processing method, the consumption of computing resources and storage space can be reduced, the cost can be reduced, and various intelligent management functions can be realized, thereby improving the intelligence level of underground parking lots, improving parking efficiency, and reducing carbon emissions.

[0064] Compared with the prior art, the above technical solutions and embodiments of the present invention can solve at least the following technical problems:

[0065] Technical issues with limited accuracy and efficiency: Due to the limitations of image processing technology, existing underground parking management technologies are prone to misidentification and missed identification when identifying vehicles and parking spaces, and their accuracy and efficiency are limited to a certain extent. The technical solution of the present invention improves the accuracy and efficiency of vehicle and parking space identification by adopting a deep learning convolutional neural network algorithm.

[0066] Technical problem of large consumption of computing resources and storage space: Existing underground parking lot management technology requires a large amount of computing resources and storage space, which is costly for large-scale parking lots. The technical solution of the present invention reduces the consumption of computing resources and storage space and reduces costs by optimizing algorithms and data processing methods.

[0067] Technical problems of single function and low intelligence: The existing underground parking lot management technology still has problems such as single function and low intelligence in terms of vehicle search and parking lot lighting control. The technical solution of the present invention improves the intelligence of underground parking lots, improves parking efficiency and reduces carbon emissions by realizing multiple intelligent management functions.

[0068] Due to the advanced nature of the technical solution of the present invention, it can be widely used in the fields of intelligent transportation systems, underground parking lot management, and deep learning technology.

[0069] First, in the field of intelligent transportation systems, the technical solution of the present invention can realize real-time monitoring and management of underground parking lots, improve the utilization efficiency of parking lots, reduce the time and energy consumption of vehicle search, thereby alleviating urban traffic pressure and improving the overall efficiency of the transportation system.

[0070] At the same time, by judging the position relationship between vehicles and parking spaces, it is possible to achieve intelligent guidance of vehicles in the parking lot, reduce the driving distance of vehicles in the parking lot, and further reduce energy consumption and carbon emissions. Secondly, in the field of underground parking lot management, the technical solution of the present invention can realize real-time monitoring and management of vehicles and parking spaces in the parking lot, improve the management efficiency of the parking lot, and reduce labor costs. At the same time, by judging the position relationship between vehicles and parking spaces, it is possible to achieve intelligent guidance of vehicles in the parking lot, improve parking efficiency, reduce the driving distance of vehicles in the parking lot, and reduce energy consumption and carbon emissions.

[0071] In addition, by determining the positional relationship between the vehicle and the parking space, it is also possible to quickly locate the vehicle in the parking lot and perform reverse vehicle search, thus improving the user experience of the car owner.

[0072] Finally, in the field of deep learning technology, the technical solution of the present invention can realize vehicle feature extraction and license plate recognition through deep learning convolutional neural network algorithm and license plate recognition algorithm, improve the accuracy and efficiency of image processing technology, and reduce the consumption of computing resources and storage space.

[0073] In addition, by judging the position relationship between vehicles and parking spaces, it is also possible to achieve intelligent guidance and rapid positioning of vehicles in parking lots, thereby improving the application value of deep learning technology in the fields of intelligent transportation systems and security monitoring.

[0074] In general, the technical solution of the present invention has broad application prospects in the fields of intelligent transportation systems, underground parking lot management, and deep learning technology, with huge market demand, and is expected to promote technological progress and industrial development in related fields.

[0075] The foregoing description of several embodiments of the present invention is presented for illustrative purposes. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise steps and / or forms disclosed, and it is apparent that many modifications and variations may be made in light of the above teachings. The scope of the present invention and all equivalents are intended to be defined by the appended claims.

Claims

1. An intelligent management system for underground parking lots based on deep learning, characterized in that: include: a database module configured to store vehicle perception data, parking camera information, parking space information, area information, and electronic device information; An image capture module configured to capture image data from an entrance / exit camera and / or a parking camera; a vehicle detection and tracking module configured to detect and track vehicles in the image data using a deep learning object detection algorithm; A vehicle cutting module configured to identify vehicle position information in the image data and to cut out a vehicle image from the image data; A vehicle information extraction module configured to extract information from the cut vehicle image using a deep learning algorithm; A feature matching module configured to compare the feature information extracted by the vehicle information extraction module with the vehicle features stored in the database, and to perform a secondary comparison in combination with the identified vehicle license plate data, so as to achieve rapid matching and positioning of the vehicle; and The management module is configured to save and update the real-time information of all vehicles and parking spaces in the underground parking lot into a database in real time, and realize various intelligent management.

2. The underground parking lot intelligent management system according to claim 1 is characterized in that: The database module further contains the following information: Parking camera information, including the rtsp stream address, camera number, camera name and camera location information of the surveillance camera in the underground parking lot; Parking space information, including the parking space number, parking space type, parking space camera number and the location data of the parking space in the parking space camera in the underground parking lot; Zone information, including the zone number, zone description, and camera data under the zone in the underground parking lot; and Electronic device information, including the number, category and area coding information of various electronic devices in the underground parking lot.

3. The underground parking lot intelligent management system according to claim 1 or 2, characterized in that: The image capture module is configured to pull the rtsp stream through the video tools ffmpeg and gstream, so as to obtain the real-time image from the parking camera, and send the real-time image information and the parking camera information corresponding to the real-time image to the vehicle detection and tracking module.

4. The underground parking lot intelligent management system according to claim 1 is characterized in that: The vehicle detection and tracking module further includes: A vehicle detection module is configured to perform size transformation, normalization and standardization on the vehicle image, and input the image into a deep learning convolutional neural network for inference, and output the location data of the vehicle; and The vehicle tracking module is configured to assign and update a tracking number to the vehicle based on the position change of the vehicle in the plurality of vehicle images.

5. The underground parking lot intelligent management system according to claim 1 is characterized in that: The vehicle information extraction module includes: A vehicle feature extraction module configured to convert the vehicle image into a 512-dimensional feature matrix through a convolutional neural network; and The vehicle license plate recognition result module is configured to recognize the license plate number in the vehicle image as characters.

6. The underground parking lot intelligent management system according to claim 5 is characterized in that: The vehicle license plate number recognition module further includes: A license plate detection module configured to detect position information of the license plate in the vehicle image; and The license plate number recognition module is configured to pre-process the cut license plate image and then send it to the license plate number recognition module, and output the recognized vehicle license plate number.

7. The underground parking lot intelligent management system according to claim 1 is characterized in that: The feature matching module is configured to judge the relationship between the vehicle and the parking space, and match the vehicle position information obtained by the vehicle detection and tracking module with the parking space information corresponding to the parking space camera in the database. The matching algorithm uses the intersection and union ratio and long edge comprehensive judgment method for matching; if the parking space is matched, the vehicle and the parking space are associated in the database, and the data on whether the vehicle is parked, the parking space number of the vehicle, and the location data of the underground parking lot where the vehicle is located are updated to the database.

8. The intelligent management system for underground parking lots according to claim 1 or 7, characterized in that: The feature matching module is also configured to determine whether the vehicle in the latest N frames of images is moving, and update the information on whether the vehicle is active into the database, where N is an integer greater than or equal to 1.

9. The underground parking lot intelligent management system according to claim 1, characterized in that: The management module is configured to save and update the real-time information of all vehicles and parking spaces in the underground parking lot to the database in real time, and realize at least one of the following intelligent management functions: Real-time dispatch of underground parking lots; Showing the congestion level of underground parking lots; Display the occupancy of each parking space in real time; Provide intelligent guidance for parking; Enable the car owner to find the car in reverse according to the license plate number; Dynamically control the lighting of underground parking lots according to whether there are active vehicles in different areas; and Provide data support for real-time dynamic situation of underground parking lot three-dimensional reconstruction.

10. A method for intelligent management of underground parking lots based on deep learning, characterized in that: The method is performed by the underground parking lot intelligent management system according to any one of claims 1 to 9, and the method comprises: The database module establishes a database, and sets in the database the video stream information of each parking space camera, the location information of the parking space camera, the parking space information, the association information between the parking space and the parking space camera, the location information of the parking space on the camera page, and the information of the electronic equipment in the underground parking lot; The data capture module is used to capture the real-time images of the cameras in each parking space; The real-time captured image is sent to the vehicle detection and tracking module, which uses a deep learning convolutional neural network algorithm to detect and track the vehicle in the image, and crops the detected and tracked vehicle to obtain the vehicle's image data; The image data of each vehicle acquired by the vehicle detection and tracking module is sent to the vehicle information extraction module. The vehicle information extraction module uses a deep learning convolutional neural network algorithm to extract vehicle features, obtains vehicle feature data, and uses a deep learning license plate recognition algorithm to recognize the vehicle license plate and obtain the vehicle license plate data; The extracted vehicle feature data and license plate data are sent to the feature matching module, and the feature matching module matches the vehicle feature data and the license plate data in the database by using the cosine similarity matching method. If the match is successful in the database, the latest data of the vehicle including the license plate data and the vehicle feature data is updated to the database; if the database does not match successfully, the vehicle information is added to the database and the last detected time of the vehicle is updated; The relationship between the vehicle and the parking space is determined by the vehicle detection and tracking module, wherein the acquired vehicle position information is matched with the parking space information corresponding to the parking space camera in the database by using the intersection and union ratio and long edge comprehensive judgment method. If the parking space is matched, the vehicle and the parking space are associated in the database, and the vehicle's parking data, the vehicle parking space number and the vehicle's underground parking lot location data are updated in the database; Determine whether the vehicle is moving in the latest N frames of images based on the vehicle detection and tracking module, and update the information of whether the vehicle is active into the database, where N is an integer greater than or equal to 1; and The management module saves and updates the real-time information of all vehicles and parking spaces in the underground parking lot to the database in real time, and performs at least one of the following intelligent management based on the real-time information in the database: Real-time dispatch of underground parking lots; Showing the congestion level of underground parking lots; Display the occupancy of each parking space in real time; Provide intelligent guidance for parking; Enable the car owner to find the car in reverse according to the license plate number; Dynamically control the lighting of underground parking lots based on the presence or absence of active vehicles in different areas; and Provide data support for real-time dynamic situation of underground parking lot three-dimensional reconstruction.

Images