A parking lot management method and system based on multi-target intelligent recognition
Through drone aerial photography and multi-object detection algorithms, a three-dimensional model of parking lots is built to realize intelligent monitoring of vehicles and optimal parking space allocation, solving the problems of low efficiency and safety hazards of traditional parking lots, and improving the intelligence of parking lots and Refined management level.
Patent Information
- Application Number
- CN202410805180.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-21
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The management of existing parking lots relies on traditional manual scheduling and monitoring, which is inefficient, inaccurate environmental perception, invalid vehicle supervision, unoptimized resource allocation, and lack of intelligent management, resulting in large errors in safety hazards and cost calculations.
Aerial photography of aerial photography of a drone is used to acquire parking lot images, and a three-dimensional environmental model is built with image recognition algorithms. A multi-objective detection algorithm is used to monitor vehicles in real time, track vehicles and calculate the shortest path for optimal parking space allocation, and use the three-dimensional model to plan navigation paths and analyze vehicle behavior , and the fee details are generated based on the parking time for settlement.
It realizes accurate and intelligent monitoring of vehicles, dynamic optimization of parking space resources, and active detection and response of vehicle abnormal behaviors, improves the intelligent and refined management level of parking lots, and improves operational efficiency and safety.
Smart Images

Figure CN118379902B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a parking lot management method and system based on multi-target intelligent recognition. Background Art
[0002] The current parking lot management mainly adopts the traditional manual scheduling and monitoring mode. Usually, a small number of surveillance cameras are set up, and the on-duty personnel observe the situation through the display. This management method can complete basic on-site supervision, but it is inefficient. On the one hand, it is impossible to process a large number of surveillance images in real time, which is easy to miss; on the other hand, the monitoring perspective is limited, it is impossible to fully observe the complex environment, and the scene information is not accurately obtained. At the same time, there is a lack of active analysis and judgment functions for vehicle behavior, and there are many safety hazards. In addition, traditional management relies too much on manual charging and accounting, and settlement information is only based on manual memory, resulting in large errors in fee calculation.
[0003] In recent years, the development of computer vision, 3D reconstruction, target tracking and pattern recognition technologies has provided hope for building an intelligent parking management system. However, there are still problems to be solved, such as how to deeply integrate multi-source data, realize all-weather intelligent tracking and monitoring, optimize resource allocation and realize active vehicle scheduling, and realize the identification of abnormal vehicle behavior. Summary of the invention
[0004] In view of the fact that the existing parking lot management mainly relies on simple monitoring means, there are problems such as inaccurate environmental perception, ineffective vehicle supervision, non-optimal resource allocation and low management intelligence, the present invention is proposed.
[0005] Therefore, the problem to be solved by the present invention is how to achieve accurate intelligent monitoring of vehicles, dynamic optimization allocation of parking resources, and active detection and response to abnormal vehicle behaviors, so as to achieve a higher level of intelligent and refined management.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] In the first aspect, the embodiment of the present invention provides a parking lot management method based on multi-target intelligent recognition, which includes using drones to obtain parking lot images, and combining image recognition algorithms to build a three-dimensional parking lot environment model; using a method based on The multi-target detection algorithm detects vehicles in real time and matches the owner information based on the detected vehicle information; combined with The algorithm continuously tracks the target vehicle, calculates the available parking spaces corresponding to the shortest path, and makes optimal parking space allocation; it uses a three-dimensional environmental model to plan the optimal parking path and provide navigation guidance, while analyzing and judging vehicle behavior; when the system detects that the vehicle has been safely parked in the designated parking space, it automatically generates a fee breakdown and settles the bill based on the parking time and area.
[0008] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, the construction of the parking lot three-dimensional environment model includes: using 3D reconstruction algorithm, calculates the mapping relationship between feature points in adjacent images to restore the camera shooting position and generate sparse point cloud; The multi-view algorithm generates a high-density 3D point cloud, and uses ground control points and sensor data to geometrically correct the irregular 3D point cloud to make it conform to the actual parking lot size. The semantic segmentation network is used to extract the parking space area in the point cloud, and the boundary is corrected to generate a standardized parking space 3D boundary. The corrected 3D point cloud and the extracted standard parking space are combined to construct an accurate 3D digital model of the parking lot environment. The 3D reconstruction algorithm includes the following formula:
[0009] ;
[0010] in, and Respectively represent the i-th point of the reference point cloud and the point cloud to be registered, and They represent the rotation matrix and the translation vector respectively. represents the weight coefficient, represents the registration error of all point pairs, Indicates the number of matching point pairs in the reference point cloud and the point cloud to be registered.
[0011] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, wherein: constructing a three-dimensional environment model of the parking lot also includes: if the number of feature point matches is small, the mapping relationship is unreliable, or the quality of the sparse point cloud does not meet expectations, recalculating the feature point matches or adjusting the parameters of the three-dimensional reconstruction algorithm; if the point cloud density generated by the multi-view algorithm is insufficient or the quality is poor, adjusting the algorithm parameters or increasing the image view angle to improve the point cloud quality; if the ground control point matching error is large, reprocessing the ground control point data or adjusting the parameters of the correction algorithm; if the sensor data is inaccurate or the noise is large, recalibrating the sensor or filtering the data; if the parking space area extracted by the semantic segmentation network has a large error, re-performing semantic segmentation or optimizing the network model to improve accuracy; if the corrected parking space boundary does not match the actual parking space, re-calibrating or adjusting the parameters of the correction algorithm.
[0012] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, the optimal parking space allocation includes the following steps: for vehicles in the multi-target tracking queue, using The model extracts a unique visual feature signature for each vehicle; when the same vehicle is detected in multiple consecutive frames, a tracker is initialized and the vehicle’s Feature signature; in subsequent frames, newly detected vehicles are compared with existing trackers Feature matching: if the feature signature matches successfully, the status of the tracker is updated; when multiple consecutive frames fail to successfully match the features, the tracker is retained for a certain period of time and then deleted to avoid accumulated false matches; the Kalman filter algorithm is used to smooth the tracking frame and output a continuous and stable vehicle trajectory, and the corresponding tracker is terminated when the vehicle leaves the scene; the information of available parking spaces in the current scene is calculated based on the three-dimensional environment model, and the shortest path to the adjacent available parking space is planned for each tracked vehicle; the owner information and parking space attributes are considered to perform intelligent allocation and dynamic adjustment of parking spaces.
[0013] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, the newly detected vehicle is compared with the existing tracker. Feature matching includes the following steps: For each newly detected vehicle target in a frame, use The model extracts the feature vector of the vehicle; matches the feature vector of the newly detected vehicle with the feature signature of the existing tracker; if the cosine similarity of the corresponding feature signature exceeds the preset threshold, it is considered to be the same vehicle and the feature matching is successful; if the feature matching is successful, the status information of the corresponding tracker including position, speed and appearance is updated; if the newly detected vehicle target cannot be successfully matched with the feature signature of any existing tracker, a new tracker is created as needed and the status information of the new tracker is initialized; for trackers that have not matched features for a long time, determine whether to retain this tracker or terminate it and release related resources based on the set maximum number of tracking frames.
[0014] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, wherein: planning the shortest path to the adjacent vacant parking space for each tracked vehicle includes the following steps: loading a pre-constructed three-dimensional parking lot environment model, which contains coordinate range information of each parking space; retrieving the current position coordinates of each vehicle in the real-time tracking system, and marking the occupied parking spaces in the environment model; calculating the unmarked areas in the environment model as vacant parking spaces, and adjusting and dividing the specific boundaries of the vacant parking spaces according to the parking space spacing requirements; for each tracked vehicle, taking the current position coordinates of the owner as the starting point of path planning; in the navigation network of the environment model, taking the center point of the vacant parking space as the planned end point; using the A* algorithm to calculate the shortest path from the starting point to each end point, and returning the shortest path to the nearest vacant space as the planned path; iteratively executing the steps to dynamically update the vacant parking spaces and the planned path.
[0015] As a preferred solution of the parking lot management method based on multi-target intelligent recognition described in the present invention, the vehicle behavior is analyzed and judged, including: if it is detected that the vehicle's forward direction is opposite to the expected driving direction, it is determined to be reverse driving, and the driver is reminded by a sound alarm to avoid hidden dangers; if it is detected that the vehicle's speed change exceeds the parking lot's speed limit, it is determined to be speeding, and the driver is reminded to slow down by a warning message or a sound alarm, and the speeding vehicle information is monitored and recorded by a camera; if it is detected that the vehicle stays in an area outside the navigation path, it is determined to be illegal parking, and a warning message is sent to the driver, and the vehicle is monitored by a camera to promptly discover and deal with illegal parking behaviors; if it is detected that the vehicle's direction and position in the parking space are inconsistent with the regulations, it is determined to be a parking posture error, and a warning message is sent to the driver or manager to adjust the vehicle posture; if the vehicle is safely parked in the designated parking space, it is determined to be in a safe parking state; repeat the detection until it enters a safe parking state.
[0016] In the second aspect, the embodiment of the present invention provides a parking lot management system based on multi-target intelligent recognition, which includes a model building module for using drone aerial photography to obtain parking lot images and combining image recognition algorithms to build a three-dimensional parking lot environment model; a detection module for using The multi-target detection algorithm monitors vehicles in real time and combines The algorithm continuously tracks the target vehicle; the parking space allocation module is used to match the owner information according to the detected vehicle information, and calculate the available parking spaces corresponding to the shortest path to make the optimal parking space allocation; the judgment module is used to use the three-dimensional environmental model to plan the optimal parking space path and provide navigation guidance, while analyzing and judging the vehicle behavior; the settlement module is used to automatically generate fee details and settle the bill according to the parking time and area after the system detects that the vehicle has been safely parked in the designated parking space.
[0017] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the parking lot management method based on multi-target intelligent recognition as described in the first aspect of the present invention are implemented.
[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the parking lot management method based on multi-target intelligent recognition as described in the first aspect of the present invention are implemented.
[0019] The beneficial effects of the present invention are: it comprehensively utilizes drone low-altitude photogrammetry, image processing and computer vision algorithms, deep learning target detection and tracking, path planning and scheduling algorithms, and vehicle behavior pattern recognition and other technical means to achieve real-time monitoring and analysis of each vehicle's movement process, intelligently plan the best path for the vehicle and navigate to the vacant parking space, actively discover and warn of abnormal vehicle behavior, and perform settlement management based on accurate vehicle entry and exit information. Compared with traditional manual management or monitoring management methods, the present invention is more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 The present invention is a flow chart of a parking lot management method based on multi-target intelligent recognition.
[0022] Figure 2 This is a flow chart of the parking lot three-dimensional environment model of the parking lot management method based on multi-target intelligent recognition.
[0023] Figure 3 Computer equipment diagram for a parking lot management method based on multi-target intelligent recognition. DETAILED DESCRIPTION
[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0027] Example 1
[0028] Reference Figure 1~Figure 3, which is the first embodiment of the present invention, and provides a parking lot management method based on multi-target intelligent recognition, comprising:
[0029] S1: Use drones to obtain parking lot images, and use image recognition algorithms to build a three-dimensional environment model of the parking lot.
[0030] Specifically, choose a six-rotor drone with long flight time and stable shooting capabilities, and carry a 1-inch The drone uses a high-definition camera with a sensor; designs a route planning algorithm to plan the flight route according to the outline of the parking lot area to ensure low-altitude bird's-eye view of each area (the flight altitude is controlled at about 10 meters); during the automatic flight of the drone, it automatically takes photos at fixed time intervals and uses a gimbal-stabilized camera to ensure image clarity; the captured images are transmitted to the edge computing server for image enhancement preprocessing, including denoising and distortion correction.
[0031] Further, use The 3D reconstruction algorithm calculates the mapping relationship between feature points in adjacent images to restore the camera shooting position and generate a sparse point cloud. The multi-view algorithm generates a high-density 3D point cloud, and uses ground control points and sensor data to geometrically correct the irregular 3D point cloud to make it conform to the actual parking lot size. The semantic segmentation network is used to extract the parking space area in the point cloud, and the boundaries are corrected to generate standardized 3D boundaries of parking spaces. The corrected 3D point cloud and the extracted standard parking spaces are combined to construct an accurate 3D digital model of the parking lot environment.
[0032] It should be noted that if the number of feature point matches is small, the mapping relationship is unreliable, or the quality of the sparse point cloud does not meet expectations, the feature point matches are recalculated or the parameters of the 3D reconstruction algorithm are adjusted; if the point cloud density generated by the multi-view algorithm is insufficient or the quality is poor, the algorithm parameters are adjusted or the image view angle is increased to improve the point cloud quality; if the ground control point matching error is large, the ground control point data is reprocessed or the parameters of the correction algorithm are adjusted; if the sensor data is inaccurate or the noise is large, the sensor is recalibrated or the data is filtered; if the parking space area extracted by the semantic segmentation network has a large error, the semantic segmentation is re-performed or the network model is optimized to improve accuracy; if the corrected parking space boundary does not match the actual parking space, the correction algorithm parameters are recalibrated or adjusted.
[0033] Preferably, in the process of feature point matching and 3D reconstruction, the specific formula for optimizing the point cloud registration error is as follows:
[0034] ;
[0035] in, and Respectively represent the i-th point of the reference point cloud and the point cloud to be registered, and They represent the rotation matrix and the translation vector respectively. represents the weight coefficient, represents the registration error of all point pairs, Indicates the number of matching point pairs in the reference point cloud and the point cloud to be registered.
[0036] Preferably, through iterative optimization , to obtain more accurate and , reduce the registration error and improve the accuracy of the three-dimensional model.
[0037] S2: Use based The multi-target detection algorithm monitors vehicles in real time and matches the owner information based on the detected vehicle information.
[0038] Specifically, the trained and optimized multi-target detection model is used to monitor vehicles in the video stream in real time; if no vehicle is detected, empty is returned directly; if a vehicle is detected, the vehicle is added to the multi-target tracking queue; the license plate of the newly appeared vehicle in the tracking queue is captured, and the quality of the license plate image is judged to filter out poor quality images; fuzzy query is applied to obtain matching owner information from the blockchain owner database; new entering vehicles are continuously detected, and the process of vehicle detection, license plate capture and matching owner information is repeated; the location information of the vehicle in the tracking queue is continuously updated, and the vehicle is deleted from the tracking queue when the vehicle leaves.
[0039] Furthermore, judging the quality of the license plate image to filter out poor-quality images includes at least one of the following situations: detecting the angle and inclination of the license plate in the image, if the angle of the license plate is too large or there is an inclination, it is judged as a poor-quality image; judging whether the image is blurred according to the clarity index in the image processing technology, if the variance difference is less than a preset blurriness threshold, it is judged as a poor-quality image; detecting whether there is an obstruction in the license plate image, if the image is severely obstructed, it is judged as a poor-quality image; detecting whether there is noise interference in the license plate image, if there are spots in the image or the details of the image are destroyed, it is judged as a poor-quality image; checking the integrity of the license plate image, if the license plate area is cropped or truncated, it is judged as a poor-quality image.
[0040] S3: Combination The algorithm continuously tracks the target vehicle and calculates the available parking spaces corresponding to the shortest path to make the optimal parking space allocation.
[0041] Preferably, the method comprises the following steps:
[0042] S3.1: For vehicles in a multi-target tracking queue, use The model extracts a unique visual feature signature for each vehicle.
[0043] Preferably, select a suitable pre-trained vehicle Model (the present invention adopts a network structure based on ResNet-50); prepare a vehicle image dataset containing various models and perspectives, and use the image dataset to fine-tune model to improve the feature expression ability; in the process of target detection and tracking, the target image is input into the fine-tuned In the model, the global and local features of the image are obtained through forward calculation; the global and local features obtained through forward calculation are converted into feature vectors respectively, and the cosine similarity is used to evaluate the distance between the feature vectors as the unique visual feature signature.
[0044] S3.2: When the same vehicle is detected in multiple consecutive frames, a tracker is initialized and the vehicle’s Feature signature.
[0045] S3.3: In subsequent frames, the newly detected vehicle is compared with the existing trackers. Feature matching, if the feature signature matches successfully, the status of the tracker is updated.
[0046] Specifically, for each newly detected vehicle target in a frame, use The model extracts the feature vector of the vehicle; matches the feature vector of the newly detected vehicle with the feature signature of the existing tracker; if the cosine similarity (or other similarity measurement) of the corresponding feature signature exceeds the preset threshold, it is considered to be the same vehicle and the feature matching is successful; if the feature matching is successful, the status information of the corresponding tracker including position, speed and appearance is updated; if the newly detected vehicle target cannot be successfully matched with the feature signature of any existing tracker, a new tracker is created as needed and the status information of the new tracker is initialized; for trackers that have not matched features for a long time, determine whether to retain this tracker or terminate it and release related resources based on the set maximum number of tracking frames.
[0047] It should be noted that for some relatively simple scenarios (such as parking lots and residential areas), the preset threshold is set lower (such as 0.5), which makes it easier to match the target vehicle; but for some more complex scenarios (such as urban roads and stations), due to the presence of more interference and changing factors, the preset threshold needs to be set higher (such as 0.8 or 0.9), which can reduce false matches and improve matching accuracy.
[0048] S3.4: When multiple consecutive frames fail to successfully match features, the tracker is retained for a certain period of time and then deleted to avoid accumulation of false matches.
[0049] It should be noted that the essence of this strategy is based on the reliability and stability of the tracker. When the tracker fails to track the target through feature matching for multiple consecutive frames, it may indicate that the tracker is drifting, the target is changing, or the tracker is invalid. At this time, the retained tracker cannot provide accurate tracking results and may cause mismatching.
[0050] S3.5: Use the Kalman filter algorithm to smooth the tracking frame and output a continuous and stable vehicle trajectory, and terminate the corresponding tracker when the vehicle exits the scene.
[0051] Specifically, in the tracking frame smoothing process, the specific formula of the state prediction equation used by the Kalman filter is as follows:
[0052] ;
[0053] in, Indicated in The state prediction at the moment, Indicates at time The state estimate (a priori estimate) of Indicated in The control vector at time, represents the external input matrix, represents the state transfer matrix, represents the process noise and is assumed to follow a zero-mean Gaussian distribution.
[0054] S3.6: Calculate the available parking space information in the current scene based on the three-dimensional environment model, and plan the shortest path to the adjacent available parking space for each tracked vehicle.
[0055] Preferably, the three-dimensional environment model of the parking lot pre-built in S1 is loaded, and the model contains the coordinate range information of each parking space; the current position coordinates of each vehicle in the real-time tracking system are retrieved, and the occupied parking spaces are marked in the environment model; the unmarked areas in the environment model are calculated as vacant parking spaces, and the specific boundaries of the vacant parking spaces are adjusted and divided according to the parking space spacing requirements; for each tracked vehicle, the owner's current position coordinates are taken as the starting point of path planning; in the navigation network of the environment model, the center point of the vacant parking space is taken as the planned end point; the A* algorithm is used to calculate the shortest path from the starting point to each end point, and the shortest path to the nearest vacant space is returned as the planned path; the above process is repeated to dynamically update the vacant parking spaces and the planned path.
[0056] S3.7: Consider the owner information and parking space attributes to perform intelligent allocation and dynamic adjustment of parking spaces.
[0057] S4: Use the 3D environment model to plan the optimal parking path and provide navigation guidance, while analyzing and judging vehicle behavior.
[0058] Specifically, the three-dimensional parking environment model constructed in step S1 is loaded (the model includes navigation network topology information), and the topological node sequence of the vehicle planning path output in step S3 is extracted according to the navigation network topology information; the topological node sequence is traversed, and the corresponding three-dimensional scene division in the environment model is retrieved to obtain the expected driving route of the vehicle; the key points in the driving route are sent to the vehicle terminal for route navigation and guidance; target detection and tracking are performed on the driving picture obtained by the vehicle camera, and the real-time motion state and posture of the vehicle target are analyzed to determine whether there is abnormal behavior.
[0059] It should be noted that abnormal behaviors include but are not limited to the following situations: if it is detected that the vehicle's forward direction is opposite to the expected driving direction, it is determined to be driving in the opposite direction. At this time, a sound alarm is used to remind the driver to avoid hidden dangers; if it is detected that the vehicle's speed change exceeds the speed limit of the parking lot, it is determined to be speeding. At this time, a warning message or a sound alarm is used to remind the driver to slow down, and the camera is used to monitor and record the speeding vehicle information; if the vehicle is detected to be parked in an area outside the navigation path, it is determined to be illegal parking. At this time, a warning message is sent to the driver, and the vehicle is monitored by the camera to promptly discover and deal with illegal parking behaviors; if it is detected that the direction and position of the vehicle in the parking space are inconsistent with the regulations, it is determined to be an incorrect parking posture. At this time, a warning message is sent to the driver or manager to adjust the vehicle posture; if the vehicle is safely parked in the designated parking space, it is determined to be in a safe parking state; repeat the detection until it enters a safe parking state.
[0060] It should be noted that the vehicle position is continuously obtained through the tracking algorithm, the direction cosine value of the vehicle position change trajectory is calculated, and compared with the navigation path direction. If the direction is completely opposite, it is judged to be reverse driving; the vehicle position change is statistically processed, the displacement within the time is calculated to obtain the speed change, and compared with the limited speed range. If it exceeds the parking lot speed limit, it is judged to be speeding; the navigation path range is defined as a legal parking area, and the stationary time of the vehicle in the non-navigation path range is detected. If it exceeds the time limit, it is judged to be illegal parking; the timer is turned on to record the time when the vehicle enters the unit parking space. When the vehicle is stationary and in the same parking space, the time is accumulated. If it exceeds the limited time, it is judged to be a stay timeout.
[0061] It should be noted that the speed limit in a parking lot needs to take into account safety factors, parking lot structure, traffic flow and regulatory requirements, and is usually between 10 km / h and 20 km / h.
[0062] S5: When the system detects that the vehicle has been safely parked in the designated parking space, it automatically generates fee details and settles the fees based on the parking time and area.
[0063] Specifically, when the system detects that the vehicle has been safely parked in the designated parking space, it starts calculating the parking fee;
[0064] A parking fee breakdown is generated based on the specific parking space detected and the actual parking time of the vehicle, and in accordance with the parking lot's charging standards; billing ends when the vehicle leaves, and total fee information is generated; the owner pays the fee by providing the license plate number through the on-board payment system or the parking lot's self-service payment terminal; after confirming that the payment is complete, the parking settlement is completed.
[0065] Furthermore, the present embodiment also provides a parking lot management system based on multi-target intelligent recognition, including a model building module for using drone aerial photography to obtain parking lot images and combining image recognition algorithms to build a three-dimensional parking lot environment model; a detection module for using The multi-target detection algorithm monitors vehicles in real time and combines The algorithm continuously tracks the target vehicle; the parking space allocation module is used to match the owner information according to the detected vehicle information, and calculate the available parking spaces corresponding to the shortest path to make the optimal parking space allocation; the judgment module is used to use the three-dimensional environmental model to plan the optimal parking space path and provide navigation guidance, while analyzing and judging the vehicle behavior; the settlement module is used to automatically generate fee details and settle the bill according to the parking time and area after the system detects that the vehicle has been safely parked in the designated parking space.
[0066] This embodiment also provides a computer device, which is suitable for the case of a parking lot management method based on multi-target intelligent recognition, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the parking lot management method based on multi-target intelligent recognition proposed in the above embodiment.
[0067] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0068] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the following steps are implemented: using a drone to obtain an image of the parking lot, and combining an image recognition algorithm to construct a three-dimensional environment model of the parking lot; using a computer program based on The multi-target detection algorithm detects vehicles in real time and matches the owner information based on the detected vehicle information; combined with The algorithm continuously tracks the target vehicle, calculates the available parking spaces corresponding to the shortest path, and makes optimal parking space allocation; it uses a three-dimensional environmental model to plan the optimal parking path and provide navigation guidance, while analyzing and judging vehicle behavior; when the system detects that the vehicle has been safely parked in the designated parking space, it automatically generates a fee breakdown and settles the bill based on the parking time and area.
[0069] In summary, the present invention uses a combination of drone low-altitude photogrammetry, image processing and computer vision algorithms, deep learning target detection and tracking, path planning and scheduling algorithms, and vehicle behavior pattern recognition to achieve real-time monitoring and analysis of each vehicle's movement process, intelligently plan the best path for the vehicle and navigate to an empty parking space, actively discover and warn of abnormal vehicle behavior, and manage settlements based on accurate vehicle entry and exit information. Compared with traditional manual management or monitoring management methods, the present invention is more intelligent.
[0070] Example 2
[0071] Reference Figure 1~Figure 3 , which is the second embodiment of the present invention, and this embodiment provides a parking lot management method based on multi-target intelligent recognition. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
[0072] Specifically, the intelligent parking lot management system is applied in an underground parking lot of a commercial complex in District B of City A. The parking lot has a total area of 8,000 square meters and is equipped with 1,000 parking spaces.
[0073] Further, use Hexacopter drone, 1 inch Sensor, 20 Camera. Eight flight routes were designed, with a flight altitude of 10 meters and an average speed of 2m / s. 2,500 images were automatically taken along the way. The collected images were transmitted to the edge server for pre-processing such as denoising and correction. A 3D point cloud containing 2 million points was generated using a 3D reconstruction algorithm with an accuracy of 10 cm and a parking space extraction accuracy of 90%.
[0074] Furthermore, based on The vehicle detection model takes the drone image stream as input and outputs the detection box, category and coordinates in real time. After a new vehicle is detected, it is added to the tracking queue, the license plate image is captured for recognition, and the license plate recognition is compared with the blockchain car owner database to accurately identify the owner information. New vehicles are continuously detected and the above process is repeated.
[0075] Preferably, use a vehicle based on ResNet-50 fine-tuning The model extracts unique visual features for each detected vehicle for tracking and matching, and updates the tracker status after a successful match. The 3D scene model is used to calculate that there are currently 90 vacant parking spaces, and the best path to the nearest vacant space is planned for each vehicle. At the same time, the owner's information is considered and the nearest parking space is allocated first.
[0076] Furthermore, the navigation network constructed based on the three-dimensional model extracts the topological nodes of the vehicle's planned path and sends them to the on-board terminal for route navigation. The driving process is monitored and if the vehicle is speeding, a voice prompt is given to slow down, and the license plate information of the speeding vehicle is recorded.
[0077] Furthermore, the fee is charged after the vehicle is safely parked in the parking space. According to the location of the parking space and the actual parking time, the fee details are automatically generated at a rate of 0.5 yuan per hour. When the vehicle leaves the parking lot, the fee is paid by scanning the code and providing the license plate number through the on-board payment system or self-service payment terminal.
[0078] Preferably, the comparison indicators of the method of the present invention and the prior art are shown in Table 1.
[0079] Table 1 Comparison index between the method of the present invention and the prior art
[0080]
[0081] Furthermore, by comparing with the traditional manual management method, it can be seen that the present invention has realized intelligent and precise management in various aspects such as digital modeling, vehicle monitoring, parking space planning, abnormal identification, and fee settlement, which has greatly improved the operation efficiency, resource optimization, navigation performance, and safety of the parking lot. In contrast, traditional management has problems such as extensive management, low efficiency, and random resource allocation.
[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A parking lot management method based on multi-target intelligent recognition, characterized in that: include, Use drones to obtain parking lot images, and use image recognition algorithms to build a three-dimensional parking lot environment model; Use based The multi-target detection algorithm detects vehicles in real time and matches the owner information based on the detected vehicle information; Combination The algorithm continuously tracks the target vehicle and calculates the available parking spaces corresponding to the shortest path to perform optimal parking space allocation; Use the 3D environment model to plan the optimal parking path and provide navigation guidance, while analyzing and judging vehicle behavior; When the system detects that the vehicle has been safely parked in the designated parking space, it automatically generates fee details and settles the bill based on the parking time and area; The construction of the parking lot three-dimensional environment model includes: use 3D reconstruction algorithm, which calculates the mapping relationship between feature points in adjacent images to restore the camera shooting position and generate a sparse point cloud; use The multi-view algorithm generates a high-density 3D point cloud and uses ground control points and sensor data to geometrically correct the irregular 3D point cloud to make it conform to the actual parking lot size. A semantic segmentation network is used to extract the parking space area in the point cloud and correct the boundaries to generate a standardized 3D boundary of the parking space; Combine the corrected 3D point cloud and the extracted standard parking spaces to build an accurate 3D digital model of the parking lot environment; Said The 3D reconstruction algorithm includes the following formula: ; in, and Respectively represent the i-th point of the reference point cloud and the point cloud to be registered, and They represent the rotation matrix and the translation vector respectively. represents the weight coefficient, represents the registration error of all point pairs, Indicates the number of matching point pairs in the reference point cloud and the point cloud to be registered; The construction of the parking lot three-dimensional environment model also includes: If the number of feature point matches is small, the mapping relationship is unreliable, or the quality of the sparse point cloud does not meet expectations, recalculate the feature point matches or adjust the parameters of the 3D reconstruction algorithm; If the point cloud generated by the multi-view algorithm is not dense enough or of poor quality, adjust the algorithm parameters or increase the image viewing angle to improve the point cloud quality; If the ground control point matching error is large, reprocess the ground control point data or adjust the parameters of the correction algorithm; If the sensor data is inaccurate or noisy, recalibrate the sensor or filter the data; If the parking space area extracted by the semantic segmentation network has a large error, re-perform semantic segmentation or optimize the network model to improve accuracy; If the corrected parking space boundary does not match the actual parking space, recalibrate or adjust the parameters of the correction algorithm; The optimal parking space allocation comprises the following steps: For vehicles in a multi-target tracking queue, The model extracts a unique visual feature signature for each vehicle; When the same vehicle is detected in multiple consecutive frames, a tracker is initialized and the vehicle’s Feature signature; In subsequent frames, newly detected vehicles are compared with existing trackers. Feature matching: if the feature signature matches successfully, the status of the tracker is updated; When multiple consecutive frames fail to successfully match features, the tracker is retained for a certain period of time and then deleted to avoid accumulation of false matches; The Kalman filter algorithm is used to smooth the tracking frame and output a continuous and stable vehicle trajectory. When the vehicle leaves the scene, the corresponding tracker is terminated. Calculate the available parking spaces in the current scene based on the 3D environment model, and plan the shortest path to the adjacent available parking spaces for each tracked vehicle; Considering the owner information and parking space attributes, intelligent allocation and dynamic adjustment of parking spaces are performed; The newly detected vehicle is compared with the existing tracker Feature matching includes the following steps: For each newly detected vehicle target in a frame, use The model extracts the feature vector of the vehicle; Match the feature vector of the newly detected vehicle with the feature signature of the existing tracker; If the cosine similarity of the corresponding feature signatures exceeds the preset threshold, it is considered to be the same vehicle and the feature matching is successful; If the feature matching is successful, the status information of the corresponding tracker including position, speed and appearance is updated; If the newly detected vehicle target cannot be successfully matched with the feature signature of any existing tracker, a new tracker is created as needed and the state information of the new tracker is initialized; For trackers that have not matched features for a long time, determine whether to keep the tracker or terminate it and release related resources based on the set maximum number of tracking frames; Planning the shortest path for each tracked vehicle to an adjacent vacant parking space comprises the following steps: Load the pre-built 3D parking environment model, which contains the coordinate range information of each parking space; Retrieve the current position coordinates of each vehicle in the real-time tracking system and mark the occupied parking spaces in the environment model; Calculate the unmarked areas in the environment model as vacant parking spaces, and adjust and divide the specific boundaries of the vacant parking spaces according to the parking space spacing requirements; For each tracked vehicle, the current position coordinates of the vehicle owner are taken as the starting point of the path planning; in the navigation network of the environment model, the center point of the vacant parking space is taken as the end point of the planning; Use the A* algorithm to calculate the shortest path from the starting point to each end point, and return the shortest path to the nearest empty space as the planned path; Iterate the execution steps to dynamically update the available parking spaces and planned paths; The analysis and judgment of vehicle behavior includes: If the vehicle is detected to be traveling in the opposite direction to the expected direction, it is considered to be traveling in the opposite direction. At this time, an audible alarm is used to remind the driver to avoid hidden dangers. If the speed of the vehicle is detected to exceed the speed limit of the parking lot, it is judged as speeding. At this time, the driver is reminded to slow down through warning information or sound alarms, and the speeding vehicle information is monitored and recorded with the help of cameras; If the vehicle is detected to be parked outside the navigation path, it is considered illegal parking. A warning message will be sent to the driver, and the vehicle will be monitored through the camera to detect and deal with illegal parking in a timely manner. If it is detected that the orientation and position of the vehicle in the parking space are not in accordance with the regulations, it is determined that the parking posture is wrong, and a warning message is sent to the driver or management personnel to adjust the vehicle posture; If the vehicle is safely parked in the designated parking space, it is considered to be in a safe parking state; Repeat the test until the vehicle enters a safe parking state.
2. A parking lot management system based on multi-target intelligent recognition, based on the parking lot management method based on multi-target intelligent recognition according to claim 1, characterized in that: include, The model building module is used to obtain parking lot images using drone aerial photography and build a three-dimensional parking lot environment model in combination with image recognition algorithms; Detection module for using The multi-target detection algorithm monitors vehicles in real time and combines The algorithm continues to track the target vehicle; The parking space allocation module is used to match the owner information according to the detected vehicle information, calculate the available parking spaces corresponding to the shortest path, and perform optimal parking space allocation; The judgment module is used to plan the optimal parking path and provide navigation guidance using the three-dimensional environment model, while analyzing and judging vehicle behavior; The settlement module is used to automatically generate fee details and settle accounts based on parking time and area after the system detects that the vehicle has been safely parked in the designated parking space.
