Parking space detection system and method

Abstract

The application discloses a parking space detection system and method, and relates to the technical field of intelligent transportation, the system comprising: an image acquisition module, which is used for collecting parking space images through a camera; a data processing module, which is used for pre-processing the parking space images and analyzing image data through an image recognition algorithm; a parking space detection module, which is used for detecting parking space states through a dTOF laser radar; and a GPS positioning module, which is used for determining the real-time position of a patrol device.According to the technical scheme of the application, the identification function of vehicles temporarily parked near parking spaces can be realized, and the phenomenon of incorrect charging of temporarily parked vehicles by the patrol device is avoided, so that the application has high application value.

Description

[0001] This application is a divisional application of Chinese invention patent application No. "202411614214.8", application date "2024.11.13", and invention title "A berth detection system and method based on dTOF lidar". Technical Field

[0002] This invention relates to the field of intelligent transportation technology, and in particular to a parking space detection system and method. Background Technology

[0003] In recent years, the number of motor vehicles in my country has continued to grow, and cars have become an indispensable means of transportation for people's daily travel, making the parking problem increasingly prominent. Especially in busy commercial areas, residential communities, and near public transportation hubs, finding a suitable parking space has become a headache for many drivers. To improve the utilization rate of on-street parking spaces, many intelligent road inspection devices have emerged on the market. These devices improve the efficiency of road inspection while reducing manual intervention. The inspection devices determine whether a vehicle is parked in a parking space by detecting information such as the relative position of the license plate, the size of the license plate, and the rotation angle of the license plate in the captured image. In some areas, on-street parking spaces are adjacent to the shoulder. When a vehicle is parked on the shoulder or sidewalk, the inspection device will be triggered by the license plate of such a vehicle. Because the license plate information of the vehicle parked in this case is very similar to that of the vehicle parked in the parking space, the inspection device often mistakenly matches the vehicle parked near the parking space to the parking space, resulting in abnormal inspection data. Therefore, based on this problem, this application proposes a parking space detection system and method based on dTOF lidar. Summary of the Invention

[0004] Technical Purpose

[0005] To address the aforementioned problems, the present invention aims to provide a parking space detection system and method that can not only identify temporarily parked vehicles near parking spaces, but also identify the types of vehicles parked within the parking spaces. This method and system have a wide range of applications and prevent the occurrence of incorrect charges for temporarily parked vehicles by inspection equipment.

[0006] Technical solution

[0007] To achieve the above objectives, this invention provides a parking space detection system and method based on dTOF (Direct Time-of-Flight) lidar. It identifies vehicle license plates through image segmentation and OCR text recognition algorithms, and extracts vehicle contour information and calculates the relative position of vehicles by emitting laser pulses into the parking space area and receiving reflected signals through the dTOF lidar, thereby realizing the identification function of temporarily parked vehicles near the parking space.

[0008] In a first aspect, the present invention provides a berth detection system, comprising:

[0009] The image acquisition module is used to acquire images of the berth via a camera;

[0010] The data processing module is used to preprocess the berth images and analyze the image data using image recognition algorithms;

[0011] The berth detection module is used to detect the berth status using a dTOF lidar. The detection data provided by the lidar includes a depth map, a point cloud map, and a confidence map.

[0012] The structured light module is used to solve the wrapping phase and absolute phase using a four-step phase shifting method and a multi-frequency heterodyne method, and to obtain the vehicle's contour information by projecting a specific light pattern and analyzing its deformation.

[0013] The GPS positioning module is used to determine the real-time location of the inspection equipment.

[0014] Furthermore, the image acquisition module acquires parking space images through an intelligent license plate recognition camera installed on the inspection equipment.

[0015] Furthermore, after receiving the parking space image video stream, the data processing module preprocesses the frames in the video stream and segments the license plate characters within the frames to obtain the vehicle's license plate information. It can accurately locate and segment the license plate area from the captured image and improve image quality and highlight license plate features through image preprocessing; it can quickly and accurately identify vehicle license plate information, thereby improving the operational efficiency and management level of the parking space.

[0016] Furthermore, the data processing module preprocesses the frames in the video stream, specifically including: labeling the text regions of the collected sample data to determine the position of the text in the document, thereby constructing a model to learn the spatial structure of the text and improving the accuracy of text recognition; while extracting the text content within the text region, establishing label data corresponding to the text content; and cleaning and preprocessing the extracted text content to ensure the quality of the text data.

[0017] Furthermore, the image recognition algorithm includes an OCR text recognition algorithm, which supports multiple models, including support vector machines, Bayesian classification algorithms, and deep learning-based neural network models. The type of model is determined according to the specific application scenario and requirements. After the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm through dataset optimization and feature selection. Specifically, dataset optimization includes using a more accurate and comprehensive dataset for training, and optimizing the dataset by removing duplicate samples and increasing randomness. Feature selection specifically includes selecting features with high weight for the recognition task and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm. Feature selection is based on the feature evaluation method of information gain.

[0018] Furthermore, in the berth detection module, the detection of berth status using dTOF lidar specifically includes: emitting laser pulses into the berth area using dTOF lidar and receiving reflected signals to extract vehicle contour information and calculate the vehicle's relative position; and associating the berth status with the specific berth based on the real-time position of the inspection equipment provided by the GPS positioning module, thereby obtaining continuous berth status images; if the data processing module identifies the vehicle's license plate information, it pairs the vehicle's license plate information with the lidar's imaging information. This provides accurate location information; provides stable measurement results even in poor lighting conditions; and improves berth operation efficiency by quickly and accurately identifying vehicle contour information and relative vehicle positions.

[0019] Furthermore, the dTOF lidar is installed at the front end of the inspection equipment and connected to the edge computing host via an uplink serial cable.

[0020] Furthermore, the berth status information includes the distance between the current inspection equipment and the curb side of the berth, whether there are objects in the berth, and whether the objects in the berth are pedestrians, bicycles, electric vehicles, motorcycles, small vehicles, large vehicles, or other objects.

[0021] Furthermore, if the parking space detection module fails to extract the vehicle's outline information within the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module will be attributed to a vehicle outside the parking space.

[0022] Furthermore, the structured light module solves for the wrapping phase and absolute phase using a four-step phase-shifting method and a multi-frequency heterodyne method. By projecting a specific light pattern and analyzing its deformation, it obtains the vehicle's contour information. This improves the accuracy and precision of LiDAR measurements, expanding the measurement range while maintaining high precision, making the detection system suitable for a wider range of applications; it also reduces system complexity, enabling a more compact system design.

[0023] Furthermore, the system also includes an edge detection module, which protects the edge details of targets in the radar image through a LiDAR adaptive threshold denoising method based on Contourlet transform, performs edge detection using the Sobel operator, and calculates the local range image edge threshold using a median filtering method. It can further analyze and process the depth information acquired by the dTOF LiDAR, extracting key edge features, enabling the detection system to maintain high recognition accuracy even in complex road environments, thus improving the overall robustness of the system; it can identify fine edges and structures, making it suitable for high-precision berth detection scenarios.

[0024] Secondly, the present invention also provides a berth detection method, the method being based on the system described in the first aspect above, comprising:

[0025] Images of the berths are captured using cameras;

[0026] Preprocess the berth images and analyze the image data using image recognition algorithms;

[0027] The dTOF lidar emits laser pulses into the parking area and receives reflected signals to extract the vehicle's outline information and calculate its relative position.

[0028] The system determines whether a vehicle is in a parking space based on the parking space status and the real-time location of the inspection equipment.

[0029] Thirdly, the present invention also provides a computer device, including a processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, and the processor being used to execute the computer program stored in the memory, so that the computer device performs the aforementioned berth detection method.

[0030] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned berth detection method.

[0031] This invention identifies vehicle license plates using image segmentation and OCR text recognition algorithms; it extracts vehicle contour information and calculates relative position by emitting laser pulses into the parking area using a dTOF lidar and receiving reflected signals; it solves for the wrapping phase and absolute phase using a four-step phase-shifting method and a multi-frequency heterodyne method, and obtains vehicle contour information by projecting specific light patterns and analyzing their deformation; it preserves edge details of targets in the lidar image using a lidar adaptive thresholding denoising method based on Contourlet transform, performs edge detection using the Sobel operator, and calculates local distance image edge thresholds using a median filtering method. This system and method have wide applications, enabling the identification of temporarily parked vehicles near parking spaces and preventing incorrect charging of temporarily parked vehicles by inspection equipment.

[0032] Beneficial effects

[0033] By implementing the berth detection system and method based on dTOF lidar provided by the present invention, the following technical effects are achieved:

[0034] (1) The system uses image segmentation and OCR text recognition algorithms to identify vehicle license plates. It can accurately locate and segment the license plate area from the captured image and improve the image quality and highlight the features of the license plate through image preprocessing. It can quickly and accurately identify vehicle license plate information, thereby improving the operation efficiency and management level of parking spaces.

[0035] (2) The dTOF lidar emits laser pulses to the parking area and receives reflected signals to extract the vehicle's outline information and calculate the vehicle's relative position. It can provide accurate position information and still provide stable measurement results even in poor lighting conditions. It can quickly and accurately identify the vehicle's outline information and relative position.

[0036] (3) The system adds a four-step phase shift method and a multi-frequency heterodyne method to solve the wrapping phase and absolute phase. By projecting a specific light pattern and analyzing its deformation, the contour information of the vehicle is obtained, which further improves the accuracy and precision of the lidar measurement. While maintaining high precision, the measurement range is expanded, making the detection system suitable for a wider range of application scenarios, reducing the complexity of the system, and thus achieving a more compact system design.

[0037] (4) The adaptive threshold denoising method of lidar based on Contourlet transform protects the edge details of the target in the lidar image. The Sobel operator is used for edge detection and the local distance image edge threshold is calculated by median filtering. The depth information obtained by dTOF lidar is further analyzed and processed. By extracting key edge features, the detection system can maintain a high recognition accuracy in complex road environments, which improves the overall robustness of the system. It can identify small edges and structures and is suitable for berth detection scenarios with high precision requirements. Attached Figure Description

[0038] To make the above-described berth detection system and method based on dTOF lidar of the present invention more apparent and understandable, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 A schematic diagram illustrating a berth detection method based on dTOF lidar;

[0040] Figure 2 This diagram shows the inspection equipment berth detection system.

[0041] Figure 3 This diagram illustrates the deployment location of the dTOF lidar.

[0042] Figure 4 A diagram showing the parking situation of vehicles in on-street parking spaces. Detailed Implementation

[0043] Example 1:

[0044] A berth detection system and method based on dTOF lidar are provided. The steps of the dTOF lidar-based berth detection method are as follows: Figure 1 As shown, the inspection equipment berth detection system is as follows: Figure 2 As shown, the system includes: an image acquisition module, a data processing module, a berth detection module, a structured light module, and a GPS positioning module, as described in detail below.

[0045] The image acquisition module is used to capture images of the berth via a camera.

[0046] The image acquisition module uses an intelligent license plate recognition camera installed on the inspection equipment to acquire images of parking spaces.

[0047] After receiving the parking space image video stream, the data processing module preprocesses the frame images in the video stream and segments the license plate characters in the frame images to obtain the vehicle's license plate information.

[0048] The data processing module is used to preprocess berth images and analyze image data using image recognition algorithms.

[0049] The data processing module preprocesses the frames in the video stream, specifically including: labeling the text regions of the collected sample data to determine the position of the text in the document, thereby constructing a model to learn the spatial structure of the text and improving the accuracy of text recognition; while extracting the text content within the text region, establishing label data corresponding to the text content; and cleaning and preprocessing the extracted text content to ensure the quality of the text data.

[0050] Image recognition algorithms include OCR text recognition algorithms, which support multiple models, including support vector machines, Bayesian classification algorithms, and deep learning-based neural network models. The type of model is determined according to the specific application scenario and requirements. After the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm through dataset optimization and feature selection. Specifically, dataset optimization includes using a more accurate and comprehensive dataset for training, and optimizing the dataset by removing duplicate samples and increasing randomness. Feature selection specifically includes selecting features with high weight for the recognition task and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm. Feature selection is based on the feature evaluation method of information gain.

[0051] The berth detection module is used to detect the berth status using a dTOF lidar. The detection data provided by the lidar includes a depth map, a point cloud map, and a confidence map.

[0052] The structured light module is used to solve the wrapping phase and absolute phase using a four-step phase shifting method and a multi-frequency heterodyne method, and to obtain the vehicle's contour information by projecting a specific light pattern and analyzing its deformation.

[0053] The structured light module specifically includes: calibrating the system to determine the geometric relationship and intrinsic parameters between the camera, LiDAR, and the structured light module; projecting an coded grating onto the surface of the object being measured via the structured light module, with the camera acquiring the reflected image; emitting short pulses of light via the dTOF LiDAR and measuring the time of the returning light to obtain vehicle distance information; for the structured light data, solving for the wrapper phase and absolute phase using a four-step phase shift method and a multi-frequency heterodyne method; achieving a three-dimensional coordinate correspondence between the phase and the measured surface by using the relative relationship between the calibrated camera and projection module coordinate systems; achieving high-precision stitching using an iterative nearest-point method; establishing feature descriptors for adjacent point clouds and using the Euclidean distance of the feature descriptor vectors as a criterion to determine the similarity of feature points; and adjusting the stitching matrix to minimize the deviation between corresponding points in the stitched point cloud to achieve global optimization. The application of the structured light module can improve the detection accuracy of the LiDAR by approximately 8%.

[0054] The GPS positioning module is used to determine the real-time location of the inspection equipment.

[0055] In the parking space detection module, the detection of parking space status using dTOF lidar specifically includes: emitting laser pulses into the parking space area using dTOF lidar and receiving reflected signals to extract the vehicle's outline information and calculate its relative position; associating the parking space status with the specific parking space based on the real-time position of the inspection equipment provided by the GPS positioning module to obtain continuous parking space status images; if the data processing module identifies the vehicle's license plate information, then pairing the vehicle's license plate information with the lidar's imaging information.

[0056] dTOF lidar deployment locations are as follows Figure 3 As shown, the dTOF lidar is installed at the front end of the inspection equipment and connected to the edge computing host via an uplink serial cable.

[0057] The berth status information includes the distance between the current inspection equipment and the curb of the berth, whether there are objects in the berth, and whether the objects in the berth are pedestrians, bicycles, electric vehicles, motorcycles, small vehicles, large vehicles, or other objects.

[0058] Parking situation of vehicles in on-street parking spaces Figure 4 As shown, if the parking space detection module fails to extract the vehicle's outline information within the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module will be attributed to the vehicle outside the parking space.

[0059] Example 2:

[0060] Based on the aforementioned embodiments, the system adds an edge detection module, which protects the edge details of the target in the radar image by using a lidar adaptive threshold denoising method based on Contourlet transform, performs edge detection by Sobel operator, and calculates the local range image edge threshold by median filtering method.

[0061] Environmental scanning is performed using dTOF lidar to obtain distance images of vehicles.

[0062] An adaptive thresholding denoising method based on Contourlet transform for imaging lidar is adopted to suppress range anomalous noise while preserving the edge details of targets in the radar image. The Contourlet transform formula is as follows:

[0063]

[0064] in, This represents the Contourlet transform coefficients after threshold adjustment; Represents the Contourlet transform coefficients; This represents the dynamic threshold adjustment factor, used to enhance the noise reduction effect; This represents the median of the absolute values ​​of the Contourlet transform coefficients, used to estimate the noise level.

[0065] Edge information in an image is extracted using an edge detection algorithm, and edge detection is performed using the Sobel operator. The accuracy of edge localization is improved by increasing the gradient detection direction, and a local distance image edge threshold is calculated using median filtering. This achieves adaptive threshold Sobel edge detection. The Sobel operator formula is as follows:

[0066]

[0067] in, Indicates at point The gradient magnitude at that point, i.e., the edge strength at that point; This represents the gradient in the horizontal direction, i.e., the gradient of the image in the horizontal direction. The rate of change of brightness in the direction; This represents the gradient in the vertical direction, i.e., the gradient of the image in the vertical direction. The rate of change of brightness in the direction; This represents the gradient intensity adjustment factor, used to adjust the sensitivity of edge detection.

[0068] Berth detection involves analyzing edge information of a specific area to determine whether there are vacant berths.

[0069] The results of edge detection and berth detection are fused to generate a point cloud data containing distance and edge information.

[0070] For example, a parking lot can be scanned using a dTOF lidar system to detect vacant parking spaces.

[0071] Assuming the absolute deviation of the median coefficients after the Contourlet transformation is 0.5, using... To enhance noise reduction:

[0072]

[0073] Suppose that at a certain point, the horizontal and vertical gradients calculated by the Sobel operator are Gx=2 and Gy=1, respectively, and the gradient strength is adjusted using β=1.2:

[0074]

[0075] The results of the edge detection module are shown in Table 1.

[0076] Table 1. Summary of Edge Detection Module Results

[0077]

[0078] As shown in Table 1, the method optimized by the edge detection module significantly improves the detection accuracy of parking spaces, while reducing the false detection rate and detection time. This indicates that the edge detection module can effectively improve the recognition accuracy of the parking space detection system for temporarily parked vehicles.

[0079] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media containing computer-usable program code.

[0080] The present invention can provide computer program instructions to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which are executed by the processor of the computer or other programmable data processing device, produce means for implementing the system.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that perform the functions of the system.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions of the system.

Claims

1. A parking space detection system, characterized in that: include: The image acquisition module is used to acquire images of the berth via a camera; The data processing module is used to preprocess the berth images and analyze the image data using image recognition algorithms; The berth detection module is used to detect berth status using a dTOF lidar installed at the front end of the inspection equipment. The dTOF lidar is connected to the edge computing host via an uplink serial cable. Specifically, the berth detection module includes: emitting laser pulses into the berth area using the dTOF lidar and receiving reflected signals to extract vehicle contour information and calculate the vehicle's relative position; and associating the berth status with the specific berth based on the real-time location of the inspection equipment provided by the GPS positioning module, thereby obtaining continuous berth status images. The berth status information includes the current location of the inspection equipment and the curb side of the berth. The data processing module determines the distance to the parking space, whether there are objects within the parking space, and whether the objects in the parking space are pedestrians, bicycles, electric vehicles, motorcycles, small vehicles, large vehicles, or other objects. If the data processing module identifies the vehicle's license plate information, it pairs the vehicle's license plate information with the imaging information of the lidar. If the parking space detection module fails to extract the vehicle's outline information within the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module is attributed to a vehicle outside the parking space. The detection data provided by the lidar includes depth maps, point cloud maps, and confidence maps. The structured light module calibrates the system to determine the geometric relationships and intrinsic parameters between the camera, LiDAR, and the structured light module. It projects an coded grating onto the surface of the object being measured, the camera captures the reflected image, and the dTOF LiDAR emits short pulses of light and measures the time of the returning light to obtain the vehicle's distance information. For the structured light data, a four-step phase-shifting method and a multi-frequency heterodyne method are used to solve for the wrapping phase and absolute phase. A specific light pattern is projected, and its deformation is analyzed to obtain the vehicle's contour information. The relative relationship between the calibrated camera and projection module coordinate systems is used to achieve the 3D coordinate correspondence between the phase and the measured surface. High-precision stitching is achieved using an iterative nearest-point method. Feature descriptors for adjacent point clouds are established, and the similarity of feature points is judged using the Euclidean distance of the feature descriptor vectors. Global optimization is achieved by adjusting the stitching matrix to minimize the deviation between corresponding points in the stitched point cloud. The GPS positioning module is used to determine the real-time location of the inspection equipment; The edge detection module uses an adaptive thresholding denoising method based on Contourlet transform for imaging lidar to suppress range anomalous noise while preserving the edge details of targets in the radar image. It performs edge detection using the Sobel operator and calculates the local range image edge threshold using the median filtering method.

2. The parking stall detection system of claim 1, wherein: After receiving the parking space image video stream, the data processing module preprocesses the frame images in the video stream and segments the license plate characters in the frame images to obtain the vehicle's license plate information.

3. The parking stall detection system of claim 2, wherein: The data processing module pre-processes picture frames in the video stream, specifically including: determining the position of the text in the document by labeling the text area of the sample data; establishing label data corresponding to the text content while extracting the text content in the text area; and cleaning and pre-processing the extracted text content.

4. The parking stall detection system of claim 1, wherein: The formula of the Contourlet transform is as follows: wherein, denotes the threshold-adjusted Contourlet transform coefficients; denotes the Contourlet transform coefficients; denotes a dynamic threshold adjustment factor for enhancing the denoising effect; denotes the median of the absolute values of the Contourlet transform coefficients for estimating the noise level.

5. The parking stall detection system of claim 4, wherein: The edge information in the image is extracted through an edge detection algorithm, edge detection is performed through a Sobel operator, the accuracy of edge positioning is improved by increasing gradient detection directions, and a local distance image edge threshold is calculated through a median filtering method to realize adaptive threshold Sobel edge detection, wherein the formula of the Sobel operator is as follows: wherein, represents the gradient magnitude at a point , i.e. the edge strength of the point; represents the gradient in the horizontal direction, i.e. the rate of change of brightness of the image in direction; represents the gradient in the vertical direction, i.e. the rate of change of brightness of the image in direction; represents a gradient strength adjustment factor for adjusting the sensitivity of edge detection.

6. A parking stall detection method, characterized in that: The implementation of the method is based on the system of any one of claims 1-5: The method comprises: A parking stall image is collected through a camera; The parking stall image is pre-processed and image data is analyzed through an image recognition algorithm; A dTOF laser radar emits laser pulses to the parking stall area and receives reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle; Whether the vehicle is located in the parking stall is determined according to the parking stall state and the real-time position of the inspection equipment.

Images