Parking space detection system and method

Abstract

The invention discloses a berth detection system and method, and relates to the technical field of intelligent transportation, and the system comprises an image collection module which is used for collecting a berth image through a camera; the data processing module is used for preprocessing the parking space image and analyzing image data through an image recognition algorithm; the berth detection module is used for detecting the berth state through the dTOF laser radar; and the GPS positioning module is used for determining the real-time position of the inspection equipment. According to the technical scheme of the invention, the method can achieve the recognition of the temporarily-parked vehicle near the parking space, avoids the wrong charging of the temporarily-parked vehicle through the inspection equipment, and is higher in application value.

Description

[0001] This application is a divisional application of a Chinese invention patent application with the application number "202411614214.8", the application date "November 13, 2024", and the invention title "A Berth Detection System and Method Based on dTOF Lidar". Technical Field

[0002] The present invention relates to the field of intelligent transportation technology, and particularly to a berth detection system and method. Background Art

[0003] In recent years, the number of motor vehicles in China has continued to grow. Automobiles have become an essential means of transportation for people's daily travel, and the problem of parking difficulty has become increasingly prominent. Especially near busy commercial areas, residential communities, and public transportation hubs, finding a suitable parking space has become a headache for many drivers. In order to improve the utilization rate of road berths, many road intelligent inspection devices have emerged on the market. These inspection devices improve the efficiency of road inspections while reducing manual participation. The inspection device determines whether a vehicle is parked within a berth by detecting information such as the relative position of the license plate, the size of the license plate, and the rotation angle of the license plate included in the captured image. In some areas, the road berths are adjacent to the road shoulder. When a vehicle is parked on the sidewalk of the road shoulder, when the inspection device passes by, it will be triggered by the license plate of such a vehicle to start the inspection process. Since the license plate information of the vehicle in this case is very similar to the license plate information of the vehicle parked within the berth, the inspection device often mismatches the vehicle parked near the berth to the berth, resulting in abnormal inspection data. Therefore, based on this problem, this application proposes a berth detection system and method based on dTOF lidar. Summary of the Invention

[0004] Technical Objectives In order to solve the above problems, the objective of the present invention is to provide a berth detection system and method that can not only identify temporarily parked vehicles near the berth but also identify the types of vehicles parked within the berth. The application scenarios of this method and system are extensive, avoiding the occurrence of incorrect charging for temporarily parked vehicles by the inspection device.

[0005] Technical Solutions In order to achieve the above objective, the present invention provides a berth detection system and method based on dTOF (Direct Time-of-Flight) lidar. It identifies the license plate of the vehicle through image segmentation and OCR text recognition algorithms, and emits laser pulses to the berth area through the dTOF lidar and receives the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, thereby realizing the identification function of temporarily parked vehicles near the berth.

[0006] In a first aspect, the present invention provides a berth detection system, comprising: An image acquisition module for acquiring berth images through a camera; A data processing module for preprocessing the berth images and analyzing the image data through an image recognition algorithm; A berth detection module for detecting the berth status through a dTOF lidar, wherein the detection data provided by the lidar includes a depth map, a point cloud map, and a confidence map; A structured light module for solving the wrapped phase and the absolute phase through a four-step phase-shifting method and a multi-frequency heterodyne method, and obtaining the contour information of the vehicle by projecting a specific light pattern and analyzing its deformation; A GPS positioning module for determining the real-time position of the inspection device.

[0007] Further, in the image acquisition module, the berth images are acquired through an intelligent license plate recognition camera installed on the inspection device.

[0008] Further, after receiving the berth image video stream, the data processing module preprocesses the frames in the video stream and segments the license plate characters in the frames, so as to obtain the license plate information of the vehicle. It can accurately locate and segment the license plate area from the captured images, improve the image quality through image preprocessing, and highlight the features of the license plate; it can quickly and accurately identify the license plate information of the vehicle, thereby improving the operation efficiency and management level of the berth.

[0009] Further, the data processing module preprocesses the frames in the video stream, specifically including: annotating the text areas of the collected sample data to determine the positions of the texts in the document, thereby constructing a model to learn the spatial structure of the texts and improve the recognition accuracy of the texts; establishing label data corresponding to the text content while extracting the text content within the text area; cleaning and preprocessing the extracted text content to ensure the quality of the text data.

[0010] Further, the image recognition algorithm includes an OCR text recognition algorithm, which supports multiple models, including a support vector machine, a Bayesian classification algorithm, and a neural network model based on deep learning. The type of the model is determined according to specific application scenarios and requirements; after the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm through dataset optimization and feature selection. Among them, dataset optimization specifically includes training with a more accurate and comprehensive dataset, and optimizing the dataset by deleting duplicate samples, increasing randomness, etc.; feature selection specifically includes screening features with high weights for the recognition task and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm. The selection of features is based on an information gain-based feature evaluation method.

[0011] Further, in the berth detection module, the specific process of detecting the berth status by the dTOF lidar includes: emitting laser pulses to the berth area by the dTOF lidar and receiving the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, and associating the berth status with the specific berth according to the real-time position of the inspection device provided by the GPS positioning module, so as to obtain a continuous berth status image; if the data processing module recognizes the license plate information of the vehicle, the license plate information of the vehicle and the imaging information of the lidar are paired. It can provide accurate position information; it can provide stable measurement results under poor light conditions; it can improve the operation efficiency of the berth by quickly and accurately identifying the contour information of the vehicle and the relative position of the vehicle.

[0012] Further, the dTOF lidar is installed at the front end of the inspection device and is connected to the edge computing host through an uplink serial cable.

[0013] Further, the berth status information includes the distance between the current inspection device and the roadside of the berth, whether there are objects in the berth, and whether the objects on the berth are pedestrians, bicycles, electric vehicles, motorcycles, small vehicles, large vehicles or other objects, etc.

[0014] Further, if the berth detection module fails to extract the contour information of the vehicle in the berth area or detects that the relative position between the vehicle and the berth exceeds a certain distance, the license plate information recognized by the data processing module is attributed to the vehicle outside the berth.

[0015] Further, the structured light module solves the wrapped phase and absolute phase through the four-step phase-shifting method and the multi-frequency heterodyne method, and projects a specific light pattern and analyzes its deformation to obtain the contour information of the vehicle. It can improve the accuracy and precision of lidar measurement, expand the measurement range while maintaining high precision, making the detection system applicable to a wider range of application scenarios; it can reduce the complexity of the system, thus realizing a more compact system design.

[0016] Further, the system further includes an edge detection module, which protects the edge details of the target in the radar image through the lidar adaptive threshold denoising method based on the Contourlet transform, performs edge detection through the Sobel operator and calculates the local distance image edge threshold through the median filtering method. It can further analyze and process the depth information obtained by the dTOF lidar, and by extracting the key edge features, the detection system can maintain a high recognition accuracy in a complex road environment, improving the overall robustness of the system; it can identify small edges and structures, and is suitable for berth detection scenarios with high precision requirements.

[0017] In a second aspect, the present invention further provides a berth detection method, which is based on the system described in the aforementioned first aspect and includes: Collecting a berth image through a camera; Preprocessing the berth image and analyzing the image data through an image recognition algorithm; Emitting laser pulses to the berth area through a dTOF lidar and receiving the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle; Judging whether the vehicle is located within the berth according to the berth state and the real-time position of the inspection device.

[0018] In a third aspect, the present invention further provides a computer device, including a processor and a memory. The processor is connected to the memory. The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the computer device executes and implements the aforementioned berth detection method.

[0019] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the aforementioned berth detection method is implemented.

[0020] The present invention recognizes the license plate of a vehicle through image segmentation and OCR character recognition algorithms; emits laser pulses to the berth area through a dTOF lidar and receives the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle; solves the wrapped phase and absolute phase through the four-step phase-shifting method and the multi-frequency heterodyne method, and obtains the contour information of the vehicle by projecting a specific light pattern and analyzing its deformation; protects the edge details of the target in the lidar image through the Contourlet transform-based lidar adaptive threshold denoising method, performs edge detection through the Sobel operator, and calculates the local distance image edge threshold through the median filtering method. The application scenarios of the system and method are extensive, realizing the identification function of temporarily parked vehicles near the berth and avoiding the occurrence of incorrect charging of temporarily parked vehicles by inspection devices.

[0021] Beneficial effects By implementing the berth detection system and method based on dTOF lidar provided by the present invention as described above, the following technical effects are achieved: (1) The system uses image segmentation and OCR character recognition algorithms to recognize the license plate of a vehicle, can accurately locate and segment the license plate area from the captured image, improve the quality of the image through image preprocessing and highlight the features of the license plate, quickly and accurately recognize the license plate information of the vehicle, thereby improving the operation efficiency and management level of the berth.

[0022] (2) The dTOF lidar emits laser pulses towards the berth area and receives the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle. It can provide accurate position information and still offer stable measurement results under poor lighting conditions, quickly and accurately identifying the contour information of the vehicle and the relative position of the vehicle.

[0023] (3) The system adds a four-step phase-shift method and a multi-frequency heterodyne method to solve the wrapped phase and absolute phase. By projecting a specific light pattern and analyzing its deformation to obtain the contour information of the vehicle, it further improves the accuracy and precision of lidar measurement. While maintaining high precision, it expands the measurement range, making the detection system applicable to a wider range of application scenarios, reducing the system complexity, and thus achieving a more compact system design.

[0024] (4) The lidar adaptive threshold denoising method based on Contourlet transform protects the edge details of the target in the radar image. Edge detection is performed through the Sobel operator and the local distance image edge threshold is calculated through the median filtering method. Further analyzing and processing the depth information obtained by the dTOF lidar, by extracting the key edge features, the detection system can maintain a high recognition accuracy even in complex road environments, improving the overall robustness of the system, being able to identify fine edges and structures, and being applicable to berth detection scenarios with high precision requirements. Brief Description of the Drawings

[0025] To make the above-mentioned berth detection system and method based on dTOF lidar of the present invention more obvious and understandable, the drawings required for the specific implementation manners of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative efforts.

[0026] Figure 1 It shows a schematic diagram of the berth detection method based on dTOF lidar; Figure 2 It shows a schematic diagram of the berth detection system of the inspection equipment; Figure 3 It shows a schematic diagram of the deployment position of the dTOF lidar; Figure 4 It shows a schematic diagram of the vehicle parking situation in the on-street berth. Detailed Description of the Invention

[0027] Example 1: A berth detection system and method based on dTOF lidar are provided. The steps of the berth detection method based on dTOF lidar are as Figure 1 shown, and the berth detection system of the inspection equipment is as Figure 2As shown, the system includes: an image acquisition module, a data processing module, a berth detection module, a structured light module, and a GPS positioning module, which are described in detail as follows.

[0028] The image acquisition module is used to acquire berth images through a camera.

[0029] In the image acquisition module, intelligent license plate recognition cameras installed on inspection equipment are used to acquire berth images.

[0030] After receiving the berth image video stream, the data processing module preprocesses the frames in the video stream and segments the license plate characters in the frames to obtain the license plate information of the vehicle.

[0031] The data processing module is used to preprocess the berth images and analyze the image data through image recognition algorithms.

[0032] The data processing module preprocesses the frames in the video stream, specifically including: annotating the text areas of the collected sample data to determine the positions of the texts in the document, thereby constructing a model to learn the spatial structure of the texts and improving the recognition accuracy of the texts; establishing label data corresponding to the text content while extracting the text content within the text area; cleaning and preprocessing the extracted text content to ensure the quality of the text data.

[0033] The image recognition algorithm includes the OCR text recognition algorithm, which supports multiple models, including support vector machines, Bayesian classification algorithms, and neural network models based on deep learning. The type of the model is determined according to specific application scenarios and requirements; after the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm through dataset optimization and feature selection. Among them, dataset optimization specifically includes training with a more accurate and comprehensive dataset and optimizing the dataset by deleting duplicate samples, increasing randomness, etc.; feature selection specifically includes screening features with high weights for the recognition task and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm. The selection of features is based on the feature evaluation method of information gain.

[0034] The berth detection module is used to detect the berth status through a dTOF lidar. Among them, the detection data provided by the lidar includes depth maps, point cloud maps, and confidence maps.

[0035] The structured light module is used to solve the wrapped phase and absolute phase through the four-step phase-shifting method and the multi-frequency heterodyne method, and obtain the contour information of the vehicle by projecting a specific light pattern and analyzing its deformation.

[0036] The structured light module specifically includes: calibrating the system to determine the geometric relationship and internal parameters among the camera, lidar, and structured light module; projecting a coded grating onto the surface of the object to be measured through the structured light module, collecting the reflected image by the camera, and emitting short pulse light and measuring the time of the returned light by the dTOF lidar to obtain the distance information of the vehicle; for the structured light data, solving the wrapped phase and absolute phase through the four-step phase-shift method and multi-frequency heterodyne method, and realizing the three-dimensional coordinate correspondence solution of the phase and the measured surface through the relative relationship between the calibrated camera and the projection module coordinate system; realizing high-precision stitching through the iterative closest point method; establishing the feature descriptors of adjacent point clouds, and using the Euclidean distance of the feature description vectors as the judgment criterion to judge the similarity of feature points, and adjusting the stitching matrix to minimize the deviation between the corresponding points of the stitched point cloud to achieve global optimization. The application of the structured light module can improve the detection accuracy of the lidar by about 8%.

[0037] The GPS positioning module is used to determine the real-time position of the inspection equipment.

[0038] In the berth detection module, the specific process of detecting the berth state by the dTOF lidar includes: emitting laser pulses to the berth area by the dTOF lidar and receiving the reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, and associating the berth state with the specific berth according to the real-time position of the inspection equipment provided by the GPS positioning module, so as to obtain a continuous berth state image; if the data processing module recognizes the license plate information of the vehicle, then pair the license plate information of the vehicle with the imaging information of the lidar.

[0039] The deployment position of the dTOF lidar is as Figure 3 shown. The dTOF lidar is installed at the front end of the inspection equipment and is connected to the edge computing host through an uplink serial cable.

[0040] The berth state information includes the distance between the current inspection equipment and the curb side of the berth, whether there is an object in the berth, and whether the object on the berth is a pedestrian, bicycle, electric vehicle, motorcycle, small vehicle, large vehicle, or other object, etc.

[0041] The vehicle parking situation in the on-street berth is as Figure 4 shown. If the berth detection module fails to extract the contour information of the vehicle in the berth area or detects that the relative position of the vehicle and the berth exceeds a certain distance, then the license plate information recognized by the data processing module is attributed to the vehicle outside the berth.

[0042] Embodiment 2: Based on the foregoing embodiments, the system is additionally provided with an edge detection module, which protects the edge details of the targets in the radar image through a lidar adaptive threshold denoising method based on the Contourlet transform, performs edge detection through the Sobel operator, and calculates the local distance image edge threshold through the median filtering method.

[0043] The environment is scanned by a dTOF lidar to obtain the distance image of the vehicle.

[0044] An imaging lidar adaptive threshold denoising method based on the Contourlet transform is used to suppress the distance abnormal noise while protecting the edge details of the targets in the radar image. The Contourlet transform formula is as follows:

[0045] Where represents the Contourlet transform coefficient after threshold adjustment; represents the Contourlet transform coefficient; represents the dynamic threshold adjustment factor for enhancing the denoising effect; represents the median of the absolute values of the Contourlet transform coefficients for estimating the noise level.

[0046] The edge information in the image is extracted through an edge detection algorithm. Edge detection is performed through the Sobel operator. The accuracy of edge positioning is improved by increasing the gradient detection direction, and the local distance image edge threshold is calculated through the median filtering method, thereby realizing the Sobel edge detection with an adaptive threshold. The Sobel operator formula is as follows:

[0047] Where represents the gradient amplitude at point , that is, the edge intensity of this point; represents the gradient in the horizontal direction, that is, the brightness change rate of the image in the direction; represents the gradient in the vertical direction, that is, the brightness change rate of the image in the direction; represents the gradient intensity adjustment factor for adjusting the sensitivity of edge detection.

[0048] Berth detection involves analyzing the edge information of a specific area to determine whether there are vacant berths.

[0049] The results of edge detection and berth detection are fused to generate a point cloud data containing distance and edge information.

[0050] For example, a parking lot is scanned by a dTOF lidar with the aim of detecting vacant parking spaces.

[0051] Assume that the median absolute deviation of the coefficients after Contourlet transform is 0.5, and is used to enhance the denoising effect:

[0052] Assume that at a certain point, the horizontal and vertical gradients calculated by the Sobel operator are Gx = 2 and Gy = 1 respectively, and β = 1.2 is used to adjust the gradient intensity:

[0053] The effect of the edge detection module is shown in Table 1.

[0054] Table 1. Summary of the effect of the edge detection module

[0055] As shown in Table 1, the method optimized by the edge detection module has significantly improved the detection accuracy of parking spaces, and the false detection rate and detection time have both decreased; this indicates that the edge detection module can effectively improve the recognition accuracy of the parking detection system for temporarily parked vehicles.

[0056] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable non-transitory storage media containing computer-usable program code.

[0057] The present invention can provide computer program instructions to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the system.

[0058] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions of the system.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions of the system.

Claims

1. Berth detection system, characterized by: include: An image acquisition module, used for acquiring berth images through a camera; A data processing module, used to pre-process the berth image and analyze the image data through an image recognition algorithm; A berth detection module is used to detect the berth status through a dTOF laser radar, wherein the detection data provided by the laser radar includes a depth map, a point cloud map and a confidence map; The structured light module is used to solve the wrapped phase and absolute phase through the four-step phase shift method and the multi-frequency heterodyne method, and to obtain the vehicle's profile information by projecting a specific light pattern and analyzing its deformation; GPS positioning module is used to determine the real-time location of the inspection equipment.

2. The berth detection system according to claim 1, characterized in that: After receiving the parking space image video stream, the data processing module pre-processes the picture frames in the video stream and segments the license plate characters in the picture frames, thereby obtaining the vehicle license plate information.

3. The berth detection system according to claim 2, characterized in that: The data processing module preprocesses the picture frames in the video stream, specifically including: determining the position of the text in the document by marking the text area of ​​the sample data; establishing label data corresponding to the text content while extracting the text content in the text area; and cleaning and preprocessing the extracted text content.

4. The berth detection system according to claim 1, characterized in that: In the berth detection module, the detection of the berth status by the dTOF laser radar specifically includes: emitting laser pulses to the berth area by the dTOF laser radar and receiving reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, and associating the berth status with the specific berth according to the real-time position of the inspection equipment provided by the GPS positioning module, thereby obtaining a continuous berth status image; if the data processing module recognizes the license plate information of the vehicle, the license plate information of the vehicle is paired with the imaging information of the laser radar.

5. The berth detection system according to claim 4, characterized in that: If the parking space detection module fails to extract the outline information of the vehicle in the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module is attributed to the vehicle outside the parking space.

6. The berth detection system according to claim 1, characterized in that: The structured light module specifically includes: determining the geometric relationship and internal parameters between the camera, the laser radar and the structured light module by calibrating the system; projecting a coded grating onto the surface of the object to be measured by the structured light module, collecting the reflected image by the camera, emitting short pulse light by the dTOF laser radar and measuring the time of the return light to obtain the distance information of the vehicle; for the structured light data, solving the wrapped phase and absolute phase by the four-step phase shift method and the multi-frequency heterodyne method, and realizing the correspondence between the phase and the three-dimensional coordinates of the measured surface by the relative relationship between the calibrated camera and projection module coordinate systems; realizing high-precision splicing by the iterative nearest point method; establishing feature descriptors of adjacent point clouds, and judging the similarity of feature points by using the Euclidean distance of the feature description vector as the judgment criterion, and minimizing the deviation between the corresponding points of the spliced ​​point cloud by adjusting the splicing matrix to achieve global optimization.

7. The berth detection system according to any one of claims 1 to 6, characterized in that: The system also includes an edge detection module, which protects the edge details of the target in the radar image through a lidar adaptive threshold denoising method based on Contourlet transform, performs edge detection through a Sobel operator, and calculates the edge threshold of the local range image through a median filtering method.

8. The berth detection system according to claim 7, characterized in that: The imaging lidar adaptive threshold denoising method based on Contourlet transform is used to suppress abnormal distance noise while protecting the edge details of the target in the radar image. The Contourlet transform formula is as follows: in, Represents the Contourlet transform coefficients after threshold adjustment; Represents the Contourlet transform coefficient; Represents the dynamic threshold adjustment factor, which is used to enhance the denoising effect; Represents the median of the absolute values ​​of the Contourlet transform coefficients, used to estimate the noise level.

9. The berth detection system according to claim 6, characterized in that: The edge information in the image is extracted through the edge detection algorithm, and the edge detection is performed through the Sobel operator. The accuracy of edge positioning is improved by increasing the gradient detection direction, and the local distance image edge threshold is calculated through the median filtering method, thereby realizing the Sobel edge detection with adaptive threshold. The Sobel operator formula is as follows: in, Indicates at point The gradient amplitude at , that is, the edge strength of the point; Represents the gradient in the horizontal direction, that is, the image The rate of change of brightness in a direction; Represents the gradient in the vertical direction, that is, the image The rate of change of brightness in a direction; Represents the gradient strength adjustment factor, which is used to adjust the sensitivity of edge detection.

10. A berth detection method, characterized in that: The method is implemented based on the system according to any one of claims 1 to 9: The method comprises: Collect berth images through cameras; Preprocessing the berth image and analyzing the image data through image recognition algorithms; The dTOF laser radar emits laser pulses to the berth area and receives reflected signals to extract the vehicle's contour information and calculate the vehicle's relative position; Determine whether the vehicle is in the berth based on the berth status and the real-time position of the inspection equipment.

Images