A berth detection system and method based on dTOF laser radar

Abstract

The present invention discloses a berth detection system and method based on dTOF laser radar, which relates to the field of intelligent transportation technology. The system includes: an image acquisition module for acquiring berth images through a camera; a data processing module for preprocessing berth images and analyzing image data through an image recognition algorithm; a berth detection module for detecting berth status through a dTOF laser radar; and a GPS positioning module for determining the real-time position of inspection equipment. According to the technical solution of the present application, the function of identifying temporarily parked vehicles near berths can be realized, avoiding the occurrence of incorrect charging of temporarily parked vehicles by inspection equipment, and having high application value.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a berth detection system and method based on dTOF laser radar. Background Art

[0002] In recent years, the number of motor vehicles in my country has continued to grow, and cars have become an indispensable means of transportation for people's daily travel, and the problem of parking difficulties has become increasingly prominent. Especially in busy commercial areas, residential areas and near public transportation hubs, finding a suitable parking space has become a headache for many drivers. In order to improve the utilization rate of road parking spaces, many road intelligent inspection devices have appeared on the market, which reduce manual participation and improve the efficiency of road inspection. The inspection equipment determines whether the vehicle is parked in the berth by detecting the relative position of the license plate, the size of the license plate, the rotation angle of the license plate and other information contained in the captured image. In some areas, the road parking spaces are adjacent to the shoulders. When a vehicle is parked at the position of the shoulder sidewalk, the inspection equipment passes by, and the inspection process is triggered by the license plate of such vehicles. Because the license plate information of the vehicle in this case is very similar to the license plate information of the vehicle parked in the berth, the inspection equipment often mistakenly matches the vehicle parked near the berth to the berth, resulting in abnormal inspection data. Therefore, based on this problem, the present application proposes a berth detection system and method based on dTOF laser radar. Summary of the invention

[0003] Technical Purpose

[0004] In order to solve the above problems, the purpose of the present invention is to provide a berth detection system and method based on dTOF laser radar, which can not only realize the identification function of temporarily parked vehicles near the berth, but also can realize the identification of the type of parked vehicles in the berth. The method and system have a wide range of application scenarios and avoid the occurrence of incorrect charging of temporarily parked vehicles by inspection equipment.

[0005] Technical Solution

[0006] In order to achieve the above-mentioned purpose, the present invention provides a parking space detection system and method based on dTOF (Direct Time-of-Flight) laser radar, which recognizes the vehicle license plate through image segmentation and OCR text recognition algorithm, and emits laser pulses to the parking space area through the dTOF laser radar and receives the reflected signal to extract the vehicle's contour information and calculate the vehicle's relative position, thereby realizing the identification function of temporarily parked vehicles near the parking space.

[0007] In a first aspect, the present invention provides a berth detection system based on dTOF laser radar, comprising:

[0008] An image acquisition module, used for acquiring berth images through a camera;

[0009] A data processing module, used to pre-process the berth image and analyze the image data through an image recognition algorithm;

[0010] A berth detection module is used to detect the berth status through a dTOF laser radar, wherein the detection data provided by the laser radar includes a depth map, a point cloud map and a confidence map;

[0011] The structured light module is used to solve the wrapped phase and absolute phase through the four-step phase shift method and the multi-frequency heterodyne method, and to obtain the vehicle's profile information by projecting a specific light pattern and analyzing its deformation;

[0012] GPS positioning module is used to determine the real-time location of the inspection equipment.

[0013] Furthermore, in the image acquisition module, the parking space image is acquired by an intelligent license plate recognition camera installed on the inspection equipment.

[0014] Furthermore, after receiving the parking space image video stream, the data processing module obtains the vehicle license plate information by preprocessing the picture frames in the video stream and segmenting the license plate characters in the picture frames. It can accurately locate and segment the license plate area from the captured image and improve the image quality and highlight the characteristics of the license plate through image preprocessing; it can quickly and accurately identify the vehicle license plate information, thereby improving the operational efficiency and management level of the parking space.

[0015] Furthermore, the data processing module pre-processes the picture frames in the video stream, specifically including: marking the text area of ​​the collected sample data to determine the position of the text in the document, thereby building a model to learn the spatial structure of the text and improve the recognition accuracy of the text; while extracting the text content in the text area, establishing label data corresponding to the text content; and cleaning and pre-processing the extracted text content to ensure the quality of the text data.

[0016] Furthermore, the image recognition algorithm includes an OCR text recognition algorithm, which supports multiple models, including support vector machines, Bayesian classification algorithms and neural network models based on deep learning, and the type of model is determined according to specific application scenarios and requirements; after the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm through data set optimization and feature selection, wherein the data set optimization specifically includes using a more accurate and comprehensive data set for training, and optimizing the data set by deleting duplicate samples, increasing randomness, etc.; feature selection specifically includes screening features with high weight for the recognition task, and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm, and the feature selection is based on the feature evaluation method of information gain.

[0017] Furthermore, in the berth detection module, the berth status detection by dTOF laser radar specifically includes: emitting laser pulses to the berth area by dTOF laser radar and receiving reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, and correlating the berth status with the specific berth according to the real-time position of the inspection equipment provided by the GPS positioning module, so as to obtain a continuous berth status image; if the data processing module recognizes the license plate information of the vehicle, the license plate information of the vehicle is paired with the imaging information of the laser radar. It can provide accurate location information; can provide stable measurement results under poor lighting conditions; and improve the operational efficiency of the berth by quickly and accurately identifying the contour information and relative position of the vehicle.

[0018] Furthermore, the dTOF laser radar is installed at the front end of the inspection equipment and connected to the edge computing host through an uplink serial cable.

[0019] Furthermore, the berth status information includes the distance between the current inspection equipment and the curb side of the berth, whether there is an object in the berth, whether the object in the berth is a pedestrian, a bicycle, an electric vehicle, a motorcycle, a small vehicle, a large vehicle, or other objects, etc.

[0020] Furthermore, if the parking space detection module fails to extract the outline information of the vehicle in the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module is attributed to the vehicle outside the parking space.

[0021] Furthermore, the structured light module solves the wrapped phase and absolute phase through the four-step phase shift method and the multi-frequency heterodyne method, and obtains the vehicle's profile information by projecting a specific light pattern and analyzing its deformation. It can improve the precision and accuracy of LiDAR measurements, expand the measurement range while maintaining high precision, and make the detection system suitable for a wider range of application scenarios; it can reduce the complexity of the system, thereby achieving a more compact system design.

[0022] Furthermore, the system also includes an edge detection module, which protects the edge details of the target in the radar image through a lidar adaptive threshold denoising method based on Contourlet transform, performs edge detection through the Sobel operator, and calculates the local distance image edge threshold through the median filtering method. It can further analyze and process the depth information obtained by the dTOF lidar, and by extracting key edge features, the detection system can maintain a high recognition accuracy in complex road environments, thereby improving the overall robustness of the system; it can recognize small edges and structures, and is suitable for parking detection scenarios with high precision requirements.

[0023] In a second aspect, the present invention further provides a berth detection method based on dTOF laser radar, the method is based on the system described in the first aspect, and comprises:

[0024] Collect berth images through cameras;

[0025] Preprocessing the berth image and analyzing the image data through image recognition algorithms;

[0026] The dTOF laser radar emits laser pulses to the berth area and receives reflected signals to extract the vehicle's contour information and calculate the vehicle's relative position;

[0027] Determine whether the vehicle is in the berth based on the berth status and the real-time position of the inspection equipment.

[0028] In a third aspect, the present invention also provides a computer device, comprising a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the computer device executes the aforementioned berth detection method based on dTOF lidar.

[0029] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the aforementioned berth detection method based on dTOF laser radar is implemented.

[0030] The present invention recognizes the license plate of the vehicle through image segmentation and OCR text recognition algorithm; emits laser pulses to the berth area through dTOF laser radar and receives reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle; solves the wrapped phase and absolute phase through the four-step phase shift method and multi-frequency heterodyne method, and obtains the contour information of the vehicle by projecting a specific light pattern and analyzing its deformation; protects the edge details of the target in the radar image through the laser radar adaptive threshold denoising method based on Contourlet transform, performs edge detection through the Sobel operator and calculates the local distance image edge threshold through the median filtering method. The system and method have a wide range of application scenarios, realize the identification function of temporarily parked vehicles near the berth, and avoid the occurrence of incorrect charging of temporarily parked vehicles by inspection equipment.

[0031] Beneficial Effects

[0032] By implementing the berth detection system and method based on dTOF laser radar provided by the present invention, the following technical effects are achieved:

[0033] (1) The system uses image segmentation and OCR text recognition algorithms to identify vehicle license plates. It can accurately locate and segment the license plate area from the captured image and improve the image quality and highlight the features of the license plate through image preprocessing. It can quickly and accurately identify the vehicle's license plate information, thereby improving the operational efficiency and management level of the parking space.

[0034] (2) The dTOF lidar emits laser pulses to the berth area and receives reflected signals to extract the vehicle's contour information and calculate the vehicle's relative position. It can provide accurate position information and provide stable measurement results even in poor lighting conditions, quickly and accurately identifying the vehicle's contour information and relative position.

[0035] (3) The system adds a four-step phase shift method and a multi-frequency heterodyne method to solve the wrapped phase and absolute phase. By projecting a specific light pattern and analyzing its deformation to obtain the vehicle's contour information, the system further improves the precision and accuracy of the lidar measurement. While maintaining high precision, it expands the measurement range, making the detection system suitable for a wider range of application scenarios, reducing the complexity of the system, and thus achieving a more compact system design.

[0036] (4) The Contourlet transform-based lidar adaptive threshold denoising method protects the edge details of the target in the radar image. The Sobel operator is used for edge detection and the median filtering method is used to calculate the edge threshold of the local distance image. The depth information obtained by the dTOF lidar is further analyzed and processed. By extracting key edge features, the detection system can maintain a high recognition accuracy even in complex road environments, thereby improving the overall robustness of the system and being able to identify small edges and structures. It is suitable for berth detection scenarios with high precision requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the above-mentioned berth detection system and method based on dTOF laser radar of the present invention more obvious and easy to understand, the drawings required for use in the specific implementation of the present invention are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic diagram showing a berth detection method based on dTOF laser radar;

[0039] Figure 2 It shows the schematic diagram of the berth detection system of the inspection equipment;

[0040] Figure 3 A schematic diagram showing the deployment location of the dTOF laser radar;

[0041] Figure 4 Schematic diagram showing the parking situation of vehicles in on-street parking spaces. DETAILED DESCRIPTION

[0042] Embodiment 1:

[0043] A berth detection system and method based on dTOF laser radar are provided. The steps of the berth detection method based on dTOF laser radar are as follows: Figure 1 As shown in the figure, the inspection equipment berth detection system is as follows Figure 2 As shown, the system includes: an image acquisition module, a data processing module, a berth detection module, a structured light module and a GPS positioning module, as described in detail below.

[0044] The image acquisition module is used to acquire berth images through a camera.

[0045] In the image acquisition module, the parking space image is acquired by an intelligent license plate recognition camera installed on the inspection equipment.

[0046] After receiving the parking space image video stream, the data processing module pre-processes the picture frames in the video stream and segments the license plate characters in the picture frames, thereby obtaining the license plate information of the vehicle.

[0047] The data processing module is used to pre-process the berth image and analyze the image data through an image recognition algorithm.

[0048] The data processing module preprocesses the picture frames in the video stream, specifically including: marking the text area of ​​the collected sample data to determine the position of the text in the document, thereby building a model to learn the spatial structure of the text and improve the recognition accuracy of the text; while extracting the text content in the text area, establishing label data corresponding to the text content; and cleaning and preprocessing the extracted text content to ensure the quality of the text data.

[0049] The image recognition algorithm includes an OCR text recognition algorithm, which supports multiple models, including support vector machines, Bayesian classification algorithms and neural network models based on deep learning. The type of model is determined according to the specific application scenario and requirements. After the model training is completed, the data processing module improves the recognition accuracy of the OCR text recognition algorithm by means of data set optimization and feature selection. Among them, the data set optimization specifically includes using a more accurate and comprehensive data set for training, and optimizing the data set by deleting duplicate samples, increasing randomness, etc.; feature selection specifically includes screening features with high weight for the recognition task and reducing the proportion of unnecessary features to reduce the computational complexity of the OCR text recognition algorithm. The feature selection is based on the feature evaluation method of information gain.

[0050] The berth detection module is used to detect the berth status through the dTOF laser radar, where the detection data provided by the laser radar includes a depth map, a point cloud map and a confidence map.

[0051] The structured light module is used to solve the wrapped phase and absolute phase through the four-step phase shift method and the multi-frequency heterodyne method, and to obtain the vehicle's contour information by projecting a specific light pattern and analyzing its deformation.

[0052] The structured light module specifically includes: calibrating the system to determine the geometric relationship and internal parameters between the camera, laser radar and structured light module; projecting the coded grating to the surface of the object to be measured through the structured light module, the camera collects the reflected image, and the dTOF laser radar emits short pulse light and measures the time of the return light to obtain the distance information of the vehicle; for the structured light data, the wrapped phase and absolute phase are solved by the four-step phase shift method and the multi-frequency heterodyne method, and the phase and the three-dimensional coordinates of the measured surface are solved by the relative relationship between the calibrated camera and projection module coordinate systems; high-precision splicing is achieved by the iterative nearest point method; by establishing feature descriptors of adjacent point clouds, and using the Euclidean distance of the feature description vector as the judgment criterion to judge the similarity of feature points, the splicing matrix is ​​adjusted to minimize the deviation between the corresponding points of the spliced ​​point cloud to achieve global optimization. The application of the structured light module can improve the detection accuracy of the laser radar by about 8%.

[0053] The GPS positioning module is used to determine the real-time location of the inspection equipment.

[0054] In the berth detection module, the detection of the berth status by the dTOF laser radar specifically includes: emitting laser pulses to the berth area by the dTOF laser radar and receiving reflected signals to extract the contour information of the vehicle and calculate the relative position of the vehicle, and associating the berth status with the specific berth according to the real-time position of the inspection equipment provided by the GPS positioning module, thereby obtaining a continuous berth status image; if the data processing module recognizes the license plate information of the vehicle, the license plate information of the vehicle is paired with the imaging information of the laser radar.

[0055] dTOF LiDAR deployment location Figure 3 As shown in the figure, the dTOF laser radar is installed at the front end of the inspection equipment and connected to the edge computing host through an uplink serial cable.

[0056] The berth status information includes the distance between the current inspection equipment and the curb side of the berth, whether there is an object in the berth, whether the object in the berth is a pedestrian, a bicycle, an electric vehicle, a motorcycle, a small vehicle, a large vehicle, or other objects, etc.

[0057] The parking situation of vehicles in on-street parking spaces is as follows Figure 4As shown, if the parking space detection module fails to extract the contour information of the vehicle in the parking space area or detects that the relative position of the vehicle and the parking space exceeds a certain distance, the license plate information identified by the data processing module is attributed to the vehicle outside the parking space.

[0058] Embodiment 2:

[0059] On the basis of the above-mentioned embodiment, the system adds an edge detection module, which protects the edge details of the target in the radar image through a lidar adaptive threshold denoising method based on Contourlet transform, performs edge detection through the Sobel operator, and calculates the local range image edge threshold through the median filtering method.

[0060] The dTOF lidar is used to scan the environment to obtain the vehicle's range image.

[0061] The imaging lidar adaptive threshold denoising method based on Contourlet transform is used to suppress abnormal distance noise while protecting the edge details of the target in the radar image. The Contourlet transform formula is as follows:

[0062]

[0063] in, Represents the Contourlet transform coefficients after threshold adjustment; Represents the Contourlet transform coefficient; Represents the dynamic threshold adjustment factor, which is used to enhance the denoising effect; Represents the median of the absolute values ​​of the Contourlet transform coefficients, used to estimate the noise level.

[0064] The edge information in the image is extracted through the edge detection algorithm, and the edge detection is performed through the Sobel operator. The accuracy of edge positioning is improved by increasing the gradient detection direction, and the local distance image edge threshold is calculated through the median filtering method, thereby realizing the Sobel edge detection with adaptive threshold. The Sobel operator formula is as follows:

[0065]

[0066] in, Indicates at point The gradient amplitude at , that is, the edge strength of the point; Represents the gradient in the horizontal direction, that is, the image The rate of change of brightness in a direction; Represents the gradient in the vertical direction, that is, the image The rate of change of brightness in a direction; Represents the gradient strength adjustment factor, which is used to adjust the sensitivity of edge detection.

[0067] Berth detection involves analyzing edge information of a specific area to determine if there are any vacant berths.

[0068] The results of edge detection and parking space detection are fused to generate a point cloud data containing distance and edge information.

[0069] For example, a parking lot is scanned by a dTOF lidar to detect vacant parking spaces.

[0070] Assuming that the median absolute deviation of the coefficients after Contourlet transformation is 0.5, use To enhance the denoising effect:

[0071]

[0072] Assume that at a certain point, the horizontal and vertical gradients calculated by the Sobel operator are and ,use To adjust the gradient strength:

[0073]

[0074] The effect of edge detection module is shown in Table 1.

[0075] Table 1. Summary of edge detection module effects

[0076]

[0077] As shown in Table 1, the method optimized by the edge detection module has significantly improved the detection accuracy of parking spaces, and the false detection rate and detection time have been reduced; this shows that the edge detection module can effectively improve the recognition accuracy of the parking space detection system for temporarily parked vehicles.

[0078] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transient storage media containing computer-usable program code.

[0079] The present invention can provide computer program instructions to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the system.

[0080] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions of the system.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions of the system.

Claims

1. A berth detection system based on dTOF laser radar, characterized in that: include: An image acquisition module, used for acquiring berth images through a camera; A data processing module, used to pre-process the berth image and analyze the image data through an image recognition algorithm; The berth detection module is used to detect the berth status through the dTOF laser radar, specifically including: emitting laser pulses to the berth area through the radar and receiving the reflected signal to extract the contour information of the vehicle and calculate the relative position of the vehicle, and associating the berth status with the specific berth according to the real-time position of the inspection equipment; if the data processing module recognizes the license plate information of the vehicle, the license plate information of the vehicle is paired with the imaging information of the radar, and if the contour information of the vehicle is not extracted in the berth area or the relative position of the vehicle is detected to exceed a certain distance, the recognized license plate information is attributed to the vehicle outside the berth; the detection data provided by the radar includes a depth map, a point cloud map and a confidence map; The structured light module calibrates the system to determine the geometric relationship and internal parameters between the camera, lidar and structured light module. The structured light module projects a coded grating onto the surface of the object being measured, and the camera collects the reflected image. The dTOF lidar emits short pulses of light and measures the time of the return light to obtain the distance information of the vehicle. For structured light data, the four-step phase shift method and multi-frequency heterodyne method are used to solve the wrapped phase and absolute phase, and the contour information of the vehicle is obtained by projecting a specific light pattern and analyzing its deformation. GPS positioning module is used to determine the real-time location of the inspection equipment.

2. The system according to claim 1, characterized in that: After receiving the parking space image video stream, the data processing module pre-processes the picture frames in the video stream and segments the license plate characters in the picture frames, thereby obtaining the license plate information of the vehicle.

3. The system according to claim 2, characterized in that: The data processing module preprocesses the picture frames in the video stream, specifically including: determining the position of the text in the document by marking the text area of ​​the sample data; establishing label data corresponding to the text content while extracting the text content in the text area; and cleaning and preprocessing the extracted text content.

4. The system according to any one of claims 1 to 3, characterized in that: The system also includes an edge detection module, which protects the edge details of the target in the radar image through a lidar adaptive threshold denoising method based on Contourlet transform, performs edge detection through a Sobel operator, and calculates the edge threshold of the local range image through a median filtering method.

5. The system according to claim 4, characterized in that: The Contourlet transform formula is as follows: in, Represents the Contourlet transform coefficients after threshold adjustment; Represents the Contourlet transform coefficient; Represents the dynamic threshold adjustment factor, which is used to enhance the denoising effect; Represents the median of the absolute values ​​of the Contourlet transform coefficients, used to estimate the noise level.

6. A berth detection method based on dTOF laser radar, characterized in that: The method is implemented based on the system according to any one of claims 1 to 5: The method comprises: Collect berth images through cameras; Preprocessing the berth image and analyzing the image data through image recognition algorithms; The dTOF laser radar emits laser pulses to the berth area and receives reflected signals to extract the vehicle's contour information and calculate the vehicle's relative position; Determine whether the vehicle is in the berth based on the berth status and the real-time position of the inspection equipment.

7. A computer device comprising a processor and a memory, wherein the processor is connected to the memory, and the memory is used to store a computer program, wherein: The processor is configured to execute the computer program stored in the memory, so that the computer device performs at least one step in the method of claim 6.

8. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed, at least one step of the method according to claim 6 is implemented.

Images