Toll vehicle type detection method and system based on fusion of laser radar and bayonet camera

Abstract

The invention relates to the technical field of intelligent traffic, in particular to a toll vehicle type detection method and system based on fusion of a laser radar and a gate camera, and the method comprises the steps: scanning a vehicle entering a toll station through the laser radar, and obtaining the three-dimensional point cloud data of the vehicle; performing feature extraction on the three-dimensional point cloud data to obtain vehicle size information; acquiring two-dimensional image data of a vehicle entering a toll station through a bayonet camera, and identifying the type of the vehicle to obtain vehicle type information; comparing and matching the vehicle size information and the vehicle type information, and analyzing the vehicle size information and the vehicle type information through a multi-modal fusion algorithm to obtain complete vehicle information; and based on the vehicle complete information, calculating the receivable fee of the vehicle according to the vehicle type charging rule. According to the invention, the vehicle identification precision in a complex traffic environment is effectively improved, the dual verification of the vehicle type is realized, false detection and missing detection are effectively reduced, and the overall reliability of a toll collection system is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a toll vehicle type detection method and system based on the fusion of a laser radar and a bayonet camera. Background Art

[0002] In the field of vehicle model detection for toll collection, traditional license plate recognition and vehicle appearance classification methods are widely used in toll collection systems. These technologies mainly rely on bayonet cameras for vehicle recognition, and determine the model by matching license plate information and vehicle image features. However, in complex traffic scenarios, such as at night, in low visibility, with strong reflections, and in dense traffic, methods that rely solely on image recognition are easily affected by the environment, resulting in reduced accuracy in model classification and detection. In particular, when faced with models of similar size and shape, traditional image recognition methods are prone to misjudgment and are difficult to meet high-precision toll collection requirements.

[0003] In recent years, driven by the intelligent transportation system, laser radar and image fusion technology have been gradually introduced for toll vehicle type detection. With the construction of smart cities and the growing demand for automated toll collection, domestic intelligent toll system technology has also been gradually updated. In view of special situations such as dense traffic and diversified vehicle types, domestic research focuses on solving the problem of vehicle type detection and toll collection in complex environments, and explores target detection methods based on laser radar to overcome the limitations of traditional cameras in low visibility environments. Through multi-sensor joint modeling, point cloud and image fusion are used for vehicle recognition and classification, and multimodal features are combined for transfer learning to enhance the generalization ability of the model in different toll stations and road environments. In the vehicle toll model detection technology at home and abroad, although the laser radar and image fusion technology has made certain progress, there are still problems such as insufficient recognition accuracy in complex environments, high algorithm model calculation cost, difficulty in real-time application on edge devices, and relatively limited data resources. Therefore, the toll vehicle type detection method based on the fusion of laser radar and bayonet camera has become one of the current research directions. Summary of the invention

[0004] The present invention provides a toll vehicle type detection method and system based on the fusion of laser radar and bayonet camera, which can effectively solve the problems in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: A toll vehicle type detection method based on the fusion of laser radar and bayonet camera, the method comprising: Scan the vehicles entering the toll booths through LiDAR to obtain the three-dimensional point cloud data of the vehicles; Extracting features from the three-dimensional point cloud data to obtain vehicle size information; Collecting two-dimensional image data of vehicles entering the toll station through a bayonet camera, identifying the type of the vehicle according to the two-dimensional image data, and obtaining vehicle type information; Comparing and matching the vehicle size information and the vehicle type information, and analyzing the vehicle size information and the vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; Based on the complete vehicle information, the chargeable fees for the vehicle are calculated according to the vehicle type charging rules.

[0006] Furthermore, obtaining the three-dimensional point cloud data of the vehicle includes: continuously emitting high-frequency laser pulses through the laser radar, and receiving signals reflected from the surface of the vehicle to construct the three-dimensional point cloud data of the vehicle.

[0007] Furthermore, the extracting features of the three-dimensional point cloud data to obtain vehicle size information includes: Preprocessing the three-dimensional point cloud data; The 3D object detection algorithm is optimized, and feature extraction is performed on the three-dimensional point cloud data using the optimized 3D object detection algorithm to obtain vehicle size information.

[0008] Furthermore, the collecting of two-dimensional image data of a vehicle entering a toll station by a bayonet camera, identifying the type of the vehicle according to the two-dimensional image data, and obtaining the vehicle type information includes: Collecting two-dimensional image data of the vehicle entering the toll station through the bayonet camera; Preprocessing the two-dimensional image data, identifying the vehicle type of the vehicle using a deep learning algorithm, and outputting a vehicle bounding box of the vehicle; Based on the vehicle boundary frame, the license plate number of the vehicle is extracted by a license plate recognition algorithm, and the vehicle type and the license plate number are integrated to obtain vehicle type information.

[0009] Furthermore, it also includes using image processing technology to optimize the license plate area when extracting the license plate area in the vehicle boundary frame.

[0010] Furthermore, the acquisition frequencies of the laser radar and the bayonet camera are aligned through clock synchronization technology.

[0011] Furthermore, it also includes using a high-precision clock source to achieve time synchronization between the laser radar and the bayonet camera.

[0012] Furthermore, comparing and matching the vehicle size information and the vehicle type information includes: The three-dimensional point cloud data collected by the laser radar and the two-dimensional image data collected by the bayonet camera are mapped into the same coordinate system through a spatial alignment algorithm.

[0013] A toll vehicle type detection system based on the fusion of laser radar and bayonet camera, the system comprising: The laser radar module scans the vehicle entering the toll station through the laser radar to obtain the three-dimensional point cloud data of the vehicle; extracts features from the three-dimensional point cloud data to obtain the vehicle size information; A bayonet camera module collects two-dimensional image data of vehicles entering the toll station through a bayonet camera, identifies the type of the vehicle according to the two-dimensional image data, and obtains vehicle type information; A time synchronization module, which aligns the acquisition frequencies of the laser radar and the bayonet camera through clock synchronization technology; A data fusion module compares and matches the vehicle size information and the vehicle type information, and analyzes the vehicle size information and the vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; The charging calculation module calculates the chargeable fee of the vehicle based on the complete information of the vehicle and the charging rules of the vehicle model.

[0014] Furthermore, it also includes integrating auditing and recording functions in the fee calculation module to record detailed information of each receivable fee.

[0015] The technical solution of the present invention can achieve the following technical effects: 1. Improve the accuracy of vehicle type recognition: By integrating the detection information of LiDAR and bayonet camera, the system can identify the vehicle type more accurately. LiDAR provides the three-dimensional size data of the vehicle, while the image recognition of the bayonet camera supplements the vehicle appearance feature information. The combination of the two can effectively improve the recognition accuracy.

[0016] 2. More accurate size determination: The system can more accurately determine the size of large trucks, light vehicles, buses and other vehicles. The geometric size information captured by the lidar helps to distinguish vehicles of similar size but different classifications, such as distinguishing large trucks from small trucks or buses, to avoid recognition errors caused by similar appearance.

[0017] 3. Dual verification of license plate recognition and size information: By combining license plate recognition with the vehicle's geometric size data, the system implements dual verification of vehicle type. By verifying each other through two independent information sources, it can effectively reduce false detections and missed detections, and further improve the accuracy of vehicle type recognition.

[0018] 4. Improve the reliability of the toll collection system: The multimodal fusion method significantly enhances the overall reliability of the toll collection system. The mutual verification of multiple detection information ensures that a high recognition accuracy can be maintained even in complex traffic environments, thereby reducing the risk of erroneous charging and adapting to the actual needs of various traffic scenarios.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flowchart of a toll vehicle type detection method based on the fusion of laser radar and bayonet camera; Figure 2 A schematic diagram of a process for obtaining vehicle type information; Figure 3 This is a schematic diagram of the structure of the toll vehicle model detection system based on the fusion of lidar and bayonet camera. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0024] Embodiment 1: like Figure 1 As shown, a toll vehicle type detection method based on the fusion of laser radar and bayonet camera includes: S1: Scan the vehicle entering the toll station through the laser radar to obtain the three-dimensional point cloud data of the vehicle; S2: Extract features from the 3D point cloud data to obtain vehicle size information; Specifically, step S1 scans the vehicle entering the toll station by emitting high-frequency laser pulses. When the vehicle enters the toll station, the laser radar module of the system starts immediately and starts scanning the vehicle to generate the three-dimensional point cloud data of the vehicle. The high-frequency laser pulses emitted should cover the area where the vehicle passes through, and scan from multiple angles and time points to ensure the rapid generation of high-resolution three-dimensional point cloud data; step S2 can accurately extract the geometric size characteristics of the vehicle by processing the point cloud data. This information is the key basis for distinguishing different types of vehicles, such as trucks, buses, and small cars. Compared with traditional visual sensors, laser radar is not affected by changes in light, so it can still stably capture the three-dimensional data of the vehicle in complex environments such as night, cloudy days, and strong light, ensuring the reliability of the system under all-weather and various weather conditions; in addition, the three-dimensional data captured by the laser radar can lay the foundation for the subsequent multi-modal data fusion with the bayonet camera. The three-dimensional size information provides richer input features for image recognition, which can supplement the shortcomings of image detection in size judgment, thereby improving the overall recognition accuracy of the system. This fusion not only improves the accuracy and reliability of vehicle model recognition, but also reduces the wrong charges caused by misjudgment of a single sensor.

[0025] S3: collecting two-dimensional image data of vehicles entering the toll station through the bayonet camera, identifying the type of the vehicle according to the two-dimensional image data, and obtaining vehicle type information; In this embodiment, the toll booth camera is responsible for collecting high-definition images of vehicles passing through the toll booth. After the images are collected, advanced algorithms can be used to identify the vehicle information. The image information collected by the toll booth camera not only improves the accuracy of vehicle identification, but also provides image feature-level support for multimodal data fusion. Combined with the three-dimensional size data captured by the lidar, the system is more accurate in vehicle model identification and toll judgment.

[0026] S4: compare and match the vehicle size information and the vehicle type information, and analyze the vehicle size information and the vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; Specifically, after completing the data processing of the laser radar and the bayonet camera, the system compares and matches the vehicle size information measured by the laser radar with the vehicle type identified by the bayonet camera to ensure that the two data correspond to the same position and size of the same vehicle. If the vehicle type identification of the bayonet camera is inaccurate, the system will perform a secondary verification based on the size information provided by the laser radar. After the data alignment is completed, the type and geometric size of the vehicle can be comprehensively analyzed through the multimodal fusion algorithm. Combining the three-dimensional size data provided by the laser radar and the license plate and appearance information captured by the bayonet camera, the system can have higher accuracy in vehicle classification and size judgment; the data fusion algorithm can associate the geometric features of the laser radar with the image features of the camera through deep learning models or rule matching, thereby forming an overall judgment of the vehicle. With the help of multimodal data fusion, this solution can effectively identify different models, distinguish vehicles of similar sizes, and reduce false detections through dual confirmation of license plates and sizes; this multimodal data fusion greatly improves the accuracy and robustness of vehicle recognition, allowing the system to maintain efficient recognition performance in complex traffic scenes.

[0027] S5: Based on the complete vehicle information, calculate the vehicle's chargeable fees according to the vehicle model charging rules.

[0028] Specifically, the complete vehicle information received in this step includes the vehicle's identification results and geometric size features. At the same time, the system will also extract the license plate number to ensure accurate association with the charging record. After obtaining this information, it can be automatically calculated according to the pre-set charging standards. These charging standards usually include vehicle type classification, size differences, and possible traffic policies, such as charging adjustments during peak hours and holidays. By matching and processing vehicle information, the receivables can be quickly generated according to the corresponding charging rules, reducing manual intervention, improving charging efficiency, and reducing the risk of inaccurate charging due to human errors.

[0029] The present invention effectively improves the vehicle recognition accuracy in complex traffic environments by integrating the detection information of the laser radar and the bayonet camera, realizes double verification of vehicle types, effectively reduces false detections and missed detections, further improves the accuracy of vehicle type recognition, and enhances the overall reliability of the toll collection system.

[0030] On the basis of the above-mentioned embodiment, obtaining the three-dimensional point cloud data of the vehicle includes: continuously emitting high-frequency laser pulses through a laser radar, and receiving signals reflected from the surface of the vehicle to construct the three-dimensional point cloud data of the vehicle.

[0031] This step generates a three-dimensional point cloud image of the vehicle by emitting high-frequency laser pulses and receiving signals reflected from the vehicle surface. When the vehicle passes through the toll station, the lidar collects data in a continuous high-frequency scanning manner to construct a complete three-dimensional view of the vehicle.

[0032] As a preferred embodiment of this invention, feature extraction is performed on the three-dimensional point cloud data to obtain vehicle size information including: S21: preprocessing the three-dimensional point cloud data; S22: Optimize the 3D target detection algorithm, and extract features from the three-dimensional point cloud data using the optimized 3D target detection algorithm to obtain vehicle size information.

[0033] On the basis of the above-mentioned embodiments, after obtaining the three-dimensional point cloud data, the data can be preprocessed, including noise filtering and data compression operations. Subsequently, the 3D target detection PointPillar algorithm can be used to effectively separate the vehicle from the background and extract the three-dimensional features of the vehicle. Through algorithm analysis, the system can accurately measure the length, width and height of the vehicle, extract the geometric size features of the vehicle, and provide a basis for subsequent vehicle classification and fee calculation. The PointPillars network can convert the point cloud into cylinders to form pseudo-image data, and then use the pseudo-image data to use the SSD network for feature extraction, and finally generate a prediction box, that is, the target three-dimensional frame. The most important feature of the PointPillars network is that it achieves a balance between detection accuracy and speed, including data preprocessing, neural network and post-processing. The neural network part includes feature extraction, 2D convolution and SSD target detection head.

[0034] In order to further improve the performance of its PointPillars network, multi-scale feature fusion technology can be introduced to use feature pyramid networks to enhance the detection capabilities of vehicles of different sizes; secondly, data enhancement methods can be used in the feature extraction stage to increase the diversity of training data and improve the generalization ability of the model; attention mechanisms can also be introduced, such as SE modules or CBAM, to further enhance the representation of important features in the 2D convolution stage and improve the accuracy and robustness of overall detection; in addition, post-processing strategies can be improved, such as optimizing the non-maximum suppression process, which helps to reduce the number of overlapping boxes and improve the quality of the final detection results; end-to-end joint training can also be considered to facilitate the network to achieve better synergy in feature learning. Through these improvements, the PointPillars network can achieve more efficient vehicle detection in complex traffic scenarios and improve the overall performance of the toll collection system.

[0035] Furthermore, if Figure 2 As shown, the two-dimensional image data of the vehicle entering the toll station is collected by the bayonet camera, and the type of the vehicle is identified according to the two-dimensional image data, and the vehicle type information obtained includes: S31: Collecting two-dimensional image data of vehicles entering the toll station through a bayonet camera; Specifically, the bayonet camera uses a high-resolution lens and a high-speed shutter to capture the image of the vehicle at the moment it enters the toll station, and the bayonet camera should be configured at the optimal angle to ensure that the license plate and the entire appearance of the vehicle can be clearly captured; the system can automatically start the camera when the vehicle enters the detection area through a vehicle sensor or lidar trigger mechanism to ensure the time synchronization of the acquisition of two-dimensional image data and lidar data; the image received by the bayonet camera can be pre-processed, including image enhancement and denoising steps, to improve recognition accuracy.

[0036] S32: pre-processing the two-dimensional image data, identifying the vehicle type of the vehicle using a deep learning algorithm, and outputting a vehicle bounding box of the vehicle; Specifically, in order to achieve real-time and accurate license plate recognition, a dynamic license plate target recognition algorithm based on Yolov7-LPRNet can be used, which combines advanced deep learning technologies for target detection and license plate recognition; the system captures video streams of passing vehicles in real time through high-frame rate, high-resolution cameras installed at toll booths or intersections to ensure clear vehicle images; in the target detection stage, the Yolov7 model is used to quickly identify vehicles in video frames. As an efficient target detection model, Yolov7 can achieve a high detection speed while maintaining accuracy; for each frame of the image, the system performs input preprocessing, including scaling and normalization, to meet the input requirements of Yolov7, and finally outputs the bounding box, category label and confidence of each vehicle, which is then compared and verified with the size data of the lidar.

[0037] S33: Based on the vehicle boundary box, the license plate number of the vehicle is extracted by a license plate recognition algorithm, and the vehicle type and the license plate number are integrated to obtain vehicle type information.

[0038] In this embodiment, the system can extract possible license plate areas from the detected vehicle bounding box. In the license plate recognition stage, the extracted license plate area is input into the LPRNet model. LPRNet is specially designed for license plate recognition and can process license plate images of different styles and complex backgrounds. At this time, the system performs image preprocessing on the license plate area to improve recognition accuracy, and extracts features through the convolutional neural network of LPRNet to finally identify the license plate number.

[0039] After the recognition is completed, the system integrates the vehicle information output by Yolov7 with the license plate information recognized by LPRNet to form the final recognition result, including the license plate number, vehicle type and detection confidence; these recognition results will be output to the user interface or related management system in real time and stored in the database for subsequent query and management. In order to maintain the efficiency of the system, its performance, including recognition accuracy and processing speed, can be monitored regularly, and the model can be updated and trained according to actual application needs to cope with the emergence of new models and license plate formats. Through this process, the dynamic license plate target recognition algorithm based on Yolov7-LPRNet can achieve efficient and accurate license plate recognition in complex traffic environments, providing strong support for traffic management and toll collection systems.

[0040] In order to optimize the license plate positioning, the image processing technology is also used to optimize the license plate area when extracting the license plate area in the vehicle boundary box.

[0041] In order to optimize license plate positioning, after the system extracts the possible license plate area from the detected vehicle bounding box, it can use image processing algorithms to process the extracted license plate area, such as edge detection and morphological processing.

[0042] In order to improve the data fusion effect, the acquisition frequencies of the lidar and the bayonet camera are aligned through clock synchronization technology.

[0043] Specifically, due to the different sampling frequencies of the bayonet camera and the lidar, there are differences in the data captured by the two sensors. Under normal circumstances, the sampling frequency of the bayonet camera is 30Hz, and the sampling frequency of the lidar is 10~20Hz. The camera completes the acquisition of 3 frames of images before the lidar can complete the acquisition of 1 frame of point cloud data. Therefore, the data of the bayonet camera and the lidar will not correspond at the same time. If there is no precise time synchronization, the system may mistakenly associate the data of different vehicles, resulting in recognition bias and classification errors, affecting the fusion detection results. Therefore, time synchronization can be performed. Through precise clock synchronization technology, the acquisition frequencies of the lidar and the bayonet camera can be aligned so that both can capture relevant information of the same vehicle at the same time.

[0044] Based on the above embodiments, a high-precision clock source is also used to achieve time synchronization between the laser radar and the bayonet camera.

[0045] As a preferred embodiment of this embodiment, a high-precision clock source, such as GPS timing or PTP (precision time protocol), can be used to achieve time alignment within milliseconds or even microseconds to ensure that the three-dimensional point cloud data of the laser radar matches the image data of the bayonet camera; this high-precision synchronization technology can effectively avoid the time misalignment problem of data in the multi-sensor fusion process, so that the system can reliably fuse and process multi-modal data, which not only improves the accuracy of vehicle model recognition, but also significantly improves the stability and reliability of the entire system, ensuring that vehicle information can be continuously and accurately captured and analyzed in high-speed and complex traffic environments. The present invention uses the time synchronization function of the ROS platform, takes the laser radar time as the reference, and selects the camera data closest to that moment to retain as the corresponding image frame.

[0046] Furthermore, comparing and matching the vehicle size information and the vehicle type information includes: The three-dimensional point cloud data collected by the lidar and the two-dimensional image data collected by the camera are mapped into the same coordinate system through the spatial alignment algorithm.

[0047] In this step, the output data of the LiDAR module and the bayonet camera module are spatially synchronized, and the information of the two is combined to achieve comprehensive judgment; in data fusion, the two-dimensional image data of the bayonet camera and the three-dimensional point cloud data of the LiDAR are spatially registered, and the results of the two are combined through the Kalman filter algorithm to output more accurate vehicle type information. The spatial synchronization process depends on accurate sensor calibration and coordinate conversion to ensure that the point cloud and image information can be seamlessly superimposed, making subsequent data processing more accurate. Spatial synchronization is the joint calibration process of the bayonet camera and the LiDAR. The projection of the point cloud data of the LiDAR coordinate system to the two-dimensional coordinate system of the bayonet camera requires the external parameters of the bayonet camera. The conversion of the bayonet camera coordinate system to the pixel coordinate system requires the internal parameters of the bayonet camera. For example, suppose a point in space In the laser radar coordinate system , the coordinates in the camera coordinate system are , in the image coordinate system , in the pixel coordinate system the coordinates are , then the relationship between the laser radar and the pixel coordinate system is expressed as: ; In the formula is the external parameter matrix of the mount camera; f is the focal length of the mount camera; , for , Pixel conversion unit of axis; , is the offset of the center of the projection screen relative to the optical axis; then the point in the three-dimensional coordinate system of the laser radar Convert to pixel coordinates The problem is converted into a process of obtaining the internal and external parameters of the bayonet camera. The bayonet camera parameters can be calibrated using Zhang Zhengyou's chessboard calibration method. For example, the calibration image is: 11×9 chessboard, with a grid width of 25mm.

[0048] After completing the spatiotemporal synchronization of the sensors, the laser radar point cloud detection results are fused with the bayonet camera detection results. The bayonet camera and laser radar detection frames are displayed on the same frame of the picture. Then, the projection bounding box detected by the laser radar and the bounding box detected by the bayonet camera are associated and matched through IOU. When the overlapping area is greater than or equal to the set threshold of 0.5, it will be considered a successful match, and the distance information of the fused laser radar and the category information of the bayonet camera sensor will be output; when the overlapping area is less than the set threshold of 0.5, only the target information detected by the bayonet camera will be output. In data fusion, when using a deep learning network to detect targets in image and point cloud data, the set confidence probability threshold will filter out some targets with low confidence, but in the actual experimental process, some of the identified targets will be filtered out due to the set confidence threshold, so the present invention uses a sigmoid function to correct the confidence of the detected targets.

[0049] Embodiment 2: like Figure 3 As shown in the figure, the toll vehicle detection system based on the fusion of laser radar and bayonet camera includes: The laser radar module scans the vehicles entering the toll station through the laser radar to obtain the three-dimensional point cloud data of the vehicle; it extracts features from the three-dimensional point cloud data to obtain the vehicle size information; the laser radar module is one of the core components of the system, responsible for capturing the three-dimensional point cloud data of vehicles passing through the toll station and extracting the geometric size features of the vehicle, including length, width and height.

[0050] The bayonet camera module collects two-dimensional image data of vehicles entering the toll station through the bayonet camera, identifies the type of vehicle based on the two-dimensional image data, and obtains vehicle type information; the bayonet camera module is responsible for collecting high-definition images of vehicles passing through the toll station and extracting key identification information.

[0051] The time synchronization module aligns the acquisition frequencies of the LiDAR and the bayonet camera through clock synchronization technology; the time synchronization module plays a vital role in the system and is responsible for ensuring the time consistency between the LiDAR and the bayonet camera during the data acquisition process.

[0052] The data fusion module compares and matches the vehicle size information and vehicle type information, and analyzes the vehicle size information and vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; the data fusion module is a key part of the system, responsible for spatially aligning the output data of the lidar module and the bayonet camera module, and combining the information of the two to achieve comprehensive judgment.

[0053] The toll calculation module calculates the tolls payable by the vehicle based on the vehicle's complete information and the vehicle model toll rules. The toll calculation module is one of the core functions of the system, responsible for automatically calculating the tolls payable by the vehicle at the toll booth based on the vehicle's license plate information, model and size.

[0054] Based on the above embodiment, it also includes integrating auditing and recording functions in the fee calculation module to record detailed information of each chargeable fee.

[0055] In order to ensure the transparency and accuracy of charges, the toll calculation module may also integrate auditing and recording functions to record the detailed information of each charge for subsequent query and review. This not only improves the efficiency of charging, but also reduces manual intervention and possible errors, ensuring the reliability and standardization of the charging process. Through this module, the system can complete the fee calculation the moment the vehicle passes through the toll station, achieving a seamless and efficient charging experience.

[0056] The above-mentioned adjustment system in the present invention can effectively implement the toll vehicle type detection method based on the fusion of laser radar and bayonet camera. The technical effects that can be achieved are as described in the above-mentioned embodiments and will not be repeated here.

[0057] Although the present application has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application as defined herein, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application is intended to include these modifications and variations.

Claims

1. A toll vehicle type detection method based on the fusion of laser radar and bayonet camera, characterized in that: The method comprises: Scan the vehicles entering the toll booths with laser radar to obtain the three-dimensional point cloud data of the vehicles; Extracting features from the three-dimensional point cloud data to obtain vehicle size information; Collecting two-dimensional image data of vehicles entering the toll station through a bayonet camera, identifying the type of the vehicle according to the two-dimensional image data, and obtaining vehicle type information; Comparing and matching the vehicle size information and the vehicle type information, and analyzing the vehicle size information and the vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; Based on the complete vehicle information, the chargeable fees for the vehicle are calculated according to the vehicle type charging rules.

2. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 1 is characterized in that: The obtaining of the three-dimensional point cloud data of the vehicle includes: continuously emitting high-frequency laser pulses through the laser radar, and receiving signals reflected from the surface of the vehicle to construct the three-dimensional point cloud data of the vehicle.

3. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 2 is characterized in that: The extracting features of the three-dimensional point cloud data to obtain the vehicle size information includes: Preprocessing the three-dimensional point cloud data; The 3D object detection algorithm is optimized, and features of the three-dimensional point cloud data are extracted by the optimized 3D object detection algorithm to obtain vehicle size information.

4. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 1 is characterized in that: The two-dimensional image data of the vehicle entering the toll station is collected by the bayonet camera, and the type of the vehicle is identified according to the two-dimensional image data to obtain the vehicle type information, including: Collecting two-dimensional image data of the vehicle entering the toll station through the bayonet camera; Preprocessing the two-dimensional image data, identifying the vehicle type of the vehicle using a deep learning algorithm, and outputting a vehicle bounding box of the vehicle; Based on the vehicle boundary frame, the license plate number of the vehicle is extracted by a license plate recognition algorithm, and the vehicle type and the license plate number are integrated to obtain vehicle type information.

5. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 4 is characterized in that: The method also includes optimizing the license plate area by using image processing technology when extracting the license plate area in the vehicle boundary frame.

6. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 1 is characterized in that: It includes aligning the acquisition frequencies of the laser radar and the bayonet camera through clock synchronization technology.

7. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 6 is characterized in that: It also includes using a high-precision clock source to achieve time synchronization between the laser radar and the bayonet camera.

8. The toll vehicle type detection method based on the fusion of laser radar and bayonet camera according to claim 1 is characterized in that: Comparing and matching the vehicle size information and the vehicle type information includes: The three-dimensional point cloud data collected by the laser radar and the two-dimensional image data collected by the bayonet camera are mapped into the same coordinate system through a spatial alignment algorithm.

9. The toll vehicle type detection system based on the fusion of laser radar and bayonet camera is characterized by: The system comprises: The laser radar module scans the vehicle entering the toll station through the laser radar to obtain the three-dimensional point cloud data of the vehicle; extracts features from the three-dimensional point cloud data to obtain the vehicle size information; A bayonet camera module collects two-dimensional image data of vehicles entering the toll station through a bayonet camera, identifies the type of the vehicle according to the two-dimensional image data, and obtains vehicle type information; A time synchronization module, which aligns the acquisition frequencies of the laser radar and the bayonet camera through clock synchronization technology; A data fusion module compares and matches the vehicle size information and the vehicle type information, and analyzes the vehicle size information and the vehicle type information through a multimodal fusion algorithm to obtain complete vehicle information; The charging calculation module calculates the chargeable fee of the vehicle based on the complete information of the vehicle and the charging rules of the vehicle model.

10. The toll vehicle type detection system based on the fusion of laser radar and bayonet camera according to claim 9 is characterized in that: It also includes integrating auditing and recording functions in the fee calculation module to record detailed information of each receivable fee.

Images