A method and system for highway toll auditing based on video and laser radar

Abstract

The present invention provides a method and system for auditing highway tolls based on video and laser radar, which relates to the field of data analysis technology, including: obtaining the license plate recognition results and vehicle model detection results of vehicles entering and exiting the toll station based on the analysis of target data obtained by preprocessing vehicle data collected by using a preset data collection tool; combining the license plate recognition results with the vehicle model detection results for analysis to complete the audit of entering and exiting the toll station. The target data is obtained by preprocessing the data of vehicles entering and exiting the toll station collected by using a preset data collection tool related to video and laser radar; the license plate recognition and vehicle model detection of the vehicle are realized by analyzing the target data; the audit of vehicles entering and exiting the toll station is completed based on the combined analysis of the license plate recognition results and the vehicle model detection results, which can accurately identify the behavior of large vehicles sticking ETC on-board identification of small passenger cars and reduce toll evasion, so as to improve the accuracy and efficiency of highway toll collection.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a method and system for highway toll auditing based on video and laser radar. Background Art

[0002] In recent years, with the continuous expansion of the national expressway network, the complexity and importance of toll management have become increasingly prominent. Toll management is directly related to the maintenance, operation and service quality of highways.

[0003] In the process of ETC toll collection on highways, the problem of "big car with small label" frequently occurs, that is, large vehicles are affixed with ETC vehicle labels for small passenger cars, and are charged and deducted according to the toll standards for small passenger cars when collecting tolls on highways, resulting in toll evasion or underpayment. However, under the traditional highway toll auditing method, data collection is often not accurate enough, and it is easy to miss toll evasion, which has a serious impact on the normal operation and revenue of highways. Therefore, how to effectively identify the "big car with small label" behavior and avoid toll evasion has become one of the current research focuses.

[0004] Therefore, the present invention provides a method and system for highway toll auditing based on video and lidar, which are used to effectively avoid the "big car with small label" problem and toll evasion behavior, and improve the accuracy and efficiency of highway toll collection. Summary of the invention

[0005] The present invention provides a method and system for auditing highway tolls based on video and laser radar, which are used to obtain target data by pre-processing the data of vehicles entering and leaving a toll station collected by using preset data collection tools related to video and laser radar; analyzing the target data to realize license plate recognition and vehicle model detection of the vehicle; completing the audit of vehicles entering and leaving the toll station based on the combined analysis of the license plate recognition result and the vehicle model detection result, and can accurately identify the behavior of large vehicles sticking ETC on-board identification of small passenger cars and reduce toll evasion behavior, so as to improve the accuracy and efficiency of highway toll collection.

[0006] The present invention provides a highway toll auditing method based on video and laser radar, comprising:

[0007] Step 1: Use a preset data acquisition tool to collect vehicle body data in real time, and perform data preprocessing to obtain target data, wherein the preset data acquisition tool is related to video and lidar;

[0008] Step 2: Based on the analysis of the target data, the license plate recognition results and vehicle type detection results of the vehicles entering and exiting the toll station are obtained;

[0009] Step 3: Based on the combined analysis of the license plate recognition results and the vehicle type detection results, complete the entry and exit toll station audit.

[0010] Preferably, the vehicle body data is collected in real time using a preset data collection tool, and the data is preprocessed to obtain target data, including:

[0011] For vehicles traveling to the entry-detection area and the exit-detection area, a first vehicle video is captured by a camera installed at a preset position;

[0012] Extracting key frames from the first vehicle video to obtain a plurality of vehicle frame images;

[0013] De-noising and contrast enhancement processing is performed on the vehicle frame image to obtain a target image and output it as target data;

[0014] After filtering, denoising and registering the vehicle point cloud data acquired by the laser radar installed at the preset position, the target point cloud data is obtained and output as the target data;

[0015] The data application direction is determined for the target data based on the detection area to which the target data belongs.

[0016] Preferably, extracting key frames from the first vehicle video to obtain a vehicle frame image includes:

[0017] Using a convolutional autoencoder to extract features from each video frame in the first vehicle video to obtain first video features;

[0018] Clustering the video frames using a preset clustering algorithm and based on the first video feature to obtain video clusters;

[0019] Performing definition analysis on the video frames in each video cluster to obtain a first definition;

[0020] Mark the video frames whose first definition is not less than the set definition threshold in each video cluster as the first frames, and collect all the first frames to establish a first frame list of the corresponding video cluster;

[0021] Calculating a frame content representative value of each first frame in the first frame list, and performing weighted averaging in combination with the first definition to obtain a frame evaluation coefficient;

[0022] The calculation formula of the frame content representative value is as follows:

[0023] In the formula, r i It is represented as the representative value of the frame content of the first frame of the i-th frame; s i It is represented as the density of important information contained in the first frame of i; a ij β is represented by the value of the jth first main feature of the i-th first frame, where j = 1, 2, 3, ..., n; n represents the total number of first main features;j It is represented by the impact evaluation weight of the jth first main feature; γ is represented by the calculated loss factor; C0 i It is represented as the feature density of the video cluster to which the first frame of the i-th video belongs; C0 q It is represented as the feature density mean of the remaining clusters except the video cluster to which the i-th first frame belongs; Cr i It is represented as the unit feature density after unit box selection of the cluster distribution under the video cluster to which the i-th first frame belongs;

[0024] If there is only a single first frame with a frame evaluation coefficient greater than the set evaluation threshold in the current first frame list, the current first frame is regarded as a vehicle frame image;

[0025] If there are multiple first frames with frame evaluation coefficients greater than the set evaluation threshold in the current first frame list, all first frames with frame evaluation coefficients greater than the set evaluation threshold are marked as analysis frames;

[0026] Obtain the frame distance between two adjacent frames of the analysis frame on the time axis, and compare it with the set distance threshold;

[0027] When the frame distance between two adjacent analysis frames is less than the set distance threshold, the analysis frame with the smaller frame evaluation coefficient among the two adjacent analysis frames is deleted, and the remaining analysis frame is regarded as the vehicle frame image;

[0028] Otherwise, all analyzed frames are regarded as vehicle frame images output.

[0029] Preferably, based on the analysis of the target data, obtaining the license plate recognition results and vehicle type detection results of vehicles entering and exiting the toll station includes:

[0030] Performing edge detection on the target image using a Canny operator, and then performing morphological operations to obtain a first processed image;

[0031] Based on the color features and shape features of the license plate, the license plate area is located on the first processed image to obtain a target area location result;

[0032] Performing character segmentation on the target area positioning result to obtain a character image;

[0033] Extracting features from the character image to obtain a first feature;

[0034] Inputting the first feature into a character recognition model pre-built based on a neural network, and outputting a character recognition result;

[0035] Summarizing the character recognition results to obtain a first recognition result;

[0036] If all the first recognition results obtained are consistent and meet the license plate number rules, any first recognition result is output as the license plate recognition result;

[0037] If there is inconsistency in the first recognition results obtained, the character recognition results with the highest confidence are selected and summarized to obtain the second recognition results;

[0038] Verifying the second recognition result according to the license plate number rule, and outputting the second recognition result after successful verification as the license plate recognition result;

[0039] When the data application direction of the target image is detection before entering a toll station, the current license plate recognition result is regarded as an entry-license plate recognition result output;

[0040] When the data application direction of the target image is detection before exiting a toll station, the current license plate recognition result is regarded as an exit-license plate recognition result output;

[0041] After averaging the data spacing of the target point cloud data by using an interpolation algorithm, a three-dimensional reconstruction of the vehicle contour is performed to obtain a three-dimensional contour image;

[0042] Graying the three-dimensional contour image, and then removing noise using morphological opening operation to obtain a two-dimensional grayscale image;

[0043] Extracting a vehicle model feature from the two-dimensional grayscale image to obtain a first vehicle model feature;

[0044] Inputting the first vehicle type feature into a pre-built vehicle type recognition model, and outputting a vehicle type detection result;

[0045] When the data application direction of the target point cloud data is detection before entering a toll station, the current vehicle type detection result is regarded as the entry-vehicle type detection result output;

[0046] When the data application direction of the target point cloud data is detection before exiting a toll station, the current vehicle type detection result is output as an exit-vehicle type detection result.

[0047] Preferably, based on the color features and shape features of the license plate, the license plate area is located on the first processed image to obtain a target area location result, including:

[0048] Based on the color features and shape features of the license plate, a preset image processing function is used to locate the license plate area of ​​the first processed image to obtain a first area positioning result;

[0049] If there is only a single first area positioning result currently, and the confidence level is greater than the set area threshold, the first area positioning result is output as the target area positioning result;

[0050] If there is only a single first area positioning result at present, and the confidence level is not greater than the set area threshold, the parameters of the preset image processing function are adjusted, and the adjusted preset image processing function is used to re-position the license plate area of ​​the first processed image to obtain a new area positioning result;

[0051] The new area positioning result with the highest confidence and greater than the set area threshold is output as the target area positioning result;

[0052] If there are currently multiple first area positioning results, then calculating the regional comprehensive similarity of the first area positioning results;

[0053] Output the first region positioning result with the highest comprehensive regional similarity as the target region positioning result;

[0054] If there is no first region positioning result currently, grayscale processing is performed on the first processed image to obtain a first grayscale image;

[0055] Acquire and analyze a grayscale histogram of the first grayscale image to obtain a first peak feature;

[0056] Determining a first segmentation threshold based on the first wave peak feature;

[0057] Based on the first segmentation threshold, binarizing the first grayscale image to obtain a binarized image;

[0058] Performing morphological processing on the binary image to obtain a second processed image;

[0059] Based on the shape feature of the license plate, using a preset image processing function to locate the license plate area on the second processed image to obtain a second area positioning result;

[0060] If there is only a single second area positioning result currently, the second area positioning result is output as the target area positioning result;

[0061] If there are currently multiple second area positioning results, then calculating the regional comprehensive similarity of the second area positioning results;

[0062] The second region positioning result with the highest comprehensive regional similarity is output as the target region positioning result.

[0063] Preferably, the calculation formula of the regional comprehensive similarity is as follows:

[0064] Where P s It is expressed as the regional comprehensive similarity of the s-th regional positioning result; z s Represents the confidence of the positioning result of the sth region; x shrepresents the parameter value of the hth similarity evaluation parameter of the sth region positioning result, where h = 1, 2, ..., m; m represents the total number of similarity evaluation parameters; b h,max It is represented as the maximum value set for the hth similarity evaluation parameter; b h,min It is represented as the set minimum value of the h-th similarity evaluation parameter; ε h It is expressed as the compensation coefficient of the impact of the h-th similarity evaluation parameter on the comprehensive similarity of the evaluation area; w s It is expressed as the location rationality coefficient of the s-th regional positioning result; δ is expressed as the contribution weight of location rationality to the comprehensive similarity of the evaluation area.

[0065] Preferably, based on the combined analysis of the license plate recognition results and the vehicle type detection results, the audit of entering and exiting the toll station is completed, including:

[0066] Based on the vehicle model information database in the system background, the vehicle model is matched with the corresponding entry-license plate recognition results of the vehicle before entering the toll station to determine the first registered vehicle model;

[0067] Compare the first registered vehicle type with the entry-vehicle type detection result, and if they are consistent, allow the current vehicle to pass smoothly in the ETC lane;

[0068] If there is a discrepancy, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual verification;

[0069] Obtain the payment record of the vehicle currently exiting the toll station according to the corresponding exit-license plate recognition result of the vehicle exiting the toll station;

[0070] The actual vehicle type in the payment record is extracted and compared with the exit-vehicle type detection result of the vehicle currently leaving the toll station. If they are consistent, the backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle type and mileage.

[0071] If there is any inconsistency, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual vehicle model verification to obtain the real vehicle model;

[0072] The backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle model and mileage;

[0073] The payable toll is compared with the paid toll in the payment record. When there is an underpayment, an audit report is generated and the driver of the vehicle currently leaving the toll station is notified to make the supplementary payment.

[0074] The present invention provides a highway toll audit system based on video and laser radar, comprising:

[0075] Data acquisition module: used to collect vehicle body data in real time using preset data acquisition tools, and perform data preprocessing to obtain target data, wherein the preset data acquisition tools are related to video and laser radar;

[0076] Vehicle identification and detection module: used to obtain license plate recognition results and vehicle model detection results of vehicles entering and leaving the toll station based on the analysis of the target data;

[0077] Audit module: used to complete the audit of entering and exiting toll stations based on the combined analysis of license plate recognition results and vehicle type detection results.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] By using preset data collection tools related to video and lidar, the data of vehicles entering and leaving the toll station are preprocessed to obtain target data; the target data is analyzed to realize vehicle license plate recognition and vehicle model detection; based on the combined analysis of license plate recognition results and vehicle model detection results, the audit of vehicles entering and leaving the toll station is completed. It can accurately identify the behavior of large vehicles affixing ETC on-board logos of small passenger cars and reduce toll evasion, so as to improve the accuracy and efficiency of highway toll collection.

[0080] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0081] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0083] Figure 1 It is a flow chart of a method for highway toll audit based on video and laser radar in an embodiment of the present invention;

[0084] Figure 2 The structure diagram of a highway toll audit system based on video and laser radar in an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0086] The embodiment of the present invention provides a highway toll audit method based on video and laser radar, such as Figure 1 As shown, including:

[0087] Step 1: Use a preset data acquisition tool to collect vehicle body data in real time, and perform data preprocessing to obtain target data, wherein the preset data acquisition tool is related to video and lidar;

[0088] Step 2: Based on the analysis of the target data, the license plate recognition results and vehicle type detection results of the vehicles entering and exiting the toll station are obtained;

[0089] Step 3: Based on the combined analysis of the license plate recognition results and the vehicle type detection results, complete the entry and exit toll station audit.

[0090] In this embodiment, the preset data acquisition tools refer to high-definition cameras and laser radars; vehicle body data refers to vehicle driving videos collected by high-definition cameras and vehicle point cloud data collected by laser radars; target data refers to target images and target point cloud data, wherein the target image refers to an image obtained by denoising and enhancing the contrast of a vehicle frame image obtained by extracting key frames from a vehicle driving video captured by a high-definition camera; target point cloud data refers to the image obtained after filtering, denoising and alignment of vehicle point cloud data acquired by the laser radar.

[0091] In this embodiment, the license plate recognition result includes two recognition results: an entry-license plate recognition result and an exit-license plate recognition result. The entry-license plate recognition result refers to the license plate recognition result obtained by performing license plate recognition on a target image whose data application direction is detection before entering a toll station, and the exit-license plate recognition result refers to the license plate recognition result obtained by performing license plate recognition on a target image whose data application direction is detection before exiting a toll station. The data application direction includes detection before entering a toll station and detection before exiting a toll station, and is determined according to the detection area to which the target data belongs. When the target data acquisition detection area is the entry-detection area, the data application direction is detection before entering a toll station, and when the target data acquisition detection area is the exit-detection area, the data application direction is detection before exiting a toll station.

[0092] In this embodiment, the vehicle type detection results include entry-vehicle type detection results and exit-vehicle type detection results. The entry-vehicle type detection results are the results of vehicle type identification using a vehicle type recognition model for target point cloud data whose data application direction is detection before entering a toll station; the exit-vehicle type detection results are the results of vehicle type identification using a vehicle type recognition model for target point cloud data whose data application direction is detection before exiting a toll station; vehicle types include large passenger buses, trucks, semi-trailer tractors, special-purpose vehicles, small passenger cars, and sedans, etc.; the vehicle type recognition model refers to a model constructed by training a neural network using a pre-acquired vehicle image set containing rich vehicle type samples as a training data set, which is used for vehicle type identification.

[0093] The beneficial effects of the above technical solution are: by using preset data collection tools related to video and lidar, the data of vehicles entering and leaving the toll station are pre-processed to obtain target data; the target data is analyzed to realize the license plate recognition and vehicle model detection of the vehicle; based on the combined analysis of the license plate recognition results and the vehicle model detection results, the audit of vehicles entering and leaving the toll station is completed, and the behavior of large vehicles affixing ETC on-board identification of small passenger cars can be accurately identified and the behavior of evading tolls can be reduced, so as to improve the accuracy and efficiency of highway toll collection.

[0094] The embodiment of the present invention provides a highway toll audit method based on video and laser radar, which uses a preset data acquisition tool to collect vehicle body data in real time, and performs data preprocessing to obtain target data, including:

[0095] For vehicles traveling to the entry-detection area and the exit-detection area, a first vehicle video is captured by a camera installed at a preset position;

[0096] Extracting key frames from the first vehicle video to obtain a plurality of vehicle frame images;

[0097] De-noising and contrast enhancement processing is performed on the vehicle frame image to obtain a target image and output it as target data;

[0098] After filtering, denoising and registering the vehicle point cloud data acquired by the laser radar installed at the preset position, the target point cloud data is obtained and output as the target data;

[0099] The data application direction is determined for the target data based on the detection area to which the target data belongs.

[0100] In this embodiment, the preset data acquisition tools refer to high-definition cameras and laser radars; the entry-detection area is a preset area before the entrance to the toll station; the exit-detection area is a preset area before the exit of the toll station; the preset position is pre-set, such as in front of the toll station entrance, above the toll station exit channel, at the ETC gantry, etc.; the first vehicle video refers to a vehicle driving video obtained by shooting a vehicle traveling to the entry-detection area and the exit-detection area using a high-definition camera installed at a preset position; the vehicle frame image refers to a frame image obtained by extracting key frames from the first vehicle video; the target image refers to the vehicle frame image after denoising and contrast enhancement processing.

[0101] In this embodiment, the laser radar detects the target's position, speed and other characteristic quantities by emitting a laser beam and receiving the reflected signal, and by combining scanning at multiple angles, three-dimensional point cloud data of the target object can be constructed; the vehicle point cloud data is generated by scanning the vehicles entering the entry-detection area and the exit-detection area using the laser radar; the target point cloud data is obtained by filtering, denoising and aligning the vehicle point cloud data; the data application directions include detection before entering the toll station and detection before exiting the toll station, which are determined according to the detection area to which the target data belongs; when the acquisition detection area of ​​the target data is the entry-detection area, the data application direction is detection before entering the toll station, and when the acquisition detection area of ​​the target data is the exit-detection area, the data application direction is detection before exiting the toll station.

[0102] The beneficial effect of the above technical solution is: by preprocessing the video data collected by high-definition cameras installed in different detection areas and the point cloud data collected by the lidar to obtain the target data, and determining the data application direction of the target data, the accuracy and efficiency of data processing can be effectively improved, laying a data support for subsequent license plate recognition and vehicle model detection analysis.

[0103] The embodiment of the present invention provides a highway toll audit method based on video and laser radar, which extracts key frames from the first vehicle video to obtain a vehicle frame image, including:

[0104] Using a convolutional autoencoder to extract features from each video frame in the first vehicle video to obtain first video features;

[0105] Clustering the video frames using a preset clustering algorithm and based on the first video feature to obtain video clusters;

[0106] Performing definition analysis on the video frames in each video cluster to obtain a first definition;

[0107] Mark the video frames whose first definition is not less than the set definition threshold in each video cluster as the first frames, and collect all the first frames to establish a first frame list of the corresponding video cluster;

[0108] Calculating a frame content representative value of each first frame in the first frame list, and performing weighted averaging in combination with the first definition to obtain a frame evaluation coefficient;

[0109] The calculation formula of the frame content representative value is as follows:

[0110] In the formula, r i It is represented as the representative value of the frame content of the first frame of the i-th frame; s i It is represented as the density of important information contained in the first frame of i; a ij β is represented by the value of the jth first main feature of the i-th first frame, where j = 1, 2, 3, ..., n; n represents the total number of first main features; j It is represented by the impact evaluation weight of the jth first main feature; γ is represented by the calculated loss factor; C0 i It is represented as the feature density of the video cluster to which the first frame of the i-th video belongs; C0 q It is represented as the feature density mean of the remaining clusters except the video cluster to which the i-th first frame belongs; Cr i It is represented as the unit feature density after unit box selection of the cluster distribution under the video cluster to which the i-th first frame belongs;

[0111] If there is only a single first frame with a frame evaluation coefficient greater than the set evaluation threshold in the current first frame list, the current first frame is regarded as a vehicle frame image;

[0112] If there are multiple first frames with frame evaluation coefficients greater than the set evaluation threshold in the current first frame list, all first frames with frame evaluation coefficients greater than the set evaluation threshold are marked as analysis frames;

[0113] Obtain the frame distance between two adjacent frames of the analysis frame on the time axis, and compare it with the set distance threshold;

[0114] When the frame distance between two adjacent analysis frames is less than the set distance threshold, the analysis frame with the smaller frame evaluation coefficient among the two adjacent analysis frames is deleted, and the remaining analysis frame is regarded as the vehicle frame image;

[0115] Otherwise, all analyzed frames are regarded as vehicle frame images output.

[0116] In this embodiment, the first video feature is obtained by extracting features from each video frame in the first vehicle video using a convolutional autoencoder, wherein the convolutional autoencoder is an autoencoder model based on a convolutional neural network, which uses a convolutional layer to replace the fully connected layer in a traditional autoencoder to better process image data; the preset clustering algorithm is pre-set, such as the K-means algorithm; and the video cluster is obtained by clustering the video frames based on the first feature using the preset clustering algorithm.

[0117] In this embodiment, the first clarity is a value obtained by calculating the resolution and peak signal-to-noise ratio of the video frame and performing weighted averaging, and is used to characterize the clarity of the video frame; the clarity threshold is set in advance; the first frame refers to a video frame whose first clarity is not less than the set clarity threshold; and the first frame list is a list established by aggregation of the first frames.

[0118] In this embodiment, for example, if the resolution of video frame 1 is e1 and the peak signal-to-noise ratio is e2, the first definition of video frame 1 is determined based on the weight g1 of the resolution and the weight g2 of the peak signal-to-noise ratio.

[0119] In this embodiment, the frame content representative value is used to express the content richness and representativeness of the analysis frame; the frame evaluation coefficient is used to characterize the availability of the first frame, which is obtained by weighted averaging the frame content representative value and the first clarity according to the assigned weights; the important information density is used to quantify the amount of important information contained in the first frame in a unit space, wherein the important information includes vehicle information, road information and environmental information, wherein the vehicle information includes vehicle type, vehicle color and license plate, etc., the road information includes road type, road signs and markings, etc., and the environmental information includes surrounding landscape, light conditions, etc.; the first main feature includes color features, texture features, shape features, etc.; setting the evaluation threshold is set in advance; setting the distance threshold is pre-set; the vehicle frame image refers to the first frame remaining after the first frame list corresponding to each video cluster is selected and deleted based on the frame evaluation coefficient.

[0120] In this embodiment, for example, the frame evaluation coefficients of the first frames w1, w2, and w3 in the first frame list 1 are f1, f2, and f3, respectively, where f1 and f2 are both greater than the set evaluation threshold, and f3 is equal to the set evaluation threshold, then the first frames w1 and w2 are both marked as analysis frames.

[0121] In this embodiment, for example, there are analysis frames d1 and d2 in the first frame list 2 that are adjacent and the frame distance is less than the set distance threshold; the frame evaluation coefficients of the analysis frames d1 and d2 are l 1 and l 2 respectively, and l 1>l 2, at this time, the analysis frame d2 is deleted from the first frame list 2.

[0122] In this embodiment, the value range of the first main feature is (0, 1).

[0123] The beneficial effect of the above technical solution is: by using a convolutional autoencoder to extract features from each video frame in the first vehicle video, clustering the video frames using a preset clustering algorithm, and then performing clarity analysis and frame evaluation on the video frames, and finally determining the vehicle frame image, the accuracy of key frame screening can be effectively improved, ensuring the high quality of the output vehicle frame image, and providing strong support for subsequent license plate recognition.

[0124] The embodiment of the present invention provides a method for auditing highway tolls based on video and laser radar, which obtains license plate recognition results and vehicle type detection results of vehicles entering and leaving a toll station based on analysis of the target data, including:

[0125] Performing edge detection on the target image using a Canny operator, and then performing morphological operations to obtain a first processed image;

[0126] Based on the color features and shape features of the license plate, the license plate area is located on the first processed image to obtain a target area location result;

[0127] Performing character segmentation on the target area positioning result to obtain a character image;

[0128] Extracting features from the character image to obtain a first feature;

[0129] Inputting the first feature into a character recognition model pre-built based on a neural network, and outputting a character recognition result;

[0130] Summarizing the character recognition results to obtain a first recognition result;

[0131] If all the first recognition results obtained are consistent and meet the license plate number rules, any first recognition result is output as the license plate recognition result;

[0132] If there is inconsistency in the first recognition results obtained, the character recognition results with the highest confidence are selected and summarized to obtain the second recognition results;

[0133] Verifying the second recognition result according to the license plate number rule, and outputting the second recognition result after successful verification as the license plate recognition result;

[0134] When the data application direction of the target image is detection before entering a toll station, the current license plate recognition result is regarded as an entry-license plate recognition result output;

[0135] When the data application direction of the target image is detection before exiting a toll station, the current license plate recognition result is regarded as an exit-license plate recognition result output;

[0136] After averaging the data spacing of the target point cloud data by using an interpolation algorithm, a three-dimensional reconstruction of the vehicle contour is performed to obtain a three-dimensional contour image;

[0137] Graying the three-dimensional contour image, and then removing noise using morphological opening operation to obtain a two-dimensional grayscale image;

[0138] Extracting a vehicle model feature from the two-dimensional grayscale image to obtain a first vehicle model feature;

[0139] Inputting the first vehicle type feature into a pre-built vehicle type recognition model, and outputting a vehicle type detection result;

[0140] When the data application direction of the target point cloud data is detection before entering a toll station, the current vehicle type detection result is regarded as the entry-vehicle type detection result output;

[0141] When the data application direction of the target point cloud data is detection before exiting a toll station, the current vehicle type detection result is output as an exit-vehicle type detection result.

[0142] In this embodiment, the target image refers to a vehicle frame image after denoising and contrast enhancement processing, wherein the vehicle frame image is an image obtained by extracting key frames from a first vehicle video captured by a high-definition camera installed at a preset position.

[0143] In this embodiment, the Canny operator is an edge detection algorithm used to implement edge detection of vehicle license plates; the first processed image is an image obtained by first performing edge detection on the target image using the Canny operator, and then processed using a morphological operation, wherein the morphological operation refers to a closing operation.

[0144] In this embodiment, the target area positioning result refers to the area obtained by screening and analyzing the first area positioning result obtained by positioning the license plate area of ​​the first processed image based on the color features and shape features of the license plate, in combination with the confidence level, wherein the confidence level is obtained by respectively calculating the similarity between the current first positioning area result and the area, aspect ratio and color consistency of the standard license plate, and combining the weighted summation with the corresponding assigned weights, and the formula is as follows:

[0145] Confidence = q1×area similarity+q2×aspect ratio similarity+q3×color consistency similarity;

[0146] Among them, q1 is represented by the weight of area similarity; q2 is represented by the weight of aspect ratio similarity; q3 is represented by the weight of color consistency similarity; area similarity is represented by the area percentage error between the first positioning area result and the standard license plate; aspect ratio similarity is represented by the aspect ratio percentage error between the first positioning area result and the standard license plate; color consistency similarity is represented by the color consistency percentage error between the first positioning area result and the standard license plate;

[0147] The first area positioning result is obtained by first using the findContours() function in OpenCV to detect the contours in the first processed image, then obtaining the license plate candidate area based on the license plate shape features, and finally applying a mask to the license plate candidate area and re-positioning it in combination with the color features of the vehicle, where color features refer to brightness, saturation, hue, and consistency, and shape features refer to aspect ratio, area, and edge contours.

[0148] In this embodiment, the character image is an image obtained by performing character segmentation on the target area positioning result using the projection method, wherein the projection method is a license plate character segmentation technology that realizes character segmentation based on the projection distribution of the license plate characters in the horizontal and vertical directions; the first feature includes contour features, pixel features, texture features, etc.; the character recognition model is a model constructed by training a neural network with a large number of vehicle character images with pre-labeled character types as training data, wherein the character types include Chinese characters, letters, numbers and special symbols; the character recognition result is the recognition result obtained by inputting the first feature obtained by extracting the character image features into the character recognition model.

[0149] In this embodiment, the first recognition result is obtained by arranging all the obtained character recognition results in sequence; the license plate number rule refers to the pre-set character type, length and arrangement order; the confidence refers to the confidence score attached to the character recognition result output by the character recognition model, which is used for the model's confidence level in the character recognition result; the second recognition result is obtained by selecting the character recognition result with the highest confidence and arranging them in sequence when the obtained first recognition results are inconsistent; the data application direction includes two directions: detection before entering the toll station and detection before exiting the toll station.

[0150] In this embodiment, the entry-license plate recognition result refers to the license plate recognition result obtained by performing license plate recognition on the target image whose data application direction is detection before entering the toll station; the exit-license plate recognition result refers to the license plate recognition result obtained by performing license plate recognition on the target image whose data application direction is detection before exiting the toll station, wherein the target image refers to the vehicle frame image after denoising and contrast enhancement processing, and the vehicle frame image is an image obtained by performing key frame extraction on the first vehicle video captured by a high-definition camera installed at a preset position; the data application directions include detection before entering the toll station and detection before exiting the toll station.

[0151] In this embodiment, the three-dimensional contour image is an image obtained by averaging the data spacing of the target point cloud data using an interpolation algorithm and then reconstructing the vehicle contour in three dimensions, wherein the target point cloud data refers to the data obtained after filtering, denoising and alignment processing of the vehicle point cloud data acquired by a laser radar installed at a preset position; the purpose of using the interpolation algorithm is to make the point distribution in the target point cloud data more uniform and reduce the impact of sparse or dense data areas on subsequent processing.

[0152] In this embodiment, the two-dimensional grayscale image is obtained by first graying the three-dimensional contour image and then denoising it using a morphological opening operation; the first vehicle model feature refers to the vehicle model feature extracted from the two-dimensional grayscale image based on a deep learning method, including size, shape, axle configuration, etc.

[0153] In this embodiment, the vehicle type recognition model is a model constructed by training a neural network using a pre-acquired vehicle image set containing rich vehicle type samples as a training data set, and is used for vehicle type recognition. The vehicle types include large passenger buses, trucks, semi-trailer tractors, special-purpose vehicles, sedans, etc.; the vehicle type detection results include entry-vehicle type detection results and exit-vehicle type detection results. Among them, the entry-vehicle type detection result is the result of vehicle type recognition using the vehicle type recognition model for target point cloud data whose data application direction is detection before entering a toll station; the exit-vehicle type detection result is the result of vehicle type recognition using the vehicle type recognition model for target point cloud data whose data application direction is detection before exiting a toll station.

[0154] The beneficial effects of the above technical solution are: by locating the license plate area based on the color features and shape features of the license plate, the target image after Canny operator edge detection and morphological processing is processed to obtain the target area positioning result; character segmentation and character recognition are performed on the target area positioning result to obtain the license plate recognition result, which can effectively ensure the accuracy of license plate positioning and character recognition, and improve the recognition accuracy of license plates.

[0155] The embodiment of the present invention provides a method for highway toll auditing based on video and laser radar, which locates the license plate area of ​​the first processed image based on the color characteristics and shape characteristics of the license plate to obtain a target area positioning result, including:

[0156] Based on the color features and shape features of the license plate, a preset image processing function is used to locate the license plate area of ​​the first processed image to obtain a first area positioning result;

[0157] If there is only a single first area positioning result currently, and the confidence level is greater than the set area threshold, the first area positioning result is output as the target area positioning result;

[0158] If there is only a single first area positioning result at present, and the confidence level is not greater than the set area threshold, the parameters of the preset image processing function are adjusted, and the adjusted preset image processing function is used to re-position the license plate area of ​​the first processed image to obtain a new area positioning result;

[0159] The new area positioning result with the highest confidence and greater than the set area threshold is output as the target area positioning result;

[0160] If there are currently multiple first area positioning results, then calculating the regional comprehensive similarity of the first area positioning results;

[0161] Output the first region positioning result with the highest comprehensive regional similarity as the target region positioning result;

[0162] If there is no first region positioning result currently, grayscale processing is performed on the first processed image to obtain a first grayscale image;

[0163] Acquire and analyze a grayscale histogram of the first grayscale image to obtain a first peak feature;

[0164] Determining a first segmentation threshold based on the first wave peak feature;

[0165] Based on the first segmentation threshold, binarizing the first grayscale image to obtain a binarized image;

[0166] Performing morphological processing on the binary image to obtain a second processed image;

[0167] Based on the shape feature of the license plate, using a preset image processing function to locate the license plate area on the second processed image to obtain a second area positioning result;

[0168] If there is only a single second area positioning result currently, the second area positioning result is output as the target area positioning result;

[0169] If there are currently multiple second area positioning results, then calculating the regional comprehensive similarity of the second area positioning results;

[0170] The second region positioning result with the highest comprehensive regional similarity is output as the target region positioning result.

[0171] In this embodiment, the color features refer to brightness, saturation, hue and consistency, and the shape features refer to aspect ratio, area, and edge contour; the preset image processing function is pre-set, generally referring to the findContours() function in OpenCV.

[0172] In this embodiment, the first area positioning result refers to first detecting the contour in the first processed image using the findContours() function in OpenCV, then obtaining the license plate candidate area based on the license plate shape feature, and finally applying a mask to the license plate candidate area, and relocating the area in combination with the color features of the vehicle; the set area threshold is pre-set; the confidence is obtained by respectively calculating the similarity between the current first positioning area result and the area, aspect ratio and color consistency of the standard license plate, and combining the corresponding assigned weights for weighted summation, and the formula is as follows:

[0173] Confidence = q1×area similarity+q2×aspect ratio similarity+q3×color consistency similarity;

[0174] Among them, q1 is represented by the weight of area similarity; q2 is represented by the weight of aspect ratio similarity; q3 is represented by the weight of color consistency similarity; area similarity is represented by the area percentage error between the first positioning area result and the standard license plate; aspect ratio similarity is represented by the aspect ratio percentage error between the first positioning area result and the standard license plate; color consistency similarity is represented by the color consistency percentage error between the first positioning area result and the standard license plate.

[0175] In this embodiment, the new area positioning result refers to the area obtained by re-positioning the license plate area of ​​the first processed image using a preset image processing function with adjusted parameters; the regional comprehensive similarity is used to quantify the degree of similarity between the first area positioning result and the area of ​​the vehicle standard license plate.

[0176] In this embodiment, for example, there are new area positioning results 1, 2, and 3 whose corresponding confidences are k1, k2, and k3, respectively, where k1 and k2 are both greater than the set area threshold, and k1>k2, and k3 is equal to the set area threshold, then the new area positioning result 1 is used as the target area positioning result.

[0177] In this embodiment, the first grayscale image is an image obtained after grayscale processing of the first processed image; the first peak feature is obtained by finding the peak from the histogram of the first grayscale image; the first segmentation threshold refers to the valley value between two adjacent and most significant peaks in the histogram; the binary image is an image obtained by setting all pixels in the first grayscale image whose grayscale values ​​are greater than the first segmentation threshold to white and the rest to black.

[0178] In this embodiment, the second processed image is an image obtained by performing morphological processing on the binary image, wherein the morphological processing refers to an opening operation; the second area positioning result is an area obtained by performing license plate area positioning on the second processed image based on the shape features of the license plate using a preset image processing function, wherein the shape features of the license plate include aspect ratio, area, and edge contour.

[0179] The beneficial effect of the above technical solution is: by analyzing and screening the regional positioning results obtained by performing license plate area positioning on the first processed image based on the color features and shape features of the license plate, the regional positioning result with the highest matching degree is obtained as the target regional positioning result output, which can effectively ensure the accuracy of license plate area positioning and improve the recognition accuracy of license plates.

[0180] The embodiment of the present invention provides a highway toll audit method based on video and laser radar, and the calculation formula of the regional comprehensive similarity is as follows:

[0181] Where P s It is expressed as the regional comprehensive similarity of the s-th regional positioning result; z s Represents the confidence of the positioning result of the sth region; x sh represents the parameter value of the hth similarity evaluation parameter of the sth region positioning result, where h = 1, 2, ..., m; m represents the total number of similarity evaluation parameters; b h,max It is represented as the maximum value set for the hth similarity evaluation parameter; b h,min It is represented as the set minimum value of the h-th similarity evaluation parameter; ε h It is expressed as the compensation coefficient of the impact of the h-th similarity evaluation parameter on the comprehensive similarity of the evaluation area; w s It is expressed as the location rationality coefficient of the s-th regional positioning result; δ is expressed as the contribution weight of location rationality to the comprehensive similarity of the evaluation area.

[0182] In this embodiment, the similarity evaluation parameters refer to color, aspect ratio, area and contour curvature; the set maximum value refers to the upper limit of the parameter value range of the similarity evaluation parameters of the local standard license plate; the set minimum value refers to the lower limit of the parameter value range of the similarity evaluation parameters of the local standard license plate; the reasonable position coefficient is used to express the reasonableness of the position of the area, which is obtained by calculating the horizontal and vertical deviations between the center point of the current area positioning result and the preset standard position, where the preset standard position generally refers to the center of the front of the vehicle.

[0183] The value range of the similarity evaluation parameter is (0, 1).

[0184] The beneficial effect of the above technical solution is: by calculating the comprehensive similarity of the regions and screening out the region positioning result with the highest matching degree as the target region positioning result, the accurate positioning of the license plate can be effectively guaranteed, which is beneficial to improving the recognition accuracy of the license plate.

[0185] The embodiment of the present invention provides a highway toll audit method based on video and laser radar, which completes the entry and exit toll station audit based on the combined analysis of license plate recognition results and vehicle type detection results, including:

[0186] Based on the vehicle model information database in the system background, the vehicle model is matched with the corresponding entry-license plate recognition results of the vehicle before entering the toll station to determine the first registered vehicle model;

[0187] Compare the first registered vehicle type with the entry-vehicle type detection result, and if they are consistent, allow the current vehicle to pass smoothly in the ETC lane;

[0188] If there is a discrepancy, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual verification;

[0189] Obtain the payment record of the vehicle currently exiting the toll station according to the corresponding exit-license plate recognition result of the vehicle exiting the toll station;

[0190] The actual vehicle type in the payment record is extracted and compared with the exit-vehicle type detection result of the vehicle currently leaving the toll station. If they are consistent, the backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle type and mileage.

[0191] If there is any inconsistency, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual vehicle model verification to obtain the real vehicle model;

[0192] The backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle model and mileage;

[0193] The payable toll is compared with the paid toll in the payment record. When there is an underpayment, an audit report is generated and the driver of the vehicle currently leaving the toll station is notified to make the supplementary payment.

[0194] In this embodiment, the vehicle model information database of the system background is composed of vehicle-related information, wherein the vehicle-related information includes vehicle type, vehicle size, license plate information and vehicle characteristics, such as color and structure; the first registered vehicle model refers to the vehicle model in the vehicle model information database of the system background that matches the vehicle model of the entry-license plate recognition result; the ETC lane refers to the electronic toll collection system lane; the roadside information display device refers to a device pre-installed on the side of the road for displaying warning signals and content to toll station personnel and passing vehicle drivers.

[0195] In this embodiment, the payment record consists of the actual vehicle model, license plate number and paid fees; the back-end toll settlement system is used for automatic fee calculation, account reconciliation management and generation of settlement reports; the real vehicle model refers to the vehicle model obtained by the toll station personnel through manual vehicle model verification of the current vehicle when the exit-vehicle model detection result of the vehicle leaving the toll station is inconsistent with the actual vehicle model in the corresponding payment record; the audit report consists of vehicle information, paid fees, payable tolls and audit time, among which vehicle information refers to the license plate number and vehicle model.

[0196] The beneficial effects of the above technical solution are: by matching the license plate and model analysis of vehicles before entering the toll station, it is possible to prevent "large vehicles with small license plates" from entering the toll section, and by auditing and making up the payment of vehicles before leaving the toll station, it is possible to avoid toll losses and effectively improve the accuracy and efficiency of highway toll collection.

[0197] The embodiment of the present invention provides a highway toll collection and verification system based on video and laser radar, such as Figure 2 As shown, including:

[0198] Data acquisition module: used to collect vehicle body data in real time using preset data acquisition tools, and perform data preprocessing to obtain target data, wherein the preset data acquisition tools are related to video and laser radar;

[0199] Vehicle identification and detection module: used to obtain license plate recognition results and vehicle model detection results of vehicles entering and leaving the toll station based on the analysis of the target data;

[0200] Audit module: used to complete the audit of entering and exiting toll stations based on the combined analysis of license plate recognition results and vehicle type detection results.

[0201] The beneficial effects of the above technical solution are: by using preset data collection tools related to video and lidar, the data of vehicles entering and leaving the toll station are pre-processed to obtain target data; the target data is analyzed to realize the license plate recognition and vehicle model detection of the vehicle; based on the combined analysis of the license plate recognition results and the vehicle model detection results, the audit of vehicles entering and leaving the toll station is completed, and the behavior of large vehicles affixing ETC on-board identification of small passenger cars can be accurately identified and the behavior of evading tolls can be reduced, so as to improve the accuracy and efficiency of highway toll collection.

[0202] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for highway toll auditing based on video and laser radar, characterized in that: include: Step 1: Use a preset data acquisition tool to collect vehicle body data in real time, and perform data preprocessing to obtain target data, wherein the preset data acquisition tool is related to video and lidar; Step 2: Based on the analysis of the target data, the license plate recognition results and vehicle type detection results of the vehicles entering and exiting the toll station are obtained; Step 3: Based on the combined analysis of the license plate recognition results and the vehicle type detection results, complete the entry and exit toll station audit; Among them, the vehicle body data is collected in real time using the preset data collection tool, and the data is preprocessed to obtain the target data, including: For vehicles traveling to the entry-detection area and the exit-detection area, a first vehicle video is captured by a camera installed at a preset position; Extracting key frames from the first vehicle video to obtain a plurality of vehicle frame images; De-noising and contrast enhancement processing is performed on the vehicle frame image to obtain a target image and output it as target data; After filtering, denoising and registering the vehicle point cloud data acquired by the laser radar installed at the preset position, the target point cloud data is obtained and output as the target data; Determining a data application direction for the target data based on the detection area to which the target data belongs; wherein, a convolutional autoencoder is used to extract features from each video frame in the first vehicle video to obtain first video features; Clustering the video frames using a preset clustering algorithm and based on the first video feature to obtain video clusters; Performing definition analysis on the video frames in each video cluster to obtain a first definition; Mark the video frames whose first definition is not less than the set definition threshold in each video cluster as the first frames, and collect all the first frames to establish a first frame list of the corresponding video cluster; Calculating a frame content representative value of each first frame in the first frame list, and performing weighted averaging in combination with the first definition to obtain a frame evaluation coefficient; The calculation formula of the frame content representative value is as follows: In the formula, r i It is represented as the representative value of the frame content of the first frame of the i-th frame; s i It is represented as the density of important information contained in the first frame of i; a ij β is represented by the value of the jth first main feature of the i-th first frame, where j = 1, 2, 3, ..., n; n represents the total number of first main features; j It is represented by the impact evaluation weight of the jth first main feature; γ is represented by the calculated loss factor; C0 i It is represented as the feature density of the video cluster to which the first frame of the i-th video belongs; C0 q It is represented as the feature density mean of the remaining clusters except the video cluster to which the i-th first frame belongs; Cr i It is represented as the unit feature density after unit box selection of the cluster distribution under the video cluster to which the i-th first frame belongs; If there is only a single first frame with a frame evaluation coefficient greater than the set evaluation threshold in the current first frame list, the current first frame is regarded as a vehicle frame image; If there are multiple first frames with frame evaluation coefficients greater than the set evaluation threshold in the current first frame list, all first frames with frame evaluation coefficients greater than the set evaluation threshold are marked as analysis frames; Obtain the frame distance between two adjacent frames of the analysis frame on the time axis, and compare it with the set distance threshold; When the frame distance between two adjacent analysis frames is less than the set distance threshold, the analysis frame with the smaller frame evaluation coefficient among the two adjacent analysis frames is deleted, and the remaining analysis frame is regarded as the vehicle frame image; Otherwise, all analyzed frames are regarded as vehicle frame images output.

2. A method for highway toll auditing based on video and laser radar according to claim 1, characterized in that: Based on the analysis of the target data, the license plate recognition results and vehicle type detection results of vehicles entering and leaving the toll station are obtained, including: Performing edge detection on the target image using a Canny operator, and then performing morphological operations to obtain a first processed image; Based on the color features and shape features of the license plate, the license plate area is located on the first processed image to obtain a target area location result; Performing character segmentation on the target area positioning result to obtain a character image; Extracting features from the character image to obtain a first feature; Inputting the first feature into a character recognition model pre-built based on a neural network, and outputting a character recognition result; Summarizing the character recognition results to obtain a first recognition result; If all the first recognition results obtained are consistent and meet the license plate number rules, any first recognition result is output as the license plate recognition result; If there is inconsistency in the first recognition results obtained, the character recognition results with the highest confidence are selected and summarized to obtain the second recognition results; Verifying the second recognition result according to the license plate number rule, and outputting the second recognition result after successful verification as the license plate recognition result; When the data application direction of the target image is detection before entering a toll station, the current license plate recognition result is regarded as an entry-license plate recognition result output; When the data application direction of the target image is detection before exiting a toll station, the current license plate recognition result is regarded as an exit-license plate recognition result output; After averaging the data spacing of the target point cloud data by using an interpolation algorithm, a three-dimensional reconstruction of the vehicle contour is performed to obtain a three-dimensional contour image; Graying the three-dimensional contour image, and then removing noise using morphological opening operation to obtain a two-dimensional grayscale image; Extracting a vehicle model feature from the two-dimensional grayscale image to obtain a first vehicle model feature; Inputting the first vehicle type feature into a pre-built vehicle type recognition model, and outputting a vehicle type detection result; When the data application direction of the target point cloud data is detection before entering a toll station, the current vehicle type detection result is regarded as the entry-vehicle type detection result output; When the data application direction of the target point cloud data is detection before exiting a toll station, the current vehicle type detection result is output as an exit-vehicle type detection result.

3. A method for highway toll auditing based on video and laser radar according to claim 2, characterized in that: Based on the color features and shape features of the license plate, the license plate area is located on the first processed image to obtain a target area location result, including: Based on the color features and shape features of the license plate, a preset image processing function is used to locate the license plate area of ​​the first processed image to obtain a first area positioning result; If there is only a single first area positioning result currently, and the confidence level is greater than the set area threshold, the first area positioning result is output as the target area positioning result; If there is only a single first area positioning result at present, and the confidence level is not greater than the set area threshold, the parameters of the preset image processing function are adjusted, and the adjusted preset image processing function is used to re-position the license plate area of ​​the first processed image to obtain a new area positioning result; The new area positioning result with the highest confidence and greater than the set area threshold is output as the target area positioning result; If there are currently multiple first area positioning results, then calculating the regional comprehensive similarity of the first area positioning results; Output the first region positioning result with the highest comprehensive regional similarity as the target region positioning result; If there is no first region positioning result currently, grayscale processing is performed on the first processed image to obtain a first grayscale image; Acquire and analyze a grayscale histogram of the first grayscale image to obtain a first peak feature; Determining a first segmentation threshold based on the first wave peak feature; Based on the first segmentation threshold, binarizing the first grayscale image to obtain a binarized image; Performing morphological processing on the binary image to obtain a second processed image; Based on the shape feature of the license plate, using a preset image processing function to locate the license plate area on the second processed image to obtain a second area positioning result; If there is only a single second area positioning result currently, the second area positioning result is output as the target area positioning result; If there are currently multiple second area positioning results, then calculating the regional comprehensive similarity of the second area positioning results; The second region positioning result with the highest comprehensive regional similarity is output as the target region positioning result.

4. A method for highway toll auditing based on video and laser radar according to claim 3, characterized in that: The calculation formula of the comprehensive similarity of the region is as follows: Where P s It is expressed as the regional comprehensive similarity of the s-th regional positioning result; z s Represents the confidence of the positioning result of the sth region; x sh represents the parameter value of the hth similarity evaluation parameter of the sth region positioning result, where h = 1, 2, ..., m; m represents the total number of similarity evaluation parameters; b h,max It is represented as the maximum value set for the hth similarity evaluation parameter; b h,min It is represented as the set minimum value of the h-th similarity evaluation parameter; ε h It is expressed as the compensation coefficient of the impact of the h-th similarity evaluation parameter on the comprehensive similarity of the evaluation area; w s It is expressed as the location rationality coefficient of the s-th regional positioning result; δ is expressed as the contribution weight of location rationality to the comprehensive similarity of the evaluation area.

5. The method for highway toll collection based on video and laser radar according to claim 1 is characterized in that: Based on the combined analysis of license plate recognition results and vehicle type detection results, the audit of entering and exiting toll stations is completed, including: Based on the vehicle model information database in the system background, the vehicle model is matched with the corresponding entry-license plate recognition results of the vehicle before entering the toll station to determine the first registered vehicle model; Compare the first registered vehicle type with the entry-vehicle type detection result, and if they are consistent, allow the current vehicle to pass smoothly in the ETC lane; If there is a discrepancy, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual verification; Obtain the payment record of the vehicle currently exiting the toll station according to the corresponding exit-license plate recognition result of the vehicle exiting the toll station; The actual vehicle type in the payment record is extracted and compared with the exit-vehicle type detection result of the vehicle currently leaving the toll station. If they are consistent, the backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle type and mileage. If there is any inconsistency, a warning signal will be issued through the roadside information display device to notify the toll station personnel to intercept and guide the current vehicle driver to the designated area for manual vehicle model verification to obtain the real vehicle model; The backend toll settlement system is used to automatically calculate the toll payable based on the actual vehicle model and mileage; The payable toll is compared with the paid toll in the payment record. When there is an underpayment, an audit report is generated and the driver of the vehicle currently leaving the toll station is notified to make the supplementary payment.

6. A highway toll audit system based on video and laser radar, characterized in that: include: Data acquisition module: used to collect vehicle body data in real time using preset data acquisition tools, and perform data preprocessing to obtain target data, wherein the preset data acquisition tools are related to video and laser radar; Vehicle identification and detection module: used to obtain license plate recognition results and vehicle model detection results of vehicles entering and leaving the toll station based on the analysis of the target data; Audit module: used to complete the audit of entering and exiting toll stations based on the combined analysis of license plate recognition results and vehicle model detection results; Among them, the data acquisition module is used to: For vehicles traveling to the entry-detection area and the exit-detection area, a first vehicle video is captured by a camera installed at a preset position; Extracting key frames from the first vehicle video to obtain a plurality of vehicle frame images; De-noising and contrast enhancement processing is performed on the vehicle frame image to obtain a target image and output it as target data; After filtering, denoising and registering the vehicle point cloud data acquired by the laser radar installed at the preset position, the target point cloud data is obtained and output as the target data; Determining a data application direction for the target data based on the detection area to which the target data belongs; wherein, a convolutional autoencoder is used to extract features from each video frame in the first vehicle video to obtain first video features; Clustering the video frames using a preset clustering algorithm and based on the first video feature to obtain video clusters; Performing definition analysis on the video frames in each video cluster to obtain a first definition; Mark the video frames whose first definition is not less than the set definition threshold in each video cluster as the first frames, and collect all the first frames to establish a first frame list of the corresponding video cluster; Calculating a frame content representative value of each first frame in the first frame list, and performing weighted averaging in combination with the first definition to obtain a frame evaluation coefficient; The calculation formula of the frame content representative value is as follows: In the formula, r i It is represented as the representative value of the frame content of the first frame of the i-th frame; s i It is represented as the density of important information contained in the first frame of i; a ij β is represented by the value of the jth first main feature of the i-th first frame, where j = 1, 2, 3, ..., n; n represents the total number of first main features; j It is represented by the impact evaluation weight of the jth first main feature; γ is represented by the calculated loss factor; C0 i It is represented as the feature density of the video cluster to which the first frame of the i-th video belongs; C0 q It is represented as the feature density mean of the remaining clusters except the video cluster to which the i-th first frame belongs; Cr i It is represented as the unit feature density after unit box selection of the cluster distribution under the video cluster to which the i-th first frame belongs; If there is only a single first frame with a frame evaluation coefficient greater than the set evaluation threshold in the current first frame list, the current first frame is regarded as a vehicle frame image; If there are multiple first frames with frame evaluation coefficients greater than the set evaluation threshold in the current first frame list, all first frames with frame evaluation coefficients greater than the set evaluation threshold are marked as analysis frames; Obtain the frame distance between two adjacent frames of the analysis frame on the time axis, and compare it with the set distance threshold; When the frame distance between two adjacent analysis frames is less than the set distance threshold, the analysis frame with the smaller frame evaluation coefficient among the two adjacent analysis frames is deleted, and the remaining analysis frame is regarded as the vehicle frame image; Otherwise, all analyzed frames are regarded as vehicle frame images output.

Images