Truck charging auditing method and electronic equipment

Abstract

The invention provides a truck charging auditing method and electronic equipment, and relates to the technical field of intelligent traffic management. The method comprises the following steps: acquiring a plurality of vehicle image frames, carrying out object identification, and cutting to obtain a region-of-interest image containing a target truck; axle detection is carried out based on the region-of-interest image, and an initial axle detection result of the target truck is determined; removing axle frames belonging to non-truck axles from the initial axle detection result, and updating the initial axle number to obtain the target axle number of the target truck; and according to the target axle number and the registration information of the target truck, charging auditing is carried out on the target truck. The highway truck charging auditing work is realized under the condition that the installation cost is not increased, the abnormal axle frame is removed from the initial axle detection result, it is ensured that only the axle really belonging to the target truck is counted, the charging error caused by the non-truck axle is avoided, and the charging efficiency is improved. Therefore, the accuracy and effectiveness of truck charging auditing are ensured.

Description

Technical Field

[0001] This application relates to the technical field of intelligent transportation management. Specifically, it relates to a truck toll auditing method and an electronic device. Background Art

[0002] With the popularization of the Electronic Toll Collection (ETC) system on highways, the accuracy of vehicle type identification has become a key factor in ensuring toll accuracy. Especially for trucks, since there are significant differences between different toll types for trucks, if the toll type of a truck is set incorrectly, it will lead to a large discrepancy in truck tolls.

[0003] Currently, the toll standard for trucks is mainly determined by the number of their axles. Existing vehicle axle identification technologies mainly include pressure contact type axle identification and contour detection technology in static or low-speed states. Among them, the pressure contact type axle identification method relies on the pressure change generated when the vehicle passes through a special road surface induction belt to identify the number of axles. This method requires damaging the existing road surface structure and cannot effectively solve the problem of floating axles of trucks, which is prone to identification errors between consecutive vehicles. The contour detection technology is based on the overall shape of the vehicle for intelligent analysis to determine the number of axles, usually requiring additional cameras or other hardware devices to be installed beside the highway.

[0004] Although the above vehicle axle identification technologies can all provide a certain identification ability, there are still obvious limitations in the actual application scenario of highways, which in turn causes highway toll auditing personnel to be unable to accurately and effectively conduct truck toll auditing. Summary of the Invention

[0005] The purpose of this application is to provide a truck toll auditing method and an electronic device for the deficiencies in the above existing technologies, so as to solve the problem that the vehicle axle identification technology in the existing technology still has obvious limitations in the actual application scenario of highways, which in turn causes highway toll auditing personnel to be unable to accurately and effectively conduct truck toll auditing.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of this application provides a truck toll auditing method, and the method includes:

[0008] Obtain a plurality of vehicle image frames, where the vehicle image frames are obtained by frame processing of the vehicle video collected by an image acquisition device installed on a highway gantry;

[0009] Perform object recognition on the multiple vehicle image frames, and crop an image of the region of interest containing the target truck from the multiple vehicle image frames;

[0010] Perform axle detection based on the image of the region of interest to determine the initial axle detection result of the target truck, where the initial axle detection result includes: axle bounding boxes of at least one axle in the image of the region of interest and the initial number of axles;

[0011] Remove the axle bounding boxes belonging to non-truck axles from the initial axle detection result, and update the initial number of axles to obtain the target number of axles of the target truck;

[0012] Perform toll auditing on the target truck according to the target number of axles and the registration information of the target truck.

[0013] As a possible implementation, the performing object recognition on the multiple vehicle image frames and cropping an image of the region of interest containing the target truck from the multiple vehicle image frames includes:

[0014] For each of the vehicle image frames, input the vehicle image frame into a pre-trained truck detection model, and have the truck detection model perform vehicle recognition on the vehicle image frame to determine the position information of the target truck in the vehicle image frame;

[0015] Generate a bounding box in the vehicle image frame based on the position information, and crop the vehicle image frame according to the bounding box to obtain the image of the region of interest.

[0016] As a possible implementation, the performing axle detection based on the image of the region of interest to determine the initial axle detection result of the target truck includes:

[0017] Input the image of the region of interest into a pre-trained truck axle detection model, and have the truck axle detection model perform axle recognition on the image of the region of interest to determine the axle bounding boxes of at least one axle in the image of the region of interest, and use the position information of each axle and the initial number of axles as the initial axle detection result.

[0018] As a possible implementation, the removing the axle bounding boxes belonging to non-truck axles from the initial axle detection result and updating the initial number of axles to obtain the target number of axles of the target truck includes:

[0019] For each axle bounding box corresponding to the axle in the image of the region of interest, determine the center point and the center point coordinates of each axle bounding box, and combine the center points and the center point coordinates of each vehicle target box to obtain a set of center points;

[0020] Perform fitting processing based on the set of center points to obtain multiple candidate fitting lines, and determine the number of inliers in each candidate fitting line, where each inlier is respectively used to indicate the center point of an axle box;

[0021] Determine the target fitting line according to the number of inliers in each candidate fitting line;

[0022] Determine the target axle number of the target freight car according to the number of inliers in the target fitting line.

[0023] As a possible implementation manner, the performing fitting processing based on the set of center points to obtain multiple candidate fitting lines, and determining the number of inliers in each candidate fitting line includes:

[0024] A. Randomly extract two center points from the set of center points as the first data set;

[0025] B. Construct an initial line according to the center point coordinates of the two center points in the first data set;

[0026] C. For each target center point in the set of center points except the two center points in the first data set, determine the Euclidean distance between the target center point and the initial line, and determine the point attribute of the target center point according to the Euclidean distance, and determine the consistency set corresponding to the first data set according to the point attributes of each target center point, where the point attribute is used to indicate that the target center point is an inlier or an outlier, and the consistency set is a data set composed of inliers;

[0027] D. Repeat the above steps A - C to obtain multiple consistency sets;

[0028] E. Perform fitting processing on each of the consistency sets to obtain multiple of the candidate fitting lines, and use the number of points in the consistency set as the number of inliers in the candidate fitting line.

[0029] As a possible implementation manner, the determining the target fitting line according to the number of inliers in each candidate fitting line includes:

[0030] Sort all candidate fitting lines according to the number of inliers in each candidate fitting line to obtain a line sequence;

[0031] Obtain the first M candidate fitting lines in the line sequence, where M is an integer greater than 0;

[0032] For each candidate fitting line in the first M candidate fitting lines, respectively determine the actual slope of the candidate fitting line, and determine the difference value between the actual slope and the preset slope;

[0033] Determine the target fitting line from the first M candidate fitting lines according to the difference value corresponding to each candidate fitting line among the first M candidate fitting lines.

[0034] As a possible implementation, the determining the target fitting line according to the number of inliers in each candidate fitting line includes:

[0035] Respectively determine the actual slope of each candidate fitting line, and determine the difference range corresponding to each candidate fitting line according to each actual slope and a preset slope;

[0036] Determine the target fitting line according to the number of inliers within the difference range corresponding to each candidate fitting line.

[0037] As a possible implementation, the performing toll audit on the target truck according to the target axle number and the registration information of the target truck includes:

[0038] Compare the target axle number with the axle number in the registration information. If the axle numbers are the same, determine the payment information of the target truck according to the vehicle type of the target truck and the truck toll standard.

[0039] As a possible implementation, the method further includes:

[0040] Obtain a vehicle driving video, and perform frame splitting on the vehicle driving video to obtain a plurality of video frames;

[0041] Perform tagging processing on the truck objects in each of the video frames to obtain truck tags corresponding to each of the video frames, and crop each of the video frames according to the truck tags corresponding to each of the video frames to obtain a plurality of truck images, wherein each truck image carries a corresponding truck tag;

[0042] Perform tagging processing on the axle objects in each of the truck images to obtain axle tags corresponding to each of the truck images;

[0043] Train a target detection model according to each truck image carrying the truck tag to obtain a truck detection model, and train a target detection model according to each truck image carrying the axle tag to obtain a truck axle detection model.

[0044] In a second aspect, an embodiment of the present application provides a truck toll audit device, and the device includes:

[0045] An acquisition module, configured to acquire a plurality of vehicle image frames, where the vehicle image frames are obtained by performing frame division on a vehicle video collected by an image acquisition device installed on a high-speed gantry;

[0046] An identification module, configured to perform object identification on the plurality of vehicle image frames, and crop an image of an area of interest containing a target truck from the plurality of vehicle image frames;

[0047] An axle detection module, configured to perform axle detection based on the image of the area of interest, and determine an initial axle detection result of the target truck, where the initial axle detection result includes: axle frames of at least one axle in the image of the area of interest and an initial number of axles;

[0048] An update module, configured to remove axle frames belonging to non-truck axles from the initial axle detection result, and update the initial number of axles to obtain the target number of axles of the target truck;

[0049] A toll auditing module, configured to perform toll auditing on the target truck according to the target number of axles and the registration information of the target truck.

[0050] As a possible implementation manner, the identification module is specifically configured to:

[0051] For each of the vehicle image frames, input the vehicle image frame into a pre-trained truck detection model, and the truck detection model performs vehicle identification on the vehicle image frame to determine the position information of the target truck in the vehicle image frame;

[0052] Generate a bounding box in the vehicle image frame based on the position information, and crop the vehicle image frame according to the bounding box to obtain the image of the area of interest.

[0053] As a possible implementation manner, the axle detection module is specifically configured to:

[0054] Input the image of the area of interest into a pre-trained truck axle detection model, and the truck axle detection model performs axle identification on the image of the area of interest to determine the axle frames of at least one axle in the image of the area of interest, and use the position information of each axle and the initial number of axles as the initial axle detection result.

[0055] As a possible implementation manner, the update module is specifically configured to:

[0056] For the axle frame corresponding to each axle in the image of the area of interest, determine the center point and the center point coordinates of each axle frame, and combine the center points and the center point coordinates of each vehicle target frame to obtain a set of center points;

[0057] Perform fitting processing based on the set of center points to obtain multiple candidate fitting lines, and determine the number of inliers in each candidate fitting line, where each inlier is used to indicate the center point of an axle box;

[0058] Determine the target fitting line according to the number of inliers in each candidate fitting line;

[0059] Determine the target axle number of the target freight car according to the number of inliers in the target fitting line.

[0060] As a possible implementation manner, the updating module is specifically configured to:

[0061] A. Randomly extract two center points from the set of center points as the first data set;

[0062] B. Construct an initial line according to the center point coordinates of the two center points in the first data set;

[0063] C. For each target center point in the set of center points except the two center points in the first data set, determine the Euclidean distance between the target center point and the initial line, and determine the point attribute of the target center point according to the Euclidean distance, and determine the consistency set corresponding to the first data set according to the point attributes of each target center point, where the point attribute is used to indicate that the target center point is an inlier or an outlier, and the consistency set is a data set composed of inliers;

[0064] D. Repeat the above steps A - C to obtain multiple consistency sets;

[0065] E. Perform fitting processing on each of the consistency sets to obtain multiple candidate fitting lines, and use the number of points in the consistency set as the number of inliers in the candidate fitting line.

[0066] As a possible implementation manner, the updating module is specifically configured to:

[0067] Sort all candidate fitting lines according to the number of inliers in each candidate fitting line to obtain a line sequence;

[0068] Obtain the first M candidate fitting lines in the line sequence, where M is an integer greater than 0;

[0069] For each candidate fitting line in the first M candidate fitting lines, respectively determine the actual slope of the candidate fitting line, and determine the difference value between the actual slope and the preset slope;

[0070] Determine the target fitting line from the first M candidate fitting lines according to the difference value corresponding to each candidate fitting line among the first M candidate fitting lines.

[0071] As a possible implementation manner, the updating module is specifically configured to:

[0072] Determine the actual slope of each candidate fitting line respectively, and determine the difference range corresponding to each candidate fitting line according to each actual slope and the preset slope;

[0073] Determine the target fitting line according to the number of inliers within the difference range corresponding to each candidate fitting line.

[0074] As a possible implementation manner, the toll audit module is specifically configured to:

[0075] Compare the target number of axles with the number of axles in the registration information. If the number of axles is the same, determine the toll payment information of the target truck according to the vehicle type of the target truck and the truck toll standard.

[0076] As a possible implementation manner, the truck toll audit device further includes a training module, and the training module is specifically configured to:

[0077] Obtain a vehicle driving video, and perform frame splitting on the vehicle driving video to obtain a plurality of video frames;

[0078] Perform labeling processing on the truck objects in each video frame to obtain truck labels corresponding to each video frame, and crop each video frame according to the truck labels corresponding to each video frame to obtain a plurality of truck images, wherein each truck image carries a corresponding truck label;

[0079] Perform labeling processing on the axle objects in each truck image to obtain axle labels corresponding to each truck image;

[0080] Perform model training on the target detection model according to each truck image carrying the truck label to obtain a truck detection model, and perform model training on the target detection model according to each truck image carrying the axle label to obtain a truck axle detection model.

[0081] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the truck toll audit method according to any one of the first aspects described above.

[0082] Fourthly, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the truck toll auditing method as described in any one of the above first aspects.

[0083] According to the truck toll auditing method and the electronic device of the embodiments of the present application, multiple vehicle image frames are obtained, object recognition is performed on the multiple vehicle image frames, a region of interest image containing the target truck is cropped from the multiple vehicle image frames, axle detection is performed based on the region of interest image to determine the initial axle detection result of the target truck, axle frames belonging to non-truck axles are excluded from the initial axle detection result, and the initial axle quantity is updated to obtain the target axle quantity of the target truck. According to the target axle quantity and the registration information of the target truck, toll auditing is performed on the target truck. According to the embodiments of the present application, a camera installed on a highway gantry is used to automatically capture videos of passing vehicles and perform frame division processing to generate high-quality vehicle image frames, thereby reducing the cost and complexity of hardware deployment and improving the reliability and efficiency of data collection. On this basis, by performing object recognition on the vehicle image frames, the position of the target truck is accurately located, a region of interest image containing the target truck is cropped, and the region of interest image is used as the input for axle detection, ensuring that the axle position can be accurately identified even in a complex background environment, providing a reliable initial axle detection result, and excluding abnormal axle frames by excluding axle frames belonging to non-truck axles from the initial axle detection result, ensuring that only the axles truly belonging to the target truck are counted, guaranteeing the accuracy of axle quantity statistics, and avoiding billing errors caused by non-truck axles. Therefore, according to the embodiments of the present application, not only the limitations of the existing vehicle axle recognition technology in highway toll auditing are effectively solved, but also the full process automation from image acquisition to axle detection to toll auditing is realized, greatly improving the accuracy and reliability of the entire toll auditing process. This operation can not only reduce the operation cost, but also improve the service quality and user experience, bringing significant benefits to highway management. Description of the Drawings

[0084] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0085] Figure 1 The flowchart showing a truck toll auditing method provided by an embodiment of the present application is shown;

[0086] Figure 2Shows a schematic diagram of an axle box of a non-truck axle provided by an embodiment of the present application;

[0087] Figure 3 Shows a schematic flow chart of a model training method provided by an embodiment of the present application;

[0088] Figure 4 Shows a schematic flow chart of a method for determining an image of an area of interest provided by an embodiment of the present application;

[0089] Figure 5 Shows a schematic diagram of an image of an area of interest provided by an embodiment of the present application;

[0090] Figure 6 Shows a schematic flow chart of a method for determining the number of target axles provided by an embodiment of the present application;

[0091] Figure 7 Shows a schematic flow chart of a fitting processing method provided by an embodiment of the present application;

[0092] Figure 8 Shows a schematic diagram of inliers and outliers provided by an embodiment of the present application;

[0093] Figure 9 Shows a schematic flow chart of a method for determining a target fitting line provided by an embodiment of the present application;

[0094] Figure 10 Shows a schematic diagram of a fitting line provided by an embodiment of the present application;

[0095] Figure 11 Shows a schematic diagram of the structure of a truck toll auditing device provided by an embodiment of the present application;

[0096] Figure 12 Shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0097] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application only serve the purpose of illustration and description, and are not used to limit the protection scope of the present application. Additionally, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0098] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and illustrated in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application claimed, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0099] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude adding other features.

[0100] In view of the problems existing in the prior art, the embodiments of the present application provide a truck toll auditing method, which can realize the toll auditing work of highway trucks without increasing the installation cost. Specifically, the axle number of the target truck is detected by a target detection model for the truck video collected by the image acquisition device on the highway gantry, and the non-truck axles are removed, and finally the axle number detection of the target truck is realized, so as to realize the auditing work of truck tolls, which can greatly reduce the labor auditing cost.

[0101] Figure 1 The flowchart of a truck toll auditing method provided by an embodiment of the present application is shown. Refer to Figure 1 As shown, the method specifically includes the following steps:

[0102] S101. Obtain a plurality of vehicle image frames.

[0103] Optionally, the vehicle image frames are obtained by frame processing the vehicle video collected by the image acquisition device installed on the highway gantry. Among them, the image acquisition device is, for example, a camera installed on the gantry. Continuous image frames are extracted from the vehicle video captured by the camera, and specifically, individual image frames can be obtained by frame processing the vehicle video.

[0104] S102. Perform object recognition on the plurality of vehicle image frames, and crop the region-of-interest image containing the target truck from the plurality of vehicle image frames.

[0105] Optionally, the region of interest (ROI) image is cropped from the vehicle image frame, that is, the key area containing the target truck is located and extracted. This key area is also the region of interest (ROI). By cropping the entire vehicle image frame and only retaining the ROI as the processing object for subsequent axle detection, the computing resources and time required for subsequent processing can be significantly reduced. Moreover, the unnecessary background information is removed from the ROI, making feature extraction and object detection more focused and accurate.

[0106] Optionally, using an object recognition algorithm, such as using the target detection network YOLOv9 to detect all vehicles in each vehicle image frame, and outputting the position information of each detected vehicle in the vehicle image frame. This position information is usually a vehicle bounding box indicating the position of the vehicle in the image. Then, based on the detected vehicle bounding box, the partial image containing the target truck is cropped. This cropped image is the region of interest image. Refer to Figure 2 As shown, only the part of the target truck is included in this region of interest image, reducing background interference and other impacts.

[0107] S103. Perform axle detection based on the region of interest image to determine the initial axle detection result of the target truck.

[0108] Optionally, since only the part of the target truck is included in the region of interest image, reducing background interference and other impacts, axle detection can be performed more accurately. On this basis, use the target detection network YOLOv9 to perform axle detection on the region of interest image, identify and mark the axle positions of the target truck, and output at least one axle box and a preliminary estimate of the number of axles to obtain the initial axle detection result. This initial axle detection result includes at least one axle box of the axles in the region of interest image and the initial number of axles.

[0109] Optionally, the feature extraction network in the target detection network YOLOv9 is mainly composed of convolutional layers and pooling layers. The convolutional layers use different convolutional kernels to perform convolutional operations on the input data to extract local features, and the pooling layers are used to reduce the spatial size of the feature map while retaining the main features. Through the stacking of multiple convolutional layers and pooling layers, more abstract and high-level features are gradually extracted. Therefore, the complete vehicle image frame is fed into the target detection network in a certain size, and the features obtained through the feature extraction network Figure 1Generally, it is 1 / 32 of the input image. Since the proportion of the truck captured on the high-speed gantry in the entire vehicle image frame is not large, and the proportion of the truck tires is even smaller. If the entire vehicle image frame is directly fed into the feature extraction network of the object detection network and undergoes 5 times of downsampling, the receptive field of the tires on the feature map is almost negligible. In response to this, the present application proposes a second-order truck axle detection. Specifically, first, the complete vehicle image frame is input into the object detection network to detect the position information of the target truck in the vehicle image frame. Then, according to the position information, the truck area, that is, the ROI area, is cropped out. Then, it enters the axle detection stage. The cropped area map is regarded as the entire image. At this time, the proportion of the truck axles is relatively large, and a large amount of feature information can still be retained after 1 / 32 times of downsampling by the object detection network. Therefore, the detection accuracy of the truck axles can be improved.

[0110] S104. Remove the axle frames belonging to non-truck axles from the initial axle detection results, and update the initial number of axles to obtain the target number of axles of the target truck.

[0111] Optionally, since the axles detected based on the ROI area may not necessarily be the axles of the target truck, refer to Figure 2 As shown, for example, the target truck is towing multiple cars, or the emergency rescue trailer's carriage is towing a broken-down car, or other vehicles are too close to the target truck. When cropping the ROI area, other vehicles close to the target truck will also be cropped. In view of the above situation, the axles of other vehicles may be recognized during axle detection, resulting in inaccurate initial axle detection results, and non-truck axles need to be removed from them.

[0112] Exemplarily, refer to Figure 2 As shown, the initial axle detection results include the axle frames of the axles of the target truck in the ROI area and the axle frames of the axles of multiple cars towed by the target truck in the ROI area (such as Figure 2 the green frames shown). Obviously Figure 2 the green frames marked in

[0113] S105. Conduct toll auditing on the target truck according to the target number of axles and the registration information of the target truck.

[0114] Optionally, obtain the relevant registration information of the target truck, such as license plate number, vehicle type, registration information, etc., and perform the correct charging operation based on the actually detected number of target axles and the vehicle registration information. Specifically, compare the actually detected number of target axles with the number of axles recorded during vehicle registration. If they are the same, calculate the fee that the target truck should pay according to the pre-set charging standard, perform the charging audit process to ensure the accuracy of the charging amount, and record the relevant information for subsequent auditing or querying.

[0115] Based on this, according to the truck charging audit method provided by the embodiments of the present application, the camera installed on the highway gantry automatically captures the videos of passing vehicles, performs frame-by-frame processing, and generates high-quality vehicle image frames, thereby reducing the cost and complexity of hardware deployment, while improving the reliability and efficiency of data collection. On this basis, by performing object recognition on the vehicle image frames, accurately locate the position of the target truck, crop out the region-of-interest image containing the target truck, and use the region-of-interest image as the input for axle detection, ensuring that even in a complex background environment, the axle position can be accurately identified, providing a reliable initial axle detection result, and by removing the axle frames belonging to non-truck axles from the initial axle detection result to exclude abnormal axle frames, ensuring that only the axles truly belonging to the target truck are counted, guaranteeing the accuracy of axle number statistics and avoiding billing errors caused by non-truck axles. Therefore, according to the embodiments of the present application, not only effectively solves the limitations of the existing vehicle wheel axle recognition technology in highway charging audit, but also realizes the full-process automation from image acquisition to axle detection and then to charging audit, greatly improving the accuracy and reliability of the entire charging audit process. This operation can not only reduce the operating cost, but also improve the service quality and user experience, bringing significant benefits to highway management.

[0116] Figure 3 The flowchart of a model training method provided by the embodiments of the present application is shown. Refer to Figure 3 As shown, the method specifically includes the following steps:

[0117] S301. Obtain the vehicle driving video and perform frame-by-frame processing on the vehicle driving video to obtain a plurality of video frames.

[0118] Exemplarily, collect the video data of the vehicle driving through the camera installed on the highway gantry, use a video processing tool (such as OpenCV) to perform batch frame-by-frame processing on the video, and remove duplicate, blurred or low-quality pictures, thereby obtaining a plurality of high-quality video frames.

[0119] S302. Perform marking processing on the truck objects in each video frame to obtain truck labels corresponding to each video frame, and crop each video frame according to the truck labels corresponding to each video frame to obtain multiple truck images. Among them, each truck image carries the corresponding truck label.

[0120] Exemplarily, for each video frame, use an annotation tool (such as LabelImg) to annotate the trucks in the video frame to generate a corresponding label file. This label file is usually in XML format, and the labels in the label file record the position information of the trucks, such as the coordinates of the truck bounding box. Based on the annotation information, crop the part containing the truck from the original video frame to generate a new image file, that is, obtain multiple truck images with truck labels, and use the images containing all truck-labeled images and their corresponding label files as the truck detection dataset.

[0121] S303. Perform marking processing on the axle objects in each truck image to obtain axle labels corresponding to each truck image.

[0122] Exemplarily, annotate the axles in the cropped truck images to generate corresponding label files. Specifically, use the annotation tool (such as LabelImg) again to annotate the axles in the truck images to generate label files for the axles. The labels in the label files record the position information of each axle, such as the coordinates of the axle bounding box. Further, save the annotation results (axle labels) as label files corresponding to the image files to obtain the axle detection dataset. This axle detection dataset contains all truck images with axle labels and their corresponding label files.

[0123] S304. Train the target detection model according to each truck image carrying the truck label to obtain a truck detection model, and train the target detection model according to each truck image carrying the axle label to obtain a truck axle detection model.

[0124] Exemplarily, each truck image carrying the truck label and each truck image carrying the axle label are used as the annotated dataset to train two different target detection models, such as the truck detection model for truck detection and the truck axle detection model for axle detection. Specifically, select a target detection model such as YOLOv9, input the truck detection dataset into the selected target detection model for model training, adjust the hyperparameters to optimize the model performance, and use cross-validation and the test set to evaluate the model performance to train the truck detection model. Similarly, input the axle detection dataset into the target detection model for model training and optimize the model parameters, and use cross-validation and the test set to evaluate the model performance to train the truck axle detection model.

[0125] Based on this, by collecting data and marking it, and training the target detection network, a truck detection model and a truck axle detection model applicable to the specific application scenario of truck toll auditing can be efficiently and accurately constructed. Thus, when obtaining vehicle image frames and region-of-interest (ROI) images, the trained truck detection model can accurately identify the target truck from the vehicle image frames and crop the ROI image containing the target truck, and the trained truck axle detection model can accurately identify and mark the axle frames of the target truck from the ROI images.

[0126] Figure 4 FIG. shows a schematic flowchart of a method for determining a region-of-interest (ROI) image provided by an embodiment of the present application. Referring to Figure 4 as shown, the above step S102 performs object recognition on multiple vehicle image frames and crops the ROI image containing the target truck from the multiple vehicle image frames, including:

[0127] S401. For each vehicle image frame, input the vehicle image frame into a pre-trained truck detection model, and the truck detection model performs vehicle recognition on the vehicle image frame to determine the position information of the target truck in the vehicle image frame.

[0128] Exemplarily, the truck detection model has been trained on a large number of labeled datasets. When a new vehicle image frame is input, the truck detection model will output the position information of all detected trucks in the vehicle image frame, and this position information is also the boundary box coordinates where the trucks are located.

[0129] S402. Generate a boundary box in the vehicle image frame based on the position information, and crop the vehicle image frame according to the boundary box to obtain the ROI image.

[0130] Exemplarily, draw a boundary box in the original vehicle image frame according to the boundary box coordinates, and crop the part of the image containing the truck as the region-of-interest (ROI) image according to the drawn boundary box, so as to obtain the ROI image as shown in Figure 5 FIG.

[0131] Based on this, input the vehicle image frame into the pre-trained truck detection model, utilize the powerful recognition ability of the model to automatically detect and locate the trucks in the image, and then crop the part of the image containing the trucks as the ROI image according to the positioning information. This process not only improves the detection efficiency but also enhances the accuracy of the results, laying a foundation for further axle detection and toll auditing.

[0132] As a possible implementation, the above-mentioned step S103 performs axle detection based on the region of interest image to determine the initial axle detection result of the target truck, including: inputting the region of interest image into a pre-trained truck axle detection model, and the truck axle detection model identifies the axles in the region of interest image, determines the axle frames and the initial number of axles of at least one axle in the region of interest image, and takes the position information of each axle and the initial number of axles as the initial axle detection result.

[0133] Exemplarily, the truck axle detection model has been trained on a large number of labeled data sets. When a new ROI image is input, the truck axle detection model analyzes the input ROI image, identifies all possible axle positions, and outputs the positions (i.e., bounding box coordinates) and confidence scores of each detected axle in the ROI image. Among them, the bounding box coordinates identify the upper left and lower right coordinates of the axle frame, and the confidence score represents the confidence level of the truck axle detection model in the detection result.

[0134] Exemplarily, according to the output result of the truck axle detection model, the final initial axle detection result is further processed. Specifically, a confidence threshold is set, and only the detection results with confidence scores greater than this confidence threshold are retained. The number of retained axle frames is counted as the initial number of axles, and the position information of each retained axle frame is recorded.

[0135] Based on this, the region of interest image ROI is input into the pre-trained truck axle detection model. By using the powerful recognition ability of the model, the axles on the truck are automatically detected and located, so as to obtain the initial axle detection result including the position information of the axle frame and the initial number of axles. This process not only improves the detection efficiency but also enhances the accuracy of the result, laying a foundation for further axle screening and toll auditing.

[0136] Figure 6 shows a schematic flowchart of a method for determining the number of target axles provided by an embodiment of the present application. Refer to Figure 6 As shown, the above-mentioned step S104 excludes the axle frames belonging to non-truck axles from the initial axle detection result and updates the initial number of axles to obtain the target number of axles of the target truck, which specifically includes the following steps:

[0137] S601. For each axle frame corresponding to an axle in the region of interest image, determine the center point and the center point coordinates of each axle frame, and combine the center points and the center point coordinates of each vehicle target frame to obtain a center point set.

[0138] Exemplarily, obtain the position information of each axle box. Taking the position information including the coordinates of the upper left corner A(x1, y1) and the lower right corner B(x2, y2) as an example, the center point O(x, y) corresponding to the axle box has the center point coordinates of ((x1 + x2) / 2, (y1 + y2) / 2). And so on, calculate the center points and the center point coordinates of all axle boxes in the region of interest image to obtain the center point set S.

[0139] S602. Perform fitting processing based on the center point set to obtain multiple candidate fitting lines, and determine the number of inliers in each candidate fitting line.

[0140] Optionally, each inlier is used to indicate the center point of an axle box. In this application, the Random Sample Consensus (RANSAC) algorithm can be used to perform fitting processing on the center point set. The RANSAC algorithm is an iterative algorithm for correctly estimating the parameters of a mathematical model from a set of data containing "outliers". Among them, "outliers" generally refer to the noise in the data, such as mismatches in matching and outliers in the estimated curve. Therefore, the RANSAC algorithm is also an outlier detection algorithm.

[0141] Optionally, referring to Figure 7 As shown, the above step S602 performs fitting processing based on the center point set to obtain multiple candidate fitting lines, and determine the number of inliers in each candidate fitting line, including:

[0142] S701. Randomly select two center points from the center point set as the first data set.

[0143] Exemplarily, taking the center point set S including six center points C, D, E, F, G, and H as an example, randomly select two points from the center point set S. For example, randomly select the center point C and the center point D as the first data set S1.

[0144] S702. Construct an initial line according to the center point coordinates of the two center points in the first data set.

[0145] Exemplarily, continuing with the above example where the center point set includes six center points C, D, E, F, G, and H, and the center point C and the center point D are selected as the first data set S1, construct an initial line y = a*x + b according to the center point coordinates corresponding to the center point C and the center point D.

[0146] S703. For each target center point in the center point set except the two center points in the first data set, determine the Euclidean distance between the target center point and the initial line, and determine the point attribute of the target center point according to the Euclidean distance, and determine the consistency set corresponding to the first data set according to the point attributes of each target center point.

[0147] Exemplarily, continuing with the example where the set S of central points includes six central points C, D, E, F, G, and H, and selecting central points C and D as the first data set, the target central points are the remaining central points. For the remaining central points (E, F, G, H) in the set S of central points, the Euclidean distance from each remaining central point to the initial line y = a * x + b is determined. If the Euclidean distance from the remaining central point to the initial line is less than the threshold T, then the remaining central point is determined to be an inlier, otherwise it is an outlier, thereby determining the point attributes of the remaining central points, which are used to indicate whether the target central point is an inlier or an outlier. Among them, the threshold T is generally set to a preset multiple of the standard deviation of the calculated Euclidean distance, such as 1.5 times or 2 times.

[0148] Further, according to the point attributes of each target central point, all inliers are grouped into a data set to obtain a consensus set, that is, the consensus set is the data set composed of inliers. Exemplarily, referring to Figure 8 as shown Figure 8 in the red points are outliers and the black points are inliers, and the data set composed of all black points is denoted as the consensus set S1*.

[0149] S704. Repeat the above steps S701 - S703 to obtain multiple consensus sets.

[0150] Exemplarily, two other central points are re - selected from the set S of central points as the first data set, and an initial line is constructed based on the first data set. Then, the Euclidean distance between the target central points in the set S of central points except the two selected central points and the initial line is determined, and according to the Euclidean distance, it is determined whether the target central point is an inlier or an outlier, thereby determining the consensus set S1* corresponding to the first data set. And so on, repeating the above process, multiple consensus sets S1* can be obtained.

[0151] S705. Perform fitting processing on each consensus set respectively to obtain multiple candidate fitting lines, and use the number of points in the consensus set as the number of inliers in the candidate fitting lines.

[0152] Exemplarily, the least - squares method can be used to perform fitting processing on each consensus set S1* to obtain multiple candidate fitting lines. Taking the example where the consensus set S1* includes n inliers, and the coordinates of each inlier are (x1, y1), (x2, y2), (x3, y3)…(xn, yn), a line y = a * x + b can be fitted based on these n inliers, and the sum of the squared deviations of these n inliers from the line y = a * x + b is shown in the following formula (1):

[0153]

[0154] where S2 represents the sum of squared deviations, \(x_i\) represents the abscissa of each inlier, \(y_i\) represents the ordinate of each inlier, \(a\) and \(b\) represent constant terms, and \(n\) represents the number of inliers.

[0155] Exemplarily, taking the partial derivatives of the above formula (1), when the two partial derivatives are 0, the minimum point of \(S\) 2 is solved for the two constant terms \(a\) and \(b\), and the specific meaning of the straight line \(y = ax + b\) can be obtained. The solution of the partial derivatives is specifically shown in the following formulas (2) and (3):

[0156]

[0157] Exemplarily, according to the above calculation method, for each consistency set \(S1^*\), a straight line \(y = ax + b\) can be fitted according to the coordinates of the inliers in the consistency set \(S1^*\), and the specific meaning of the parameter terms can be obtained by taking the partial derivatives, so as to fit multiple candidate fitting straight lines, and the number of points in the consistency set is used as the number of inliers in the candidate fitting straight lines.

[0158] S603. Determine the target fitting straight line according to the number of inliers in each candidate fitting straight line.

[0159] Optionally, the target fitting straight line can be the candidate fitting straight line with the largest number of inliers among multiple candidate fitting straight lines. Refer to Figure 9 As shown, the above step S603 determines the target fitting straight line according to the number of inliers in each candidate fitting straight line, and specifically includes the following steps:

[0160] S901. Sort all candidate fitting straight lines according to the number of inliers in each candidate fitting straight line to obtain a straight line sequence.

[0161] Exemplarily, assume that there are candidate fitting straight line 1 as \(y = a_1x + b_1\), candidate fitting straight line 2 as \(y = a_2x + b_2\), candidate fitting straight line 3 as \(y = a_3x + b_3\), candidate fitting straight line 4 as \(y = a_4x + b_4\), and the number of inliers in candidate fitting straight line 1 is 6, the number of inliers in candidate fitting straight line 2 is 7, the number of inliers in candidate fitting straight line 3 is 8, and the number of inliers in candidate fitting straight line 4 is 5. Then, sort the candidate fitting straight lines in descending order according to the number of inliers in each candidate fitting straight line. The order of the candidate fitting straight lines in the obtained straight line sequence is candidate fitting straight line 3, candidate fitting straight line 2, candidate fitting straight line 1, and candidate fitting straight line 4.

[0162] S902. Obtain the first \(M\) candidate fitting straight lines in the straight line sequence, where \(M\) is an integer greater than 0.

[0163] Exemplarily, taking the order of each candidate fitting line in the above straight line sequence as candidate fitting line 3, candidate fitting line 2, candidate fitting line 1, and candidate fitting line 4 in turn, and taking M as 3, the first 3 candidate fitting lines are obtained from the straight line sequence, that is, candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1.

[0164] S903. For each candidate fitting line among the first M candidate fitting lines, respectively determine the actual slope of the candidate fitting line, and determine the difference value between the actual slope and the preset slope.

[0165] Exemplarily, for each candidate fitting line among the first M candidate fitting lines, taking the above candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1 as examples, respectively determine the actual slopes of candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1, and respectively determine the difference values between the actual slopes of candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1 and the preset slope set in advance. Among them, the preset slope is, for example, 0.2, and this preset slope can be adjusted accordingly according to the test results during actual application.

[0166] S904. According to the difference values corresponding to each candidate fitting line among the first M candidate fitting lines, determine the target fitting line from the first M candidate fitting lines.

[0167] Exemplarily, continuing to take the above first M candidate fitting lines as candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1 as examples, the difference between the actual slope of candidate fitting line 3 and the preset slope is 0.11, the difference between the actual slope of candidate fitting line 2 and the preset slope is 0.15, and the difference between the actual slope of candidate fitting line 1 and the preset slope is 0.09. It can be obtained that the difference between the actual slope of candidate fitting line 1 and the preset slope is the smallest, so candidate fitting line 1 can be used as the target fitting line.

[0168] In another embodiment, the above step S603 determines the target fitting line according to the number of inliers in each candidate fitting line, including: respectively determining the actual slope of each candidate fitting line, and determining the difference range corresponding to each candidate fitting line according to each actual slope and the preset slope; determining the target fitting line according to the number of inliers within the difference range corresponding to each candidate fitting line.

[0169] Optionally, the difference range is an angular interval. For each candidate fitting line, calculate the difference between its actual slope and the preset slope, and set an allowable maximum difference range (such as ±5 degrees). On the premise of meeting this difference range, select the fitting line with the largest number of inliers as the target fitting line. For example, continuing with the above-mentioned first M candidate fitting lines as candidate fitting line 3, candidate fitting line 2, and candidate fitting line 1, the slope of candidate fitting line 3 is 0.05, the intercept is 10, and the number of inliers is 4; the slope of candidate fitting line 2 is 0.1, the intercept is 8, and the number of inliers is 3; the slope of candidate fitting line 1 is 0.01, the intercept is 12, and the number of inliers is 5. Since the axles of the target truck are usually horizontally arranged under normal circumstances, the preset slope can be set to 0, and the allowable maximum slope difference range is ±0.05 (i.e., ±5 degrees). It is calculated that the slope difference of candidate fitting line 3 is 0.05, the slope difference of candidate fitting line 2 is 0.1, and the slope difference of candidate fitting line 1 is 0.01. It can be seen that candidate fitting line 2 does not meet the requirements, and the fitting lines that meet the conditions are candidate fitting line 3 and candidate fitting line 1. However, since the number of inliers of candidate fitting line 1 is the largest at 5, candidate fitting line 1 can be used as the final target fitting line.

[0170] Exemplarily, referring to Figure 10 as shown, Figure 10 the red line in it is a candidate fitting line. The number of inliers included in the finally determined target fitting line is the largest, and the target fitting line represents the connection line of the center points of each axle box (such as Figure 10 shown by the green box in it), that is, the axle connection line.

[0171] 604. Determine the target axle number of the target truck according to the number of inliers in the target fitting line.

[0172] Exemplarily, take the number of inliers in the target fitting line as the target axle number of the target truck. For example, if the number of inliers in the target fitting line is 4, then it is determined that the target axle number of the target truck is also 4.

[0173] Based on this, the axle boxes that do not belong to the target truck are effectively excluded from the initial axle detection results, and the actual axle number of the target truck is accurately determined, improving the accuracy of axle detection.

[0174] As a possible implementation, step S105 above performs toll auditing on the target truck according to the target axle number and the registration information of the target truck, including: comparing the target axle number with the axle number in the registration information. If the axle numbers are the same, then determine the payment information of the target truck according to the vehicle type of the target truck and the truck toll standard.

[0175] Exemplarily, obtain the registration information of the target truck from the vehicle registration information system, including the number of axles, vehicle type, etc. Compare the actual number of axles of the target truck obtained through image processing and model detection with the number of axles in the registration information. If the actually detected number of axles does not match the registration information, there may be license plate cloning or other abnormal situations, and further investigation is required, such as measures like checking the authenticity of the license plate and contacting the vehicle owner for confirmation to determine whether there is license plate cloning behavior.

[0176] Exemplarily, since different types of trucks (such as light-duty and heavy-duty) may have different toll rates, if the number of axles is the same, further determine the corresponding toll rate according to the vehicle type, and calculate the payable fee based on this toll rate, that is, obtain the toll payment information of the target truck.

[0177] Based on this, by comparing the actually detected number of axles of the target truck with the number of axles in its registration information, ensure the consistency of the two, thereby excluding the possibility of license plate cloned vehicles. And if the number of axles is the same, further determine the final toll amount according to the vehicle type, the number of axles, and other relevant factors, such as the total vehicle weight. This process not only ensures the accuracy of toll collection but also improves the safety and reliability of the entire highway toll collection system.

[0178] Based on the same inventive concept, an embodiment of the present application also provides a truck toll auditing device corresponding to the truck toll auditing method. Since the principle of solving problems by the truck toll auditing device in the embodiment of the present application is similar to the above-mentioned truck toll auditing method in the embodiment of the present application, the implementation of the truck toll auditing device can refer to the implementation of the truck toll auditing method, and the repeated parts will not be elaborated.

[0179] Refer to Figure 11 As shown, it is a schematic structural diagram of a truck toll auditing device provided by an embodiment of the present application. The truck toll auditing device 1100 includes: an acquisition module 1101, an identification module 1102, an axle detection module 1103, an update module 1104, and a toll auditing module 1105, where:

[0180] The acquisition module 1101 is used to acquire multiple vehicle image frames, and the vehicle image frames are obtained by frame-dividing the vehicle video collected by the image acquisition device installed on the highway gantry.

[0181] The identification module 1102 is used to perform object recognition on multiple vehicle image frames and crop the region of interest image containing the target truck from the multiple vehicle image frames.

[0182] The axle detection module 1103 is used to detect axles based on the image of the region of interest, and determine the initial axle detection result of the target truck. The initial axle detection result includes: the axle frames of at least one axle in the image of the region of interest and the initial number of axles;

[0183] The update module 1104 is used to remove the axle frames belonging to non-truck axles from the initial axle detection result, and update the initial number of axles to obtain the target number of axles of the target truck;

[0184] The toll auditing module 1105 is used to conduct toll auditing on the target truck according to the target number of axles and the registration information of the target truck.

[0185] Based on this, the truck toll auditing device according to the embodiments of the present application uses a camera installed on the highway gantry to automatically capture the videos of passing vehicles, and performs frame-by-frame processing to generate high-quality vehicle image frames, thereby reducing the cost and complexity of hardware deployment, while improving the reliability and efficiency of data collection. On this basis, by performing object recognition on the vehicle image frames, the position of the target truck is accurately located, and the image of the region of interest containing the target truck is cropped, and the image of the region of interest is used as the input for axle detection, ensuring that even in a complex background environment, the axle position can be accurately identified, providing a reliable initial axle detection result, and by removing the axle frames belonging to non-truck axles from the initial axle detection result to exclude abnormal axle frames, ensuring that only the axles truly belonging to the target truck are counted, guaranteeing the accuracy of axle number statistics and avoiding billing errors caused by non-truck axles. Therefore, according to the embodiments of the present application, not only the limitations of the existing vehicle wheel and axle recognition technology in highway toll auditing are effectively solved, but also the full-process automation from image acquisition to axle detection to toll auditing is realized, greatly improving the accuracy and reliability of the entire toll auditing process. This operation can not only reduce the operating cost, but also improve the service quality and user experience, bringing significant benefits to highway management.

[0186] In a possible implementation manner, the recognition module 1102 is specifically used for:

[0187] For each vehicle image frame, input the vehicle image frame into a pre-trained truck detection model, and the truck detection model performs vehicle recognition on the vehicle image frame to determine the position information of the target truck in the vehicle image frame;

[0188] Generate a bounding box in the vehicle image frame based on the position information, and crop the vehicle image frame according to the bounding box to obtain the image of the region of interest.

[0189] In a possible implementation manner, the axle detection module 1103 is specifically used for:

[0190] Input the image of the region of interest into the pre-trained truck axle detection model. The truck axle detection model identifies axles in the image of the region of interest, determines the axle frames of at least one axle in the image of the region of interest, and takes the position information of each axle and the initial number of axles as the initial axle detection result.

[0191] In a possible implementation, the update module 1104 is specifically configured to:

[0192] For each axle frame corresponding to an axle in the image of the region of interest, determine the center point and the center point coordinates of each axle frame, and combine the center points and the center point coordinates of each vehicle target frame to obtain a set of center points;

[0193] Perform a fitting process based on the set of center points to obtain multiple candidate fitting lines, and determine the number of inliers in each candidate fitting line, where each inlier is used to indicate the center point of an axle frame;

[0194] Determine the target fitting line according to the number of inliers in each candidate fitting line;

[0195] Determine the number of target axles of the target truck according to the number of inliers in the target fitting line.

[0196] In a possible implementation, the update module 1104 is specifically configured to:

[0197] A. Randomly select two center points from the set of center points as the first data set;

[0198] B. Construct an initial line according to the center point coordinates of the two center points in the first data set;

[0199] C. For each target center point in the set of center points except the two center points in the first data set, determine the Euclidean distance between the target center point and the initial line, and determine the point attribute of the target center point according to the Euclidean distance, and determine the consistency set corresponding to the first data set according to the point attributes of each target center point, where the point attribute is used to indicate whether the target center point is an inlier or an outlier, and the consistency set is a data set composed of inliers;

[0200] D. Repeat the above steps A - C to obtain multiple consistency sets;

[0201] E. Perform a fitting process on each consistency set respectively to obtain multiple candidate fitting lines, and take the number of points in the consistency set as the number of inliers in the candidate fitting line.

[0202] In a possible implementation, the update module 1104 is specifically configured to:

[0203] Sort all candidate fitting lines according to the number of inliers in each candidate fitting line to obtain a line sequence;

[0204] Obtain the first M candidate fitting lines in the line sequence, where M is an integer greater than 0;

[0205] For each candidate fitting line among the first M candidate fitting lines, determine the actual slope of the candidate fitting line and determine the difference value between the actual slope and the preset slope;

[0206] Determine the target fitting line from the first M candidate fitting lines according to the difference values corresponding to each candidate fitting line among the first M candidate fitting lines.

[0207] In a possible implementation manner, the update module 1104 is specifically configured to:

[0208] Determine the actual slopes of the candidate fitting lines respectively, and determine the difference range corresponding to each candidate fitting line according to each actual slope and the preset slope;

[0209] Determine the target fitting line according to the number of inliers within the difference range corresponding to each candidate fitting line.

[0210] In a possible implementation manner, the toll audit module 1105 is specifically configured to:

[0211] Compare the target axle number with the axle number in the registration information. If the axle numbers are the same, determine the payment information of the target truck according to the vehicle type of the target truck and the truck toll standard.

[0212] In a possible implementation manner, the truck toll audit device 1100 further includes a training module, and the training module is specifically configured to:

[0213] Obtain the vehicle driving video, and perform frame division processing on the vehicle driving video to obtain a plurality of video frames;

[0214] Perform labeling processing on the truck objects in each video frame to obtain the truck labels corresponding to each video frame, and crop each video frame according to the truck labels corresponding to each video frame to obtain a plurality of truck images, wherein each truck image carries the corresponding truck label;

[0215] Perform labeling processing on the axle objects in each truck image to obtain the axle labels corresponding to each truck image;

[0216] Train the target detection model according to the truck images carrying the truck labels to obtain a truck detection model, and train the target detection model according to the truck images carrying the axle labels to obtain a truck axle detection model.

[0217] Descriptions of the processing flows of the various modules in the device and the interaction flows between the modules can refer to the relevant descriptions in the above method embodiments, and will not be elaborated here.

[0218] An embodiment of the present application also provides an electronic device 1200, as Figure 12 shown, which is a schematic structural diagram of the electronic device 1200 provided by the embodiment of the present application, including: a processor 1201, a memory 1202. Optionally, a bus 1203 may also be included. The memory 1202 stores machine-readable instructions executable by the processor 1201. When the electronic device 1200 runs, the processor 1201 communicates with the memory 1202 through the bus 1203. When the machine-readable instructions are executed by the processor 1201, the steps in any one of the above truck toll auditing methods are executed.

[0219] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps in any one of the above truck toll auditing methods are executed.

[0220] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, and will not be elaborated in the present application. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces, and the indirect couplings or communication connections of the devices or modules can be in electrical, mechanical or other forms.

[0221] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0222] The above are only specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application.

Claims

1. A truck toll audit method, characterized in that, Including: Obtaining a plurality of vehicle image frames, which are obtained by performing frame division on a vehicle video collected by an image acquisition device installed on a high-speed gantry; Performing object recognition on the plurality of vehicle image frames, and cropping from the plurality of vehicle image frames an image of an interested region containing a target truck; Performing axle detection based on the image of the interested region to determine an initial axle detection result of the target truck, where the initial axle detection result includes: axle frames of at least one axle in the image of the interested region and an initial number of axles; Removing axle frames belonging to non-truck axles from the initial axle detection result, and updating the initial number of axles to obtain the target number of axles of the target truck; Performing toll audit on the target truck according to the target number of axles and the registration information of the target truck.

2. The method according to claim 1, characterized in that The performing object recognition on the plurality of vehicle image frames and cropping from the plurality of vehicle image frames an image of an interested region containing a target truck includes: For each of the vehicle image frames, inputting the vehicle image frame into a pre-trained truck detection model, and performing vehicle recognition on the vehicle image frame by the truck detection model to determine the position information of the target truck in the vehicle image frame; Generating a bounding box in the vehicle image frame based on the position information, and cropping the vehicle image frame according to the bounding box to obtain the image of the interested region.

3. The method according to claim 1, wherein The performing axle detection based on the image of the interested region to determine the initial axle detection result of the target truck includes: Inputting the image of the interested region into a pre-trained truck axle detection model, and performing axle recognition on the image of the interested region by the truck axle detection model to determine axle frames of at least one axle in the image of the interested region, and taking the position information of each axle and the initial number of axles as the initial axle detection result.

4. The method according to claim 1, wherein The removing axle frames belonging to non-truck axles from the initial axle detection result and updating the initial number of axles to obtain the target number of axles of the target truck includes: For each axle frame corresponding to an axle in the image of the interested region, determining the center point and the center point coordinates of each axle frame, and combining the center points and the center point coordinates of each vehicle target frame to obtain a set of center points; Performing fitting processing based on the set of center points to obtain a plurality of candidate fitting lines, and determining the number of inliers in each candidate fitting line, where each inlier is respectively used to indicate the center point of an axle frame; Determining a target fitting line according to the number of inliers in each candidate fitting line; Determining the target number of axles of the target truck according to the number of inliers in the target fitting line.

5. The method according to claim 4, characterized in that, The performing fitting processing based on the set of center points to obtain a plurality of candidate fitting lines and determining the number of inliers in each candidate fitting line includes: A. Randomly extracting two center points from the set of center points as a first data set; B. Constructing an initial line according to the center point coordinates of the two center points in the first data set; C. For each target center point in the set of center points except for the two center points in the first data set, determine the Euclidean distance between the target center point and the initial straight line, determine the point attribute of the target center point according to the Euclidean distance, and determine the consistency set corresponding to the first data set according to the point attributes of the target center points, where the point attribute is used to indicate whether the target center point is an inlier or an outlier, and the consistency set is a data set composed of inliers; D. Repeat the above steps A - C to obtain multiple consistency sets; E. Perform fitting processing on each of the consistency sets to obtain multiple candidate fitting straight lines, and use the number of points in the consistency set as the number of inliers in the candidate fitting straight line.

6. The method according to claim 4, characterized in that, The determining the target fitting straight line according to the number of inliers in each candidate fitting straight line includes: Sort all candidate fitting straight lines according to the number of inliers in each candidate fitting straight line to obtain a straight line sequence; Obtain the first M candidate fitting straight lines in the straight line sequence, where M is an integer greater than 0; For each candidate fitting straight line in the first M candidate fitting straight lines, respectively determine the actual slope of the candidate fitting straight line and determine the difference value between the actual slope and the preset slope; Determine the target fitting straight line from the first M candidate fitting straight lines according to the difference values corresponding to the candidate fitting straight lines in the first M candidate fitting straight lines.

7. The method according to claim 4, wherein The determining the target fitting straight line according to the number of inliers in each candidate fitting straight line includes: Respectively determine the actual slope of each candidate fitting straight line, and determine the difference range corresponding to each candidate fitting straight line according to each actual slope and the preset slope; Determine the target fitting straight line according to the number of inliers within the difference range corresponding to each candidate fitting straight line.

8. The method according to claim 1, wherein The performing toll audit on the target truck according to the number of target axles and the registration information of the target truck includes: Compare the number of target axles with the number of axles in the registration information. If the number of axles is the same, determine the payment information of the target truck according to the vehicle type of the target truck and the truck toll standard.

9. The method according to claim 1, wherein The method further includes: Obtain a vehicle driving video, and perform frame - by - frame processing on the vehicle driving video to obtain multiple video frames; Perform labeling processing on the truck objects in each of the video frames to obtain truck labels corresponding to each of the video frames, and crop each of the video frames according to the truck labels corresponding to each of the video frames to obtain multiple truck images, where each truck image carries a corresponding truck label; Perform labeling processing on the axle objects in each of the truck images to obtain axle labels corresponding to each of the truck images; Train a target detection model using each truck image carrying the truck label to obtain a truck detection model, and train a target detection model using each truck image carrying the axle label to obtain a truck axle detection model.

10. An electronic device, characterized in that, Includes: A processor and a memory, the memory storing machine-readable instructions executable by the processor, and when the electronic device operates, the processor executes the machine-readable instructions to perform the steps of the truck toll auditing method according to any one of claims 1 to 9.

Images