Vehicle charging type identification method and vehicle charging type identification system

Abstract

The invention provides a vehicle charging type identification method and a vehicle charging type identification system, and relates to the technical field of intelligent traffic management. The method comprises the following steps: acquiring vehicle driving data; vehicle identification and tracking are carried out based on the plurality of video frames, and first feature information of a target vehicle running on the target lane is determined; performing protocol analysis on the radar data, and determining a plurality of pieces of second feature information; according to the lane number of the target lane and a preset trigger line, performing fusion matching on the first feature information and the multiple pieces of second feature information, and determining attribute information of the target vehicle; and determining the charging type of the target vehicle according to the attribute information. According to the invention, the target vehicle is identified and tracked more comprehensively and accurately by simultaneously using the accurate position, speed and other information provided by the radar device and the visual information provided by the image acquisition device, and time synchronization and accurate definition are carried out on the feature information from different data sources based on the preset trigger line and the lane number. And fusing and determining the charging type of the target vehicle.

Description

Technical Field

[0001] This application relates to the technical field of intelligent traffic management. Specifically, it relates to a method and a system for identifying vehicle toll types. Background Art

[0002] With the rapid development of intelligent traffic technology, the highway vehicle toll system has become an indispensable part of modern traffic management. Along with the continuous increase in the number of highway toll stations, how to collect vehicle tolls efficiently and accurately has become an urgent problem to be solved.

[0003] Currently, traditional toll collection methods include the Electronic Toll Collection (ETC) method and the manual toll collection method. Among them, due to factors such as privacy protection and security issues, the usage rate of the ETC toll collection method has not reached an ideal state and has not achieved full coverage. The manual toll collection method captures license plate images by setting up cameras at toll stations and then completes toll collection by identifying license plate numbers based on the license plate images. However, the manual toll collection method usually requires vehicles to stop briefly at toll stations to ensure the effectiveness of license plate recognition.

[0004] Although the above traditional toll collection methods have improved the toll collection efficiency to a certain extent, they still face many challenges in practical applications. For example, adverse weather conditions, the diversity of usage scenarios in different regions and time periods, and license plate styles will all limit license plate recognition, thereby affecting the toll collection efficiency and reliability of the highway vehicle toll system. Summary of the Invention

[0005] The purpose of this application is to provide a method and a system for identifying vehicle toll types to solve the problem that license plate recognition in the prior art has limitations, thereby affecting the toll collection efficiency and reliability of the highway vehicle toll system, aiming at the deficiencies in the above-mentioned prior art.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, the embodiments of this application provide a method for identifying vehicle toll types, which is applied to an edge device in a vehicle toll type identification system. The vehicle toll type identification system includes the edge device, a plurality of radar devices and a plurality of image acquisition devices arranged in the area where the vehicle toll station is located. Each image acquisition device is respectively used to collect video frames on one lane in the area where the vehicle toll station is located. The method includes:

[0008] Obtain vehicle driving data, where the vehicle driving data includes: radar data of vehicles driving on each lane collected by the radar device, and multiple video frames of each vehicle driving on each lane collected by the image acquisition device;

[0009] Based on the multiple video frames, perform vehicle identification and tracking to determine first feature information of a target vehicle driving on a target lane;

[0010] Perform protocol parsing on the radar data to determine multiple second feature information, and each of the second feature information is respectively used to indicate the feature information of each vehicle driving on each lane;

[0011] According to the lane number of the target lane and a preset trigger line, perform fusion matching on the first feature information and the multiple second feature information to determine the attribute information of the target vehicle. The preset trigger line is a boundary line set on the target road for triggering the identification of the toll type of the vehicle, and the attribute information includes vehicle type, license plate number, and number of axles;

[0012] According to the attribute information of the target vehicle, determine the toll type of the target vehicle.

[0013] As a possible implementation manner, the performing vehicle identification and tracking based on the multiple video frames to determine first feature information of a target vehicle driving on a target lane includes:

[0014] Perform preprocessing on each of the video frames to obtain multiple processed video frames;

[0015] Based on each first processed video frame, perform vehicle detection, obtain first position information of the target vehicle in each first processed video frame, and assign a unique identifier to the target vehicle when the target vehicle is first detected. Among them, the first processed video frame is a video frame collected before the target vehicle enters the trigger area where the preset trigger line is located;

[0016] Based on the unique identifier of the target vehicle, perform multi-target tracking on the target vehicle to determine whether the vehicle detected in each second processed video frame is the same vehicle as the target vehicle. Among them, the second processed video frame is a video frame collected after the target vehicle enters the trigger area where the preset trigger line is located;

[0017] If so, then based on the second position information of the target vehicle in each of the second processed video frames, crop each of the second processed video frames to obtain multiple target vehicle images;

[0018] Based on multiple of the target vehicle images, perform feature recognition to determine the first feature information of the target vehicle.

[0019] As a possible implementation, for multi-object tracking of the target vehicle based on the unique identifier of the target vehicle and determining whether the vehicles detected in each second processed video frame are the same as the target vehicle, it includes:

[0020] According to the unique identifier of the target vehicle and the first position information of the target vehicle in each first processed video frame, determine the movement trajectory of the target vehicle, and predict the predicted position of the target vehicle in each second processed video frame based on the movement trajectory;

[0021] Compare the predicted position of the target vehicle in each second processed video frame with the position of the vehicle detected in each second processed video frame. If the position difference is less than a preset value, determine that the detected vehicle is the same as the target vehicle.

[0022] As a possible implementation, for feature recognition based on multiple target vehicle images to determine the first feature information of the target vehicle, it includes:

[0023] For each target vehicle image, determine the vehicle type of the target vehicle based on the target vehicle image. If the vehicle type is a truck, perform axle detection on the target vehicle to obtain an axle detection result;

[0024] Perform license plate detection on the target vehicle image to determine the position information of the target license plate in the target vehicle image, crop the target vehicle image according to the position information to obtain a license plate image, input the license plate image into a pre-trained license plate detection model, convert the license plate image into a time series feature sequence, perform bidirectional feature extraction processing on the time series feature sequence to determine the context information of each character in the time series feature sequence, determine the probability distribution of each character according to the context information of each character, and determine the license plate number of the target license plate based on the probability distribution of each character;

[0025] Take the vehicle type, axle detection result, and license plate number as the initial feature information corresponding to the target vehicle image;

[0026] Compare and fuse the initial feature information corresponding to each target vehicle image to determine the first feature information of the target vehicle.

[0027] As a possible implementation, the license plate detection model is an improved object detection model. Specifically, at least one full-dimensional dynamic convolution layer is added to the original object detection model, and the intermediate layer network in the original object detection model is replaced with a generalized feature pyramid network to obtain the license plate detection model;

[0028] The full-dimensional dynamic convolution layer is used to convert the license plate image into a time series feature sequence;

[0029] The generalized feature pyramid network is used to perform bidirectional feature extraction processing on the time series feature sequence, determine the context information of each character in the time series feature sequence, and determine the probability distribution of each character according to the context information of each character.

[0030] As a possible implementation manner, the fusing and matching of the first feature information and the multiple second feature information according to the lane number of the target lane and the preset trigger line to determine the attribute information of the target vehicle includes:

[0031] Performing feature matching on the first feature information and each of the second feature information according to the position information of the preset trigger line and the lane number to obtain target second feature information that matches the first feature information, where the position information of the preset trigger line includes the coordinate range of two trigger points on the first coordinate axis and the pixel range on the second coordinate axis;

[0032] Fusing the first feature information and the target second feature information to determine the attribute information of the target vehicle.

[0033] As a possible implementation manner, the performing feature matching on the first feature information and each of the second feature information according to the position information of the preset trigger line and the lane number to obtain target second feature information that matches the first feature information includes:

[0034] Obtaining the vehicle information of each vehicle from each of the second feature information, performing coordinate transformation on the vehicle information of each vehicle, and determining whether the transformed coordinates corresponding to each of the second feature information satisfy the effective range condition. If so, taking the second feature information as an available feature information, where the effective range condition is that the vehicle is located within a preset coordinate range, and the preset coordinate range is determined based on the position information of the preset trigger line;

[0035] Screening out at least one candidate feature information belonging to the lane number from the multiple available feature information;

[0036] Obtaining the body width and center point coordinates of the target vehicle from the first feature information, and determining whether the body width and center point coordinates of the target vehicle satisfy the effective range condition. If so, determining that the target vehicle is in the target lane, and performing matching on the first feature information and each of the candidate feature information to obtain target second feature information that matches the first feature information.

[0037] As a possible implementation, parsing the radar data to determine multiple second feature information includes:

[0038] Preprocessing the radar data to obtain processed radar data, and parsing the processed radar data according to the radar communication protocol to extract effective vehicle information from the processed radar data;

[0039] Determining multiple pieces of the second feature information according to the effective vehicle information, where the second feature information at least includes vehicle length, lane number of the lane where the vehicle is located, and driving speed.

[0040] As a possible implementation, determining the toll type of the target vehicle according to the attribute information of the target vehicle includes:

[0041] Determining the toll type of the target vehicle according to the attribute information of the target vehicle and a preset discrimination criterion, where the preset discrimination criterion is used to indicate the correspondence between the toll type and the vehicle attribute information.

[0042] In a second aspect, an embodiment of the present application provides a vehicle toll type recognition system, which includes an edge device, multiple radar devices arranged in the area where the vehicle toll station is located, and multiple image acquisition devices. Among them, each image acquisition device is respectively used to acquire video frames on one lane in the area where the vehicle toll station is located;

[0043] The edge device is used to execute the steps of the vehicle toll type recognition method according to any one of the first aspects.

[0044] In a third aspect, an embodiment of the present application provides an edge device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the edge device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the steps of the vehicle toll type recognition method according to any one of the above first aspects.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the vehicle toll type recognition method according to any one of the above first aspects.

[0046] A vehicle toll type identification method and a vehicle toll type identification system according to an embodiment of the present application obtain vehicle driving data, where the vehicle driving data includes: radar data of vehicles driving on each lane collected by a radar device, and multiple video frames of each vehicle driving on each lane collected by an image acquisition device. Based on the multiple video frames, vehicle identification and tracking are performed to determine first feature information of a target vehicle driving on a target lane. The radar data is subjected to protocol analysis to determine multiple second feature information. According to the lane number of the target lane and a preset trigger line, the first feature information and the multiple second feature information are fused and matched to determine the attribute information of the target vehicle. According to the attribute information of the target vehicle, the toll type of the target vehicle is determined. According to the embodiment of the present application, by combining radar data and video frame analysis, the target vehicle is more comprehensively and accurately identified and tracked by simultaneously using information such as precise position and speed provided by the radar device and visual information provided by the image acquisition device, and the feature information from different data sources is time-synchronized and precisely defined based on the preset trigger line and lane number. Through time synchronization processing of the camera and the radar, it is ensured that the data of both can be aligned at the same moment, thereby realizing precise positioning of the vehicle position, and the attribute information of the target vehicle is obtained through fusion and matching, so that vehicle toll type identification can better adapt to different application scenarios and user requirements. In this way, not only the limitations of license plate recognition in the prior art are solved, but also the efficiency and reliability of the entire highway toll system are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 FIG. shows a schematic architecture diagram of a vehicle toll type identification system provided by an embodiment of the present application;

[0049] Figure 2 FIG. shows an identification flowchart of a vehicle toll type identification system provided by an embodiment of the present application;

[0050] Figure 3 FIG. shows a schematic flowchart of a vehicle toll type identification method provided by an embodiment of the present application;

[0051] Figure 4 FIG. shows a schematic flowchart of a method for determining first feature information of a target vehicle provided by an embodiment of the present application;

[0052] Figure 5Shows a schematic flowchart of another method for determining the first feature information of the target vehicle provided by an embodiment of the present application;

[0053] Figure 6 Shows a schematic flowchart of a method for determining the attribute information of a target vehicle provided by an embodiment of the present application;

[0054] Figure 7 Shows a schematic flowchart of a method for determining the target second feature information provided by an embodiment of the present application;

[0055] Figure 8 Shows a schematic diagram of a database monitoring and retransmission process provided by an embodiment of the present application;

[0056] Figure 9 Shows a schematic diagram of the structure of an edge device provided by an embodiment of the present application. Detailed implementation manners

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0058] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0059] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0060] Figure 1 Shows a schematic architecture diagram of a vehicle toll type recognition system provided by an embodiment of the present application. Refer to Figure 1As shown in the figure, the vehicle toll type recognition system includes edge devices (i.e., edge intelligent computing boxes), multiple radar devices installed in the area where the vehicle toll station is located, and multiple image acquisition devices. Among them, each image acquisition device is respectively used to acquire video frames on one lane in the area where the vehicle toll station is located.

[0061] Optionally, in the embodiments of this application, the image acquisition device and the radar device jointly form the front-end perception module of the vehicle toll type recognition system. The image acquisition device is, for example, a camera installed on the highway gantry, and the camera is responsible for acquiring vehicle image information. The radar device is, for example, a lidar or a millimeter-wave radar, and the radar device is mainly responsible for acquiring radar data, such as vehicle length, speed, lane number and other information. The edge device is the edge computing module of the vehicle toll type recognition system. The edge device is, for example, an edge intelligent computing box installed on the highway gantry, which can quickly receive the radar data acquired by the radar device and the vehicle image information acquired by the image acquisition device, such as multiple video frames, and quickly and accurately identify the vehicle toll type.

[0062] Optionally, the cloud computing module further includes a cloud-edge collaborative operation and maintenance management platform and a cloud-edge collaborative model automatic training platform. Among them, the cloud-edge collaborative operation and maintenance management platform is mainly responsible for collaborative management, monitoring and operation and maintenance, application management, hardware resource management, algorithm management, training task distribution, and cloud-edge model automatic deployment between the cloud (cloud server) and the edge (edge device). The cloud-edge collaborative model automatic training platform is mainly responsible for data preprocessing, model and data management, model performance evaluation and optimization, model compression and acceleration, operator automatic scheduling, and continuous learning. Specifically, the edge node accesses the cloud-edge collaborative operation and maintenance management platform through a certificate, returns the structured data and sample images processed by the algorithm to the cloud, the cloud-edge collaborative operation and maintenance management platform issues training tasks to the cloud-edge collaborative model automatic training platform, the cloud-edge collaborative model automatic training platform preprocesses the data, optimizes and trains the model, compresses and accelerates it, and the cloud-edge collaborative operation and maintenance management platform updates the trained new model to the model library on the edge side to realize the automatic upgrade and optimization of the model.

[0063] In addition, the vehicle toll type recognition system further includes a support and guarantee module, which includes a support member, a communication control box, and a fill light. The support member provides installation support for the front-end perception module. The communication control box includes a power module, a switch, and a relay, which are mainly used to provide power supply, communication, and installation of the edge intelligent computing box for the vehicle toll type recognition system. Through the switch, the vehicle toll recognition system can synchronize and integrate data with the highway toll system platform. The fill light is mainly used to provide fill light for the camera in low-light conditions. Among them, the edge intelligent computing box is the edge computing module of the vehicle toll type recognition system. In addition to the edge intelligent computing box, the edge computing module also includes highway vehicle toll type recognition software, such as some algorithm models for identifying vehicle toll types. The cloud server can also perform data collaboration on the edge computing module, perform online updates on the algorithm models, and achieve unified management of cloud-edge collaboration in other resources, applications, services, etc.

[0064] Optionally, referring to Figure 2 As shown, for the vehicle toll type recognition system provided by the embodiment of the present application, the oncoming vehicles are respectively sensed and detected by the camera and the radar, the video algorithm analysis is performed on multiple video frame data, and the protocol analysis is performed on the radar data to obtain the data analysis result. Then, according to the preset trigger line and lane number, the data in the data analysis result are fused and matched to determine the attribute information of the same target vehicle, so as to determine the toll type of the target vehicle in combination with the preset discrimination criterion. In addition, the relevant information of the target vehicle, such as vehicle type, vehicle length, vehicle speed, lane number, license plate number, etc., is synchronously sent to the highway toll system platform for storage in the database. If the sending fails, the relevant information can be resent in a rollback manner. In this way, the cloud server can optimize and upgrade the algorithm model by continuously accumulating vehicle data.

[0065] Based on this, the vehicle toll type recognition system provided by the embodiment of the present application can perform intelligent recognition based on multi-sensor fusion, simultaneously obtain video frames and radar data, perform license plate detection algorithm to identify the license plate number for multi-frame fusion and error correction, and use cloud-edge collaboration technology to achieve online update of the model. While identifying the vehicle toll type, it synchronizes and updates information with the highway toll system platform, thereby improving the toll collection efficiency and reliability of the highway vehicle toll system.

[0066] Next, in combination with the content described in the vehicle toll type recognition system shown above Figure 1 the vehicle toll type recognition method provided by the embodiment of the present application will be described in detail.

[0067] Figure 3 is a flowchart of a vehicle toll type recognition method provided by the embodiment of the present application. Referring to Figure 3As shown, the execution entity of this method is the edge device in the above vehicle toll type recognition system, and this method specifically includes the following steps:

[0068] S301. Obtain vehicle driving data. Among them, the vehicle driving data includes: radar data of vehicles driving on each lane collected by radar devices, and multiple video frames of each vehicle driving on each lane collected by image acquisition devices.

[0069] Optionally, in the embodiments of the present application, the radar devices installed on the gantry cover all lanes on the target road. The radar devices continuously emit electromagnetic waves and receive reflected signals. By analyzing the time difference and frequency change of the reflected signals, information such as the speed, distance, and direction of the vehicle is calculated. And the radar data is usually output in the form of point clouds or target lists, including but not limited to vehicle IDs for uniquely identifying each vehicle, vehicle positions (latitude and longitude or relative coordinates), vehicle speeds, vehicle directions, lane numbers, etc.

[0070] Optionally, an image acquisition device (such as a camera) is used to capture the appearance features, license plate information of the vehicle, and the overall situation on the target lane. The camera is also installed on the gantry, and one camera is installed corresponding to each lane, and the angle of the camera is adjusted to avoid blind spots. Exemplarily, the camera can capture videos at a fixed frame rate (such as 30 frames per second) to record the driving conditions of vehicles on the target lane.

[0071] S302. Perform vehicle identification and tracking based on multiple video frames, and determine the first feature information of the target vehicle driving on the target lane.

[0072] Optionally, based on multiple consecutive video frames collected by the camera, perform preprocessing operations such as denoising and enhancement on the video frames, detect the positions and bounding boxes of all vehicles in each frame, then track the same target vehicle in multiple consecutive frames to obtain the tracking result, and extract the first feature information of the target vehicle from the tracking result.

[0073] Optionally, in the embodiments of the present application, a target detection algorithm can be used for vehicle identification and tracking. For example, a deep learning model is used to detect vehicles in each frame, output the bounding boxes and vehicle types of each vehicle in each frame, such as sedans, buses, trucks, etc., and then a multi-target tracking algorithm is used to track the same target vehicle in consecutive frames to obtain the cross-frame trajectory information of the target vehicle, and feature extraction is performed based on the cross-frame trajectory information to obtain the first feature information of the target vehicle, including license plate number, vehicle type, number of axles, vehicle speed, vehicle direction, etc.

[0074] S303. Perform protocol parsing on the radar data to determine multiple second feature information, and each second feature information is respectively used to indicate the feature information of each vehicle driving on each lane.

[0075] Optionally, a radar device usually outputs data according to a specific communication protocol. Common protocols include, for example, the CAN bus protocol, the UDP / TCP protocol, and private protocols, etc. The radar device outputs the raw data stream through a network or a serial port. The raw data stream is usually presented in the form of a target list, and each target represents a detected vehicle. On this basis, the raw data stream is parsed to obtain the target ID for uniquely identifying the vehicle, the vehicle position information (such as distance, angle, height, etc.), the speed information (such as longitudinal speed, lateral speed), the direction information (such as the direction angle of the vehicle's travel), and the confidence level for identifying the reliability of the detection result, etc. Then, according to the lane number or the vehicle position information, the detected vehicles are assigned to the corresponding lanes, so as to obtain the characteristic information of each vehicle traveling on each lane, that is, a plurality of second characteristic information is obtained.

[0076] S304. According to the lane number of the target lane and the preset trigger line, fuse and match the first characteristic information and the plurality of second characteristic information to determine the attribute information of the target vehicle.

[0077] Optionally, the preset trigger line is a boundary line set on the target road for triggering the identification of the vehicle toll type. The attribute information includes the vehicle type, the license plate number, and the number of axles.

[0078] Optionally, use the camera calibration parameters to convert the image coordinates into the road coordinate system, determine whether the vehicle crosses the preset trigger line, and combine the vehicle position information to confirm the lane number where the vehicle is located and whether it crosses the preset trigger line. When it is detected that the vehicle crosses the preset trigger line, for each target vehicle that crosses the preset trigger line, fuse its first characteristic information (license plate number, vehicle type, number of axles, etc.) with the second characteristic information (speed, driving direction, etc.) to obtain the attribute information of the target vehicle. Among them, the vehicle type can be directly obtained from the vehicle type information identified in the video frame, or more accurate classification can be achieved by referring to the vehicle size information in the radar data. The license plate number is extracted from the video frame through the optical character recognition technology (OCR). In addition, for trucks, to determine the vehicle toll type, it is also necessary to determine the number of axles of the vehicle.

[0079] S305. According to the attribute information of the target vehicle, determine the toll type of the target vehicle.

[0080] Optionally, based on the determined attribute information of the target vehicle, the charging type of the target vehicle can be determined in combination with a preset charging criterion. The preset charging criterion is a charging standard defined according to different application scenarios. For example, classified by vehicle type, it can be divided into small vehicles, large vehicles, and special vehicles, etc., and the charging standard for each type is different. It can also be classified according to the number of axles. For large vehicles such as trucks, the charging may be adjusted according to the number of axles. For example, there may be different charging standards for two-axle vehicles, three-axle vehicles, and vehicles with four or more axles.

[0081] Based on this, according to the vehicle charging type recognition method provided by the embodiments of the present application, by combining radar data and video frame analysis, the target vehicle can be more comprehensively and accurately identified and tracked by simultaneously using the precise position, speed, and other information provided by the radar device and the visual information provided by the image acquisition device. And based on the preset trigger line and lane number, the feature information from different data sources is synchronized in time and accurately defined. Through the time synchronization process of the camera and the radar, it is ensured that the data of both can be aligned at the same moment, so as to achieve precise positioning of the vehicle position, fuse and match to obtain the attribute information of the target vehicle, enabling the vehicle charging type recognition to better adapt to different application scenarios and user needs. In this way, not only the limitations of license plate recognition in the prior art are solved, but also the efficiency and reliability of the entire highway toll system are greatly improved.

[0082] Figure 4 The flowchart showing the method for determining the first feature information of a target vehicle provided by the embodiments of the present application is shown. As a possible implementation, refer to Figure 4 As shown, the above step S302 performs vehicle recognition and tracking based on multiple video frames to determine the first feature information of the target vehicle traveling on the target lane, which specifically includes the following steps:

[0083] S401. Preprocess each video frame to obtain multiple processed video frames.

[0084] Optionally, since the quality of video frame acquisition is easily affected by weather, for example, in foggy days or scenes with poor lighting at night, the quality of the acquired video frames is also low. In the embodiments of the present application, to ensure the accuracy of video frame processing, an image dehazing algorithm and an image enhancement algorithm are used to preprocess the video frames acquired in foggy days and scenes with poor lighting at night. Among them, the image dehazing algorithm uses a dehazing algorithm based on the dark channel of the image. Its core idea is to use the dark channel prior of the image to estimate the atmospheric light and the transmission map, so as to restore the haze-free image. This algorithm first calculates the dark channel, then determines the atmospheric light value A, then obtains the transmission rate map, and finally restores the clear image according to the formula. The image enhancement algorithm uses a night image processing algorithm. This algorithm first sharpens the image using the Laplace operator to enhance the image details, then compares the local gray mean with the global gray mean and the local gray variance with the global gray variance to determine the dark background area of the night image, and segments the dark background area. The histogram equalization algorithm with local contrast enhancement is used to adjust the gray level of the segmented area to improve the contrast of the image. Finally, the non-linear image processing technology - Gamma transformation is used to soften the image brightness.

[0085] Exemplarily, for each video frame, the dehazing process includes: first calculating its dark channel map, selecting the brightest pixel point in the dark channel map as the atmospheric light value A, and estimating the transmission rate map based on the dark channel map and the atmospheric light value A, and then restoring the clear image based on the dehazing formula.

[0086] Exemplarily, for each video frame, the night enhancement process includes: applying the Laplace transform or other sharpening techniques to enhance the image details, analyzing the local and global gray means and variances, identifying and separating the dark areas that need to be enhanced, and implementing histogram equalization or similar techniques on the selected dark areas to improve the local contrast, and using Gamma transformation for correction to appropriately adjust the brightness distribution of the entire image to make the image look more comfortable and natural.

[0087] S402. Perform vehicle detection based on each first processed video frame, obtain the first position information of the target vehicle in each first processed video frame, and assign a unique identifier to the target vehicle when the target vehicle is first detected.

[0088] Wherein, the first processed video frame is a video frame acquired before the target vehicle enters the trigger area where the preset trigger line is located among multiple processed video frames.

[0089] Exemplarily, in the embodiments of the present application, a target detection algorithm based on YOLOv5 is used to detect vehicles in the first processed video frame obtained by preprocessing, and the position information of the target vehicle in the first processed video frame is obtained, such as bounding box coordinates. And when the target vehicle is detected for the first time, a unique identifier ID is assigned to it, and this identifier ID is used for subsequent multi-object tracking. That is to say, in the embodiments of the present application, each detected target vehicle will be assigned a unique identifier ID to distinguish different vehicles.

[0090] S403. Perform multi-object tracking on the target vehicle based on the unique identifier of the target vehicle, and determine whether the vehicle detected in each second processed video frame is the same vehicle as the target vehicle.

[0091] Among them, the second processed video frame is a video frame collected after the target vehicle drives into the trigger area where the preset trigger line is located among multiple processed video frames.

[0092] Exemplarily, check whether the target vehicle enters the preset trigger area, such as whether it passes the preset trigger line. If it does not enter, continue to monitor. If it has entered, use the unique identifier ID of the target vehicle to perform multi-object tracking in the subsequent collected second processed video frames to determine whether the currently detected vehicle is the same vehicle as the target vehicle that was previously assigned a unique identifier.

[0093] Optionally, determining whether the vehicle detected in each second processed video frame is the same vehicle as the target vehicle includes: determining the movement trajectory of the target vehicle according to the unique identifier of the target vehicle and the first position information of the target vehicle in each first processed video frame, and predicting the predicted position of the target vehicle in each second processed video frame based on the movement trajectory; comparing the predicted position of the target vehicle in each second processed video frame with the position of the vehicle detected in each second processed video frame. If the position difference is less than the preset value, it is determined that the detected vehicle is the same vehicle as the target vehicle.

[0094] Exemplarily, the first position information refers to the position of the target vehicle in the first processed video frame, such as bounding box coordinates. By analyzing the change of the position of the target vehicle in each first processed video frame over time, the movement trajectory of the target vehicle can be constructed. This movement trajectory refers to the position sequence of the target vehicle in consecutive frames, reflecting the movement path and speed of the target vehicle.

[0095] Exemplarily, based on the historical position data of the target vehicle, various algorithms such as Kalman filters, particle filters, etc. can be used to predict its position in subsequent frames. The above algorithms all consider factors such as the speed and direction of the vehicle and can provide relatively accurate predicted positions. Thus, for each second processed video frame, a predicted position will be generated. Further, in each second processed video frame, the vehicle toll type recognition system will attempt to re-detect all vehicles, record their positions, and compare the predicted position with the positions of all vehicles detected in the current frame. If the difference between the position of a detected vehicle and the predicted position is less than a preset value, then this detected vehicle is considered to be the target vehicle to be tracked. Among them, the preset value is a standard for determining whether two positions are close enough to be considered the same object. In practical applications, the preset value can be adjusted according to specific situations such as the speed of the vehicle, the angle of the camera, etc. to achieve the best tracking effect.

[0096] S404. If so, crop each second processed video frame based on the second position information of the target vehicle in each second processed video frame to obtain multiple target vehicle images.

[0097] Exemplarily, if the vehicle detected in the second processed video frame is the same vehicle as the target vehicle, record the position information of the vehicle in each second processed video frame, that is, the second position information, and crop each second processed video frame based on the second position information of the target vehicle in each second processed video frame to obtain multiple target vehicle images containing the target vehicle. The multiple target vehicle images respectively correspond to the same vehicle in different frames.

[0098] S405. Perform feature recognition based on the multiple target vehicle images to determine the first feature information of the target vehicle.

[0099] Optionally, by comparing and fusing the recognition results of the same vehicle in multiple frames of images, correcting the deviation in the recognition results, misjudgment can be reduced, and the stability and accuracy of the recognition of the first feature information can be improved. Figure 5 The flowchart shows another method for determining the first feature information of the target vehicle provided by the embodiment of the present application. Refer to Figure 5 As shown, the above step S405 performs feature recognition based on the multiple target vehicle images to determine the first feature information of the target vehicle, which specifically includes the following steps:

[0100] S501. For each target vehicle image, determine the vehicle type of the target vehicle based on the target vehicle image. If the vehicle type is a truck, perform axle detection on the target vehicle to obtain an axle detection result.

[0101] Exemplarily, for each target vehicle image, it is first necessary to determine the type of vehicle included in the target vehicle image, such as a sedan, a truck, a bus, a freight car, etc. Determining the vehicle type can be accomplished through a trained deep learning model, such as the YOLOv5 model. The model input is the target vehicle image, and the model output is the vehicle type. Once the vehicle type is determined to be a freight car, axle detection needs to be performed. Axle detection can use a specially designed algorithm or model to identify and count the number of axles of the freight car.

[0102] Exemplarily, the core of axle detection is to identify the position of the axles at the bottom of the freight car and count the number of axles. However, in this application, since the image acquisition device is installed at a certain angle during erection, the vehicle included in the obtained target vehicle image is also the vehicle image on the same side photographed at the corresponding angle. Therefore, in this application, the number of axles can be determined by identifying the number of tires in the target vehicle image. In an ideal state, if the number of tires on one side of the freight car in the target vehicle image is detected to be 5, the number of axles of the freight car is also 5.

[0103] S502. Perform license plate detection on the target vehicle image, determine the position information of the target license plate in the target vehicle image, and crop the target vehicle image according to the position information to obtain a license plate image. Then input the license plate image into a pre-trained license plate detection model, convert the license plate image into a time series feature sequence, perform bidirectional feature extraction processing on the time series feature sequence, determine the context information of each character in the time series feature sequence, and determine the probability distribution of each character according to the context information of each character, and determine the license plate number of the target license plate based on the probability distribution of each character.

[0104] Optionally, after determining the vehicle type, perform license plate detection on the target vehicle image, identify the position information of the target license plate in the target vehicle image. This position information is, for example, the coordinate information of the license plate frame corresponding to the target license plate, such as the upper left and lower right coordinates of the license plate frame. Then crop the target vehicle image according to the position information of the target license plate in the target vehicle image, extract the partial image of the target vehicle image that contains the target license plate to obtain a license plate image. Specifically, the license plate image can be cropped from the target vehicle image through bounding box recognition or region of interest (ROI) extraction method, and the license plate image is input into a pre-trained license plate detection model, convert the license plate image into a time series feature sequence, and perform bidirectional feature extraction (such as using bidirectional LSTM) on the obtained time series feature sequence to capture the context information between characters, and then calculate the probability distribution of each character, that is, a most likely character and a probability value will be given for each character at each position. Further, according to the probability distribution of each character, select the character combination with the greatest possibility to form the final license plate number.

[0105] In the embodiments of the present application, the license plate detection model is an improved object detection model. Among them, at least one full-dimensional dynamic convolution layer is added to the original object detection model, and the intermediate layer network in the original object detection model is replaced with a generalized feature pyramid network to obtain the license plate detection model. The full-dimensional dynamic convolution layer is used to convert the license plate image into a time series feature sequence, and the generalized feature pyramid network is used to perform bidirectional feature extraction processing on the time series feature sequence, determine the context information of each character in the time series feature sequence, and determine the probability distribution of each character according to the context information of each character.

[0106] Exemplarily, the object detection model is the YOLOv7 model, the intermediate layer network is the Neck network, and the generalized feature pyramid network is the GFPN network. The embodiments of the present application improve the license plate detection algorithm. Specifically, on the basis of the object detection model YOLOv7, by introducing the plug-and-play full-dimensional dynamic convolution (ODConv) and replacing the intermediate layer network (Neck network) in the original YOLOv7 with a generalized feature pyramid network (Generalized FPN, GFPN), the feature expression ability and feature fusion ability of the license plate detection model can be improved, so as to better adapt to the license plate detection task in complex scenarios. Among them, the full-dimensional dynamic convolution (ODConv) can dynamically generate convolution kernel weights according to the input features to adapt to the feature distributions of different inputs, thereby improving the expression ability and robustness of the convolutional network. Compared with the fixed convolution kernel, it can better capture features of different sizes and shapes, which helps to improve the detection accuracy and can ensure that the expression ability of the license plate detection model is improved without significantly increasing the amount of computation. The generalized feature pyramid network (GFPN network) can better perform multi-scale feature fusion and make full use of feature information at different levels. And by using the GFPN network to replace the traditional FPN network in the Neck network of the YOLOv7 model, the feature fusion ability of the YOLOv7 model can be enhanced, especially in multi-scale detection tasks.

[0107] Optionally, based on the improvement of the license plate recognition algorithm in the present application, the feature extraction layer of the improved YOLOv7 model is used to extract distinguishable spatial features from the input license plate image, such as edges, textures, etc., and higher-level semantic features are gradually extracted through multi-layer convolution and pooling operations to obtain a feature sequence. And based on this feature sequence, the feature vectors at each time step are processed, considering both past and future information, to capture the context dependencies between characters, so as to better understand the structure and semantics of the character sequence and obtain the license plate number contained in the license plate image.

[0108] S503. Use the vehicle type, axle detection result, and license plate number as the initial feature information corresponding to the target vehicle image, and compare and fuse the initial feature information corresponding to each target vehicle image to determine the first feature information of the target vehicle.

[0109] Exemplarily, since multiple frames of images may involve the same target vehicle, it is necessary to compare and fuse the initial feature information in these different frames to eliminate errors in single-frame recognition and improve the overall recognition accuracy and stability. Specifically, for the license plate number, the most reliable license plate number can be confirmed by comparing the recognition results of multiple frames. For the vehicle type and the number of axles, their accuracy can also be verified through consistency checks of multiple-frame data.

[0110] Based on this, key feature information about a specific target vehicle is extracted and integrated from a series of video frames, thereby providing a more accurate and reliable vehicle recognition result. This method not only improves the accuracy of single-frame recognition but also further enhances the robustness of the system through the fusion of cross-frame information.

[0111] Figure 6 The flowchart shows a method for determining the attribute information of a target vehicle provided by an embodiment of the present application. Refer to Figure 6 As shown, in step S304 above, according to the lane number of the target lane and the preset trigger line, the first feature information and multiple second feature information are fused and matched to determine the attribute information of the target vehicle, which specifically includes the following steps:

[0112] S601. According to the position information of the preset trigger line and the lane number, perform feature matching on the first feature information and each second feature information to obtain the target second feature information that matches the first feature information.

[0113] Among them, the position information of the preset trigger line includes the coordinate range of two trigger points on the first coordinate axis and the pixel range on the second coordinate axis. The first coordinate axis is the X-axis along the road direction, and the second coordinate axis is the Y-axis perpendicular to the road direction. In the embodiment of the present application, the camera calibration parameters are used to convert the pixel coordinates in the image into the actual road coordinate system, and the relative position of the center point or the bottom edge of the vehicle bounding box and the trigger line is compared to determine whether the current position of the vehicle crosses the preset trigger line.

[0114] It should be noted that according to the laws of our country, the maximum speed of vehicles on the highway is 120 km / h. Calculated at 20% over the speed limit, when the vehicle speed is 144 km / h, the distance traveled per second is 144×1000 / 3600 = 38 meters, that is, 3.8 meters are traveled every 100 milliseconds. The average time-consuming of the vehicle recognition algorithm is 20 milliseconds, the average time-consuming of the license plate recognition algorithm is 40 milliseconds, and the data transmission frequency of the radar is 100 milliseconds. According to the above parameter analysis, the interval range of the preset trigger line must be able to accommodate the target to travel within this interval for more than 100 milliseconds. After calculation, when this interval is about 4 meters, when each vehicle travels through this interval at a speed of 144 km / h, the video algorithm can identify it twice, and the millimeter-wave radar can identify it once. For vehicles with a speed exceeding 144 km / h, the video can still ensure one identification.

[0115] As a possible implementation manner, Figure 7 FIG. shows a schematic flow chart of a method for determining target second feature information provided by an embodiment of the present application. Refer to Figure 7 As shown, the above step S601 specifically includes the following steps:

[0116] S701. Obtain the vehicle information of each vehicle from the second feature information, perform coordinate transformation on the vehicle information of each vehicle, and determine whether the transformed coordinates corresponding to the second feature information meet the valid range condition. If so, use the second feature information as an available feature information.

[0117] Exemplarily, extract the relevant information of all vehicles from the radar data. Since the radar data is usually represented by the actual physical distance (meters), and the video frame uses pixel coordinates, the camera calibration parameters can be used to convert the coordinates in the radar data to the image coordinate system corresponding to the video frame. For each transformed coordinate, check whether it meets the preset valid range condition. Among them, the valid range condition is that the vehicle is located within the preset coordinate range, and the preset coordinate range is determined based on the position information of the preset trigger line.

[0118] Exemplarily, the valid range check based on the valid range condition includes X coordinate judgment and Y coordinate judgment. Among them, the X coordinate judgment means that two-thirds of the body width (X2 - X1) of the target vehicle must be within the X coordinate range of the two trigger points, that is, it means that if most (at least two-thirds) of the vehicle is within the X coordinate range defined by the trigger line, it is considered that the vehicle may be in the target lane. The Y coordinate judgment means that the center point (Y2 - Y1) / 2 of the target vehicle should be within ±50 pixels of the trigger point Y coordinate, that is, it means that if the center point of the vehicle is within this range, it is considered that it is more likely to be the target vehicle. On this basis, for the vehicles that meet the above valid range conditions, retain their second feature information as available feature information.

[0119] S702. Screen at least one candidate feature information belonging to the lane number from multiple available feature information.

[0120] Exemplarily, screen the target vehicle information belonging to a specific lane number from all available feature information. For example, if the target road has a total of four lanes and the camera captures that the target vehicle is driving in the third lane, then the specific lane number is 3. Among the multiple available feature information, there may be feature information of the four lanes. At this time, the feature information with the lane number 3 can be screened out from the multiple available feature information and used as the candidate feature information.

[0121] S703. Obtain the body width and the center point coordinates of the target vehicle from the first feature information, and determine whether the body width and the center point coordinates of the target vehicle meet the effective range conditions. If so, determine that the target vehicle is in the target lane, and match the first feature information with each candidate feature information to obtain the target second feature information that matches the first feature information.

[0122] Exemplarily, combine the first feature information (from video frame analysis) and the candidate feature information (filtered radar data) to further confirm the target vehicle and complete the feature matching. Specifically, verify again whether the body width of the target vehicle (X2 - X1)*2 / 3 is within the preset X coordinate range, and whether the center point of the target vehicle (Y2 - Y1) / 2 is within the range of the trigger point Y coordinate plus or minus 50 pixels. If the above conditions are all met, it can be confirmed that the target vehicle is indeed in the target lane, and the first feature information is compared with each candidate feature information. Specifically, the matching can be performed by evaluating the matching degree between the two. When there are multiple targets that meet the conditions, the vehicle with the closest Y coordinate can be selected as the final matching result, and finally the best target second feature information that matches the first feature information is obtained.

[0123] Furthermore, to ensure accuracy, it is also necessary to synchronize the time of the camera and the radar. The synchronization process includes time synchronization and data matching processing. Time synchronization means ensuring that the timestamps of the camera and the radar are the same or very close (for example, the time difference does not exceed 60 ms). If the time difference exceeds 60 ms, wait for the next round of data. And data matching means extracting all the target information in the radar information that is the same as the preset lane number, and further filtering the Y coordinate distance, and selecting the target vehicle with Y belonging to the range of the radar trigger distance plus or minus 2 meters. If there are multiple targets that meet the conditions, the vehicle with the closest Y coordinate is selected as the target vehicle.

[0124] S602. Fuse the first feature information and the target second feature information to determine the attribute information of the target vehicle.

[0125] Exemplarily, the first feature information is combined with the target second feature information to confirm and supplement the missing information. For example, if the vehicle type is missing in the first feature information, it can be obtained from the second feature information, and vice versa. In addition, if there are conflicting or inconsistent information, such as the first feature information shows a sedan, while the second feature information shows a truck, further verification or correction using additional data sources is required. Finally, the complete attribute information of the target vehicle is obtained, including but not limited to vehicle type, license plate number, speed, driving direction, number of axles, lane number where the vehicle is located, etc.

[0126] Exemplarily, the preset trigger line is defined as the X-axis coordinate range of [100, 150] meters and the Y-axis pixel range of [200, 300] pixels. Suppose there is a truck passing through the toll gate of a highway, and the toll gate is set with a preset trigger line for automatic vehicle information recognition. Through license plate recognition, it is known that this is a blue truck with the license plate number "ABCD1234". The radar data shows that the speed of the vehicle is 60 km / h, it is located in the third lane, and the estimated number of axles is 4. When it is detected that the truck has crossed the preset trigger line and is within the third lane, then the first feature information obtained from license plate recognition is matched with the second feature information in the radar data, and it is found that their timestamps and lane numbers are consistent. Therefore, it is considered a valid match, and the first feature information (vehicle type is truck, license plate number is ABCD1234) is fused with the second feature information (speed is 60 km / h, number of axles is 4) to form the complete vehicle attribute information.

[0127] Based on this, the most compliant target vehicle is effectively screened out from complex multi-source data, and accurate feature matching is completed to obtain the complete attribute information of the target vehicle. This not only improves the accuracy of vehicle detection and recognition but also provides reliable data support for subsequent operations such as toll type determination. And in this process, the advantages of video frame analysis and radar data are combined, making full use of the advantages of different sensors to achieve more comprehensive and accurate vehicle monitoring and management.

[0128] As a possible implementation, in step S303 above, protocol analysis is performed on the radar data to determine multiple second feature information, including: preprocessing the radar data to obtain the processed radar data, and parsing the processed radar data according to the radar communication protocol to extract effective vehicle information from the processed radar data; determining multiple second feature information according to the effective vehicle information.

[0129] Among them, the second feature information at least includes vehicle length, lane number of the lane where the vehicle is located, and driving speed.

[0130] Exemplarily, before performing protocol parsing on radar data, in order to ensure the quality and consistency of the radar data, the radar data may be preprocessed, such as removing noise in the radar data through a filtering algorithm, correcting the radar data according to specific parameters of the radar device, and, if necessary, fusing it with video frames or other sensor data to ensure that the timestamp of the radar data is consistent with other data sources.

[0131] Furthermore, the communication protocol format used by the radar equipment is determined. Radar data usually contains multiple fields. When parsing the radar data, the position and meaning of each field is identified according to the communication protocol format, so as to effectively extract valid information related to the vehicle, such as vehicle length, lane number of the lane, driving speed, etc., and use the valid vehicle information as the second feature information.

[0132] Based on this, by parsing the radar data protocol, not only the utilization efficiency of radar data is improved, but also a solid foundation is provided for subsequent vehicle detection, tracking and feature fusion. Further, by combining video frame analysis and other sensor data, more comprehensive and accurate vehicle monitoring can be achieved.

[0133] As a possible implementation method, the above step S305 determines the charging type of the target vehicle according to the attribute information of the target vehicle, including: determining the charging type of the target vehicle according to the attribute information of the target vehicle and the preset judgment criteria. Among them, the preset judgment criteria define the charging standards that should be used for different types of vehicles, usually referring to a set of clear rules or standards for determining the charging type according to the different attributes of the vehicle, and these criteria are usually formulated by relevant traffic management departments or operating units, and will take into account various factors, such as the degree of wear and tear of the vehicle on the road, environmental protection requirements, etc. The specific classification of vehicle types is shown in Table 1 below:

[0134] Table 1 Description of vehicle type classification

[0135]

[0136] Exemplarily, based on the vehicle type classification details shown in Table 1 above, the preset judgment criteria include: classification by vehicle type: small cars, medium-sized cars, large cars, etc. have different charging standards; classification by the number of axles: trucks are charged different fees according to the number of axles (such as two axles, three axles, four axles and above); classification by vehicle purpose: military vehicles and emergency service vehicles may enjoy the right to free passage, and new energy vehicles may enjoy discounts; classification by road sections and time periods: certain road sections or peak hours may have additional surcharges.

[0137] As a possible implementation, refer to Figure 8As shown, after determining the attribute information of the target vehicle and the corresponding toll type, these information can be stored in the database, sent to the highway toll system platform, and it is judged whether the send return value is successful according to the sending situation. If the sending fails, the information is marked for rollback sending.

[0138] Exemplarily, the present application monitors the total number of data and the date situation in the database at all times by establishing a database monitoring thread and makes corresponding processing, such as clearing the data stored for more than the maximum number of days, clearing the data exceeding the preset total number, and resending the marked data that has not been sent successfully every 10 minutes, etc.

[0139] Based on this, not only the automation level of the highway toll system platform is improved, the possibility of human intervention is reduced, but also the fairness and accuracy of toll collection can be ensured. At the same time, flexible adjustment of the discrimination criteria can also quickly respond to the needs of policy changes or special situations.

[0140] The embodiment of the present application also provides an edge device 900, as Figure 9 shown, which is a schematic structural diagram of the edge device 900 provided by the embodiment of the present application, including: a processor 901, a memory 902, and optionally, a bus 903 can also be included. The memory 902 stores machine-readable instructions executable by the processor 901. When the edge device 900 runs, the processor 901 communicates with the memory 902 through the bus 903, and when the machine-readable instructions are executed by the processor 901, the steps of the vehicle toll type recognition method described in any one of the above are executed.

[0141] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the vehicle toll type recognition method described in any one of the above are executed.

[0142] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, which will not be repeated in the present application. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, and the indirect coupling or communication connection of the devices or modules may be in electrical, mechanical or other forms.

[0143] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0144] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application.

Claims

1. A vehicle toll type recognition method, characterized in that, An edge device applied to a vehicle toll type identification system. The vehicle toll type identification system includes the edge device, a plurality of radar devices arranged in the area where the vehicle toll station is located, and a plurality of image acquisition devices. Each image acquisition device is respectively used to acquire video frames on one lane in the area where the vehicle toll station is located. The method includes: Obtain vehicle driving data, where the vehicle driving data includes: radar data of vehicles driving on each lane collected by the radar devices, and a plurality of video frames of each vehicle driving on each lane collected by the image acquisition devices; Based on the plurality of video frames, perform vehicle identification and tracking to determine first feature information of a target vehicle driving on a target lane; Perform protocol parsing on the radar data to determine a plurality of second feature information, and each of the second feature information is respectively used to indicate the feature information of each vehicle driving on each lane; According to the lane number of the target lane and a preset trigger line, perform fusion matching on the first feature information and the plurality of second feature information to determine the attribute information of the target vehicle. The preset trigger line is a boundary line set on the target road for triggering the identification of the vehicle toll type, and the attribute information includes vehicle type, license plate number, and number of axles; According to the attribute information of the target vehicle, determine the toll type of the target vehicle.

2. The method according to claim 1, wherein The step of performing vehicle identification and tracking based on the plurality of video frames to determine first feature information of a target vehicle driving on a target lane includes: Perform preprocessing on each of the video frames to obtain a plurality of processed video frames; Based on each of the first processed video frames, perform vehicle detection, obtain first position information of the target vehicle in each of the first processed video frames, and assign a unique identifier to the target vehicle when the target vehicle is first detected. The first processed video frame is a video frame collected before the target vehicle enters the trigger area where the preset trigger line is located among the plurality of processed video frames; Based on the unique identifier of the target vehicle, perform multi-target tracking on the target vehicle to determine whether the vehicles detected in each of the second processed video frames are the same vehicle as the target vehicle. The second processed video frame is a video frame collected after the target vehicle enters the trigger area where the preset trigger line is located among the plurality of processed video frames; If so, based on the second position information of the target vehicle in each of the second processed video frames, crop each of the second processed video frames to obtain a plurality of target vehicle images; Based on the plurality of target vehicle images, perform feature recognition to determine the first feature information of the target vehicle.

3. The method according to claim 2, wherein The step of performing multi-target tracking on the target vehicle based on the unique identifier of the target vehicle to determine whether the vehicles detected in each of the second processed video frames are the same vehicle as the target vehicle includes: According to the unique identifier of the target vehicle and the first position information of the target vehicle in each of the first processed video frames, determine the movement trajectory of the target vehicle, and predict the predicted position of the target vehicle in each of the second processed video frames based on the movement trajectory; Compare the predicted position of the target vehicle in each second-processed video frame with the position of the vehicle detected in each second-processed video frame. If the position difference is less than a preset value, it is determined that the detected vehicle and the target vehicle are the same vehicle.

4. The method according to claim 2, wherein Performing feature recognition based on multiple target vehicle images to determine first feature information of the target vehicle, including: For each of the target vehicle images, determine the vehicle type of the target vehicle based on the target vehicle image. If the vehicle type is a truck, perform axle detection on the target vehicle to obtain an axle detection result; Perform license plate detection on the target vehicle image to determine the position information of the target license plate in the target vehicle image, crop the target vehicle image according to the position information to obtain a license plate image, input the license plate image into a pre-trained license plate detection model, convert the license plate image into a time series feature sequence, perform two-way feature extraction processing on the time series feature sequence, determine the context information of each character in the time series feature sequence, determine the probability distribution of each character according to the context information of each character, and determine the license plate number of the target license plate based on the probability distribution of each character; Use the vehicle type, axle detection result, and license plate number as the initial feature information corresponding to the target vehicle image; Compare and fuse the initial feature information corresponding to each of the target vehicle images to determine the first feature information of the target vehicle.

5. The method according to claim 4, wherein The license plate detection model is an improved object detection model. Specifically, at least one full-dimensional dynamic convolution layer is added to the original object detection model, and the intermediate layer network in the original object detection model is replaced with a generalized feature pyramid network to obtain the license plate detection model; The full-dimensional dynamic convolution layer is used to convert the license plate image into a time series feature sequence; The generalized feature pyramid network is used to perform two-way feature extraction processing on the time series feature sequence, determine the context information of each character in the time series feature sequence, and determine the probability distribution of each character according to the context information of each character.

6. The method according to claim 1, wherein Fusing and matching the first feature information and multiple second feature information according to the lane number of the target lane and a preset trigger line to determine the attribute information of the target vehicle, including: Perform feature matching on the first feature information and each of the second feature information according to the position information of the preset trigger line and the lane number to obtain target second feature information that matches the first feature information. The position information of the preset trigger line includes the coordinate range of two trigger points on the first coordinate axis and the pixel range on the second coordinate axis; Fuse the first feature information and the target second feature information to determine the attribute information of the target vehicle.

7. The method according to claim 6, wherein Performing feature matching on the first feature information and each of the second feature information according to the position information of the preset trigger line and the lane number to obtain target second feature information that matches the first feature information, including: Obtain the vehicle information of each vehicle from each of the second feature information, perform coordinate conversion on the vehicle information of each vehicle, and determine whether the converted coordinates corresponding to each of the second feature information meet the valid range condition. If so, use the second feature information as an available feature information, where the valid range condition is that the vehicle is located within a preset coordinate range, and the preset coordinate range is determined based on the position information of the preset trigger line; Screen out at least one candidate feature information belonging to the lane number from the multiple available feature information; Obtain the body width and center point coordinates of the target vehicle from the first feature information, and determine whether the body width and center point coordinates of the target vehicle meet the valid range condition. If so, determine that the target vehicle is in the target lane, and match the first feature information with each of the candidate feature information to obtain the target second feature information that matches the first feature information.

8. The method according to claim 1, characterized in that The protocol analysis of the radar data to determine a plurality of second feature information includes: Preprocess the radar data to obtain processed radar data, and parse the processed radar data according to the radar communication protocol, and extract valid vehicle information from the processed radar data; Determine a plurality of the second feature information according to the valid vehicle information, and the second feature information at least includes vehicle length, lane number of the lane where the vehicle is located, and driving speed.

9. The method according to claim 1, characterized in that, The determining the toll type of the target vehicle according to the attribute information of the target vehicle includes: Determine the toll type of the target vehicle according to the attribute information of the target vehicle and a preset discrimination criterion, and the preset discrimination criterion is used to indicate the corresponding relationship between the toll type and the vehicle attribute information.

10. A vehicle toll type identification system, characterized in that, The vehicle toll type recognition system includes edge devices, a plurality of radar devices arranged in the area where the vehicle toll station is located, and a plurality of image acquisition devices, where each image acquisition device is respectively used to acquire video frames on one lane in the area where the vehicle toll station is located; The edge device is used to execute the steps of the vehicle toll type recognition method according to any one of claims 1 to 9.

Images