An intelligent vehicle toll audit system based on deep learning

Abstract

This invention discloses a deep learning-based intelligent vehicle toll audit system, belonging to the field of toll audit technology. It includes a feature extraction module, a trajectory prediction module, a matching analysis module, an evidence collection module, and an audit processing module. The feature extraction module extracts features from gantry images to generate vehicle passage data packets; the trajectory prediction module uses a convolutional neural network for behavior recognition and predicts vehicle trajectories; the matching analysis module matches trajectory codes with payment data and assesses risk; the evidence collection module generates an audit evidence sequence based on risk scores; and the audit processing module calculates the outstanding amount and generates a collection notice. This invention achieves intelligent recognition and trajectory prediction of vehicle behavior through deep learning and multimodal data fusion technology, improving audit accuracy; it reduces the system's false alarm rate by employing feature matching and risk assessment mechanisms; and its evidence chain-based audit processing method ensures the reliability and traceability of audit results.

Description

Technical Field

[0001] This invention relates to the field of toll collection and auditing technology, and in particular to an intelligent toll collection and auditing system for vehicles based on deep learning. Background Technology

[0002] With the nationwide rollout of the unified toll collection system for expressways, the road network environment is becoming increasingly complex, exacerbating toll evasion. Traditional auditing methods, when faced with complex road networks and diverse toll evasion behaviors, have revealed problems such as fragmented audit structures, difficulty in identifying toll evasion types, challenges in verifying suspected vehicles, and inaccurate calculation of evaded toll amounts. This not only leads to high labor costs and inefficient verification work but also fails to fundamentally curb toll evasion.

[0003] Currently, the industry primarily relies on manual review of toll collection data at entrances and exits, examination of images or videos captured by traffic cameras, and the use of self-developed or third-party toll evasion detection algorithms. However, these methods suffer from inefficiencies, high manpower requirements, and generally low accuracy. In particular, fixed algorithms struggle to cope with constantly evolving toll evasion behaviors, leading to a gradual decline in algorithm accuracy. Furthermore, the processes of obtaining evidence, verifying, calculating amounts, and submitting work orders for toll evasion data also face numerous challenges. Summary of the Invention

[0004] In view of the problems existing in the audit system in terms of data collection, behavior recognition, evidence chain construction and amount calculation, this invention is proposed.

[0005] Therefore, the problem to be solved by this invention is how to achieve intelligent identification of toll evasion behavior, automatic collection of evidence, and intelligent processing of the audit process through deep learning and big data analysis technology.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a deep learning-based intelligent vehicle toll audit system, comprising: a feature extraction module for extracting vehicle feature parameters from gantry images to form a vehicle toll data packet; a trajectory prediction module for extracting spatiotemporal features from the vehicle toll data packet, using a convolutional neural network for behavior recognition and classification, and training a vehicle trajectory prediction model based on the behavior classification results to output a trajectory code; a matching analysis module for matching the trajectory code with payment data, calculating the path matching degree, training a multilayer perceptron anomaly detection model based on the path matching degree, and generating an audit risk score; an evidence collection module for filtering vehicle toll images based on the audit risk score, extracting vehicle toll information, and generating an audit evidence sequence; and an audit processing module for calculating the outstanding amount based on the audit evidence sequence and generating an audit collection notice.

[0008] As a preferred embodiment of the intelligent vehicle toll audit system based on deep learning described in this invention, the workflow of the feature extraction module is as follows: A deep learning object detection network is used to detect vehicles in the gantry image to obtain a vehicle region image; feature segmentation is performed on the vehicle region image to extract the license plate region, body region, and wheel region; an optical character recognition model is used to extract the license plate number from the license plate region, a color segmentation algorithm is used to extract the body color from the body region, and an image classification network is used to extract the vehicle model identification code from the body region; the body length is calculated based on the body region bounding box coordinates, and the number of axles is extracted from the wheel region image to obtain vehicle feature parameters; the vehicle feature parameters are combined with gantry time and geographical coordinates to generate a vehicle toll data package.

[0009] As a preferred embodiment of the intelligent vehicle toll audit system based on deep learning described in this invention, the vehicle characteristic parameters include license plate number, vehicle body color, vehicle type identification code, vehicle body length, and number of axles; the gantry time includes the passage period and the collection time.

[0010] As a preferred embodiment of the intelligent vehicle toll audit system based on deep learning described in this invention, the trajectory prediction module's workflow is as follows: A vehicle feature matrix is ​​constructed based on vehicle feature parameters; a time series matrix is ​​constructed based on gantry time; and a spatial location matrix is ​​constructed based on geographic coordinates. Tensor concatenation operations are performed on the vehicle feature matrix, time series matrix, and spatial location matrix to obtain a multidimensional feature tensor. The multidimensional feature tensor is separated into vehicle feature sub-tensors, time feature sub-tensors, and spatial feature sub-tensors. These feature sub-tensors are input into a multi-layer bidirectional convolutional neural network for feature extraction to obtain a behavior feature map. A retrieval index is constructed based on the license plate number. The vehicle behavior probability distribution is calculated based on the behavior feature map, and the behavior category label corresponding to the maximum probability is extracted. The behavior category label and vehicle feature parameters are input into the trajectory prediction model, and a trajectory point prediction vector is output. A trajectory point coordinate sequence is generated based on the trajectory point prediction vector, and the trajectory point coordinate sequence is converted into a trajectory code using a vector quantization encoding method.

[0011] As a preferred embodiment of the intelligent vehicle toll fee auditing system based on deep learning described in this invention, the method of converting the trajectory point coordinate sequence into trajectory code using vector quantization encoding includes the following steps: calculating the covariance matrix of the trajectory point prediction vector using principal component analysis to obtain eigenvalues ​​and eigenvectors; determining the number of principal components based on the cumulative contribution rate of the eigenvalues; projecting the trajectory point prediction vector onto the eigenvector space to generate a dimensionality-reduced feature matrix; standardizing the dimensionality-reduced feature matrix to obtain a normalized feature sequence and generating a feature transformation parameter matrix; and inputting the normalized feature sequence into the K-means clustering algorithm. The algorithm first obtains the cluster center vector set, performs Schmitt orthogonalization on the cluster center vector set, and generates an initial codeword matrix. It then calculates the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix, executes the nearest neighbor search algorithm to obtain the codeword index corresponding to the minimum distance, and generates a quantized index sequence. The quantized index sequence is multiplied by the initial codeword matrix to obtain the reconstructed feature sequence, and the reconstruction error is calculated to construct the codebook optimization objective function. The stochastic gradient descent algorithm is used to optimize the codebook optimization objective function, update the initial codeword matrix, and obtain the optimized codeword matrix. Finally, the normalized feature sequence and the optimized codeword matrix are matched for nearest neighbor matching to output the trajectory code.

[0012] As a preferred embodiment of the intelligent vehicle toll audit system based on deep learning described in this invention, the workflow of the matching analysis module is as follows: The payment amount and toll station number are extracted from the payment data to generate a payment path sequence. The payment path sequence is converted into a payment path code using vector quantization encoding. A path alignment matrix is ​​constructed, and the trajectory code and payment path code are mapped using feature mapping. The path matching degree is calculated using cosine distance. Vehicle feature parameters and gantry time corresponding to the path matching degree are extracted. The path matching degree, vehicle feature parameters, and gantry time are combined to form a training sample matrix. A multilayer perceptron network structure is constructed, comprising an input layer, three hidden layers with Dropout layers, and an output layer. The hidden layers use the LeakyReLU activation function, and the output layer uses a normalized activation function. The training sample matrix is ​​input into the multilayer perceptron network, cross-entropy is used as the loss function, and the Adam optimizer is used to optimize the multilayer perceptron network parameters. Vehicle toll data packets are input into the trained multilayer perceptron network, and the network output value is obtained to generate an audit risk score.

[0013] As a preferred embodiment of the intelligent vehicle toll fee audit system based on deep learning described in this invention, the workflow of the evidence collection module is as follows: Extracting the license plate number and passage time corresponding to the audit risk score; retrieving the gantry image sequence of the license plate number within the passage time; performing feature enhancement processing on the gantry image sequence to extract license plate features, vehicle body features, and axle features from the gantry image sequence, generating a feature vector group; inputting the feature vector group into the similarity calculation module, using Euclidean distance to calculate the feature similarity between images, generating an image matching matrix; calculating the feature similarity of adjacent images based on the image matching matrix, combining adjacent gantry images with feature similarity higher than the matching threshold to form an image evidence chain; extracting the vehicle feature parameters and gantry time corresponding to the image evidence chain, and generating an audit evidence sequence in chronological order.

[0014] As a preferred embodiment of the intelligent vehicle toll audit system based on deep learning described in this invention, the workflow of the audit processing module is as follows: Extracting gantry time and geographical coordinates from the audit evidence sequence; constructing the vehicle travel path based on the geographical coordinates; calculating the actual mileage of the vehicle; determining the vehicle toll rate based on the vehicle identification number and the number of axles; multiplying the actual mileage of the vehicle by the vehicle toll rate to calculate the amount due; extracting the actual amount paid from the payment data; determining the difference between the amount due and the actual amount paid as the outstanding amount; calculating the payment deadline based on the date the audit result is generated; and combining the license plate number, outstanding amount, payment deadline, travel time, vehicle identification number, actual mileage of the vehicle, and number of axles to generate an audit collection notice.

[0015] The beneficial effects of this invention are as follows: The feature extraction module accurately extracts vehicle feature parameters, enabling automated collection and standardized processing of vehicle traffic data; the trajectory prediction module uses deep learning methods to classify vehicle behavior and predict trajectories, improving the accuracy of trajectory recognition; the matching analysis module intelligently matches trajectory data with payment data and uses a multilayer perceptron for anomaly detection, effectively identifying vehicles with a high risk of toll evasion; and the evidence collection and audit processing modules automatically generate complete audit evidence chains and collection notices, improving audit efficiency and reducing labor costs. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a system architecture diagram of a deep learning-based intelligent vehicle toll audit system.

[0018] Figure 2 This is a flowchart of the feature extraction module of a deep learning-based intelligent vehicle toll fee audit system.

[0019] Figure 3 This is a flowchart of the trajectory prediction module of a deep learning-based intelligent vehicle toll fee audit system. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1

[0024] Reference Figures 1-3 This is the first embodiment of the present invention, which provides a deep learning-based intelligent vehicle toll audit system. The system architecture diagram is shown below. Figure 1 As shown, it includes the following functional modules:

[0025] The feature extraction module is used to extract vehicle feature parameters from the gantry image and form a vehicle passage data packet;

[0026] The trajectory prediction module is used to extract spatiotemporal features from vehicle traffic data packets, use convolutional neural networks for behavior recognition and classification, and train a vehicle trajectory prediction model based on the behavior classification results to output trajectory codes.

[0027] The matching analysis module is used to match trajectory codes with payment data, calculate path matching degree, train a multilayer perceptron anomaly detection model based on path matching degree, and generate an audit risk score.

[0028] The evidence collection module is used to filter vehicle passage images based on audit risk scores, extract vehicle passage information, and generate audit evidence sequences.

[0029] The audit processing module is used to calculate the amount of outstanding fees based on the sequence of audit evidence and generate an audit collection notice.

[0030] In practice, the flowchart of the feature extraction module is as follows: Figure 2 As shown, the process includes: using a deep learning object detection network to detect vehicles in gantry images and obtain vehicle region images; performing feature segmentation on the vehicle region images to extract license plate region, body region, and wheel region; using an optical character recognition model to extract the license plate number from the license plate region, using a color segmentation algorithm to extract the body color from the body region, using an image classification network to extract the vehicle model identification code from the body region, calculating the body length based on the body region bounding box coordinates, and extracting the number of axles from the wheel region image to obtain vehicle feature parameters; combining the vehicle feature parameters with gantry time and geographic coordinates to generate a vehicle passage data packet. The vehicle feature parameters include license plate number, body color, vehicle model identification code, body length, and number of axles; the gantry time includes the passage period and the data collection time.

[0031] In the feature extraction module, the deep learning object detection network adopts the YOLOv5 model, which achieves fast and accurate vehicle detection through pre-trained weights; the feature segmentation adopts the U-Net network structure to achieve accurate segmentation of each region of the vehicle; the optical character recognition model adopts the CRNN+CTC architecture to improve the accuracy of license plate recognition; the color segmentation algorithm is based on the HSV color space and the K-means clustering method to achieve stable extraction of vehicle body color; and the image classification network adopts the ResNet50 structure to provide reliable vehicle classification results.

[0032] In existing technologies, traditional trajectory prediction methods mainly rely on historical trajectory data for simple time-series predictions, failing to fully consider the impact of vehicle behavior characteristics on trajectory prediction. Furthermore, the prediction results are easily affected by data noise, resulting in poor prediction accuracy and robustness. In addition, existing methods have high computational complexity when processing long-sequence trajectory data, making it difficult to meet real-time prediction requirements.

[0033] To address the above problems, this invention designs a trajectory prediction module based on behavior recognition. The flowchart of the trajectory prediction module is as follows: Figure 3 As shown, the process includes: constructing a vehicle feature matrix based on vehicle feature parameters, constructing a time series matrix based on gantry time, and constructing a spatial location matrix based on geographic coordinates; performing tensor concatenation operations on the vehicle feature matrix, time series matrix, and spatial location matrix to obtain a multidimensional feature tensor; and separating the multidimensional feature tensor to obtain vehicle feature sub-tensors, time feature sub-tensors, and spatial feature sub-tensors, where each sub-tensor has dimensions of (batch_size, features, sequence_length), features are 16-dimensional vehicle features, 8-dimensional time features, and 4-dimensional spatial features, and sequence_length is uniformly set to 12.

[0034] Furthermore, the feature sub-tensors are input into a multi-layer bidirectional convolutional neural network for feature extraction to obtain a behavioral feature map. The process is as follows: First, the three feature sub-tensors are input into two one-dimensional convolutional layers for feature extraction, with a kernel size of 3, a stride of 1, a same padding method, and a ReLU activation function. Then, max pooling is performed on the convolutional feature map with a kernel size of 2. Next, the pooled feature map is fed into a bidirectional LSTM layer for temporal feature learning, with a hidden layer dimension of 128. Finally, the feature map output by the LSTM is dimensionality-reduced through a fully connected layer to obtain a 256-dimensional behavioral feature map.

[0035] Furthermore, a retrieval index is constructed based on the license plate number, and the probability distribution of vehicle behavior is calculated according to the behavior feature map. The behavior category label corresponding to the maximum probability is extracted. Specifically, cosine similarity is used to calculate the similarity score between the behavior feature map and each category feature in the pre-trained behavior template library. The similarity score is normalized to a probability distribution using the Softmax function, and the behavior category corresponding to the maximum probability is selected as the prediction result. The pre-trained behavior template library contains 10,000 labeled samples, covering 4 behavior categories, with at least 1,000 valid samples in each category.

[0036] In addition, the behavior category labels include normal behavior categories, violation categories, abnormal behavior categories, and special situation categories. For example, normal behavior categories include normal passage and compliant detours; violation categories include detouring to avoid tolls, paying in segments, vehicle type mismatch, obscured / altered / inconsistent license plates, exceeding the time limit for stopping, driving in the wrong direction, merging into other lanes and running toll stations, and frequent lane changes; abnormal behavior categories include equipment malfunction, missing data, duplicate billing, and pending confirmation; and special situation categories include emergency vehicles, green channel vehicles, official vehicles, and other vehicles eligible for free passage.

[0037] Furthermore, the behavior category labels and vehicle feature parameters are input into the trajectory prediction model, which includes bidirectional gated recurrent units and outputs trajectory point prediction vectors. The training process of the trajectory prediction model includes the following steps: one-hot encoding of the behavior category labels; conversion of the vehicle model identification code into a numerical vector; normalization of the vehicle length and axle count; concatenation of the processed feature data to form the input feature vector; construction of a bidirectional gated recurrent unit network, including an input layer, two bidirectional gated recurrent layers, a fully connected layer, and an output layer. The input layer dimension matches the feature vector dimension, and the number of hidden units in the bidirectional gated recurrent layer is 128; setting the time step to 12, expanding the input feature vector according to the time step, and inputting it into the forward and backward layers of the bidirectional gated recurrent unit network to generate bidirectional features. The sequence is processed by mapping the bidirectional feature sequence to a two-dimensional coordinate space through a fully connected layer and activating it with a hyperbolic tangent function to obtain the predicted coordinates of trajectory points. The mean squared error is used as the loss function to calculate the error between the predicted and actual trajectory point coordinates, and the Adam optimizer is used to update the network parameters. The bidirectional gated recurrent unit network is trained iteratively, and training is stopped when the validation set loss value converges or the preset number of iterations is reached. The trained network is then used as the trajectory prediction model. The new input feature vector is input into the trajectory prediction model, and the output trajectory point prediction vector contains the predicted coordinates of 12 consecutive time steps.

[0038] Furthermore, a trajectory point coordinate sequence is generated based on the trajectory point prediction vector, and the trajectory point coordinate sequence is converted into trajectory code using a vector quantization encoding method. In trajectory encoding, principal component analysis is used to calculate the covariance matrix of the trajectory point prediction vector to obtain eigenvalues ​​and eigenvectors; the number of principal components is determined based on the cumulative contribution rate of the eigenvalues ​​(the threshold for the cumulative contribution rate of eigenvalues ​​in this invention is set to 0.95), and the trajectory point prediction vector is projected onto the eigenvector space to generate a dimensionality-reduced feature matrix; the dimensionality-reduced feature matrix is ​​standardized to obtain a normalized feature sequence, and a feature transformation parameter matrix is ​​generated; the normalized feature sequence is input into a K-means clustering algorithm to obtain a cluster center vector group, and the cluster center vector group is subjected to Schmitt orthogonalization to generate an initial codeword matrix; the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix is ​​calculated, and a nearest neighbor search algorithm is executed to obtain the codeword index corresponding to the minimum distance, generating a quantized index sequence; the quantized index sequence is multiplied by the initial codeword matrix to obtain the reconstructed feature sequence, the reconstruction error is calculated, and a codebook optimization objective function is constructed, the specific formula of which is as follows:

[0039]

[0040] Where X is the normalized feature sequence, Q(X) is the quantized reconstructed sequence, and C is the initial codeword matrix. Let |||·||1 be the squared Euclidean distance, ||·||1 be the L1 norm, and λ be the regularization coefficient (set to 0.01). This objective function integrates PCA dimensionality reduction, K-means clustering, and vector quantization encoding into a unified optimization framework, through the reconstruction term... The coding quality is directly measured, and the regularization term λ||C||1 controls the sparsity of the codebook, achieving efficient compression while ensuring the ability to express trajectory features.

[0041] Furthermore, the stochastic gradient descent algorithm is used to optimize the codebook objective function, update the initial codeword matrix, and obtain the optimized codeword matrix; the normalized feature sequence is then matched with the optimized codeword matrix to perform nearest neighbor matching, and the trajectory code is output.

[0042] Preferably, the trajectory prediction module achieves effective fusion of vehicle features, temporal features, and spatial features through the construction and separation of multidimensional feature tensors; it uses a multi-layer bidirectional convolutional neural network to extract behavioral features, improving the accuracy of behavior recognition; it guides trajectory prediction based on behavior category labels, significantly improving prediction accuracy; and it achieves efficient compression and representation of trajectory data through vector quantization encoding.

[0043] In existing technologies, traditional path matching methods often use simple path overlap calculations without considering path temporal relationships and connectivity characteristics, which can easily lead to matching errors; and they lack effective risk assessment mechanisms, making it difficult to accurately identify abnormal traffic behavior.

[0044] To address the aforementioned issues, this invention proposes a path alignment-based matching analysis module, the workflow of which is as follows: The payment amount and toll station number are extracted from the payment data to generate a payment path sequence. This sequence is then converted into a payment path code using vector quantization encoding. A path alignment matrix is ​​constructed, and the trajectory code and payment path code are mapped using features. The path matching degree is calculated using cosine distance. Specifically, the number of rows in the path alignment matrix equals the dimension of the trajectory code, and the number of columns equals the dimension of the payment path code.

[0045] Furthermore, the temporal relationship between adjacent coding units in the trajectory coding is extracted to generate a trajectory temporal relationship vector. The connectivity between adjacent coding units in the payment path coding is extracted to generate a payment connectivity vector. The temporal relationship vector represents the normalized value of the time interval between adjacent trajectory points, and the connectivity vector represents the normalized value of the spatial distance between adjacent payment points. The dot product of the trajectory temporal relationship vector and the payment connectivity vector is calculated to obtain the path correlation degree, which ranges from [0,1]. The path correlation degree is then filled into the corresponding positions in the path alignment matrix. Based on the dynamic programming algorithm, the optimal alignment path is searched in the path alignment matrix, and the path correlation degree sequence on the optimal alignment path is extracted. The path correlation degree sequence is then subjected to min-max normalization to generate a path matching degree sequence. The mean of the path matching degree sequence is calculated to obtain the path matching degree.

[0046] Furthermore, vehicle feature parameters and gantry time corresponding to the path matching degree are extracted, and the path matching degree, vehicle feature parameters, and gantry time are combined to form a training sample matrix. A multilayer perceptron network structure is constructed, which includes an input layer, three hidden layers with dropout layers, and an output layer. The hidden layers use the LeakyReLU activation function, and the output layer uses the normalized activation function. The training sample matrix is ​​input into the multilayer perceptron network, and cross-entropy is used as the loss function. The Adam optimizer is used to optimize the parameters of the multilayer perceptron network. The vehicle passage data packets are input into the trained multilayer perceptron network, and the network output value is obtained to generate an audit risk score with a value range of [0,1]. A score greater than 0.8 is considered high risk.

[0047] In addition, in the multilayer perceptron network, the dropout rate of the Dropout layer is set to 0.3, the slope of the negative half-axis of the LeakyReLU activation function is 0.01, and the initial learning rate is set to 0.001.

[0048] Ideally, the matching analysis module achieves accurate matching between trajectory codes and payment path codes through the path alignment matrix; the introduction of temporal relationships and connectivity features improves the accuracy of matching; and a risk assessment model is established using a multilayer perceptron network to effectively identify abnormal passage behavior.

[0049] In existing technologies, traditional evidence collection methods often focus only on a single gantry image and lack systematic analysis of continuous passage records; moreover, the image quality is unstable, the feature extraction effect is poor, and it is difficult to construct a reliable chain of evidence.

[0050] To address the aforementioned issues, this invention proposes an evidence collection module based on feature matching. Its workflow is as follows: Extract the license plate number and passage time corresponding to the audit risk score; retrieve the gantry image sequence of the license plate number within the passage time; perform feature enhancement processing on the gantry image sequence, extracting license plate features, vehicle body features, and axle features to generate a feature vector group; input the feature vector group into a similarity calculation module, using Euclidean distance to calculate the feature similarity between images, generating an image matching matrix; calculate the feature similarity of adjacent images based on the image matching matrix, combining adjacent gantry images with feature similarity higher than the matching threshold to form an image evidence chain; extract the vehicle feature parameters and gantry time corresponding to the image evidence chain, generating an audit evidence sequence in chronological order.

[0051] In the evidence acquisition module, feature enhancement processing includes adaptive contrast equalization and Gaussian noise suppression to enhance image details. Adaptive contrast equalization uses the CLAHE algorithm, dividing the image into 8×8 equal-sized blocks, limiting the contrast threshold to 3.0, performing histogram equalization on each block, and eliminating artificial boundaries between blocks through bilinear interpolation. Gaussian noise suppression uses a bilateral filter with a spatial domain standard deviation of 75, a value domain standard deviation of 75, and a window size of 5, considering both spatial distance and grayscale differences to effectively suppress noise while preserving edge information.

[0052] Furthermore, the feature vector group extracts 1024-dimensional feature vectors using ResNet50; similarity calculation uses weighted Euclidean distance, assigning weight coefficients of 0.5, 0.3, and 0.2 to the license plate, vehicle body, and wheel axle features, respectively; the matching threshold is determined through ROC curve analysis and set to 0.85 to ensure matching accuracy while avoiding breaks in the evidence chain; the image evidence chain is constructed using a sliding window method with a window size of 5, a step size of 1, and a 50% overlap between windows. Adjacent windows share 2-3 frames of images to ensure the continuity and integrity of the evidence. The sliding window moves sequentially in time, performing feature matching on the image sequence within each window to ensure the continuity and integrity of the acquired evidence.

[0053] In practice, the audit processing module's workflow is as follows: It extracts the gantry time and geographical coordinates from the audit evidence sequence; constructs the vehicle's travel route based on the geographical coordinates; calculates the vehicle's actual mileage; determines the basic toll rate corresponding to each vehicle type according to the "Vehicle Classification for Toll Roads" (JT / T489-2019) standard; determines the vehicle toll rate based on the vehicle identification number and the number of axles, with vehicles having ≥6 axles having a 20% higher toll rate than the basic rate; and calculates the payable by multiplying the vehicle's actual mileage by the vehicle toll rate. The process involves extracting the actual amount paid from the payment data and determining the difference between the amount due and the actual amount paid as the outstanding amount. The payment period is calculated based on the date the audit results are generated, with the following rules: 15 days for outstanding amounts under 1000 yuan; 30 days for outstanding amounts between 1000 and 5000 yuan; and 45 days for outstanding amounts over 5000 yuan. A collection notice is then generated by combining the license plate number, outstanding amount, payment period, travel time, vehicle identification number, actual mileage, and number of axles.

[0054] In summary, this invention achieves automated collection and standardized processing of vehicle traffic data by accurately extracting vehicle feature parameters through a feature extraction module; improves the accuracy of trajectory recognition by classifying vehicle behavior and predicting trajectories using deep learning methods through a trajectory prediction module; effectively identifies vehicles with a high risk of toll evasion by intelligently matching trajectory data with payment data through a matching analysis module and using a multilayer perceptron for anomaly detection through a multilayer perceptron; and improves audit efficiency and reduces labor costs by automatically generating a complete audit evidence chain and a toll collection notice through an evidence collection module and an audit processing module.

[0055] Example 2

[0056] Reference Figures 1-3 This is the second embodiment of the present invention, which provides a vehicle toll fee intelligent auditing system based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0057] The research team conducted a three-month experimental verification on the highway network of a certain province. The experimental data came from actual data collected from 124 toll gantries in the province, involving approximately 6.8 million vehicle passage records, including 6.52 million normal passage records and 280,000 abnormal passage records. The experimental environment used an Intel Xeon Gold 6248R processor, 256GB of memory, and an NVIDIA A100 GPU.

[0058] In the feature extraction module, the research team used the YOLOv5s model to detect vehicles in gantry images. The model input size was set to 640×640 pixels, the batch size to 32, the initial learning rate to 0.01, and the training epochs to 100. License plate recognition employed an improved CRNN+CTC model, trained on 80,000 highway vehicle images. Vehicle type classification used a ResNet50 network, trained on a dataset containing 12 vehicle types.

[0059] In the trajectory prediction module, researchers constructed a behavior template library containing 8000 labeled samples, including 5600 normal behavior samples (70%), 1600 violation behavior samples (20%), 600 abnormal behavior samples (7.5%), and 200 special case samples (2.5%). The bidirectional gated recurrent unit network uses a 128-dimensional hidden layer, with a dropout rate of 0.5, a training batch size of 64, and a learning rate of 0.001.

[0060] As shown in Table 1, a comparative experiment was conducted between the present invention and existing deep learning methods, and the performance of the most widely used deep learning methods was selected for evaluation.

[0061] Table 1. Experimental Results of System Performance Comparison

[0062] Evaluation indicators Method of the present invention Deep learning methods Vehicle inspection accuracy rate (%) 94.6 93.2 Vehicle recognition accuracy (%) 93.1 91.7 Behavior recognition accuracy (%) 89.2 87.5 Trajectory prediction error (m) 12.3 14.2 Risk identification accuracy (%) 87.8 85.9 System processing latency (ms) 82 95 Memory usage (GB) 38 45

[0063] As can be seen from the comparison results in Table 1, the method of the present invention significantly improves system efficiency while maintaining high accuracy. In particular, it achieves optimizations of 13.7% and 15.6% in system processing latency and memory usage, respectively. This is mainly due to the improved feature extraction strategy and optimized network structure design.

[0064] In the evidence collection module experiments, the CLAHE algorithm was used for contrast equalization in feature enhancement processing. The image was divided into 8×8 blocks, and the contrast threshold was limited to 3.0. A bilateral filter was used for noise suppression, with both the spatial and value domain standard deviations set to 75. The signal-to-noise ratio of the processed image was improved by 6.2 dB. The image evidence chain was constructed using a sliding window method, with the best results achieved when the window size was set to 5 frames, resulting in an evidence integrity rate of 91.5%.

[0065] In actual abnormal passage data processing, the system successfully generated 252,000 audit and recovery notices, with a recovery accuracy rate of 90.3%. The system's average processing time was 2.8 seconds per notice, representing a 25% improvement in efficiency compared to existing deep learning methods. The distribution of recovered amounts showed that 45% were below 500 yuan, 35% were between 500 and 2000 yuan, 15% were between 2000 and 5000 yuan, and 5% were above 5000 yuan. These results demonstrate that the present invention has significant performance and efficiency advantages in practical applications.

[0066] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based intelligent vehicle toll audit system, characterized in that: include, The feature extraction module is used to extract vehicle feature parameters from the gantry image and form a vehicle passage data packet; The trajectory prediction module is used to extract spatiotemporal features from the vehicle traffic data packet, use a convolutional neural network for behavior recognition and classification, and train a vehicle trajectory prediction model based on the behavior classification results to output trajectory codes. The matching analysis module is used to match the trajectory code with the payment data, calculate the path matching degree, train a multilayer perceptron anomaly detection model based on the path matching degree, and generate an audit risk score. The evidence collection module is used to filter vehicle passage images based on the audit risk score, extract vehicle passage information, and generate an audit evidence sequence. The audit processing module is used to calculate the amount of outstanding fees based on the audit evidence sequence and generate an audit collection notice. The trajectory prediction module includes: Principal component analysis is used to calculate the covariance matrix of the trajectory point prediction vectors, and eigenvalues ​​and eigenvectors are obtained. The number of principal components is determined based on the cumulative contribution rate of the eigenvalues, and the trajectory point prediction vector is projected onto the feature vector space to generate a dimension-reduced feature matrix. The reduced feature matrix is ​​standardized to obtain a normalized feature sequence, and a feature transformation parameter matrix is ​​generated. The normalized feature sequence is input into the K-means clustering algorithm to obtain the cluster center vector group. The cluster center vector group is then subjected to Schmitt orthogonalization to generate the initial codeword matrix. Calculate the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix, execute the nearest neighbor search algorithm, obtain the codeword index corresponding to the minimum distance, and generate a quantized index sequence; Multiply the quantized index sequence by the initial codeword matrix to obtain the reconstructed feature sequence, calculate the reconstruction error, and construct the codebook optimization objective function; The codebook optimization objective function is optimized using the stochastic gradient descent algorithm, and the initial codeword matrix is ​​updated to obtain the optimized codeword matrix; The normalized feature sequence is matched with the optimized codeword matrix to perform nearest neighbor matching, and the trajectory code is output.

2. The intelligent vehicle toll audit system based on deep learning as described in claim 1, characterized in that: The workflow of the feature extraction module is as follows: A deep learning object detection network is used to detect vehicles in gantry images to obtain vehicle region images; The vehicle area image is segmented to extract the license plate area, vehicle body area, and wheel area; An optical character recognition model is used to extract the license plate number from the license plate area, a color segmentation algorithm is used to extract the body color from the body area, and an image classification network is used to extract the vehicle model identification code from the body area. The vehicle length is calculated based on the coordinates of the vehicle body region bounding box, and the number of axles is extracted from the wheel region image to obtain vehicle feature parameters; The vehicle characteristic parameters are combined with gantry time and geographic coordinates to generate a vehicle passage data packet.

3. The intelligent vehicle toll audit system based on deep learning as described in claim 2, characterized in that: The vehicle characteristic parameters include license plate number, body color, vehicle identification number, body length, and number of axles; the gantry time includes the passage period and the collection time.

4. The intelligent vehicle toll audit system based on deep learning as described in claim 1, characterized in that: The trajectory prediction module also includes: A vehicle feature matrix is ​​constructed based on vehicle feature parameters, a time series matrix is ​​constructed based on gantry time, and a spatial location matrix is ​​constructed based on geographic coordinates. Tensor concatenation operations are performed on the vehicle feature matrix, the time series matrix, and the spatial location matrix to obtain a multidimensional feature tensor. The multidimensional feature tensor is separated into vehicle feature sub-tensor, time feature sub-tensor, and spatial feature sub-tensor. The feature sub-tensors are then input into a multi-layer bidirectional convolutional neural network for feature extraction to obtain a behavior feature map. A retrieval index is constructed based on the license plate number, and the probability distribution of vehicle behavior is calculated based on the behavior feature map. The behavior category label corresponding to the maximum probability value is extracted. Input the behavior category label and the vehicle feature parameters into the trajectory prediction model, and output the trajectory point prediction vector; A trajectory point coordinate sequence is generated based on the trajectory point prediction vector, and the trajectory point coordinate sequence is converted into trajectory code using a vector quantization encoding method.

5. The intelligent vehicle toll audit system based on deep learning as described in claim 1, characterized in that: The workflow of the evidence collection module is as follows: Extract the license plate number and passage time corresponding to the audit risk score, and retrieve the gantry image sequence of the license plate number within the passage time. The gantry image sequence is subjected to feature enhancement processing to extract license plate features, vehicle body features, and wheel axle features from the gantry image sequence, and feature vector groups are generated. The feature vector group is input into the similarity calculation module, and the feature similarity between images is calculated using Euclidean distance to generate an image matching matrix; The feature similarity of adjacent images is calculated based on the image matching matrix, and adjacent gantry images with feature similarity higher than the matching threshold are combined to form an image evidence chain; Extract the vehicle feature parameters and gantry time corresponding to the image evidence chain, and generate an audit evidence sequence in chronological order.

6. The intelligent vehicle toll audit system based on deep learning as described in claim 1, characterized in that: The workflow of the audit processing module is as follows: Extract the gantry time and geographic coordinates from the audit evidence sequence, construct the vehicle travel path based on the geographic coordinates, and calculate the actual mileage of the vehicle. The vehicle toll rate is determined based on the vehicle identification number and the number of axles. The amount payable is calculated by multiplying the actual mileage of the vehicle by the vehicle toll rate. Extract the actual amount paid from the payment data, and determine the difference between the amount due and the actual amount paid as the amount owed; The payment period is calculated based on the date generated by the audit results; The system combines the license plate number, outstanding amount, payment deadline, permitted travel time, vehicle identification number, actual mileage, and number of axles to generate an audit and collection notice.

Images