Vehicle pass fee intelligent auditing system based on deep learning

Abstract

The invention discloses a vehicle pass fee intelligent auditing system based on deep learning, which relates to the technical field of charging auditing and comprises a feature extraction module, a track prediction module, a matching analysis module, an evidence acquisition module and an auditing processing module. The feature extraction module performs feature extraction on the portal frame image to generate a vehicle passing data packet; the trajectory prediction module adopts a convolutional neural network to perform behavior recognition and predict a vehicle trajectory; the matching analysis module matches the track code with the payment data and evaluates the risk; the evidence collection module generates an auditing evidence sequence according to the risk score; the auditing processing module is responsible for calculating the arrearage amount and generating a reclaiming notice. According to the invention, through deep learning and multi-modal data fusion technologies, intelligent identification and track prediction of vehicle behaviors are realized, and the auditing accuracy is improved; a feature matching and risk assessment mechanism is adopted, so that the false alarm rate of the system is reduced; the auditing processing method based on the evidence chain ensures the reliability and traceability of the auditing result.

Description

Technical Field

[0001] The present invention relates to the technical field of toll auditing, and in particular to an intelligent vehicle toll auditing system based on deep learning. Background Art

[0002] With the full promotion of the "one network" toll collection model for national highways, the road network environment has become increasingly complex, and the problem of toll evasion has become more serious. When faced with complex road networks and diverse toll evasion behaviors, traditional audit methods have exposed problems such as fragmented audit structures, difficulty in identifying toll evasion types, difficulty in verifying suspected vehicles, and inaccurate calculation of toll evasion amounts. This not only leads to high labor costs and inefficient verification work, but also makes it difficult to fundamentally curb the phenomenon of toll evasion.

[0003] At present, the industry's audit and fight against evasion mainly adopts methods such as manually checking the entrance and exit toll data, checking the lane snapshot pictures or videos, and using self-built or third-party evasion detection algorithms. However, these methods have problems such as low efficiency, large manpower investment, and generally low evasion detection accuracy. In particular, fixed algorithms are difficult to cope with the ever-changing evasion behavior, resulting in a gradual decrease in algorithm accuracy. In addition, there are also many difficulties in the collection, verification, amount calculation and work order submission of evasion data. Summary of the invention

[0004] In view of the problems existing in the existing audit system in data collection, behavior recognition, evidence chain construction and amount accounting, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is how to realize intelligent identification of fee evasion behavior, automatic collection of evidence and intelligent processing of the audit process through deep learning and big data analysis technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a vehicle toll intelligent audit system based on deep learning, which includes a feature extraction module for extracting vehicle feature parameters in a gantry image to form a vehicle toll data packet; a trajectory prediction module for extracting spatiotemporal features of the vehicle toll data packet, using a convolutional neural network for behavior recognition and classification, and training a vehicle trajectory prediction model based on the behavior classification results to output a trajectory code; a matching analysis module for matching the trajectory code with the payment data, calculating the path matching degree, training a multi-layer perceptron anomaly detection model based on the path matching degree, and generating an audit risk score; an evidence collection module for screening vehicle toll images based on the audit risk score, extracting vehicle toll information, and generating an audit evidence sequence; an audit processing module for calculating the amount of arrears based on the audit evidence sequence and generating an audit recovery notice.

[0008] As a preferred solution of the deep learning-based vehicle toll intelligent audit system described in the present invention, the working process of the feature extraction module is as follows: use the deep learning target detection network to perform vehicle detection on the gantry image to obtain the vehicle area image; perform feature segmentation on the vehicle area image to extract the license plate area, body area, and wheel area; use the optical character recognition model to extract the license plate number from the license plate area, use the color segmentation algorithm to extract the body color from the body area, and use the image classification network to extract the vehicle model identification code from the body area; calculate the body length based on the body area boundary box coordinates, extract the number of axles from the wheel area image, and obtain the vehicle feature parameters; combine the vehicle feature parameters with the gantry time and geographic coordinates to generate a vehicle pass data packet.

[0009] As a preferred solution of the deep learning-based intelligent vehicle toll audit system described in the present invention, the vehicle characteristic parameters include license plate number, body color, vehicle model identification code, body length, and number of axles; the gantry time includes the passage period and collection time.

[0010] As a preferred solution of the vehicle toll intelligent audit system based on deep learning described in the present invention, the workflow of the trajectory prediction module is as follows: construct a vehicle feature matrix according to vehicle feature parameters, construct a time series matrix according to gantry time, and construct a spatial position matrix according to geographic coordinates; perform tensor splicing operations on the vehicle feature matrix, the time series matrix, and the spatial position matrix to obtain a multidimensional feature tensor; separate the multidimensional feature tensor to obtain vehicle feature sub-tensors, time feature sub-tensors, and spatial feature sub-tensors, and input the feature sub-tensors into a multi-layer bidirectional convolutional neural network for feature extraction to obtain a behavior feature graph; construct a retrieval index based on the license plate number, calculate the vehicle behavior probability distribution according to the behavior feature graph, and extract the behavior category label corresponding to the maximum probability value; input the behavior category label and vehicle feature parameters into the trajectory prediction model, and output a trajectory point prediction vector; generate a trajectory point coordinate sequence according to the trajectory point prediction vector, and convert the trajectory point coordinate sequence into a trajectory code using a vector quantization coding method.

[0011] As a preferred solution of the vehicle toll intelligent audit system based on deep learning described in the present invention, the vector quantization coding method is used to convert the trajectory point coordinate sequence into trajectory coding, which includes the following steps: the principal component analysis method is used to calculate the covariance matrix of the trajectory point prediction vector to obtain the eigenvalues ​​and eigenvectors; the number of principal components is determined according to the cumulative contribution rate of the eigenvalues, and the trajectory point prediction vector is projected into the eigenvector space to generate a reduced dimension feature matrix; the reduced dimension feature matrix is ​​standardized to obtain a normalized feature sequence, and a feature transformation parameter matrix is ​​generated; the normalized feature sequence is input into the K-means clustering algorithm , obtain the cluster center vector group, perform Schmidt orthogonalization on the cluster center vector group, and generate the initial codeword matrix; calculate the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix, execute the nearest neighbor search algorithm, obtain the codeword index corresponding to the minimum distance, and generate a quantization index sequence; multiply the quantization index sequence with the initial codeword matrix to obtain the reconstructed feature sequence, calculate the reconstruction error, and construct the codebook optimization objective function; use the stochastic gradient descent algorithm to optimize the codebook optimization objective function, update the initial codeword matrix, and obtain the optimized codeword matrix; perform nearest neighbor matching on the normalized feature sequence and the optimized codeword matrix, and output the trajectory code.

[0012] As a preferred solution of the vehicle toll intelligent audit system based on deep learning described in the present invention, the working process of the matching analysis module is as follows: extract the payment amount and toll station number in the payment data, generate a payment path sequence, and convert the payment path sequence into a payment path code using a vector quantization encoding method; construct a path alignment matrix, perform feature mapping on the trajectory code and the payment path code, and use cosine distance calculation to obtain the path matching degree; extract the vehicle characteristic parameters and gantry time corresponding to the path matching degree, and combine the path matching degree, vehicle characteristic parameters, and gantry time to form a training sample matrix; construct a multi-layer perceptron network structure, the multi-layer perceptron network includes an input layer, three hidden layers with a Dropout layer, and an output layer, the hidden layer uses a LeakyReLU activation function, and the output layer uses a normalized activation function; input the training sample matrix into the multi-layer perceptron network, use cross entropy as the loss function, and use the Adam optimizer to optimize the multi-layer perceptron network parameters; input the vehicle pass data packet into the trained multi-layer perceptron network, obtain the network output value, and generate an audit risk score.

[0013] As a preferred solution of the deep learning-based intelligent audit system for vehicle tolls described in the present invention, the workflow of the evidence collection module is as follows: extract the license plate number and traffic time period corresponding to the audit risk score, and retrieve the gantry image sequence of the license plate number within the traffic time period; perform feature enhancement processing on the gantry image sequence, extract the license plate features, vehicle body features, and axle features in the gantry image sequence, and generate a feature vector group; input the feature vector group into the similarity calculation module, use Euclidean distance to calculate the feature similarity between images, and generate an image matching matrix; calculate the feature similarity of adjacent images according to the image matching matrix, and combine adjacent gantry images with feature similarity higher than the matching threshold to form an image evidence chain; extract the vehicle feature parameters and gantry time corresponding to the image evidence chain, and generate an audit evidence sequence in chronological order.

[0014] As a preferred solution of the vehicle toll intelligent audit system based on deep learning described in the present invention, the workflow of the audit processing module is as follows: extract the gantry time and geographic coordinates in the audit evidence sequence, construct the vehicle travel path based on the geographic coordinates, and calculate the actual mileage of the vehicle; determine the vehicle toll rate according to the vehicle model identification code and the number of axles, and multiply the actual mileage of the vehicle by the vehicle toll rate to calculate the amount payable; extract the actual paid amount from the payment data, and determine the difference between the amount payable and the actual paid amount as the amount in arrears; calculate the payment period according to the audit result generation date; combine the license plate number, arrears amount, payment period, travel time period, vehicle model identification code, actual vehicle mileage, and number of axles to generate an audit and collection notice.

[0015] The beneficial effects of the present invention are as follows: the present invention accurately extracts vehicle feature parameters through a feature extraction module, thereby realizing the automatic collection and standardized processing of vehicle traffic data; the deep learning method is used to classify vehicle behaviors and predict trajectories through a trajectory prediction module, thereby improving the accuracy of trajectory recognition; the trajectory data is intelligently matched with payment data through a matching analysis module, and anomaly detection is performed using a multi-layer perceptron to effectively identify vehicles with a higher risk of evading fees; a complete audit evidence chain and a recovery notice are automatically generated through an evidence collection module and an audit processing module, thereby improving the efficiency of audit work and reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is the system architecture diagram of the vehicle toll intelligent audit system based on deep learning.

[0018] Figure 2 This is the flow chart of the feature extraction module of the vehicle toll intelligent audit system based on deep learning.

[0019] Figure 3 This is the flow chart of the trajectory prediction module of the vehicle toll intelligent audit system based on deep learning. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1

[0024] Reference Figure 1 to Figure 3 , which is the first embodiment of the present invention, and provides a vehicle toll intelligent audit system based on deep learning. The system architecture diagram is as follows Figure 1 As shown, it includes the following functional modules:

[0025] A feature extraction module is used to extract vehicle feature parameters from the gantry image to form a vehicle traffic data packet;

[0026] The trajectory prediction module is used to extract the spatiotemporal features of vehicle traffic data packets, use convolutional neural networks for behavior recognition and classification, and train the vehicle trajectory prediction model based on the behavior classification results to output the trajectory code;

[0027] The matching analysis module is used to match the trajectory code with the payment data, calculate the path matching degree, train the multi-layer perceptron anomaly detection model based on the path matching degree, and generate the audit risk score;

[0028] The evidence collection module is used to screen vehicle traffic images according to the audit risk score, extract vehicle traffic information, and generate an audit evidence sequence;

[0029] The audit processing module is used to calculate the amount of outstanding fees based on the audit evidence sequence and generate an audit recovery notice.

[0030] In the specific implementation, the feature extraction module flow chart is as follows Figure 2 As shown, it includes: using a deep learning target detection network to detect vehicles on the gantry image to obtain a vehicle area image; performing feature segmentation on the vehicle area image to extract the license plate area, body area, and wheel area; using an optical character recognition model to extract the license plate number from the license plate area, using a color segmentation algorithm to extract the body color from the body area, using an image classification network to extract the vehicle model identification code from the body area, calculating the body length based on the body area boundary box coordinates, extracting the number of axles from the wheel area image, and obtaining vehicle feature parameters; combining the vehicle feature parameters with the gantry time and geographic coordinates to generate a vehicle traffic data packet. Among them, the vehicle feature parameters include the license plate number, body color, vehicle model identification code, body length, and number of axles; the gantry time includes the traffic period and the collection time.

[0031] In the feature extraction module, the deep learning target detection network adopts the YOLOv5 model, which achieves fast and accurate vehicle detection through pre-trained weights; the feature segmentation adopts the U-Net network structure to achieve accurate segmentation of various areas of the vehicle; the optical character recognition model adopts the CRNN+CTC architecture to improve the accuracy of license plate recognition; the color segmentation algorithm is based on the HSV color space and K-means clustering method to achieve stable extraction of vehicle body color; the image classification network adopts the ResNet50 structure to provide reliable vehicle model classification results.

[0032] In the prior art, traditional trajectory prediction methods mainly perform simple time series prediction based on historical trajectory data, without fully considering the impact of vehicle behavior characteristics on trajectory prediction. In addition, the prediction results are easily affected by data noise, and the prediction accuracy and robustness are poor. In addition, the existing methods have high computational complexity when processing long sequence trajectory data, making it difficult to meet real-time prediction requirements.

[0033] To solve the above problems, the present invention designs a trajectory prediction module based on behavior recognition. The flow chart of the trajectory prediction module is as follows: Figure 3 As shown, it includes: constructing a vehicle feature matrix according to vehicle feature parameters, constructing a time series matrix according to the gantry time, and constructing a space position matrix according to the geographic coordinates; performing tensor splicing operations on the vehicle feature matrix, the time series matrix, and the space position matrix to obtain a multi-dimensional feature tensor; separating the multi-dimensional feature tensor to obtain a vehicle feature sub-tensor, a time feature sub-tensor, and a space feature sub-tensor, wherein the dimensions of each sub-tensor are (batch_size, features, sequence_length), features are 16 dimensions for vehicle features, 8 dimensions for time features, and 4 dimensions for space features, and sequence_length is uniformly 12.

[0034] Furthermore, the feature sub-tensors are input into a multi-layer bidirectional convolutional neural network for feature extraction to obtain a behavior feature map. The process is as follows: first, the three feature sub-tensors are respectively input into two layers of one-dimensional convolutional layers for feature extraction, the convolution kernel size is 3, the step size is 1, the padding method is the same, and the activation function is ReLU; then the convolved feature map is subjected to a maximum pooling operation with a pooling kernel size of 2; then the pooled feature map is sent to a bidirectional LSTM layer for temporal feature learning, and the hidden layer dimension is 128; finally, the feature map output by the LSTM is reduced in dimension through a fully connected layer to obtain a 256-dimensional behavior feature map.

[0035] Furthermore, a retrieval index is constructed based on the license plate number, the vehicle behavior probability distribution is calculated based on the behavior feature graph, and the behavior category label corresponding to the maximum probability is extracted. Specifically, the cosine similarity is used to calculate the similarity score between the behavior feature graph and the features of each category in the pre-trained behavior template library, and the similarity score is normalized to a probability distribution through the Softmax function, and the behavior category corresponding to the maximum probability is selected as the prediction result. The pre-trained behavior template library contains 10,000 annotated samples, covering 4 behavior categories, and each category contains at least 1,000 valid samples.

[0036] In addition, the behavior category labels include normal behavior category, violation behavior category, abnormal behavior category and special situation category. For example, the normal behavior category includes normal passage, compliant detour, etc.; the violation behavior category includes detour to avoid tolls, segment payment, vehicle type mismatch, license plate obstruction / alteration / mismatch, overtime stop, reverse driving, merging and passing stations, frequent lane changes, etc.; the abnormal behavior category includes equipment failure, data missing, duplicate billing, pending confirmation, etc.; the special situation category includes emergency vehicles, green channels, official vehicles and other free policy vehicles.

[0037] Furthermore, the behavior category label and vehicle feature parameters are input into the trajectory prediction model, which includes a bidirectional gated recurrent unit and outputs a trajectory point prediction vector. The training process of the trajectory prediction model includes the following steps: uniquely encode the behavior category label, convert the vehicle model identification code into a numerical vector, normalize the vehicle body length and the number of axles, and concatenate the processed feature data to form an input feature vector; construct a bidirectional gated recurrent unit network, including an input layer, two bidirectional gated recurrent layers, a fully connected layer, and an output layer. The input layer dimension matches the feature vector dimension, and the number of hidden units in the bidirectional gated recurrent layer is 128; set the time step to 12, expand the input feature vector according to the time step, and input it into the forward and reverse layers of the bidirectional gated recurrent unit network to generate a bidirectional feature vector. sequence; map the bidirectional feature sequence to the two-dimensional coordinate space through a fully connected layer, activate it with the hyperbolic tangent function, and obtain the predicted value of the trajectory point coordinates; use the mean square error as the loss function, calculate the error between the predicted value of the trajectory point coordinates and the true trajectory point coordinates, and use the Adam optimizer to update the network parameters; iteratively train the bidirectional gated recurrent unit network, stop training when the validation set loss value converges or reaches the preset number of iterations, and use the trained network as the trajectory prediction model; input the new input feature vector into the trajectory prediction model, and output the trajectory point prediction vector, which contains the coordinate prediction values ​​of 12 consecutive time steps.

[0038] Further, a trajectory point coordinate sequence is generated according to the trajectory point prediction vector, and the trajectory point coordinate sequence is converted into trajectory coding by using a vector quantization coding method. In terms of trajectory coding, the principal component analysis method is used to calculate the covariance matrix of the trajectory point prediction vector to obtain eigenvalues ​​and eigenvectors; the number of principal components is determined according to the cumulative contribution rate of the eigenvalues ​​(the threshold of the cumulative contribution rate of the eigenvalues ​​in the present invention is set to 0.95), and the trajectory point prediction vector is projected into the eigenvector space to generate a reduced dimension feature matrix; the reduced dimension feature matrix is ​​standardized to obtain a normalized feature sequence, and a feature transformation parameter matrix is ​​generated; the normalized feature sequence is input into the K-means clustering algorithm to obtain a cluster center vector group, and the cluster center vector group is Schmidt orthogonalized to generate an initial codeword matrix; the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix is ​​calculated, and the nearest neighbor search algorithm is executed to obtain the codeword index corresponding to the minimum distance, and a quantization index sequence is generated; the quantization index sequence is multiplied by the initial codeword matrix to obtain a reconstructed feature sequence, the reconstruction error is calculated, and the codebook optimization objective function is constructed. The specific formula is as follows:

[0039]

[0040] Among them, X is the normalized feature sequence, Q(X) is the quantized reconstruction sequence, C is the initial codeword matrix, is the square of the Euclidean distance, ||·||1 is the L1 norm, and λ is the regularization coefficient (set to 0.01). This objective function integrates PCA dimensionality reduction, K-means clustering, and vector quantization coding into a unified optimization framework. The coding quality is directly measured, and the regularization term λ||C||1 controls the sparsity of the codebook, achieving efficient compression while ensuring the ability to express trajectory features.

[0041] Furthermore, the stochastic gradient descent algorithm is used to optimize the codebook optimization objective function, and the initial codeword matrix is ​​updated to obtain the optimized codeword matrix; the normalized feature sequence is nearest neighbor matched with the optimized codeword matrix, and the trajectory code is output.

[0042] Preferably, the trajectory prediction module realizes the effective fusion of vehicle features, time features and spatial features through the construction and separation of multi-dimensional feature tensors; uses a multi-layer bidirectional convolutional neural network to extract behavior features, which improves the accuracy of behavior recognition; guides trajectory prediction based on behavior category labels, which significantly improves the prediction accuracy; and realizes efficient compression and representation of trajectory data through vector quantization coding methods.

[0043] In the prior art, traditional path matching methods often use simple path overlap calculations without considering the path temporal relationship and connectivity characteristics, which can easily lead to matching errors. In addition, they lack an effective risk assessment mechanism and are difficult to accurately identify abnormal traffic behaviors.

[0044] In view of the above problems, the present invention proposes a matching analysis module based on path alignment, and its working process is as follows: extract the payment amount and toll station number in the payment data, generate the payment path sequence, and use the vector quantization coding method to convert the payment path sequence into the payment path code; construct a path alignment matrix, perform feature mapping on the trajectory code and the payment path code, and use the cosine distance calculation to obtain the path matching degree. Among them, the number of matrix rows of the path alignment matrix is ​​equal to the dimension of the trajectory code, and the number of matrix columns is equal to the dimension of the payment path code.

[0045] Furthermore, the temporal relationship between adjacent coding units in the trajectory coding is extracted to generate a trajectory temporal relationship vector, and the connectivity between adjacent coding units in the payment path coding is extracted to generate a payment connectivity relationship vector, wherein the temporal relationship vector represents the normalized value of the time interval between adjacent trajectory points, and the connectivity relationship vector represents the normalized value of the spatial distance between adjacent payment points; the dot product of the trajectory temporal relationship vector and the payment connectivity relationship vector is calculated to obtain the path correlation degree, the path correlation degree range is [0,1], and the path correlation degree is filled into the corresponding position of the path alignment matrix; based on the dynamic programming algorithm, the optimal alignment path is searched in the path alignment matrix, and the path correlation degree sequence on the optimal alignment path is extracted; the path correlation degree sequence is subjected to min-max normalization processing to generate a path matching degree sequence, and the mean of the path matching degree sequence is calculated to obtain the path matching degree.

[0046] Furthermore, the vehicle characteristic parameters and gantry time corresponding to the path matching degree are extracted, and the path matching degree, vehicle characteristic parameters and gantry time are combined to form a training sample matrix; a multi-layer perceptron network structure is constructed, and the multi-layer perceptron network includes an input layer, three hidden layers with a Dropout layer and an output layer. The hidden layer uses the LeakyReLU activation function, and the output layer uses the normalized activation function; the training sample matrix is ​​input into the multi-layer perceptron network, and the cross entropy is used as the loss function. The Adam optimizer is used to optimize the multi-layer perceptron network parameters; the vehicle traffic data packet is input into the trained multi-layer perceptron network, the network output value is obtained, and an audit risk score with a value range of [0,1] is generated, where a score greater than 0.8 is judged as high risk.

[0047] In addition, in the multi-layer perceptron network, the dropout rate of the Dropout layer is set to 0.3, the negative semi-axis slope of the LeakyReLU activation function is 0.01, and the initial value of the learning rate is set to 0.001.

[0048] Preferably, the matching analysis module achieves accurate matching of trajectory coding and payment path coding through the path alignment matrix; introduces temporal relationship and connectivity features to improve the accuracy of matching; and uses a multi-layer perceptron network to establish a risk assessment model to achieve effective identification of abnormal traffic behavior.

[0049] In the existing technology, traditional evidence collection methods often only focus on a single gantry image and lack systematic analysis of continuous traffic records; in addition, the image quality is unstable and the feature extraction effect is poor, making it difficult to build a reliable chain of evidence.

[0050] In view of the above problems, the present invention proposes an evidence collection module based on feature matching, and its working process is as follows: extract the license plate number and the traffic time period corresponding to the audit risk score, and retrieve the gantry image sequence of the license plate number within the traffic time period; perform feature enhancement processing on the gantry image sequence, extract the license plate features, vehicle body features, and axle features in the gantry image sequence, and generate a feature vector group; input the feature vector group into the similarity calculation module, use the Euclidean distance to calculate the feature similarity between images, and generate an image matching matrix; calculate the feature similarity of adjacent images according to the image matching matrix, and combine adjacent gantry images with feature similarity higher than the matching threshold to form an image evidence chain; extract the vehicle feature parameters and gantry time corresponding to the image evidence chain, and generate an audit evidence sequence in chronological order.

[0051] In the evidence collection module, feature enhancement processing includes contrast adaptive equalization and Gaussian noise suppression to enhance image details; contrast adaptive equalization uses the CLAHE algorithm to divide the image into 8×8 blocks of equal size, limit the contrast threshold to 3.0, perform histogram equalization on each block, and eliminate artificial boundaries between blocks through bilinear interpolation. Gaussian noise suppression uses a bilateral filter with a spatial domain standard deviation of 75, a value domain standard deviation of 75, and a window size of 5. It also considers spatial distance and gray value differences, effectively suppressing noise while maintaining edge information.

[0052] In addition, the feature vector group is extracted with 1024-dimensional feature vectors through ResNet50; the similarity calculation adopts weighted Euclidean distance, and the license plate, body and axle features are assigned weight coefficients of 0.5, 0.3 and 0.2 respectively; the matching threshold is determined by ROC curve analysis and is set to 0.85, which ensures the matching accuracy while avoiding the break of the evidence chain; the image evidence chain is constructed by sliding window method with a window size of 5 and a step size of 1. The overlap rate between windows is 50%, and adjacent windows share 2-3 frames of images to ensure the continuity and integrity of the evidence. The sliding window moves sequentially in time sequence, and feature matching is performed on the image sequence in each window to ensure the continuity and integrity of the acquired evidence.

[0053] In specific implementation, the workflow of the audit processing module is as follows: extract the gantry time and geographic coordinates in the audit evidence sequence, construct the vehicle travel path based on the geographic coordinates, and calculate the actual mileage of the vehicle; determine the basic rate corresponding to the vehicle model according to the "Vehicle Type Classification for Toll Road Vehicle Tolls" (JT / T489-2019) standard; determine the vehicle toll rate based on the vehicle model identification code and the number of axles, where the rate for vehicles with ≥6 axles is increased by 20% over the basic rate, and multiply the actual mileage of the vehicle by the vehicle toll rate to calculate the payable amount Amount; extract the actual payment amount from the payment data, and determine the difference between the amount payable and the actual payment amount as the amount in arrears; calculate the payment period based on the date when the audit result is generated, and the rules are as follows: if the amount in arrears is less than 1,000 yuan, a 15-day payment period is given; if the amount in arrears is 1,000-5,000 yuan, a 30-day payment period is given; if the amount in arrears is more than 5,000 yuan, a 45-day payment period is given; combine the license plate number, amount in arrears, payment period, travel time, vehicle type identification code, actual mileage of the vehicle, and number of axles to generate an audit and collection notice.

[0054] In summary, the present invention accurately extracts vehicle feature parameters through the feature extraction module, realizes the automatic collection and standardized processing of vehicle traffic data; uses the deep learning method to classify vehicle behavior and predict trajectory through the trajectory prediction module, thereby improving the accuracy of trajectory recognition; uses the matching analysis module to intelligently match trajectory data with payment data, and uses a multi-layer perceptron for anomaly detection to effectively identify vehicles with a higher risk of evading fees; and automatically generates a complete audit evidence chain and recovery notice through the evidence collection module and the audit processing module, thereby improving the efficiency of audit work and reducing labor costs.

[0055] Example 2

[0056] Reference Figure 1 to Figure 3 , which is the second embodiment of the present invention, provides a vehicle toll intelligent audit system based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0057] The research team conducted a three-month experimental verification on a provincial highway network. The experimental data came from the actual data collected by 124 toll gates in the province, involving about 6.8 million vehicle traffic records, including 6.52 million normal traffic data and 280,000 abnormal traffic data. The experimental environment uses an Intel Xeon Gold 6248R processor, 256GB memory, and a GPU of NVIDIA A100.

[0058] In the feature extraction module, the research team used the YOLOv5s model to detect vehicles on gantry images. The model input size was set to 640×640 pixels, the batch size was set to 32, the initial learning rate was 0.01, and the training rounds were 100. The license plate recognition used an improved CRNN+CTC model and was trained on 80,000 highway vehicle images. The vehicle type classification used the ResNet50 network and was trained on a data set containing 12 vehicle types.

[0059] In the trajectory prediction module, the researchers built a behavior template library containing 8,000 labeled samples, including 5,600 normal behavior samples (70%), 1,600 illegal behavior samples (20%), 600 abnormal behavior samples (7.5%), and 200 special case samples (2.5%). The bidirectional gated recurrent unit network uses a 128-dimensional hidden layer, the dropout rate is set to 0.5, the training batch size is 64, and the learning rate is 0.001.

[0060] As shown in Table 1, a comparative experiment was conducted between the present invention and the existing deep learning method, and a currently widely used deep learning method was selected for performance evaluation.

[0061] Table 1 System performance comparison experimental results

[0062] Evaluation Metrics Method of the present invention Deep Learning Methods Vehicle detection accuracy (%) 94.6 93.2 Vehicle recognition accuracy (%) 93.1 91.7 Behavior recognition accuracy (%) 89.2 87.5 Trajectory prediction error (m) 12.3 14.2 Risk identification accuracy (%) 87.8 85.9 System processing delay (ms) 82 95 Memory usage (GB) 38 45

[0063] From the comparison results in Table 1, it can be seen that the method of the present invention significantly improves the system efficiency while maintaining a high accuracy rate. In particular, it achieves 13.7% and 15.6% optimization in system processing delay and memory usage, respectively. This is mainly due to the improved feature extraction strategy and optimized network structure design.

[0064] In the experiment of the evidence collection module, the feature enhancement processing uses the CLAHE algorithm for contrast equalization, divides the image into 8×8 blocks, and limits the contrast threshold to 3.0. A bilateral filter is used for noise suppression, and the standard deviation of the spatial domain and the range is set to 75. The signal-to-noise ratio of the processed image is improved by 6.2dB. The image evidence chain is constructed using the sliding window method, and the effect is best when the window size is set to 5 frames, achieving an evidence completeness rate of 91.5%.

[0065] In the actual abnormal traffic data processing, the system successfully generated 252,000 audit and recovery notices, with a recovery accuracy rate of 90.3%. The average system processing time is 2.8 seconds per item, which is 25% more efficient than the existing deep learning method. The recovery amount distribution shows that: less than 500 yuan accounts for 45%, 500-2000 yuan accounts for 35%, 2000-5000 yuan accounts for 15%, and more than 5000 yuan accounts for 5%. These results show that the present invention has good performance and efficiency advantages in practical applications.

[0066] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A vehicle toll intelligent audit system based on deep learning, characterized by: include, A feature extraction module is used to extract vehicle feature parameters from the gantry image to form a vehicle traffic data packet; A trajectory prediction module is used to extract spatiotemporal features of the vehicle traffic data packets, use a convolutional neural network to perform behavior recognition and classification, and train a vehicle trajectory prediction model based on the behavior classification results to output a trajectory code; A matching analysis module, used to match the trajectory code with the payment data, calculate the path matching degree, train the multi-layer perceptron anomaly detection model according to the path matching degree, and generate an audit risk score; An evidence collection module, used to screen vehicle traffic images according to the audit risk score, extract vehicle traffic information, and generate an audit evidence sequence; The audit processing module is used to calculate the amount of outstanding fees based on the audit evidence sequence and generate an audit collection notice.

2. The vehicle toll intelligent audit system based on deep learning as claimed in claim 1, characterized in that: The workflow of the feature extraction module is as follows: Use deep learning target detection network to detect vehicles on gantry images and obtain vehicle area images; Performing feature segmentation on the vehicle area image to extract the license plate area, the vehicle body area, and the wheel area; The optical character recognition model is used to extract the license plate number from the license plate area, the color segmentation algorithm is used to extract the body color from the body area, and the image classification network is used to extract the vehicle model identification code from the body area; The vehicle body length is calculated based on the coordinates of the bounding box of the vehicle body area, and the number of axles is extracted from the wheel area image to obtain the vehicle characteristic parameters; The vehicle characteristic parameters are combined with the gantry time and geographic coordinates to generate a vehicle traffic data packet.

3. The vehicle toll intelligent audit system based on deep learning as claimed in claim 2, characterized in that: The vehicle characteristic parameters include license plate number, body color, vehicle model identification code, body length, and number of axles; the gantry time includes the passage period and collection time.

4. The vehicle toll intelligent audit system based on deep learning as claimed in claim 1, characterized in that: The workflow of the trajectory prediction module is as follows: Construct a vehicle feature matrix based on vehicle feature parameters, construct a time series matrix based on gantry time, and construct a spatial position matrix based on geographic coordinates; Performing tensor concatenation operation on the vehicle feature matrix, the time series matrix, and the spatial position matrix to obtain a multi-dimensional feature tensor; Separating the multidimensional feature tensor into a vehicle feature sub-tensor, a time feature sub-tensor, and a space feature sub-tensor, inputting the feature sub-tensors into a multi-layer bidirectional convolutional neural network for feature extraction, and obtaining a behavior feature graph; Building a retrieval index based on the license plate number, calculating the vehicle behavior probability distribution according to the behavior feature graph, and extracting the behavior category label corresponding to the maximum probability value; Inputting the behavior category label and the vehicle characteristic parameter into a trajectory prediction model, and outputting a trajectory point prediction vector; A trajectory point coordinate sequence is generated according to the trajectory point prediction vector, and the trajectory point coordinate sequence is converted into trajectory code by using a vector quantization coding method.

5. The vehicle toll intelligent audit system based on deep learning as claimed in claim 4, characterized in that: The method of converting the trajectory point coordinate sequence into trajectory code by using the vector quantization coding method comprises the following steps: The principal component analysis method is used to calculate the covariance matrix of the trajectory point prediction vector to obtain eigenvalues ​​and eigenvectors; Determine the number of principal components according to the cumulative contribution rate of the eigenvalues, project the trajectory point prediction vector into the eigenvector space, and generate a reduced-dimensional feature matrix; Standardizing the dimension-reduced feature matrix to obtain a normalized feature sequence and generate a feature transformation parameter matrix; Inputting the normalized feature sequence into a K-means clustering algorithm to obtain a cluster center vector group, performing Schmidt orthogonalization processing on the cluster center vector group, and generating an initial codeword matrix; Calculating the Euclidean distance matrix between the normalized feature sequence and the initial codeword matrix, executing a nearest neighbor search algorithm, obtaining a codeword index corresponding to a minimum distance, and generating a quantized index sequence; Multiplying the quantization index sequence by the initial codeword matrix to obtain a reconstruction feature sequence, calculating a reconstruction error, and constructing a codebook optimization objective function; The codebook optimization objective function is optimized by using a stochastic gradient descent algorithm, and the initial codeword matrix is ​​updated to obtain an optimized codeword matrix; The normalized feature sequence is matched with the optimized codeword matrix by nearest neighbor, and the trajectory code is output.

6. The vehicle toll intelligent audit system based on deep learning as claimed in claim 1, characterized in that: The workflow of the matching analysis module is as follows: Extracting the payment amount and toll station number from the payment data, generating a payment path sequence, and converting the payment path sequence into a payment path code using a vector quantization coding method; Constructing a path alignment matrix, performing feature mapping between the trajectory code and the payment path code, and using cosine distance calculation to obtain the path matching degree; Extracting vehicle characteristic parameters and gantry time corresponding to the path matching degree, and combining the path matching degree, the vehicle characteristic parameters, and the gantry time to form a training sample matrix; Construct a multi-layer perceptron network structure, wherein the multi-layer perceptron network includes an input layer, three hidden layers with Dropout layers, and an output layer, wherein the hidden layer uses a LeakyReLU activation function, and the output layer uses a normalized activation function; Input the training sample matrix into the multi-layer perceptron network, use cross entropy as the loss function, and use Adam optimizer to optimize the multi-layer perceptron network parameters; The vehicle traffic data packet is input into the trained multi-layer perceptron network to obtain the network output value and generate an audit risk score.

7. The vehicle toll intelligent audit system based on deep learning as claimed in claim 1, characterized in that: The workflow of the evidence collection module is as follows: Extract the license plate number and traffic time period corresponding to the audit risk score, and retrieve the gantry image sequence of the license plate number within the traffic time period; Performing feature enhancement processing on the mast image sequence, extracting license plate features, vehicle body features, and wheel axle features in the mast image sequence, and generating a feature vector group; The feature vector group is input into a similarity calculation module, and the feature similarity between images is calculated using Euclidean distance to generate an image matching matrix; Calculating feature similarities of adjacent images according to the image matching matrix, and combining adjacent gantry images whose feature similarities are higher than a matching threshold to form an image evidence chain; The vehicle characteristic parameters and gantry time corresponding to the image evidence chain are extracted, and an audit evidence sequence is generated in chronological order.

8. The vehicle toll intelligent audit system based on deep learning as claimed in claim 1, characterized in that: The workflow of the audit processing module is as follows: Extract the gantry time and geographic coordinates in the audit evidence sequence, construct the vehicle travel path based on the geographic coordinates, and calculate the actual mileage of the vehicle; Determine the vehicle toll rate based on the vehicle model identification code and the number of axles, and calculate the amount payable by multiplying the actual mileage of the vehicle by the vehicle toll rate; Extracting the actual payment amount from the payment data, and determining the difference between the amount payable and the actual payment amount as the amount in arrears; Calculate the payment deadline based on the date the audit results were generated; The license plate number, amount of outstanding fees, payment deadline, traffic time period, vehicle type identification code, actual vehicle mileage, and number of axles are combined to generate an audit and collection notice.

Images