A vehicle ETC data matching method and system based on convolutional neural network

Abstract

The present invention discloses a vehicle ETC data matching method based on a convolutional neural network, including: using a monitoring camera to collect a vehicle video to form a data set, constructing and training a vehicle ETC data matching network for detecting vehicle information, obtaining valid information of the passing vehicle from the ETC system, and finally using the data matching network to perform two matches, and when the two matches are correct, the vehicle is allowed to pass. The present invention uses multi-level feature extraction and multi-stage feature fusion to optimize the matching accuracy of vehicle information, which can not only effectively improve the accuracy of vehicle information matching, but also enhance the adaptability of the model to complex scenes through fine feature processing, thereby improving the accuracy of license plate and vehicle model information recognition, reducing the phenomenon of fee evasion, and avoiding inconvenience to normal vehicles.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation systems, and more particularly to a vehicle ETC data matching method and system based on a convolutional neural network. Background Art

[0002] With the rapid economic development in my country in recent years, highway construction is advancing rapidly, and the number of motor vehicles and drivers in the country is also growing rapidly. This rapid development has brought higher requirements for highway management, and the management work is becoming increasingly heavy and complex, requiring continuous response to new problems and challenges.

[0003] In this context, the Electronic Toll Collection System (ETC) was introduced and widely used. As an efficient road toll collection technology, it communicates with the toll station antenna through the electronic tags installed on the vehicle, so that the vehicle can automatically pay tolls without stopping, thereby significantly improving the vehicle's travel speed and overall traffic efficiency.

[0004] At present, most ETC systems mainly rely on radio frequency identification (RFID) technology. Although RFID technology is relatively mature, there are still some problems, such as the risk of unauthorized tag use. This leads to the possibility that some unauthorized vehicles may evade payment by following closely behind authorized vehicles or cutting in line. The main disadvantage of ETC lanes using infrared grating detectors and roadside units (RSU) is the high alarm probability, which to some extent limits its widespread application. The high alarm rate may cause inconvenience to normal vehicles and increase management costs and operational complexity. Summary of the invention

[0005] In order to overcome the problems existing in the background technology, the present invention provides a vehicle ETC data matching method based on convolutional neural network.

[0006] The main contents of the present invention are as follows:

[0007] Step 1: Use the monitoring camera at the entrance of the toll station to record the real-time video of vehicles passing through the ETC lane, extract video frames at a certain time interval, and use these video frames as data set images for subsequent vehicle information recognition;

[0008] Step 2, obtain the valid information of the vehicle from the ETC system, including the license plate number and vehicle model;

[0009] Step 3, construct and train a vehicle ETC data matching network, including a feature extraction module, a feature fusion module, a vehicle detection module, and a vehicle matching module;

[0010] The feature extraction module includes a convolution layer and n sub-feature extraction units, which operate in series. The features extracted by the n sub-feature extraction units are P1, P2, ..., Pn in sequence. The feature fusion module performs fusion processing on P2-Pn and outputs n-2 features. The vehicle detection module detects the n-2 features output by P1 to obtain vehicle model information and license plate information. Finally, the vehicle matching module performs two matches, namely, license plate matching and vehicle model matching. When the two matches are correct, the vehicle is allowed to pass.

[0011] Furthermore, the specific processing process of the feature extraction module is as follows;

[0012] When the input image passes through the convolution layer, the image size is halved in height and width, and the number of channels of the output image is equal to the number of channels of the convolution layer. The convolution layer consists of a convolution kernel of size 3×3 and stride 2, an h-swish activation function layer and a batch normalization layer, and the number of channels is 16. The formula of the h-swish activation function layer is as follows:

[0013]

[0014] In the above formula, x represents the input or element of the feature map of a certain layer, and ReLU is the activation function;

[0015] In the sub-feature extraction unit structure, an input feature map is accepted and first input into a first inverted residual unit. The purpose of the first inverted residual unit is to extract features and reduce the dimension to achieve feature dimensionality reduction. The reduced feature map will be input into a second inverted residual unit, and the dimension remains unchanged. The feature maps output by the first inverted residual unit and the second inverted residual unit are superimposed or spliced ​​to achieve simple feature fusion in the first stage.

[0016] Furthermore, the specific structure of the first inverted residual unit is to convert the input feature map into a tensor. The input tensor will first pass through a convolution layer with a stride of 2 and a convolution kernel size of 3. In this convolution layer, convolution operations and downsampling are performed to reduce the dimension. Then, the dilation factor R1 is set. If the dilation factor R1 is 1, a 1x1 dilated convolution will be performed. The number of output channels of this dilated convolution is the product of the dilation factor and the number of channels of the input feature map. If the dilation factor is not 1, no dilated convolution is performed. The two paths finally pass through a 1*1 convolution to convert the number of output channels to C1.

[0017] The specific structure of the second inverse residual unit is as follows: first, a 1x1 dilated convolution is performed on the input feature map to increase the number of channels of the input feature map. The number of output channels is the product of the dilation factor and the number of channels of the input feature map. Then, a depthwise separable convolution is performed, in which the convolution kernel and the step size are the same. The depthwise separable convolution divides the conventional convolution operation into two independent operations: depthwise convolution and point convolution, which reduces the amount of calculation and model complexity. Finally, a point convolution operation is performed to compress the number of channels from the expanded number of channels back to the original output number of channels.

[0018] Furthermore, n takes a value of 4, and the first sub-feature extraction unit and the second sub-feature extraction unit adopt a superposition method to realize the feature fusion of the first stage. During this process, the dimension of the fused feature map remains unchanged, and the number of channels is added; the third sub-feature extraction unit and the fourth sub-feature extraction unit adopt a splicing method to realize the feature fusion of the first stage. During this process, the number of channels of the fused feature map remains unchanged, the dimension is seamlessly spliced ​​in the same space, and the dimension of the feature map is increased; the features extracted by the four sub-feature extraction units are P1, P2, P3, and P4 respectively.

[0019] Furthermore, in the feature fusion module, only one intermediate node is set in the P3 layer, which receives feature inputs from the P2, P3, and P4 layers, and outputs the feature information of the three layers of P2, P3, and P4 to the output node of the P3 layer after integrating them;

[0020] The output node of the P3 layer receives the output of the intermediate node and the features of the P4 layer at the same time. The two features are fused at the output node of the P3 layer and transmitted upward to the output node of the P2 layer.

[0021] The output node of the P2 layer receives the feature input of two levels, the output of the output nodes of the feature P2 and P3 layers, and after the features of the two levels are fused, they are output to the output node of the feature P1 layer, completing the last feature fusion with the feature P1;

[0022] The features output by the output nodes of the P1, P2, and P3 layers are the features that need to be detected in the end. The vehicle detection module includes a detection head and an OCR. The detection head detects the features output by the output nodes of the P1, P2, and P3 layers to obtain the vehicle model information. The OCR detection is used to obtain the license plate information, where the vehicle model information and the license plate information are both given in the form of prediction boxes.

[0023] Furthermore, the fusion formula is as follows:

[0024]

[0025] In the above formula, O is the fused feature, P is the fused feature, w is the weight of feature P during fusion, M is the number of feature maps during fusion, and Resize is the operation of changing the dimension of the feature map.

[0026] Furthermore, the total loss function used in training the vehicle ETC data matching network is:

[0027] L=λ·L cha +β·L re +σ·L cls

[0028] In the total loss function above, λ is the character matching loss L cha The weight, β is the regression loss L re The weight, σ is the classification loss L cls The weight occupied.

[0029] Furthermore, in the first matching, the license plate information obtained by the ETC system is compared with the license plate information obtained by the OCR license plate detection in a character-by-character manner, and the loss is calculated at the same time. The license plate character matching loss L cha The calculation formula is:

[0030]

[0031] In the above formula, if the characters are the same q i =0, if the characters are different, then q i =1, N represents the total number of license plate characters.

[0032] Furthermore, in the second matching process, the vehicle models are matched, and the classification loss L calculated by the VFL loss function is cls for;

[0033]

[0034] Where p is the scalar of the predicted classification score vector, for the positive samples in training, q is the intersection over union (IoU) between the generated predicted box and the true box, for the negative samples in training, q is 0, Ψ is the loss weight of the image foreground and background, and γ is the weight of each sample;

[0035] Use CIoU function as the regression loss L of the bounding box re for:

[0036]

[0037] The parameters α and v are as follows:

[0038]

[0039] In the above formula, ρ represents the Euclidean distance between the two rectangular boxes of the predicted box and the real box, IoU is the intersection over union ratio, b1 and b2 represent the center points of the predicted box and the real box, c is the diagonal length of the minimum bounding box covering the two bounding boxes, v is a consistency parameter used to measure the relative proportion of the predicted box and the real box, α is a balance parameter, w1, h1, w2 and h2 are the width and height of the predicted box and the real box respectively.

[0040] The present invention also provides a vehicle ETC data matching system based on a convolutional neural network, comprising the following units:

[0041] The data acquisition unit is used to use the monitoring camera at the entrance of the toll station to record the real-time video of the vehicle passing through the ETC lane, extract the video frames at a certain time interval, and use these video frames as the data set images for subsequent vehicle information recognition;

[0042] Vehicle information extraction unit, used to obtain valid vehicle information from the ETC system, including license plate number and vehicle model;

[0043] A model building and training unit, used to build and train a vehicle ETC data matching network, including a feature extraction module, a feature fusion module, a vehicle detection module, and a vehicle matching module;

[0044] The feature extraction module includes a convolution layer and n sub-feature extraction units, which operate in series. The features extracted by the n sub-feature extraction units are P1, P2, ..., Pn in sequence. The feature fusion module performs fusion processing on P2-Pn and outputs n-2 features. The vehicle detection module detects the n-2 features output by P1 to obtain vehicle model information and license plate information. Finally, the vehicle matching module performs two matches, namely, license plate matching and vehicle model matching. When the two matches are correct, the vehicle is allowed to pass.

[0045] The vehicle ETC data matching method of the present invention has the following beneficial effects compared with the prior art:

[0046] (1) By constructing and training a vehicle ETC data matching network, the present invention adopts multi-level feature extraction and multi-stage feature fusion to optimize the matching accuracy of vehicle information, which can not only effectively improve the accuracy of vehicle information matching, but also enhance the adaptability of the model to complex scenarios through fine feature processing, thereby improving the accuracy of license plate and vehicle model information recognition, reducing the phenomenon of toll evasion, and avoiding inconvenience to normal vehicles.

[0047] (2) By effectively reducing the dimension of the feature graph and using appropriate fusion strategies, the amount of computation is reduced, allowing the model to achieve fast matching without sacrificing too much computing resources. Especially during peak hours, this fast matching capability can significantly improve traffic efficiency and alleviate congestion at toll booths. In addition, it also helps to capture and respond to abnormal events in a timely manner, such as the intrusion of unauthorized vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flow chart of the method provided by the present invention;

[0049] Figure 2 Provides a structural diagram of a vehicle ETC data matching network for the present invention;

[0050] Figure 3 It is a structural diagram of the sub-feature extraction unit in superposition mode;

[0051] Figure 4 This is the structural diagram of the sub-feature extraction unit in splicing mode. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be fully described below in conjunction with the accompanying drawings in the present invention.

[0053] like Figure 1 As shown, a vehicle ETC data matching method based on a convolutional neural network provided by an embodiment of the present invention includes the following steps:

[0054] Step 1: Data collection;

[0055] The monitoring camera at the entrance of the toll station is used to record the real-time video of vehicles passing through the ETC lane, and the video frames are extracted at a time interval of 0.5 seconds. These video frames are used as the data set images for subsequent vehicle information recognition; then Python is used to extract frames from the recorded video, and video frames are extracted at a time interval of 0.5 seconds. These video frames are used as the data set images for subsequent vehicle information recognition;

[0056] Step 2: Obtain valid vehicle information from the ETC system;

[0057] When a vehicle enters the ETC lane, the ETC system obtains the vehicle's information through radio frequency recognition and obtains the vehicle's valid information from the system for data matching network;

[0058] The valid information includes license plate number and vehicle model;

[0059] Step 3: Build and train the vehicle ETC data matching network;

[0060] Step 3-1, in the backbone network, the feature extraction structure includes a first convolution layer, four sub-feature extraction units, namely, a first sub-feature extraction unit, a second sub-feature extraction unit, a third sub-feature extraction unit, and a fourth sub-feature extraction unit. The first convolution layer and the four sub-feature extraction units are operated in series;

[0061] Step 3-1-1, the height and width of the input image are halved when passing through the first convolutional layer, and the number of channels of the output image is equal to the number of channels of the convolutional layer. The first convolutional layer consists of a convolution kernel of size 3×3 and stride 2, an h-swish activation function layer and a batch normalization (BN) layer, and the number of channels is 16; the formula of the h-swish activation function layer is as follows:

[0062]

[0063] In the above formula, x represents the input or element of the feature map of a layer in the neural network, and ReLU is the activation function.

[0064] Step 3-1-2, in the sub-feature extraction unit structure, an input feature map is accepted. The feature map is first input into a first inverted residual unit. The purpose of the first inverted residual unit is to extract features and reduce the dimension to achieve feature dimensionality reduction. The reduced feature map will be input into a second inverted residual unit, and the dimension remains unchanged. The feature maps output by the first inverted residual unit and the second inverted residual unit are superimposed or spliced ​​to achieve simple feature fusion in the first stage;

[0065] The specific structure of the first inverse residual unit mentioned above is to convert the input feature map into a tensor. The input tensor will first pass through a convolution layer with a stride of 2 and a convolution kernel size of 3. In this convolution layer, convolution operations and downsampling are performed to reduce the dimension. Then, the dilation factor R1 is set. If the dilation factor R1 is 1, a 1x1 dilated convolution will be performed. The number of output channels of this convolution layer is the product of the dilation factor and the number of channels of the input feature map. If the dilation factor is not 1, no dilated convolution is performed. The two paths finally pass through a 1*1 convolution to convert the number of output channels to C1.

[0066] The specific structure of the second inverse residual unit in the above is that, first, a 1x1 dilated convolution is performed on the input feature map to increase the number of channels of the input feature map. The number of output channels is the product of the dilation factor and the number of channels of the input feature map. Then, a depthwise separable convolution is performed, in which the convolution kernel and the step size are the same. The depthwise separable convolution divides the conventional convolution operation into two independent operations: depthwise convolution and point convolution, which reduces the amount of calculation and model complexity. Finally, a point convolution operation is performed to compress the number of channels from the expanded number of channels back to the original number of output channels.

[0067] The feature maps output by the first inverted residual unit and the second inverted residual unit are superimposed or spliced ​​to realize the simple feature fusion of the first stage, wherein the first sub-feature extraction unit and the second sub-feature extraction unit are superimposed to realize the feature fusion of the first stage. In this process, the dimension of the fused feature map remains unchanged, and the number of channels is added; the third sub-feature extraction unit and the fourth sub-feature extraction unit are spliced ​​to realize the feature fusion of the first stage. In this process, the number of channels of the fused feature map remains unchanged, and the dimension is seamlessly spliced ​​in the same space, and the dimension of the feature map increases; the features extracted by the four sub-feature extraction units are P1, P2, P3, and P4 respectively;

[0068] Step 3-2, construct a feature fusion structure that meets the ETC vehicle information matching, realize the second stage of feature fusion, and fuse the four features P1~P4 in step 3-1 into this structure to realize the linkage of 4 layers of features;

[0069] In the feature fusion structure, only one intermediate node is set in the P3 layer, which accepts feature inputs from the P2, P3, and P4 layers, and outputs the feature information of the three layers of P2, P3, and P4 to the output node of the P3 layer;

[0070] The output node of the P3 layer receives the output of the intermediate node and the features of the P4 layer at the same time. The two features are fused at the output node of the P3 layer and transmitted upward to the output node of the P2 layer.

[0071] The output node of the P2 layer receives the feature input of the two layers, the output of the output node of the feature P2 and P3 layers, and after the features of the two layers are fused, they are output to the output node of the feature P1 layer, completing the last feature fusion with the feature P1;

[0072] The fusion formula is as follows:

[0073]

[0074] In the above formula, O is the fused feature, P is the feature to be fused, w is the weight of feature P during fusion, and the sum of the weights of all features is 1. Due to the different dimensions of different features, some feature maps need to be enlarged / reduced by N times. For example, if it is enlarged by 2 times, N is 2, if it does not need to be enlarged, N is 1, and if it needs to be reduced by 2 times, N is 0.5. For the fusion of the intermediate nodes, the magnification of P2 is 0.5, P3 does not need to be enlarged and is 1, and P4 needs to be enlarged and is 2. The weight of P3 is 1 / (0.5+1+2), the weight of P2 is 0.5 / (0.5+1+2), and the weight of P4 is 2 / (0.5+1+2); M represents the number of feature maps during fusion, and Resize represents the operation of changing the dimension of the feature map;

[0075] In the above feature fusion structure, feature P1 does not participate in the intermediate node fusion, but participates in the final upward transmission feature fusion. This is done to directly and efficiently output shallow semantic features, speed up the prediction speed of the model in actual scenarios, shorten the model's prediction time for vehicles, and meet real-time requirements;

[0076] The purpose of setting only intermediate nodes in the P3 layer is to integrate deep semantic information more comprehensively and efficiently, avoid unnecessary feature fusion, and reduce the complexity of the model network without losing accuracy.

[0077] Reduced the amount of calculation and improved the performance and processing speed of the network;

[0078] Step 3-3, the features output by the output nodes of the P1, P2, and P3 layers are the features that need to be detected in the end. The features output by the output nodes of the P1, P2, and P3 layers are detected by the detection head to obtain the vehicle model information. The license plate information is obtained by using OCR license plate detection, where the vehicle model information and license plate information are both given in the form of prediction boxes; a vehicle information matching module is constructed to obtain the captured vehicle information (license plate and vehicle model) from the ETC system and match it with the vehicle information detected by the network;

[0079] Specifically, when a vehicle enters an ETC toll lane, the vehicle is sensed by the ETC system, and after obtaining the current vehicle's license plate information and vehicle model information from the ETC system, the current vehicle begins to be detected, and the detected license plate information and vehicle model information are matched with the information sensed by the ETC system;

[0080] Furthermore, when the current vehicle is detected, the license plate is detected first, and the first data matching is performed with the vehicle information of the ETC system. After the matching is completely correct, the vehicle model is detected, and then the second data matching is performed to check whether the matching vehicle model is correct. When the two matches are correct, it is allowed to pass;

[0081] Step 3-4: In the first match, the license plate information obtained by the ETC system is compared with the license plate information obtained by the OCR license plate detection in a character-by-character manner, and the loss is calculated at the same time. The loss of license plate character matching is L cha The formula is:

[0082]

[0083] In the above formula, if the characters are the same q i =0, if the characters are different, then q i =1, N represents the total number of license plate characters;

[0084] In the second matching process, the car models are matched, and the classification loss L calculated by the VFL loss function cls for;

[0085]

[0086] Where p is the scalar of the predicted classification score vector. The model will classify the vehicle type when making predictions and output a vector containing the predicted probability distribution of each category. p is a value in this probability distribution. For positive samples in training, q is the intersection-over-union ratio (IoU) between the generated predicted box and the true box. For negative samples in training, q is 0. Ψ is the loss weight of the foreground and background of the image. The foreground is the target object in the image, that is, the specific category you want to detect. The background is all other parts except the target object. It is usually set to 0.2 or 0.5. γ is the weight of each sample. This value is determined according to the characteristics of the data set. Through artificial setting, when the sample distribution is relatively uniform, the same fixed value is set for each sample, such as the same γ for cars, trucks, and buses. If the samples of a certain category are particularly scarce, a higher γ value will be assigned to it.

[0087] In addition, by minimizing the loss function, the CIoU function is used as the regression loss L of the bounding box re ,

[0088]

[0089] The parameters α and v are as follows:

[0090]

[0091] In the above formula, ρ represents the Euclidean distance between the two rectangular boxes of the predicted box and the real box, IoU is the intersection over union ratio, b1 and b2 represent the center points of the predicted box and the real box, c is the diagonal length of the smallest bounding box covering the two bounding boxes, v is a consistency parameter used to measure the relative proportion of the predicted box and the real box, α is a balance parameter, w1, h1, w2 and h2 are the width and height of the predicted box and the real box respectively;

[0092] The total loss function is:

[0093] L=λ·L cha +β·L re +σ·L cls

[0094] In the total loss function of the above formula, λ is the weight of character loss, β is the weight of regression loss, and σ is the weight of classification loss.

[0095] On the other hand, an embodiment of the present invention further provides a vehicle ETC data matching system based on a convolutional neural network, comprising the following units:

[0096] The data acquisition unit is used to use the monitoring camera at the entrance of the toll station to record the real-time video of the vehicle passing through the ETC lane, extract the video frames at a certain time interval, and use these video frames as the data set images for subsequent vehicle information recognition;

[0097] Vehicle information extraction unit, used to obtain valid vehicle information from the ETC system, including license plate number and vehicle model;

[0098] A model building and training unit, used to build and train a vehicle ETC data matching network, including a feature extraction module, a feature fusion module, a vehicle detection module, and a vehicle matching module;

[0099] The feature extraction module includes a convolution layer and n sub-feature extraction units, which operate in series. The features extracted by the n sub-feature extraction units are P1, P2, ..., Pn in sequence. The feature fusion module performs fusion processing on P2-Pn and outputs n-2 features. The vehicle detection module detects the n-2 features output by P1 to obtain vehicle model information and license plate information. Finally, the vehicle matching module performs two matches, namely, license plate matching and vehicle model matching. When the two matches are correct, the vehicle is allowed to pass.

[0100] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions and modifications without departing from the scope of protection of the claims of the present invention, all of which fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A vehicle ETC data matching method based on convolutional neural network, characterized in that: The steps include: Step 1: Use the monitoring camera at the entrance of the toll station to record the real-time video of vehicles passing through the ETC lane, extract video frames at a certain time interval, and use these video frames as data set images for subsequent vehicle information recognition; Step 2, obtain the valid information of the vehicle from the ETC system, including the license plate number and vehicle model; Step 3, construct and train a vehicle ETC data matching network, including a feature extraction module, a feature fusion module, a vehicle detection module, and a vehicle matching module; The feature extraction module includes a convolution layer and n sub-feature extraction units, and the convolution layer and the n sub-feature extraction units are operated in series; the features extracted by the n sub-feature extraction units are P1, P2, ..., Pn in sequence; the feature fusion module performs fusion processing on P2-Pn, outputs n-2 features, and detects P1 and the output n-2 features through the vehicle detection module to obtain vehicle model information and license plate information, and finally performs two matches of license plate matching and vehicle model matching in the vehicle matching module. When the two matches are correct, it is allowed to pass; The value of n is 4. In the feature fusion module, only one intermediate node is set in the P3 layer. The intermediate node receives the feature input from the P2, P3, and P4 layers, and outputs the feature information of the three layers of P2, P3, and P4 to the output node of the P3 layer. The output node of the P3 layer receives the output of the intermediate node and the features of the P4 layer at the same time. The two features are fused at the output node of the P3 layer and transmitted upward to the output node of the P2 layer. The output node of the P2 layer receives two levels of feature input: the output of the P3 layer output node and feature P2. After the two levels of features are fused, they are output to the output node of the feature P1 layer and complete the last feature fusion with feature P1. The features output by the output nodes of the P1, P2, and P3 layers are the features that need to be detected in the end. The vehicle detection module includes a detection head and an OCR. The detection head detects the features output by the output nodes of the P1, P2, and P3 layers to obtain the vehicle model information. The OCR detection is used to obtain the license plate information, where the vehicle model information and the license plate information are both given in the form of prediction boxes.

2. A vehicle ETC data matching method based on a convolutional neural network as claimed in claim 1, characterized in that: The specific processing process of the feature extraction module is as follows; When the input image passes through the convolution layer, the image size is halved in height and width, and the number of channels of the output image is equal to the number of channels of the convolution layer. The convolution layer consists of a convolution kernel of size 3×3 and stride 2, an h-swish activation function layer and a batch normalization layer, and the number of channels is 16. The formula of the h-swish activation function layer is as follows: ； In the above formula, x represents the input or element of the feature map of a certain layer, and ReLU is the activation function; In the sub-feature extraction unit structure, an input feature map is accepted and first input into a first inverted residual unit. The purpose of the first inverted residual unit is to extract features and reduce the dimension to achieve feature dimensionality reduction. The reduced feature map will be input into a second inverted residual unit, and the dimension remains unchanged. The feature maps output by the first inverted residual unit and the second inverted residual unit are superimposed or spliced ​​to achieve simple feature fusion in the first stage.

3. A vehicle ETC data matching method based on a convolutional neural network as claimed in claim 2, characterized in that: The specific structure of the first inverted residual unit is to convert the input feature map into a tensor. The input tensor will first pass through a convolution layer with a stride of 2 and a convolution kernel size of 3. In this convolution layer, convolution operations and downsampling are performed to reduce the dimension. Then the dilation factor R1 is set. If the dilation factor R1 is 1, a 1x1 dilated convolution will be performed. The number of output channels of this dilated convolution is the product of the dilation factor and the number of channels of the input feature map. If the dilation factor is not 1, no dilated convolution is performed. The two paths finally pass through a 1*1 convolution to convert the number of output channels to C1. The specific structure of the second inverse residual unit is as follows: first, a 1x1 dilated convolution is performed on the input feature map to increase the number of channels of the input feature map. The number of output channels is the product of the dilation factor and the number of channels of the input feature map. Then, a depthwise separable convolution is performed, in which the convolution kernel and the step size are the same. The depthwise separable convolution divides the conventional convolution operation into two independent operations: depthwise convolution and point convolution. Finally, a point convolution operation is performed to compress the number of channels from the expanded number of channels back to the original output number of channels.

4. A vehicle ETC data matching method based on a convolutional neural network as claimed in claim 3, characterized in that: The first sub-feature extraction unit and the second sub-feature extraction unit adopt a superposition method to realize the feature fusion of the first stage. During this process, the dimension of the fused feature map remains unchanged, and the number of channels is added; the third sub-feature extraction unit and the fourth sub-feature extraction unit adopt a splicing method to realize the feature fusion of the first stage. During this process, the number of channels of the fused feature map remains unchanged, the dimensions are seamlessly spliced ​​in the same space, and the dimension of the feature map is increased; the features extracted by the four sub-feature extraction units are P1, P2, P3, and P4 respectively.

5. The vehicle ETC data matching method based on convolutional neural network according to claim 1, characterized in that: The fusion formula is as follows: ； In the above formula, O is the fused feature, P is the feature to be fused, w is the weight of feature P during fusion, M is the number of feature maps during fusion, and Resize is the operation of changing the dimension of the feature map.

6. A vehicle ETC data matching method based on convolutional neural network as claimed in claim 1, characterized in that: The total loss function used to train the vehicle ETC data matching network is: ； In the total loss function above, λ is the character matching loss L cha The weight, β is the regression loss L re The weight, σ is the classification loss L cls The weight occupied.

7. A vehicle ETC data matching method based on a convolutional neural network as claimed in claim 6, characterized in that: In the first match, the license plate information obtained by the ETC system is compared with the license plate information obtained by OCR license plate detection in a character-by-character manner, and the loss is calculated at the same time. The license plate character matching loss L cha The calculation formula is: ； In the above formula, if the characters are the same q i =0, if the characters are different, then q i =1, N represents the total number of license plate characters.

8. A vehicle ETC data matching method based on a convolutional neural network as claimed in claim 6, characterized in that: In the second matching process, the car models are matched, and the classification loss L calculated by the VFL loss function cls for: ； Where p is the scalar of the predicted classification score vector, for the positive samples in training, q is the intersection over union (IoU) between the generated predicted box and the true box, for the negative samples in training, q is 0, Ψ is the loss weight of the image foreground and background, and γ is the weight of each sample; Use CIoU function as the regression loss L of the bounding box re for: ； The calculation formulas of parameters α and v are as follows: ； ； In the above formula, ρ represents the Euclidean distance between the two rectangular boxes of the predicted box and the real box, IoU is the intersection over union ratio, b1 and b2 represent the center points of the predicted box and the real box, c is the diagonal length of the minimum bounding box covering the two bounding boxes, v is a consistency parameter used to measure the relative proportion of the predicted box and the real box, α is a balance parameter, w1, h1, w2 and h2 are the width and height of the predicted box and the real box respectively.

9. A vehicle ETC data matching system based on convolutional neural network, characterized in that: The following units are included: The data acquisition unit is used to use the monitoring camera at the entrance of the toll station to record the real-time video of the vehicle passing through the ETC lane, extract the video frames at a certain time interval, and use these video frames as the data set images for subsequent vehicle information recognition; Vehicle information extraction unit, used to obtain valid vehicle information from the ETC system, including license plate number and vehicle model; A model building and training unit, used to build and train a vehicle ETC data matching network, including a feature extraction module, a feature fusion module, a vehicle detection module, and a vehicle matching module; The feature extraction module includes a convolution layer and n sub-feature extraction units, and the convolution layer and the n sub-feature extraction units are operated in series; the features extracted by the n sub-feature extraction units are P1, P2, ..., Pn in sequence; the feature fusion module performs fusion processing on P2-Pn, outputs n-2 features, and detects P1 and the output n-2 features through the vehicle detection module to obtain vehicle model information and license plate information, and finally performs two matches of license plate matching and vehicle model matching in the vehicle matching module. When the two matches are correct, it is allowed to pass; The value of n is 4. In the feature fusion module, only one intermediate node is set in the P3 layer. The intermediate node receives the feature input from the P2, P3, and P4 layers, and outputs the feature information of the three layers of P2, P3, and P4 to the output node of the P3 layer. The output node of the P3 layer receives the output of the intermediate node and the features of the P4 layer at the same time. The two features are fused at the output node of the P3 layer and transmitted upward to the output node of the P2 layer. The output node of the P2 layer receives two levels of feature input: the output of the P3 layer output node and feature P2. After the two levels of features are fused, they are output to the output node of the feature P1 layer and complete the last feature fusion with feature P1. The features output by the output nodes of the P1, P2, and P3 layers are the features that need to be detected in the end. The vehicle detection module includes a detection head and an OCR. The detection head detects the features output by the output nodes of the P1, P2, and P3 layers to obtain the vehicle model information. The OCR detection is used to obtain the license plate information, where the vehicle model information and the license plate information are both given in the form of prediction boxes.

Images