Non-standard electric three-wheeler and four-wheeler row tube number detection method
By spraying the tube number on the rear plate of non-standard electric three- and four-wheelers and using Faster R-CNN model for detection, the problem of difficult to accurately identify and supervise non-standard electric three- and four-wheelers in the existing technology is solved, and the problem of accurate identification and supervision of non-standard electric three- and four-wheelers is realized. Efficient traffic management significantly improves regulatory efficiency and safety.
Patent Information
- Application Number
- CN202411279684.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-27
AI Technical Summary
The existing technology is difficult to accurately identify and supervise non-standard electric three- and four-wheelers, resulting in frequent traffic violations and poses a threat to the safety of people's lives and property.
The column tube number spraying technology is used in combination with computer vision technology, and the tail plate position and column tube number of non-standard electric three- and four-wheeled vehicles are detected through the Faster R-CNN model to achieve accurate identification and supervision.
The supervision efficiency of non-standard electric three- and four-wheeled vehicles has been improved, and the inspection rate of accurately identifying the number of the ducts has reached 94%, effectively reducing the occurrence of traffic accidents and ensuring the safety of people's lives and property.
Smart Images
Figure CN119206732A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent traffic management, and in particular to a method for detecting control numbers of non-standard electric three- and four-wheeled vehicles, aiming to improve the supervision efficiency of non-standard electric three- and four-wheeled vehicles. Background Art
[0002] Over the years, due to the lack of timely standardization in the production, sales, and management of electric three- and four-wheeled vehicles, non-standard electric three- and four-wheeled vehicles that do not meet the conditions for license plate registration have been illegally produced, sold, and put on the road. Most of these vehicles are used in rural areas. Due to their weak traffic safety awareness, drivers often commit traffic violations such as running red lights and illegally carrying passengers, resulting in a high incidence of traffic accidents, causing significant losses to the lives and property of the people. Traffic management departments are in urgent need of a technical means to accurately identify the vehicle information of non-standard electric three- and four-wheeled vehicles to improve regulatory efficiency.
[0003] In recent years, with the rapid development of computer vision technology, image recognition technology has been gradually applied in the traffic management industry. However, the precise supervision of non-standard electric three- and four-wheeled vehicles still faces a series of challenges, especially how to accurately identify and lock non-standard electric three- and four-wheeled vehicles and drivers. At present, the management method for such vehicles is driver identification based on face matching. Since the face recognition camera is only 9 million pixels and uses a large-format, small-cutout method for face matching, the accuracy rate is only about 40%. It is also limited by the fixed installation position of the face camera and the detection frame is only at both ends of the crosswalk. The comprehensive illegal detection ratio is less than 10%, and the actual supervision efficiency is poor.
[0004] In view of these drawbacks, a method is needed to accurately identify the vehicle information of non-standard electric three- and four-wheeled vehicles, which can solve the problem of difficult management of non-standard electric three- and four-wheeled vehicles. This technology has many applications, such as non-standard electric three- and four-wheeled vehicle trajectory analysis, vehicle violation identification, etc. Summary of the invention
[0005] The present invention discloses a method for realizing accurate identification and supervision of non-standard electric three- and four-wheeled vehicles by spraying the serial number in combination with advanced computer vision technology. The method comprises the following steps:
[0006] Step 1: Prepare the registration number for non-standard electric three- and four-wheeled vehicles.
[0007] Step 2: Spray the serial number on the tail plate of non-standard electric three- and four-wheel vehicles.
[0008] Step 3: Collect the image data of electric three- and four-wheeled vehicles passing by collected by the electronic police, and mark the tailgate position to form a tailgate position dataset.
[0009] Step 4: Use the tailgate position dataset to train the first Faster R-CNN model to obtain the tailgate position detection model.
[0010] Step 5: Use the labeled tailgate position dataset to crop the tailgate image, and annotate the column and tube number characters sprayed in the tailgate image one by one to construct the tailgate character dataset.
[0011] Step 6: Use the tail plate character dataset to train and optimize the second Faster R-CNN model to obtain the column number detection model.
[0012] Step 7: Deploy the tailgate position detection model and the vehicle registration number detection model. Predict the images of electric three- and four-wheeled vehicles passing by collected by the electronic police to identify and obtain the vehicle registration numbers.
[0013] Furthermore, the preparation and spraying of the serial number on the tail plate of the non-standard electric three- and four-wheeled vehicles in the described steps 1 and 2 is because the non-standard electric three- and four-wheeled vehicles do not meet the requirements for license registration, so the serial number is sprayed on the tail plate for vehicle management, and the specific steps are:
[0014] Step 1: Prepare an 8-digit serial number, the first two characters are the district code, the third character is the town code, and the last five characters are the serial number code of non-standard electric three-wheeled and four-wheeled vehicles;
[0015] Step 2 uses template spray painting method to spray the serial number on the rear of non-standard electric three- and four-wheeled vehicles. We have actually sprayed the serial number on the rear of 43,500 non-standard electric three- and four-wheeled vehicles.
[0016] Furthermore, in step 3 described above, the tailgate position data set is collected, and the specific steps are as follows:
[0017] Step 3-1 manually screened the vehicle passing image data collected by the electronic police, and retained the images with non-standard electric three- and four-wheeled vehicles and their tail plates and sprayed serial numbers. We actually manually screened out 650 images of non-standard electric three- and four-wheeled vehicles with sprayed serial numbers;
[0018] Step 3-2: Use the labelMG tool to rectangularly mark the tailgate positions of non-standard electric three- and four-wheeled vehicles with the serial numbers to form a data set in VOC format.
[0019] Furthermore, in step 4 described above, the first Faster R-CNN model is trained and optimized, and the specific steps are as follows:
[0020] Step 4-1 Use openCV to simulate random illumination transformation of the non-standard electric three- and four-wheeled vehicles with serial numbers sprayed on them obtained in step 3 to expand the data set;
[0021] In step 4-2, the Faster R-CNN model is trained using the transfer learning method. Because only two categories are distinguished: the background and the tailgate area of electric three- and four-wheeled vehicles with painted serial numbers, we actually achieved the best loss result in the first batch of 200 rounds of training.
[0022] Furthermore, in step 5 described above, a tailboard character dataset is constructed, and the specific steps are as follows:
[0023] Step 5-1: Use step 3 to manually mark and crop the tailgate area of non-standard electric three- and four-wheel vehicles from the passing vehicle images, and remove the tailgate area images with unclear digital painting to avoid duplication of work. We actually obtained 566 tailgate images with clear spray numbers;
[0024] Step 5-2: Use the labelMG tool to rectangularly label each character in the tail plate area to form a VOC format dataset. Our actual labels are the ten numbers 0-9 and the two characters l and b. Since the background class is necessary in the Faster R-CNN model training, we have a total of 13 categories.
[0025] Furthermore, in step 6 described above, the second Faster R-CNN pipe number detection model is trained and optimized, and the specific steps are as follows:
[0026] Step 6-1 Use openCV to simulate random illumination transformation on the tailgate area image with character annotations obtained in step 5 to expand the data set;
[0027] Step 6-2 randomly initializes the weights and performs the first training on the Faster R-CNN model to obtain the column number detection model under a small data set. The reason why we choose the Faster R-CNN model for string detection is that the tailgate area of non-standard electric three- and four-wheeled vehicles has different shapes, colors, and spraying qualities. This model solves the problem that the traditional OCR recognition method has low detection accuracy and the string detection method based on CRNN is difficult to avoid character duplication;
[0028] Step 6-3 uses the control number detection model under the small data set obtained in step 6-2 to detect 2 million vehicle images collected by the electronic police, and writes the prediction results into an XML file to form a VOC format data set, and obtains 9,200 images of non-standard electric three- and four-wheeled vehicle tailgates sprayed with control numbers;
[0029] In step 6-4, the labelMG tool is used to manually check the 9,200 images of non-standard electric three-wheeler and four-wheeler tailboards sprayed with column numbers obtained in step 6-3, delete the blurred images of non-standard electric three-wheeler and four-wheeler tailboards, and modify the character annotations of the erroneously detected ones. Finally, we obtained 7,263 images of non-standard electric three-wheeler and four-wheeler tailboards with clear column numbers sprayed and the dataset composed of them.
[0030] In step 6-5, the VOC format character annotation dataset of non-standard electric three-wheeler and four-wheeler tailboards obtained in step 5 and step 6-4 is used, and the openCV is used to simulate random illumination transformation to further expand the dataset for the second training of the Faster R-CNN model. After 200 rounds of training, our loss value reached 0.053.
[0031] Furthermore, in step 7, the tailboard position detection model and the column number detection model are deployed. The specific steps are as follows:
[0032] In step 7-1, connect to the remote Oracle vehicle passing record database and the illegal record database to obtain the vehicle passing time, the vehicle passing intersection, and the URL data of the recorded vehicle passing images;
[0033] In step 7-2, read the URL one by one and pass the vehicle passing image data into the tailboard position detection model;
[0034] In step 7-3, discard the vehicle passing images without tailboard position detection results, crop the vehicle passing images with the anchor box with the highest score for the vehicle passing images with tailboard position detection results, and pass them into the column number detection model;
[0035] In step 7-4, discard the anchor boxes with scores lower than 0.8 for the results of the column number detection model, and use the non-maximum suppression method for the results with an intersection over union greater than 50% in the remaining anchor boxes to eliminate the problem of repeated predicted characters;
[0036] In step 7-5, sort the results of the column number detection model according to the upper left position of the anchor box, and map the results to column numbers according to the anchor box labels;
[0037] In step 7-6, since the actually sprayed column numbers start with the string "L Bei", and the third digit represents the town or street with a number range from 1 to 8, and the fourth to eighth digits are all numbers, so we perform a forced match on the prediction results of step 7-5 and write the data that meets the format conditions back to the database. Our final column number detection accuracy is 94%.
[0038] In summary, the present invention has the following beneficial technical effects: The present invention proposes a method for painting the serial number on the tail plate of a non-standard electric three- and four-wheeled vehicle and using real vehicle passing data to train the first Faster R-CNN model for detecting the tail plate area in the image and the second first Faster R-CNN model for identifying the specific content of the serial number painted on the tail plate. By accurately identifying the serial number of non-standard electric three- and four-wheeled vehicles and conducting supervision by traffic management departments, the occurrence of accidents involving such vehicles can be effectively reduced, and the safety of people's lives and property can be protected. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a flow chart of the method for identifying the column number in the example of this application;
[0040] Figure 2 is a schematic diagram of the output display of the tailgate position detection model in an embodiment of the present application;
[0041] Figure 3 It is a schematic diagram of the output display of the column tube number detection model in the embodiment of the present application;
[0042] Figure 4 It is a flowchart of the actual application of the method for detecting the serial number of non-standard electric three- and four-wheeled vehicles in the embodiment of the present application; DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following Figure 1-4 The present invention is described in further detail.
[0044] The embodiment of the present application discloses a method for detecting the serial number of a non-standard electric three-wheeled or four-wheeled vehicle, which is used to detect and identify the serial number sprayed on the tail plate of a non-standard electric three-wheeled or four-wheeled vehicle. The identification method can accurately detect the serial number of a non-standard electric three-wheeled or four-wheeled vehicle, thereby further determining the identity information of the vehicle owner, and is used for road traffic safety management of such vehicles. The identification method uses two Faster R-CNN model recognitions to detect the position and serial number of the tail plate of a non-standard electric three-wheeled or four-wheeled vehicle, respectively.
[0045] Faster R-CNN stands for Faster Region-based Convolutional Neural Network, a deep learning model based on convolutional neural networks that enables end-to-end training and prediction from original images to target detection results. Instead of relying on external components to generate candidate regions, it introduces a region proposal network that automatically generates high-quality candidate regions from images. Faster R-CNN includes convolutional feature extraction, region proposal network, RoIPooling, as well as classification and bounding box regression. Finally, the category label of each candidate region and its corresponding bounding box coordinates are output, and redundant detections are reduced through non-maximum suppression.
[0046] Reference Figure 1 , the method comprises the following steps:
[0047] S1, prepare and spray the serial number on the tail plate of non-standard electric three- and four-wheel vehicles.
[0048] Specifically, the electric three- and four-wheeled vehicle registration numbers should be reasonably compiled based on the number of non-standard electric three- and four-wheeled vehicles in the jurisdiction of the traffic management department and the distribution of towns and streets. For example: compile an 8-digit registration number, the first two characters are the district and county code, the third character is the town and street code, and the last 5 digits are the non-standard electric three- and four-wheeled vehicle serial number code. Refer to the compilation format "L Bei 510186", where "L Bei" represents the filing jurisdiction, the third digit 5 represents the town and street, and 4 to 8 represent the serial number of the filed vehicle. After the compilation is completed, spray paint is used on the tailgate of the non-standard electric three- and four-wheeled vehicle. The size of each sprayed character is 7.5 cm wide and 17 cm high.
[0049] S2, collects non-standard electric three- and four-wheeled vehicle passing images collected by electronic police, annotates them into VOC format datasets using labelImg software, and imports them into the first Faster R-CNN model for training to obtain the tailgate position detection model.
[0050] Specifically, the addresses of the images of non-standard electric three- and four-wheeled vehicles passing by collected by the electronic police are stored in the database in the form of URLs. Each URL is a link to an image of a non-standard electric three- and four-wheeled vehicle passing by. We read the URL links one by one and save the images of non-standard electric three- and four-wheeled vehicles passing by with the serial numbers sprayed on them to the local computer hard disk. This completes the collection of images of non-standard electric three- and four-wheeled vehicles passing by collected by the electronic police. Our model training adopts the method of transfer learning. Since this step only needs to detect one category of electric tricycle tailgates, it is sufficient to collect 600 to 1,000 images of electric vehicle tailgates for training.
[0051] Furthermore, the labelImg software was used to annotate the collected non-standard electric three- and four-wheeled vehicle passing images in VOC format. Although the Faster R-CNN model also calculates the image background as a category, in fact, in the labelImg software, only the tailgate area needs to be annotated with a rectangular box. This completes the construction of the electric vehicle tailgate position dataset.
[0052] Further, the data set is imported into the first Faster R-CNN model for training. In an implementation example, the training of the Faster R-CNN model includes image enhancement, image transformation, definition of optimizer, definition of loss function and training process. In the image enhancement part, since the data in the collected data set are all in real scenes, openCV is used to randomly simulate the passing of non-standard electric three- and four-wheeled vehicles under different lighting conditions. Expanding the data set by 3 times can improve the generalization of the model; in the image transformation part, the three-channel RGB non-standard electric three- and four-wheeled vehicle passing images are converted into tensors; in the optimizer definition part, the stochastic gradient descent method is used, and the learning rate is set to 0.005; in the loss function definition part, Faster The output of the R-CNN model in training mode includes 4 categories, namely classification loss (classifier), bounding box regression loss (box_reg), objectness loss (objectness), and RPN bounding box regression loss (rpn_box_reg). In the tailgate position detection model, classification loss and bounding box regression loss are the most important. The classification loss represents the accuracy of the model in detecting whether the tailgate of a non-standard electric three- or four-wheeled vehicle is the background or the tailgate area. The cross entropy loss is used to calculate this value. The bounding box regression loss represents the accuracy of the model in detecting the tailgate position of non-standard electric three- or four-wheeled vehicles. In the training process, the batch size, that is, the number of electric three- or four-wheeled vehicle images read by the model at one time, is set to 2, and the total number of training rounds is set to 10. In fact, we used the transfer learning method for the tailgate position detection model and achieved the best results in the first of 200 training rounds. After training, you can save only the training parameters or the entire model with parameters.
[0053] S3, use the VOC labels obtained in the previous step to crop the images of non-standard electric three- and four-wheeled vehicles collected by the electronic police to obtain tailgate images, use labelImg software to annotate the characters in the tailgate, and import the second FasterR-CNN model for training to obtain the column number detection model.
[0054] Specifically, the labels in the VOC format dataset of non-standard electric three- and four-wheeled vehicle passing images obtained in S2 record the upper left corner fixed-point coordinates and length and width of the tailgate position. We use the label information to crop the non-standard electric three- and four-wheeled vehicle passing images to obtain the tailgate image with the column number sprayed on it.
[0055] Furthermore, the cropped tailgate image was annotated in VOC format using labelImg software. The annotation categories were two English characters "l" and "b" and 10 numbers from 0 to 9. Together with the background, there were a total of 13 categories, among which "l" and "b" corresponded to "L" and "spare" in the real tube number respectively, and the labels were 11 and 12, 9 from 1 to 9 corresponded to the nine labels of 1-9, label 0 corresponded to the background, and label 10 corresponded to the number 0 in the tube number, thus completing the construction of the tube number detection model dataset.
[0056] Furthermore, the dataset is imported into the second Faster R-CNN model for training. Image enhancement, image transformation, definition of optimizer, and definition of loss function are described in step S2 to obtain a column number detection model.
[0057] Furthermore, the second Faster R-CNN model has 13 categories, and each correctly predicted result has at least 8 characters, character classifications, and bounding box information, so a larger dataset is needed to train this model. In order to simplify the dataset construction process, we input 2 million vehicle images collected by the electronic police into the tailgate position detection model for prediction, cut out the tailgate images, and then use the column number detection model for prediction, obtaining 9,200 labeled tailgate images and the labels corresponding to each tailgate image. By manually screening using labelImg software, we finally obtained 7,263 cleaned tailgate images and their datasets. Repeating the training process of the second Faster R-CNN model, the final column number detection model can be obtained.
[0058] S4, combined with the tailgate position detection model and the control number detection model, is deployed to predict the images of non-standard electric three- and four-wheeled vehicles passing by collected by the electronic police to identify and obtain the control numbers of non-standard electric three- and four-wheeled vehicles.
[0059] Specifically, the training process of the tailgate position detection model and the tube number detection model has been completed in steps S1-S3. Figure 4The flowchart of the practical application of the non-standard electric three- and four-wheeled vehicle control number detection method shown in the figure illustrates step S4 and the practical application. The violation data and violation images of non-standard electric three- and four-wheeled vehicles collected by the electronic police are stored in the database, including the violation time, violation type, violation location and the URL corresponding to the three violation photos, of which the first violation photo is the image of the non-standard electric three- and four-wheeled vehicle before crossing the stop line, and the tailgate and control number features are the clearest. By reading the first violation photo and passing it into the tailgate position detection model and the control number detection model for prediction, the final control number is obtained and then written back to the database for the traffic management department to implement various management measures such as interviews, warnings, and penalties, so as to achieve the ultimate goal of standardizing the driving order of non-standard electric three- and four-wheeled vehicles and protecting the lives and property of the people.
[0060] Furthermore, the output of the Faster R-CNN model is the target classification, bounding box information, and score. After the vehicle image is input into the tailgate position detection model, the target detection results may be 0 to N. For the detection results of 0 targets, the image should be discarded directly and no longer input into the column number detection model to save computing resource overhead. For the detection results of 1-N targets, Figure 4 In the actual application process shown, only the results with a score greater than 0.8 and the highest score in the same image are retained (such as Figure 2 ).
[0061] Furthermore, the bounding box information obtained by the above-mentioned tailgate position detection model is used to crop the illegal images of electric three- and four-wheeled vehicles, and the tailgate images are obtained and passed into the column control number prediction model for prediction. The prediction results have multiple classification labels and bounding boxes and scores corresponding to the classification labels. The Faster R-CNN model may have multiple predictions with different bounding boxes, labels, and scores for the same target. We only retain the prediction results with a score greater than 0.9, and use the non-maximum suppression method to eliminate the prediction results with an intersection-over-union ratio greater than 50% to obtain the column control number prediction label. The labels are sorted by row according to the X-axis value of the upper-left corner coordinate of the bounding box corresponding to each label, and mapped according to label 11 corresponding to "L", label 12 corresponding to "standby", label 10 corresponding to 0, and labels 1 to 9 corresponding to numbers 1-9 respectively, to obtain the column control number (such as Figure 3 ).
[0062] Furthermore, the prediction results of the control number, the violation time, violation type, violation location and the URLs corresponding to the three violation photos of the non-standard electric three- and four-wheeled vehicles are written back to the database for the traffic management department to implement relevant measures.
[0063] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.
Claims
1. A method for detecting the serial number of non-standard electric three- and four-wheeled vehicles, the method comprising the following steps: Step 1: Prepare the registration number for non-standard electric three- and four-wheeled vehicles. Step 2: Spray the serial number on the tail plate of non-standard electric three- and four-wheel vehicles. Step 3: Collect the image data of electric three- and four-wheeled vehicles passing by collected by the electronic police, and mark the tailgate position to form a tailgate position dataset. Step 4: Use the tailgate position dataset to train the first Faster R-CNN model to obtain the tailgate position detection model. Step 5: Use the labeled tailgate position dataset to crop the tailgate image, and annotate the column and tube number characters sprayed in the tailgate image one by one to construct the tailgate character dataset. Step 6: Use the tail plate character dataset to train and optimize the second Faster R-CNN model to obtain the column number detection model. Step 7: Deploy the tailgate position detection model and the vehicle registration number detection model. Predict the images of electric three- and four-wheeled vehicles passing by collected by the electronic police to identify and obtain the vehicle registration numbers.
2. A method for detecting the serial number of a non-standard electric three- and four-wheeled vehicle according to claim 1, characterized in that: The preparation and spraying of the serial number on the tail plate of non-standard electric three- and four-wheeled vehicles in the described steps 1 and 2 is because non-standard electric three- and four-wheeled vehicles do not meet the requirements for license registration. Therefore, the serial number is sprayed on the tail plate for vehicle management. The specific steps are: Step 1: Prepare an 8-digit serial number, the first two characters are the district code, the third character is the town code, and the last five digits are the serial number code of non-standard electric three- and four-wheeled vehicles; Step 2: Use template spray painting method to spray the serial number on the rear of non-standard electric three- and four-wheeled vehicles. We actually sprayed the serial number on the rear of 43,500 non-standard electric three- and four-wheeled vehicles.
3. A method for detecting the serial number of a non-standard electric three-wheeled or four-wheeled vehicle according to claim 1, characterized in that: In step 3 described above, the tailgate position data set is collected, and the specific steps are as follows: Step 3-1: Manually screen the vehicle passing image data collected by the electronic police, and retain the images with non-standard electric three- and four-wheeled vehicles and their tail plates and sprayed serial numbers. We actually manually screened out 650 images of non-standard electric three- and four-wheeled vehicles with sprayed serial numbers; Step 3-2: Use the labelImg tool to rectangularly mark the tailgate positions of non-standard electric three- and four-wheeled vehicles with the serial numbers to form a data set in VOC format.
4. A method for detecting the serial number of a non-standard electric three- and four-wheeled vehicle according to claim 1, characterized in that: In step 4 described above, the first Faster R-CNN model is trained and optimized. The specific steps are: Step 4-1: Use openCV to simulate random illumination transformation of the non-standard electric three- and four-wheeled vehicles with serial numbers sprayed on them obtained in step 3 to expand the data set; Step 4-2: Use the transfer learning method to train the Faster R-CNN model. Because we only distinguish between the background and the tailgate area of electric three- and four-wheeled vehicles with sprayed serial numbers, we actually achieved the best loss result in the first batch of 200 rounds of training.
5. A method for detecting the serial number of a non-standard electric three- and four-wheeled vehicle according to claim 1, characterized in that: In step 5 described above, the tailboard character dataset is constructed, and the specific steps are as follows: Step 5-1: Use step 3 to manually mark and crop the tailgate area of non-standard electric three- and four-wheel vehicles from the passing vehicle images, and remove the tailgate area images with unclear digital painting to avoid duplication of work. We actually obtained 566 tailgate images with clear spray numbers; Step 5-2: Use the labelMG tool to rectangularly label each character in the tail plate area to form a VOC format dataset. Our actual labels are the ten numbers 0-9 and the two characters l and b. Since the background class is necessary in the Faster R-CNN model training, we have a total of 13 categories.
6. A method for detecting the serial number of a non-standard electric three-wheeled or four-wheeled vehicle according to claim 1, characterized in that: In step 6 described above, the second Faster R-CNN pipe number detection model is trained and optimized. The specific steps are as follows: Step 6-1: Use openCV to simulate random illumination transformation on the tailgate area image with character annotations obtained in step 5 to expand the data set; Step 6-2: Randomly initialize the weights and perform the first training on the Faster R-CNN model to obtain the column number detection model under the small data set. The reason why we choose the Faster R-CNN model for string detection is that the tailgate area of non-standard electric three- and four-wheel vehicles has different shapes, colors, and spraying quality. This model solves the problem that the traditional OCR recognition method has low detection accuracy and the string detection method based on CRNN is difficult to avoid character duplication; Step 6-3: Use the control number detection model under the small data set obtained in step 6-2 to detect 2 million vehicle images collected by the electronic police, and write the prediction results into an XML file to form a VOC format data set, and obtain 9,200 images of non-standard electric three- and four-wheeled vehicle tailgates sprayed with control numbers; Step 6-4: Use the labelMG tool to manually check the 9,200 non-standard electric three- and four-wheeled vehicle tailgate images with serial numbers obtained in step 6-3, delete the blurred non-standard electric three- and four-wheeled vehicle tailgate images, and modify the incorrectly detected character labels. We finally obtained 7,263 non-standard electric three- and four-wheeled vehicle tailgate images with clear serial numbers and the dataset they constitute. Step 6-5: Use the VOC format non-standard electric three- and four-wheeled vehicle tailgate character annotation dataset obtained in steps 5 and 6-4, and use openCV to simulate random illumination changes, further expand the dataset and train the Faster R-CNN model for the second time. After 200 rounds of training, our loss value reached 0.
053.
7. A method for detecting the serial number of a non-standard electric three-wheeled or four-wheeled vehicle according to claim 1, characterized in that: In step 7 described above, the tailgate position detection model and the tube number detection model are deployed. The specific steps are as follows: Step 7-1: Connect to the remote Oracle vehicle passing record database and violation record database to obtain the passing time, passing intersection and URL data of the passing image; Step 7-2: Read the URLs one by one and transfer the vehicle image data to the tailgate position detection model; Step 7-3: discard the vehicle images without tailgate position detection results, obtain the highest-scoring anchor frame for the vehicle images with tailgate position detection results, crop the vehicle images, and pass them into the column number detection model; Step 7-4: Discard the anchor boxes with scores lower than 0.8 in the results of the column pipe number detection model, and use the non-maximum suppression method for the results with an intersection over union greater than 50% among the remaining anchor boxes to eliminate the problem of duplicate predicted characters; Step 7-5: Sort the results of the column pipe number detection model according to the upper left corner position of the anchor box, and map the results to the column pipe number according to the anchor box label; Step 7-6: Since the column pipe numbers we actually spray start with the string "L Bei", and the third digit represents the town or street with a number range of 1-8, and the fourth to eighth digits are all numbers, we perform a forced match on the prediction results of Step 7-5 and write the data that meets the format conditions back to the database. Our final accuracy rate for column pipe number detection is 94%.