Electric bicycle illegal actor identification method based on feature vector extraction and comparison

Abstract

The invention belongs to the field of intelligent traffic management, and provides an electric bicycle illegal actor identification method based on feature vector extraction and comparison. The method comprises the following steps: acquiring illegal and passing images of an electric bicycle shot by an electronic police and a public security monitor, and respectively forming a background data set and a foreground data set; training an optimized Faster R-CNN model for target detection of an electric bicycle and a driver thereof, and training an OSNet full-scale feature extraction model for feature extraction; extracting feature vectors from the images and storing the feature vectors in a database; illegal and passing images are matched according to time and places, and the same vehicle and driver are determined by calculating the Euclidean distance between the feature vectors; and finally, the illegal actor is confirmed in combination with a face recognition technology. According to the method, a solution for effectively managing illegal behaviors of the electric bicycle is provided for a traffic management department.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic management, and in particular to a method for identifying electric bicycle offenders based on feature vector extraction and comparison, aiming to improve the efficiency of supervision of electric bicycle violations. Background Art

[0002] Currently, there are three main methods for identifying e-bike offenders: the first uses pedestrian red light cameras installed at both ends of crosswalks to directly capture three images of the e-bike's front and the process of running the red light. These images are then used to identify the offender through facial recognition. The second method uses electronic police cameras installed on non-motorized vehicle lanes to capture vehicle violations and identify license plates to identify offenders. The third method uses RFID radio frequency combination electronic police cameras installed on non-motorized vehicle lanes to detect violations by vehicles with new national standard license plates equipped with radio frequency modules. All three methods have significant drawbacks. Firstly, they require the installation of new electronic police equipment at both ends of non-motorized vehicle lanes or crosswalks, resulting in high deployment costs. Secondly, the second license plate detection method and the third RFID identification method require the e-bike to be properly registered. In reality, the standard registration rate for e-bikes in most Chinese cities is currently very low, and changes in ownership due to vehicle sales are not properly registered. As a result, the percentage of offenders that can be identified is less than 20%, resulting in poor regulatory effectiveness.

[0003] To address these drawbacks, there is an urgent need for a technical method that does not require the construction of new road sensing equipment and can accurately identify electric bicycle offenders, so as to solve the existing problems in the management of electric bicycle violations and improve the level of urban road traffic safety management. Summary of the Invention

[0004] In response to the above-mentioned cost, management and technical issues, the present invention combines advanced computer vision technology to propose a method for identifying electric bicycle offenders based on feature vector extraction and comparison. The method includes the following steps:

[0005] Step 1: Collect the electric bicycle violation image data collected by the electronic police, obtain the background image of the passing vehicle, and annotate the electric bicycles and drivers in it to form the electric bicycle background dataset.

[0006] Step 2: Collect the electric bicycle passing data collected by the public security monitoring and binocular camera within 200 meters around the electronic police in step 1 above, obtain the passing vehicle foreground image, and annotate the electric bicycles and drivers in it to form the electric bicycle foreground dataset.

[0007] Step 3: Use the electric bicycle background dataset and electric bicycle foreground dataset obtained in steps 1 and 2 above to train and optimize the Faster R-CNN target detection model to obtain the electric bicycle and driver target detection model, which is used to detect the target electric bicycle and driver in the violation image.

[0008] Step 4: Use the electric bicycle background dataset and electric bicycle foreground dataset obtained in steps 1 and 2 above to train and optimize the OSNet full-scale feature extraction model to obtain a feature extraction model for matching the foreground and background images of the same electric bicycle and driver.

[0009] Step 5: Use the electric bicycle and driver target detection model to extract the background of electric bicycles and drivers in the violation photos captured by the electronic police, use the feature extraction model to extract the background feature vectors of electric bicycles and drivers, and store the violation time, violation location, violation photos, and feature vectors in the electric bicycle violation capture database table.

[0010] Step 6: Use the electric bicycle and driver target detection model to extract the foreground of electric bicycles and drivers in the electric bicycle passing photos collected by public security monitoring and binocular cameras, use the feature extraction model to extract the foreground feature vectors of electric bicycles and drivers, and store the passing time, passing location, passing photos, and feature vectors in the electric bicycle and driver foreground passing database table.

[0011] Step 7: Based on the violation time and location in the electric bicycle violation capture database table, filter the electric bicycle and driver foreground passing vehicles from the electric bicycle and driver foreground passing vehicle database table, whose passing time is within 5 minutes before and after the violation time and whose passing location is within 200 meters around the electronic police security surveillance and captured by binocular cameras.

[0012] Step 8: Use the Euclidean Reed distance to calculate the electric bicycle violation data in step 7 and the feature vectors of the electric bicycle and driver foreground vehicles selected in step 7, and select the electric bicycle and driver foreground vehicle with the smallest Euclidean Reed distance to obtain the violation photo and foreground vehicle pair.

[0013] Step 9: Use the face recognition model to identify the identity of the electric bicycle and the driver in the foreground vehicle obtained in step 8, and confirm the offender.

[0014] Furthermore, in the described steps 1 and 2, the illegal electric bicycle image data and the foreground image of the passing vehicle are collected and annotated to form a data set. The specific steps are as follows:

[0015] Step 1-1: Use the X-AnyLabeling tool to annotate the rectangular area containing the target e-bike and driver in the e-bike violation image. The annotation information includes the horizontal and vertical coordinates of the upper left corner of the rectangular area relative to the entire image, the width and height of the area, and the identification number. We actually screened and annotated 15,000 e-bike violation images.

[0016] Step 2-1: Use the X-AnyLabeling tool to rectangularly label the rectangular area containing the target electric bicycle and driver in the electric bicycle passing image. The labeling information includes the horizontal and vertical coordinates of the upper left corner of the rectangular area where the electric bicycle and driver are located relative to the entire image, the width, height and identification number of the area. Among them, the electric bicycle and driver foreground collected in this step and the electric bicycle and driver background collected in step 1 are the same electric bicycle, the same driver, and the same continuous passing, and the labeling numbers are the same. We actually screened and labeled 15,000 electric bicycle passing images, and corresponded them to the 15,000 electric bicycle violation images collected and labeled in step 1-1.

[0017] Furthermore, in step 3 described above, the electric bicycle and driver target detection model is trained and optimized, and the specific steps are as follows:

[0018] Step 3-1: Use Python programming language to build the Faster R-CNN object detection model structure.

[0019] Step 3-2: Combine the 15,000 images of electric bicycle violations and the 15,000 images of electric bicycle passing vehicles obtained in steps 1 and 2 into an electric bicycle and driver target detection dataset consisting of a total of 30,000 images.

[0020] Step 3-3: Expand the electric bicycle and driver target detection dataset obtained in step 3-2 to 120,000 images by simulating random lighting and angle transformation using OpenCV.

[0021] Step 3-4: Randomly initialize the weights and input the Faster R-CNN target detection model obtained in step 3-1.

[0022] Step 3-5: Read the e-bike and rider object detection dataset obtained in Step 3-2 and input it into the Faster R-CNN object detection model for training. After 200 rounds of training, our loss value reached 0.032.

[0023] Furthermore, in step 4 described above, the feature extraction model is trained and optimized, and the specific steps are as follows:

[0024] Step 4-1: Use Python programming language to build the OSNet full-scale feature extraction model structure.

[0025] Step 4-2: The 15,000 images of electric bicycle violations and the 15,000 images of electric bicycle passing vehicles obtained in steps 1 and 2 were expanded to 60,000 images respectively by simulating random lighting, angle transformation and adding noise through openCV.

[0026] Step 4-3: Randomly initialize the weights and input the OSNet full-scale feature extraction model obtained in step 4-1.

[0027] Step 4-4: The 60,000 expanded e-bike violation images and 60,000 e-bike passing images obtained in Step 4-2 were fed into the OSNet full-scale feature extraction model for training. Each e-bike violation image corresponds to an e-bike passing image, provided they have the same label number. This means that a set of e-bike violation images and e-bike passing images represents the same consecutive e-bike passings by the same driver. We set the initial learning rate to 0.001 and used cosine annealing to adjust the learning rate based on the cosine function. Each training batch size was 64, and using triplet loss as the loss function, we achieved a final loss value of 0.012, resulting in an accuracy of 92.31% on the validation set.

[0028] Furthermore, in step 5, the background feature vectors of the electric bicycle and the driver are extracted and stored in the electric bicycle illegal capture database table. The specific steps are as follows:

[0029] Step 5-1: Connect to the remote Oracle violation record database, obtain the time and location of the electric bicycle violation and the URL address of the violation photo, and read the violation photo according to the URL.

[0030] Step 5-2: Use the electric bicycle and driver target detection model obtained in step 3 to detect the illegal electric bicycle and driver in the illegal photo to obtain the background image of the illegal electric bicycle and driver.

[0031] Step 5-3: Use the feature extraction model obtained in step 4 to perform feature extraction on the background images of the illegal electric bicycle and the driver obtained in step 5-2 to obtain background feature vectors of the illegal electric bicycle and the driver.

[0032] Step 5-4: Store the electric bicycle violation time, violation location, and URL address of the violation photo obtained in step 5-1 and the background feature vector of the illegal electric bicycle and driver obtained in step 5-3 into the electric bicycle violation capture database table.

[0033] Furthermore, in step 6, the foreground feature vectors of the electric bicycle and the driver are extracted and stored in the electric bicycle and the driver foreground vehicle database table. The specific steps are:

[0034] Step 6-1: Connect to the remote Oracle vehicle passing record database, obtain the time and location of the electric bicycle passing and the URL address of the passing photo, and read the passing photo according to the URL.

[0035] Step 6-2: Use the electric bicycle and driver target detection model obtained in step 3 to detect the electric bicycle and driver in the passing vehicle photo to obtain the foreground image of the electric bicycle and driver.

[0036] Step 6-3: Use the feature extraction model obtained in step 4 to perform feature extraction on the foreground images of the electric bicycle and the driver obtained in step 6-2 to obtain feature vectors of the foreground of the electric bicycle and the driver.

[0037] Step 6-4: Store the electric bicycle passing time, passing location, and URL address of the passing photo obtained in step 6-1 and the electric bicycle and driver foreground feature vectors obtained in step 6-3 into the electric bicycle and driver foreground passing database table.

[0038] Furthermore, in step 7, the electric bicycle and driver foreground vehicles are filtered according to the records in the electric bicycle illegal capture database table. The specific steps are:

[0039] Step 7-1: Count the addresses of electronic police, public security monitoring, and binocular cameras included in all projects, and map the electronic police addresses to public security monitoring in a 1:N ratio. That is, one electronic police device corresponds to multiple public security monitoring and binocular cameras. The corresponding condition is that the actual geographical location of the public security monitoring and binocular cameras is within 200 meters of the actual geographical location of the corresponding electronic police, forming an electronic police and public security monitoring pair record table.

[0040] Step 7-2: Use the SQL intersection method to connect the electric bicycle violation capture data table obtained in step 5 and the electric bicycle and driver foreground passing vehicle data table obtained in step 6. The intersection condition is that the time when the electric bicycle and driver foreground passed the electric bicycle is within 5 minutes before or after the electric bicycle violation capture time, and the electronic police and public security monitoring and binocular camera pair of the same violation data is in the electronic police and public security monitoring pair record table obtained in step 7-1.

[0041] Furthermore, in step 8 described above, the illegal photo and the foreground vehicle pair are obtained, and the specific steps are as follows:

[0042] Step 8-1: Calculate the Euclidean Reed distance between the background feature vectors of illegal electric bicycles and drivers and the foreground feature vectors of electric bicycles and drivers obtained from the intersection of the electric bicycle violation capture database table obtained in step 7-2 and the electric bicycle and driver foreground passing vehicle data table.

[0043] Step 8-2: For the same electric bicycle violation record, compare the Euclidean distances between the background feature vectors of the illegal electric bicycle and the driver, and the foreground feature vectors of the electric bicycle and the driver obtained in 8-1, and select the electric bicycle and the foreground vehicle with the smallest Euclidean distance to obtain the violation photo and foreground vehicle pair.

[0044] Furthermore, in step 9 described above, the identity of the offender is confirmed, and the specific steps are as follows:

[0045] Step 9-1: Take out the foreground images of the electric bicycle and the driver obtained by performing target detection on the electric bicycle and the driver in the passing vehicle photo in step 6-2.

[0046] Step 9-2: Input the foreground image of the electric bicycle and the driver into the face recognition module to obtain the driver's identity information and confirm the identity of the offender.

[0047] In summary, compared with the prior art, the present invention has the following beneficial technical effects: utilizing the existing electronic police installed on the road that can cover non-motorized vehicle lanes to capture electric bicycle violations, utilizing public security monitoring around the geographical location of the electronic police, and binocular cameras to capture images of electric bicycles passing by from the front, without the need for new front-end equipment, thus saving capital costs. At the same time, this method associates the back of the electric bicycle violation image with the front of the electric bicycle passing by through feature extraction, and determines the offender through face recognition, thus solving the problem of low installation rate of electric bicycle license plates and inaccurate identification of offenders due to license plate recognition caused by private transactions of vehicles. By using existing equipment to accurately identify electric bicycle offenders and handing over the violation data to the traffic management department for law enforcement management, the occurrence of accidents involving such vehicles can be effectively reduced, and the safety of people’s lives and property can be protected. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of a specific implementation method in the example of this application;

[0049] Figure 2 This is a schematic diagram of the system architecture in the embodiment of the present application;

[0050] Figure 3 Schematic diagram of OSNet full-scale feature extraction and determination of offenders by using Euclidean Reed distance to compare similarities in an embodiment of the present application; DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following Figure 1 The present invention is described in further detail.

[0052] The embodiment of the present application discloses a method for identifying electric bicycle offenders based on feature vector extraction and comparison. The method aims to utilize existing urban traffic monitoring resources, and through a technical solution that combines computer vision and deep learning technology, concatenates illegal photos and portrait photos taken from different types of camera perspectives to achieve efficient and accurate identification of electric bicycle offenders, thereby improving urban traffic management efficiency and reducing the traffic accident rate caused by electric bicycle violations.

[0053] Reference Figure 1 , the method comprises the following steps:

[0054] S1, collects images of electric bicycle violations taken by electronic police and annotates them to form an electric bicycle background dataset.

[0055] Specifically, the X-AnyLabeling tool is used to rectangularly label the area containing the target electric bicycle and the driver in the electric bicycle violation image. The labeling of each electric bicycle background image includes the horizontal and vertical coordinates of the upper left corner of the rectangular area where the electric bicycle and the driver are located relative to the entire image, the width, height and identification number of the area.

[0056] Furthermore, the electronic police installed at the intersection can accurately record illegal photos such as electric bicycles running red lights and not wearing safety helmets. After recording the violation, the electronic police will upload the violation time, location, and type of violation to the Oracle database through the terminal device at the intersection. At the same time, the violation photos will be saved and a URL address will be generated and written into the database.

[0057] Furthermore, we need to read the URL address corresponding to the electric bicycle violation time, violation location, violation behavior, and violation photos from the Oracle database that records vehicle violation data. We use HTTP requests to access the URL address link corresponding to the violation photo and read the violation photo.

[0058] Furthermore, the identification number marked in the electric bicycle violation image marked with the rectangle by the X-AnyLabeling tool is unique for each violator.

[0059] S2 collects images of electric bicycles passing by taken by public security surveillance and binocular cameras, and annotates them to form an electric bicycle foreground dataset.

[0060] Specifically, the X-AnyLabeling tool is used to rectangularly label the rectangular area containing the target electric bicycle and the driver in the electric bicycle passing image. The labeling information includes the horizontal and vertical coordinates of the upper left corner of the rectangular area where the electric bicycle and the driver are located relative to the entire image, the width, height and identification number of the area.

[0061] Furthermore, the public security monitoring and binocular camera equipment installed near the intersection can record photos of electric bicycles, pedestrians, etc. passing through the shooting area. After recording the photos, the equipment will upload the images of electric bicycles and pedestrians passing by to the Oracle database, and at the same time save the photos and generate a URL address and write it into the database.

[0062] We then need to filter out the time, location, and URL of the electric bicycle's passing time from the Oracle database that records electric bicycle and pedestrian traffic, which corresponds to the photos of electric bicycle violations captured by the electronic police. We then access the URL corresponding to the violation photo through an HTTP request and retrieve the photo. This will yield a frontal photo of the offending electric bicycle.

[0063] Furthermore, the identification number marked in the electric bicycle passing image marked by the X-AnyLabeling tool's rectangle corresponds to the identification number marked in the electric bicycle violation image, and the numbers of the same consecutive passing images of the same violator riding the same electric bicycle are consistent.

[0064] S3 trains and optimizes the Faster R-CNN target detection model to obtain the electric bicycle and driver target detection model.

[0065] Specifically, in steps S1 and S2, we collected and annotated images of illegal e-bikes and vehicles passing by. Combining these two datasets yields the e-bike object detection dataset. We annotated 15,000 images of illegal e-bikes and 15,000 images of vehicles passing by, respectively. After merging, we obtained a total of 30,000 images and annotations, comprising the e-bike and driver object detection dataset.

[0066] Furthermore, we used openCV computer vision methods to simulate random lighting and angle transformations for each image in the electric bicycle and driver target detection dataset consisting of 30,000 images and annotations, and expanded the dataset to 120,000 images.

[0067] Furthermore, we use the Python programming language to build the Faster R-CNN target detection model. Faster R-CNN stands for Faster Region-based Convolutional Neural Network. It is a deep learning model based on convolutional neural networks that can achieve end-to-end training and prediction from original images to target detection results. Instead of relying on external components to generate candidate regions, it introduces a region proposal network that can automatically generate high-quality candidate regions from images. Faster R-CNN includes convolutional feature extraction, region proposal network, RoIPooling, as well as classification and bounding box regression. Finally, the category label of each candidate region and its corresponding bounding box coordinates are output, and redundant detections are reduced through non-maximum suppression.

[0068] We then imported the data into the Faster R-CNN target detection model for training. In one implementation, the Faster R-CNN model training process included image enhancement, image transformation, optimizing, loss function definition, and training. For image enhancement and transformation, we expanded the dataset to 120,000 images using simulated random lighting and angle transformation to improve the model's generalization. For image transformation, we converted three-channel RGB images of e-bikes and drivers into tensors. For optimizing, we used stochastic gradient descent with a learning rate of 0.002. For loss function definition, we used cross-entropy loss to calculate target detection model performance. After 200 rounds of training, we achieved an e-bike and driver target detection model with optimal parameters, achieving a loss of 0.032.

[0069] S4, train and optimize the OSNet full-scale feature extraction model to obtain the electric bicycle feature extraction model.

[0070] Specifically, we used the e-bike violation images and passing vehicle images collected and labeled in steps S1 and S2 to train the OSNet full-scale feature extraction model. To enhance the model's generalization and robustness, we augmented the original data.

[0071] We then used OpenCV to simulate random lighting and angle changes, and added noise to expand the total to 60,000 images of each of the 15,000 images of electric bicycle violations and 15,000 images of electric bicycles passing by vehicles. Each violation image was paired with a corresponding image of a vehicle passing by, ensuring that they were from consecutive passing behaviors of the same electric bicycle and the same driver.

[0072] Furthermore, we used the Python programming language to build the OSNet (Omni-Scale Feature Learning) model structure. OSNet is a lightweight yet powerful deep learning model that can capture the key features of target objects from different viewpoints and poses.

[0073] Furthermore, during training, we employed a triplet loss function, optimizing for maximizing the similarity between identical individuals and minimizing the similarity between different individuals. The initial learning rate was set to 0.001, dynamically adjusted using a cosine annealing strategy, and the training batch size was set to 64. After 200 rounds of training, the final loss reached 0.012, and the validation set accuracy reached 92.31%.

[0074] S5, target detection and extraction of feature vectors of illegal electric bicycle images taken by electronic police, and the time, location and feature vectors are entered into the database table.

[0075] Specifically, we have completed the data collection and neural network model training process of the present invention in steps S1 to S4. This step begins with the deployment phase of the invention content. Figure 1 、 2 Provide detailed explanation.

[0076] We then read the URLs corresponding to the time, location, behavior, and photos of electric bicycle violations from the Oracle database that records vehicle violation data. We then access the URLs corresponding to the violation photos through HTTP requests to retrieve the photos.

[0077] Furthermore, we input the electric bicycle violation photos into the electric bicycle and driver target detection model obtained in step S3 to obtain the anchor frames corresponding to the electric bicycle and driver targets in the electric bicycle violation photos.

[0078] Furthermore, we crop the illegal electric bicycle photo according to the anchor frame and input the cropped electric bicycle and driver into the electric bicycle feature extraction model obtained in step S4 to obtain the background feature vector of the illegal electric bicycle and driver in the illegal photo.

[0079] Furthermore, we store the corresponding electric bicycle violation time, violation location, URL address of the violation photo, and background feature vectors of the illegal electric bicycle and driver in the violation photo into the electric bicycle violation capture database table.

[0080] S6, target detection and extraction of feature vectors of the electric bicycle passing images taken by public security monitoring and binocular cameras, and the time, location and feature vectors are entered into the database table.

[0081] Specifically, in step S5, we obtained the background feature vectors of the illegal electric bicycle and the driver in the illegal photo. Our ultimate goal is to associate the back of the driver in the illegal photo with the foreground of the driver captured by public security monitoring and binocular cameras through the feature vector, so as to determine the identity of the offender through face recognition. Therefore, we need to obtain the foreground feature vector of the driver captured by public security monitoring and binocular cameras in this step.

[0082] Furthermore, we access the database that stores public security monitoring, binocular camera shooting records and the URLs corresponding to the photos, access the URL address links corresponding to all photos through HTTP requests, and read all photos of pedestrians passing by and electric bicycles passing by.

[0083] We then input all photos of pedestrians and e-bikes into the e-bike and driver object detection model generated in step S3. Because the model was trained with e-bike drivers as input, the object detection model only detects e-bike drivers and automatically filters out invalid data such as pedestrians. Based on the model's computational results, we obtain anchor frames corresponding to the foreground objects of the e-bike and driver.

[0084] Furthermore, we crop the foreground photo of the electric bicycle and the driver according to the anchor frame, and extract the electric bicycle feature model obtained in step S4 from the cropped foreground photo of the electric bicycle and the driver to obtain the foreground feature vector of the electric bicycle and the driver in the foreground photo of the electric bicycle and the driver.

[0085] Furthermore, we store the time and location of the electric bicycle passing, the URL address of the passing photo, and the foreground feature vectors of the electric bicycle and the driver in the electric bicycle and driver foreground passing database table.

[0086] S7, filter the images of electric bicycles passing by according to the time and place of the electric bicycle violation.

[0087] Specifically, in actual deployment scenarios, due to the current large number of e-bike violations and the large number of vehicles passing through, and for specific groups such as food delivery drivers, the backgrounds of different e-bikes and drivers are relatively small. Therefore, we need to narrow the comparison range of feature vectors.

[0088] Furthermore, the addresses of the electronic police, public security monitoring, and binocular cameras included in all projects are counted, and the electronic police addresses are matched with the public security monitoring in a 1:N ratio. That is, one electronic police device corresponds to multiple public security monitoring and binocular cameras. The corresponding condition is that the actual geographical location of the public security monitoring and binocular camera is within 200 meters of the actual geographical location of the corresponding electronic police, thus forming an electronic police and public security monitoring pair record table.

[0089] Furthermore, we used the SQL intersection method to join the electric bicycle violation capture data table with the electric bicycle and driver foreground vehicle data table. The intersection condition was that the electric bicycle and driver foreground vehicle passed within 5 minutes of the electric bicycle violation capture time, and that the same violation data was in the electronic police and public security surveillance, and binocular camera pair record table. In our actual deployment, a single violation record was mapped to up to 197 images of electric bicycles and drivers foreground vehicles captured by public security surveillance and binocular cameras.

[0090] S8, calculate the Euclidean Reed distance between the feature vectors of the violation photo and the passing vehicle image pair, and select the one with the smallest distance as the violation photo and foreground photo pair of the same person and the same vehicle.

[0091] Specifically, in S5 and S6, we obtained the feature vectors of the electric bicycle violation images and the feature vectors of the electric bicycle passing images taken by the electronic police, respectively. In S7, we associated a feature vector of an electric bicycle violation image taken by the electronic police with the feature vectors of electric bicycle passing images taken at the same location and in a similar time period. By calculating the Euclidean Reed distance between each pair of feature vectors of the electric bicycle violation image and the feature vectors of the electric bicycle passing image taken by the electronic police, we selected the pair of feature vectors with the smallest distance to determine the electric bicycle passing image corresponding to the violation photo.

[0092] S9, performs facial recognition on electric bicycles and drivers passing by in the foreground to identify the offender.

[0093] Specifically, the images of electric bicycle violations captured by electronic police officers include the violation process but not the person's face, while the images of electric bicycles passing by captured by security cameras and binocular cameras include the person's face but not the violation process. In step S8, we correlate the images of electric bicycle violations with those of electric bicycles passing by, resulting in a set of photos of electric bicycle violations that include faces. Simply performing facial recognition on the images of electric bicycles passing by can identify the offender. The facial recognition interface is provided by the public security department and is not within the scope of this invention or the scope of protection of this technical solution.

[0094] Although the present invention has been disclosed above in terms of preferred embodiments, this is not intended to limit the present invention. Any person skilled in the art may, without departing from the spirit and scope of the present invention, utilize the methods and technical contents disclosed above to make possible changes and modifications to the technical solutions of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the scope of protection of the technical solutions of the present invention.

Claims

1. A method for identifying electric bicycle offenders based on feature vector extraction and comparison, the method comprising the following steps: Step 1: Collect the electric bicycle violation image data collected by the electronic police, obtain the background image of the passing vehicle, and annotate the electric bicycles and drivers in it to form the electric bicycle background dataset. Step 2: Collect the electric bicycle passing data collected by the public security monitoring and binocular camera within 200 meters around the electronic police in step 1 above, obtain the passing vehicle foreground image, and annotate the electric bicycles and drivers in it to form the electric bicycle foreground dataset. Step 3: Use the electric bicycle background dataset and electric bicycle foreground dataset obtained in steps 1 and 2 above to train and optimize the Faster R-CNN target detection model to obtain the electric bicycle and driver target detection model, which is used to detect the target electric bicycle and driver in the violation image. Step 4: Use the electric bicycle background dataset and electric bicycle foreground dataset obtained in steps 1 and 2 above to train and optimize the OSNet full-scale feature extraction model to obtain a feature extraction model for matching the foreground and background images of the same electric bicycle and driver. Step 5: Use the electric bicycle and driver target detection model to extract the background of electric bicycles and drivers in the violation photos captured by the electronic police, use the feature extraction model to extract the background feature vectors of electric bicycles and drivers, and store the violation time, violation location, violation photos, and feature vectors in the electric bicycle violation capture database table. Step 6: Use the electric bicycle and driver target detection model to extract the foreground of electric bicycles and drivers in the electric bicycle passing photos collected by public security monitoring and binocular cameras, use the feature extraction model to extract the foreground feature vectors of electric bicycles and drivers, and store the passing time, passing location, passing photos, and feature vectors in the electric bicycle and driver foreground passing database table. Step 7: Based on the violation time and location in the electric bicycle violation capture database table, filter the electric bicycle and driver foreground passing vehicles from the electric bicycle and driver foreground passing vehicle database table, whose passing time is within 5 minutes before and after the violation time and whose passing location is within 200 meters around the electronic police security surveillance and captured by binocular cameras. Step 8: Use the Euclidean Reed distance to calculate the electric bicycle violation data in step 7 and the feature vectors of the electric bicycle and driver foreground vehicles selected in step 7, and select the electric bicycle and driver foreground vehicle with the smallest Euclidean Reed distance to obtain the violation photo and foreground vehicle pair. Step 9: Use the face recognition model to identify the identity of the electric bicycle and the driver in the foreground vehicle obtained in step 8, and confirm the offender.

2. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In steps 1 and 2 described above, we collect illegal electric bicycle image data and foreground images of passing vehicles, and annotate them to form a dataset. The specific steps are as follows: Step 1-1: Use the X-AnyLabeling tool to annotate the rectangular area containing the target e-bike and driver in the e-bike violation image. The annotation information includes the horizontal and vertical coordinates of the upper left corner of the rectangular area relative to the entire image, the width and height of the area, and the identification number. We actually screened and annotated 15,000 e-bike violation images. Step 2-1: Use the X-AnyLabeling tool to rectangularly label the rectangular area containing the target electric bicycle and driver in the electric bicycle passing image. The labeling information includes the horizontal and vertical coordinates of the upper left corner of the rectangular area where the electric bicycle and driver are located relative to the entire image, the width, height and identification number of the area. Among them, the electric bicycle and driver foreground collected in this step and the electric bicycle and driver background collected in step 1 are the same electric bicycle, the same driver, and the same continuous passing, and the labeling numbers are the same. We actually screened and labeled 15,000 electric bicycle passing images, and corresponded them to the 15,000 electric bicycle violation images collected and labeled in step 1-1.

3. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 3 described above, the training and optimization of the electric bicycle and driver target detection model is as follows: Step 3-1: Use Python programming language to build the Faster R-CNN object detection model structure. Step 3-2: Combine the 15,000 images of electric bicycle violations and the 15,000 images of electric bicycle passing vehicles obtained in steps 1 and 2 into an electric bicycle and driver target detection dataset consisting of a total of 30,000 images. Step 3-3: Expand the electric bicycle and driver target detection dataset obtained in step 3-2 to 120,000 images by simulating random lighting and angle transformation using OpenCV. Step 3-4: Randomly initialize the weights and input the Faster R-CNN target detection model obtained in step 3-1. Step 3-5: Read the electric bicycle and driver target detection dataset obtained in step 3-2 and input it into the FasterR-CNN target detection model for training. After 200 epochs of training, our loss reached 0.

032.

4. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 4 described above, the feature extraction model is trained and optimized. The specific steps are: Step 4-1: Use Python programming language to build the OSNet full-scale feature extraction model structure. Step 4-2: The 15,000 images of electric bicycle violations and the 15,000 images of electric bicycle passing vehicles obtained in steps 1 and 2 were expanded to 60,000 images respectively by simulating random lighting, angle transformation and adding noise through openCV. Step 4-3: Randomly initialize the weights and input the OSNet full-scale feature extraction model obtained in step 4-1. Step 4-4: The 60,000 expanded e-bike violation images and 60,000 e-bike passing images obtained in Step 4-2 were fed into the OSNet full-scale feature extraction model for training. Each e-bike violation image corresponds to an e-bike passing image, provided they have the same label number. This means that a set of e-bike violation images and e-bike passing images represents the same consecutive e-bike passings by the same driver. We set the initial learning rate to 0.001 and used cosine annealing to adjust the learning rate based on the cosine function. Each training batch size was 64, and using triplet loss as the loss function, we achieved a final loss value of 0.012, resulting in an accuracy of 92.31% on the validation set.

5. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 5 described above, the background feature vectors of the electric bicycle and the driver are extracted and stored in the electric bicycle illegal capture database table. The specific steps are as follows: Step 5-1: Connect to the remote Oracle violation record database, obtain the time and location of the electric bicycle violation and the URL address of the violation photo, and read the violation photo according to the URL. Step 5-2: Use the electric bicycle and driver target detection model obtained in step 3 to detect the illegal electric bicycle and driver in the illegal photo to obtain the background image of the illegal electric bicycle and driver. Step 5-3: Use the feature extraction model obtained in step 4 to perform feature extraction on the background images of the illegal electric bicycle and the driver obtained in step 5-2 to obtain background feature vectors of the illegal electric bicycle and the driver. Step 5-4: Store the electric bicycle violation time, violation location, and URL address of the violation photo obtained in step 5-1 and the background feature vector of the illegal electric bicycle and driver obtained in step 5-3 into the electric bicycle violation capture database table.

6. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 6 described above, the foreground feature vectors of the electric bicycle and the driver are extracted and stored in the foreground vehicle database table of the electric bicycle and the driver. The specific steps are: Step 6-1: Connect to the remote Oracle vehicle passing record database, obtain the time and location of the electric bicycle passing, and the URL address of the passing photo, and read the passing photo according to the URL. Step 6-2: Use the electric bicycle and driver target detection model obtained in step 3 to detect the electric bicycle and driver in the passing vehicle photo to obtain the foreground image of the electric bicycle and driver. Step 6-3: Use the feature extraction model obtained in step 4 to perform feature extraction on the foreground images of the electric bicycle and the driver obtained in step 6-2 to obtain feature vectors of the foreground of the electric bicycle and the driver. Step 6-4: Store the electric bicycle passing time, passing location, and URL address of the passing photo obtained in step 6-1 and the electric bicycle and driver foreground feature vectors obtained in step 6-3 into the electric bicycle and driver foreground passing database table.

7. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 7 described above, electric bicycles and drivers are screened for foreground vehicles according to the records in the electric bicycle illegal capture database table. The specific steps are: Step 7-1: Count the addresses of electronic police, public security monitoring, and binocular cameras included in all projects, and map the electronic police addresses to public security monitoring in a 1:N ratio. That is, one electronic police device corresponds to multiple public security monitoring and binocular cameras. The corresponding condition is that the actual geographical location of the public security monitoring and binocular cameras is within 200 meters of the actual geographical location of the corresponding electronic police, forming an electronic police and public security monitoring pair record table. Step 7-2: Use the SQL intersection method to connect the electric bicycle violation capture data table obtained in step 5 and the electric bicycle and driver foreground passing vehicle data table obtained in step 6. The intersection condition is that the time when the electric bicycle and driver foreground passed the electric bicycle is within 5 minutes before or after the electric bicycle violation capture time, and the electronic police and public security monitoring and binocular camera pair of the same violation data is in the electronic police and public security monitoring pair record table obtained in step 7-1.

8. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 8 described above, the illegal photo and the foreground vehicle pair are obtained. The specific steps are: Step 8-1: Calculate the Euclidean Reed distance between the background feature vectors of illegal electric bicycles and drivers and the foreground feature vectors of electric bicycles and drivers obtained from the intersection of the electric bicycle violation capture database table obtained in step 7-2 and the electric bicycle and driver foreground passing vehicle data table. Step 8-2: For the same electric bicycle violation record, compare the Euclidean distances between the background feature vectors of the illegal electric bicycle and the driver, and the foreground feature vectors of the electric bicycle and the driver obtained in 8-1, and select the electric bicycle and the foreground vehicle with the smallest Euclidean distance to obtain the violation photo and foreground vehicle pair.

9. The method for identifying electric bicycle offenders based on feature vector extraction and comparison according to claim 1 is characterized by: In step 9 described above, the identity of the offender is confirmed. The specific steps are: Step 9-1: Take out the foreground images of the electric bicycle and the driver obtained by performing target detection on the electric bicycle and the driver in the passing vehicle photo in step 6-2. Step 9-2: Input the foreground image of the electric bicycle and the driver into the face recognition module to obtain the driver's identity information and confirm the identity of the offender.

Images