An electric bicycle re-identification method

Abstract

This invention provides a method for re-identifying electric bicycles, which can achieve targeted secondary identification of electric bicycles and effectively improve the accuracy of identification. It constructs the backbone network of a vehicle feature extraction model based on EfficientNet, introduces a generalized mean pooling layer at the feature output end, and then maps the input image into a multi-dimensional normalized feature vector based on a bottleneck layer. The method obtains vehicle images from the registration domain and intersection domains collected at traffic intersections as input images, uses an object detection network to extract vehicle bounding boxes, constructs an asymmetric cross-domain dataset, and feeds it into the vehicle feature extraction model for multi-dimensional normalized feature vector extraction. Finally, it calculates the similarity between the two images based on the two multi-dimensional normalized feature vectors.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, specifically to a method for re-identifying electric bicycles. Background Technology

[0002] In recent years, with the increasing popularity of electric bicycles in urban transportation, higher requirements have been placed on the intelligent and refined management of electric bicycles. Among these, secondary identification technology for electric bicycles has become one of the key research directions. Similar to the approach of secondary identification (re-identification) for motor vehicles, electric bicycle re-identification mainly builds upon existing hardware such as electronic monitoring equipment, checkpoint equipment, and radio frequency reading equipment. Based on image recognition technology or radio frequency technology to obtain basic information such as vehicle license plate and vehicle body images, richer vehicle features are extracted through algorithms or AI models. These features are then compared with data in the electric bicycle information database to identify violations such as license plate forgery, alteration, and cloned license plates. Alternatively, it can achieve image-based vehicle search to quickly match similar vehicles from massive images and locate the vehicle.

[0003] In intelligent traffic management systems, accurately verifying the consistency between the vehicle's license plate and registration information is a core component. Existing technologies include a wealth of computer vision recognition and vehicle re-identification technologies for motor vehicles. However, due to significant differences in size and shape between motor vehicles and electric bicycles, existing computer vision and vehicle re-identification technologies face the following technical bottlenecks when dealing with the unique characteristics of electric bicycles: 1) Traditional license plate recognition technology that relies solely on OCR suffers from the phenomenon of "visual information silos".

[0004] Most existing traffic intersection systems in China adopt a "crossroads data acquisition + OCR license plate extraction" model. This technology can only extract text characters from the license plate and cannot effectively bind "license plate information" with the "physical appearance of the vehicle." To achieve more rigorous vehicle information consistency verification, cross-referencing the physical appearance features (such as vehicle frame, fenders, and trunk shape) of dynamically acquired images (crossroads images) with static images (registration images) in the system archive has become an important direction for technological development. However, this "crossroads-registration" cross-domain image comparison currently faces technical challenges such as high overlap between people and vehicles, huge parallax, and inaccurate cross-domain measurements.

[0005] 2) Mainstream general vehicle re-identification methods suffer from overfitting and interference from "human occlusion".

[0006] Most common re-identification schemes, both domestically and internationally, are designed for motor vehicles, whose appearance features are relatively stable. However, images of electric bicycles captured at intersections inevitably contain large areas of occlusion caused by the rider. If such general neural network models are directly applied, the network is prone to overfitting, incorrectly using environmental interference factors such as "the color of the rider's clothing or the style of their raincoat" as the core discrimination criterion, rather than the physical characteristics of the vehicle itself. Furthermore, when mainstream schemes handle cross-domain retrieval between intersection images (usually rear-view) and registration images (usually side-view), the lack of geometric deformation adversarial strategies for such complex perspective transitions leads to low cross-domain feature matching accuracy.

[0007] 3) Existing cross-camera tracking solutions are unable to solve the problem of "cross-domain measurement inaccuracy".

[0008] Some existing technologies propose cross-camera electric vehicle recognition schemes based on temporal trajectories. These schemes heavily rely on continuous multi-frame video from the same camera position to compensate for single-frame occlusion. However, in the comparison scenario of "intersection dynamic images" and "registration still images," the archive images often only contain a single frame and lack temporal information. In addition, traditional schemes often use symmetrical global hard example mining during model training (i.e., only comparing the differences between "intersection images"), which prevents the model from fully learning and mining key features in cross-domain (dynamic and static, high-altitude overhead and eye-level) scenarios, ultimately resulting in poor system performance in judging vehicle identity consistency.

[0009] In summary, how to achieve accurate electric bicycle identity consistency determination under complex conditions of high occlusion, cross-viewpoint, and heterogeneous data through lightweight models and asymmetric metric learning is a technical challenge that urgently needs to be solved in the fields of computer vision and intelligent transportation. Summary of the Invention

[0010] To address the problem that existing vehicle re-identification methods for motor vehicles are not applicable to electric bicycles, this invention provides an electric bicycle re-identification method that can specifically achieve secondary identification of electric bicycles and effectively improve the accuracy of identification.

[0011] The technical solution of the present invention is as follows: a method for re-identifying electric bicycles, characterized in that it includes the following steps: S1: Construct the target detection model; The target detection model identifies electric bicycles in the input image; the input of the target detection model is an image including the electric bicycle to be identified, and the output is the image to be processed with the vehicle bounding box and the coordinates of the vehicle bounding box. The vehicle bounding box is the smallest bounding rectangle of the electric bicycle identified by the model; S2: Construct a vehicle feature extraction model; The vehicle feature extraction model includes: a backbone network, a generalized mean pooling layer (GeM), a bottleneck layer, and a classification head; The classification head in the EfficientNet model is removed and used as the backbone network. The bottleneck layer consists of Linear, BN and PReLU connected in sequence. The classification head is implemented based on a fully connected layer. The input to the vehicle feature extraction model is the image to be processed; The output of the bottleneck layer is the vehicle query feature V. query The output of the classification head is the vehicle query feature V. query Corresponding license plate number Key; S3: Construct training samples based on existing electric bicycle data, and train the target detection model and the vehicle feature extraction model to obtain the trained target detection model and vehicle feature extraction model; S4: Constructed as a re-identification feature extraction module; The re-identification feature extraction module includes: a trained target detection model and a trained vehicle feature extraction model set sequentially; S5: Build a photo library of registered electric bicycles; The registration image library stores registration information for registered electric bicycles. The format of the registration information is: [Registration Number Key] reg Registration feature V reg ]; Among them, registration feature V reg The registration image captured when registering electric bicycles is based on vehicle image features extracted by the re-identification feature extraction module; S6: Arrange the re-identification feature extraction module in a computing device and perform the following operations: a1: The monitoring equipment set up at the intersection collects images of electric bicycles on the road, denoted as: the intersection image to be identified; the intersection image to be identified is sent to the re-identification feature extraction module to extract the vehicle query feature V corresponding to the electric bicycle to be identified. query ; a2: Identify the vehicle license plate corresponding to the image of the intersection to be identified, denoted as: vehicle license plate of the intersection to be identified; a3: Based on the license plate number of the vehicle at the intersection to be identified, find the corresponding registration information [Registration License Plate Key] in the registration image database. reg Registration feature V reg ]; a4: Calculate vehicle query features V query and registration feature V regThe similarity between them is denoted as: the similarity to be identified (SIM); a5: Compare the target similarity SIM with a preset similarity threshold SimTh; If SIM < SimTh, it indicates an anomaly, and an alarm message for abnormal re-identification result is issued. Otherwise, it indicates that the two are similar and no abnormal alarm message needs to be issued.

[0012] Its further features are: The re-identification feature extraction module also includes: an adaptive semantic occlusion module; The adaptive semantic occlusion module is located between the target detection model and the vehicle feature extraction model. When the input image is an image of the intersection to be identified collected by a monitoring device based on the intersection, the image to be processed with vehicle bounding boxes output by the target detection model is first sent to the adaptive semantic occlusion module for processing, and then sent to the vehicle feature extraction model. The operation performed in the adaptive semantic occlusion module is as follows: b1: Calculate the aspect ratio R of the vehicle bounding box, and determine the dynamic erase boundary ratio P of the human torso occlusion based on the aspect ratio R. cut ; b2: For vehicles within the bounding box that are higher than P cut The image region of × Height is occluded to force the network to focus on the physical features of the lower half of the region; where Height is the height of the vehicle bounding box. b3: The focused image to be processed is then fed into the vehicle feature extraction model for feature extraction. The hyperparameter p of the generalized mean pooling layer GeM in the vehicle feature extraction model is set to a value greater than 1. The aspect ratio is calculated as follows: R = Height / Width; In the formula, Height is the height of the vehicle bounding box, and Width is the width of the vehicle bounding box; The method for calculating the dynamic erasure boundary ratio is as follows: P cut =min(max(0.3,γlog(R)),0.5); In the formula, P cut The dynamic erase boundary ratio is defined by γ, which is an adjustment coefficient, and R is the aspect ratio of the vehicle enclosure frame. In step S3, for the training and construction of the vehicle feature extraction model, the total training loss function L is jointly trained. total ; L total =L ID_cam +LID_reg +L triplet ; Among them, L ID_cam L is the loss function for identity classification in the intersection domain; ID_reg For the registration domain identity classification loss function; L triplet For dynamic boundary asymmetric cross-domain triplet loss function; The method for calculating the dynamic boundary asymmetric cross-domain triplet loss function is as follows: ; In the formula, N c N represents the number of intersection area image samples in the sample data; c =P×K c P is the number of vehicle license plates, K c The number of features in the intersection domain image; N r N represents the number of registered domain image samples in the sample data. r =P×K r K r The number of image features in the registration domain; D C→R pos [i] represents the most difficult positive sample with the farthest distance in the same license plate registration map corresponding to the i-th intersection map; D R→C pos [j] represents the most difficult positive sample with the furthest distance among the intersection maps with the same license plate number corresponding to the j-th registration map; D C→R neg [i] represents the closest, most difficult negative sample among the different license plate registration maps corresponding to the i-th intersection map; D R→C neg [j] represents the closest, most difficult negative sample among the different license plate intersections corresponding to the j-th registration map; α i The dynamic boundary parameters corresponding to the i-th intersection image are obtained based on training. α j The dynamic boundary parameters corresponding to the j-th intersection image are obtained based on training. C is the intersection domain feature matrix output by the vehicle feature extraction model, and R is the registration domain feature matrix. The method for calculating the intersection domain identity classification loss function is as follows: ; In the formula, M represents the total number of vehicle identity categories in the training set; p C The probability distribution of vehicle identity predictions in an intersection image by a vehicle feature extraction model; q CThe vehicles in the intersection map are accurately represented by their actual labels. The calculation method for the registration domain identity classification loss function is as follows: ; In the formula, p R The probability distribution of vehicle identity predicted by the vehicle feature extraction model from the registration map; q R To register the distribution of the true identity tags of the vehicles in the image; In step a2, the method for recognizing vehicle license plates at the intersection to be identified includes the following operations: a21: For each vehicle within the bounding box of the intersection image to be identified, confirm whether the complete vehicle license plate can be identified using the OCR model based on image recognition technology; If the complete license plate of the electric bicycle in the vehicle bounding box can be identified based on the OCR model, then the license plate of the vehicle to be identified at the intersection will be obtained, and a3 will be executed. Otherwise, execute a22; a22: For electric bicycles within the vehicle enclosure, license plate data is collected using radio frequency identification equipment; If a complete license plate can be captured, the license plate number of the vehicle at the intersection to be identified will be obtained, and step a3 will be executed; Otherwise, execute a23; a23: This will retrieve the vehicle query feature V corresponding to vehicles for which license plate numbers cannot be collected. query The image is added to the abnormal image feature database and will not be included in the subsequent calculations of this re-identification process. The abnormal image feature database stores the following abnormal data processing operations; c1: When a vehicle query feature V query When the image is added to the abnormal image feature database, a similarity comparison is performed with the data in the database. If similar data can be found, increment its corresponding counter CA by 1 and proceed to step c2; Otherwise, save it as a new data item and initialize its counter CA to 1; c2: Compare the counter CA with the preset abnormal threshold; If the value exceeds the preset abnormal threshold, an abnormal alarm message will be issued; Otherwise, proceed to step c3; c3: Perform periodic zeroing operations on the abnormal image feature database according to a preset frequency.

[0013] This application provides a method for re-identifying electric bicycles. It constructs the backbone network of a vehicle feature extraction model based on EfficientNet, discarding the original global average pooling layer of EfficientNet and introducing a generalized average pooling layer at the feature output. Through adaptive learning of the pooling parameter p, it enhances the network's nonlinear response to local highlighted features, such as modification stickers and specific tail box shapes, thereby improving the accuracy of the final recognition result. Then, based on a bottleneck layer, the input image is mapped into a multi-dimensional normalized feature vector. The method obtains vehicle images from the registration domain and intersection domains collected at traffic intersections as input images. It uses an object detection network to extract vehicle bounding boxes, constructs an asymmetric cross-domain dataset, and feeds it into the vehicle feature extraction model for multi-dimensional normalized feature vector extraction. Finally, it calculates the similarity between the two images based on the two multi-dimensional normalized feature vectors, which can effectively improve the recognition accuracy between intersection domain images and registration domain images. This application proposes an adaptive upper-half semantic erasure technique for electric bicycle images captured by monitoring equipment in the intersection domain. This technique forces the decoupling of human and vehicle features, significantly improving the stability of feature extraction when dealing with riders wearing raincoats or different clothing. It effectively solves the problem of electric bicycle re-identification technology being hampered by rider clothing interference. Furthermore, this application proposes an asymmetric cross-domain triplet loss for cross-domain image recognition between the intersection and registration domains. This concentrates the computational power for hard example mining on the "intersection-registration" cross-domain gap, improving the accuracy of cross-domain feature matching. Attached Figure Description

[0014] Figure 1 A schematic diagram of the electric bicycle re-identification method; Figure 2 Network architecture diagram of the vehicle feature extraction model; Figure 3 An example of a vehicle bounding box output by an object detection model; Figure 4 Based on Figure 3 This example demonstrates the effect of adaptive semantic occlusion. Detailed Implementation

[0015] like Figure 1 As shown, this application includes a method for re-identifying electric bicycles, which includes the following steps.

[0016] S1: Construct a target detection model for electric bicycles.

[0017] The object detection model identifies electric bicycles in the input image. The input of the object detection model is an image including the electric bicycle to be identified, and the output is the image to be processed with the vehicle bounding box and the coordinates of the vehicle bounding box.

[0018] The specific application involves an electric bicycle target detection model. This model can be implemented using existing technologies such as CNN or YOLO, which are capable of detecting electric bicycles. In this embodiment, a target detection model for electric bicycles is built based on the YOLO model. The output includes an image with detection boxes, the coordinates of the detection boxes, and the confidence scores corresponding to the detection boxes. The origin of the detection box coordinates is the lower left corner of the image. Specifically, the detection box, or vehicle bounding box, is the smallest bounding rectangle of the electric bicycle identified by the model, such as... Figure 3 The green box shown is the vehicle bounding box in the output detection result of the object detection model.

[0019] The data used for electric bicycle identification re-identification (secondary identification) in this application includes: registration domain data and intersection domain data. Registration domain data refers to the vehicle photos required for registration when a user purchases and registers an electric bicycle. According to regulations, these photos are head-on views of the entire vehicle at a 45-degree angle to the left front and / or a 45-degree angle to the left rear. Requirements vary slightly by region; some regions require only one image, while others require two to three. All registration vehicle photos are saved in a vehicle image database, corresponding to the registration domain images in this application. Secondary vehicle identification utilizes image acquisition devices at traffic intersections, such as surveillance cameras or checkpoints, to capture images of passing electric bicycles. These images are typically top-down rear-view images of the vehicles, corresponding to the intersection domain images in this application.

[0020] Both the registration domain image and the intersection domain image need to be fed into the object detection model for object recognition. However, intersection domain images will inevitably contain riders on electric bicycles, which leads to human occlusion problems. To reduce the interference of riders on vehicle feature recognition, this application designs an adaptive semantic occlusion method for intersection domain images to perform data augmentation. The specific operations included in the adaptive semantic occlusion method are as follows: b1: Calculate the aspect ratio R of the vehicle bounding box, and determine the dynamic erase boundary ratio P of the human torso occlusion based on the aspect ratio R. cut ; b2: For vehicles with a height above P within the bounding box cut The image region is occluded, forcing the network to focus on the physical features of the lower half of the region; b3: The focused image to be processed is then fed into the vehicle feature extraction model for feature extraction.

[0021] The aspect ratio is calculated as follows: R = Height / Width; In the formula, Height is the height of the vehicle bounding box, and Width is the width of the vehicle bounding box; Dynamic erase boundary ratio P cut The calculation method is as follows: P cut =min(max(0.3,γlog(R)),0.5); In the formula, γ is the adjustment coefficient, and R is the aspect ratio of the vehicle's bounding box. The specific adjustment coefficient γ is collected based on historical data; in this embodiment, γ is set to 0.45.

[0022] Vehicle bounding boxes are extracted using an object detection network, and an adaptive upper half-region erasure strategy is implemented for intersection domain images.

[0023] Because the size of the electric bicycle in each intersection image is related to the distance between the electric bicycle and the intersection image acquisition device, its size is uncertain. This method dynamically determines the proportion threshold P of human torso occlusion by calculating the aspect ratio R of the vehicle bounding box. cut The object detection model outputs the coordinates of each vehicle bounding box, such as the coordinates of the bottom left and top right corners of the bounding box. This allows us to determine the highest and lowest points of the bounding box. For vehicle bounding boxes, coordinates exceeding a threshold P are also output. cut The image region of ×Height (i.e., the upper half of the vehicle bounding box containing the cyclist) is probabilistically masked, forcing the network to focus on high-stability physical features such as the frame, fenders, and tail hub in the lower half of the image.

[0024] During training, the system triggers the enhancement method with a 40% probability, moving the vehicle bounding box from its highest point to P. cut The upper half of the ×Height area has all pixels set to zero (painted black). For example... Figure 4 As shown, it is based on Figure 3 The image shows the effect of adaptive semantic occlusion of the vehicle bounding box. This operation preserves the lower chassis, license plate area, and rear wheel hub while reducing the backpropagation probability of clothing color gradients at the physical level from the input.

[0025] This method overcomes the feature shift problem caused by "severe occlusion" by employing an adaptive semantic occlusion approach. Compared to mainstream international re-identification schemes that are constrained by the interference of cyclists' clothing, the "adaptive upper half semantic erasure" technique proposed in this invention forcibly decouples human and vehicle features, significantly improving the stability of feature extraction when dealing with cyclists wearing raincoats or different clothing.

[0026] S2: Construct a vehicle feature extraction model; the vehicle feature extraction model includes: a backbone network, a generalized mean pooling (GeM) layer, a bottleneck layer, and a classification head. The classification head from the EfficientNet model is removed and used as the backbone network. The bottleneck layer consists of sequentially connected Linear, BN, and PReLU layers; the classification head is implemented based on a fully connected layer. The input to the vehicle feature extraction model is the image to be processed; after processing through the backbone network and GeM layer, the output of the bottleneck layer is the vehicle query feature V. query The output of the classification head is the vehicle query feature V. query The corresponding license plate number Key; where the classification head only participates in training to improve the model's feature extraction capabilities, and does not participate in the actual inference process.

[0027] Considering that the initial business often involves only a few thousand vehicle data points, directly using large networks (such as EfficientNet-B7 or Swin-Transformer) could easily lead to overfitting. Therefore, this application selects the lightweight convolutional neural network EfficientNet-B3 as the backbone extraction network, abandoning the original global average pooling layer of EfficientNet-B3 and introducing a generalized mean pooling (GeM) layer at the feature output. By adaptively learning the pooling parameter p, the network's nonlinear response to local highlighted features (such as modification stickers or specific trunk shapes) is enhanced, ultimately mapping the input image into a D-dimensional normalized feature vector.

[0028] The backbone network of the vehicle feature extraction model outputs a feature map with dimensions [B, C, H, W], where B is the batch size (number of samples), H is the spatial height of the feature map, W is the spatial width of the feature map, k is the number of convolutional kernels, and C is the number of channels (number of feature maps). The feature map output by the backbone network is input into a generalized mean pooling (GeM) layer. In this embodiment, the pooling kernel k is set to 3, and the number of channels C is 1536.

[0029] For a local region X={x1,x2,…x} in the input feature map n The standard calculation formula for the generalized mean pooling GeM layer is: ; In the formula, x i Let |X| be the value of the i-th element in the feature map, |X| be the total number of elements in the local region, and ε be a minimum value to prevent the base from being negative, with a value of 1×10. -6 ; GeM pooling introduces an adjustable hyperparameter p to control the shape of the pooling. When p=1, GeM pooling is equivalent to average pooling; when p approaches positive infinity, GeM pooling is equivalent to max pooling. In this embodiment, p is set to a value greater than 1 and initialized to 3.0. Compared to traditional global average pooling (p=1), a learnable p value (p>1) allows the network to adaptively focus on local highlight features (such as stickers on takeout boxes), improving detail recognition capabilities.

[0030] Image features extracted by the backbone network are fed into the GeM layer to enhance local detail recognition. Finally, a bottleneck layer (containing Linear, BatchNorm1d, and PReLU) reduces the dimensionality of the features to 512 dimensions. Since the intersection domain image and the registration domain image are captured from different angles, this application uses the multi-dimensional image feature vector output from the bottleneck layer to compare the two images, which effectively improves the accuracy of the comparison results.

[0031] Compared to existing mainstream technologies such as ResNet50 (approximately 25.6M parameters) or temporal multi-frame solutions, this invention improves upon EfficientNet-B3, resulting in a model with only approximately 12.2M network parameters. On GPU devices, the complete inference latency for a single video stream is only about 15ms (compared to approximately 35-42ms for traditional solutions), making it extremely suitable for deployment in high-concurrency scenarios.

[0032] S3: Based on existing electric bicycle data, construct training samples to train the target detection model and vehicle feature extraction model, and obtain the trained target detection model and vehicle feature extraction model.

[0033] In real-world scenarios, intersection cameras typically capture the continuous movement trajectory of electric bicycles (5-10 intersection images per bicycle), while registration files often only contain 2-3 static photos. Due to the severe asymmetry in both sample quantity and perspective between the "registration domain" and the "intersection domain," traditional methods of mixing images from the same domain to calculate triplet loss waste significant computational resources comparing "two extremely similar registration images" and cannot adapt to the varying quantities of heterogeneous samples across domains. To overcome these shortcomings, this application proposes an asymmetric cross-domain triplet joint training mechanism adapted to imbalanced multi-sample environments. The specific steps include the following operations: In each training batch, a cross-domain P×K sampling strategy is adopted: P electric bicycles with different identities are randomly selected (i.e., P vehicle license plate IDs), and for each identity, K are randomly selected. c Zhanglukou area image features and K r Zhang registered domain image features, such as K c =6,K r=2. At this time, the intersection domain feature matrix output by the network is Where N c =P×K c The corresponding identity tag sequence is Y. c The registration domain feature matrix is , where N r =P×K r The corresponding identity tag sequence is Y. R .

[0034] Constructing the asymmetric cross distance matrix: Calculating the cross Euclidean distance matrix between the intersection domain and the registration domain features. ; where matrix element D i,j =||C i -R j ||2, completely shield the distance calculation within the same domain (intersection to intersection, registration to registration).

[0035] The most difficult aspect of cross-domain positive sample mining: (1) For the i-th intersection map (labeled y) C i Among all the corresponding registration images of the same identity (same vehicle license plate), the one with the greatest distance (the largest difference) is selected as the most difficult positive sample: ; (2) For the j-th registration image (labeled y) R j In the corresponding intersection graphs of all intersections with the same identity, find the most distant and hardest positive sample: ; Cross-domain most challenging negative sample mining: (1) For the i-th intersection map, find the hardest negative sample that is closest (with the smallest difference) among all the registration maps of different identities (different vehicle license plates): ; (2) For the j-th registration map, find the nearest least difficult negative sample among all intersection maps with different identities: .

[0036] For the sample data in the aforementioned asymmetric domain, an asymmetric cross-domain triplet loss is constructed during the model training phase.

[0037] This application designs the most difficult positive samples and the most difficult negative samples across domains to train the model, which can make feature vectors of the same class as close as possible and feature vectors of different classes as far apart as possible. It is particularly suitable for training the model on asymmetric domain training datasets. Each set of samples includes two images: an intersection domain image and a registration domain image. Each image uses the license plate as an identity label. After the bottleneck layer of the vehicle feature extraction model outputs the two sets of extracted feature vectors, it determines whether the two sets of feature vectors belong to the same identity or different identities based on the identity label. The model adjusts its own parameters based on the identity determination result and the loss function.

[0038] This application calculates the cross-distance matrix only between the "intersection domain" and the "registration domain," excluding redundant negative sample mining within the same domain. Simultaneously, a dynamic boundary related to the intersection image degradation factor is introduced, and the network is optimized end-to-end for multiple tasks using a joint identity cross-entropy loss. Specifically, for the training and construction of the vehicle feature extraction model, a joint training total loss function L... total .

[0039] L total =L ID_cam +L ID_reg +L triplet ; Among them, L ID_cam L is the loss function for identity classification in the intersection domain; ID_reg For the registration domain identity classification loss function; L triplet For dynamic boundary asymmetric cross-domain triplet loss function.

[0040] The dynamic boundary asymmetric cross-domain triplet loss function is composed of the sum of the bidirectional asymmetric losses of "intersection retrieval registration" and "registration retrieval intersection", and the calculation method is as follows: ; In the formula, N c N represents the number of intersection area image samples in the sample data; c =P×K c P is the number of vehicle license plates, K c The number of features in the intersection domain image; N r N represents the number of registered domain image samples in the sample data. r =P×K r K r The number of image features in the registration domain; D C→R pos [i] represents the most difficult positive sample with the farthest distance in the same license plate registration map corresponding to the i-th intersection map; D R→C pos[j] represents the most difficult positive sample with the furthest distance among the intersection maps with the same license plate number corresponding to the j-th registration map; D C→R neg [i] represents the closest, most difficult negative sample among the different license plate registration maps corresponding to the i-th intersection map; D R→C neg [j] represents the closest, most difficult negative sample among the different license plate intersections corresponding to the j-th registration map; α i The dynamic boundary parameters corresponding to the i-th intersection image are obtained based on training. α j The dynamic boundary parameters corresponding to the j-th intersection image are obtained based on training. C is the intersection domain feature matrix output by the vehicle feature extraction model, and R is the registration domain feature matrix.

[0041] α i and α j As dynamic boundary parameters, if a certain intersection image C i If the confidence level of the target detector output decreases due to low illumination, motion blur, or occlusion, the system will adaptively lower the α value of that sample. i The value is reduced, for example, from the standard 0.3 to 0.15. This mechanism effectively prevents low-quality intersection samples from generating abnormally large gradients during training, protecting the stability of the feature space under a mixed distribution of high- and low-quality samples.

[0042] This application incorporates identity classification training during the training phase. By training the identity prediction capability of the vehicle feature extraction model's classification head, the accuracy of image feature extraction capabilities in the backbone network, GeM layer, and bottleneck layer is improved. Identity classification training enables the feature network to differentiate itself both locally in the metric space and globally in the global feature space. In this embodiment, after outputting 512-dimensional features from the bottleneck layer, a fully connected classification layer is connected to predict the identity ID and calculate the cross-entropy loss. Because the sample size and feature distribution are highly unbalanced between the intersection domain and the registration domain, this invention decouples the identity classification loss across domains, calculating the intersection domain identity classification loss L separately. ID_cam And the loss of identity classification in the registration domain L ID_reg .

[0043] The method for calculating the identity classification loss function at intersections is as follows: ; In the formula, M represents the total number of vehicle identity categories in the training set; p C The probability distribution of vehicle identity predictions in an intersection image by a vehicle feature extraction model; q C The vehicles in the intersection map are exactly the distribution of their actual labels.

[0044] The calculation method for the registration domain identity classification loss function is as follows: ; In the formula, p R The probability distribution of vehicle identity predicted by the vehicle feature extraction model from the registration map; q R This is to show the distribution of the vehicle's true identity tags in the registration image.

[0045] Through this multi-task joint loss function L total Perform end-to-end backpropagation optimization, where L ID_cam and L ID_reg L is responsible for coarse-grained identity clustering in the global feature space (identifying who owns the car). triplet The two are responsible for fine-grained push-pull of difficult examples in asymmetric cross-domain space (overcoming the gap between viewpoint and occlusion). They complement each other and jointly train the backbone network and generalized mean pooling layer GeM in a lightweight vehicle feature extraction model with strong robustness.

[0046] Compared to existing patents that use symmetric domain training samples and symmetric loss functions, this invention uses asymmetric domain training sample data to construct an asymmetric cross-domain triplet loss function, concentrating hard example mining computational power on the "intersection-registration" cross-domain gap. Experiments show that the cross-domain feature matching accuracy (mAP) jumps from the existing 78.7% to 94.6%.

[0047] S4: In order to deploy the model in engineering, the trained model is built into a re-identification feature extraction module; The re-identification feature extraction module includes: a pre-trained target detection model and a pre-trained vehicle feature extraction model, set sequentially.

[0048] The re-identification feature extraction module also includes: an adaptive semantic occlusion module, which performs human occlusion processing on the intersection domain image; The adaptive semantic occlusion module is positioned between the target detection model and the vehicle feature extraction model. When the input image is an image of the intersection to be identified collected by a monitoring device based on the intersection, the image to be processed with vehicle bounding boxes output by the target detection model is first sent to the adaptive semantic occlusion module for processing, and then sent to the vehicle feature extraction model.

[0049] S5: Build a photo library of registered electric bicycles; The registration image library stores the registration information of registered electric bicycles. The format of the registration information is: [Registration Number Key] reg Registration feature V reg ]; among them, registration feature V reg The vehicle image features are extracted from the registration images collected when registering electric bicycles based on the re-identification feature extraction module.

[0050] Specifically, the system reads registration images of 10,000 electric vehicles offline, performs detection and cropping using a YOLO object detection model, and then inputs them into a pre-trained re-identification feature extraction module. The module then extracts and outputs a 512-dimensional feature vector V through the backbone network and generalized mean pooling layer GeM in a lightweight vehicle feature extraction model. reg The images are then stored in a database. In this embodiment, the registered image library is built based on the FAISS database.

[0051] S6: Deploy the trained re-identification feature extraction module in the computing device and perform the following inference operations.

[0052] a1: Images of electric bicycles on the road are collected by monitoring equipment set up at intersections, denoted as: the intersection image to be identified; the intersection image to be identified is sent to the re-identification feature extraction module. In the re-identification feature extraction model, the input image is processed by the object detection model to extract vehicle bounding boxes. If it is an intersection domain image, it is processed by an adaptive semantic occlusion module to mask the human figure before being sent to the vehicle feature extraction model. In the vehicle feature extraction model, the input image is processed by the backbone network to extract image features. The extracted image features are then sent to the GeM layer to improve local detail recognition. Finally, a bottleneck layer is used to reduce the feature dimensionality to 512 dimensions. That is, the vehicle query feature V corresponding to the electric bicycle to be identified is extracted. query ; a2: Identify the vehicle license plate corresponding to the image of the intersection to be identified, denoted as: vehicle license plate of the intersection to be identified; a3: Based on the license plate number of the vehicle at the intersection to be identified, find the corresponding registration information in the registration image database [Registration License Plate Key]. reg Registration feature V reg ]; a4: Calculate vehicle query features V query and registration feature V reg The similarity between them is denoted as: the similarity to be identified (SIM); Two features V query and V reg Both are multi-dimensional image feature vectors extracted and output by the vehicle feature extraction model. The two sets of feature vectors can be compared directly based on mathematical methods such as cosine similarity and Euclidean distance. The comparison is efficient and meets the requirements of real time. Moreover, the comparison results after multi-dimensional vector calculation are more accurate. a5: After prolonged use, electric bicycles may exhibit inconsistencies in some image features. This inconsistency can also be caused by issues such as shooting angle or abnormal occlusion. Therefore, this application uses a similarity threshold to measure whether two images are consistent. The similarity to be identified (SIM) is compared with the preset similarity threshold (SimTh). If SIM < SimTh, it indicates an anomaly, and an alarm message for abnormal re-identification result is issued. Otherwise, it means that the two are similar and no alarm message needs to be issued.

[0053] During the actual reasoning phase, online prediction is performed. When the intersection camera captures an image, the system uses a multi-threaded mechanism to process it concurrently.

[0054] The first thread uses an OCR model or radio frequency information from image recognition technology to extract the license plate number; the second thread uses a trained lightweight feature extraction network to extract the vehicle's current feature vector V. query Based on the license plate number, retrieve the vehicle's registration feature vector V from the database. reg Calculate the cosine similarity (V) between the two. query V reg In this embodiment, the system presets a similarity threshold SimTh=0.6. When Cosine(V) query V reg When the expression is less than 0.6, it indicates that the two are not similar, which is an abnormal situation.

[0055] In this application, vehicle license plates are collected based on images or radio frequency data of electric bicycles traveling on the road. The registration image of the license plate is found in the registration database. The image features of the electric bicycle are compared with those of the registered image and the image collected by the intersection image acquisition device. If the similarity between the two does not meet the preset similarity threshold, it indicates that the electric bicycle traveling on the road is suspected of using a fake license plate or a counterfeit license plate.

[0056] The computing device used for inference can be a remote server, or it can be a cloud server, an edge computing device, or an existing roadside device with computing capabilities.

[0057] Specifically, in step a2, the method for recognizing vehicle license plates at the intersection to be identified includes the following operations.

[0058] a21: For each vehicle within the bounding box of the intersection image to be identified, confirm whether the complete vehicle license plate can be identified using the OCR model based on image recognition technology; If the complete license plate of the electric bicycle in the vehicle bounding box can be identified based on the OCR model, then the license plate of the vehicle to be identified at the intersection will be obtained, and a3 will be executed; Otherwise, execute a22.

[0059] The specific method for recognizing vehicle license plates based on OCR models and image recognition technology can be implemented using existing technologies.

[0060] a22: For electric bicycles within the vehicle enclosure, license plate data is collected using radio frequency identification equipment; If a complete license plate can be captured, the license plate number of the vehicle at the intersection to be identified will be obtained, and step a3 will be executed; Otherwise, execute a23.

[0061] In practice, the collection of license plate data for electric bicycles using radio frequency identification (RFID) devices involves various scenarios, such as... Figure 3 In the scenario shown, if there is only one vehicle in the intersection image, the license plate data collected by the RFID device can correspond to the electric bicycle identified in the image. However, if there are multiple electric bicycles in the image, the license plates of the electric bicycles can be matched and confirmed using existing RFID devices and video data matching methods, such as the method disclosed in document CN119559587A.

[0062] a23: This will retrieve the vehicle query feature V corresponding to vehicles for which license plate numbers cannot be collected. query The image is added to the abnormal image feature database and will not be included in the subsequent calculations of this re-identification process.

[0063] In the secondary identity verification process, the inability to collect license plate data for electric bicycles on the road is itself an anomaly. Sometimes, the inability to collect the license plate may be due to occasional obstruction preventing image capture, or it may be due to a damaged license plate chip. However, if multiple attempts fail to capture the license plate, manual verification is required. Therefore, the following anomaly data processing operations are performed on the data stored in the abnormal image feature database; c1: When a vehicle query feature V query When adding images to the abnormal image feature database, a similarity comparison is performed with the data in the database. If similar data can be found, increment its corresponding counter CA by 1 and proceed to step c2; Otherwise, save it as a new data item and initialize its counter CA to 1; c2: Compare the counter CA with the preset abnormal threshold; If the value exceeds the preset abnormal threshold, an abnormal alarm message will be issued, prompting manual confirmation. Otherwise, execute c1~c2 repeatedly, and execute step c3 only when the preset time point is reached; c3: Performs periodic zeroing operations on the abnormal image feature database according to a preset frequency.

[0064] In this embodiment, the anomaly threshold is set to 5. If license plate data cannot be collected five times consecutively, it indicates that the license plate itself may be abnormal. After an anomaly alarm is issued, traffic management personnel need to manually confirm and handle subsequent matters. Because the same type of problem for the same vehicle is only processed once on the same day, the clearing operation of the abnormal image feature database is set to 24 hours, and the abnormal image feature database is cleared once every 24 hours.

[0065] Using the technical solution of this invention, firstly, adaptive upper half-region erasure is performed on the intersection image, and the human body region is dynamically masked according to the target aspect ratio to achieve physical-level decoupling of human and vehicle features; secondly, multi-scale generalized mean pooling is introduced based on a lightweight network to enhance the feature extraction of local vehicle details; subsequently, an asymmetric cross-domain triplet loss adapted to multi-sample imbalance is proposed to mine the most difficult positive and negative samples between the intersection and registration domains, and is jointly optimized by combining it with the identity classification loss of cross-domain decoupling, overcoming the problem of measurement inaccuracy caused by the unequal number of cross-domain samples; finally, OCR image recognition technology and feature retrieval technology are combined through multi-threaded concurrency to realize vehicle query feature V. query and registration feature V reg Online comparison. This invention features high cross-domain matching accuracy and low single-path inference latency, making it suitable for deployment in high-concurrency scenarios. It solves the problems of low recognition accuracy caused by occlusion and the "intersection-registration" cross-domain perspective gap in existing electric bicycle identity consistency recognition.

Claims

1. A method for re-identifying electric bicycles, characterized in that, It includes the following steps: S1: Construct the target detection model; The target detection model identifies electric bicycles in the input image; The target detection model takes as input an image including the electric bicycle to be identified, and outputs as the image to be processed with the vehicle bounding box and the coordinates of the vehicle bounding box. The vehicle bounding box is the smallest bounding rectangle of the electric bicycle identified by the model; S2: Construct a vehicle feature extraction model; The vehicle feature extraction model includes: a backbone network, a generalized mean pooling layer (GeM), a bottleneck layer, and a classification head; The classification head in the EfficientNet model is removed and used as the backbone network. The bottleneck layer consists of Linear, BN and PReLU connected in sequence. The classification head is implemented based on a fully connected layer. The input to the vehicle feature extraction model is the image to be processed; The output of the bottleneck layer is the vehicle query feature V. query The output of the classification head is the vehicle query feature V. query Corresponding license plate number Key; S3: Construct training samples based on existing electric bicycle data, and train the target detection model and the vehicle feature extraction model to obtain the trained target detection model and vehicle feature extraction model; S4: Constructed as a re-identification feature extraction module; The re-identification feature extraction module includes: a trained target detection model and a trained vehicle feature extraction model set sequentially; S5: Build a photo library of registered electric bicycles; The registration image library stores registration information for registered electric bicycles. The format of the registration information is: [Registration Number Key] reg Registration feature V reg ]; Among them, registration feature V reg The registration image captured when registering electric bicycles is based on vehicle image features extracted by the re-identification feature extraction module; S6: Arrange the re-identification feature extraction module in a computing device and perform the following operations: a1: The monitoring equipment set up at the intersection collects images of electric bicycles on the road, denoted as: the intersection image to be identified; the intersection image to be identified is sent to the re-identification feature extraction module to extract the vehicle query feature V corresponding to the electric bicycle to be identified. query ; a2: Identify the vehicle license plate corresponding to the image of the intersection to be identified, denoted as: vehicle license plate of the intersection to be identified; a3: Based on the license plate number of the vehicle at the intersection to be identified, find the corresponding registration information [Registration License Plate Key] in the registration image database. reg Registration feature V reg ]; a4: Calculate vehicle query features V query and registration feature V reg The similarity between them is denoted as: the similarity to be identified (SIM); a5: Compare the target similarity SIM with a preset similarity threshold SimTh; If SIM < SimTh, it indicates an anomaly, and an alarm message for abnormal re-identification result is issued. Otherwise, it indicates that the two are similar and no abnormal alarm message needs to be issued.

2. The electric bicycle re-identification method according to claim 1, characterized in that: The re-identification feature extraction module also includes: an adaptive semantic occlusion module; The adaptive semantic occlusion module is located between the target detection model and the vehicle feature extraction model. When the input image is an image of the intersection to be identified collected by a monitoring device based on the intersection, the image to be processed with vehicle bounding boxes output by the target detection model is first sent to the adaptive semantic occlusion module for processing, and then sent to the vehicle feature extraction model. The operation performed in the adaptive semantic occlusion module is as follows: b1: Calculate the aspect ratio R of the vehicle bounding box, and determine the dynamic erase boundary ratio P of the human torso occlusion based on the aspect ratio R. cut ; b2: For vehicles within the bounding box that are higher than P cut The image region of × Height is occluded to force the network to focus on the physical features of the lower half of the region; where Height is the height of the vehicle bounding box. b3: The focused image to be processed is then fed into the vehicle feature extraction model for feature extraction.

3. The electric bicycle re-identification method according to claim 1, characterized in that: The hyperparameter p of the generalized mean pooling layer GeM in the vehicle feature extraction model is set to a value greater than 1.

4. The electric bicycle re-identification method according to claim 2, characterized in that: The aspect ratio is calculated as follows: R = Height / Width; In the formula, Height is the height of the vehicle bounding box, and Width is the width of the vehicle bounding box; The method for calculating the dynamic erasure boundary ratio is as follows: P cut =min(max(0.3,γlog(R)),0.5); In the formula, P cut The dynamic erase boundary ratio is defined by γ, which is an adjustment coefficient, and R is the aspect ratio of the vehicle's bounding box.

5. The electric bicycle re-identification method according to claim 1, characterized in that: In step S3, for the training and construction of the vehicle feature extraction model, the total training loss function L is jointly trained. total ; L total =L ID_cam +L ID_reg +L triplet ; Among them, L ID_cam L is the loss function for identity classification in the intersection domain; ID_reg For the registration domain identity classification loss function; L triplet For dynamic boundary asymmetric cross-domain triplet loss function.

6. The electric bicycle re-identification method according to claim 5, characterized in that: The method for calculating the dynamic boundary asymmetric cross-domain triplet loss function is as follows: ； In the formula, N c N represents the number of intersection area image samples in the sample data; c =P×K c P is the number of vehicle license plates, K c The number of features in the intersection domain image; N r N represents the number of registered domain image samples in the sample data. r =P×K r K r The number of image features in the registration domain; D C→R pos [i] represents the most difficult positive sample with the farthest distance in the same license plate registration map corresponding to the i-th intersection map; D R→C pos [j] represents the most difficult positive sample with the furthest distance among the intersection maps with the same license plate number corresponding to the j-th registration map; D C→R neg [i] represents the closest, most difficult negative sample among the different license plate registration maps corresponding to the i-th intersection map; D R→C neg [j] represents the closest, most difficult negative sample among the different license plate intersections corresponding to the j-th registration map; α i The dynamic boundary parameters corresponding to the i-th intersection image are obtained based on training. α j The dynamic boundary parameters corresponding to the j-th intersection image are obtained based on training. C is the intersection domain feature matrix output by the vehicle feature extraction model, and R is the registration domain feature matrix.

7. The electric bicycle re-identification method according to claim 6, characterized in that: The method for calculating the intersection domain identity classification loss function is as follows: ； In the formula, M represents the total number of vehicle identity categories in the training set; p C The probability distribution of vehicle identity predictions in an intersection image by a vehicle feature extraction model; q C The vehicles in the intersection map are exactly as labeled.

8. The electric bicycle re-identification method according to claim 6, characterized in that: The calculation method for the registration domain identity classification loss function is as follows: ； In the formula, p R The probability distribution of vehicle identity predicted by the vehicle feature extraction model from the registration map; q R This is to show the distribution of the vehicle's true identity tags in the registration image.

9. The electric bicycle re-identification method according to claim 1, characterized in that: In step a2, the method for recognizing vehicle license plates at the intersection to be identified includes the following operations: a21: For each vehicle within the bounding box of the intersection image to be identified, confirm whether the complete vehicle license plate can be identified using the OCR model based on image recognition technology; If the complete license plate of the electric bicycle in the vehicle bounding box can be identified based on the OCR model, then the license plate of the vehicle to be identified at the intersection will be obtained, and a3 will be executed. Otherwise, execute a22; a22: For electric bicycles within the vehicle enclosure, license plate data is collected using radio frequency identification equipment; If a complete license plate can be captured, the license plate number of the vehicle at the intersection to be identified will be obtained, and step a3 will be executed; Otherwise, execute a23; a23: This will retrieve the vehicle query feature V corresponding to vehicles for which license plate numbers cannot be collected. query The image is added to the abnormal image feature database and will not be included in the subsequent calculations of this re-identification process.

10. The electric bicycle re-identification method according to claim 9, characterized in that: The abnormal image feature database stores the following abnormal data processing operations; c1: When a vehicle query feature V query When the image is added to the abnormal image feature database, a similarity comparison is performed with the data in the database. If similar data can be found, increment its corresponding counter CA by 1 and proceed to step c2; Otherwise, save it as a new data item and initialize its counter CA to 1; c2: Compare the counter CA with the preset abnormal threshold; If the value exceeds the preset abnormal threshold, an abnormal alarm message will be issued; Otherwise, proceed to step c3; c3: Perform periodic zeroing operations on the abnormal image feature database according to a preset frequency.

Images