Vehicle identification method and device based on remote sensing image, and electronic equipment

By adopting the identification model of residual network and spatial attention network in vehicle re-identification, combining diversity regularization and consistency regularization technologies, the problem of insufficient vehicle identification accuracy and efficiency is solved, and higher identification accuracy is achieved Sex and robustness.

CN118982797APending Publication Date: 2024-11-19HAINAN UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410998743.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing vehicle re-identification methods have insufficient accuracy and efficiency, especially insufficient global features and incomplete local features, resulting in poor accuracy and low efficiency of vehicle recognition, and are susceptible to viewing angle changes, distortion, occlusion, The influence of factors such as motion fuzzy.

Method used

Using the recognition model based on residual network and spatial attention network, the comprehensive loss function is constructed for end-to-end training through partial diversity regularization and partial consistency regularization, combining cross entropy loss and triple metric loss, and the image is extracted Features and enhances the robustness and generalization capabilities of the model.

Benefits of technology

Improves the accuracy and efficiency of vehicle identification, enhances the robustness of the model to complex backgrounds and multiple shooting conditions, avoids overfitting, and improves performance on different tasks and data sets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982797A_ABST
    Figure CN118982797A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of vehicle re-identification, and particularly discloses a vehicle identification method and device based on a remote sensing image and electronic equipment, and the method comprises the steps: obtaining a to-be-identified remote sensing image of a vehicle; inputting the remote sensing image into a trained recognition model to obtain a recognition result of the remote sensing image; wherein the recognition model is established by using a residual network and a space attention network, the recognition model performs feature map extraction, partial diversity regularization and partial consistency regularization on a remote sensing image sample to determine regularization constraints, and a comprehensive loss function is determined according to the regularization constraints and model training loss; and carrying out end-to-end training on the model by utilizing a comprehensive loss function. According to the invention, the accuracy and efficiency of vehicle identification can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of vehicle re-identification, and more specifically, to a vehicle identification method, device and electronic equipment based on remote sensing images. Background Art

[0002] Vehicle re-identification (Re-ID) aims to retrieve a given query image from a large-scale image database. With the rapid development of road traffic networks, vehicle re-identification plays an increasingly critical role in intelligent transportation systems and smart cities.

[0003] In the physical world, this is a challenging task that cannot be ignored due to the high inter-class similarity and substantial intra-class differences between vehicles. The reasons for this should be related to the following points: substantial intra-class differences caused by different viewpoints, low resolution, and high exposure; and the high inter-class similarity caused by the density of vehicles of the same model.

[0004] To address the above issues, some ongoing work focuses on learning global features of vehicles, such as vehicle attributes, license plate information, spatiotemporal information, etc. There are also many works that pay more attention to the combination of loss functions and backbone networks to enhance the focus on global features. Global features can well describe the overall structure of the vehicle, but so far there are still some shortcomings, such as redundant information and lack of ability to capture finer-grained visual differences, which is crucial in vehicle re-identification. In addition, they are easily affected by factors such as perspective changes, distortion, occlusion, motion blur, and lighting. Vehicle re-identification models need to have the ability to capture distinguishable fine-grained features, so current technology is difficult to guarantee the accuracy and efficiency of vehicle recognition.

[0005] In summary, the current vehicle re-identification methods still have some defects and problems, such as insufficient global features, limited annotation data, and incomplete local features, which lead to poor accuracy and low efficiency of vehicle re-identification. Summary of the invention

[0006] In view of the defects of the prior art, the purpose of this application is to provide a vehicle identification method, device and electronic equipment based on remote sensing images, aiming to solve the problems of poor accuracy and low efficiency of re-identification.

[0007] In a first aspect, the present application provides a vehicle recognition method based on remote sensing images, comprising:

[0008] Acquire a remote sensing image of a vehicle to be identified;

[0009] Inputting the remote sensing image into a trained recognition model to obtain a recognition result of the remote sensing image;

[0010] Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

[0011] This application uses partial diversity regularization and partial consistency regularization to help the model better generalize to new vehicle images during the training process and avoid overfitting, thereby improving the accuracy of vehicle recognition. It also comprehensively considers regularization constraints and model training losses, which helps to optimize the overall performance of the model and further improve the accuracy and efficiency of vehicle recognition.

[0012] Optionally, the training method of the recognition model includes:

[0013] Use residual network and spatial attention network to build the initial model;

[0014] Inputting the remote sensing image sample into the initial model to obtain a classification result of the remote sensing image sample, and obtaining a cross entropy loss according to the classification result and the vehicle identification label;

[0015] Obtain N feature images corresponding to the remote sensing image samples according to the residual network; N is a positive integer, and the triplet metric loss is determined using the feature images;

[0016] Obtaining N attention maps from the N feature images according to the spatial attention network, and obtaining a partial diversity regularization constraint according to the N attention maps and a unit matrix;

[0017] Performing global average pooling on the feature image and the attention map to obtain diversified partial features, and obtaining partial consistency regularization constraints according to the diversified partial features and the central features;

[0018] Determining the comprehensive loss function according to the cross entropy loss, the triplet metric loss, the partial diversity regularization constraint, and the partial consistency regularization constraint;

[0019] The initial model is iteratively updated using the comprehensive loss function, and model parameters are adjusted until the model converges to obtain the trained recognition model.

[0020] The present application can effectively extract image features and solve the gradient vanishing problem through residual connections, which helps to build deeper networks to improve expression capabilities. The spatial attention network can enhance the model's attention to important image areas, thereby improving the model's recognition accuracy and robustness in complex backgrounds.

[0021] Cross entropy loss is used to measure the difference between the model output and the true label in the classification task. It is a commonly used supervised learning loss function and helps to optimize the model's classification ability. Triplet loss is used to measure the similarity of features of vehicles with the same identity in the feature space. By minimizing the distance between features of vehicles with the same identity and maximizing the distance between features of vehicles with different identities, the model's performance in vehicle re-identification tasks is improved. Partial diversity regularization constraints enhance the model's ability to learn diverse features through attention maps, which helps to improve the model's robustness to complex backgrounds and various shooting conditions. Partial consistency regularization constraints help the model better integrate different feature perspectives to improve the consistency and stability of feature expression.

[0022] Taking into account multiple loss functions and regularization terms, we can fully optimize the model's training objectives, making the model perform better on different tasks and data sets. By iteratively updating and optimizing the comprehensive loss function, we can effectively improve the model's ability to fit the training data and achieve better generalization performance on the validation data.

[0023] Optionally, the process of acquiring the attention map further includes:

[0024] Partial diversity regularization is used to make each attention map focus on different parts of the feature image, so as to achieve the positioning of the N attention maps to different semantic parts of the same vehicle.

[0025] Optionally, the determining the triplet metric loss by using the feature image comprises:

[0026] Determine the source sample, positive sample and negative sample of the triplet in the k-th feature image, and obtain the global high-dimensional feature vectors corresponding to the source sample, positive sample and negative sample to obtain the source sample feature vector, the positive sample feature vector and the negative sample feature vector, where k is a positive integer;

[0027] Obtaining a first Euclidean distance between the source sample feature vector and the positive sample feature vector, and a second Euclidean distance between the source sample feature vector and the negative sample feature vector;

[0028] The triplet metric loss is determined according to the first Euclidean distance, the second Euclidean distance, and a boundary threshold of the triplet loss.

[0029] Optionally, obtaining a partial diversity regularization constraint according to the N attention maps and the identity matrix includes:

[0030] Sum the N attention maps to obtain an attention matrix;

[0031] Determine a transposed matrix of the attention matrix, and determine the partial diversity regularization constraint according to a matrix norm of the attention matrix, the transposed matrix, and the identity matrix.

[0032] Optionally, the partial consistency regularization constraint makes the partial features gradually approach the central feature by minimizing the inter-class variance of the partial features learned by the k-th branch;

[0033] The central feature is initialized from 0 and updated based on the moving average.

[0034] Optionally, obtaining N attention maps from the N feature images according to the spatial attention network includes:

[0035] Using the convolutional layer of the spatial attention network to obtain point-to-point spatial information of the feature image;

[0036] The spatial information is activated by using a sigmoid function, and weights are allocated to key target areas of the feature image to obtain the attention map.

[0037] In a second aspect, a vehicle identification device based on remote sensing images comprises:

[0038] An acquisition unit, used for acquiring a remote sensing image of a vehicle to be identified;

[0039] A recognition unit, used for inputting the remote sensing image into a trained recognition model to obtain a recognition result of the remote sensing image;

[0040] Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

[0041] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0043] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0044] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0045] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0046] (1) This application uses partial diversity regularization and partial consistency regularization to help the model better generalize to new vehicle images during the training process and avoid overfitting, thereby improving the accuracy of vehicle recognition. It also comprehensively considers regularization constraints and model training losses, which helps to optimize the overall performance of the model and further improve the accuracy and efficiency of vehicle recognition.

[0047] (2) This application uses residual networks and spatial attention networks to establish a recognition model. The residual network can effectively train deep neural networks and help extract features from complex images. The spatial attention network helps the model focus on the most important parts of the image, thereby improving recognition accuracy. And through end-to-end training, it is ensured that the model can learn directly from the original input data without the need to manually design complex feature extractors, thereby improving the efficiency of model training and thus improving the efficiency of vehicle recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a vehicle identification method based on remote sensing images provided in an embodiment of the present application;

[0049] Figure 2 It is a framework diagram of the recognition model in the embodiment of the present application;

[0050] Figure 3 It is a complete structural diagram of the spatial attention network in the embodiment of the present application;

[0051] Figure 4 Schematic diagram of the relationship between the value of N in the embodiment of the present application and the mean average precision on the VeRi-776 data set;

[0052] Figure 5 This is a schematic diagram of the effect of changing the number of branches on the model in the embodiment of the present application;

[0053] Figure 6 is a schematic diagram of the structure of a vehicle identification device based on remote sensing images provided in an embodiment of the present application;

[0054] Figure 7It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] The term "and / or" in this application is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this application indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0057] The terms "first" and "second" in the specification and claims of this application are used to distinguish different objects rather than to describe a specific order of objects. For example, the first Euclidean distance and the second Euclidean distance are used to distinguish different Euclidean distances rather than to describe a specific order of Euclidean distances.

[0058] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0059] Next, the technical solutions provided in the embodiments of the present application are introduced.

[0060] Reference Figure 1 , the present application provides a vehicle recognition method based on remote sensing images, comprising:

[0061] S101. Acquire a remote sensing image of a vehicle to be identified;

[0062] S102. Inputting the remote sensing image into the trained recognition model to obtain the recognition result of the remote sensing image;

[0063] Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

[0064] Specifically, firstly, images that need to be used for vehicle recognition are obtained. These images can come from a camera, a video stream recorded by a camera, or a static image file.

[0065] The remote sensing image is then input into a pre-trained recognition model. The recognition model is constructed by using a residual network (ResNet) and a spatial attention network, which can effectively extract complex features in the image, and the spatial attention network can focus on the most critical areas in the image.

[0066] The model first extracts feature maps from remote sensing image samples, that is, extracts deep feature representations from input images through a residual network. In order to improve the generalization ability and accuracy of the model, partial diversity regularization and partial consistency regularization are used. Regularization technology helps the model better handle the diversity of data during training and maintain consistency to avoid overfitting. The comprehensive loss function is obtained by combining the regularization constraint and the model training loss. The comprehensive loss function is used to optimize the model during training so that it can effectively identify vehicles under various input conditions.

[0067] Using a comprehensive loss function to perform end-to-end training on the entire recognition model can optimize the performance of the model to the greatest extent, making it more stable and reliable in practical applications.

[0068] The embodiments of the present application can effectively implement vehicle recognition tasks and improve recognition accuracy and efficiency by combining advanced network architecture, regularization technology and end-to-end training methods.

[0069] Optionally, the training method of the recognition model includes:

[0070] Use residual network and spatial attention network to build the initial model;

[0071] Inputting the remote sensing image sample into the initial model to obtain a classification result of the remote sensing image sample, and obtaining a cross entropy loss according to the classification result and the vehicle identification label;

[0072] Obtain N feature images corresponding to the remote sensing image samples according to the residual network; N is a positive integer, and the triplet metric loss is determined using the feature images;

[0073] Obtaining N attention maps from the N feature images according to the spatial attention network, and obtaining a partial diversity regularization constraint according to the N attention maps and a unit matrix;

[0074] Performing global average pooling on the feature image and the attention map to obtain diversified partial features, and obtaining partial consistency regularization constraints according to the diversified partial features and the central features;

[0075] Determining the comprehensive loss function according to the cross entropy loss, the triplet metric loss, the partial diversity regularization constraint, and the partial consistency regularization constraint;

[0076] The initial model is iteratively updated using the comprehensive loss function, and model parameters are adjusted until the model converges to obtain the trained recognition model.

[0077] This embodiment is a detailed process of model training.

[0078] Reference Figure 2 , Figure 2 It is a framework diagram of the recognition model in the embodiment of the present application.

[0079] Specifically, the backbone of the model is the residual network ResNet-50, which performs well in target re-identification tasks. After the res conv4 module, the network is divided into N independent branches, each of which points to a res conv5 module, where the final spatial downsampling step is changed from 2 to 1. Through this operation, N feature maps {F k}, where F k ∈Rh×w×C. Next, these feature maps are forced to learn different semantic information in subsequent operations.

[0080] Before performing the partial diversity regularization operation, the semantic information of these features {Fk} is nearly repeated. Because each res conv5 module is initialized with the same parameters and is subject to the same constraints during training. Through the regularization operation, the feature maps will be represented as different parts of the vehicle. That is, through partial diversity regularization, these high-dimensional feature maps are constrained to represent different parts of the vehicle, enhancing the network's ability to learn different semantic attributes.

[0081] Partial diversity regularization is used to make each attention map focus on different parts of the feature image, so as to achieve the positioning of the N attention maps to different semantic parts of the same vehicle.

[0082] After the above stages, multiple similar semantic features of a vehicle are obtained {F k}. We further use multiple spatial attention networks to guide the corresponding branches to autonomously locate different salient parts of a vehicle. Figure 3 , the complete structure of SAM is shown in the figure. For the feature map F corresponding to the kth branch k, which is fed into two 1×1 convolutional layers to obtain point-to-point spatial information, and then a Sigmoid layer is applied to calculate the spatial attention map A k ∈Rh×w, as shown below:

[0083] A k =(W1(0( k )))

[0084] Among them, σ represents the sigmoid function, W1 and W0 represent the parameters of the convolution layer. Through the above steps, N attention maps can be obtained.

[0085] Further:

[0086]

[0087] In this embodiment, each attention map obtained contains the importance weights of each area in the input image. These weights can be used to determine which areas are considered more important for vehicle re-identification. However, when looking at each attention map individually, the information of these areas is redundant, that is, multiple attention maps may focus on the same or similar image areas and cannot effectively distinguish different local features. Therefore, it is necessary to introduce a partial diversity regularization to force each attention map to focus on different parts of the input image, ensuring that each branch can locate different semantic parts of the same vehicle.

[0088] Optionally, obtaining a partial diversity regularization constraint according to the N attention maps and the identity matrix includes:

[0089] Sum the N attention maps to obtain an attention matrix;

[0090] Determine a transposed matrix of the attention matrix, and determine the partial diversity regularization constraint according to a matrix norm of the attention matrix, the transposed matrix, and the identity matrix.

[0091] Furthermore, this application introduces a new penalty term, namely the partial diversity regularization constraint, to encourage the spatial attention model to focus on different salient areas:

[0092]

[0093] Among them, A T Indicates A T The transpose of . ‖·‖ F represents the Frobenius norm of the matrix, and I represents the identity matrix. This penalty term Ld will be multiplied by a coefficient and minimized together with other losses. Minimizing this penalty term means learning an orthogonal matrix so that each row in A only focuses on one aspect of the semantics.

[0094] Optionally, the partial consistency regularization constraint makes the partial features gradually approach the central feature by minimizing the inter-class variance of the partial features learned by the k-th branch;

[0095] The central feature is initialized from 0 and updated based on the moving average.

[0096] Partial consistency regularization is as follows:

[0097] It should be noted that the spatial misalignment between features will seriously affect the performance of the re-identification task. Therefore, for any input image, it is expected that a single branch of the network can always capture consistent parts. Inspired by the center loss, partial consistency regularization is proposed to achieve this goal. Specifically, firstly, a global average pooling (GAP) operation is used to extract diverse partial features:

[0098] f k =AP(A k ⊙F k )

[0099] Where ⊙ represents an element-wise product operation.

[0100] Then, minimize the feature f learned by the kth branch of the network k The inter-class variance makes the feature gradually approach the central feature c of the part k . The loss function can be expressed as:

[0101]

[0102] Among them, c k It can be initialized from zero and updated by a moving average.

[0103] c k ←c k -(c k -k)

[0104] Among them, α controls c k The update rate.

[0105] Optionally, obtaining N attention maps from the N feature images according to the spatial attention network includes:

[0106] Using the convolutional layer of the spatial attention network to obtain point-to-point spatial information of the feature image;

[0107] The spatial information is activated by using a sigmoid function, and weights are allocated to key target areas of the feature image to obtain the attention map.

[0108] For the feature map F of the kth branch, it first passes through two 1×1 convolutional layers to obtain point-to-point spatial information, and then uses the sigmoid function to activate the spatial information so that it assigns greater weights to more important areas. The final spatial attention activation map A is obtained. k .

[0109] Optionally, the determining the triplet metric loss by using the feature image comprises:

[0110] Determine the source sample, positive sample and negative sample of the triplet in the k-th feature image, and obtain the global high-dimensional feature vectors corresponding to the source sample, positive sample and negative sample to obtain the source sample feature vector, the positive sample feature vector and the negative sample feature vector, where k is a positive integer;

[0111] Obtaining a first Euclidean distance between the source sample feature vector and the positive sample feature vector, and a second Euclidean distance between the source sample feature vector and the negative sample feature vector;

[0112] The triplet metric loss is determined according to the first Euclidean distance, the second Euclidean distance, and a boundary threshold of the triplet loss.

[0113] Specifically, the weakly supervised multi-component vehicle re-identification network aims to extract features with global expressiveness and local discriminability for vehicle identity. Therefore, the network of this application adopts a multi-task training method, and the loss function is mainly divided into four parts: the first part is the cross entropy loss, which calculates the loss value of the network with vehicle identity as the label. The second part is the triplet metric loss, which aims to push negative sample pairs away and pull positive sample pairs closer. The third part is the partial diversity regularization constraint, which allows multiple attention models to focus on different parts of the same input. The fourth part is the partial consistency regularization constraint, which ensures that each branch can always extract the same part of different inputs.

[0114] After obtaining the local feature vector {F k}, a batch normalization layer BN, a linear layer W, and a softmax layer are used to calculate the classification loss to improve the generalization ability of the network.

[0115] As shown in the following formula:

[0116] p i =softmax(W*BN( k ))

[0117]

[0118] In addition, for each branch’s local feature vector {F k}, a hard triplet loss is used to reduce the distance between positive sample pairs and push the distance between negative sample pairs further. Ultimately, vehicle images with the same identity form clusters in the feature space, achieving the goal of vehicle re-identification.

[0119] L t =d(f ka ,kp)-d(f ka ,kn)+] +

[0120] Among them, f ka 、f kp 、f kn They represent the global high-dimensional feature vectors of the source samples, positive samples, and negative samples that constitute the triplet in the k-th branch. ka ,kp) and d(1 ka ,kn) represent the Euclidean distance between the positive sample and the source sample and the Euclidean distance between the negative sample pair and the source sample, respectively. They represent the Euclidean distance of the positive sample pair and the negative sample pair, respectively. m represents the boundary threshold of the triplet loss, and + represents a positive value.

[0121] The final loss function includes the triplet loss function, the classification loss function, partial diversity regularization, and partial consistency regularization for each branch. The total loss function of a single branch is:

[0122] L=L s + r +1L d +1L λ

[0123] Among them, λ1 and λ2 are loss function weights, which are used to balance the influence of each loss function.

[0124] Furthermore, in order to evaluate the MRL method proposed in this application, this application conducted experiments on three public large-scale vehicle re-identification datasets: VehicleID, VeRi-776, and VRIC.

[0125] A. Datasets and Evaluation Metrics

[0126] The VehicleID dataset includes 221,763 images of 26,267 vehicles captured by multiple non-overlapping real-world surveillance cameras during the day. Each vehicle has a front view or a rear view. In the VehicleID dataset, the images used for training include 110,178 images of 13,134 vehicles, while the images used for testing include 111,585 images of 13,133 vehicles. There are three subsets in the test set, namely the small test set includes 7,332 images of 800 vehicles, the medium test set includes 12,995 images of 1,600 vehicles, and the large test set includes 20,038 images of 2,400 vehicles.

[0127] VeRi-776 contains more than 50,000 images of 776 vehicles, captured by 20 cameras over 24 hours. Each vehicle was captured by 2-18 cameras with different viewpoints, lighting, resolutions, and occlusions. All datasets are divided into a training set, including 37,778 images of 576 vehicles, and a test set, including 11,579 images of 200 vehicles.

[0128] The VRIC dataset has unique characteristics in that vehicle images are subject to more realistic and unconstrained variations in resolution (scale), motion blur, lighting, occlusion, and perspective. It contains 60,430 images covering 5,622 vehicle identities captured by 60 different cameras in heterogeneous road traffic scenes at day and night. The training set contains 54,808 images of 2,811 vehicles, and the remaining set is used for testing, including 5,622 images of another 2,811 vehicle identities.

[0129] Evaluation Metrics. To measure the performance of vehicle re-identification, this application uses the cumulative matching characteristic (CMC) and mean average precision (mAP) as evaluation criteria. CMC calculates the cumulative percentage of correct matches before the top K candidates appear. This application reports the top 1 and top 5 scores to represent the CMC criterion. Given a query image, the mean average precision (AP) is the area under the Precision-Recall curve, while mAP is the average of the APs of all query images. The mAP criterion reflects both precision and recall, providing a more convincing evaluation of the re-identification task.

[0130] B. Experimental Setup

[0131] The model of this application uses ResNet-50 pre-trained on ImageNet as the backbone network, and the res conv5 in different branches do not share parameters. In addition, the model removes the last spatial downsampling operation to increase the spatial size of the output feature map, and uses an additional batch normalization (BN) operation before the Softmax layer.

[0132] During the training phase, the input images were resized to 256×256 and then augmented by random horizontal flipping, normalization, and random erasing. The network of this application was trained using 2 2080Ti GPUs with a batch size of 64. Each batch contains 16 identities, each with 4 instances. This application uses the Adam optimizer with an initial learning rate of 2e-4 and then decays by 0.1 every 40 epochs. The total number of training rounds is 130. Training takes about 3 hours on the VeRi-776 dataset. 60,430 images of 5,622 vehicle identities captured in heterogeneous road traffic scenes at night. The training set has 54,808 images of 2,811 vehicles, and the rest is used for testing, including another 5,622 images of 2,811 vehicle identities.

[0133] Evaluation Metrics. To measure the performance of vehicle re-identification, this application uses the cumulative matching characteristic (CMC) and mean average precision (mAP) as evaluation criteria. CMC calculates the cumulative percentage of correct matches before the top K candidates appear. This application reports the top 1 and top 5 scores to represent the CMC criterion. Given a query image, the mean average precision (AP) is the area under the Precision-Recall curve, while mAP is the average of the APs of all query images. The mAP criterion reflects both precision and recall, providing a more convincing evaluation of the re-identification task.

[0134] Reference Figure 4 , Figure 4 It is a schematic diagram of the relationship between the value of N and the average precision mean on the VeRi-776 data set. It can be seen from the figure that when N=3, the average precision reaches the highest level of 82.6%.

[0135] Reference Figure 5 , Figure 5 This is a schematic diagram of the effect of changing the number of branches on the model. In the figure, the blue line indicates branch 1, the orange line indicates branch 5, and the green line indicates branch 10. As can be seen from the figure, as the number of branches increases, the accuracy of the model will continue to improve.

[0136] Further, the table shows that the method proposed in this application exhibits superior performance. Compared with the PRN model, the MCEN method introduced in this study shows an 8.3% improvement in mAP and a 3.1% enhancement in Top-1 accuracy. Traditional feature-based methods (FACT+Plate-SNN+STR) may face challenges in adapting to complex and dynamic environments. Methods that rely on local features extracted by detectors (PRN, AAVER, OIFE) may produce inaccurate detection results when the camera angle changes, resulting in inaccurate recognition. The use of hard partitioning techniques (RAM) for local features may destroy the spatial structure of the vehicle, resulting in incomplete feature representation. The MCEN method adopts an end-to-end learning strategy, and its partial diversity regularization enables the network to recognize multiple semantic features in the input image, enhancing its anti-occlusion ability in complex environments. Partial consistency regularization ensures a high degree of alignment between features and solves the problem of feature space misalignment caused by changes in camera angles. Therefore, the MCEN model achieves excellent re-identification results.

[0137] When the proposed method is compared with the state-of-the-art methods on the VehicleID dataset, the results are shown in Tables 1 and 2. The images in this dataset are taken by surveillance cameras from the front or rear of the vehicle, and they are often affected by overexposure, color distortion, and significant resolution variations. This requires a model with high generalization ability and robustness.

[0138] Unlike other methods, the MCEN network uses partial consistency regularization and partial diversity regularization to extract multiple consistent feature representations of vehicles. In the VehicleID dataset, images are not affected by changes in perspective, and the partial consistency regularization term enables the network to have a stronger perception of local features with the same semantic attributes. This enables the network to match between the same parts, making the network more stable. Experimental results show that the MCEN network achieves higher results on small, medium, and large subsets of the VehicleID dataset, verifying the effectiveness and superiority of the model algorithm proposed in this application.

[0139] Table 1 Comparison with the most advanced methods

[0140]

[0141] Table 2 Comparison of detection effects at different scales

[0142]

[0143] Reference Figure 6 The present application also provides a vehicle identification device based on remote sensing images, comprising:

[0144] An acquisition unit 610 is used to acquire a remote sensing image of a vehicle to be identified;

[0145] The recognition unit 620 is used to input the remote sensing image into the trained recognition model to obtain the recognition result of the remote sensing image;

[0146] Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

[0147] Optionally, the training process of the recognition model includes:

[0148] A model building unit, which is used to build an initial model using a residual network and a spatial attention network;

[0149] A cross entropy loss acquisition unit, used for inputting the remote sensing image sample into the initial model to obtain a classification result of the remote sensing image sample, and obtaining a cross entropy loss according to the classification result and the vehicle identification label;

[0150] A triplet metric loss acquisition unit is used to obtain N feature images corresponding to the remote sensing image sample according to the residual network, wherein N is a positive integer, and the triplet metric loss is determined by using the feature image;

[0151] A partial diversity regularization constraint unit, used for obtaining N attention maps from the N feature images according to the spatial attention network, and obtaining a partial diversity regularization constraint according to the N attention maps and a unit matrix;

[0152] A partial consistency regularization constraint unit, used for performing global average pooling on the feature image and the attention map to obtain diversified partial features, and obtaining partial consistency regularization constraints according to the diversified partial features and the central features;

[0153] A comprehensive loss unit, used for determining the comprehensive loss function according to the cross entropy loss, the triple metric loss, the partial diversity regularization constraint and the partial consistency regularization constraint;

[0154] The iterative updating unit is used to iteratively update the initial model using the comprehensive loss function, adjust the model parameters until the model converges, and obtain the trained recognition model.

[0155] Optionally, the partial diversity regularization constraint unit is further used for:

[0156] Partial diversity regularization is used to make each attention map focus on different parts of the feature image, so as to achieve the positioning of the N attention maps to different semantic parts of the same vehicle.

[0157] Optionally, the triplet metric loss acquisition unit is used to:

[0158] Determine the source sample, positive sample and negative sample of the triplet in the k-th feature image, and obtain the global high-dimensional feature vectors corresponding to the source sample, positive sample and negative sample to obtain the source sample feature vector, the positive sample feature vector and the negative sample feature vector, where k is a positive integer;

[0159] Obtaining a first Euclidean distance between the source sample feature vector and the positive sample feature vector, and a second Euclidean distance between the source sample feature vector and the negative sample feature vector;

[0160] The triplet metric loss is determined according to the first Euclidean distance, the second Euclidean distance, and a boundary threshold of the triplet loss.

[0161] Optionally, a partial diversity regularization constraint unit is used to:

[0162] Sum the N attention maps to obtain an attention matrix;

[0163] Determine a transposed matrix of the attention matrix, and determine the partial diversity regularization constraint according to a matrix norm of the attention matrix, the transposed matrix, and the identity matrix.

[0164] Optionally, the partial consistency regularization constraint makes the partial features gradually approach the central feature by minimizing the inter-class variance of the partial features learned by the k-th branch;

[0165] The central feature is initialized from 0 and updated based on the moving average.

[0166] Optionally, the partial diversity regularization constraint unit is used to:

[0167] Using the convolutional layer of the spatial attention network to obtain point-to-point spatial information of the feature image;

[0168] The spatial information is activated by using a sigmoid function, and weights are allocated to key target areas of the feature image to obtain the attention map.

[0169] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0170] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0171] Reference Figure 7 Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, which may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the method in the above embodiment.

[0172] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0173] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0174] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0175] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0176] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0177] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0178] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0179] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A vehicle recognition method based on remote sensing images, characterized in that: include: Acquire a remote sensing image of a vehicle to be identified; Inputting the remote sensing image into a trained recognition model to obtain a recognition result of the remote sensing image; Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

2. The vehicle recognition method based on remote sensing images according to claim 1, characterized in that: The training method of the recognition model includes: Use residual network and spatial attention network to build the initial model; Inputting the remote sensing image sample into the initial model to obtain a classification result of the remote sensing image sample, and obtaining a cross entropy loss according to the classification result and the vehicle identification label; Obtain N feature images corresponding to the remote sensing image samples according to the residual network; N is a positive integer, and the triplet metric loss is determined using the feature images; Obtaining N attention maps from the N feature images according to the spatial attention network, and obtaining a partial diversity regularization constraint according to the N attention maps and a unit matrix; Performing global average pooling on the feature image and the attention map to obtain diversified partial features, and obtaining partial consistency regularization constraints according to the diversified partial features and the central features; Determining the comprehensive loss function according to the cross entropy loss, the triplet metric loss, the partial diversity regularization constraint, and the partial consistency regularization constraint; The initial model is iteratively updated using the comprehensive loss function, and model parameters are adjusted until the model converges to obtain the trained recognition model.

3. The vehicle recognition method based on remote sensing images according to claim 2 is characterized in that: The process of acquiring the attention map further includes: Partial diversity regularization is used to make each attention map focus on different parts of the feature image, so as to achieve the positioning of the N attention maps to different semantic parts of the same vehicle.

4. The vehicle recognition method based on remote sensing images according to claim 2 is characterized in that: The determining the triplet metric loss by using the feature image comprises: Determine the source sample, positive sample and negative sample of the triplet in the k-th feature image, and obtain the global high-dimensional feature vectors corresponding to the source sample, positive sample and negative sample to obtain the source sample feature vector, the positive sample feature vector and the negative sample feature vector, where k is a positive integer; Obtaining a first Euclidean distance between the source sample feature vector and the positive sample feature vector, and a second Euclidean distance between the source sample feature vector and the negative sample feature vector; The triplet metric loss is determined according to the first Euclidean distance, the second Euclidean distance, and a boundary threshold of the triplet loss.

5. The vehicle recognition method based on remote sensing images according to claim 2 is characterized in that: The obtaining of a partial diversity regularization constraint according to the N attention maps and the identity matrix includes: Sum the N attention maps to obtain an attention matrix; Determine a transposed matrix of the attention matrix, and determine the partial diversity regularization constraint according to a matrix norm of the attention matrix, the transposed matrix, and the identity matrix.

6. The vehicle recognition method based on remote sensing images according to claim 2 is characterized in that: The partial consistency regularization constraint makes the partial features gradually approach the central features by minimizing the inter-class variance of the partial features learned by the k-th branch; The central feature is initialized from 0 and updated based on the moving average.

7. The vehicle recognition method based on remote sensing images according to claim 2 is characterized in that: The step of obtaining N attention maps from the N feature images according to the spatial attention network includes: Using the convolutional layer of the spatial attention network to obtain point-to-point spatial information of the feature image; The spatial information is activated by using a sigmoid function, and weights are allocated to key target areas of the feature image to obtain the attention map.

8. A vehicle identification device based on remote sensing images, characterized in that: include: An acquisition unit, used for acquiring a remote sensing image of a vehicle to be identified; A recognition unit, used for inputting the remote sensing image into a trained recognition model to obtain a recognition result of the remote sensing image; Among them, the recognition model is established using a residual network and a spatial attention network. The recognition model determines the regularization constraints by performing feature map extraction, partial diversity regularization, and partial consistency regularization on remote sensing image samples, determines the comprehensive loss function based on the regularization constraints and model training losses, and uses the comprehensive loss function to perform end-to-end training on the model.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Vehicle re-recognition method based on visual representation cross-sensor invariance

    CN113792686A

  • Optical remote sensing target detection method for accurate positioning

    CN114821356A

  • Vehicle re-identification method based on multi-loss joint learning

    CN115049989A