Dual attention cross-modal vehicle re-identification method, system, device and medium

Abstract

The present invention provides a dual attention cross-modal vehicle re-identification method, system, device and medium. The identification method includes: obtaining a vehicle image to be checked and multiple reference vehicle images; inputting multiple reference vehicle images and the vehicle image to be checked into a feature extraction network in a re-identification model, forming target features from global features, local features and global-local features corresponding to the reference vehicle image, and forming features to be checked from global features, local features and global-local features corresponding to the vehicle image to be checked; the classification network in the re-identification model determines the feature comprehensive similarity between the vehicle image to be checked and each reference vehicle image according to the target feature and the feature to be checked; the classification network in the re-identification model selects multiple vehicle images whose feature comprehensive similarity with the vehicle image to be checked meets preset matching conditions from multiple reference vehicle images according to the feature comprehensive similarity. The present invention has the advantage of high vehicle image re-identification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle image recognition, and in particular to a dual-attention cross-modal vehicle re-identification method, system, device and medium. Background Art

[0002] Vehicle re-identification is a technology for retrieving designated vehicles across non-overlapping areas. Vehicle re-identification technology can be used to search and track vehicles under the monitoring equipment of the entire network, so that designated target vehicles can be tracked and searched across regions. In the existing field of vehicle re-identification, a vehicle image is given, and the image most similar to the vehicle image is found in the vehicle image library as a re-identification image. However, in the existing vehicle image library, no training is performed on the vehicle images within it, resulting in low vehicle image re-identification accuracy. Therefore, there is room for improvement. Summary of the invention

[0003] The present invention provides a dual-attention cross-modal vehicle re-identification method, system, device and medium to improve the problem of low vehicle image re-identification accuracy in the prior art.

[0004] The present invention provides a dual-attention cross-modal vehicle re-identification method, the method comprising:

[0005] Acquire a to-be-checked vehicle image and a plurality of reference vehicle images;

[0006] Inputting a plurality of the reference vehicle images and the vehicle image to be checked into a feature extraction network in a re-identification model, performing feature extraction in the spatial domain and the channel domain to obtain global features, performing feature extraction in the spatial domain and the channel domain and performing multi-equal division processing to obtain local features, fusing the corresponding global features and local features to obtain global-local features, forming target features with the global features, local features, and global-local features corresponding to the reference vehicle image, and forming features to be checked with the global features, local features, and global-local features corresponding to the vehicle image to be checked;

[0007] The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each of the reference vehicle images according to the target feature and the to-be-checked feature;

[0008] The classification network in the re-identification model selects, from the multiple reference vehicle images, multiple vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet preset matching conditions according to the comprehensive feature similarity, and records them as a re-identification image set. In the re-identification image set, the multiple vehicle images are sorted based on the comprehensive feature similarity.

[0009] In one embodiment of the present invention, the step of acquiring the image of the vehicle to be checked and a plurality of reference vehicle images comprises:

[0010] Through different visible light surveillance cameras, images of vehicles to be inspected and multiple reference vehicle images are collected from different perspectives and scenes;

[0011] The image of the vehicle to be checked and the plurality of the reference vehicle images are sequentially subjected to size adjustment, grayscale conversion, denoising, and image graying processing, so that the image of the vehicle to be checked and the plurality of the reference vehicle images have consistent image format and image quality.

[0012] In one embodiment of the present invention, the step of inputting the plurality of reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model and extracting features in a spatial domain and a channel domain to obtain global features comprises:

[0013] Input the plurality of reference vehicle images and the vehicle image to be checked into the residual block and the hybrid module of the feature extraction network of the re-identification model, respectively extract vehicle features from the plurality of reference vehicle images and the vehicle image to be checked, and obtain corresponding input features;

[0014] Through the dual attention mechanism module of the feature extraction network, feature processing is performed on the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked in the spatial domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, and feature processing is performed on the channel domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain;

[0015] The global pooling module of the feature extraction network performs global average pooling processing on the output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain and the output features in the channel domain to obtain global features.

[0016] In one embodiment of the present invention, the step of performing feature processing on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked in the spatial domain to obtain output features of the plurality of reference vehicle images and the vehicle image to be checked in the spatial domain comprises:

[0017] The input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked are processed by convolutional neural network in sequence. The mapping relationship is processed to obtain the corresponding feature weights, wherein the The mapping relationship is used to map the feature vector into a probability distribution;

[0018] Multiplying the input feature by the feature weight to obtain a global context feature;

[0019] The global context features are subjected to bottleneck conversion processing to obtain output features of the plurality of reference vehicle images and the vehicle images to be checked in the spatial domain.

[0020] In one embodiment of the present invention, the step of performing feature processing on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked in the channel domain to obtain output features of the plurality of reference vehicle images and the vehicle image to be checked in the channel domain comprises:

[0021] The input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked are dimensionally transformed to obtain a first feature map, and the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked are dimensionally transformed and transposed in sequence to obtain a second feature map;

[0022] Multiply the first feature map by the second feature map, and perform Mapping relationship processing to obtain the channel attention map;

[0023] Multiplying the channel attention map with the first feature map, and multiplying the multiplication result by a scale coefficient and then performing a dimension conversion process to obtain a channel weighting matrix, wherein the shape of the channel weighting matrix is ​​the same as the shape of the input feature;

[0024] The channel weighting matrix is ​​added to the input feature to obtain output features of a plurality of the reference vehicle images and the vehicle image to be checked in the channel domain.

[0025] In one embodiment of the present invention, the step of inputting the plurality of reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model, extracting features in a spatial domain and a channel domain, and performing multi-equal division processing to obtain local features comprises:

[0026] Inputting the input features corresponding to the plurality of the reference vehicle images and the vehicle image to be checked into the bilinear filter and the dual attention mechanism module of the two-layer multi-scale feature extraction module of the feature extraction network, performing feature processing to obtain a first feature, and performing feature processing on the output features corresponding to the plurality of the reference vehicle images and the vehicle image to be checked on the bilinear filter of the two-layer multi-scale feature extraction module to obtain a second feature;

[0027] The first feature and the second feature are propagated from top to bottom and from bottom to top in the bidirectional feature pyramid network of the two-layer multi-scale feature extraction module to perform feature fusion to obtain deep multi-scale features;

[0028] The deep multi-scale features are subjected to average pooling processing, multi-equal division processing, and dimensionality reduction processing in sequence to obtain local features.

[0029] In one embodiment of the present invention, the re-identification model is trained by the following method:

[0030] Obtaining a training sample set, wherein vehicle images in the training sample set are provided with corresponding vehicle labels;

[0031] Selecting different batches of training vehicle images from the training sample set;

[0032] Inputting different batches of the training vehicle images into a vehicle recognition model to be trained for training, and determining a recognition loss value of a target loss function of the vehicle recognition model to be trained, wherein the vehicle recognition model includes a feature extraction network and a classification network, the feature extraction network is connected to the classification network, and the target loss function includes a feature learning loss corresponding to the global-local features on the feature extraction network, and a total loss of classification losses corresponding to the global features and the local features on the classification network;

[0033] Based on the recognition loss value, the parameters of the vehicle recognition model to be trained are adjusted to obtain a re-recognition model.

[0034] The present invention also proposes a dual-attention cross-modal vehicle re-identification system, the system comprising:

[0035] An acquisition unit, used for acquiring a to-be-checked vehicle image and a plurality of reference vehicle images;

[0036] an extraction unit, configured to input the plurality of the reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model, perform feature extraction in a spatial domain and a channel domain to obtain global features, perform feature extraction in a spatial domain and a channel domain and perform multi-equal division processing to obtain local features, fuse the corresponding global features and local features to obtain global-local features, form target features with the global features, local features, and global-local features corresponding to the reference vehicle images, and form to-be-checked features with the global features, local features, and global-local features corresponding to the to-be-checked vehicle images;

[0037] a processing unit, configured to determine, based on the classification network in the re-identification model, a comprehensive feature similarity between the to-be-checked vehicle image and each of the reference vehicle images according to the target feature and the to-be-checked feature; and

[0038] A matching unit is used to select, based on the classification network in the re-identification model and according to the comprehensive feature similarity, a plurality of vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet a preset matching condition from the plurality of reference vehicle images, record them as a re-identification image set, and sort the plurality of vehicle images in the re-identification image set based on the comprehensive feature similarity.

[0039] The present invention also proposes an electronic device, comprising a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the dual-attention cross-modal vehicle re-identification method as described in any one of the above items.

[0040] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to enable a computer to execute the dual-attention cross-modal vehicle re-identification method as described in any one of the above items.

[0041] Beneficial effects of the present invention: The present invention proposes a dual-attention cross-modal vehicle re-identification method, system, device and medium, wherein a reference vehicle image and a to-be-checked vehicle image are input into a re-identification model, and a feature extraction network of the re-identification model extracts target features corresponding to the reference vehicle image and to-be-checked features corresponding to the to-be-checked vehicle image in the spatial domain and the channel domain. The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each reference vehicle image based on the target features and to-be-checked features, so as to select multiple vehicle images whose comprehensive feature similarity with the to-be-checked vehicle image meets preset matching conditions, thereby obtaining a re-identified image. The present invention has the advantage of high accuracy in vehicle image re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic flow chart of a dual-attention cross-modal vehicle re-identification method provided by an embodiment of the present invention.

[0044] Figure 2 A structural block diagram of an image recognition model provided by one embodiment of the present invention.

[0045] Figure 3 An embodiment of the present invention provides Figure 2 Structural block diagram of the dual attention mechanism module.

[0046] Figure 4 An embodiment of the present invention provides Figure 2 Structural block diagram of the two-layer multi-scale feature extraction module.

[0047] Figure 5 A structural block diagram of a dual-attention cross-modal vehicle re-identification system provided by an embodiment of the present invention.

[0048] Figure 6 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0051] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0052] Please refer to Figures 1 to 6 The present invention proposes a dual-attention cross-modal vehicle re-identification method, system, device and medium, which can be applied in the field of image recognition, for example, in the field of vehicle image recognition, and can search and track vehicle images under the whole network monitoring equipment to realize vehicle re-identification. For example, given a vehicle image to be checked, the vehicle re-identification system performs similarity judgment on the to-be-checked features of the to-be-checked vehicle image and the target features of multiple benchmark vehicle images taken by other cameras, so as to match the most similar vehicle image, and then obtain the re-identified image to complete the vehicle re-identification task. The following is a detailed description through specific embodiments.

[0053] See also Figure 1 , the present invention provides a dual attention cross-modal vehicle re-identification method, which may include the following steps.

[0054] Step S10: Acquire a to-be-checked vehicle image and a plurality of reference vehicle images.

[0055] Step S20, input multiple reference vehicle images and the vehicle image to be checked into the feature extraction network in the re-identification model, perform feature extraction in the spatial domain and the channel domain to obtain global features, perform feature extraction in the spatial domain and the channel domain and perform multi-equal division processing to obtain local features, fuse the corresponding global features and local features to obtain global-local features, form target features with the global features, local features and global-local features corresponding to the reference vehicle image, and form features to be checked with the global features, local features and global-local features corresponding to the vehicle image to be checked.

[0056] Step S30: The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each reference vehicle image according to the target feature and the to-be-checked feature.

[0057] Step S40, the classification network in the re-identification model selects, from multiple reference vehicle images, multiple vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet preset matching conditions based on the comprehensive feature similarity, and records them as a re-identification image set. In the re-identification image set, the multiple vehicle images are sorted based on the comprehensive feature similarity.

[0058] The above steps are described in detail below through specific embodiments.

[0059] Step S10: Acquire a to-be-checked vehicle image and a plurality of reference vehicle images.

[0060] In one embodiment of the present invention, multiple vehicle images are obtained, and the multiple vehicle images are divided into vehicle images to be checked and reference vehicle images. Multiple vehicle images under different viewing angles and different scenes can be collected, and the multiple vehicle images can come from different visible light monitoring cameras or infrared monitoring cameras, and the multiple vehicle images are divided into vehicle images to be checked and reference vehicle images.

[0061] Specifically, the vehicle image to be checked is a vehicle image to be tested, and the vehicle image to be checked can be specified by a user. The reference vehicle image refers to a vehicle image in an image database. For the vehicle image to be checked specified by the user, the most similar vehicle image needs to be matched among multiple vehicle images to be checked and used as the re-recognition image of the vehicle image to be checked.

[0062] Step S20, input multiple reference vehicle images and the vehicle image to be checked into the feature extraction network in the re-identification model, perform feature extraction in the spatial domain and the channel domain to obtain global features, perform feature extraction in the spatial domain and the channel domain and perform multi-equal division processing to obtain local features, fuse the corresponding global features and local features to obtain global-local features, form target features with the global features, local features and global-local features corresponding to the reference vehicle image, and form features to be checked with the global features, local features and global-local features corresponding to the vehicle image to be checked.

[0063] In one embodiment of the present invention, after training the vehicle recognition model to be trained, a re-recognition model can be obtained, that is, the re-recognition model is already trained. The re-recognition model refers to a technology that uses a computer to process, analyze and understand images to identify vehicle images in various modes, and is a recognition model that applies a deep learning algorithm. The purpose of the re-recognition model is to input a vehicle image to be checked into the trained re-recognition model, and to obtain several similar vehicle images from the reference vehicle image.

[0064] Specifically, the training phase corresponds to the process of training the vehicle recognition model to be trained. The training sample set is used for vehicle images trained in the vehicle recognition model to be trained. After the vehicle recognition model to be trained is trained, a trained re-recognition model can be obtained.

[0065] In one embodiment of the present invention, in the testing phase, it corresponds to the process of applying the re-identification model to test the vehicle image to be checked. First, multiple reference vehicle images and the vehicle image to be checked are input into the feature extraction network in the trained re-identification model for feature extraction, and the target features corresponding to the reference vehicle images and the features to be checked corresponding to the vehicle image to be checked can be obtained. Secondly, based on the target features and the features to be checked, the comprehensive similarity of the features between the vehicle image to be checked and each reference vehicle image is determined. A feature is an attribute or characteristic that describes an instance, also known as an independent variable or input variable. The feature can be any type of data, such as numbers, text, images, or audio. In this embodiment, the vehicle features can be the color distribution, license plate number, vehicle outline, etc. of the vehicle.

[0066] Specifically, Figure 2 and Figure 3 As shown, feature extraction is performed on multiple reference vehicle images in the spatial domain and the channel domain to obtain global features corresponding to the multiple reference vehicle images. Feature extraction is performed on the image to be checked in the spatial domain and the channel domain to obtain global features corresponding to the image to be checked.

[0067] like Figure 2 and Figure 3As shown, after extracting features from multiple reference vehicle images in the spatial domain and channel domain and performing multi-equal division processing, local features corresponding to multiple reference vehicle images are obtained. After extracting features from the image to be checked in the spatial domain and channel domain and performing multi-equal division processing, local features corresponding to the image to be checked are obtained.

[0068] like Figure 2 and Figure 3 As shown, the global features and local features corresponding to the multiple reference vehicle images are fused to obtain the global-local features corresponding to the multiple reference vehicle images. The global features and local features corresponding to the image of the vehicle to be checked are fused to obtain the global-local features corresponding to the image of the vehicle to be checked.

[0069] like Figure 2 and Figure 3 As shown, the global features, local features and global-local features of the reference vehicle image are formed into corresponding target features, and the global features, local features and global-local features of the to-be-checked vehicle image are formed into corresponding to-be-checked features.

[0070] Specifically, the spatial domain transforms the spatial domain information in the vehicle image accordingly, so that the key information can be extracted, the space mask is generated and scored. The attention mechanism in the spatial domain ignores the information in the channel domain and treats the image features in each channel equally. In addition, the attention mechanism in the channel domain adds corresponding weights to the signal on each channel to represent the relevance of the channel to the key information. The larger the weight, the higher the relevance.

[0071] By inputting multiple reference vehicle images and vehicle images to be checked into the feature extraction network in the re-identification model and performing feature extraction in the spatial domain and channel domain, the performance of the vehicle re-identification model in cross-regional tasks and the ability to express features can be improved.

[0072] Step S30: The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each reference vehicle image according to the target feature and the to-be-checked feature.

[0073] In one embodiment of the present invention, in step S20, the global features, local features, and global-local features of the reference vehicle image are connected to form the corresponding target features, and the global features, local features, and global-local features of the vehicle image to be checked are connected to form the corresponding features to be checked. In the classification network of the re-identification model, the target features and the features to be checked are compared to determine the comprehensive similarity of features between the vehicle image to be checked and each reference vehicle image. There is no need to compare the global features, local features, and global-local features separately in the vehicle image to be checked and each reference vehicle image.

[0074] Step S40, the classification network in the re-identification model selects, from multiple reference vehicle images, multiple vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet preset matching conditions based on the comprehensive feature similarity, and records them as a re-identification image set. In the re-identification image set, the multiple vehicle images are sorted based on the comprehensive feature similarity.

[0075] In one embodiment of the present invention, multiple vehicle images are selected from multiple reference vehicle images based on the feature comprehensive similarity, and the feature comprehensive similarity between the multiple vehicle images selected from the multiple reference vehicle images and the vehicle image to be checked meets the preset matching condition, and the multiple vehicle images selected from the multiple reference vehicle images are recorded as a re-identification image set. In the re-identification image set, the multiple vehicle images can be sorted based on the feature comprehensive similarity. For example, in the re-identification image set, the multiple vehicle images are sorted in order from high to low based on the feature comprehensive similarity.

[0076] See also Figure 1 In one embodiment of the present invention, step S10 may include step S110 and step S120. Step S110 may be represented as collecting the image of the vehicle to be checked and multiple reference vehicle images at different viewing angles and in different scenes through different visible light monitoring cameras. Step S120 may be represented as sequentially performing size adjustment, grayscale conversion, denoising, and image graying processing on the image of the vehicle to be checked and multiple reference vehicle images, so that the image of the vehicle to be checked and the multiple reference vehicle images have consistent image format and image quality.

[0077] See also Figure 1 and Figure 2 In one embodiment of the present invention, in step S20, a plurality of reference vehicle images and a vehicle image to be checked are input into a feature extraction network in a re-identification model, for example, using The network performs feature extraction to meet the following requirements:

[0078] ;in, It is represented by the vehicle features extracted from multiple reference vehicle images and the vehicle images to be checked, It can be expressed as or network, Represented as multiple reference vehicle images and vehicle images to be checked.

[0079] See also Figure 1 , Figure 2 In one embodiment of the present invention, in step S20, a plurality of reference vehicle images and the image of the vehicle to be checked are input into the feature extraction network in the re-identification model, and the step of extracting features in the spatial domain and the channel domain to obtain global features may include step S210, step S211, and step S212.

[0080] Step S210 can be represented as inputting multiple reference vehicle images and the vehicle image to be checked into the residual block and the hybrid module of the feature extraction network of the re-identification model, respectively extracting vehicle features from the multiple reference vehicle images and the vehicle image to be checked to obtain corresponding input features.

[0081] Step S211 can be represented as a dual attention mechanism module of a feature extraction network, which performs feature processing in the spatial domain on the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked, and obtains output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, and performs feature processing in the channel domain, and obtains output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain.

[0082] Step S212 can be represented as performing global average pooling processing on the output features of multiple reference vehicle images and the vehicle images to be checked in the spatial domain and the output features in the channel domain through the global pooling module of the feature extraction network to obtain global features.

[0083] In one embodiment of the present invention, Figure 2 As shown, the backbone network This can include residual blocks, hybrid modules, and dual attention mechanism modules. For example, in the backbone network It includes the first residual block (Conv1, Conv2), the hybrid module (EFDMx1, EFDMx2), the second residual block (Conv3), the third residual block (Conv4), the fourth residual block (Conv5) and the dual attention mechanism (DAM) module.

[0084] Specifically, after the fourth residual block (Conv5) of the backbone network, a global branch is constructed to extract global features of multiple benchmark vehicle images and the image of the vehicle to be checked. In order to effectively identify vehicles of the same model and similar appearance but different identities in vehicle re-identification, the dual attention mechanism (DAM) module proposed in this paper is inserted after the second residual block (Conv3) of the backbone network.

[0085] like Figure 3 As shown, through the dual attention mechanism module (DAM), the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked are feature processed in the spatial domain to obtain the output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, and feature processed in the channel domain to obtain the output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain.

[0086] See also Figure 1 , Figure 2 and Figure 3 In one embodiment of the present invention, in step S212, the input features corresponding to multiple reference vehicle images and the vehicle images to be checked are processed in the spatial domain to obtain the output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, which may include step S2120, step S2121, and step S2122.

[0087] Step S2120 can be represented as sequentially performing convolutional neural network on the input features corresponding to the multiple reference vehicle images and the vehicle image to be checked. deal with, The mapping relationship is processed to obtain the corresponding feature weights, where: The mapping relationship is used to map the feature vector to a probability distribution.

[0088] Step S2121 can be expressed as multiplying the input feature by the feature weight to obtain the global context feature.

[0089] Step S2122 can be represented as performing bottleneck conversion processing on the global context features to obtain output features of multiple reference vehicle images and the vehicle image to be checked in the spatial domain. .

[0090] Among them, the output features satisfy:

[0091] ;

[0092] Expressed as feature weight, , , Represented as a convolutional neural network processing, Represents bottleneck conversion processing, Represented as layer normalization processing, It is represented by a nonlinear activation function. are the input features corresponding to multiple reference vehicle images and the vehicle image to be checked, For each point in the input feature, the feature map corresponding to the input feature has Points, for , Represented as the height and width of the feature map corresponding to the input feature.

[0093] Specifically, Figure 2 and Figure 3 As shown in FIG. 1 , the dual attention mechanism (DAM) module obtains input features from multiple reference vehicle images through two parallel paths, and obtains input features from the image of the vehicle to be checked through two parallel paths, such as Figure 3 As shown, one is in the spatial domain and the other is in the channel domain. Figure 3 The upper part of the figure is the processing performed on the input features corresponding to the multiple reference vehicle images and the vehicle image to be checked in the spatial domain. Figure 3 The lower part of is the processing of input features corresponding to multiple reference vehicle images and the vehicle image to be checked in the channel domain.

[0094] Specifically, Figure 3 As shown, for the input feature maps of multiple reference vehicle images and the vehicle images to be checked (C×H×W), (C×H×W) represent the channel, height, and width of the input feature map, respectively. Input feature map of a benchmark vehicle image , corresponding to multiple input features on the reference vehicle image. Input feature map of a vehicle image to be checked , corresponding to multiple input features of the vehicle image to be checked. In the spatial domain, a global attention pooling is first performed, introducing a depth-wise separable convolution (1×H×W) and Function (HW×1×1) is used to obtain the feature weights corresponding to the input features.

[0095] Secondly, by summing the feature weights and the input feature map The (C×H×W) matrix is ​​multiplied to obtain the global context feature (C×1×1). Then, the bottleneck operation is used to reduce the amount of calculation. Here, the depth-separable convolution (C / r×1×1), (C×1×1) can reduce the number of parameters and further extract feature information. In addition, the GELU (Gaussian Error Linear Unit) activation function is introduced into the spatial domain to improve the overall performance of the full-text context module. The layer normalization (LN, LayerNorm) processing is introduced before GELU to overcome the optimization difficulty caused by the two-layer feature conversion and act as a regularizer that helps generalization.

[0096] See also Figure 1 , Figure 2 and Figure 3 In one embodiment of the present invention, in step S212, the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked are processed in the channel domain to obtain the output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain, which may include step S2123, step S2124, step S2125 and step S2126.

[0097] Step S2123 can be expressed as performing dimension conversion on the input features corresponding to multiple reference vehicle images and the vehicle images to be checked to obtain a first feature map, and performing dimension conversion and transposition processing on the input features corresponding to multiple reference vehicle images and the vehicle images to be checked in sequence to obtain a second feature map.

[0098] Step S2124 can be represented as multiplying the first feature map by the second feature map and performing Mapping relationship processing to obtain channel attention map .

[0099] Step S2125 may represent the channel attention map Multiply it with the first feature map and multiply the result by the scale factor After that, the dimension conversion process is performed to obtain the channel weighted matrix, and the shape of the channel weighted matrix is ​​the same as the shape of the input feature.

[0100] Step S2126 can be expressed as adding the channel weighting matrix to the input feature to obtain the output features of multiple reference vehicle images and the vehicle image to be checked in the channel domain. .

[0101] Among them, it meets:

[0102] , ;

[0103] is the channel attention map The elements in Expressed as Channel pair The impact of the channels, Represented as output feature The elements in , the scale factor Initialized to 0, and gradually assigned larger weights during training, is the feature matrix of the input features, is the transposed matrix of the input features, is the number of channels corresponding to the input features.

[0104] Specifically, Figure 3 As shown, for the input feature maps of multiple reference vehicle images and the vehicle images to be checked (C×H×W). In the channel domain, the input feature map (C×H×W) is transformed into Reshape (C×H×W) to obtain the first feature map. Perform dimension conversion Reshape and transpose Transpose (H×W×C) to obtain the second feature map. Multiply the first feature map obtained by the second feature map, and then Function processing to obtain channel attention map (C×C).

[0105] Then, the channel attention map With the input feature map Do matrix multiplication and multiply the result by the scaling factor The latter dimension is converted to the original shape (C×H×W) to obtain the channel weight matrix , the channel weighting matrix With the input feature map Add together to get the output features in the channel domain Finally, the input feature map , output features in the spatial domain And the output features in the channel domain Perform feature aggregation to obtain the output features of the dual attention mechanism (DAM) module .

[0106] .

[0107] See also Figure 1 , Figure 2 In one embodiment of the present invention, in step S30, a plurality of reference vehicle images and the vehicle image to be checked are input into the feature extraction network in the trained re-identification model, and the steps of extracting features in the spatial domain and the channel domain and performing multi-equal division processing to obtain local features may include step S310, step S320, and step S330.

[0108] Step S310 can be represented as inputting the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked into the bilinear filter and the dual attention mechanism module of the two-layer multi-scale feature extraction module, performing feature processing to obtain the first feature, and performing feature processing on the output features corresponding to the multiple reference vehicle images and the vehicle images to be checked on the bilinear filter of the two-layer multi-scale feature extraction module to obtain the second feature.

[0109] Step S320 can be represented as fusing the first feature and the second feature in a bidirectional feature pyramid network of a two-layer multi-scale feature extraction module, by propagating the features from top to bottom and from bottom to top, to obtain a deep multi-scale feature.

[0110] Step S330 can be represented as performing average pooling, multi-equal division, and dimensionality reduction processing on the deep multi-scale features in sequence to obtain local features.

[0111] Specifically, Figure 2 As shown, the backbone network Can include residual blocks, hybrid modules and dual attention mechanism modules, such as backbone networks It includes the first residual block (Conv1, Conv2), the hybrid module (EFDMx1, EFDMx2), the second residual block (Conv3), the third residual block (Conv4), the fourth residual block (Conv5) and the dual attention mechanism (DAM) module.

[0112] Specifically, after the fourth residual block (Conv5) of the backbone network, a global branch is constructed to extract global features of multiple benchmark vehicle images and the image of the vehicle to be checked. In order to effectively identify vehicles of the same model and similar appearance but different identities in vehicle re-identification, the dual attention mechanism (DAM) module proposed in this paper is inserted after the second residual block (Conv3) of the backbone network. After performing a global average pooling (GAP) operation on the output of the backbone network, the 2048-dimensional vehicle features are used as global features. Then, BNNeck (Batch Normalization Neck) is used to transform the global features. Perform normalization to obtain normalized features .

[0113] See also Figure 1 , Figure 2 , Figure 3 and Figure 4In one embodiment of the present invention, in the backbone network After the second residual block (Conv3) and the third residual block (Conv4), a depth-by-depth multi-scale feature pyramid branch is introduced to obtain more local details of the vehicle image to be checked and multiple reference vehicle images. The depth-by-depth multi-scale feature pyramid branch consists of a two-layer multi-scale feature extraction module DBL-BiFPN and a horizontal three-block structure based on PCB.

[0114] Specifically, Figure 4 As shown, the two-layer multi-scale feature extraction module includes a bilinear filter (DBL) and a bidirectional feature pyramid network (BiFPN). The input features corresponding to multiple reference vehicle images and the vehicle images to be checked are input to the bilinear filter (DBL) and the dual attention mechanism module (DAM) of the two-layer multi-scale feature extraction module, and feature processing is performed to obtain the first feature. The output features corresponding to multiple reference vehicle images and the vehicle images to be checked are processed on the bilinear filter (DBL) of the two-layer multi-scale feature extraction module to obtain the second feature. The first feature and the second feature are propagated from top to bottom and from bottom to top in the bidirectional feature pyramid network (BiFPN) of the two-layer multi-scale feature extraction module to perform feature fusion to obtain deep multi-scale features. In addition, a cross orthogonality regularization (COR) module may also be involved, and the cross orthogonality regularization module can reduce collaborative redundancy. For example, the first feature and the second feature are input into the cross-orthogonalization module to extract mutually orthogonal features, thereby enriching the diversity of features and reducing feature redundancy to build a more compact and efficient convolutional neural network (CNN) model.

[0115] The deep multi-scale features are processed by global average pooling (GAP), multi-equal division, and dimension reduction modules in turn, compressing the 1024-dimensional information to 256-dimensional information. The 256-dimensional vehicle features are recorded as local features, which are used to optimize the classification loss of the deep multi-scale feature pyramid branch during the training process. The local features output by the depth-by-depth multi-scale feature pyramid branch are fused to form global-local features.

[0116] See also Figure 2 In one embodiment of the present invention, a method for training a re-identification model may include the following steps:

[0117] Step S220: obtaining a training sample set, wherein the vehicle images in the training sample set are provided with corresponding vehicle labels.

[0118] Step S221: Select different batches of training vehicle images from the training sample set.

[0119] Step S222: input different batches of training vehicle images into the vehicle recognition model to be trained for training, and determine the recognition loss value of the target loss function of the vehicle recognition model to be trained, wherein the vehicle recognition model includes a feature extraction network and a classification network, the feature extraction network is connected to the classification network, and the target loss function includes the feature learning loss corresponding to the global-local features on the feature extraction network, and the total loss of the classification loss corresponding to the global features and local features on the classification network.

[0120] Step S223: Based on the recognition loss value, the parameters of the vehicle recognition model to be trained are adjusted to obtain a re-recognition model.

[0121] In one embodiment of the present invention, the local features corresponding to the multiple reference vehicle images and the vehicle images to be checked can be used to optimize the classification loss of the depth-by-depth multi-scale feature pyramid branch during the training process. The global features corresponding to the multiple reference vehicle images and the vehicle images to be checked can be used to optimize the backbone network during the training process. Classification loss of the global branch. The classification loss is mainly used to supervise the classification task in the re-identification model to ensure that the re-identification model can correctly predict the category of the sample.

[0122] Specifically, the loss function of the classification loss is a crucial part of the image recognition model training process. The loss function defines the gap between the prediction results of the vehicle recognition model to be trained and the true label, which can be used to guide the update of the vehicle recognition model to be trained, so that the vehicle recognition model to be trained gradually learns how to make predictions more accurately, thereby obtaining a re-identification model. Among them, the label is the target value or expected output associated with the vehicle image, for example, the label can be the object category in the picture, etc.

[0123] In one embodiment of the present invention, for global features The global-local features formed by fusing the local features output by the depth-by-depth multi-scale feature pyramid branch can be used to optimize the depth-by-depth multi-scale feature pyramid branch and the backbone network during the training process. Feature learning loss corresponding to the global branch. Feature learning loss is usually used to ensure that the re-ID model can learn effective feature representations that can distinguish different categories or samples.

[0124] During the training phase of the vehicle recognition model to be trained, the global features and normalized features , which are used to optimize the Euclidean distance and cosine distance between vehicle features of different vehicle images, respectively, to avoid the problem of loss function oscillation when multiple loss functions optimize a vehicle feature at the same time. Among them, the Euclidean distance refers to the straight-line distance between two vehicle features in multidimensional space. The Euclidean distance is suitable for occasions where the modulus (length) of the vehicle feature is more important. The cosine distance, also known as the cosine similarity, can be obtained by subtracting the similarity from 1. The cosine distance focuses on the angle between two vehicle features, not their length. The cosine distance is more suitable for occasions where the direction of the vehicle feature is more important.

[0125] Specifically, for the target features corresponding to the reference vehicle image and the features to be checked corresponding to the vehicle image to be checked, the Euclidean distance and cosine distance between the corresponding target features and the features to be checked can be calculated to measure the similarity between the reference vehicle image and the vehicle image to be checked.

[0126] See also Figure 1 , Figure 2 , Figure 3 and Figure 4 In one embodiment of the present invention, in step S222, in the process of training the vehicle recognition model to be trained, local features, global features, The steps of optimizing the cross entropy loss in the vehicle recognition model and optimizing the triplet loss and marginal loss of difficult sample mining in the vehicle recognition model using global-local features may include steps S2220, S2221, S2222, S2223, S2224 and S2225.

[0127] Step S2220 can be expressed as dividing the global-local features into anchor vehicle features according to the sample type , positive sample vehicle features and negative sample vehicle features .

[0128] Specific, anchor vehicle features It is represented as a specific sample selected from a category of the training sample set. During the training process of the vehicle recognition model to be trained, we will try to adjust the parameters so that the anchor vehicle feature Compared with other samples of the same category (positive sample vehicle features ) as small as possible, and the distance between samples of different categories (negative sample vehicle features ) as large as possible.

[0129] For example, the positive sample vehicle features Is the vehicle feature with anchor point Samples belonging to the same category. Ideally, the vehicle recognition model to be trained should learn the characteristics of the anchor vehicle and positive sample vehicle features The distance between them should be as small as possible. In other words, after the vehicle recognition model to be trained is trained, the re-identification model should be able to recognize that they are instances of the same category. Negative sample vehicle features Is different from the anchor point vehicle characteristics One of the goals of the re-identification model is to make the anchor vehicle features and negative sample vehicle features The distance between them is as large as possible to distinguish samples of different categories.

[0130] Step S2221 can be expressed as setting a discriminative feature embedding function , and satisfies .

[0131] Step S2222 can be expressed as the anchor vehicle feature corresponding to the global-local feature , positive sample vehicle features and negative sample vehicle features , calculate the marginal loss of the vehicle recognition model to be trained ,satisfy ,in, Expressed as and The angle between Expressed as and The angle between is expressed as the scale factor of the hypersphere radius, is represented as the slack allowance and , The number of classes represented as anchor vehicles, positive vehicles, and negative vehicles.

[0132] Step S2223 can be expressed as the anchor vehicle feature corresponding to the global-local feature , positive sample vehicle features and negative sample vehicle features , calculate the triplet loss of hard sample mining for image recognition models ,satisfy ,in, Expressed as a distance metric, Represents the desired positive sample vehicle features and anchor vehicle features The distance between the negative sample vehicle features and anchor vehicle features The minimum gap between the distances.

[0133] Step S2224 can be expressed as: , calculate the cross entropy loss of the image recognition model ,satisfy ,in, Including local features and global features , Expressed as The corresponding sample labels are Expressed as predicted probability.

[0134] For global features and local features, and corresponding test labels can be generated according to the fully connected layer (FC layer) and related algorithms. The test label is the target value or expected output associated with a vehicle image. For example, the test label can be the object category in the image, and the sample label is the actual label corresponding to the vehicle image, which is known in advance. By comparing the gap between the predicted label and the true label, it can be used to guide the update of the vehicle recognition model to be trained, so that the vehicle recognition model to be trained gradually learns how to make predictions more accurately, thereby obtaining a re-recognition model.

[0135] Step S2225 can be expressed as: , Triplet loss for difficult sample mining , cross entropy loss Add together to get the total loss function of the vehicle recognition model to be trained , using the total loss function The network model parameters in the vehicle recognition model to be trained are adjusted, where: , Expressed as marginal loss The weight coefficient of .

[0136] See also Figure 5 In one embodiment of the present invention, a dual-attention cross-modal vehicle re-identification system is proposed, which may include an acquisition unit 110, an extraction unit 120, a processing unit 130, and a training unit 140.

[0137] The acquisition unit 110 is used to acquire a to-be-checked vehicle image and a plurality of reference vehicle images.

[0138] The extraction unit 120 is used to input multiple reference vehicle images and the vehicle image to be checked into the feature extraction network in the re-identification model, perform feature extraction in the spatial domain and the channel domain to obtain global features, perform feature extraction in the spatial domain and the channel domain and obtain local features after multi-equal division processing, fuse the corresponding global features and local features to obtain global-local features, form target features with the global features, local features and global-local features corresponding to the reference vehicle image, and form features to be checked with the global features, local features and global-local features corresponding to the vehicle image to be checked.

[0139] The processing unit 130 is used to determine the comprehensive feature similarity between the vehicle image to be checked and each reference vehicle image based on the classification network in the re-identification model according to the target feature and the feature to be checked.

[0140] The matching unit 140 is used to select, based on the classification network in the re-identification model and according to the comprehensive feature similarity, multiple vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet the preset matching conditions from multiple reference vehicle images, record them as a re-identification image set, and sort the multiple vehicle images in the re-identification image set based on the comprehensive feature similarity.

[0141] See also Figure 6 In one embodiment of the present invention, the electronic device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a dual-attention cross-modal vehicle re-identification program.

[0142] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code for dual attention cross-modal vehicle re-identification, etc., but also can be used to temporarily store data that has been output or is to be output.

[0143] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect various components of the entire electronic device 1, and executes or executes programs or modules (such as a power battery capacity correction program, etc.) stored in the memory 12, and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0144] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above dual attention cross-modal vehicle re-identification method.

[0145] Exemplarily, the computer program may be divided into one or more modules, one or more modules are stored in the memory 12, and executed by the processor 13 to complete the present application. One or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition unit 110, an extraction unit 120, a processing unit 130, and a matching unit 140.

[0146] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, which can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the dual-attention cross-modal vehicle re-identification method of each embodiment of the present application.

[0147] In summary, the present invention discloses a dual-attention cross-modal vehicle re-identification method, system, device and medium, wherein a reference vehicle image and a to-be-checked vehicle image are input into a re-identification model, and a feature extraction network of the re-identification model extracts target features corresponding to the reference vehicle image and to-be-checked features corresponding to the to-be-checked vehicle image in the spatial domain and the channel domain. The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each reference vehicle image based on the target features and to-be-checked features, so as to select multiple vehicle images whose comprehensive feature similarity with the to-be-checked vehicle image meets preset matching conditions, thereby obtaining a re-identified image. The present invention has the advantage of high vehicle image re-identification accuracy.

[0148] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A dual attention cross-modal vehicle re-identification method, characterized in that: The method comprises: Acquire a to-be-checked vehicle image and a plurality of reference vehicle images; Inputting a plurality of the reference vehicle images and the vehicle image to be checked into a feature extraction network in a re-identification model, performing feature extraction in the spatial domain and the channel domain to obtain global features, performing feature extraction in the spatial domain and the channel domain and performing multi-equal division processing to obtain local features, fusing the corresponding global features and local features to obtain global-local features, forming target features with the global features, local features, and global-local features corresponding to the reference vehicle image, and forming features to be checked with the global features, local features, and global-local features corresponding to the vehicle image to be checked; The classification network in the re-identification model determines the comprehensive feature similarity between the to-be-checked vehicle image and each of the reference vehicle images according to the target feature and the to-be-checked feature; The classification network in the re-identification model selects, from the plurality of reference vehicle images, a plurality of vehicle images whose feature comprehensive similarities with the vehicle image to be checked meet a preset matching condition according to the feature comprehensive similarity, and records them as a re-identification image set, and sorts the plurality of vehicle images in the re-identification image set based on the feature comprehensive similarity; The step of inputting the plurality of reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model and extracting features in a spatial domain and a channel domain to obtain global features comprises: Input the plurality of reference vehicle images and the vehicle image to be checked into the residual block and the hybrid module of the feature extraction network of the re-identification model, respectively extract vehicle features from the plurality of reference vehicle images and the vehicle image to be checked, and obtain corresponding input features; Through the dual attention mechanism module of the feature extraction network, feature processing is performed on the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked in the spatial domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, and feature processing is performed on the channel domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain; By using the global pooling module of the feature extraction network, a global average pooling process is performed on the output features of the plurality of reference vehicle images and the to-be-checked vehicle images in the spatial domain and the output features in the channel domain to obtain a global feature; The step of performing feature processing on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked in the spatial domain to obtain output features of the plurality of reference vehicle images and the vehicle image to be checked in the spatial domain comprises: The convolutional neural network is sequentially performed on the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked. deal with, The mapping relationship is processed to obtain the corresponding feature weights, wherein the The mapping relationship is used to map the feature vector into a probability distribution; Multiplying the input feature by the feature weight to obtain a global context feature; The global context features are processed by bottleneck transformation to obtain output features of the plurality of reference vehicle images and the vehicle images to be checked in the spatial domain. ; Output Features satisfy: ； Expressed as feature weight, , , Represented as a convolutional neural network processing, Represents bottleneck conversion processing, Represented as layer normalization processing, Represented as a nonlinear activation function, are the input features corresponding to multiple reference vehicle images and the vehicle image to be checked, For each point in the input feature, the feature map corresponding to the input feature has Points, for , Represented as the height and width of the feature map corresponding to the input feature; The step of performing feature processing on the channel domain for the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked to obtain output features of the plurality of reference vehicle images and the vehicle image to be checked on the channel domain comprises: The input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked are dimensionally transformed to obtain a first feature map, and the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked are dimensionally transformed and transposed in sequence to obtain a second feature map; Multiply the first feature map by the second feature map, and perform Mapping relationship processing to obtain channel attention map ; Multiply the channel attention map with the first feature map, and multiply the multiplication result by the scale coefficient Then, a dimension conversion process is performed to obtain a channel weighting matrix, the shape of which is the same as the shape of the input feature; The channel weighting matrix is ​​added to the input feature to obtain output features of the reference vehicle image and the vehicle image to be checked in the channel domain. ; satisfy: ， ； is the channel attention map The elements in Expressed as Channel pair The impact of the channels, Represented as output feature The elements in , the scale factor Initialized to 0, and gradually assigned larger weights during training, is the feature matrix of the input features, is the transposed matrix of the input features, is the number of channels corresponding to the input feature; Expressed as a natural constant.

2. The dual attention cross-modal vehicle re-identification method according to claim 1, characterized in that: The step of acquiring the image of the vehicle to be checked and a plurality of reference vehicle images comprises: Through different visible light surveillance cameras, images of vehicles to be inspected and multiple reference vehicle images are collected from different perspectives and scenes; The image of the vehicle to be checked and the plurality of the reference vehicle images are sequentially subjected to size adjustment, grayscale conversion, denoising, and image graying processing, so that the image of the vehicle to be checked and the plurality of the reference vehicle images have consistent image format and image quality.

3. The dual attention cross-modal vehicle re-identification method according to claim 1, characterized in that: The step of inputting the plurality of reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model, extracting features in a spatial domain and a channel domain, and performing multi-equal division processing to obtain local features comprises: Inputting the input features corresponding to the plurality of the reference vehicle images and the vehicle image to be checked into the bilinear filter and the dual attention mechanism module of the two-layer multi-scale feature extraction module of the feature extraction network, performing feature processing to obtain a first feature, and performing feature processing on the output features corresponding to the plurality of the reference vehicle images and the vehicle image to be checked on the bilinear filter of the two-layer multi-scale feature extraction module to obtain a second feature; The first feature and the second feature are propagated from top to bottom and from bottom to top in the bidirectional feature pyramid network of the two-layer multi-scale feature extraction module to perform feature fusion to obtain deep multi-scale features; The deep multi-scale features are subjected to average pooling processing, multi-equal division processing, and dimensionality reduction processing in sequence to obtain local features.

4. The dual attention cross-modal vehicle re-identification method according to claim 1, characterized in that: The re-identification model is trained by the following method: Obtaining a training sample set, wherein vehicle images in the training sample set are provided with corresponding vehicle labels; Selecting different batches of training vehicle images from the training sample set; Inputting different batches of the training vehicle images into a vehicle recognition model to be trained for training, and determining a recognition loss value of a target loss function of the vehicle recognition model to be trained, wherein the vehicle recognition model includes a feature extraction network and a classification network, the feature extraction network is connected to the classification network, and the target loss function includes a feature learning loss corresponding to the global-local features on the feature extraction network, and a total loss of classification losses corresponding to the global features and the local features on the classification network; Based on the recognition loss value, the parameters of the vehicle recognition model to be trained are adjusted to obtain a re-recognition model.

5. A dual-attention cross-modal vehicle re-identification system, characterized in that: The system comprises: An acquisition unit, used for acquiring a to-be-checked vehicle image and a plurality of reference vehicle images; an extraction unit, configured to input the plurality of the reference vehicle images and the to-be-checked vehicle images into a feature extraction network in a re-identification model, perform feature extraction in a spatial domain and a channel domain to obtain global features, perform feature extraction in a spatial domain and a channel domain and perform multi-equal division processing to obtain local features, fuse the corresponding global features and local features to obtain global-local features, form target features with the global features, local features, and global-local features corresponding to the reference vehicle images, and form to-be-checked features with the global features, local features, and global-local features corresponding to the to-be-checked vehicle images; a processing unit, configured to determine, based on the classification network in the re-identification model, a comprehensive feature similarity between the to-be-checked vehicle image and each of the reference vehicle images according to the target feature and the to-be-checked feature; and a matching unit, configured to select, based on the classification network in the re-identification model and according to the comprehensive feature similarity, a plurality of vehicle images whose comprehensive feature similarities with the vehicle image to be checked meet a preset matching condition from the plurality of reference vehicle images, record them as a re-identification image set, and sort the plurality of vehicle images in the re-identification image set based on the comprehensive feature similarity; The extraction unit is further used to input the plurality of reference vehicle images and the vehicle image to be checked into the residual block and the hybrid module of the feature extraction network of the re-identification model, respectively extract vehicle features from the plurality of reference vehicle images and the vehicle image to be checked, and obtain corresponding input features; Through the dual attention mechanism module of the feature extraction network, feature processing is performed on the input features corresponding to the multiple reference vehicle images and the vehicle images to be checked in the spatial domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the spatial domain, and feature processing is performed on the channel domain to obtain output features of the multiple reference vehicle images and the vehicle images to be checked in the channel domain; By using the global pooling module of the feature extraction network, a global average pooling process is performed on the output features of the plurality of reference vehicle images and the to-be-checked vehicle images in the spatial domain and the output features in the channel domain to obtain a global feature; The extraction unit is further used to sequentially perform convolutional neural network on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked. deal with, The mapping relationship is processed to obtain the corresponding feature weights, wherein the The mapping relationship is used to map the feature vector into a probability distribution; Multiplying the input feature by the feature weight to obtain a global context feature; The global context features are processed by bottleneck transformation to obtain output features of the plurality of reference vehicle images and the vehicle images to be checked in the spatial domain. ; Output characteristics satisfy: ； Expressed as feature weight, , , Represented as a convolutional neural network processing, Represents bottleneck conversion processing, Represented as layer normalization processing, Represented as a nonlinear activation function, are the input features corresponding to multiple reference vehicle images and the vehicle image to be checked, For each point in the input feature, the feature map corresponding to the input feature has Points, for , Represented as the height and width of the feature map corresponding to the input feature; The extraction unit is further used to perform dimension conversion on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked to obtain a first feature map, and to perform dimension conversion and transposition processing on the input features corresponding to the plurality of reference vehicle images and the vehicle image to be checked in sequence to obtain a second feature map; Multiply the first feature map by the second feature map, and perform Mapping relationship processing to obtain channel attention map ; Multiply the channel attention map with the first feature map, and multiply the multiplication result by the scale coefficient Then, a dimension conversion process is performed to obtain a channel weighting matrix, the shape of which is the same as the shape of the input feature; The channel weighting matrix is ​​added to the input feature to obtain output features of the reference vehicle image and the vehicle image to be checked in the channel domain. ; satisfy: ， ； is the channel attention map The elements in Expressed as Channel pair The impact of the channels, Represented as output feature The elements in , the scale factor Initialized to 0, and gradually assigned larger weights during training, is the feature matrix of the input features, is the transposed matrix of the input features, is the number of channels corresponding to the input feature; Expressed as a natural constant.

6. An electronic device, characterized in that: It includes a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the dual-attention cross-modal vehicle re-identification method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to enable a computer to execute the dual-attention cross-modal vehicle re-identification method as described in any one of claims 1-4.

Images