Image-based vehicle re-identification method and device, server and storage medium

Abstract

Embodiments of the present application disclose a kind of vehicle re-identification method, device, server and storage medium based on image, vehicle image feature extracted by neural network is finally converted into one-dimensional multi-channel digitization expression feature, and one-dimensional multi-channel digitization expression feature is matched with the one-dimensional multi-channel digitization expression feature of the mass different car image obtained in advance, and the same vehicle is determined according to the approximate degree of matching.No need to compare with each vehicle image using neural network model separately classification, greatly reduce the computing power of vehicle re-identification.Through multi-scale serialization fusion, fully fuse the features in multiple feature extraction convolution in the extraction process, greatly improve the accuracy of feature extraction.And through position serialization transformation, accurately extract the local feature of vehicle to integrate and model global feature, effectively solve the problem of similar local features of different vehicle images, further improve the accuracy of vehicle re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a vehicle re-identification method and device based on images, a server and a storage medium. BACKGROUND

[0002] With the rapid development of China's economy, people's living standards are improving, urban modernization is gradually improving, the automobile manufacturing industry is developing rapidly, and the number of vehicles is also increasing. Car travel has become a common phenomenon, which brings many challenges to traffic control and traffic safety. In a complex road traffic environment, it is more difficult to crack down on illegal behavior such as vehicle fraud, illegal driving, vehicle theft, traffic accidents and hit-and-run. In order to ensure road safety, road monitoring systems are constantly improving, and road monitoring cameras are becoming more and more dense, covering key road sections. These cameras capture real-time video image information of road vehicles and record them, providing data for relevant departments to track illegal vehicles and stolen vehicles.

[0003] However, due to the complexity of road traffic, the large number of vehicles and the large number of non-overlapping cameras, it is difficult to search for target vehicles in a large number of video image libraries by manpower, which is time-consuming and laborious. Therefore, the current vehicle re-identification technology based on neural network image is generally used to replace manual search, which can quickly and accurately search, locate and track target vehicles in a large number of video image libraries.

[0004] However, the existing image recognition neural network is only good at identifying and classifying vehicles based on images. In the face of a large number of vehicles and image data, relying on image recognition neural networks from a large amount of data to separately re-identify a specific vehicle will require strong computing power to achieve, making it more and more difficult to identify a specific vehicle. If a single vehicle feature is used to implement vehicle re-identification using an image recognition neural network, the accuracy of re-identification will be reduced due to the use of only a single vehicle feature. SUMMARY

[0005] The embodiments of the present application provide a vehicle re-identification method and device based on images, a server and a storage medium to solve the technical problems of consuming strong computing power and low accuracy in re-identifying vehicles based on images in the prior art.

[0006] In a first aspect, the embodiments of the present application provide a vehicle re-identification method based on images, comprising:

[0007] preprocessing the collected vehicle images to obtain a to-be-identified image;

[0008] The image to be identified is subjected to accelerated convergence down-sampling convolution to obtain an initial down-sampling multi-channel feature map; the initial down-sampling multi-channel feature map is subjected to feature extraction convolution processing, and the image subjected to feature extraction convolution is subjected to position serialization transformation, and the image features subjected to position serialization transformation are encoded to obtain an initial label sequence;

[0009] The initial label sequence is converted into a converted initial multi-channel feature map, and the converted initial multi-channel feature map is subjected to accelerated convergence down-sampling convolution and feature extraction convolution processing, the image obtained after processing is subjected to position serialization transformation, and the image features subjected to position serialization transformation are encoded to obtain a generated label sequence;

[0010] The initial label sequence and the generated label sequence are subjected to multi-scale sequence fusion to obtain a multi-scale label sequence;

[0011] The multi-scale label sequence is converted into a multi-scale multi-channel feature map, and the multi-scale multi-channel feature map is subjected to accelerated convergence down-sampling convolution and feature extraction convolution processing, and whether the number of feature channels after processing the multi-scale multi-channel feature map meets a preset standard feature channel number is judged;

[0012] After meeting the preset standard feature channel number, a one-dimensional channel feature corresponding to the feature channel is extracted, similarity comparison is performed between the channel feature and a vehicle image channel feature stored in advance, and vehicle information of a vehicle image with the largest similarity is taken as a recognition result;

[0013] Otherwise, the multi-scale multi-channel feature map is taken as an image to be identified, and the image to be identified is subjected to accelerated convergence down-sampling convolution to obtain an initial down-sampling multi-channel feature map.

[0014] In a second aspect, an embodiment of the present application further provides a vehicle re-identification device based on an image, comprising:

[0015] A preprocessing module is configured to preprocess a collected vehicle image to obtain an image to be identified;

[0016] A first encoding module is configured to subject the image to be identified to accelerated convergence down-sampling convolution to obtain an initial down-sampling multi-channel feature map; subject the initial down-sampling multi-channel feature map to feature extraction convolution processing, and subject the image subjected to feature extraction convolution to position serialization transformation, and encode the image features subjected to position serialization transformation to obtain an initial label sequence;

[0017] a second encoding module, configured to convert the initial mark sequence into a converted initial multi-channel feature map, and perform accelerated convergence down-sampling convolution and feature extraction convolution processing on the converted initial multi-channel feature map, perform position serialization transformation on the image obtained after processing, and encode the image features after the position serialization transformation to obtain a generated mark sequence;

[0018] a fusion module, configured to perform multi-scale serialization fusion on the initial mark sequence and the generated mark sequence to obtain a multi-scale mark sequence;

[0019] a conversion module, configured to convert the multi-scale mark sequence into a multi-scale multi-channel feature map, and perform accelerated convergence down-sampling convolution and feature extraction convolution processing on the multi-scale multi-channel feature map, and determine whether a feature channel number of the multi-scale multi-channel feature map after processing meets a preset standard feature channel number;

[0020] a comparison module, configured to, after the multi-scale multi-channel feature map meets the preset standard feature channel number, extract one-dimensional channel features corresponding to the feature channels, compare the channel features with vehicle image channel features stored in advance according to similarity, and take vehicle information of a vehicle image with the largest similarity as a recognition result;

[0021] a return module, configured to, otherwise, take the multi-scale multi-channel feature map as a to-be-recognized image, and return to perform accelerated convergence down-sampling convolution on the to-be-recognized image to obtain an initial down-sampling multi-channel feature map.

[0022] In a third aspect, an embodiment of the present application further provides a server, comprising:

[0023] one or more processors;

[0024] a storage device configured to store one or more programs,

[0025] when the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle re-identification method based on images provided by the above embodiment.

[0026] In a fourth aspect, an embodiment of the present application further provides a storage medium containing computer executable instructions, which are used to execute the vehicle re-identification method based on images provided by the above embodiment when executed by a computer processor.

[0027] The embodiment of the present application provides a vehicle re-identification method and device based on images, a server and a storage medium. The vehicle image collected is preprocessed to obtain a to-be-identified image; the to-be-identified image is subjected to accelerated convergence down-sampling convolution to obtain an initial down-sampling multi-channel feature map; the initial down-sampling multi-channel feature map is subjected to feature extraction convolution processing, and the image after the feature extraction convolution is subjected to position serialization transformation, and the image features after the position serialization transformation are encoded to obtain an initial mark sequence; the initial mark sequence is converted into a converted initial multi-channel feature map, and the converted initial multi-channel feature map is subjected to accelerated convergence down-sampling convolution and feature extraction convolution processing, the image obtained after the processing is subjected to position serialization transformation, and the image features after the position serialization transformation are encoded to obtain a generated mark sequence; the initial mark sequence and the generated mark sequence are subjected to multi-scale serialization fusion to obtain a multi-scale mark sequence; the multi-scale mark sequence is converted into a multi-scale multi-channel feature map, and the multi-scale multi-channel feature map is subjected to accelerated convergence down-sampling convolution and feature extraction convolution processing, and whether the feature channel number of the multi-scale multi-channel feature map after the processing meets a preset standard feature channel number is judged; after meeting the preset standard feature channel number, one-dimensional channel features corresponding to the feature channel are extracted, similarity comparison is performed on the channel features and pre-stored vehicle image channel features, vehicle information of a vehicle image with the largest similarity is taken as an identification result; otherwise, the multi-scale multi-channel feature map is taken as the to-be-identified image, and the to-be-identified image is subjected to accelerated convergence down-sampling convolution to obtain an initial down-sampling multi-channel feature map. The vehicle image features extracted by the neural network are finally converted into one-dimensional multi-channel digital expression features, and the one-dimensional multi-channel digital expression features are matched with one-dimensional multi-channel digital expression features of a large number of different vehicle images obtained in advance, and the same vehicle is determined according to the approximate degree of the matching. Without using the neural network model to classify and compare each vehicle image, the computing power of vehicle re-identification is greatly reduced. Meanwhile, in the vehicle image feature extraction process, the features in the multiple feature extraction convolutions are fully fused through multi-scale serialization fusion, so that the accuracy of feature extraction is greatly improved. And through the position serialization transformation, the local features of the vehicle are accurately extracted for integrated global feature modeling, effectively solving the problem of similar local features of different vehicle images. The accuracy of vehicle re-identification is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0028] Other features, objects, and advantages of the present application will become more apparent from the following detailed description when read in conjunction with the accompanying drawings:

[0029] Figure 1 is a flowchart of a vehicle re-identification method based on images provided by the embodiment one of the present application;

[0030] Figure 2 is a structural schematic diagram of a Conv_Block_a convolution module in the image-based vehicle re-identification method provided by Embodiment One of the present application;

[0031] Figure 3 is a structural schematic diagram of a Conv_Block_b convolution module in the image-based vehicle re-identification method provided by Embodiment One of the present application;

[0032] Figure 4 is a flow schematic diagram of the image-based vehicle re-identification method provided by Embodiment Two of the present application;

[0033] Figure 5 is a position sequence transformation schematic diagram in the image-based vehicle re-identification method provided by Embodiment Two of the present application;

[0034] Figure 6 is a flow schematic diagram of the image-based vehicle re-identification method provided by Embodiment Three of the present application;

[0035] Figure 7 is a structural schematic diagram of the image-based vehicle re-identification device provided by Embodiment Four of the present application;

[0036] Figure 8 is a structural diagram of the server provided by Embodiment Five of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0038] Embodiment One

[0039] Figure 1 is a flowchart of the image-based vehicle re-identification method provided by Embodiment One of the present application. This embodiment can be applied to the case of a large number of different vehicle image targets, and can quickly realize the re-identification of vehicles by using the collected images. The image-based vehicle re-identification device can be used to realize the method, and the method can be integrated into a vehicle image server. The method specifically includes the following steps:

[0040] Step 110, pre-processing the collected vehicle images to obtain a to-be-identified image.

[0041] In the embodiment, the collected vehicle image can be an image containing a vehicle collected by various cameras on the road. After the vehicle image is obtained, the vehicle image can be preprocessed to facilitate subsequent vehicle feature extraction using a neural network for comparison and re-identification. For example, it can include cropping and image enhancement processing. In the embodiment, the cropping can be set to an image with a resolution of 256*256. After cropping, the cropped image can be subjected to data enhancement.

[0042] In step 120, the image to be identified is subjected to accelerated convergence down-sampling convolution to obtain an initial down-sampled multi-channel feature map; the initial down-sampled multi-channel feature map is subjected to feature extraction convolution processing, and the image subjected to feature extraction convolution is subjected to position serialization transformation, and the image feature subjected to position serialization transformation is encoded to obtain an initial mark sequence.

[0043] For example, a 256*256 3-channel vehicle image can be first converted into a 128*128 12-channel feature map through accelerated convergence down-sampling convolution operation. In the embodiment, the image can be input into the Conv_Block_a convolution module to realize accelerated convergence down-sampling. Figure 2 is a structural diagram of the Conv_Block_a convolution module in the vehicle re-identification method based on images provided by the first embodiment of the application, referring to Figure 2 The Conv_Block_a convolution module includes three main branches, wherein the first branch is composed of a convolution layer and a batch normalization layer, the convolution layer has a convolution kernel size of 5*5 and a step distance of 2; the second branch is also composed of a convolution layer and a batch normalization layer, the convolution layer of the second branch has a convolution kernel size of 3x3 and a step distance of 2; and the third branch is composed of a convolution layer with a step distance of 2 and a batch normalization layer, the convolution layer has a convolution kernel size of 1x1 and a step distance of 2.

[0044] Then, the 128*128 12-channel feature map is subjected to feature extraction convolution processing. For example, multiple branches can be used for convolution feature extraction to fully extract the features of the image. Optionally, the image subjected to accelerated convergence down-sampling can be input into the Conv_Block_b module. Figure 3 is a structural diagram of the Conv_Block_b convolution module in the vehicle re-identification method based on images provided by the first embodiment of the application, referring to Figure 2The Conv_Block_b module includes three main branches, wherein the first branch is composed of a convolution layer and a batch normalization layer, the convolution layer has a convolution kernel size of 5*5 and a stride of 1; the second branch is also composed of a convolution layer and a batch normalization layer, the convolution layer of the second branch has a convolution kernel size of 3x3 and a stride of 1; and the third branch is composed of a batch normalization layer. After the feature matrix is operated through the three branches, the feature matrix is added and then activated through an activation function. The Conv_Block_b module is used to extract image features. Further, the Conv_Block_b convolution module can be used multiple times to further extract features. In this embodiment, a 128*128 12-channel feature map can be processed through four Conv_Block_b modules to obtain a feature image.

[0045] The feature image processed through the feature extraction convolution is subjected to a serialization transformation. The serialization transformation can be to divide the feature image into a plurality of sub-image blocks, and then divide the image sub-image blocks again to obtain feature sub-blocks. The feature sub-blocks are adjusted according to a certain rule in each sub-image block according to the corresponding serial number. This is because, in the image convolution process, too much attention is paid to the features of a certain component in the vehicle image, and the correlation features between components are ignored. Through position serialization transformation and encoding of the image features after position serialization transformation using a Transformer encoder, a multi-head attention mechanism is provided in the Transformer encoder. The attention mechanism can strengthen the feature relationship between correlated components, and thus accurately extract local features of the vehicle and integrate the local features to model the global features. The local features of the vehicle, such as the vehicle lights, windows, windshield, and wheels, are integrated to obtain the global features of the vehicle, and a feature model of the entire vehicle is established, effectively solving the problem of similar local features of different vehicle images.

[0046] The serialization features token_1 of this stage are obtained through the transformer encoder, i.e., an initial token sequence.

[0047] In step 130, the initial token sequence is converted into a converted initial multi-channel feature map. The converted initial multi-channel feature map is subjected to accelerated convergence down-sampling convolution and feature extraction convolution. The image obtained after processing is subjected to position serialization transformation, and the image features after position serialization transformation are encoded to obtain a generated token sequence.

[0048] The initial marker sequence generated in the above steps can be converted into a corresponding feature image. This feature image is then subjected to accelerated convergence downsampling convolution and feature extraction convolution to reduce its size and increase its channels. Based on this, feature extraction is performed again. A positional serialization transformation is then applied, and the image features after the positional serialization transformation are encoded using a Transformer encoder. This yields the serialized feature token_2 for this stage, which is the generated marker sequence. For example, a 128*128 12-channel feature map can be converted into a 64*64 64-channel feature map through one Conv_Block_a module, and then through six Conv_Block_b modules. The 128*128 12-channel feature map is then subjected to a relative positional serialization transformation and two Transformer encoders to obtain the serialized feature token_2 for the Conv_2 stage.

[0049] Step 140: Perform multi-scale serialization fusion on the initial marker sequence and the generated marker sequence to obtain a multi-scale marker sequence.

[0050] token_1 and token_2 are cross-fused to obtain token_3, which is then passed through a Transformer encoder once and converted into a 64*64 64-channel feature map.

[0051] Step 150: Convert the multi-scale labeled sequence into a multi-scale multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the multi-scale channel feature map, and determine whether the number of feature channels after the processing of the multi-scale channel feature map meets the preset standard number of feature channels.

[0052] The method described above can be used to inversely encode the multi-scale marker sequence token_3 using a transformer encoder, converting it into a corresponding multi-scale, multi-channel feature map. Then, accelerated convergence downsampling convolution and feature extraction convolution are performed on the multi-scale channel feature map. With each accelerated convergence downsampling convolution, the number of feature channels can be continuously increased. In this embodiment, since the feature channels are numerically represented, the number of feature channels can be approximated as the number of types of digital features. When performing approximate comparisons, too many feature channels will increase computational power significantly and have limited effect on improving recognition accuracy; conversely, if the number of feature channels is too small, although the computational load will be reduced, the recognition accuracy will be lowered. Therefore, in this embodiment, the actual number of image feature channels to be compared can be determined based on the actual situation of the vehicle image. By comparing it with a preset standard number of feature channels, it can be determined whether accelerated convergence downsampling convolution and feature extraction convolution processing is still needed.

[0053] Step 160: After meeting the preset standard number of feature channels, extract the one-dimensional channel features corresponding to the feature channels, compare the similarity between the channel features and the pre-stored vehicle image channel features, and take the vehicle information of the vehicle image with the highest similarity as the recognition result.

[0054] When the preset standard number of feature channels is met, the extracted channel features are consistent with the number of feature channels of the previously extracted vehicle images. One-dimensional channel features corresponding to the feature channels can be extracted. For example, the one-dimensional channel features can be converted into corresponding digital sequences, and the digital sequences can be compared with the digital sequences of one-dimensional channel features of all previously stored vehicle images. The most similar digital sequence is selected as the comparison result, and the vehicle information of the image corresponding to the digital sequence is used as the recognition result.

[0055] Step 170: Otherwise, take the multi-scale channel feature map as the image to be identified and return to the step of performing accelerated convergent downsampling convolution on the image to be identified to obtain the initial downsampled multi-channel feature map.

[0056] If the number of feature channels in the processed multi-scale channel feature map is still less than the preset standard number of feature channels, the number of corresponding digital sequences will also be inconsistent due to the inconsistent number of channels, making feature comparison impossible. Therefore, in this embodiment, the multi-scale channel feature map obtained in the above steps is subjected to accelerated convergence downsampling convolution and feature extraction convolution again, and the above fusion steps are repeated until the image feature dimension of the obtained multi-scale channel feature map is consistent with the preset standard number of feature channels, thus meeting the comparison requirements. If the image feature dimension of the obtained multi-scale channel feature map still does not meet the requirements, the above steps are repeated, that is, the latest obtained multi-scale channel feature map is subjected to accelerated convergence downsampling convolution and feature extraction convolution, and fused with the feature map corresponding to the previously obtained multi-scale labeled sequence, and then encoded again for accelerated convergence downsampling convolution and feature extraction convolution until the feature channel number requirement is met.

[0057] In this embodiment, when it is determined that the requirements are not met, token_3 is then passed through a Transformer encoder once and converted into a 64*64 64-channel feature map. The 64*64 64-channel feature map is then converted into a 32*32 256-channel feature map using the Conv_Block_a module, and then passed through the Conv_Block_b module 8 times. After relative position serialization transformation, it is passed through the Transformer encoder twice to obtain the serialized feature token_4 of the Conv_2 stage. token_3 and token_4 are cross-fused through the Multiscale_Cross module to obtain token_5, which is then passed through a Transformer encoder once and converted into a 32*32 256-channel feature map. The 32*32 256-channel feature map is converted into a 1024-channel feature map of size 16*16 by the Conv_Block_a module, then converted into a 2048-channel feature map of size 8*8 by four Conv_Block_b modules and one more Conv_Block_a module. This meets the standard for the number of channels.

[0058] This invention preprocesses acquired vehicle images to obtain images to be identified; it then performs accelerated convergent downsampling convolution on the images to be identified to obtain an initial downsampling multi-channel feature map; it further performs feature extraction convolution on the initial downsampling multi-channel feature map, and then performs positional serialization transformation on the image after feature extraction convolution, encoding the features of the image after positional serialization transformation to obtain an initial label sequence; it then converts the initial label sequence into a transformed initial multi-channel feature map, and performs accelerated convergent downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map, performing positional serialization transformation on the processed image, and encoding the features of the image after positional serialization transformation to obtain a generated label sequence; finally, it combines the initial label sequence and the generated label sequence. The sequence is fused using multi-scale serialization to obtain a multi-scale labeled sequence. This multi-scale labeled sequence is then converted into a multi-scale multi-channel feature map. The multi-scale channel feature map is then subjected to accelerated convergence downsampling convolution and feature extraction convolution. The number of feature channels in the processed multi-scale channel feature map is checked against a preset standard number of feature channels. If the preset standard number of feature channels is met, one-dimensional channel features corresponding to each feature channel are extracted. These channel features are then compared with pre-stored vehicle image channel features for similarity. The vehicle information of the image with the highest similarity is used as the recognition result. Otherwise, the multi-scale channel feature map is used as the image to be recognized, and the process returns to performing accelerated convergence downsampling convolution on the image to be recognized to obtain an initial downsampled multi-channel feature map. By converting the vehicle image features extracted through the neural network into one-dimensional multi-channel digital representation features, and matching these one-dimensional multi-channel digital representation features with a large number of pre-obtained one-dimensional multi-channel digital representation features of different car images, the similarity of the matching is used to determine if the vehicles are the same. This eliminates the need for separate classification and comparison with each vehicle image using a neural network model, significantly reducing the computational cost of vehicle re-identification. Meanwhile, during vehicle image feature extraction, multi-scale sequential fusion is employed to fully integrate features from multiple feature extraction convolutions, significantly improving the accuracy of feature extraction. Furthermore, positional sequential transformation accurately extracts local vehicle features for unified global feature modeling, effectively addressing the issue of similar local features across different vehicle images. This further enhances the accuracy of vehicle re-identification.

[0059] Example 2

[0060] Figure 4This is a flowchart illustrating the image-based vehicle re-identification method provided in Embodiment 2 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, the position serialization transformation is specifically optimized as follows: the image is uniformly divided into a preset even number of sub-image blocks of the same size; each sub-image block is uniformly divided into a preset even number of feature image blocks; the feature image blocks within each sub-image block are numbered in order from left to right and from top to bottom; and the feature image blocks with the same number in each feature block are combined to complete the position serialization transformation.

[0061] Accordingly, the image-based vehicle re-identification method provided in this embodiment specifically includes:

[0062] Step 210: Preprocess the acquired vehicle images to obtain the images to be identified.

[0063] Step 220: Convert the initial labeled sequence into a transformed initial multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map.

[0064] Step 230: Divide the processed image into a preset even number of sub-image blocks of the same size; divide each sub-image block into a preset even number of feature image blocks.

[0065] When performing feature extraction, using a large convolution kernel results in fewer extracted features, making recognition difficult. Conversely, using a small convolution kernel, while capable of extracting features from some regions of the image, neglects image features of related parts of the vehicle, especially after multiple convolution processes, thus reducing the accuracy of vehicle recognition. Therefore, in this embodiment, intervention is used to ensure that while fully extracting local image features, some features from adjacent local images are also considered, thereby focusing on the features of related vehicle parts.

[0066] Figure 5 This is a schematic diagram of the position serialization transformation in the image-based vehicle re-identification method provided in Embodiment 2 of the present invention. See [link / reference]. Figure 5 The processed image can be segmented. In this embodiment, it is segmented into 16 image blocks of the same size.

[0067] Step 240: Number the feature image blocks within each sub-image block in order from left to right and from top to bottom, and combine the feature image blocks with the same number in each feature block to complete the positional serialization transformation.

[0068] Number the sub-image blocks of each image block in a left-to-right, top-to-bottom order, and then swap the sub-image blocks with the same number in each image block with the sub-image blocks with the same number in their adjacent sub-image blocks. This completes the positional serialization transformation.

[0069] Step 250: Encode the image features after positional serialization transformation to obtain the initial label sequence.

[0070] Step 260: Convert the initial label sequence into a transformed initial multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map. Perform position serialization transformation on the processed image, and encode the image features after position serialization transformation to obtain the generated label sequence.

[0071] Step 270: Perform multi-scale serialization fusion on the initial marker sequence and the generated marker sequence to obtain a multi-scale marker sequence.

[0072] Step 280: Convert the multi-scale labeled sequence into a multi-scale multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the multi-scale channel feature map, and determine whether the number of feature channels after the processing of the multi-scale channel feature map meets the preset standard feature channels.

[0073] Step 290: After meeting the preset standard number of feature channels, extract the one-dimensional channel features corresponding to the feature channels, compare the similarity between the channel features and the pre-stored vehicle image channel features, and take the vehicle information of the vehicle image with the highest similarity as the recognition result.

[0074] Step 2100: Otherwise, take the multi-scale channel feature map as the image to be identified, and return to perform accelerated convergent downsampling convolution on the image to be identified to obtain the initial downsampled multi-channel feature map.

[0075] This embodiment optimizes the positional serialization transformation by: uniformly dividing the image into a predetermined even number of identical sub-image blocks; uniformly dividing each sub-image block into a predetermined even number of feature image blocks; numbering the feature image blocks within each sub-image block in a left-to-right, top-to-bottom order; and combining feature image blocks with the same number within each feature block to complete the positional serialization transformation. This approach facilitates the extraction of image features using a smaller convolutional kernel while also considering other features of adjacent parts of the vehicle, further improving the accuracy of vehicle recognition.

[0076] In a preferred embodiment of this example, the encoding of the image features after positional serialization transformation is specifically optimized as follows: the processed image is segmented to generate a preset number of image blocks of the same size, and the relative position of each image block is recorded; the relative positions are flattened into corresponding serialized features, and the serialized features are input into the trained transformer encoder to obtain sequence-encoded features. The transformer encoder has an attention mechanism that can preserve key image features during the encoding process, forming a corresponding digitized sequence.

[0077] Example 3

[0078] Figure 6 This is a flowchart illustrating the image-based vehicle re-identification method provided in Embodiment 3 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, the initial label sequence and the generated label sequence are fused using multi-scale serialization to obtain a multi-scale label sequence. Specifically, the optimization involves: matching the codes in the initial label sequence and the generated label sequence according to their sequence numbers; performing digital fusion on the successfully matched codes; and adding a corresponding sequence number to the fused result to obtain the multi-scale label sequence.

[0079] Accordingly, the image-based vehicle re-identification method provided in this embodiment specifically includes:

[0080] Step 310: Preprocess the acquired vehicle images to obtain the images to be identified.

[0081] Step 320: Perform accelerated convergence downsampling convolution on the image to be identified to obtain an initial downsampling multi-channel feature map; perform feature extraction convolution on the initial downsampling multi-channel feature map, perform position serialization transformation on the processed image, and encode the image features after position serialization transformation to obtain an initial label sequence.

[0082] Step 330: Convert the initial label sequence into a transformed initial multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map. Perform position serialization transformation on the processed image, and encode the image features after position serialization transformation to obtain the generated label sequence.

[0083] Step 340: Match the elements in the initial label sequence and the generated label sequence according to their serial numbers, perform digital fusion on the successfully matched elements, and add the corresponding serial number to the fused result to obtain the multi-scale label sequence.

[0084] Optionally, each element corresponding to the initial label sequence can be matched according to its index, and the elements with corresponding indices can be added together. For example, the element matrix can be added together to achieve digital fusion. The fused element matrix inherits the original indices, and the elements with corresponding indices are fused together to form a multi-scale label sequence according to their indices.

[0085] In this embodiment, successfully matched elements can be converted into vectors in a multi-dimensional vector space according to their corresponding positions in the matrix. The vectors generated from the initial and generated label sequences are then combined to achieve fusion. This method allows label sequences of different sizes to be fused across multiple dimensions using multi-dimensional space, thus better reflecting vehicle image features.

[0086] Step 350: Convert the multi-scale labeled sequence into a multi-scale multi-channel feature map, and perform accelerated convergence downsampling convolution and feature extraction convolution on the multi-scale channel feature map, and determine whether the number of feature channels after the processing of the multi-scale channel feature map meets the preset standard number of feature channels.

[0087] Step 360: After meeting the preset standard number of feature channels, extract the one-dimensional channel features corresponding to the feature channels, compare the similarity between the channel features and the pre-stored vehicle image channel features, and take the vehicle information of the vehicle image with the highest similarity as the recognition result.

[0088] Step 370: Otherwise, take the multi-scale channel feature map as the image to be identified, and return to perform accelerated convergent downsampling convolution on the image to be identified to obtain the initial downsampled multi-channel feature map.

[0089] This embodiment obtains a multi-scale labeled sequence by multi-scale serialization fusion of the initial labeled sequence and the generated labeled sequence. Specifically, the optimization involves: matching the codes in the initial labeled sequence and the generated labeled sequence according to their sequence numbers; digitally fusing the successfully matched codes; and adding a corresponding sequence number to the fused result to obtain the multi-scale labeled sequence. This allows for the digital fusion of multi-channel feature maps extracted at different stages, preserving the image features extracted at each stage. This facilitates vehicle identification.

[0090] Example 4

[0091] Figure 7 This is a schematic diagram of the image-based vehicle re-identification device provided in Embodiment 5 of the present invention, as shown below. Figure 7 As shown, the device includes:

[0092] The preprocessing module 410 is used to preprocess the acquired vehicle images to obtain the images to be recognized;

[0093] The first encoding module 420 is used to perform accelerated convergence downsampling convolution on the image to be identified to obtain an initial downsampling multi-channel feature map; to perform feature extraction convolution on the initial downsampling multi-channel feature map, and to perform position serialization transformation on the processed image, and to encode the image features after position serialization transformation to obtain an initial label sequence.

[0094] The second encoding module 430 is used to convert the initial label sequence into a transformed initial multi-channel feature map, and to perform accelerated convergence downsampling convolution and feature extraction convolution processing on the transformed initial multi-channel feature map, to perform position serialization transformation on the processed image, and to encode the image features after position serialization transformation to obtain the generated label sequence.

[0095] The fusion module 440 is used to perform multi-scale serialization fusion of the initial marker sequence and the generated marker sequence to obtain a multi-scale marker sequence.

[0096] The conversion module 450 is used to convert the multi-scale labeled sequence into a multi-scale multi-channel feature map, and to perform accelerated convergence downsampling convolution and feature extraction convolution processing on the multi-scale channel feature map, and to determine whether the number of feature channels after the processing of the multi-scale channel feature map meets the preset standard number of feature channels.

[0097] The comparison module 460 is used to extract the one-dimensional channel features corresponding to the feature channels after meeting the preset standard number of feature channels, and to compare the similarity between the channel features and the pre-stored vehicle image channel features, and to take the vehicle information of the vehicle image with the highest similarity as the recognition result.

[0098] The return module 470 is used otherwise to take the multi-scale channel feature map as the image to be identified and return to perform accelerated convergent downsampling convolution on the image to be identified to obtain the initial downsampled multi-channel feature map.

[0099] The image-based vehicle re-identification device provided in this embodiment preprocesses the acquired vehicle images to obtain images to be identified; it then performs accelerated convergent downsampling convolution on the images to be identified to obtain an initial downsampling multi-channel feature map; it further performs feature extraction convolution on the initial downsampling multi-channel feature map, and performs positional serialization transformation on the image after feature extraction convolution, encoding the features of the image after positional serialization transformation to obtain an initial label sequence; it converts the initial label sequence into a transformed initial multi-channel feature map, and performs accelerated convergent downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map, performing positional serialization transformation on the processed image, and encoding the features of the image after positional serialization transformation to obtain a generated label sequence; finally, it converts the initial label sequence into a transformed initial multi-channel feature map. The sequence and generated label sequence are fused using multi-scale serialization to obtain a multi-scale label sequence. This multi-scale label sequence is then converted into a multi-scale multi-channel feature map. Accelerated convergence downsampling convolution and feature extraction convolution are performed on the multi-scale channel feature map. The number of feature channels in the processed multi-scale channel feature map is then determined to meet a preset standard number of feature channels. If the preset standard number of feature channels is met, one-dimensional channel features corresponding to each feature channel are extracted. The similarity of these channel features with pre-stored vehicle image channel features is compared, and the vehicle information of the image with the highest similarity is used as the recognition result. Otherwise, the multi-scale channel feature map is used as the image to be recognized, and the process returns to performing accelerated convergence downsampling convolution on the image to be recognized to obtain an initial downsampling multi-channel feature map. By converting the vehicle image features extracted through the neural network into one-dimensional multi-channel digital representation features, and matching these one-dimensional multi-channel digital representation features with a large number of pre-obtained one-dimensional multi-channel digital representation features of different car images, the similarity of the matching is used to determine if the same vehicle is identified. This eliminates the need for separate classification and comparison with each vehicle image using a neural network model, significantly reducing the computational cost of vehicle re-identification. Meanwhile, during vehicle image feature extraction, multi-scale sequential fusion is employed to fully integrate features from multiple feature extraction convolutions, significantly improving the accuracy of feature extraction. Furthermore, positional sequential transformation accurately extracts local vehicle features for unified global feature modeling, effectively addressing the issue of similar local features across different vehicle images. This further enhances the accuracy of vehicle re-identification.

[0100] Based on the above embodiments, the first encoding module and the second encoding module include:

[0101] The serialization transformation unit is used to uniformly divide the image into a preset even number of sub-image blocks of the same size;

[0102] Each sub-image block is evenly divided into a preset even number of feature image blocks;

[0103] The feature image blocks within each sub-image block are numbered in order from left to right and from top to bottom;

[0104] The feature image blocks with the same number in each feature block are combined separately to complete the positional serialization transformation.

[0105] Based on the above embodiments, the first encoding module and the second encoding module include:

[0106] Encoding unit, used for:

[0107] The processed image is segmented to generate a preset number of image blocks of the same size, and the relative position of each image block is recorded;

[0108] The relative positions are flattened into a planar image, and the planar image is input into the trained transformer encoder to obtain sequence coding features.

[0109] Based on the above embodiments, the fusion module includes:

[0110] The matching unit is used to match elements in the initial tag sequence and the generated tag sequence according to their sequence numbers;

[0111] The fusion unit is used to digitally fuse successfully matched elements, add a corresponding sequence number to the fused result, and obtain a multi-scale labeled sequence.

[0112] Based on the above embodiments, the fusion unit includes:

[0113] The vector transformation subunit is used to convert the successfully matched elements into vectors in a multidimensional vector space according to their corresponding positions in the matrix.

[0114] The fusion subunit synthesizes the vectors generated from the initial label sequence and the generated label sequence to achieve digital fusion.

[0115] Based on the above embodiments, the first encoding module and the second encoding module include:

[0116] An accelerated convergence downsampling convolutional unit is used to input the image to be recognized into the Conv_Block_a convolutional module. The Conv_Block_a convolutional module includes three main branches: the first branch consists of a convolutional layer and a batch normalization layer, with a convolution kernel size of 5*5 and a stride of 2; the second branch also consists of a convolutional layer and a batch normalization layer, with a convolution kernel size of 3x3 and a stride of 2; the third branch consists of a convolutional layer with a stride of 2, a convolution kernel size of 1x1, and a batch normalization layer, with a stride of 2.

[0117] Based on the above embodiments, the first encoding module and the second encoding module include:

[0118] The feature convolution processing unit is used to input the image after accelerated convergence downsampling into the Conv_Block_b module. The Conv_Block_b module includes three main branches. The first branch consists of a convolutional layer and a batch normalization layer. The convolutional layer has a convolution kernel size of 5*5 and a stride of 1. The second branch also consists of a convolutional layer and a batch normalization layer. The convolutional layer in the second branch has a convolution kernel size of 3x3 and a stride of 1. The third branch consists of a batch normalization layer. The feature matrix is ​​processed by the three branches, summed, and then activated by an activation function.

[0119] The image-based vehicle re-identification device provided in this embodiment of the invention can execute the image-based vehicle re-identification method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0120] Example 5

[0121] Figure 8 This is a schematic diagram of the structure of a server provided in Embodiment 5 of the present invention. Figure 8 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 8 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0122] like Figure 8 As shown, server 12 is presented as a general-purpose computing device. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0123] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0124] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0125] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0126] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0127] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable users to interact with server 12, and / or with any device that enables server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the image-based vehicle re-identification method provided in the embodiments of the present invention.

[0129] Example 7

[0130] Embodiment 7 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform any of the image-based vehicle re-identification methods provided in the above embodiments.

[0131] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0132] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0133] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0134] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An image-based vehicle re-identification method, characterized in that, include: The acquired vehicle images are preprocessed to obtain the images to be identified; The image to be identified is subjected to accelerated convergence downsampling convolution to obtain an initial downsampling multi-channel feature map; the initial downsampling multi-channel feature map is subjected to feature extraction convolution, and the processed image is subjected to position serialization transformation, and the image features after position serialization transformation are encoded to obtain an initial label sequence; The initial label sequence is converted into a transformed initial multi-channel feature map, and the transformed initial multi-channel feature map is subjected to accelerated convergence downsampling convolution and feature extraction convolution. The processed image is then subjected to position serialization transformation, and the image features after position serialization transformation are encoded to obtain the generated label sequence. The initial marker sequence and the generated marker sequence are fused using multi-scale serialization to obtain a multi-scale marker sequence; The multi-scale labeled sequence is converted into a multi-scale multi-channel feature map, and the multi-scale channel feature map is subjected to accelerated convergence downsampling convolution and feature extraction convolution. It is then determined whether the number of feature channels in the processed multi-scale channel feature map meets the preset standard number of feature channels. After meeting the preset standard number of feature channels, the one-dimensional channel features corresponding to the feature channels are extracted. The similarity of the channel features with the pre-stored vehicle image channel features is compared, and the vehicle information of the vehicle image with the highest similarity is taken as the recognition result. Otherwise, the multi-scale channel feature map is used as the image to be identified, and the process is repeated to perform accelerated convergence downsampling convolution on the image to be identified to obtain the initial downsampling multi-channel feature map; The step of performing positional serialization transformation on the processed image includes: The image is evenly divided into a preset even number of sub-image blocks of the same size; Each sub-image block is evenly divided into a preset even number of feature image blocks; The feature image blocks within each sub-image block are numbered in order from left to right and from top to bottom; The feature image blocks with the same number in each feature block are combined to complete the positional serialization transformation.

2. The method according to claim 1, characterized in that, The process of encoding the image features after positional serialization transformation to obtain a generated marker sequence includes: The processed image is segmented to generate a preset number of image blocks of the same size, and the relative position of each image block is recorded; The relative positions are flattened into a planar image, and the planar image is input into the trained transformer encoder to obtain sequence coding features.

3. The method according to claim 1, characterized in that, The step of fusing the initial marker sequence and the generated marker sequence into a multi-scale sequence includes: Match the elements in the initial label sequence and the generated label sequence according to their index; The successfully matched elements are digitally fused, and the fused result is incremented with the corresponding sequence number to obtain a multi-scale labeled sequence.

4. The method according to claim 3, characterized in that, The process of digitally fusing the successfully matched elements includes: The successfully matched elements are converted into vectors in a multidimensional vector space according to their corresponding positions in the matrix. The vectors generated from the initial label sequence and the generated label sequence are combined to achieve digital fusion.

5. The method according to claim 1, characterized in that, The step of performing accelerated convergence downsampling convolution on the image to be identified includes: The image to be recognized is input into the Conv_Block_a convolutional module, which includes three main branches. The first branch consists of a convolutional layer and a batch normalization layer, with a convolution kernel size of 5*5 and a stride of 2. The second branch also consists of a convolutional layer and a batch normalization layer, with a convolution kernel size of 3x3 and a stride of 2. The third branch consists of a convolutional layer with a stride of 2, a convolution kernel size of 1x1, and a batch normalization layer, with a stride of 2.

6. The method according to claim 5, characterized in that, The step of performing feature extraction and convolution processing on the initial downsampled multi-channel feature map includes: The image after accelerated convergence and downsampling is input into the Conv_Block_b module, which includes three main branches. The first branch consists of a convolutional layer and a batch normalization layer, with the convolutional layer having a kernel size of 5x5 and a stride of 1. The second branch also consists of a convolutional layer and a batch normalization layer, with the convolutional layer having a kernel size of 3x3 and a stride of 1. The third branch consists of a batch normalization layer. The feature matrix is ​​summed after being processed by the three branches and then activated by an activation function.

7. An image-based vehicle re-identification device, characterized in that, include: The preprocessing module is used to preprocess the acquired vehicle images to obtain the images to be recognized; The first encoding module is used to perform accelerated convergence downsampling convolution on the image to be identified to obtain an initial downsampling multi-channel feature map; to perform feature extraction convolution on the initial downsampling multi-channel feature map, and to perform position serialization transformation on the processed image, and to encode the image features after position serialization transformation to obtain an initial label sequence; The second encoding module is used to convert the initial label sequence into a transformed initial multi-channel feature map, and to perform accelerated convergence downsampling convolution and feature extraction convolution on the transformed initial multi-channel feature map. The processed image is then subjected to position serialization transformation, and the image features after position serialization transformation are encoded to obtain the generated label sequence. The fusion module is used to perform multi-scale serialization fusion of the initial marker sequence and the generated marker sequence to obtain a multi-scale marker sequence. The conversion module is used to convert the multi-scale labeled sequence into a multi-scale multi-channel feature map, and to perform accelerated convergence downsampling convolution and feature extraction convolution on the multi-scale channel feature map, and to determine whether the number of feature channels after the processing of the multi-scale channel feature map meets the preset standard number of feature channels. The comparison module is used to extract the one-dimensional channel features corresponding to the feature channels after meeting the preset standard number of feature channels, and to compare the similarity between the channel features and the pre-stored vehicle image channel features, and to take the vehicle information of the vehicle image with the highest similarity as the recognition result. The return module is used otherwise to treat the multi-scale channel feature map as the image to be identified and return to perform accelerated convergent downsampling convolution on the image to be identified to obtain the initial downsampled multi-channel feature map; The first encoding module and the second encoding module include: The serialization transformation unit is used to uniformly divide the image into a preset even number of sub-image blocks of the same size; Each sub-image block is evenly divided into a preset even number of feature image blocks; The feature image blocks within each sub-image block are numbered in order from left to right and from top to bottom; The feature image blocks with the same number in each feature block are combined to complete the positional serialization transformation.

8. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image-based vehicle re-identification method as described in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the image-based vehicle re-identification method as described in any one of claims 1-6.

Images