Vehicle re-identification method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle re-identification method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining different view angle images of a target vehicle; based on the local features of any visual angle corresponding to other visual angle images, enhancing the global features of any visual angle image to obtain the enhanced features of any visual angle image; fusing the local feature of the image at any view angle and the enhanced feature of the image at any view angle to obtain a fused feature of the image at any view angle; and carrying out vehicle re-identification on the target vehicle based on the fusion feature of any view angle image. According to the method, whether the vehicles in different images are the same vehicle can be accurately judged, the accuracy and robustness of vehicle re-identification are improved, and the problem that the traditional method only depends on a single image and is easily influenced by factors such as view angle change, illumination difference and shielding, so that the identification accuracy is reduced is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a vehicle re-identification method, device, electronic device and storage medium. Background Art

[0002] Vehicle re-identification (VRI) is a technology that can identify a specific vehicle in images captured by different cameras. As traffic monitoring networks continue to expand, VRI has become increasingly popular.

[0003] Currently, traditional vehicle re-ID methods primarily rely on single vehicle images for identification. This involves first extracting vehicle features from a single image and then matching these features with those of a query image to determine whether the query image contains the same vehicle. However, due to factors such as perspective changes, lighting variations, and occlusion, the features of the same vehicle in different images can vary significantly, reducing the accuracy and robustness of vehicle re-ID. Summary of the Invention

[0004] The present invention provides a vehicle re-identification method, device, electronic device and storage medium to solve the defects in the prior art.

[0005] The present invention provides a vehicle re-identification method, comprising the following steps: Acquire images of the target vehicle from different perspectives; Based on the local features of the images of the other perspectives corresponding to any perspective, the global features of the image of any perspective are enhanced to obtain the enhanced features of the image of any perspective; Fusing the local features of the image at any one viewing angle and the enhanced features of the image at any one viewing angle to obtain a fused feature of the image at any one viewing angle; Based on the fusion features of the images from any one of the perspectives, the target vehicle is re-identified.

[0006] According to a vehicle re-identification method provided by the present invention, the vehicle re-identification of the target vehicle based on the fusion features of the image from any one of the viewing angles includes: Perform attention calculation on the fusion features of each perspective image and determine the attention weight of each perspective image; Based on the attention weights of the images from each perspective, the fusion features of the images from each perspective are fused to obtain the perspective fusion features; Based on the perspective fusion feature, the target vehicle is re-identified.

[0007] According to a vehicle re-identification method provided by the present invention, performing attention calculation on the fusion features of each perspective image to determine the attention weight of each perspective image includes: Perform attention calculation on the fusion features of each view image to determine the initial attention weight of each view image; Based on the dependency relationship between images from each perspective, the initial attention weight of each perspective image is adjusted to obtain the attention weight of each perspective image.

[0008] According to a vehicle re-identification method provided by the present invention, the vehicle re-identification of the target vehicle based on the perspective fusion feature includes: The perspective fusion features are input into multiple expert models respectively, and each expert model extracts expert features from the perspective fusion features, where each expert feature has a different focus; Based on the weights of each expert model, the features of each expert are fused to obtain the target vehicle features; the weights of each expert model are determined based on the global features of each view image and the fusion features of each view image; Based on the target vehicle features, vehicle re-identification is performed on the target vehicle.

[0009] According to a vehicle re-identification method provided by the present invention, the global features of an image from any one perspective are enhanced based on the local features of images from other perspectives to obtain the enhanced features of the image from any one perspective; and the local features of the image from any one perspective and the enhanced features of the image from any one perspective are fused to obtain the fused features of the image from any one perspective, including: Extracting global features and local features of each perspective image based on a feature extraction model, enhancing the global features of any perspective image based on the local features of the remaining perspective images to obtain enhanced features of the any perspective image; fusing the local features of the any perspective image and the enhanced features of the any perspective image to obtain fused features of the any perspective image; The feature extraction model is trained based on multiple sample perspective images and the fusion features of each sample perspective image. The feature extraction model is trained with the goal of minimizing the difference between the global features and the fusion features of the same sample perspective image and maximizing the difference between the fusion features of different sample perspective images.

[0010] According to a vehicle re-identification method provided by the present invention, the loss value of the feature extraction model is determined based on the following steps: Determine the reconstruction loss value based on the difference between the global features and the fusion features of the same sample view image; Determine the contrast loss value based on the difference between the fusion features of images from different sample perspectives; The reconstruction loss value and the contrast loss value are fused to obtain a loss value of the feature extraction model.

[0011] According to a vehicle re-identification method provided by the present invention, each view image shares a feature extractor, and the feature extractor is used to extract global features and local features of each view image.

[0012] The present invention also provides a vehicle re-identification device, comprising the following modules: An acquisition unit, used for acquiring images of a target vehicle from different perspectives; an enhancement unit, configured to enhance the global features of an image from any perspective based on local features of images from other perspectives corresponding to any perspective, thereby obtaining enhanced features of the image from any perspective; a fusion unit, configured to fuse the local features of the image at any one viewing angle and the enhanced features of the image at any one viewing angle to obtain a fusion feature of the image at any one viewing angle; The recognition unit is used to re-identify the target vehicle based on the fusion features of the images from any one of the perspectives.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-described vehicle re-identification methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described vehicle re-identification methods.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned vehicle re-identification methods.

[0016] The vehicle re-identification method, device, electronic device, and storage medium provided by the present invention acquire images of a target vehicle from different perspectives, enhance the global features of any one perspective image using the local features of the remaining perspective images, then fuse the local and enhanced features, ultimately performing vehicle re-identification based on the fused features. Because the fused features combine the vehicle's global appearance information and local detail information, they overcome the interference caused by incomplete single-perspective image information and factors such as lighting and occlusion. Consequently, when performing vehicle re-identification based on the fused features, it is possible to more accurately determine whether vehicles in different images are the same vehicle, improving the accuracy and robustness of vehicle re-identification and avoiding the problem of traditional methods that rely solely on a single image and are easily affected by factors such as perspective changes, lighting differences, and occlusion, resulting in reduced recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is one of the flow charts of the vehicle re-identification method provided by the present invention.

[0019] Figure 2 This is the second flow chart of the vehicle re-identification method provided by the present invention.

[0020] Figure 3 This is the third flow chart of the vehicle re-identification method provided by the present invention.

[0021] Figure 4 This is the fourth flow chart of the vehicle re-identification method provided by the present invention.

[0022] Figure 5 It is a structural diagram of the feature extraction model provided by the present invention.

[0023] Figure 6 Schematic diagram of the workflow of the cross-view feature extraction module provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the vehicle re-identification device provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] At present, traditional vehicle re-identification methods mainly rely on a single vehicle image for identification, that is, first extracting vehicle features from a single vehicle image, and then matching the features with the features of the query image to determine whether the query image contains the same vehicle.

[0028] However, due to the different camera shooting angles, lighting conditions, and occlusions, the features of the same vehicle in different images often show significant differences. Specifically, changes in perspective cause the vehicle to present different geometric shapes and visible areas in different images, making it difficult for feature extraction methods based on specific perspectives to generalize effectively; lighting differences can cause changes in the color, brightness, and shadow distribution of vehicle images, thereby affecting the extraction of features that are sensitive to lighting, such as color histograms and texture features; occlusions can cause information loss in some key areas of the vehicle, such as license plates and logos, making it impossible to effectively utilize the original discriminative features. Under the influence of the above factors, traditional vehicle re-identification methods that rely solely on a single image find it difficult to extract stable and reliable vehicle feature representations, ultimately resulting in a significant reduction in recognition accuracy and robustness.

[0029] To this end, the present invention provides a vehicle re-identification method. Figure 1 This is one of the flow charts of the vehicle re-identification method provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .

[0030] Step 110: Acquire images of the target vehicle from different perspectives.

[0031] Here, the target vehicle refers to the designated vehicle for vehicle re-identification. Different-view images of the target vehicle refer to multiple images of the same target vehicle captured from different angles. For example, different-view images may include images from multiple perspectives, such as the front, side, back, and top views of the vehicle. Different-view images of the target vehicle may be captured collaboratively by multiple cameras or captured by a single camera at different time points, and this is not specifically limited in this embodiment of the present invention.

[0032] Considering that the image quality captured by different cameras may vary and that images from different perspectives may contribute differently to vehicle re-identification, after obtaining images of the target vehicle from different perspectives, these images can be preprocessed, such as image denoising, brightness correction, and perspective selection, to improve the accuracy and efficiency of subsequent feature extraction.

[0033] As an optional embodiment, the triggering conditions for acquiring images of the target vehicle from different perspectives may include: ① When a user enters the target vehicle's license plate number or uploads a picture of the target vehicle for query, the image acquisition process is triggered, and images of the target vehicle from different perspectives are retrieved from a database or real-time surveillance video stream. ② Virtual fences are set up at key intersections or areas. When the target vehicle enters these areas, the image acquisition process is automatically triggered to record images of the target vehicle from different perspectives. ③ At preset time intervals, images of the target vehicle from different perspectives are regularly acquired from various cameras.

[0034] Furthermore, to ensure image quality, you can prioritize image acquisition. For example, you can prioritize high-definition images or images containing key viewpoints. Furthermore, to reduce computational complexity, you can filter acquired images to remove duplicate or poor-quality images.

[0035] Step 120: Based on the local features of any perspective image corresponding to the other perspective images, enhance the global features of any perspective image to obtain enhanced features of any perspective image.

[0036] Specifically, the "any perspective image" refers to the target image selected from multiple perspective images for feature enhancement, and the "other perspective images" refer to the other perspective images except the "any perspective image." For example, if there are images of the front, side, and back of a vehicle, when the front image is selected as the "any perspective image," the side and back images are the "other perspective images." The "any perspective image" can be randomly selected from the multiple perspective images, or it can be selected from the multiple perspective images based on image quality or perspective completeness. For example, the perspective image with the highest image clarity can be selected as the "any perspective image," or the perspective image containing the complete outline of the vehicle can be selected as the "any perspective image."

[0037] In addition, local features refer to locally representative feature information in an image, which is used to characterize local details of the target vehicle, such as headlights, license plates, logos, windows, etc. Global features refer to features that describe overall information in an image, which are used to characterize the overall appearance of the target vehicle, such as the vehicle's color, model, overall outline, etc. Considering that the global features of an image from any perspective may be affected by factors such as lighting and occlusion, resulting in incomplete or inaccurate feature expression, if vehicle re-identification is performed based solely on the global features, misjudgment or missed detection may occur. For example, if the front image is exposed to strong light, resulting in distortion of color information, then relying solely on the global color features of the front image for identification may result in the vehicle being misjudged as a vehicle of another color.

[0038] At the same time, considering that the local features of the remaining view images contain complementary information related to the target vehicle, for example, if the front view image is occluded, the side view image may be able to provide unobstructed vehicle body information, which can compensate for the lack of global features of any one view image. Therefore, based on the local features of the remaining view images, the global features of any one view image are enhanced. The resulting enhanced features of any one view image can more comprehensively and accurately express the overall appearance and local details of the vehicle. Among them, the enhanced features can be understood as a comprehensive feature representation that integrates global and local information.

[0039] As an optional embodiment, an attention mechanism can be used to learn the importance weights of the local features of the remaining view images to the global features of any view image, and perform weighted fusion on the global features based on the weights to obtain enhanced global features.

[0040] Step 130: Fuse the local features of the image at any viewing angle and the enhanced features at any viewing angle to obtain the fused features of the image at any viewing angle.

[0041] Specifically, while the enhanced features from any one view incorporate local information from the remaining viewpoints, enhancing the expressive power of global features and more comprehensively describing the vehicle's overall appearance, the enhancement process may introduce noise or lose some detailed information. The local features from any one viewpoint preserve the vehicle's original details, such as the license plate and logo, thus compensating for the lack of detail in the enhanced features. In other words, if vehicle re-identification is performed based on the enhanced features from any one viewpoint, the loss of detailed information may lead to a decrease in recognition accuracy.

[0042] Based on this, the embodiments of the present invention combine the local features of any perspective image to make up for the possible lack of detail information in the enhanced features, so as to express the characteristic information of the vehicle more comprehensively and accurately, and improve the accuracy and robustness of vehicle re-identification.

[0043] Specifically, the embodiment of the present invention fuses the local features of an image from any perspective and the enhanced features from any perspective, so that the fused features of an image from any perspective obtained contain the overall appearance information and local detail information of the vehicle, thereby more effectively distinguishing vehicles and reducing misjudgments and missed judgments.

[0044] As an optional embodiment, when fusing the local features of an image from any perspective and the enhanced features from any perspective, the fusion can be performed in the following manner: ① The local features of an image from any perspective and the enhanced features from any perspective are concatenated to obtain corresponding fused features. ② Based on preset weights, the local features of an image from any perspective and the enhanced features from any perspective are weightedly fused to obtain corresponding fused features. The preset weights can be set based on actual conditions, or an attention mechanism can be used to determine the correlation between the local features and the enhanced features, and the preset weights can be determined based on the correlation.

[0045] Step 140: Re-identify the target vehicle based on the fusion features of the images from any perspective.

[0046] Specifically, since the fusion features of images from any perspective integrate the global appearance information and local detail information of the vehicle, they have stronger discrimination and robustness. Therefore, based on the fusion features of images from any perspective, it is possible to more accurately determine whether the vehicles in different images are the same vehicle, thereby achieving efficient and accurate vehicle re-identification.

[0047] After determining the fused features of any viewpoint image, this fused feature can be matched against a pre-established vehicle feature database or against fused features of other vehicle images acquired in real time to determine whether the target vehicle exists in the database or real-time image. This matching can be performed by calculating the distance between feature vectors (e.g., Euclidean distance, cosine similarity), or by using a trained classifier (e.g., support vector machine (SVM) or softmax classifier), though this is not specifically limited in this embodiment of the present invention.

[0048] The vehicle re-identification method provided by an embodiment of the present invention obtains images of a target vehicle from different perspectives, enhances the global features of any one perspective image using the local features of the remaining perspective images, then fuses the local and enhanced features, and ultimately performs vehicle re-identification based on the fused features. Because the fused features combine the vehicle's global appearance information and local detail information, they overcome the interference caused by incomplete single-perspective image information and factors such as lighting and occlusion. Therefore, when performing vehicle re-identification based on the fused features, it is possible to more accurately determine whether the vehicles in different images are the same vehicle, improving the accuracy and robustness of vehicle re-identification and avoiding the problem of traditional methods that rely solely on a single image and are easily affected by factors such as perspective changes, lighting differences, and occlusion, resulting in reduced recognition accuracy.

[0049] Based on the above embodiments, Figure 2 This is the second flow chart of the vehicle re-identification method provided by the present invention, such as Figure 2 As shown, the method includes: Step 210: Acquire images of the target vehicle from different perspectives.

[0050] Here, the target vehicle refers to the designated vehicle for vehicle re-identification. The target vehicle's different-view images refer to multiple images of the same target vehicle taken from different angles. Step 220: Based on the local features of the remaining perspective images, enhance the global features of any perspective image to obtain enhanced features of any perspective image.

[0051] Specifically, any one perspective image refers to a target image selected from multiple perspective images for feature enhancement, and the remaining perspective images refer to other perspective images except any one perspective image.

[0052] Furthermore, local features refer to locally representative features within an image, used to characterize the local details of a target vehicle. Global features, on the other hand, describe the overall image information and are used to characterize the overall appearance of a target vehicle. By enhancing the global features of any one viewpoint image based on the local features of the remaining viewpoint images, the resulting enhanced features for that viewpoint can more comprehensively and accurately represent the overall appearance and local details of the vehicle.

[0053] Step 230: Fuse the local features of the image at any viewing angle and the enhanced features at any viewing angle to obtain the fused features of the image at any viewing angle.

[0054] Specifically, the embodiment of the present invention fuses the local features of an image from any perspective and the enhanced features from any perspective, so that the fused features of an image from any perspective obtained contain the overall appearance information and local detail information of the vehicle, thereby more effectively distinguishing vehicles and reducing misjudgments and missed judgments.

[0055] Step 240: Perform attention calculation on the fusion features of each perspective image to determine the attention weight of each perspective image; based on the attention weight of each perspective image, fuse the fusion features of each perspective image to obtain a perspective fusion feature; based on the perspective fusion feature, perform vehicle re-identification on the target vehicle.

[0056] Specifically, the attention weight of each view image refers to its importance to the vehicle re-identification task, representing the amount of useful information contained in each view image. A larger attention weight indicates that the view image contains more discriminative information and contributes more to vehicle re-identification. A self-attention mechanism or a Transformer network can be used to calculate the attention of the fused features of each view image, although this is not specifically limited in this embodiment of the present invention. Furthermore, the fused features of each view image are determined using the method for obtaining the fused features of any view image described in the aforementioned embodiments.

[0057] After determining the attention weights for each view, the fusion features from each view are combined to create a view fusion feature. This feature combines information from different viewpoints and weights them according to their importance, enabling a more comprehensive representation of the target vehicle's characteristics. This improves the accuracy and robustness of vehicle re-identification based on the view fusion feature. Compared to vehicle re-identification based on the fusion features of any single viewpoint, this approach considers the differences in the contributions of images from different viewpoints, better utilizing multi-view information and achieving superior recognition performance.

[0058] Among them, when the fusion features of the images of each perspective are fused based on the attention weight of the images of each perspective, the fusion features of the images of each perspective can be weighted averaged, and the weight is the corresponding attention weight.

[0059] Based on any of the above embodiments, Figure 3 This is the third flow chart of the vehicle re-identification method provided by the present invention, such as Figure 3 As shown, the method includes: Step 310: Acquire images of the target vehicle from different perspectives.

[0060] Here, the target vehicle refers to the designated vehicle for vehicle re-identification. The target vehicle's different-view images refer to multiple images of the same target vehicle taken from different angles. Step 320: Based on the local features of the remaining perspective images, enhance the global features of any perspective image to obtain enhanced features of any perspective image.

[0061] Specifically, any one perspective image refers to a target image selected from multiple perspective images for feature enhancement, and the remaining perspective images refer to other perspective images except any one perspective image.

[0062] Furthermore, local features refer to locally representative features within an image, used to characterize the local details of a target vehicle. Global features, on the other hand, describe the overall image information and are used to characterize the overall appearance of a target vehicle. By enhancing the global features of any one viewpoint image based on the local features of the remaining viewpoint images, the resulting enhanced features for that viewpoint can more comprehensively and accurately represent the overall appearance and local details of the vehicle.

[0063] Step 330: Fuse the local features of the image at any viewing angle and the enhanced features at any viewing angle to obtain the fused features of the image at any viewing angle.

[0064] Specifically, the embodiment of the present invention fuses the local features of an image from any perspective and the enhanced features from any perspective, so that the fused features of an image from any perspective obtained contain the overall appearance information and local detail information of the vehicle, thereby more effectively distinguishing vehicles and reducing misjudgments and missed judgments.

[0065] Step 340: perform attention calculation on the fusion features of each perspective image to determine the initial attention weight of each perspective image; based on the dependency relationship between the images of each perspective, adjust the initial attention weight of each perspective image to obtain the attention weight of each perspective image; based on the attention weight of each perspective image, fuse the fusion features of each perspective image to obtain the perspective fusion feature; based on the perspective fusion feature, perform vehicle re-identification on the target vehicle.

[0066] Specifically, the initial attention weight refers to the importance of each perspective image calculated directly through the attention mechanism without dependency adjustment. The dependency between perspective images refers to the correlation between images from different perspectives. For example, there is a certain complementary relationship between the front image and the side image of the vehicle, that is, the front image can provide feature information of the front of the vehicle, while the side image can provide feature information of the side of the vehicle.

[0067] Considering that the initial attention weights of each view image obtained by performing attention calculation on the fused features of each view image may not fully reflect the dependency relationship between images from different views, if the fused features of each view image are fused based on the initial attention weights, important information may be ignored or noise information may be overly focused, thereby affecting the accuracy of vehicle re-identification. For example, suppose the front image of the vehicle has good lighting conditions and clear details, so the initial attention weight is high, while the side image has a low initial attention weight due to partial occlusion. However, if the side of the vehicle has a unique iconic sticker that can serve as key information for re-identification, then relying solely on the initial attention weights for fusion will ignore the useful information of the side image, resulting in recognition errors.

[0068] Based on this, the embodiments of the present invention adjust the initial attention weights of each view based on the dependencies between the images from different perspectives. This fully utilizes the complementary information between the images from different perspectives and suppresses the interference of noise information. The resulting attention weights of each view can more accurately reflect the contribution of each view to vehicle re-identification. Furthermore, when the fused features of each view are fused based on the attention weights of each view, the key features of the vehicle can be more effectively extracted, improving the accuracy and robustness of vehicle re-identification.

[0069] As an optional embodiment, a graph neural network (GNN) is constructed to model the dependencies between perspective images, and the GNN is used to learn and adjust the initial attention weights. The specific steps are as follows: each perspective image is regarded as a node in the graph, edges are established based on the correlation between perspective images (for example, adjacent perspectives), and the edge weights are obtained using manually set rules or through learning. The fused features of each perspective image are used as the initial features of the corresponding node. Using a graph convolution operation, the initial features of each node are aggregated with the initial features of its neighboring nodes to learn the dependencies between the nodes. The output of the graph convolution operation is fused with the initial attention weights (e.g., weighted averaging) to obtain the adjusted attention weights.

[0070] For example, suppose there are three views of a vehicle: the front, side, and back, with initial attention weights of 0.4, 0.3, and 0.3, respectively. If it is known that there is a strong dependency between the front and side images, GNN can be used to increase the attention weights of the front and side images, for example, to 0.5 and 0.4, respectively, while reducing the weight of the back image to 0.1. This can further highlight the importance of the front and side images and improve the accuracy of vehicle re-identification.

[0071] Based on any of the above embodiments, Figure 4 This is a fourth flow chart of the vehicle re-identification method provided by the present invention, such as Figure 4 As shown, the method includes: Step 410: Acquire images of the target vehicle from different perspectives.

[0072] Here, the target vehicle refers to the designated vehicle for vehicle re-identification. The target vehicle's different-view images refer to multiple images of the same target vehicle taken from different angles. Step 420: Based on the local features of the remaining perspective images, enhance the global features of any perspective image to obtain enhanced features of any perspective image.

[0073] Specifically, any one perspective image refers to a target image selected from multiple perspective images for feature enhancement, and the remaining perspective images refer to other perspective images except any one perspective image.

[0074] Furthermore, local features refer to locally representative features within an image, used to characterize the local details of a target vehicle. Global features, on the other hand, describe the overall image information and are used to characterize the overall appearance of a target vehicle. By enhancing the global features of any one viewpoint image based on the local features of the remaining viewpoint images, the resulting enhanced features for that viewpoint can more comprehensively and accurately represent the overall appearance and local details of the vehicle.

[0075] Step 430: Fuse the local features of the image at any viewing angle and the enhanced features at any viewing angle to obtain the fused features of the image at any viewing angle.

[0076] Specifically, the embodiment of the present invention fuses the local features of an image from any perspective and the enhanced features from any perspective, so that the fused features of an image from any perspective obtained contain the overall appearance information and local detail information of the vehicle, thereby more effectively distinguishing vehicles and reducing misjudgments and missed judgments.

[0077] Step 440: perform attention calculation on the fusion features of each view image to determine the attention weight of each view image; based on the attention weight of each view image, fuse the fusion features of each view image to obtain a view fusion feature; input the view fusion feature into multiple expert models respectively, and each expert model extracts expert features from the view fusion feature respectively, and each expert feature has a different focus point; based on the weight of each expert model, fuse the expert features to obtain a target vehicle feature; the weight of each expert model is determined based on the global features of each view image and the fusion features of each view image; based on the target vehicle feature, perform vehicle re-identification on the target vehicle.

[0078] Specifically, expert features refer to feature representations extracted from view-fusion features by different expert models, with different focuses. Expert models are used to extract specific feature information from view-fusion features. Different expert models have different network structures or training objectives, and thus the expert features output by each expert model have different focuses. The focus here can be understood as the type of features that different expert models focus on extracting, such as color features, texture features, shape features, and local detail features. For example, one expert model may focus on extracting global information such as the color and model of a vehicle, another expert model may focus on extracting local details such as the license plate and logo, and a third expert model may focus on extracting texture features.

[0079] Considering that the vehicle re-identification task requires comprehensive consideration of multiple different feature information to achieve more accurate identification, however, a single model may find it difficult to simultaneously extract all types of key features. After obtaining the view fusion feature, since the view fusion feature contains comprehensive information from images of different viewpoints, in order to more comprehensively extract the vehicle features and improve the accuracy of vehicle re-identification, the embodiment of the present invention inputs the view fusion feature into different expert models respectively, so that each expert model can focus on extracting a specific type of feature information, so that the expert features extracted by each expert model from the view fusion feature have different focuses and can complement each other.

[0080] On this basis, in order to effectively integrate the feature information extracted by different expert models to obtain a more distinctive vehicle representation, the embodiment of the present invention fuses the expert features based on the weights of each expert model to obtain the target vehicle feature. Compared with the perspective fusion feature, the target vehicle feature provides more comprehensive and detailed vehicle information. Based on the target vehicle feature, it is possible to more accurately determine whether the vehicles in different images are the same vehicle, thereby improving the accuracy of vehicle re-identification.

[0081] The weights of each expert model are determined based on the global features of each view image and the fused features of each view image. This means that the weights of each expert model can be adaptively adjusted based on the quality and importance of images from different view images. For example, if the global features of an image from a certain view indicate poor lighting conditions and distorted color information, the weight of the color feature extraction expert can be reduced, while the weights of other feature extraction experts can be increased to reduce the impact of lighting. Alternatively, if the fused features of an image from a certain view indicate that the image contains license plate information, the weight of the license plate recognition expert can be increased to more effectively utilize the license plate information.

[0082] Based on any of the above embodiments, based on the local features of the remaining perspective images, the global features of the image of any perspective are enhanced to obtain the enhanced features of the image of any perspective; the local features of the image of any perspective and the enhanced features of any perspective are fused to obtain the fused features of the image of any perspective, including: Based on the feature extraction model, the global features and local features of each view image are extracted. Based on the local features of the remaining view images, the global features of any view image are enhanced to obtain the enhanced features of any view image. The local features of any view image and the enhanced features of any view are fused to obtain the fused features of any view image. The feature extraction model is trained based on multiple sample perspective images and the fusion features of each sample perspective image. The feature extraction model is trained with the goal of minimizing the difference between the global features and the fusion features of the same sample perspective image and maximizing the difference between the fusion features of different sample perspective images.

[0083] Specifically, after obtaining different perspective images of the target vehicle, each perspective image can be input into the feature extraction model respectively, and the feature extraction model extracts the global features and local features of each perspective image. Based on the local features of the remaining perspective images, the global features of any perspective image are enhanced to obtain the enhanced features of any perspective image; the local features of any perspective image and the enhanced features of any perspective are fused to obtain the fused features of any perspective image.

[0084] The feature extraction model is trained to minimize the difference between the global features and fused features of images from the same sample viewpoint, and to maximize the difference between the fused features of images from different sample viewpoints. Minimizing the difference between the global features and fused features of images from the same sample viewpoint can be achieved by designing an appropriate loss function, such as the mean squared error loss. This allows the feature extraction model to learn how to effectively fuse global and local features, thereby improving the expressiveness and robustness of the global features.

[0085] Maximizing the difference between the fused features of images from different sample perspectives can use metric learning loss functions such as triplet loss, center loss, and contrast loss, so that the feature extraction model can learn how to extract discriminative feature representations, making the features of different vehicles as far apart as possible in the feature space, thereby improving the accuracy of vehicle re-identification.

[0086] That is to say, the feature extraction model is trained with the goal of minimizing the difference between the global features and the fused features of the same sample perspective image and maximizing the difference between the fused features of different sample perspective images. This can not only improve the quality and expressiveness of the features extracted by the model, but also enhance the model's ability to distinguish different vehicles, thereby improving the performance of vehicle re-identification.

[0087] Based on any of the above embodiments, the loss value of the feature extraction model is determined based on the following steps: Determine the reconstruction loss value based on the difference between the global features and the fusion features of the same sample view image; Determine the contrast loss value based on the difference between the fusion features of images from different sample perspectives; The reconstruction loss value and the contrast loss value are fused to obtain the loss value of the feature extraction model.

[0088] Specifically, the reconstruction loss measures the model's ability to reconstruct global features. It indicates whether the original feature information is retained after the global features are fused. The smaller the difference between the global features and the fused features of the same sample view, the smaller the reconstruction loss. This allows the feature extraction model to better learn how to fuse local features, enhancing global features while preserving as much detail as possible from the single-view image.

[0089] Contrastive loss measures the model's ability to distinguish between different samples. It represents the distance between features of different samples in feature space. The greater the difference between the fused features of images from different sample views, the smaller the contrastive loss. This allows the feature extraction model to learn how to extract more discriminative features, minimizing the distance between features of different vehicles in feature space and thus aligning semantic consistency across multiple views.

[0090] After determining the reconstruction loss and contrast loss, they can be fused based on preset weights to obtain the loss of the feature extraction model. The preset weights can be determined empirically or based on the validation set performance during model training. For example, if the model has high recognition accuracy on the validation set but low feature discrimination, the contrast loss weight can be appropriately increased; conversely, the reconstruction loss weight can be appropriately increased.

[0091] Among them, the loss value of the feature extraction model can be determined based on the following formula: in, represents the loss value of the feature extraction model, represents the contrast loss value, represents the reconstruction loss value, Represents the preset weight, which can balance the model's retention of single-view feature details and alignment of semantic consistency between multiple views, thereby improving the accuracy and robustness of vehicle re-identification.

[0092] It can be seen that the embodiment of the present invention fuses the reconstruction loss value and the contrast loss value to obtain the loss value of the feature extraction model, which enables the feature extraction model to learn to fuse multi-perspective information while not losing important details in the single-perspective image, and can effectively distinguish different vehicles, thereby improving the accuracy and robustness of vehicle re-identification and achieving more accurate and reliable vehicle re-identification.

[0093] Figure 5 It is a structural diagram of the feature extraction model provided by the present invention, such as Figure 5 As shown, the feature extraction model includes a multi-view feature extraction module, a cross-view feature extraction module and a multi-view feature fusion module.

[0094] The multi-view feature extraction module includes a backbone network shared by multiple view images, which is used to extract local and global features from each view image. The cross-view feature extraction module is used to perform view perception based on the global features of each view image and the local features of the remaining view images, enhance the global features of each view image to obtain enhanced features for each view image, and fuse the local features of each view image and the enhanced features of each view to obtain fused features for each view image.

[0095] Figure 6 This is a schematic diagram of the workflow of the cross-view feature extraction module provided by the present invention. Figure 6 As shown in the figure, visual perception is performed based on the local features (patch) of each view corresponding to the other view images and the global features (cls) of each view image to obtain the cross-view perception features of each view image. The cross-view perception features of each view image are then spliced ​​with the local features of each view image to obtain the initial enhanced features of each view image. Then, the spliced ​​features of each view image are average pooled (avg) to obtain the fused features of each view image.

[0096] The multi-view feature fusion module includes a view attention module, a gating network, and multiple view expert models. The gate network is used to determine the weights of each view expert model based on the global features of each view image output by the multi-view feature extraction module and the fusion features of each view image output by the cross-view feature extraction module. The view attention module is used to use the view attention mechanism to perform attention calculations on the fusion features of each view image, determine the attention weights of each view image, and fuse the fusion features of each view image based on the attention weights of each view image to obtain the view fusion features. Each view expert model is used to extract expert features from the view fusion features respectively, and finally fuse the expert features based on the weights of each view expert model to obtain the target vehicle features. Each view expert model is implemented by a multi-layer perceptron (MLP) and a ReLU activation function, as follows: Based on any of the above embodiments, each perspective image shares a feature extractor, and the feature extractor is used to extract global features and local features of each perspective image.

[0097] Specifically, the feature extractor is used to extract global features and local features of images from each perspective. The feature extractor can be a visual transformer (ViT) or a convolutional neural network (CNN), such as ResNet, DenseNet, etc., and this embodiment of the present invention does not specifically limit this.

[0098] If each view image uses a separate feature extractor to extract corresponding global and local features, the differing parameters of each feature extractor may lead to deviations in the extracted features, affecting the accuracy of subsequent feature fusion and vehicle re-identification. Therefore, in the embodiments of the present invention, each view image shares a feature extractor, ensuring that the extracted global and local features have a unified feature space, reducing feature deviations and ensuring the accuracy of subsequent feature fusion and vehicle re-identification. Furthermore, sharing the feature extractor reduces the number of model parameters, lowers computational complexity, and improves the model's generalization capabilities.

[0099] The vehicle re-identification device provided by the present invention is described below. The vehicle re-identification device described below and the vehicle re-identification method described above can be referenced to each other.

[0100] Based on any of the above embodiments, Figure 7 Schematic diagram of the structure of the vehicle re-identification device provided by the present invention, such as Figure 7 As shown, the device includes: An acquisition unit 710 is configured to acquire images of a target vehicle from different perspectives; An enhancement unit 720 is configured to enhance the global features of an image from any perspective based on the local features of the images from other perspectives corresponding to the image from any perspective, thereby obtaining enhanced features of the image from any perspective; A fusion unit 730 is configured to fuse the local features of the image at any perspective and the enhanced features of any perspective to obtain a fusion feature of the image at any perspective; The recognition unit 740 is used to re-identify the target vehicle based on the fusion features of the images from any perspective.

[0101] Based on any of the above embodiments, re-identifying a target vehicle based on the fusion features of images from any perspective includes: Perform attention calculation on the fusion features of each perspective image and determine the attention weight of each perspective image; Based on the attention weights of the images from each perspective, the fusion features of the images from each perspective are fused to obtain the perspective fusion features; Based on the perspective fusion features, the target vehicle is re-identified.

[0102] Based on any of the above embodiments, performing attention calculation on the fusion features of the images of each perspective to determine the attention weight of the images of each perspective includes: Perform attention calculation on the fusion features of each view image to determine the initial attention weight of each view image; Based on the dependency relationship between images from each perspective, the initial attention weight of each perspective image is adjusted to obtain the attention weight of each perspective image.

[0103] Based on any of the above embodiments, re-identifying the target vehicle based on the view fusion feature includes: The perspective fusion features are input into multiple expert models respectively, and each expert model extracts expert features from the perspective fusion features. Each expert feature has different focus points. Based on the weights of each expert model, the features of each expert are fused to obtain the target vehicle features; the weights of each expert model are determined based on the global features of each view image and the fusion features of each view image; Based on the target vehicle features, the target vehicle is re-identified.

[0104] Based on any of the above embodiments, based on the local features of the remaining perspective images, the global features of the image of any perspective are enhanced to obtain the enhanced features of the image of any perspective; the local features of the image of any perspective and the enhanced features of any perspective are fused to obtain the fused features of the image of any perspective, including: Based on the feature extraction model, the global features and local features of each view image are extracted. Based on the local features of the remaining view images, the global features of any view image are enhanced to obtain the enhanced features of any view image. The local features of any view image and the enhanced features of any view are fused to obtain the fused features of any view image. The feature extraction model is trained based on multiple sample perspective images and the fusion features of each sample perspective image. The feature extraction model is trained with the goal of minimizing the difference between the global features and the fusion features of the same sample perspective image and maximizing the difference between the fusion features of different sample perspective images.

[0105] Based on any of the above embodiments, the loss value of the feature extraction model is determined based on the following steps: Determine the reconstruction loss value based on the difference between the global features and the fusion features of the same sample view image; Determine the contrast loss value based on the difference between the fusion features of images from different sample perspectives; The reconstruction loss value and the contrast loss value are fused to obtain the loss value of the feature extraction model.

[0106] Based on any of the above embodiments, each perspective image shares a feature extractor, and the feature extractor is used to extract global features and local features of each perspective image.

[0107] Figure 8 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communications bus 840. The processor 810 may call logic instructions in the memory 830 to execute a vehicle re-identification method, which includes: acquiring images of a target vehicle from different perspectives; enhancing the global features of the image from any perspective based on the local features of the images from the other perspectives corresponding to any perspective to obtain enhanced features of the image from any perspective; fusing the local features of the image from any perspective and the enhanced features of the image from any perspective to obtain fused features of the image from any perspective; and re-identifying the target vehicle based on the fused features of the images from any perspective.

[0108] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the vehicle re-identification method provided by the above methods, which includes: obtaining different perspective images of the target vehicle; enhancing the global features of the image of any perspective based on the local features of the images of the other perspectives corresponding to any perspective to obtain the enhanced features of the image of any perspective; fusing the local features of the image of any perspective and the enhanced features of the image of any perspective to obtain the fused features of the image of any perspective; and re-identifying the target vehicle based on the fused features of the image of any perspective.

[0110] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the vehicle re-identification method provided by the above-mentioned methods, the method comprising: acquiring different perspective images of the target vehicle; enhancing the global features of the image of any perspective based on the local features of the images of the other perspectives corresponding to any perspective, to obtain the enhanced features of the image of any perspective; fusing the local features of the image of any perspective and the enhanced features of the image of any perspective to obtain the fused features of the image of any perspective; and re-identifying the target vehicle based on the fused features of the image of any perspective.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0112] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A vehicle re-identification method, characterized in that: include: Acquire images of the target vehicle from different perspectives; Based on the local features of the images of the other perspectives corresponding to any perspective, the global features of the image of any perspective are enhanced to obtain the enhanced features of the image of any perspective; Fusing the local features of the image at any one viewing angle and the enhanced features of the image at any one viewing angle to obtain a fused feature of the image at any one viewing angle; Based on the fusion features of the images from any one of the perspectives, the target vehicle is re-identified.

2. The vehicle re-identification method according to claim 1, characterized in that: The re-identifying the target vehicle based on the fusion features of the images from any one of the perspectives includes: Perform attention calculation on the fusion features of each perspective image and determine the attention weight of each perspective image; Based on the attention weights of the images from each perspective, the fusion features of the images from each perspective are fused to obtain the perspective fusion features; Based on the perspective fusion feature, the target vehicle is re-identified.

3. The vehicle re-identification method according to claim 2, characterized in that: The performing attention calculation on the fusion features of the images of each perspective to determine the attention weight of the images of each perspective includes: Perform attention calculation on the fusion features of each view image to determine the initial attention weight of each view image; Based on the dependency relationship between images from each perspective, the initial attention weight of each perspective image is adjusted to obtain the attention weight of each perspective image.

4. The vehicle re-identification method according to claim 2, characterized in that: The performing vehicle re-identification on the target vehicle based on the perspective fusion feature includes: The perspective fusion features are input into multiple expert models respectively, and each expert model extracts expert features from the perspective fusion features, where each expert feature has a different focus; Based on the weights of each expert model, the features of each expert are fused to obtain the target vehicle features; the weights of each expert model are determined based on the global features of each view image and the fusion features of each view image; Based on the target vehicle features, vehicle re-identification is performed on the target vehicle.

5. The vehicle re-identification method according to any one of claims 1 to 4, characterized in that: The global features of any one perspective image are enhanced based on the local features of the other perspective images to obtain the enhanced features of the any one perspective image; The local features of the image of any one viewing angle and the enhanced features of the image of any one viewing angle are fused to obtain the fused features of the image of any one viewing angle, including: Extracting global features and local features of each perspective image based on a feature extraction model, enhancing the global features of any perspective image based on the local features of the remaining perspective images to obtain enhanced features of the any perspective image; fusing the local features of the any perspective image and the enhanced features of the any perspective image to obtain fused features of the any perspective image; The feature extraction model is trained based on multiple sample perspective images and the fusion features of each sample perspective image. The feature extraction model is trained with the goal of minimizing the difference between the global features and the fusion features of the same sample perspective image and maximizing the difference between the fusion features of different sample perspective images.

6. The vehicle re-identification method according to claim 5, characterized in that: The loss value of the feature extraction model is determined based on the following steps: Determine the reconstruction loss value based on the difference between the global features and the fusion features of the same sample view image; Determine the contrast loss value based on the difference between the fusion features of images from different sample perspectives; The reconstruction loss value and the contrast loss value are fused to obtain a loss value of the feature extraction model.

7. The vehicle re-identification method according to any one of claims 1 to 4, characterized in that: Each view image shares a feature extractor, which is used to extract global features and local features of each view image.

8. A vehicle re-identification device, characterized in that: include: An acquisition unit, used for acquiring images of a target vehicle from different perspectives; an enhancement unit, configured to enhance the global features of an image from any perspective based on local features of images from other perspectives corresponding to any perspective, thereby obtaining enhanced features of the image from any perspective; a fusion unit, configured to fuse the local features of the image at any one viewing angle and the enhanced features of the image at any one viewing angle to obtain a fusion feature of the image at any one viewing angle; The recognition unit is used to re-identify the target vehicle based on the fusion features of the images from any one of the perspectives.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the vehicle re-identification method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle re-identification method according to any one of claims 1 to 7 is implemented.

Images