Training methods for object re-identification models, object re-identification methods and devices

Abstract

This disclosure provides a training method and apparatus for an object re-identification model, an object re-identification method and apparatus, an electronic device, and a computer-readable storage medium, relating to the field of image processing technology. The training method for the object re-identification model includes: performing occlusion processing on an original image to obtain multiple occluded images, each occluded image corresponding to a different occlusion region; sampling each occluded image to obtain a target image; and training a pre-trained object re-identification model using the original image and the target image as sample images; wherein the pre-trained object re-identification model is obtained by training a model to be trained based on the original image. This disclosure can improve the recognition accuracy of the object re-identification model.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to a training method for an object re-identification model, an object re-identification method, a training device for an object re-identification model, an object re-identification device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development and widespread application of deep learning technology, the trained model can be used for object re-identification, which can quickly obtain re-identification results. For example, vehicle re-identification can quickly match the target vehicle from a massive amount of monitoring data based on a given vehicle.

[0003] However, existing object re-identification models suffer from insufficient recognition accuracy, which affects the user experience of the models in practical applications. Summary of the Invention

[0004] The purpose of this disclosure is to provide a training method for an object re-identification model, an object re-identification method, a training device for an object re-identification model, an object re-identification device, an electronic device, and a computer-readable storage medium, thereby overcoming, at least to some extent, the problem of insufficient recognition accuracy in object re-identification models.

[0005] According to a first aspect of this disclosure, a method for training an object re-identification model is provided, comprising: performing occlusion processing on an original image to obtain multiple occluded images, each occluded image corresponding to a different occlusion region; performing sampling processing on each occluded image to obtain a target image; and training a pre-trained object re-identification model using the original image and the target image as sample images; wherein the pre-trained object re-identification model is obtained by training a model to be trained based on the original image.

[0006] In an exemplary embodiment of this disclosure, the step of occluding the original image to obtain multiple occluded images includes: determining multiple regions to be occluded in the original image using an occlusion mask; and performing occlusion processing on the original image based on each of the regions to be occluded to obtain the multiple occluded images of the original image.

[0007] In an exemplary embodiment of this disclosure, determining a plurality of regions to be occluded in the original image using an occlusion mask includes: determining the plurality of regions to be occluded in the original image in response to an adjustment operation of the occlusion mask; wherein, in response to each adjustment operation, an occlusion mask size corresponding to the adjustment operation and a sliding step size corresponding to the occlusion mask are obtained.

[0008] In an exemplary embodiment of this disclosure, the step of sampling each of the occluded images to obtain a target image includes: inputting each of the occluded images of the original image into the pre-trained object re-identification model to obtain the ground truth probability corresponding to each of the occluded images; and for each original image, sampling each of the occluded images of the original image according to the ground truth probability to obtain the target image.

[0009] In an exemplary embodiment of this disclosure, the step of sampling each occluded image of the original image according to the ground truth probability to obtain the target image for each original image includes: determining the occluded image corresponding to the target ground truth probability that is lower than a preset probability threshold as the target image.

[0010] In an exemplary embodiment of this disclosure, training a pre-trained object re-identification model using the original image and the target image as sample images includes: inputting the sample image into the pre-trained object re-identification model to obtain a model prediction result; constructing a loss function based on the model prediction result and the corresponding category label of the sample image; and adjusting the parameters of the pre-trained object re-identification model using the loss function to obtain a target object re-identification model.

[0011] In an exemplary embodiment of this disclosure, the pre-trained object re-identification model includes a first residual network layer, an attention network layer, a second residual network layer, and a fully connected network layer. The attention network layer includes a channel attention network and a spatial attention network. The step of inputting the sample image into the pre-trained object re-identification model to obtain a model prediction result includes: inputting the sample image into the first residual network for processing to obtain a first feature map; using the channel attention network to extract features from the first feature map to obtain channel attention features, and fusing the channel attention features with the first feature map to obtain a second feature map; inputting the second feature map into the spatial attention network to obtain a spatial attention feature map, and fusing the spatial attention feature map with the second feature map to obtain a third attention feature map; inputting the third attention feature map into the second residual network layer for processing, and predicting the processing result through the fully connected network layer to obtain the model prediction result.

[0012] In an exemplary embodiment of this disclosure, the channel attention network includes a first average pooling layer, a first max pooling layer, and a shared network layer. The step of using the channel attention network to extract features from the first feature map to obtain a channel attention feature map includes: aggregating features from the first feature map using the first average pooling layer to obtain first context features; aggregating features from the first feature map using the first max pooling layer to obtain second context features; inputting the first context features and the second context features into the shared network layer to map the first context features and the second context features to a shared feature space based on the shared network layer; and performing feature fusion on the output of the shared network layer to obtain the channel attention feature map.

[0013] In an exemplary embodiment of this disclosure, the spatial attention network includes a second average pooling layer, a second max pooling layer, and a convolutional layer; the step of inputting the second feature map into the spatial attention network to obtain a spatial attention feature map includes: inputting the second feature map into the second average pooling layer to obtain a third context feature; inputting the second feature map into the second max pooling layer to obtain a fourth context feature; concatenating the third context feature and the fourth context feature and then inputting them into the convolutional layer for convolution processing to obtain the spatial attention feature map.

[0014] According to a second aspect of this disclosure, an object re-identification method is provided, comprising: acquiring an image to be identified; inputting the image to be identified into a target object re-identification model to obtain a re-identification result of the image to be identified; wherein the target object re-identification model is obtained by training a training model according to the object re-identification model training method according to any one of the exemplary embodiments described above.

[0015] According to a third aspect of this disclosure, a training apparatus for an object re-identification model is provided, comprising: an occlusion processing module for performing occlusion processing on an original image to obtain multiple occluded images, each of the occluded images corresponding to a different occlusion region; an image sampling module for performing sampling processing on each of the occluded images to obtain a target image; and a model training module for training a pre-trained object re-identification model using the original image and the target image as sample images; wherein the pre-trained object re-identification model is obtained by training a model to be trained based on the original image.

[0016] According to a fourth aspect of this disclosure, an object re-identification apparatus is provided, comprising: an image acquisition module for acquiring an image to be identified; and an object recognition module for inputting the image to be identified into a target object re-identification model to obtain a re-identification result of the image to be identified; wherein the target object re-identification model is obtained by training a training model according to the object re-identification model training method according to any one of the exemplary embodiments described above.

[0017] According to a fifth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the method described above.

[0018] According to a sixth aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method described above.

[0019] The technical solution disclosed herein involves occlusion processing of the original image to obtain occluded images, sampling processing of each occluded image to obtain a target image, and then using the original image and the target image as sample images to train a pre-trained object re-identification model. During the model training phase, after training the model based on the original image, generating occluded samples enriches the training samples, simulating occlusion problems in real-world scenarios during training, increasing the model's recognition performance in handling occluded images, and improving the model's training accuracy. In the model application phase, processing the image to be identified using the trained object re-identification model makes it easier for the model to discover more discriminative features, improving the accuracy of object re-identification.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0022] Figure 1 A schematic diagram illustrating the stages involved in the object re-identification model scheme of this disclosure embodiment is shown;

[0023] Figure 2 A flowchart illustrating a training method for an object re-identification model according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 3 A flowchart illustrating an exemplary embodiment of the present disclosure is provided, showing how to acquire multiple occluded images of an original image.

[0025] Figure 4 This schematic diagram illustrates an exemplary embodiment of the present disclosure in which multiple regions to be occluded are determined by moving an occlusion mask.

[0026] Figure 5 This schematically illustrates an exemplary embodiment of the present disclosure. Figure 4 A schematic diagram of multiple occluded images corresponding to the original image;

[0027] Figure 6 The flowchart schematically illustrates an implementation method for sampling an occluded image of the original image in an exemplary embodiment of the present disclosure;

[0028] Figure 7 The flowchart schematically illustrates an implementation of training a pre-trained object re-identification model according to an exemplary embodiment of the present disclosure;

[0029] Figure 8 This schematic diagram illustrates the structure of a pre-trained object re-identification model in an exemplary embodiment of the present disclosure.

[0030] Figure 9 This schematic diagram illustrates the structure of a channel attention network according to an exemplary embodiment of the present disclosure;

[0031] Figure 10 This schematic diagram illustrates the structure of a spatial attention network in an exemplary embodiment of the present disclosure.

[0032] Figure 11 This schematically illustrates a visualization of attention features during the training phase of a model in an exemplary embodiment of the present disclosure.

[0033] Figure 12 A flowchart illustrating an object re-identification method according to an exemplary embodiment of the present disclosure is shown schematically.

[0034] Figure 13 This schematic diagram illustrates the composition of a training apparatus for an object re-identification model in an exemplary embodiment of the present disclosure.

[0035] Figure 14 This schematic diagram illustrates the composition of an object re-identification device according to an exemplary embodiment of the present disclosure;

[0036] Figure 15 A schematic diagram of an electronic device to which embodiments of the present disclosure may be applied is shown. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] Figure 1 The diagram illustrates the stages involved in the object re-identification model scheme of this disclosure embodiment, as shown below. Figure 1 As shown, the technical solution of this disclosure includes a model training stage and a model application stage.

[0040] In this embodiment, the training method for the object re-identification model can be executed by a terminal device. During the training phase, the terminal device can perform occlusion processing on the original image to obtain multiple occluded images, then sample each occluded image to obtain a target image. Finally, using the original image and the target image as sample images, the pre-trained object re-identification model is trained to obtain the target object re-identification model. The application phase of the object re-identification model can also be executed on the terminal device. In the object re-identification phase, the terminal device acquires the image to be identified and inputs it into the target object re-identification model to obtain the re-identification result. In this approach, where the training method and all steps of the object re-identification method provided in this embodiment can be executed by the terminal device, all steps can be executed by the terminal device's processor.

[0041] The terminal device can be an intelligent device with data processing capabilities, such as a monitoring device, smartphone, computer, tablet computer, in-vehicle device, wearable device, etc. The terminal device can also be referred to as a mobile terminal, terminal, mobile device, etc. This disclosure does not limit the type of terminal device.

[0042] Furthermore, the technical solutions provided in the embodiments of this disclosure can also be executed by a server. Correspondingly, in this server-executed manner, the server can respond to a triggering command to begin executing the steps in the technical solutions of the embodiments of this disclosure. This triggering command can be sent by a user-used terminal device, or it can be triggered locally by the server in response to some automated event. The server can be a backend system providing the relevant services in the embodiments of this disclosure, and may include a single electronic device with computing capabilities, such as a portable computer, desktop computer, or smartphone, or a cluster of multiple electronic devices.

[0043] Furthermore, the technical solutions of this disclosure can also be executed collaboratively by a terminal device and a server. In this method of collaborative execution by an electronic device and a server, some steps in the technical solutions provided by this disclosure are executed by the terminal device, while other steps are executed by the server. For example, the technical solutions provided by this disclosure can have the server execute the training phase of the object re-identification model and send the trained target object re-identification model to the terminal device, whereby the terminal device executes the model application phase.

[0044] It should be noted that in this method where the terminal device and the server work together, the steps executed by the terminal device and the server can be dynamically adjusted according to the actual situation, and this embodiment does not impose any special restrictions on this.

[0045] Object re-identification refers to quickly matching the same object from massive amounts of monitoring data based on a given object. The object can be a vehicle, a person, an item, etc., and the embodiments of this disclosure can be used for scenarios involving the re-identification of any object.

[0046] With the continuous development of deep learning technology, sophisticated network models are constructed to extract discriminative features, or various loss functions are used for metric learning to increase the recognition accuracy of object re-identification models. However, in practical applications, objects are often occluded; for example, vehicles can be obscured by pedestrians, trees, other vehicles, onboard objects, and other obstacles. For scenarios where objects are occluded, the aforementioned model training methods suffer from insufficient recognition accuracy.

[0047] To address one or more of the aforementioned problems, this exemplary embodiment first provides a method for training an object re-identification model. (See reference...) Figure 2 As shown, the training method for this object re-identification model may include the following steps S210 to S230:

[0048] In step S210, the original image is occluded to obtain multiple occluded images, each corresponding to a different occluded region.

[0049] In an exemplary embodiment of this disclosure, the original image is an image from the original training sample set, which includes multiple original images, X = {X1, X2, X3, ..., Xn}. For each original image, occlusion processing is performed to obtain N occluded images, and the N occluded images have different occlusion regions. That is, each original image in the original training sample set has N occluded images.

[0050] In step S220, each occluded image is sampled to obtain the target image.

[0051] In an exemplary embodiment of this disclosure, by sampling the occluded images of the original images, each original image can generate occluded images with different occluded areas, and the target image is obtained by sampling from each occluded image.

[0052] Optionally, random sampling can be performed on each occluded image to obtain the target image. Alternatively, to improve the model's predictive ability for occluded images through model training, the sampling process can also select images from among the occluded images whose predictions are inaccurate as the target image; that is, occluded images that have a significant impact on model prediction are selected as the target images. This allows for strengthening the model's predictive ability for such images in subsequent model training stages. Of course, other sampling methods can also be used in the embodiments of this disclosure, including but not limited to the sampling methods described above.

[0053] In step S230, the pre-trained object re-identification model is trained using the original image and the target image as sample images; wherein, the pre-trained object re-identification model is obtained by training the model to be trained based on the original image.

[0054] In an exemplary embodiment of this disclosure, the model to be trained is first trained based on the original image to obtain a pre-trained object re-identification model. Then, the pre-trained object re-identification model is trained using the original image and the target image as sample images. The category label of the occluded image is consistent with that of its corresponding original image.

[0055] The object re-identification model training method of this disclosure involves occlusion processing of the original image to obtain occluded images, sampling processing of each occluded image to obtain a target image, and then using the original image and the target image as sample images to train the pre-trained object re-identification model. During the model training phase, after training the model based on the original image, generating occluded samples enriches the training samples. This simulates occlusion problems in real-world scenarios during training, improving the model's recognition performance in handling occluded images and enhancing the model's training accuracy. Furthermore, in the model application phase, processing the target object re-identification model with the image to be identified allows the model to more easily uncover more discriminative features, improving the accuracy of object re-identification.

[0056] In one exemplary embodiment, an implementation method is provided for acquiring multiple occluded images of an original image. For example... Figure 3 As shown, obtaining multiple occluded images by occlusion processing of the original image may include steps S310 and S320:

[0057] Step S310: Determine multiple areas to be occluded in the original image using an occlusion mask.

[0058] Image masking is the process of using a selected occlusion mask to occlude all or part of an image to obtain a new image containing the occluded areas. The occlusion mask is a completely black image of a certain size, and by moving the occlusion mask, multiple areas to be occluded can be identified in the original image.

[0059] Figure 4 This schematically illustrates a method for determining multiple areas to be occluded by moving an occlusion mask. (Refer to...) Figure 4 An occlusion mask of a predetermined size can slide on the original image Xn with a predetermined sliding step size, and a region to be occluded is determined each time it moves to a position.

[0060] For example, given an original image of size H*W and an occlusion mask of size d*d, when the stride is S... w and S h S h Moving from left to right and from top to bottom will generate a total of X occlusion images, as shown in formula (1):

[0061]

[0062] It should be noted that, in each embodiment of this disclosure, after determining a region to be occluded, an occlusion image corresponding to that region will be subsequently obtained. For example, as shown... Figure 4When the occlusion mask moves to position 3, the occlusion image will be obtained subsequently, which includes the area to be occluded determined by position 3. The area to be occluded determined by moving to other positions is not included in the occlusion image. That is, an occlusion image only includes one area to be occluded.

[0063] Step S320: Perform occlusion processing on the original image based on each region to be occluded to obtain multiple occluded images of the original image.

[0064] After determining the regions to be occluded in the original image, occlusion processing can be performed on the original image based on each region, resulting in multiple occluded images of the original image. For example, such as... Figure 5 It shows Figure 4 The original image corresponds to multiple occluded images, of course. Figure 5 As an example only, the number of occluded images corresponding to the original image is related to the size of the occlusion mask and the sliding step size.

[0065] In this embodiment of the disclosure, the above-described occlusion process is performed on each original image to obtain multiple occluded images for each original image.

[0066] In an exemplary embodiment, to enrich the occluded image, step S320 may further include:

[0067] In response to adjustments to the occlusion mask, identify multiple areas to be occluded in the original image;

[0068] In response to each adjustment operation, the occlusion mask size and the sliding step size corresponding to the occlusion mask are obtained.

[0069] Users can adjust the size and sliding step of the occlusion mask through the interactive interface for adjusting the occlusion mask. Thus, the embodiments of this disclosure can generate occlusion images containing occlusion regions of different sizes for each original image, and control the number of occlusion images generated for each original image by adjusting the sliding step, thereby obtaining rich occlusion images to enrich the sample images and thus enable the model to have better performance.

[0070] In one exemplary embodiment, an implementation method is provided for sampling processing of an occluded image of the original image. For example... Figure 6 As shown, sampling each occluded image to obtain the target image may include steps S610 and S620:

[0071] Step S610: Input each occluded image of the original image into the pre-trained object re-identification model to obtain the ground truth probability corresponding to each occluded image.

[0072] The ground truth probabilities obtained by inputting occluded images into a pre-trained object re-identification model can reflect the degree of influence of the occluded region contained in the occluded image on the model's prediction accuracy. By inputting each occluded image of an original image into the pre-trained object re-identification model, the ground truth probability corresponding to each occluded image can be obtained.

[0073] Step S620: For each original image, sample each occluded image of the original image according to the true probability to obtain the target image.

[0074] This disclosure embodiment can select the target image from multiple occluded images of each original image that has a greater impact on the model's prediction accuracy.

[0075] In some possible embodiments, the occluded image corresponding to the ground truth probability of the target image that is below a preset probability threshold can be determined as the target image. The preset probability threshold can be set according to the actual model requirements, such as 0.35, 0.40, etc., and is not specifically limited thereto.

[0076] For example, if multiple occluded images {1,2,3,...,M} of an original image are input into a pre-trained object re-identification model, and the resulting ground truth probabilities are {0.85,0.88,...,0.11,...,0.32,...,0.90}, and the preset probability threshold is 0.35, then the occluded images with ground truth probabilities lower than 0.35 are identified as the target images.

[0077] Specifically, the ground truth probabilities of the M occluded images in each original image can be normalized to the same distribution, and sampling can be performed based on this distribution to fairly evaluate the impact of different occlusion positions on the model. For example, for each original image, sampling each occluded image based on its ground truth probability to obtain the target image can include:

[0078] The ground truth probability obtained by inputting the original image into the pre-trained object re-identification model is p. When the position of the occlusion mask slides to position l (l∈1,2,…,M), the occluded image is processed by the pre-trained object re-identification model, and the corresponding ground truth probability is p. l Then we get formula (2):

[0079]

[0080] The ground truth probabilities of the M occluded images in the original image are then normalized to the same distribution, as shown in formula (3):

[0081]

[0082] in, Let M be the distribution of the M occluded images corresponding to a given original image.

[0083] Finally, based on the distribution shown in formula (3), each occluded image of the original image is sampled from the distribution to obtain the target image.

[0084] It should be noted that, in this embodiment of the present disclosure, each original image can be sampled and processed according to the above sampling method to obtain the target image corresponding to each original image.

[0085] In one exemplary embodiment, an implementation method for training a pre-trained object re-identification model is also provided. See [link to example]. Figure 7 The process includes steps S710 to S730:

[0086] In step S710, the sample image is input into the pre-trained object re-identification model to obtain the model prediction result.

[0087] The pre-trained object re-identification model in this embodiment includes a first residual network layer, an attention network layer, a second residual network layer, and a fully connected network layer. The attention network layer includes a channel attention network and a spatial attention network, such as... Figure 8 As shown.

[0088] based on Figure 8 The network structure shown allows for the following steps: First, the sample image is input into a first residual network for processing to obtain a first feature map. Then, a channel attention network is used to extract features from the first feature map to obtain channel attention features. These channel attention features are then fused with the first feature map to obtain a second feature map. Next, the second feature map is input into a spatial attention network to obtain a spatial attention feature map. This spatial attention feature map is then fused with the second feature map to obtain a third attention feature map. Finally, the third attention feature map is input into a second residual network layer for processing, and the processing result is used for prediction through a fully connected network layer to obtain the model prediction result.

[0089] The second feature map is obtained by fusing the channel attention features with the first feature map by multiplying the corresponding feature data of the first feature map in pairs. The third attention feature map is obtained by fusing the spatial attention feature map with the second feature map by multiplying the feature data of the second feature map in pairs. During the pairwise multiplication process, the attention values ​​are broadcast sequentially, with the channel attention values ​​broadcast along the spatial dimension and vice versa. As shown in formulas (4) and (5):

[0090]

[0091]

[0092] Where X1 is the second feature map, M c Here, X represents the channel attention feature, and X is the first feature map. For positional multiplication, X2 is the third attention feature map, M s This is a spatial attention feature.

[0093] This disclosure introduces an attention mechanism and uses channel attention networks and spatial attention networks to deeply mine features in regions other than occluded areas. Based on the attention network layer of this disclosure, the extracted features are more discriminative for images with occluded areas, thereby improving the training accuracy of the model.

[0094] In one exemplary embodiment, such as Figure 9 A schematic diagram of the structure of a channel attention network according to an exemplary embodiment of the present disclosure is shown, such as... Figure 9 As shown, the channel attention network includes a first average pooling layer, a first max pooling layer, and a shared network layer.

[0095] To effectively obtain channel attention features, the input 3D features are compressed along the spatial dimension. Specifically, the channel attention feature map is obtained by extracting features from the first feature map using a channel attention network, and may include:

[0096] First, the first feature map is aggregated using the first average pooling layer to obtain the first context feature F. avg Furthermore, the first feature map is aggregated through a first max-pooling layer to obtain the second contextual feature F. max Then, the first context feature F avg Second contextual feature F max The input is fed into a shared network layer to process the first contextual features F based on the shared network layer. avg Second contextual feature F max The outputs of the shared network layers are mapped to a shared feature space. Finally, the outputs of the shared network layers are fused to obtain the channel attention feature map M. c (x)∈R C*1*1 .

[0097] The shared network may include a multilayer perceptron (MLP) and at least one hidden layer. To reduce the network overhead of the hidden layer, its output size can be set to... Where r is the reduction ratio.

[0098] The channel attention feature map can be obtained by fusing the output of the shared network layer by adding the two features output by the shared network in pairs. Specifically, the processing of the channel attention feature can be expressed by the following formula (6):

[0099]

[0100] Where AvgPool() is the first average pooling operation, MaxPool() is the first max pooling operation, and sigmoid is the activation function. It is worth noting that the weights W0 and W1 of the multilayer perceptron are shared, and the ReLU activation function is included after W0.

[0101] This disclosure generates channel attention features based on the inter-channel relationships of features. Since each channel of the feature map can be considered a feature detector, channel attention focuses on "what information" is meaningful in a given image. Therefore, this disclosure uses a channel attention network to make the model pay more attention to the features of other regions in the image besides the occluded areas, thereby improving the accuracy of object re-identification in occluded scenes after training.

[0102] In one exemplary embodiment, such as Figure 10 A schematic diagram of the structure of a spatial attention network according to an exemplary embodiment of the present disclosure is shown, such as... Figure 10 As shown, the spatial attention network includes a second average pooling layer, a second max pooling layer, and a convolutional layer.

[0103] Specifically, using a spatial attention network to extract features from the second feature map to obtain a spatial attention feature map can include:

[0104] First, the second feature map is input into the second average pooling layer to obtain the third context feature; then, the second feature map is input into the second max pooling layer to obtain the fourth context feature; finally, the third and fourth context features are concatenated and input into the convolutional layer for convolution processing to obtain the spatial attention feature map.

[0105] The processing procedure of the spatial attention network described above can be expressed by the following formula (7):

[0106] M s X = sigmoid(f 7*7 ([AvgPool(X);MaxPool(X)])) (7)

[0107] Where AvgPool() is the second average pooling operation and MaxPool() is the second max pooling operation f 7*7 The operation is a convolution with a size of 7*7. Of course, other operations are also possible, and this disclosure does not impose any special limitations on this.

[0108] Generating a spatial attention map using spatial relationships of features differs from channel attention. Spatial attention focuses on identifying regions containing meaningful information, complementing channel attention. In this embodiment, applying pooling operations along the channel dimension effectively highlights information regions. To obtain a spatial attention map, average pooling and max pooling operations are first applied along the channel dimension, and the results are concatenated to generate effective spatial features.

[0109] The pre-trained object re-identification model of this disclosure uses channel attention network and spatial attention network to mine depth information, making the model more focused on mining features in areas other than the occluded areas. It is organically combined with the occluded images in the sample images used in this disclosure. The object re-identification model based on the attention mechanism of occluded images can extract more discriminative features in the image and improve the object re-identification accuracy of the model.

[0110] For example, taking vehicle re-identification as an example, the pre-trained object re-identification model of this embodiment of the present disclosure will mine features of other parts during the training process due to occlusion images that cover the headlights and the front of the vehicle. For example, the vehicle inspection mark on the vehicle can be used as the identification of the vehicle, thereby improving the recognition accuracy of vehicle re-identification in occluded scenarios.

[0111] For example, Figure 11 The diagram illustrates the visualization results of attention features during the model training phase. Figure (1) shows the original image, Figure (2) shows the feature visualization results corresponding to the pre-trained object re-identification model, Figure (3) shows the input sample image (including occluded regions) when training the pre-trained object re-identification model, and Figure (4) shows the feature visualization results after training the pre-trained object re-identification model. It can be seen that when the discriminative information in the input model image is occluded, retraining can enable the model to discover other more discriminative regions. Figure 11 The dashed box represents the part that the model is most interested in. By occluding these parts and using them as input when training the pre-trained object re-identification model, the attention network can be encouraged to discover other discriminative parts that are worth paying attention to.

[0112] In step S720, a loss function is constructed based on the model prediction results of the sample images and the corresponding category labels.

[0113] According to actual needs, the embodiments of this disclosure can construct loss functions based on the model prediction results of sample images and corresponding category labels, such as cross-entropy loss, etc., and the embodiments of this disclosure do not impose special limitations on this.

[0114] The following section uses the construction of cross-entropy loss as an example to illustrate the construction loss function of this embodiment.

[0115] If the sample image set T includes M types of objects and N sample images, where each sample image is denoted as (I... i ,c i ), i∈{1,2,...,N}, where L i c is the class label for sample image I. i Feature information for each sample image.

[0116] The sample image is input into a pre-trained object re-identification model, which is equivalent to mapping the sample image I to the classification score vector S. i =g(c i The function ) normalizes the classification score vector to a probability distribution using the softmax function:

[0117]

[0118] Among them, Z im S represents the probability value of each category for the i-th sample image. ij Let be the probability value of the i-th sample corresponding to class j. Then, the loss function for the entire sample image set is obtained through the following formula:

[0119] l i (θ)=-log(p(y ici |I i (9)

[0120]

[0121] Where θ is the network parameter, and l(θ) is the loss function over the entire sample image set T.

[0122] In step S730, the parameters of the pre-trained object re-identification model are adjusted using the loss function to obtain the target object re-identification model.

[0123] The pre-trained object re-identification model is adjusted using the loss function until the model converges, thus obtaining the target object re-identification model.

[0124] Since minimizing the loss function is equivalent to maximizing the posterior probability of the true value, the embodiments of this disclosure use stochastic gradient descent and mini-batch samples to optimize the loss function.

[0125] It is worth noting that in the training phase of this embodiment, each object is treated as a class, and the probability of each sample image belonging to each class is obtained after passing through the classifier. During the testing phase, the classifier, i.e., the final fully connected layer, is removed, and the remaining layer serves as a feature extractor. The testing process in this embodiment is conventional and will not be described further.

[0126] The object re-identification model training method disclosed in this embodiment involves occlusion processing of the original image to obtain occluded images, sampling processing of each occluded image to obtain target images, and then using the original image and target image as sample images to train a pre-trained object re-identification model. During the model training phase, after training the model based on the original image, generating occluded samples enriches the training samples. This simulates occlusion problems in real-world scenarios during training, increasing the model's recognition performance in handling occluded images and improving the model's training accuracy. Furthermore, by introducing an attention network into the model, the model is able to discover other noteworthy discriminative features in images with occluded regions, prompting the model to escape local optima and improving the accuracy of object re-identification in occluded images.

[0127] Further reference Figure 12 As shown, an exemplary embodiment of this disclosure provides an object re-identification method, including steps S1210 and S1220:

[0128] Step S1210: Obtain the image to be recognized;

[0129] Step S1220: Input the image to be recognized into the target object re-recognition model to obtain the re-recognition result of the image to be recognized; wherein, the target object re-recognition model is obtained by training the training model of the object re-recognition model according to the training method of the object re-recognition model according to any of the above exemplary embodiments.

[0130] The model training process involved in step S1220 has been described in detail in the above exemplary embodiments and will not be repeated here.

[0131] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0132] Further reference Figure 13 As shown, an exemplary embodiment of this disclosure provides a training apparatus 1300 for an object re-identification model, including an occlusion processing module 1310, an image sampling module 1320, and a model training module 1330. Wherein:

[0133] The occlusion processing module 1310 is used to perform occlusion processing on the original image to obtain multiple occlusion images, each of which corresponds to a different occlusion area.

[0134] Image sampling module 1320 is used to sample each of the occluded images to obtain the target image;

[0135] The model training module 1330 is used to train a pre-trained object re-identification model using the original image and the target image as sample images; wherein the pre-trained object re-identification model is obtained by training the model to be trained based on the original image.

[0136] In an exemplary embodiment, the occlusion processing module 1310 is configured to perform: determining a plurality of regions to be occluded in the original image using an occlusion mask; and performing occlusion processing on the original image based on each of the regions to be occluded to obtain the plurality of occluded images of the original image.

[0137] In one exemplary embodiment, the occlusion processing module 1310 is configured to perform: in response to an adjustment operation for the occlusion mask, determining the plurality of regions to be occluded in the original image; wherein, in response to each adjustment operation, an occlusion mask size corresponding to the adjustment operation and a sliding step size corresponding to the occlusion mask are obtained.

[0138] The occlusion processing module 1310 is configured to perform the following: The image sampling module 1320 is configured to perform the following: Input each of the occluded images of the original image into the pre-trained object re-identification model to obtain the ground truth probability corresponding to each of the occluded images; For each original image, perform sampling processing on each of the occluded images of the original image according to the ground truth probability to obtain the target image.

[0139] In one exemplary embodiment, the image sampling module 1320 is configured to perform the following: determine the occluded image corresponding to the target ground truth probability that is lower than a preset probability threshold as the target image.

[0140] In an exemplary embodiment, the model training module 1330 is configured to perform: inputting the sample image into the pre-trained object re-identification model to obtain a model prediction result; constructing a loss function based on the model prediction result of the sample image and the corresponding category label; and adjusting the parameters of the pre-trained object re-identification model using the loss function to obtain a target object re-identification model.

[0141] In an exemplary embodiment, the pre-trained object re-identification model includes a first residual network layer, an attention network layer, a second residual network layer, and a fully connected network layer. The attention network layer includes a channel attention network and a spatial attention network. The model training module 1330 is configured to perform the following: inputting the sample image into the first residual network for processing to obtain a first feature map; extracting features from the first feature map using the channel attention network to obtain channel attention features, and fusing the channel attention features with the first feature map to obtain a second feature map; inputting the second feature map into the spatial attention network to obtain a spatial attention feature map, and fusing the spatial attention feature map with the second feature map to obtain a third attention feature map; inputting the third attention feature map into the second residual network layer for processing, and predicting the processing result through the fully connected network layer to obtain the model prediction result.

[0142] In an exemplary embodiment, the channel attention network includes a first average pooling layer, a first max pooling layer, and a shared network layer; the model training module 1330 is configured to perform: feature aggregation on the first feature map using the first average pooling layer to obtain a first context feature; feature aggregation on the first feature map using the first max pooling layer to obtain a second context feature; inputting the first context feature and the second context feature to the shared network layer to map the first context feature and the second context feature to a shared feature space based on the shared network layer; and performing feature fusion on the output of the shared network layer to obtain the channel attention feature map.

[0143] In an exemplary embodiment, the spatial attention network includes a second average pooling layer, a second max pooling layer, and a convolutional layer; the model training module 1330 is configured to perform: inputting the second feature map into the second average pooling layer to obtain a third context feature; inputting the second feature map into the second max pooling layer to obtain a fourth context feature; concatenating the third context feature and the fourth context feature and inputting them into the convolutional layer for convolution processing to obtain the spatial attention feature map.

[0144] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0145] In addition, such as Figure 14 As shown, this disclosure also provides an object re-identification device 1100, see [link to relevant documentation]. Figure 14 The object re-identification device 1400 includes an image acquisition module 1410 and an object recognition module 1420, specifically:

[0146] Image acquisition module 1410 is used to acquire the image to be recognized;

[0147] The object recognition module 1420 is used to input the image to be recognized into the target object re-recognition model to obtain the re-recognition result of the image to be recognized; wherein, the target object re-recognition model is obtained by training the training model according to the training method of the object re-recognition model according to any of the above exemplary embodiments.

[0148] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0149] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0150] An exemplary embodiment of this disclosure also provides an electronic device for the above-described method. This electronic device may be the aforementioned imaging device or a server. Generally, the electronic device includes at least a processor and a memory, the memory for storing executable instructions of the processor, and the processor configured to perform the above-described method by executing the executable instructions.

[0151] The following is based on Figure 15 Taking the mobile terminal 1500 as an example, the construction of the electronic device in this embodiment of the present disclosure will be described by way of example. Those skilled in the art will understand that, apart from components specifically designed for mobile purposes, Figure 15 The structure shown can also be applied to fixed-type devices. In other embodiments, the mobile terminal 1500 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. The interface connections between the components are only schematic and do not constitute a limitation on the structure of the mobile terminal 1500. In other embodiments, the mobile terminal may also adopt a similar design to... Figure 15 Different interface connection methods, or combinations of multiple interface connection methods.

[0152] like Figure 15As shown, the mobile terminal 1500 may specifically include: a processor 1501, a memory 1502, a bus 1503, a mobile communication module 1504, an antenna 1, a wireless communication module 1505, an antenna 2, a display screen 1506, a camera module 1507, an audio module 1508, a power module 1509, and a sensor module 1510.

[0153] Processor 1501 may include one or more processing units, such as: application processor (AP), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit), etc.

[0154] An encoder encodes (compresses) images or videos to reduce data size for easier storage or transmission. A decoder decodes (decompresses) the encoded data to restore the original image or video data. The mobile terminal 1500 can support one or more encoders and decoders, such as image formats like JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), and BMP (Bitmap), and video formats like MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0155] The processor 1501 can be connected to the memory 1502 or other components via the bus 1503.

[0156] The memory 1502 can be used to store executable program code, including instructions. The processor 1501 executes various functional applications and data processing of the mobile terminal 1500 by running the instructions stored in the memory 1502. The memory 1502 can also store application data, such as images, videos, and other files.

[0157] The communication functions of mobile terminal 1500 can be implemented through mobile communication module 1504, antenna 1, wireless communication module 1505, antenna 2, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 1504 can provide 3G, 4G, and 5G mobile communication solutions for mobile terminal 1500. Wireless communication module 1505 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication for mobile terminal 1500.

[0158] The display screen 1506 is used to implement display functions, such as displaying the user interface, images, videos, and displaying abnormal prompts. The camera module 1507 is used to implement shooting functions, such as capturing images and videos to acquire scene images. The audio module 1508 is used to implement audio functions, such as playing audio and capturing voice. The power module 1509 is used to implement power management functions, such as charging the battery, supplying power to the device, and monitoring battery status. The sensor module 1510 may include one or more sensors for implementing corresponding sensing and detection functions.

[0159] Furthermore, exemplary embodiments of this disclosure also provide a computer-readable storage medium storing a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0160] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0161] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0162] Furthermore, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0163] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A training method for an object re-identification model, characterized in that, include: Multiple occlusion images are obtained by occlusion processing of the original image, and each occlusion image corresponds to a different occlusion area; Each of the occluded images is sampled to obtain the target image; Using the original image and the target image as sample images, a pre-trained object re-identification model is trained; wherein, the pre-trained object re-identification model is obtained by training the model to be trained based on the original image; The step of sampling each of the occluded images to obtain the target image includes: Each of the occluded images of the original image is input into the pre-trained object re-identification model to obtain the ground truth probability corresponding to each occluded image; the ground truth probability is used to reflect the degree of influence of the occluded region contained in the occluded image on the prediction accuracy of the model; For each original image, the occluded images of the original image are sampled according to the ground truth probability to obtain the target image.

2. The method according to claim 1, characterized in that, The process of occluding the original image yields multiple occluded images, including: Multiple areas to be masked are determined in the original image by using a masking technique; The original image is occluded based on each of the regions to be occluded, so as to obtain the plurality of occluded images of the original image.

3. The method according to claim 2, characterized in that, The process of determining multiple regions to be masked in the original image using a masking technique includes: In response to an adjustment operation for the occlusion mask, the plurality of regions to be occluded are determined in the original image; In response to each adjustment operation, the occlusion mask size corresponding to the adjustment operation and the sliding step size corresponding to the occlusion mask are obtained.

4. The method according to claim 1, characterized in that, The step of sampling each occluded image of the original image according to the ground truth probability to obtain the target image includes: The occluded image corresponding to the ground truth probability of the target that is lower than the preset probability threshold is determined as the target image.

5. The method according to any one of claims 1 to 4, characterized in that, The step of training a pre-trained object re-identification model using the original image and the target image as sample images includes: The sample image is input into the pre-trained object re-identification model to obtain the model prediction result; A loss function is constructed based on the model prediction results and corresponding category labels of the sample images; The parameters of the pre-trained object re-identification model are adjusted using the loss function to obtain the target object re-identification model.

6. The method according to claim 5, characterized in that, The pre-trained object re-identification model includes a first residual network layer, an attention network layer, a second residual network layer, and a fully connected network layer. The attention network layer includes a channel attention network and a spatial attention network. The step of inputting the sample image into the pre-trained object re-identification model to obtain the model prediction result includes: The sample image is input into the first residual network for processing to obtain the first feature map; The channel attention network is used to extract features from the first feature map to obtain channel attention features, and the channel attention features are fused with the first feature map to obtain a second feature map; The second feature map is input into the spatial attention network to obtain a spatial attention feature map, and the spatial attention feature map is fused with the second feature map to obtain a third attention feature map; The third attention feature map is input into the second residual network layer for processing, and the processing result is used for prediction through the fully connected network layer to obtain the model prediction result.

7. The method according to claim 6, characterized in that, The channel attention network includes a first average pooling layer, a first max pooling layer, and a shared network layer; The step of extracting features from the first feature map using the channel attention network to obtain a channel attention feature map includes: The first feature map is aggregated using the first average pooling layer to obtain the first context features; The first feature map is aggregated using the first max pooling layer to obtain the second context features; The first context feature and the second context feature are input into the shared network layer to map the first context feature and the second context feature to the shared feature space based on the shared network layer; The output of the shared network layer is fused to obtain the channel attention feature map.

8. The method according to claim 6, characterized in that, The spatial attention network includes a second average pooling layer, a second max pooling layer, and a convolutional layer; The step of inputting the second feature map into the spatial attention network to obtain a spatial attention feature map includes: The second feature map is input into the second average pooling layer to obtain the third context feature; The second feature map is input into the second max pooling layer to obtain the fourth context feature; The third and fourth context features are concatenated and then input into the convolutional layer for convolution processing to obtain the spatial attention feature map.

9. An object re-identification method, characterized in that, include: Acquire the image to be recognized; The image to be identified is input into the target object re-identification model to obtain the re-identification result of the image to be identified; wherein, the target object re-identification model is obtained by training the model to be trained using the training method of the object re-identification model according to any one of claims 1 to 8.

10. A training device for an object re-identification model, characterized in that, include: An occlusion processing module is used to perform occlusion processing on the original image to obtain multiple occlusion images, each of which corresponds to a different occlusion area; An image sampling module is used to sample each of the occluded images to obtain a target image; The model training module is used to train a pre-trained object re-identification model using the original image and the target image as sample images; wherein the pre-trained object re-identification model is obtained by training the model to be trained based on the original image; The step of sampling each of the occluded images to obtain the target image includes: Each of the occluded images of the original image is input into the pre-trained object re-identification model to obtain the ground truth probability corresponding to each occluded image; the ground truth probability is used to reflect the degree of influence of the occluded region contained in the occluded image on the prediction accuracy of the model; For each original image, the occluded images of the original image are sampled according to the ground truth probability to obtain the target image.

11. An object re-identification device, characterized in that, include: The image acquisition module is used to acquire the image to be recognized; An object recognition module is used to input the image to be recognized into a target object re-recognition model to obtain the re-recognition result of the image to be recognized; wherein, the target object re-recognition model is obtained by training a training model according to the training method of the object re-recognition model according to any one of claims 1 to 8.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 9 by executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 9.

Images