Method and apparatus for explaining object re-identification model

Abstract

The application discloses an object re-identification model explanation method and device. A specific embodiment of the method comprises: inputting a to-be-identified image and a reference image in an image pair into a pre-trained re-identification model in sequence to determine the similarity between a to-be-identified object in the to-be-identified image and a reference object in the reference image; inputting the images in the image pair into a pre-trained explanation model in sequence, decomposing a feature tensor obtained by a second feature extraction network of the explanation model into a plurality of attribute attention maps corresponding to a plurality of attributes in a preset attribute set one by one; and obtaining a similarity component guided by each attribute in the preset attribute set based on the feature tensors obtained by a first feature extraction network and corresponding to the images in the image pair in sequence and the plurality of attribute attention maps corresponding to the images in the image pair. The application provides an object re-identification model explanation method, and improves the reliability and explainability of the object re-identification model.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an interpretation method and apparatus for an object re-identification model. Background Technology

[0002] People re-identification technology refers to finding individuals with the same identity from a large number of images captured by surveillance cameras, given an image of a person to be identified. Currently, people re-identification technology typically uses deep convolutional neural networks (DNNs) as image feature extractors and metric space learners. These DNNs are trained using large-scale labeled personnel image data and optimized using contrastive loss and cross-entropy loss functions. During the search phase, the trained DNN extracts image features, and Euclidean distance or cosine distance is used to calculate the similarity between image samples to achieve person image matching.

[0003] Deep learning models, especially deep convolutional neural networks, are black-box models that have achieved excellent accuracy in various computer vision tasks, such as image classification, object detection, semantic segmentation, and person re-identification. However, their internal principles and decision-making logic remain opaque to users. Summary of the Invention

[0004] This application provides an interpretation method and apparatus for an object re-identification model.

[0005] In a first aspect, embodiments of this application provide an interpretation method for an object re-identification model, comprising: sequentially inputting a to-be-identified image and a reference image from an image pair into a pre-trained re-identification model to determine the similarity between the to-be-identified object in the to-be-identified image and the reference object in the reference image; sequentially inputting the images from the image pair into a pre-trained interpretation model, decomposing the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set, wherein the second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network; based on the feature tensors obtained by the first feature extraction network that sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, obtaining a similarity component guided by each attribute in the preset attribute set to interpret the similarity, wherein the similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the to-be-identified object and the reference object.

[0006] In some embodiments, obtaining the similarity component guided by each attribute in a preset attribute set based on the feature tensors corresponding to the images in the image pair obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the images in the image pair, includes: obtaining multiple first feature tensors guided by attributes in the preset attribute set based on the feature tensors corresponding to the image to be identified obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the image to be identified; obtaining multiple second feature tensors guided by attributes in the preset attribute set based on the feature tensors corresponding to the reference image obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the reference image; and determining the similarity component corresponding to each attribute in the preset attribute set based on the first feature tensor and the second feature tensor corresponding to the attribute.

[0007] In some embodiments, the above-described method of sequentially inputting the image to be identified and the reference image in an image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image includes: sequentially inputting the images in the image pair into the pre-trained re-identification model, obtaining the feature tensors corresponding to the images in the image pair through a first feature extraction network; obtaining the feature vectors corresponding to the feature tensors of the images in the image pair; and determining the similarity between the object to be identified and the reference object based on the feature vectors corresponding to the images in the image pair.

[0008] In some embodiments, the re-identification model is trained as follows: A first training sample set is obtained, wherein the training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images; each training sample in the first training sample set is traversed, and a positive sample with the same identity label as the training sample and a negative sample with a different identity label are selected from the first training sample set, and the training sample, the corresponding positive sample, and the negative sample are combined into a training sample group; for each training sample group, the following operations are performed until a re-identification model is obtained: the training sample group is input into the initial re-identification model to obtain a feature vector group corresponding to the training sample group; the triplet loss value and the cross-entropy loss value are calculated based on the feature vector group, and the network parameters of the initial re-identification model are optimized based on the triplet loss value and the cross-entropy loss value.

[0009] In some embodiments, the above-mentioned calculation of triplet loss and cross-entropy loss based on feature vector sets includes: determining triplet loss based on the distance between each pair of feature vectors in the feature vector set; obtaining a classification probability vector representing the identity of the object corresponding to the feature vector in the feature vector set based on the feature vector in the feature vector set; and determining cross-entropy loss based on the classification probability vector corresponding to the feature vector set.

[0010] In some embodiments, the explanatory model is trained as follows: a second training sample set is obtained, wherein the training samples in the second training sample set include sample images and attribute label sequences representing the attributes included in the sample images; a pair of images to be trained, including two training samples, is selected from the second training sample set; the second feature extraction network of the initial explanatory model is initialized to the first feature extraction network in a pre-trained re-identification model, and the parameters of at least a portion of the network structure in the second feature extraction network that share parameters with the first feature extraction network are fixed; for each pair of images to be trained, the following operations are performed until an explanatory model is obtained: the pair of images to be trained is input into the pre-trained re-identification model to obtain the similarity of objects in the pair of images to be trained; the pair of images to be trained is input into the initial explanatory model after parameter sharing to obtain the similarity component guided by each attribute in the preset attribute set; a first loss value representing the loss between the sum of the similarity components and the similarity, and a second loss value representing the loss of the preset attribute prior knowledge corresponding to the preset attribute set are calculated, and the network parameters of the initial explanatory model are optimized based on the first loss value and the second loss value.

[0011] In some embodiments, the second loss value is determined as follows: based on the attribute label sequence of the training samples in the image pair to be trained, the same attributes and different attributes between the two training samples in the image pair are determined; the second loss value is determined based on the same attributes, different attributes, similarity components corresponding to attributes in the same attributes, and similarity components corresponding to attributes in different attributes.

[0012] In some embodiments, the preset attribute prior knowledge includes prior knowledge representing different attributes of interest; and the determination of the second loss value based on the same attribute, different attributes, similarity components corresponding to attributes in the same attribute, and similarity components corresponding to attributes in different attributes includes: determining a third loss value that, compared to the sum of similarity components corresponding to the same attribute, makes the explanatory model focus on the sum of similarity components corresponding to different attributes, and a fourth loss value that, compared to the similarity components corresponding to each attribute in the same attribute, makes the explanatory model focus on the similarity components corresponding to each attribute in different attributes; and determining the second loss value based on the third loss value and the fourth loss value.

[0013] In some embodiments, the parameters of the first and second feature extraction networks are shared for a predetermined number of feature extraction layers.

[0014] In some embodiments, the method further includes: displaying attribute attention maps and similarity components of the images in the image pair.

[0015] Secondly, embodiments of this application provide a method for verifying object re-identification results, including: interpreting the re-identification results obtained by the re-identification model, which represent that objects in different images are the same object, to obtain interpretation information, wherein the interpretation information interprets the similarity represented by the re-identification results through the similarity component guided by each attribute in a preset attribute set; and verifying the re-identification results through the interpretation information.

[0016] Thirdly, embodiments of this application provide an interpretation apparatus for an object re-identification model, comprising: a determining unit configured to sequentially input a to-be-identified image and a reference image from an image pair into a pre-trained re-identification model to determine the similarity between the to-be-identified object in the to-be-identified image and a reference object in the reference image; a decomposition unit configured to sequentially input images from the image pair into a pre-trained interpretation model to decompose the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set, wherein the second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network; and a obtaining unit configured to obtain a similarity component guided by each attribute in the preset attribute set based on the feature tensor obtained by the first feature extraction network that sequentially corresponds to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, to interpret the similarity, wherein the similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the to-be-identified object and the reference object.

[0017] In some embodiments, the obtaining unit is further configured to: obtain multiple first feature tensors guided by attributes in a preset attribute set based on the feature tensors corresponding to the image to be identified obtained by the first feature extraction network and multiple attribute attention maps corresponding to the image to be identified; obtain multiple second feature tensors guided by attributes in a preset attribute set based on the feature tensors corresponding to the reference image obtained by the first feature extraction network and multiple attribute attention maps corresponding to the reference image; and for each attribute in the preset attribute set, determine the similarity component corresponding to the attribute based on the first feature tensor and the second feature tensor corresponding to the attribute.

[0018] In some embodiments, the determining unit is further configured to: sequentially input the images in the image pair into a pre-trained re-identification model, obtain feature tensors corresponding to the images in the image pair through a first feature extraction network; obtain feature vectors corresponding to the feature tensors of the images in the image pair; and determine the similarity between the object to be identified and the reference object based on the feature vectors corresponding to the images in the image pair.

[0019] In some embodiments, the apparatus further includes: a first training unit configured to train a re-identification model based on the following manner: acquiring a first training sample set, wherein the training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images; traversing each training sample in the first training sample set, selecting a positive sample with the same identity label as the training sample and a negative sample with a different identity label from the first training sample set, and combining the training sample, the corresponding positive sample, and the negative sample into a training sample group; for each training sample group, performing the following operations until a re-identification model is obtained: inputting the training sample group into an initial re-identification model to obtain a feature vector group corresponding to the training sample group; calculating triplet loss and cross-entropy loss based on the feature vector group, and optimizing the network parameters of the initial re-identification model based on the triplet loss and cross-entropy loss.

[0020] In some embodiments, the first training unit is further configured to: determine the triplet loss value based on the distance between each pair of feature vectors in the feature vector group; obtain a classification probability vector representing the identity of the object corresponding to the feature vector in the feature vector group based on the feature vector in the feature vector group; and determine the cross-entropy loss value based on the classification probability vector corresponding to the feature vector group.

[0021] In some embodiments, the apparatus further includes: a second training unit configured to train an interpretation model based on the following: acquiring a second training sample set, wherein the training samples in the second training sample set include sample images and attribute label sequences representing the attributes included in the sample images; selecting a pair of images to be trained, including two training samples, from the second training sample set; initializing the second feature extraction network of the initial interpretation model to a first feature extraction network in a pre-trained re-identification model, and fixing the parameters of at least a portion of the network structure in the second feature extraction network that shares parameters with the first feature extraction network; for each pair of images to be trained, performing the following operations until an interpretation model is obtained: inputting the pair of images to be trained into the pre-trained re-identification model to obtain the similarity of objects in the pair of images to be trained; inputting the pair of images to be trained into the initial interpretation model after parameter sharing to obtain a similarity component guided by each attribute in a preset attribute set; calculating a first loss value representing the loss between the sum of the similarity components and the similarity, and a second loss value representing the loss of the preset attribute prior knowledge corresponding to the preset attribute set, and optimizing the network parameters of the initial interpretation model based on the first loss value and the second loss value.

[0022] In some embodiments, the second training unit is further configured to: determine the same and different attributes between two training samples in the image pair based on the attribute label sequence of the training samples in the image pair to be trained; and determine a second loss value based on the same attributes, different attributes, similarity components corresponding to attributes in the same attributes, and similarity components corresponding to attributes in different attributes.

[0023] In some embodiments, the preset attribute prior knowledge includes prior knowledge representing different attributes of interest; and the second training unit is further configured to: determine a third loss value relative to the sum of similarity components corresponding to the same attribute, such that the explanatory model focuses on the sum of similarity components corresponding to different attributes, and a fourth loss value relative to the similarity components corresponding to each attribute among the same attributes, such that the explanatory model focuses on the similarity components corresponding to each attribute among different attributes; and determine a second loss value based on the third loss value and the fourth loss value.

[0024] In some embodiments, the parameters of the first and second feature extraction networks are shared for a predetermined number of feature extraction layers.

[0025] In some embodiments, the above-described apparatus further includes a display unit configured to display an attribute attention map and a similarity component of the images in an image pair.

[0026] Fourthly, embodiments of this application provide a verification device for object re-identification results, including: an interpretation unit configured to interpret re-identification results obtained by the re-identification model, which represent objects in different images as the same object, through an interpretation model, to obtain interpretation information, wherein the interpretation information interprets the similarity represented by the re-identification results through a similarity component guided by each attribute in a preset attribute set; and a verification unit configured to verify the re-identification results through the interpretation information.

[0027] Fifthly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the method described in either the first or second aspect.

[0028] Sixthly, embodiments of this application provide an electronic device, including: one or more processors; and a storage device storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect or the second aspect.

[0029] The object re-identification model interpretation method and apparatus provided in this application embodiment determine the similarity between the object to be identified in the image to be identified and the reference image in the reference image by sequentially inputting the image to be identified and the reference image in the image to be identified into a pre-trained re-identification model. The images in the image pair are then sequentially input into a pre-trained interpretation model. The feature tensor obtained by the second feature extraction network of the interpretation model is decomposed into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set. The second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first and second feature extraction networks. Based on the feature tensors obtained by the first feature extraction network, which sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, a similarity component guided by each attribute in the preset attribute set is obtained to interpret the similarity. The similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object. This provides an interpretation method for the object re-identification model, improving the reliability and interpretability of the object re-identification model. Attached Figure Description

[0030] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0031] Figure 1 This is an exemplary system architecture diagram in which one embodiment of this application can be applied;

[0032] Figure 2 This is a flowchart of an embodiment of the interpretation method of the object re-identification model according to this application;

[0033] Figure 3 This is a schematic diagram of the attribute attention map generated based on the interpretation model of this application and the similarity components obtained by decomposition;

[0034] Figure 4 This is a schematic diagram of the positive and negative example images corresponding to each attribute according to the interpretation model of this application;

[0035] Figure 5 This is a schematic diagram illustrating an application scenario of the interpretation method of the object re-identification model according to this embodiment;

[0036] Figure 6 This is a flowchart of an embodiment of the training method for the object re-identification model according to this application;

[0037] Figure 7 This is a flowchart of one embodiment of the training method for the explanatory model according to this application;

[0038] Figure 8 This is a flowchart of an embodiment of the verification method for object re-identification results according to this application;

[0039] Figure 9 This is a structural diagram of an embodiment of the interpretation apparatus for the object re-identification model according to this application;

[0040] Figure 10 This is a structural diagram of one embodiment of the verification device for object re-identification results according to this application;

[0041] Figure 11 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of this application. Detailed Implementation

[0042] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] Figure 1 An exemplary architecture 100 of the interpretation method and apparatus for which the object re-identification model of this application can be applied is shown.

[0045] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The communication connections between terminal devices 101, 102, and 103 form a network topology. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0046] Terminal devices 101, 102, and 103 can be hardware or software that supports network connectivity for data interaction and processing. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as, for example, multiple software programs or software modules to provide distributed services, or as a single software program or software module. No specific limitations are imposed here.

[0047] Server 105 can be a server providing various services, such as acquiring images to be identified sent by users through terminal devices 101, 102, and 103, determining the similarity between the object to be identified in the image and a reference object in the reference image through a re-identification model, and interpreting the similarity determined by the re-identification model through an interpretation model. Optionally, the server can feed back the interpretation results obtained from the interpretation model to the terminal device. As an example, server 105 can be a cloud server.

[0048] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0049] It should also be noted that the object re-identification model interpretation method provided in the embodiments of this disclosure can be executed by a server, by a terminal device, or by a combination of both. Accordingly, the various parts (e.g., various units) of the object re-identification model interpretation apparatus can be entirely located in the server, entirely located in the terminal device, or separately located in the server and the terminal device.

[0050] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be included depending on implementation needs. When the electronic device on which the object re-identification model interpretation method runs does not require data transmission with other electronic devices, the system architecture may only include the electronic device (e.g., a server or terminal device) on which the object re-identification model interpretation method runs.

[0051] Continue to refer to Figure 2The flowchart 200 illustrates an embodiment of an interpretation method for an object re-identification model, including the following steps:

[0052] Step 201: Input the image to be identified and the reference image in the image pair into the pre-trained re-identification model in sequence to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image.

[0053] In this embodiment, the execution entity of the object re-identification model's interpretation method (e.g., Figure 1 The server can sequentially input the image to be identified and the reference image in the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image.

[0054] The object re-identification model represents the correspondence between the image to be identified and the reference image in an image pair, and their similarity. This correspondence can be obtained by training a neural network, including but not limited to deep convolutional neural networks (DCNNs) and residual neural networks (RNNs). In the object re-identification model, a deep convolutional neural network can be used as an image feature extractor and a metric space learner. The deep convolutional neural network is trained using large-scale image annotation data, and its parameters are optimized using contrastive loss functions and cross-entropy loss functions to obtain the trained object re-identification model. During the model usage phase, the trained deep neural network extracts image features and calculates the Euclidean distance or cosine distance between the feature vector representing the image to be identified and the feature vector representing the reference image, thus representing the similarity between the object to be identified in the image to be identified and the reference object in the reference image.

[0055] The object to be identified and the reference object can be objects of any type. For example, they can be people or animals.

[0056] The reference image can be any image from a preset image library. Image pairs are formed by combining the reference images from the preset image library with the image to be identified. A re-identification model is used to obtain the similarity score for each image pair, and these similarities are ranked. Based on the ranking results, the identification result of the object to be identified in the image to be identified is determined.

[0057] In some optional implementations of this embodiment, the execution entity can perform step 201 as follows:

[0058] First, the images in the image pair are sequentially input into a pre-trained re-identification model, and the feature tensors corresponding to the images in the image pair are obtained through the first feature extraction network.

[0059] In this implementation, the number of feature extraction layers and the network structure of each feature extraction layer in the first feature extraction network can be specifically set according to the recognition situation, and are not limited here. Through the first feature extraction network, feature tensors corresponding to the image to be recognized and the reference image can be obtained sequentially.

[0060] Second, obtain the eigenvectors corresponding to the feature tensors of the images in the image pair.

[0061] As an example, the aforementioned execution entity performs a global pooling operation on the feature tensors corresponding to the image to be identified and the reference image, respectively, to obtain feature vectors corresponding to the image to be identified and the reference image in turn.

[0062] Third, based on the feature vectors of the images corresponding to the image pair, determine the similarity between the object to be identified and the reference object.

[0063] As an example, the aforementioned execution entity can calculate the similarity between the object to be identified and the reference object using the following formula:

[0064] d q,d =D(f q f d )=||f q -f d ||

[0065] Among them, f q The representation corresponds to the feature vector of the image to be recognized, f d The representation corresponds to the feature vector of the reference image, and the L2 norm is calculated for the ||·|| representation.

[0066] Step 202: Input the images in the image pair into the pre-trained interpretation model in sequence, and decompose the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps that correspond one-to-one with multiple attributes in the preset attribute set.

[0067] In this embodiment, the execution entity can sequentially input the images in the image pair into a pre-trained interpretation model, and decompose the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps that correspond one-to-one with multiple attributes in the preset attribute set.

[0068] The interpretation model is used to determine the degree of attention the re-identification model pays to the attributes involved in the image to be identified and the reference image during image-pair-based information processing. Here, "attribute" refers to the attributes included in a preset attribute set. The preset attribute set includes various attributes related to the object and can be set based on prior knowledge of attributes from the re-identification technology. Taking a person as an example, the prior knowledge of attributes includes, but is not limited to, gender, hairstyle, clothing, and carried items.

[0069] In this embodiment, the interpretation model includes a second feature extraction network with the same structure as the first feature extraction network in the re-identification model, and at least some of the first and second feature extraction networks share parameters.

[0070] The position and proportion of the parameter-sharing network in the second feature extraction network can be flexibly set according to the actual situation, and are not limited here.

[0071] As an example, an attribute decomposition module is added to the end of the second feature extraction network. The attribute decomposition module contains a The network consists of a convolutional layer, a 1×1×M convolutional layer, and an exponential activation function. Here, c is the number of channels in the last convolutional layer of the second feature extraction network, M is the number of attributes included in the preset attribute set, and the exponential activation function is defined as follows:

[0072]

[0073] Where κ and τ are growth coefficients between (0, 1), and in this embodiment, they are set to... τ = 0.5.

[0074] The attribute decomposition module decomposes the feature tensors obtained by the second feature extraction network, which correspond to the image to be recognized and the reference image respectively, into M attribute attention maps. Each attribute attention map corresponds to one attribute in a preset attribute set.

[0075] In some optional implementations of this embodiment, the parameters of the first and second feature extraction networks are shared for the first preset number of feature extraction layers. The preset number can be flexibly set according to actual conditions and is not limited here.

[0076] Step 203: Based on the feature tensors obtained by the first feature extraction network that correspond sequentially to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, obtain the similarity component guided by each attribute in the preset attribute set to interpret the similarity.

[0077] In this embodiment, based on the feature tensors obtained by the first feature extraction network, which sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, a similarity component guided by each attribute in the preset attribute set is obtained to interpret the similarity. The similarity component is used to characterize the contribution value of the attribute corresponding to that similarity component when the re-identification model determines the similarity between the object to be identified and the reference object.

[0078] The explanation model mainly includes a second feature extraction network, an attribute decomposition module, and a network structure for operations between feature matrices and feature vectors. The second feature extraction network extracts image features to obtain a feature map; the attribute decomposition module decomposes the feature map based on multiple attributes from a preset attribute set to obtain attribute attention maps for each attribute. The network structure used for computation performs operations based on the attribute attention maps and the image's feature tensors to obtain the similarity component corresponding to each attribute.

[0079] As an example, for each attribute in the preset attribute set, the aforementioned execution entity determines the Euclidean distance or cosine distance between the attribute attention maps corresponding to the attribute and belonging to the image to be identified and the reference image as the similarity component corresponding to the attribute.

[0080] In some optional implementations of this embodiment, the execution entity can perform step 203 as follows:

[0081] First, based on the feature tensor corresponding to the image to be identified obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the image to be identified, multiple first feature tensors guided by attributes in the preset attribute set are obtained.

[0082] As an example, the feature tensor corresponding to the image to be identified obtained by the first feature extraction network is F. q The attention map corresponding to multiple attributes of the image to be recognized is as follows: Among them, A q Each element in F represents an attribute attention map of the image to be recognized. q respectively with A q Multiplying the attribute attention maps in the graphs yields multiple first feature tensors.

[0083] Second, based on the feature tensor corresponding to the reference image obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the reference image, multiple second feature tensors guided by attributes in the preset attribute set are obtained.

[0084] As an example, the feature tensor corresponding to the reference image obtained by the first feature extraction network is F. d The attention map corresponding to multiple attributes of the reference image is as follows: Among them, A d Each element in F represents an attribute attention map referenced to the image. d respectively with A d Multiplying the attribute attention maps in the graphs yields multiple second feature tensors.

[0085] Third, for each attribute in the preset attribute set, determine the similarity component corresponding to the attribute based on the first feature tensor and the second feature tensor corresponding to the attribute.

[0086] As an example, the aforementioned execution entity will use multiple first-feature tensors With multiple second feature tensors Perform global pooling to obtain multiple corresponding first feature vectors. and multiple second feature vectors

[0087] For multiple first feature vectors and multiple second feature vectors The aforementioned execution entity calculates the distance between feature vectors corresponding to the same attribute and determines the similarity component corresponding to that attribute.

[0088] In some optional implementations of this embodiment, the execution entity may further display the attribute attention map and similarity component of the image in the image pair. The attribute attention map of the image to be identified, the attribute attention map of the reference image, and the similarity component are based on attribute correspondence.

[0089] See Figure 3 The diagram 300 illustrates the attribute attention maps generated by the interpreter model and the similarity components obtained from the decomposition. It includes four image pairs: 301, 302, 303, and 304. For image pairs 301 and 302, the diagram shows the attribute attention maps for the more prevalent attributes (backpack, top, bottom, etc.), and the similarity components represented by 305 and 306, respectively. For image pairs 303 and 304, the diagram shows the attribute attention maps for the more prevalent attributes (hat, top, bottom, etc.), and the similarity components represented by 307 and 308, respectively. The similarity components can be represented using pie charts. Each pie chart represents the similarity component corresponding to one attribute.

[0090] The aforementioned executing entity can display positive and negative example images for each attribute. See also... Figure 4 The diagram 400 illustrates the explanatory model's attention to the positive and negative example images corresponding to each attribute. The positive example image represents the attention paid to the attribute corresponding to that positive example image, while the negative example image represents the attention paid to other attributes. It can be seen that for the positive example image, the explanatory model can accurately locate salient regions, while for the negative example image, there are no salient regions, and the attention is evenly distributed across all regions.

[0091] See also Figure 5 , Figure 5This is a schematic diagram 500 illustrating an application scenario of the interpretation method of the object re-identification model according to this embodiment. Figure 5 In the application scenario, user 501 sends an image 504 to be identified to server 503 via terminal device 502. The image to be identified includes an object to be identified. Server 503 acquires surveillance video 506 captured by monitoring device 505, and uses each frame of the surveillance video as a reference image 5061, forming an image pair with the image to be identified. For each image pair, firstly, server 503 sequentially inputs the image to be identified and the reference image in the image pair into a pre-trained re-identification model 507 to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image; then, it sequentially inputs the images in the image pair into a pre-trained interpretation model 508, and decomposes the feature tensor obtained by the second feature extraction network 5081 of the interpretation model into multiple attribute attention maps 509 corresponding one-to-one with multiple attributes in a preset attribute set through attribute decomposition module 5082. The second feature extraction network 5081 has the same structure as the first feature extraction network 5071 in the re-identification model 507, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network. Finally, based on the feature tensors obtained by the first feature extraction network that correspond sequentially to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, a similarity component 510 guided by each attribute in the preset attribute set is obtained to interpret the similarity. The similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object.

[0092] The method provided by the above embodiments of this disclosure determines the similarity between the object to be identified in the image to be identified and the reference object in the reference image by sequentially inputting the image to be identified and the reference image in the image to be identified into a pre-trained re-identification model; the images in the image pair are sequentially input into a pre-trained interpretation model, and the feature tensor obtained by the second feature extraction network of the interpretation model is decomposed into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set, wherein the second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network; based on the feature tensors obtained by the first feature extraction network that sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, a similarity component guided by each attribute in the preset attribute set is obtained to interpret the similarity, wherein the similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object, thereby providing an interpretation method for the object re-identification model and improving the reliability and interpretability of the object re-identification model.

[0093] Continue to refer to Figure 6 The illustration shows a schematic flow 600 of an embodiment of a training method for an object re-identification model according to this application, including the following steps:

[0094] Step 601: Obtain the first training sample set.

[0095] In this embodiment, the execution entity of the training method for the object re-identification model (e.g., Figure 1 The server in the first training sample set can obtain the first training sample set from an electronic device communicatively connected to the aforementioned execution entity via a wired or wireless connection, or from a local device. The training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images.

[0096] In this embodiment, the first training sample set includes training samples with the same objects as those in other training samples, and also includes training samples with different objects from those in other training samples.

[0097] Step 602: Iterate through each training sample in the first training sample set, select a positive sample with the same identity label as the training sample and a negative sample with a different identity label from the first training sample set, and combine the training sample, the corresponding positive sample and the negative sample into a training sample group.

[0098] In this embodiment, the execution entity selects a sample image and its identity tag as the reference sample I. a Randomly select a sample from the first training sample set that is similar to the baseline sample I. a Sample images with the same identity label are used as positive samples I p Randomly select a sample from the first training sample set that is similar to the baseline sample I. a Images with different identity labels are used as negative sample images I. n .Will a I p I n >Form triples as training sample groups.

[0099] Step 603: For each training sample group, perform the following operations until a re-identification model is obtained:

[0100] Step 6031: Input the training sample group into the initial re-identification model to obtain the feature vector group corresponding to the training sample group.

[0101] In this embodiment, the aforementioned execution entity can use the training sample group a I p I n Input the initial re-identification model to obtain the training sample group. <I​​a I p I n The corresponding feature vector group <f a ,f p f n >

[0102] Step 6032: Calculate the triplet loss value and cross-entropy loss value based on the feature vector set, and optimize the network parameters of the initial re-identification model based on the triplet loss value and cross-entropy loss value.

[0103] In this embodiment, the aforementioned execution entity can calculate the triplet loss value and cross-entropy loss value based on the feature vector set, and optimize the network parameters of the initial re-identification model using the stochastic gradient descent algorithm and the backpropagation algorithm based on the triplet loss value and cross-entropy loss value.

[0104] In some optional implementations of this embodiment, the execution entity can perform step 6032 in the following manner:

[0105] First, determine the triplet loss value based on the distance between each pair of feature vectors in the feature vector group.

[0106] As an example, the aforementioned executing entity can determine the triplet loss value using the following formula:

[0107] L t =max(0,||f a -f n ||-||f a -f p ||-m)+α||f a -f n ||

[0108] Where α is the weighting coefficient, m is the marginal coefficient, and ||·|| is the L2 norm of the vector.

[0109] Second, based on the feature vectors in the feature vector group, obtain the classification probability vector that represents the identity of the object corresponding to the feature vector in the feature vector group.

[0110] As an example, the aforementioned execution entity can input the feature vector f into a fully connected layer w(·), and then obtain a 1×Y-dimensional classification probability vector w(f) through a softmax operation, where Y is the number of all identity labels in the first training sample set.

[0111] Third, determine the cross-entropy loss value based on the classification probability vector corresponding to the feature vector group.

[0112] As an example, the aforementioned execution entity can determine the cross-entropy loss value using the following formula:

[0113]

[0114] Here, y represents the identity label.

[0115] As can be seen from this embodiment, with Figure 2 Compared with the corresponding embodiments, the training method flow 600 of the object re-identification model in this embodiment specifically illustrates the training process of the object re-identification model, provides a training method for the object re-identification model, and improves the accuracy of the object re-identification model.

[0116] Continue to refer to Figure 7 The illustration shows a schematic flow 700 of an embodiment of a training method for an interpretable model according to this application, including the following steps:

[0117] Step 701: Obtain the second training sample set.

[0118] In this embodiment, the entity that executes the training method of the explanation model (e.g., Figure 1 The server in the system can obtain the second training sample set from an electronic device communicatively connected to the aforementioned execution entity via a wired or wireless connection, or from its local storage. The training samples in the second training sample set include sample images and sequences of attribute labels representing the attributes included in the sample images.

[0119] As an example, the attribute label sequence consists of 0s and 1s. For each attribute in the preset attribute set, when the value in the attribute label sequence is 1, it indicates that the sample image includes the information represented by the attribute; when the value in the attribute label sequence is 0, it indicates that the sample image does not include the information represented by the attribute.

[0120] Step 702: Select a pair of images to be trained, which includes two training samples, from the second training sample set.

[0121] As an example, the aforementioned execution entity can randomly select two training samples from the second training sample set to form a pair of images to be trained. The images in the same pair of images to be trained can represent the same object or different objects.

[0122] Step 703: Initialize the second feature extraction network of the initial interpretation model to the first feature extraction network in the pre-trained re-identification model, and fix the parameters of at least a portion of the network structure in the second feature extraction network that share parameters with the first feature extraction network.

[0123] In this embodiment, the aforementioned execution entity fixes the network parameters to ensure that the network parameters are not updated during model training. As an example, the execution entity can fix the parameters of the first preset number of feature extraction layers in the second feature extraction network after parameter sharing, and randomly initialize other feature extraction layers in the second feature extraction network.

[0124] Step 704: For each pair of images to be trained, perform the following operations until an interpretive model is obtained:

[0125] Step 7041: Input the image pair to be trained into the pre-trained re-recognition model to obtain the similarity of objects in the image pair to be trained.

[0126] In this embodiment, the aforementioned execution entity can be executed in a manner similar to step 201, which will not be described in detail here.

[0127] Step 7042: The initial interpretation model after sharing the input parameters of the image to be trained is used to obtain the similarity component guided by each attribute in the preset attribute set.

[0128] In this embodiment, the aforementioned execution entity can be executed in a manner similar to steps 202 and 203, which will not be described in detail here.

[0129] Step 7043: Calculate a first loss value representing the loss between the sum of similarity components and the similarity, and a second loss value representing the loss of prior knowledge of preset attributes corresponding to the preset attribute set, and optimize the network parameters of the initial explanation model based on the first loss value and the second loss value.

[0130] In this embodiment, the preset attribute prior knowledge can be prior knowledge considered according to the actual situation, including but not limited to the number of attribute categories in the preset attribute set and the attention of the interpretation model to each attribute.

[0131] As an example, the first loss value can be calculated using the following formula:

[0132]

[0133]

[0134] Where, d i，j The feature vector pairs (I) representing the image pairs to be trained are represented by the corresponding feature vector pairs. i I j The similarity between ).

[0135] In some optional implementations of this embodiment, the execution entity can perform step 7043 as follows:

[0136] First, based on the attribute label sequence of the training samples in the image pair to be trained, determine the common and different attributes between the two training samples in the image pair.

[0137] In this implementation, the execution entity can determine the same and different attributes between two training samples based on the XOR operation of the attribute label sequence of the training samples in the image pair to be trained.

[0138] Second, determine the second loss value based on the similarity components corresponding to the same attribute, different attributes, attributes within the same attribute, and attributes within different attributes.

[0139] Specifically, the pre-defined attribute prior knowledge includes prior knowledge representing different attributes of interest. The aforementioned execution entity can determine a third loss value, which, compared to the sum of similarity components corresponding to the same attribute, makes the explanatory model focus on the sum of similarity components corresponding to different attributes, and a fourth loss value, which, compared to the similarity components corresponding to each attribute within the same attribute, makes the explanatory model focus on the similarity components corresponding to each attribute within different attributes; based on the third and fourth loss values, a second loss value is determined.

[0140] As an example, the aforementioned executing entity can determine the third loss value using the following formula:

[0141]

[0142] The fourth loss value is determined using the following formula:

[0143]

[0144]

[0145] Where v is the growth factor in the interval (0, 1), and M e M is the number of distinct attributes in an image pair. c This represents the number of identical attributes in the image.

[0146] As can be seen from this embodiment, with Figure 2 Compared with the corresponding embodiments, the flowchart 700 of the training method for the explanatory model in this embodiment specifically illustrates the training process of the explanatory model, provides a training method for the explanatory model, and improves the accuracy of the explanatory model.

[0147] Continue to refer to Figure 8 The illustration shows a schematic flow 800 of an embodiment of the verification method for object re-identification results according to this application, including the following steps:

[0148] Step 801: By interpreting the re-identification results obtained by the re-identification model, which represent objects in different images as the same object, interpretation information is obtained.

[0149] In this embodiment, the execution entity of the verification method for object re-identification results (e.g.) Figure 1 The server in the middle interprets the re-identification results obtained by the re-identification model, which represent objects in different images as the same object, and obtains interpretation information.

[0150] The explanatory information explains the similarity represented by the re-identification result through the similarity component guided by each attribute in the preset attribute set. The information processing procedures of the explanatory model and the object re-identification model can be performed in the manner described in Embodiment 200 above, and will not be repeated here.

[0151] Step 802: Verify the re-identification result by interpreting the information.

[0152] In this embodiment, the aforementioned execution entity can verify the re-identification result by interpreting the information.

[0153] As an example, the aforementioned executing entity can first determine the proportion of the similarity component corresponding to each attribute in the explanatory information to the total similarity, and then determine the rationality of the proportion information in order to verify the re-identification result.

[0154] The method provided in the above embodiments of this disclosure interprets the re-identification results obtained by the re-identification model, which represent objects in different images as the same object, to obtain interpretation information. The re-identification results are then verified using the interpretation information, thereby providing a method for verifying re-identification results based on interpretation information, which improves the reliability of the re-identification results.

[0155] Continue to refer to Figure 9 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an interpretation device for an object re-identification model, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0156] like Figure 9As shown, the interpretation device for the object re-identification model includes: a determination unit 901, configured to sequentially input the image to be identified and the reference image in the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image; a decomposition unit 902, configured to sequentially input the images in the image pair into the pre-trained interpretation model to decompose the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set, wherein the second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network; and a obtaining unit 903, configured to obtain a similarity component guided by each attribute in the preset attribute set based on the feature tensor obtained by the first feature extraction network that sequentially corresponds to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, to interpret the similarity, wherein the similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object.

[0157] In some embodiments, the obtaining unit 903 is further configured to: obtain multiple first feature tensors guided by attributes in a preset attribute set based on the feature tensors corresponding to the image to be identified obtained by the first feature extraction network and multiple attribute attention maps corresponding to the image to be identified; obtain multiple second feature tensors guided by attributes in a preset attribute set based on the feature tensors corresponding to the reference image obtained by the first feature extraction network and multiple attribute attention maps corresponding to the reference image; and for each attribute in the preset attribute set, determine the similarity component corresponding to the attribute based on the first feature tensor and the second feature tensor corresponding to the attribute.

[0158] In some embodiments, the determining unit 901 is further configured to: sequentially input the images in the image pair into a pre-trained re-identification model, obtain the feature tensor corresponding to the image in the image pair through a first feature extraction network; obtain the feature vector corresponding to the feature tensor of the image in the image pair; and determine the similarity between the object to be identified and the reference object based on the feature vector corresponding to the image in the image pair.

[0159] In some embodiments, the above apparatus further includes: a first training unit (not shown in the figure), configured to train a re-identification model based on the following manner: acquiring a first training sample set, wherein the training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images; traversing each training sample in the first training sample set, selecting a positive sample with the same identity label as the training sample and a negative sample with a different identity label from the first training sample set, and combining the training sample, the corresponding positive sample, and the negative sample into a training sample group; for each training sample group, performing the following operations until a re-identification model is obtained: inputting the training sample group into an initial re-identification model to obtain a feature vector group corresponding to the training sample group; calculating the triplet loss value and the cross-entropy loss value based on the feature vector group, and optimizing the network parameters of the initial re-identification model based on the triplet loss value and the cross-entropy loss value.

[0160] In some embodiments, the first training unit (not shown in the figure) is further configured to: determine the triplet loss value based on the distance between each pair of feature vectors in the feature vector group; obtain the classification probability vector representing the identity of the object corresponding to the feature vector in the feature vector group based on the feature vector in the feature vector group; and determine the cross-entropy loss value based on the classification probability vector corresponding to the feature vector group.

[0161] In some embodiments, the apparatus further includes: a second training unit (not shown in the figures), configured to train an interpretation model based on the following: acquiring a second training sample set, wherein the training samples in the second training sample set include sample images and attribute label sequences representing the attributes included in the sample images; selecting a pair of images to be trained, including two training samples, from the second training sample set; initializing the second feature extraction network of the initial interpretation model to a first feature extraction network in a pre-trained re-identification model, and fixing the parameters of at least a portion of the network structure in the second feature extraction network that shares parameters with the first feature extraction network; for each pair of images to be trained, performing the following operations until an interpretation model is obtained: inputting the pair of images to be trained into the pre-trained re-identification model to obtain the similarity of objects in the pair of images to be trained; inputting the pair of images to be trained into the initial interpretation model after parameter sharing to obtain a similarity component guided by each attribute in a preset attribute set; calculating a first loss value representing the loss between the sum of the similarity components and the similarity, and a second loss value representing the loss of the preset attribute prior knowledge corresponding to the preset attribute set, and optimizing the network parameters of the initial interpretation model based on the first loss value and the second loss value.

[0162] In some embodiments, the second training unit (not shown in the figure) is further configured to: determine the same and different attributes between two training samples in the image pair based on the attribute label sequence of the training samples in the image pair to be trained; and determine a second loss value based on the same attributes, different attributes, similarity components corresponding to attributes in the same attributes, and similarity components corresponding to attributes in different attributes.

[0163] In some embodiments, the preset attribute prior knowledge includes prior knowledge representing different attributes of interest; and the second training unit (not shown in the figure) is further configured to: determine a third loss value relative to the sum of similarity components corresponding to the same attribute, such that the explanatory model focuses on the sum of similarity components corresponding to different attributes, and a fourth loss value relative to the similarity components corresponding to each attribute among the same attributes, such that the explanatory model focuses on the similarity components corresponding to each attribute among different attributes; and determine a second loss value based on the third loss value and the fourth loss value.

[0164] In some embodiments, the parameters of the first and second feature extraction networks are shared for a predetermined number of feature extraction layers.

[0165] In some embodiments, the above-described apparatus further includes a display unit (not shown) configured to display an attribute attention map and a similarity component of the images in the image pair.

[0166] In this embodiment, the determination unit in the interpretation device of the object re-identification model sequentially inputs the image to be identified and the reference image in the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image. The decomposition unit sequentially inputs the images in the image pair into the pre-trained interpretation model and decomposes the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps that correspond one-to-one with multiple attributes in a preset attribute set. The second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network. The obtaining unit obtains the similarity component guided by each attribute in the preset attribute set based on the feature tensor obtained by the first feature extraction network that corresponds sequentially to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, to interpret the similarity. This provides an interpretation device for the object re-identification model, improving the reliability and interpretability of the object re-identification model.

[0167] Continue to refer to Figure 10 As a response to the above Figure 8 The implementation of the method shown herein provides an embodiment of a verification device for object re-identification results, which is similar to... Figure 8Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0168] like Figure 10 As shown, the verification device for object re-identification results includes: an interpretation unit 1001, configured to interpret the re-identification results obtained by the re-identification model, which represent that objects in different images are the same object, through an interpretation model, to obtain interpretation information, wherein the interpretation information interprets the similarity represented by the re-identification results through the similarity component guided by each attribute in a preset attribute set; and a verification unit 1002, configured to verify the re-identification results through the interpretation information.

[0169] The interpretation unit in the apparatus provided in the above embodiments of this disclosure interprets the re-identification results obtained by the re-identification model, which represent that objects in different images are the same object, through the interpretation model, and obtains interpretation information; the verification unit verifies the re-identification results through the interpretation information, thereby providing an apparatus for verifying re-identification results based on the interpretation information of the re-identification results, which improves the reliability of the re-identification results.

[0170] The following is for reference. Figure 11 It illustrates a device suitable for implementing embodiments of this application (e.g., Figure 1 The diagram shows the structure of the computer system 1100 of the devices 101, 102, 103, and 105. Figure 11 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0171] like Figure 11 As shown, the computer system 1100 includes a processor (e.g., CPU, Central Processing Unit) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from storage portion 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for the operation of the system 1100. The processor 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0172] The following components are connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0173] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by processor 1101, it performs the functions defined in the methods of this application.

[0174] It should be noted that the computer-readable medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0175] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the client computer, partially on the client computer, as a standalone software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the client computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0177] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, they can be described as: a processor including a determining unit, a decomposition unit, and an obtaining unit. The name of the obtaining unit does not necessarily limit the unit itself; for example, the determining unit can also be described as "a unit that, based on feature tensors obtained from a first feature extraction network that sequentially correspond to images in an image pair, and multiple attribute attention maps corresponding to images in the image pair, obtains a similarity component guided by each attribute in a preset attribute set to interpret the similarity."

[0178] In another respect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist alone and not assembled into the device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the device, cause the computer device to: sequentially input the image to be identified and the reference image in the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image; sequentially input the images in the image pair into a pre-trained interpretation model, decomposing the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps corresponding one-to-one with multiple attributes in a preset attribute set, wherein the second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network; based on the feature tensors obtained by the first feature extraction network that sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, obtain a similarity component guided by each attribute in the preset attribute set to interpret the similarity, wherein the similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object. The computer device also enables the following: by interpreting the re-identification results obtained by the re-identification model, which represent objects in different images as the same object, to obtain interpretation information. The interpretation information interprets the similarity represented by the re-identification results through the similarity component guided by each attribute in the preset attribute set; and verifies the re-identification results through the interpretation information.

[0179] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An interpretation method for an object re-identification model, comprising: The image to be identified and the reference image in the image pair are sequentially input into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image. The images in the image pair are sequentially input into a pre-trained interpretation model. The feature tensor obtained by the second feature extraction network of the interpretation model is decomposed into multiple attribute attention maps that correspond one-to-one with multiple attributes in a preset attribute set. The second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network. Based on the feature tensors obtained by the first feature extraction network, which sequentially correspond to the images in the image pair, and the multiple attribute attention maps corresponding to the images in the image pair, a similarity component guided by each attribute in the preset attribute set is obtained to interpret the similarity. The similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object.

2. The method according to claim 1, wherein, The step of obtaining a similarity component guided by each attribute in the preset attribute set based on the feature tensors obtained from the first feature extraction network, which sequentially correspond to the images in the image pair, and multiple attribute attention maps corresponding to the images in the image pair, includes: Based on the feature tensor corresponding to the image to be identified obtained by the first feature extraction network, and the multiple attribute attention maps corresponding to the image to be identified, multiple attribute-guided first feature tensors in the preset attribute set are obtained. Based on the feature tensor corresponding to the reference image obtained by the first feature extraction network, and multiple attribute attention maps corresponding to the reference image, multiple attribute-guided second feature tensors in the preset attribute set are obtained. For each attribute in the preset attribute set, the similarity component corresponding to the attribute is determined based on the first feature tensor and the second feature tensor corresponding to the attribute.

3. The method according to claim 1, wherein, The step of sequentially inputting the image to be identified and the reference image from the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image includes: The images in the image pair are sequentially input into a pre-trained re-identification model, and the feature tensors corresponding to the images in the image pair are obtained through the first feature extraction network. Obtain the feature vector corresponding to the feature tensor of the image in the image pair; The similarity between the object to be identified and the reference object is determined based on the feature vectors corresponding to the images in the image pair.

4. The method according to claim 1 or 3, wherein, The re-identification model is trained in the following manner: Obtain a first training sample set, wherein the training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images; Iterate through each training sample in the first training sample set, select a positive sample with the same identity label as the training sample and a negative sample with a different identity label from the first training sample set, and combine the training sample, the corresponding positive sample and the negative sample into a training sample group; For each training sample group, the following operations are performed until the re-identification model is obtained: input the training sample group into the initial re-identification model to obtain the feature vector group corresponding to the training sample group; calculate the triplet loss value and the cross-entropy loss value based on the feature vector group, and optimize the network parameters of the initial re-identification model according to the triplet loss value and the cross-entropy loss value.

5. The method according to claim 4, wherein, The calculation of triplet loss and cross-entropy loss based on the feature vector set includes: The triplet loss value is determined based on the distance between each pair of feature vectors in the feature vector group; Based on the feature vectors in the feature vector group, a classification probability vector representing the identity of the object corresponding to the feature vector in the feature vector group is obtained; The cross-entropy loss value is determined based on the classification probability vector corresponding to the feature vector group.

6. The method according to claim 1, wherein, The explanatory model was trained in the following manner: Obtain a second training sample set, wherein the training samples in the second training sample set include sample images and attribute label sequences representing the attributes included in the sample images; In the second training sample set, a pair of images to be trained, including two training samples, is selected; The second feature extraction network of the initial interpretation model is initialized as the first feature extraction network in the pre-trained re-identification model, and the parameters of at least a portion of the network structure in the second feature extraction network that share parameters with the first feature extraction network are fixed. For each pair of images to be trained, the following operations are performed until the interpretation model is obtained: input the pair of images to be trained into a pre-trained re-identification model to obtain the similarity of objects in the pair of images to be trained; input the pair of images to be trained into an initial interpretation model after parameter sharing to obtain the similarity component guided by each attribute in the preset attribute set; calculate a first loss value representing the loss between the sum of similarity components and the similarity, and a second loss value representing the loss of the preset attribute prior knowledge corresponding to the preset attribute set; and optimize the network parameters of the initial interpretation model based on the first loss value and the second loss value.

7. The method according to claim 6, wherein, The second loss value is determined as follows: Based on the attribute label sequence of the training samples in the image pair to be trained, determine the same and different attributes between the two training samples in the image pair; The second loss value is determined based on the same attribute, the different attribute, the similarity component corresponding to the attribute in the same attribute, and the similarity component corresponding to the attribute in the different attribute.

8. The method according to claim 7, wherein, The preset attribute prior knowledge includes prior knowledge that represents the different attributes of interest; as well as Determining the second loss value based on the same attribute, the different attribute, the similarity component corresponding to the attribute in the same attribute, and the similarity component corresponding to the attribute in the different attribute includes: A third loss value is determined, which is the sum of similarity components corresponding to the same attributes, such that the explanatory model focuses on the sum of similarity components corresponding to the different attributes; and a fourth loss value is determined, which is the sum of similarity components corresponding to each attribute among the same attributes, such that the explanatory model focuses on the similarity components corresponding to each attribute among the different attributes. The second loss value is determined based on the third loss value and the fourth loss value.

9. The method according to claim 1, wherein, For the first feature extraction network and the second feature extraction network, the parameters of the first preset number of feature extraction layers are shared.

10. The method according to claim 1, wherein, Also includes: Display the attribute attention map and similarity component of the images in the image pair.

11. A verification method for object re-identification results, comprising: The interpretation model is used to interpret the re-identification results obtained by the re-identification model, which represent that objects in different images are the same object, to obtain interpretation information. The interpretation information interprets the similarity represented by the re-identification results through the similarity component guided by each attribute in the preset attribute set. The information processing of the interpretation model and the re-identification model is performed in accordance with any one of claims 1-10. The re-identification result is verified using the explanatory information.

12. An interpretation apparatus for an object re-identification model, comprising: The determining unit is configured to sequentially input the image to be identified and the reference image in the image pair into a pre-trained re-identification model to determine the similarity between the object to be identified in the image to be identified and the reference object in the reference image; The decomposition unit is configured to sequentially input the images in the image pair into a pre-trained interpretation model, and decompose the feature tensor obtained by the second feature extraction network of the interpretation model into multiple attribute attention maps that correspond one-to-one with multiple attributes in a preset attribute set. The second feature extraction network has the same structure as the first feature extraction network in the re-identification model, and at least some network parameters are shared between the first feature extraction network and the second feature extraction network. The obtaining unit is configured to obtain a similarity component guided by each attribute in the preset attribute set based on the feature tensors obtained by the first feature extraction network, which sequentially correspond to the images in the image pair, and multiple attribute attention maps corresponding to the images in the image pair, in order to interpret the similarity. The similarity component is used to characterize the contribution value of the attribute corresponding to the similarity component when the re-identification model determines the similarity between the object to be identified and the reference object.

13. The apparatus according to claim 12, wherein, The obtaining unit is further configured as follows: Based on the feature tensor corresponding to the image to be identified obtained by the first feature extraction network, and multiple attribute attention maps corresponding to the image to be identified, multiple attribute-guided first feature tensors in the preset attribute set are obtained; based on the feature tensor corresponding to the reference image obtained by the first feature extraction network, and multiple attribute attention maps corresponding to the reference image, multiple attribute-guided second feature tensors in the preset attribute set are obtained; for each attribute in the preset attribute set, the similarity component corresponding to the attribute is determined based on the first feature tensor and the second feature tensor corresponding to the attribute.

14. The apparatus according to claim 12, wherein, The determining unit is further configured to: The images in the image pair are sequentially input into a pre-trained re-identification model, and the feature tensors corresponding to the images in the image pair are obtained through the first feature extraction network. Obtain the feature vector corresponding to the feature tensor of the image in the image pair; determine the similarity between the object to be identified and the reference object based on the feature vector corresponding to the image in the image pair.

15. The apparatus according to claim 12 or 14, wherein, It also includes: a first training unit, configured to train the re-identification model in the following manner: Obtain a first training sample set, wherein the training samples in the first training sample set include sample images and identity labels representing the identities of objects in the sample images; traverse each training sample in the first training sample set, select a positive sample with the same identity label as the training sample and a negative sample with a different identity label from the first training sample set, and combine the training sample, the corresponding positive sample, and the negative sample into a training sample group; for each training sample group, perform the following operations until the re-identification model is obtained: input the training sample group into the initial re-identification model to obtain a feature vector group corresponding to the training sample group; calculate the triplet loss value and the cross-entropy loss value based on the feature vector group, and optimize the network parameters of the initial re-identification model according to the triplet loss value and the cross-entropy loss value.

16. The apparatus according to claim 15, wherein, The first training unit is further configured as follows: The triplet loss value is determined based on the distance between each pair of feature vectors in the feature vector group; a classification probability vector representing the identity of the object corresponding to the feature vector in the feature vector group is obtained based on the feature vector in the feature vector group; and the cross-entropy loss value is determined based on the classification probability vector corresponding to the feature vector group.

17. The apparatus according to claim 12, wherein, Also includes: The second training unit is configured to train the explanatory model in the following manner: Obtain a second training sample set, wherein the training samples in the second training sample set include sample images and attribute label sequences representing the attributes included in the sample images; select a pair of images to be trained, including two training samples, from the second training sample set; initialize the second feature extraction network of the initial interpretation model to the first feature extraction network in the pre-trained re-identification model, and fix the parameters of at least a portion of the network structure in the second feature extraction network that share parameters with the first feature extraction network; for each pair of images to be trained, perform the following operations until the interpretation model is obtained: input the pair of images to be trained into the pre-trained re-identification model to obtain the similarity of objects in the pair of images to be trained; input the pair of images to be trained into the initial interpretation model after parameter sharing to obtain the similarity component guided by each attribute in the preset attribute set; Calculate a first loss value representing the loss between the sum of similarity components and the similarity, and a second loss value representing the loss of prior knowledge of preset attributes corresponding to the preset attribute set. Based on the first loss value and the second loss value, optimize the network parameters of the initial explanation model.

18. The apparatus according to claim 17, wherein, The second training unit is further configured as follows: Based on the attribute label sequence of the training samples in the image pair to be trained, determine the common attributes and different attributes between the two training samples in the image pair; based on the common attributes, the different attributes, the similarity components corresponding to the attributes in the common attributes, and the similarity components corresponding to the attributes in the different attributes, determine the second loss value.

19. The apparatus according to claim 18, wherein, The preset attribute prior knowledge includes prior knowledge representing the different attributes of interest; and The second training unit is further configured as follows: A third loss value is determined, which is the sum of similarity components corresponding to the same attributes, such that the explanatory model focuses on the sum of similarity components corresponding to the different attributes; and a fourth loss value is determined, which is the sum of similarity components corresponding to each attribute among the same attributes, such that the explanatory model focuses on the similarity components corresponding to each attribute among the different attributes. The second loss value is determined based on the third loss value and the fourth loss value.

20. The apparatus according to claim 12, wherein, For the first feature extraction network and the second feature extraction network, the parameters of the first preset number of feature extraction layers are shared.

21. The apparatus according to claim 12, wherein, Also includes: The display unit is configured to display the attribute attention map and similarity component of the images in the image pair.

22. A verification device for object re-identification results, comprising: An interpretation unit is configured to interpret the re-identification results obtained by the re-identification model, which represent objects in different images as the same object, through an interpretation model, to obtain interpretation information. The interpretation information interprets the similarity represented by the re-identification results through a similarity component guided by each attribute in a preset attribute set. The information processing of the interpretation model and the re-identification model is performed in accordance with any one of claims 12-21. The verification unit is configured to verify the re-identification result using the explanation information.

23. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-11.

24. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-11.

Images