Object re-identification method and device, electronic equipment and storage medium

Abstract

This disclosure relates to an object re-identification method, apparatus, electronic device, and storage medium. It involves determining an image to be identified, including a target object, and an image set including candidate images, where each candidate image includes at least one object. The image to be identified and the image set are input into a re-identification network to obtain target candidate images whose included objects match the target object. The re-identification network is trained in two stages: the first stage is based on sample images and corresponding first-category labels; the second stage is based on sample images, corresponding pseudo-labels, and the first-category label. The pseudo-label for each sample image is determined by the re-identification network after the first training stage. This disclosure ensures the performance of the re-identification network and improves the accuracy of the recognition results through two-stage training.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an object re-identification method and apparatus, electronic device and storage medium. Background Technology

[0002] Re-identification technology is widely used in various projects, such as the re-identification of people, vehicles, and objects. In real-world open-world applications, new situations may arise at any time, resulting in previously unseen data. Traditional re-identification algorithms require a large number of labeled samples for training, and when the dataset shifts or the domain shifts, new data or samples from new domains need to be re-labeled, which consumes a lot of human and material resources. At the same time, unsupervised re-identification methods in related technologies often suffer from low accuracy due to factors such as the scene. Summary of the Invention

[0003] This disclosure proposes an object re-identification method, apparatus, electronic device, and storage medium, aiming to improve the accuracy of re-identification results through a re-identification model obtained through unsupervised training.

[0004] According to a first aspect of this disclosure, an object re-identification method is provided, comprising:

[0005] Identify the image to be recognized, which includes the target object;

[0006] Determine an image set comprising at least one candidate image, each candidate image comprising an object;

[0007] The image to be identified and the image set are input into the re-identification network to obtain the re-identification result. If there is a target candidate image in the image set, the re-identification result includes the target candidate image, and the object included in the target candidate image matches the target object.

[0008] The re-identification network is trained in two stages. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on the at least one sample image, a pseudo label for each sample image, and a first category label. The pseudo label for each sample image is determined based on the re-identification network after the first stage training process is completed. The first category label represents the category of the corresponding image.

[0009] In one possible implementation, each candidate image has a corresponding second category label, which represents the category of the object in the corresponding image;

[0010] The method further includes:

[0011] The second category label corresponding to the target candidate image is determined as the second category label of the image to be identified.

[0012] In one possible implementation, the training process of the re-identification network includes:

[0013] Determine at least one preset image including an object, each preset image having at least one image frame for labeling the area where the object is located, and a first category label corresponding to each image frame;

[0014] Determine at least one sample image corresponding to each preset image based on at least one corresponding image frame;

[0015] The re-identification network is trained in the first stage based on the sample images and the corresponding first category labels;

[0016] The pseudo-labels of the sample images are determined based on the re-identification network after the completion of the first phase of training;

[0017] The re-identification network obtained after the first stage of training is trained in the second stage based on the sample images, the corresponding first category labels, and pseudo labels.

[0018] In one possible implementation, determining at least one preset image including an object includes:

[0019] Randomly sample from a preset image set to obtain at least one preset image containing an object.

[0020] In one possible implementation, determining at least one sample image corresponding to each preset image based on at least one corresponding image frame includes:

[0021] Each preset image is subjected to at least one data augmentation, and after each data augmentation, at least one region within the image frame is cropped as a sample image.

[0022] In one possible implementation, the preset images are preprocessed before data augmentation is performed on each preset image.

[0023] In one possible implementation, the first-stage training of the re-identification network based on each sample image and its corresponding first category label includes:

[0024] The first category label corresponding to each of the sample images is determined as the second category label;

[0025] Each of the sample images is input into the re-identification network, and the first predicted category corresponding to the sample image is output.

[0026] A first network loss is determined based on the first category label, the second category label, and the first predicted category corresponding to each sample image, and the re-identification network is adjusted based on the first network loss.

[0027] In one possible implementation, determining a first network loss based on a first category label, a second category label, and a first predicted category corresponding to each sample image, and adjusting the re-identification network based on the first network loss, includes:

[0028] A first loss is determined based on the first category label and the first predicted category corresponding to each sample image;

[0029] The second loss is determined based on the second category label and the first predicted category corresponding to each sample image;

[0030] A first network loss is determined based on the first loss and the second loss, and the re-identification network is adjusted based on the first network loss.

[0031] In one possible implementation, determining the pseudo-label of the sample image based on the re-identification network after the first stage of training includes:

[0032] Each of the sample images is input into the re-identification network after the first stage of training is completed, to obtain the feature vector after feature extraction of each of the sample images;

[0033] Cluster the feature vectors of each sample image and determine the unique identifier information corresponding to each cluster obtained after clustering;

[0034] The identifier information corresponding to each cluster is used as a pseudo-label for the sample image corresponding to each feature vector included therein.

[0035] In one possible implementation, the clustering process is based on the k-means clustering algorithm.

[0036] In one possible implementation, the second-stage training of the re-identification network obtained after the first-stage training based on the sample image, the corresponding first category label, and the pseudo-label includes:

[0037] Each of the sample images is input into the re-identification network obtained after the first stage of training, and the corresponding second predicted category is output.

[0038] A second network loss is determined based on the first category label, pseudo label, and second predicted category corresponding to each sample image, and the re-identification network is adjusted based on the second network loss.

[0039] In one possible implementation, determining a second network loss based on a first category label, a pseudo-label, and a second predicted category corresponding to each sample image, and adjusting the re-identification network based on the second network loss, includes:

[0040] The third loss is determined based on the first category label and the second predicted category corresponding to each of the sample images;

[0041] The fourth loss is determined based on the pseudo-label and the second predicted category corresponding to each of the sample images;

[0042] The second network loss is determined based on the third loss and the fourth loss, and the re-identification network is adjusted based on the second network loss.

[0043] In one possible implementation, the first loss and / or the third loss are triplet losses, and the second loss and / or the fourth loss are cross-entropy classification losses.

[0044] In one possible implementation, inputting the image to be identified and the image set into the re-identification network to obtain the re-identification result includes:

[0045] The image to be identified and the image set are input into the re-identification network, and the target object features of the image to be identified and the candidate object features of each candidate image are extracted by the re-identification network.

[0046] The similarity between each candidate image and the image to be identified is determined based on the target object features and each candidate object feature;

[0047] In response to the similarity between the candidate image and the image to be identified meeting a preset condition, it is determined that the object in the candidate image matches the target object, and the candidate image is used as the target candidate image to obtain the re-identification result.

[0048] In one possible implementation, the preset condition is that the similarity value is the largest and greater than the similarity threshold.

[0049] According to a second aspect of this disclosure, an object re-identification apparatus is provided, comprising:

[0050] The image determination module is used to determine the image to be identified, which includes the target object.

[0051] A set determination module is used to determine an image set including at least one candidate image, each candidate image including an object;

[0052] The re-identification module is used to input the image to be identified and the image set into the re-identification network to obtain the re-identification result. When there is a target candidate image in the image set, the re-identification result includes the target candidate image, and the object included in the target candidate image matches the target object.

[0053] The re-identification network is trained in two stages. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on the at least one sample image, a pseudo label for each sample image, and a first category label. The pseudo label for each sample image is determined based on the re-identification network after the first stage training process is completed. The first category label represents the category of the corresponding image.

[0054] In one possible implementation, each candidate image has a corresponding second category label, which represents the category of the object in the corresponding image;

[0055] The device further includes:

[0056] The label determination module is used to determine the second category label corresponding to the target candidate image as the second category label of the image to be identified.

[0057] In one possible implementation, the training process of the re-identification network includes:

[0058] Determine at least one preset image including an object, each preset image having at least one image frame for labeling the area where the object is located, and a first category label corresponding to each image frame;

[0059] Determine at least one sample image corresponding to each preset image based on at least one corresponding image frame;

[0060] The re-identification network is trained in the first stage based on the sample images and the corresponding first category labels;

[0061] The pseudo-labels of the sample images are determined based on the re-identification network after the completion of the first phase of training;

[0062] The re-identification network obtained after the first stage of training is trained in the second stage based on the sample images, the corresponding first category labels, and pseudo labels.

[0063] In one possible implementation, determining at least one preset image including an object includes:

[0064] Randomly sample from a preset image set to obtain at least one preset image containing an object.

[0065] In one possible implementation, determining at least one sample image corresponding to each preset image based on at least one corresponding image frame includes:

[0066] Each preset image is subjected to at least one data augmentation, and after each data augmentation, at least one region within the image frame is cropped as a sample image.

[0067] In one possible implementation, the preset images are preprocessed before data augmentation is performed on each preset image.

[0068] In one possible implementation, the first-stage training of the re-identification network based on the sample image and the corresponding first category label includes:

[0069] The first category label corresponding to each of the sample images is determined as the second category label;

[0070] Each of the sample images is input into the re-identification network, and the first predicted category corresponding to the sample image is output.

[0071] A first network loss is determined based on the first category label, the second category label, and the first predicted category corresponding to each sample image, and the re-identification network is adjusted based on the first network loss.

[0072] In one possible implementation, determining a first network loss based on a first category label, a second category label, and a first predicted category corresponding to each sample image, and adjusting the re-identification network based on the first network loss, includes:

[0073] A first loss is determined based on the first category label and the first predicted category corresponding to each sample image;

[0074] The second loss is determined based on the second category label and the first predicted category corresponding to each sample image;

[0075] A first network loss is determined based on the first loss and the second loss, and the re-identification network is adjusted based on the first network loss.

[0076] In one possible implementation, determining the pseudo-label of the sample image based on the re-identification network after the first stage of training includes:

[0077] Each of the sample images is input into the re-identification network after the first stage of training is completed, to obtain the feature vector after feature extraction of each of the sample images;

[0078] Cluster the feature vectors of each sample image and determine the unique identifier information corresponding to each cluster obtained after clustering;

[0079] The identifier information corresponding to each cluster is used as a pseudo-label for the sample image corresponding to each feature vector included therein.

[0080] In one possible implementation, the clustering process is based on the k-means clustering algorithm.

[0081] In one possible implementation, the second-stage training of the re-identification network obtained after the first-stage training based on the sample image, the corresponding first category label, and the pseudo-label includes:

[0082] Each of the sample images is input into the re-identification network obtained after the first stage of training, and the corresponding second predicted category is output.

[0083] A second network loss is determined based on the first category label, pseudo label, and second predicted category corresponding to each sample image, and the re-identification network is adjusted based on the second network loss.

[0084] In one possible implementation, determining a second network loss based on a first category label, a pseudo-label, and a second predicted category corresponding to each sample image, and adjusting the re-identification network based on the second network loss, includes:

[0085] The third loss is determined based on the first category label and the second predicted category corresponding to each of the sample images;

[0086] The fourth loss is determined based on the pseudo-label and the second predicted category corresponding to each of the sample images;

[0087] The second network loss is determined based on the third loss and the fourth loss, and the re-identification network is adjusted based on the second network loss.

[0088] In one possible implementation, the first loss and / or the third loss are triplet losses, and the second loss and / or the fourth loss are cross-entropy classification losses.

[0089] In one possible implementation, the re-identification module includes:

[0090] The image input submodule is used to input the image to be identified and the image set into the re-identification network, and extract the target object features of the image to be identified and the candidate object features of each candidate image through the re-identification network;

[0091] A similarity matching submodule is used to determine the similarity between each candidate image and the image to be identified based on the features of the target object and each feature of the candidate object;

[0092] The result output submodule is used to determine that the object in the candidate image matches the target object when the similarity between the candidate image and the image to be identified meets a preset condition, and to obtain the re-identification result by using the candidate image as the target candidate image.

[0093] In one possible implementation, the preset condition is that the similarity value is the largest and greater than the similarity threshold.

[0094] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0095] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0096] In this embodiment of the disclosure, the performance of the re-identification network is guaranteed through two-stage training, thereby improving the accuracy of the identification results.

[0097] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0098] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0099] Figure 1 A flowchart of an object re-identification method according to an embodiment of the present disclosure is shown;

[0100] Figure 2 A flowchart illustrating a training re-identification network according to an embodiment of the present disclosure is shown;

[0101] Figure 3 A schematic diagram showing a preset image according to an embodiment of the present disclosure;

[0102] Figure 4 A schematic diagram showing a sample image according to an embodiment of the present disclosure;

[0103] Figure 5 A schematic diagram illustrating a determination of a sample diagram according to an embodiment of the present disclosure is shown;

[0104] Figure 6 A schematic diagram illustrating the first-stage training process of a re-identification network according to an embodiment of the present disclosure is shown.

[0105] Figure 7 A schematic diagram illustrating a second-stage training process for a re-identification network according to an embodiment of the present disclosure is shown.

[0106] Figure 8 A schematic diagram of an object re-identification device according to an embodiment of the present disclosure is shown;

[0107] Figure 9 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0108] Figure 10 A schematic diagram of another electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0109] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0110] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0111] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0112] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0113] In one possible implementation, the object re-identification method of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be any mobile or fixed terminal, such as a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the object re-identification method of this disclosure by having its processor call computer-readable instructions stored in its memory.

[0114] The object re-identification method of this disclosure can be applied to re-identify any object, such as people, vehicles, and animals. This re-identification method can search for images or video frames containing a specific object within multiple image or video frame sequences. It can be applied to application scenarios such as searching for a specific person in images captured by multiple cameras, or tracking objects such as pedestrians and vehicles.

[0115] Figure 1 A flowchart illustrating an object re-identification method according to an embodiment of this disclosure is shown. Figure 1 As shown, the object re-identification method of this disclosure embodiment may include the following steps S10-S30.

[0116] Step S10: Determine the image to be identified, which includes the target object.

[0117] In one possible implementation, the image to be identified can be an image obtained directly from the target object, or an image obtained by cropping the area containing the target object from an image obtained from the target object. The image to be identified can be acquired through an image acquisition device built into or connected to an electronic device, or by directly receiving an image to be identified sent by another device. The target object can be any movable or immovable object, such as a person, animal, vehicle, or even furniture.

[0118] Step S20: Determine an image set including at least one candidate image, each candidate image including an object.

[0119] In one possible implementation, a set of images is determined to serve as the basis for re-identification of the image to be identified, including at least one candidate image for matching with the image to be identified. Optionally, this set of images may be pre-stored in an electronic device or in a database connected to the electronic device. Each candidate image is obtained by capturing similar objects to the target object; this can be an image obtained by directly capturing the object, or an image obtained by cropping the region where the object is located from the image obtained by capturing the object. That is, the object in each candidate image is a similar object to the target object. For example, when the target object is a person, the objects in the candidate images are also people. When the target object is a vehicle, the objects in the candidate images are also vehicles.

[0120] Optionally, each candidate image in the image set also has a corresponding second category label to characterize the category of the object in the candidate image. For example, when the object in the candidate image is a person, the second category label can be the object's name, phone number, and identification document number, etc. When the object in the candidate image is a vehicle, the second category label can be the vehicle's license plate number, owner information, and vehicle registration document number, etc.

[0121] Step S30: Input the image to be identified and the image set into the re-identification network to obtain the re-identification result.

[0122] In one possible implementation, the image to be identified and the image set are input into a re-identification network. The re-identification network identifies candidate images from multiple candidate images whose included objects match the target object. These candidate images are then used as the target candidate images to obtain the re-identification result. That is, if a target candidate image exists that matches the target object, the re-identification result may include the target candidate image. Optionally, in addition to the target candidate image, the re-identification result may also include the category of the target object. That is, after determining the target candidate image, the second category label corresponding to the target candidate image is also determined as the second category label of the image to be identified.

[0123] Furthermore, the specific process of determining the re-identification result through the re-identification network can be as follows: input the image to be identified and the image set into the re-identification network; extract the target object features of the image to be identified and the candidate object features of each candidate image through the re-identification network; then determine the similarity between each candidate image and the image to be identified based on the target object features and each candidate object feature; in response to the similarity between the candidate image and the image to be identified meeting a preset condition, determine that the object in the candidate image matches the target object, and use the candidate image as the target candidate image.

[0124] Optionally, when the image to be identified is an image obtained by directly capturing the target object, the target object features can be obtained by cropping the region where the target object is located in the image to be identified and extracting the features of that region through the feature extraction layer of the re-identification network. Similarly, when the candidate image is an image obtained by directly capturing the object, the candidate object features can also be obtained by cropping the region where the object is located in the candidate image and extracting the features of that region through the feature extraction layer of the re-identification network. The target object features and each candidate object feature can be represented by vectors, and the similarity can be obtained by calculating the distance between their corresponding vectors in the feature space. This similarity can be calculated using the following formula:

[0125]

[0126] Where similarity(A,B) is the similarity between A and B, A is the target object feature, B is the candidate object feature, n is the number of elements in the target object feature and the candidate object feature, and i represents the position of the current element in the target object feature and the candidate object feature, that is, which element the current element is.

[0127] In one possible implementation, the preset condition can be that the similarity value is the largest and greater than a similarity threshold, that is, the candidate image with the largest similarity value and greater than the similarity threshold is determined as the target candidate image. Further, the second category label of the target candidate image is determined as the second category label of the image to be identified, and a re-identification result including the target candidate image and the corresponding second category label is determined. Optionally, if there is no similarity value that meets the preset condition, that is, if there is no target candidate image whose included object matches the target object, the category of the target object in the current image to be identified can be determined as a new category, and the re-identification result can be determined as the new category.

[0128] In one possible implementation, the re-identification network of this disclosure embodiment is obtained through two-stage training. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on at least one sample image, a pseudo-label for each sample image, and the first category label. The pseudo-label for each sample image is determined based on the re-identification network after the first stage training process, and the first category label represents the category of the corresponding image. The sample image is an unannotated sample image.

[0129] Figure 2 A flowchart illustrating a training re-identification network according to an embodiment of this disclosure is shown. Figure 2 As shown, the training process of the object re-identification network in this embodiment may include the following steps S40-S80. Optionally, the electronic device performing steps S40-S80 may be an electronic device performing an object re-identification method, or other electronic devices such as terminals or servers.

[0130] Step S40: Determine at least one preset image that includes the object.

[0131] In one possible implementation, each preset image is obtained by acquiring at least one object. Each preset image has at least one image box for labeling the area where the object is located and a first category label corresponding to each image box. Each preset image has at least one image box for labeling the area where the object is located in the preset image. This image box can be obtained by any object labeling method. For example, the preset image can be input into a pre-trained object recognition model to identify the location of objects included in the preset image and output at least one image box representing the object location. The first category label represents the category of the image region within the corresponding image box and can be determined based on the acquired object. For example, when two people are acquired using an image acquisition device to obtain preset images, the location of each person in the preset image can be identified to obtain two corresponding image boxes, and each image box can be assigned a corresponding first category label as Person 1 and Person 2.

[0132] Optionally, at least one preset image can be determined by random sampling, that is, by randomly sampling a preset image set to obtain at least one preset image including the object. The preset image set can be pre-stored in the electronic device training the re-identification network, or stored in other devices, and at least one preset image can be directly extracted from these other electronic devices by the electronic device training the re-identification network.

[0133] Figure 3 This is a schematic diagram showing a preset image according to an embodiment of the present disclosure. Figure 3 As shown, the preset image 30 may include at least one object, and the preset image 30 also has image frames for marking the positions of the objects. For example, when the preset image 30 is an image obtained by capturing at least one person, the preset image 30 may have image frames for representing the positions of the faces of at least one person. When the preset image 30 includes person 1 and person 2, the preset image 30 has a first image frame 31 representing the area where the face of person 1 is located, and a second image frame 32 representing the area where the face of person 2 is located. Optionally, since the preset image 30 is obtained by capturing two people, the first category label corresponding to the first image frame 31 in the preset image 30 can be directly set as person 1, and the first category label corresponding to the second image frame 32 can be set as person 2 during the process of marking the positions of the two people.

[0134] Step S50: Determine at least one sample image corresponding to each preset image based on at least one corresponding image frame.

[0135] In one possible implementation, after determining at least one preset image, at least one sample image corresponding to each preset image is determined based on the image frame corresponding to each preset image. Each sample image is obtained by cropping a portion of the preset image. Multiple sample images can be obtained by cropping from each image frame of the preset image. Optionally, each preset image can undergo at least one data augmentation, and after each data augmentation, a region within at least one image frame is cropped as a sample image. The data augmentation process may include translating the image frame, flipping the image frame, and reducing the image frame scale, so that the sample image cropped after each data augmentation can include different regions of the object.

[0136] Furthermore, since different preset images have different formats and attributes, in order to ensure that the obtained sample images meet the format requirements for training the re-identification network, each preset image can be preprocessed before data augmentation. This data preprocessing process can include any processing method such as format conversion, image brightness adjustment, and overall noise reduction. At least one processing method can be selected in advance for data preprocessing as needed.

[0137] Figure 4 This diagram illustrates a sample image according to an embodiment of the present disclosure. In one possible implementation, after determining a preset image 30, multiple sample images corresponding to at least one object are cropped from the preset image 30. When the preset image 30 includes person 1 and person 2, and the image frames on the preset image 30 are a first image frame 31 representing the area where person 1's face is located, and a second image frame 32 representing the area where person 2's face is located, at least one first object sample image 33 corresponding to person 1 and at least one second object sample image 34 corresponding to person 2 in the preset image 30 can be determined.

[0138] Optionally, before extracting the sample image, the preset image 30 is preprocessed, and then the first image frame 31 and the second image frame 32 are translated, flipped and scaled. After each operation, the content in the image frame is cropped to obtain the corresponding first object sample image 33 and second object sample image 34.

[0139] Figure 5 A schematic diagram illustrating one method of determining a sample diagram according to an embodiment of the present disclosure is shown. Figure 5 As shown in the embodiments of this disclosure, when determining sample images for training the re-identification network, a preset image set 50 including at least one preset image can be first determined. Random sampling 51 is then performed on the preset image set 50 to obtain at least one preset image 52 including an object. Image preprocessing 53 and data augmentation 54 are then performed on the preset images 52 sequentially, and a region within the image frame of each preset image 52 is cropped to obtain a sample image 55.

[0140] In one possible implementation, the order of the process of randomly sampling a preset image from a preset image set and the process of extracting sample images from the preset image set can be changed. That is, the preset images can be randomly sampled first and then sample images can be extracted, or sample images can be extracted from each preset image in the preset image set first and then random sampling can be performed. Optionally, the order of image preprocessing and data augmentation in the sample image extraction process can also be adjusted.

[0141] Based on the above methods, the embodiments of this disclosure can obtain multiple sample images corresponding to each object through data augmentation, significantly expanding the number of sample images. Furthermore, image processing can be performed in parallel using GPUs, reducing image processing speed and unnecessary background noise. Simultaneously, this disclosure alleviates the problem of difficult-to-calculate training loss due to too many sample categories by randomly selecting preset images to determine sample images, and ensures that the selected preset images are representative and reflect the characteristics of the preset image set through random sampling.

[0142] Step S60: Perform the first stage training of the re-identification network based on the sample image and the corresponding first category label.

[0143] In one possible implementation, after determining multiple sample images, the re-identification network can be directly trained in the first stage based on each sample image and a first category label. During training, the re-identification network can output a first predicted category for the input sample image, representing the category of the object predicted by the re-identification network in the input sample image. Since the sample image contains only one object, the true image category of the sample image is the true object category. The loss can be calculated separately based on the true image category and the true object category of the sample image and the first predicted category to obtain the total re-identification network loss for network adjustment.

[0144] Optionally, to improve training efficiency and reduce manual annotation costs, manual annotation of sample images is unnecessary before training the re-identification network. The first category label of the image bounding box corresponding to the sample image can be directly used as the first category label of that sample image; that is, the category of the region where each object is located in the preset image is used as the true image category of the sample image obtained by collecting that region. Furthermore, since most preset images contain only one object, and some contain a smaller number of objects, to improve annotation efficiency, the actual second category label can be omitted based on the object category in each sample image. During the first stage of training the re-identification network, the first category label of each sample image is directly used as the second category label representing the object category therein; the correction of the true object category in the sample images is then performed during the second stage of training.

[0145] For example, given a preset image with three bounding boxes containing three people, and each bounding box having a first category label of "Person 1," "Person 2," and "Person 3," sample images are extracted from each bounding box. When manually labeling the sample images corresponding to each bounding box, the identity of each person is specifically identified, and the corresponding second category label can be "Zhang San," "Li Si," and "Wang Wu." To save time in the labeling process and improve the efficiency of the re-identification network training process, the identity of the people in each sample image can be omitted; instead, the second category label of each bounding box's corresponding sample image can be quickly labeled as "Person 1," "Person 2," and "Person 3" simply by inheriting the first category label.

[0146] Based on the aforementioned second-category label determination method, the first-stage training process for the re-identification network includes determining the first-category label for each sample image as the second-category label, inputting each sample image into the re-identification network, and outputting the corresponding first predicted category. A first network loss is determined based on the first-category label, the second-category label, and the first predicted category for each sample image, and the re-identification network is adjusted based on this first network loss. Specifically, the first loss can be determined based on the first-category label and the first predicted category of the sample image, and the second loss can be determined based on the second-category label and the first predicted category of the sample image. That is, the first loss can be determined first based on the first-category label and the first predicted category of each sample image, the second loss can be determined based on the second-category label and the first predicted category of each sample image, and then the first network loss can be determined based on both the first and second losses, and the re-identification network can be adjusted based on this first network loss. The first network loss can be obtained by calculating the weighted sum of the first and second losses.

[0147] In one possible implementation, the first loss can be a triplet loss, and the second loss can be a cross-entropy classification loss. Specifically, the first loss can be obtained by calculating the triplet loss between the first class label and the first predicted class for each sample image, and the second loss can be obtained by calculating the cross-entropy classification loss between the second class label and the first predicted class for each sample image. The triplet loss is inversely proportional to the distance between samples of the same object class and directly proportional to the distance between samples of different object classes. The triplet loss can be reduced through network adjustment to shorten the distance between samples of the same object class and widen the distance between samples of different object classes. Similarly, the cross-entropy classification loss is inversely proportional to the distance between samples of the same image class, and can be reduced through network adjustment to shorten the distance between samples of the same image class.

[0148] Alternatively, the triplet loss and cross-entropy classification loss can be calculated using the following formulas 2 and 3, respectively:

[0149]

[0150]

[0151] In Formula 2, the triplet loss is L. th P×K represents the total number of sample images, a is any sample image, p is the sample image whose feature vector has the largest distance in feature space between its corresponding feature vector and the feature vector of a among multiple sample images with the same first category label as a, and n is the sample image whose feature vector has the smallest distance in feature space between its corresponding feature vector and the feature vector of a among multiple sample images with different first category labels as a. α is a preset correction parameter. In Formula 3, the cross-entropy classification loss is L, N is the number of sample images, M is the number of second category labels, and p... ic Let y be the predicted probability that sample image i belongs to the first predicted category c. ic The value is 1 when the second category label of sample i is c, and 0 when it is not c.

[0152] After determining the first network loss, the re-identification network can undergo a first-stage adjustment until the first network loss meets a first preset condition. This first preset condition can be that the first network loss is less than a preset first threshold. Based on the characteristics of triplet loss and cross-entropy classification loss, the re-identification network after the first-stage adjustment can obtain a reasonably distributed feature space. In other words, by adjusting the feature extraction layer of the re-identification network, it can extract similar feature vectors for images of the same image category, and also extract similar feature vectors for images of the same object category.

[0153] Figure 6 This diagram illustrates the first-stage training process of a re-identification network according to an embodiment of the present disclosure. Figure 6 As shown, in this embodiment of the present disclosure, after determining the sample image 60, the first category label 61 and the second category label 62 corresponding to the sample image 60 are determined based on the first category label 61 obtained simultaneously with the acquisition of the preset image corresponding to the sample image 60. Each sample image 60 is input into the re-identification network 63 to obtain a first predicted category 64, and a first loss 65 is calculated based on the first predicted category 64 and the first category label 61 of each sample image 60. Simultaneously, a second loss 66 is calculated based on the first predicted category 64 and the second category label 62 of each sample image 60, and the re-identification network 63 is adjusted jointly based on the first loss 65 and the second loss 66. Optionally, the adjustment method can be to calculate the weighted sum of the first loss 65 and the second loss 66 to obtain the first network loss, and perform a first-stage adjustment on the re-identification network 63 until the first network loss meets a first preset condition.

[0154] Step S70: Determine the pseudo-label of the sample image based on the re-identification network after the first stage of training.

[0155] In one possible implementation, after the first stage of training the re-identification network, a pseudo-label for each sample image can be determined based on the reasonably distributed feature space of the trained network. This pseudo-label is then used in the second stage of training to represent the category of the object in the sample image. The pseudo-label can be any label, and each pseudo-label uniquely represents a class of objects.

[0156] Optionally, the pseudo-labels can be determined based on the re-identification network after the first stage of training. This can be achieved by inputting each sample image into the re-identification network after the first stage of training, obtaining feature vectors after feature extraction for each sample image. The feature vectors of each sample image are then clustered, and a unique identifier is determined for each cluster. This identifier is used as the pseudo-label for each sample image corresponding to each feature vector. The clustering process can be implemented based on the k-means clustering algorithm. The unique identifier for each cluster can be pre-defined or generated according to preset rules.

[0157] Step S80: Perform a second-stage training on the re-identification network obtained after the first-stage training based on the sample image, the corresponding first category label, and the pseudo label.

[0158] In one possible implementation, after obtaining the pseudo-label for each sample image, the first category label corresponding to each sample image during the first stage of training is taken as the real image category, and the pseudo-label corresponding to each sample image is taken as the real object category. Further, a second stage of training of the re-identification network is performed based on the real image category, the real object category, and the sample image category predicted by the re-identification network for each current sample image. That is, each sample image can be input into the re-identification network obtained after the first stage of training, and the network outputs a corresponding second predicted category. A second network loss is determined based on the first category label, pseudo-label, and second predicted category corresponding to each sample image, and the re-identification network is adjusted based on the second network loss.

[0159] Optionally, similar to the first stage training process of the re-identification network, the second stage training process calculates the loss based on the true image category and the true object category of the sample images, respectively, and the second predicted category, to obtain the total re-identification network loss for network adjustment. That is, the second stage adjustment process of the re-identification network may include determining a third loss based on the first category label and the second predicted category corresponding to each sample image, and determining a fourth loss based on the pseudo label and the second predicted category corresponding to each sample image. Then, a second network loss is determined based on the third and fourth losses, and the re-identification network is adjusted based on the second network loss. The second network loss can be obtained by calculating the weighted sum of the third and fourth losses.

[0160] In one possible implementation, the third loss can be the triplet loss, and the fourth loss can be the cross-entropy classification loss. Specifically, the third loss can be obtained by calculating the triplet loss between the first class label and the second predicted class for each sample image, and the fourth loss can be obtained by calculating the cross-entropy classification loss between the pseudo-label and the first predicted class for each sample image. The triplet loss is inversely proportional to the distance between samples of the same object class and directly proportional to the distance between samples of different object classes. The network can be adjusted to reduce the triplet loss, bringing samples of the same object class closer together and widening the distance between samples of different object classes. Similarly, the cross-entropy classification loss is inversely proportional to the distance between samples of the same image class, and the network can be adjusted to reduce the cross-entropy classification loss, bringing samples of the same image class closer together. Optionally, the calculation process for the third loss can be the same as that for the first loss, and the calculation process for the fourth loss can be the same as that for the second loss, which will not be elaborated further here.

[0161] After determining the second network loss by calculating the weighted sum of the third and fourth losses, the re-identification network can undergo a second-stage adjustment until the second network loss meets a second preset condition. This second preset condition can be that the second network loss is less than a preset second threshold. Based on the characteristics of triplet loss and cross-entropy classification loss, the re-identification network after the second-stage adjustment can obtain a more reasonably distributed feature space. In other words, by adjusting the feature extraction layer of the re-identification network, it can more accurately extract similar feature vectors from images of the same image category, and also more accurately extract similar feature vectors from images of the same object category.

[0162] Figure 7 This diagram illustrates a second-stage training process for a re-identification network according to an embodiment of the present disclosure. Figure 7As shown, in this embodiment, after determining the sample image 70, the first category label 71 and pseudo label 72 corresponding to the sample image 70 are determined based on the first category label 71 obtained simultaneously with the acquisition of the preset image corresponding to the sample image 70. Each sample image 70 is input into the re-identification network 73 to obtain the second predicted category 74, and a third loss 75 is calculated based on the second predicted category 74 and the first category label 71 of each sample image 70. Simultaneously, a fourth loss 76 is calculated based on the second predicted category 74 and the pseudo label 72 of each sample image 70, and the re-identification network 73 is adjusted jointly based on the third loss 75 and the fourth loss 76. Optionally, the adjustment method can be to calculate the weighted sum of the third loss 75 and the fourth loss 76 to obtain the second network loss, and perform a first-stage adjustment on the re-identification network 73 until the second network loss meets the second preset condition.

[0163] Based on the training method, a high-accuracy re-identification network can be trained quickly and cost-effectively using unlabeled data. This re-identification network can accurately extract similar feature vectors from images of the same category, and also accurately extract similar feature vectors from images of the same object category, thus obtaining a reasonably distributed feature space. Furthermore, the accuracy of the re-identification network can be obtained through two-stage training, enabling accurate re-identification of the image to be identified, resulting in accurate re-identification results.

[0164] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0165] In addition, this disclosure also provides an object re-identification device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the object re-identification methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0166] Figure 8 A schematic diagram of an object re-identification device according to an embodiment of the present disclosure is shown. Figure 8 As shown, the object re-identification device in this embodiment may include an image determination module 80, a set determination module 81, and a re-identification module 82.

[0167] Image determination module 80 is used to determine the image to be identified, which includes the target object;

[0168] The set determination module 81 is used to determine an image set including at least one candidate image, each candidate image including an object;

[0169] The re-identification module 82 is used to input the image to be identified and the image set into the re-identification network to obtain the re-identification result. When there is a target candidate image in the image set, the re-identification result includes the target candidate image, and the object included in the target candidate image matches the target object.

[0170] The re-identification network is trained in two stages. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on the at least one sample image, a pseudo label for each sample image, and a first category label. The pseudo label for each sample image is determined based on the re-identification network after the first stage training process is completed. The first category label represents the category of the corresponding image.

[0171] In one possible implementation, each candidate image has a corresponding second category label, which represents the category of the object in the corresponding image;

[0172] The device further includes:

[0173] The label determination module is used to determine the second category label corresponding to the target candidate image as the second category label of the image to be identified.

[0174] In one possible implementation, the training process of the re-identification network includes:

[0175] Determine at least one preset image including an object, each preset image having at least one image frame for labeling the area where the object is located, and a first category label corresponding to each image frame;

[0176] Determine at least one sample image corresponding to each preset image based on at least one corresponding image frame;

[0177] The re-identification network is trained in the first stage based on the sample images and the corresponding first category labels;

[0178] The pseudo-labels of the sample images are determined based on the re-identification network after the completion of the first phase of training;

[0179] The re-identification network obtained after the first stage of training is trained in the second stage based on the sample images, the corresponding first category labels, and pseudo labels.

[0180] In one possible implementation, determining at least one preset image including an object includes:

[0181] Randomly sample from a preset image set to obtain at least one preset image containing an object.

[0182] In one possible implementation, determining at least one sample image corresponding to each preset image based on at least one corresponding image frame includes:

[0183] Each preset image is subjected to at least one data augmentation, and after each data augmentation, at least one region within the image frame is cropped as a sample image.

[0184] In one possible implementation, the preset images are preprocessed before data augmentation is performed on each preset image.

[0185] In one possible implementation, the first-stage training of the re-identification network based on the sample image and the corresponding first category label includes:

[0186] The first category label corresponding to each of the sample images is determined as the second category label;

[0187] Each of the sample images is input into the re-identification network, and the first predicted category corresponding to the sample image is output.

[0188] A first network loss is determined based on the first category label, the second category label, and the first predicted category corresponding to each sample image, and the re-identification network is adjusted based on the first network loss.

[0189] In one possible implementation, determining a first network loss based on a first category label, a second category label, and a first predicted category corresponding to each sample image, and adjusting the re-identification network based on the first network loss, includes:

[0190] A first loss is determined based on the first category label and the first predicted category corresponding to each sample image;

[0191] The second loss is determined based on the second category label and the first predicted category corresponding to each sample image;

[0192] A first network loss is determined based on the first loss and the second loss, and the re-identification network is adjusted based on the first network loss.

[0193] In one possible implementation, determining the pseudo-label of the sample image based on the re-identification network after the first stage of training includes:

[0194] Each of the sample images is input into the re-identification network after the first stage of training is completed, to obtain the feature vector after feature extraction of each of the sample images;

[0195] Cluster the feature vectors of each sample image and determine the unique identifier information corresponding to each cluster obtained after clustering;

[0196] The identifier information corresponding to each cluster is used as a pseudo-label for the sample image corresponding to each feature vector included therein.

[0197] In one possible implementation, the clustering process is based on the k-means clustering algorithm.

[0198] In one possible implementation, the second-stage training of the re-identification network obtained after the first-stage training based on the sample image, the corresponding first category label, and the pseudo-label includes:

[0199] Each of the sample images is input into the re-identification network obtained after the first stage of training, and the corresponding second predicted category is output.

[0200] A second network loss is determined based on the first category label, pseudo label, and second predicted category corresponding to each sample image, and the re-identification network is adjusted based on the second network loss.

[0201] In one possible implementation, determining a second network loss based on a first category label, a pseudo-label, and a second predicted category corresponding to each sample image, and adjusting the re-identification network based on the second network loss, includes:

[0202] The third loss is determined based on the first category label and the second predicted category corresponding to each of the sample images;

[0203] The fourth loss is determined based on the pseudo-label and the second predicted category corresponding to each of the sample images;

[0204] The second network loss is determined based on the third loss and the fourth loss, and the re-identification network is adjusted based on the second network loss.

[0205] In one possible implementation, the first loss and / or the third loss are triplet losses, and the second loss and / or the fourth loss are cross-entropy classification losses.

[0206] In one possible implementation, the re-identification module 82 includes:

[0207] The image input submodule is used to input the image to be identified and the image set into the re-identification network, and extract the target object features of the image to be identified and the candidate object features of each candidate image through the re-identification network;

[0208] A similarity matching submodule is used to determine the similarity between each candidate image and the image to be identified based on the features of the target object and each feature of the candidate object;

[0209] The result output submodule is used to determine that the object in the candidate image matches the target object when the similarity between the candidate image and the image to be identified meets a preset condition, and to obtain the re-identification result by using the candidate image as the target candidate image.

[0210] In one possible implementation, the preset condition is that the similarity value is the largest and greater than the similarity threshold.

[0211] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0212] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0213] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0214] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0215] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0216] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0217] Figure 9 A schematic diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0218] Reference Figure 9The electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0219] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0220] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0221] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0222] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0223] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0224] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0225] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 may detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0226] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, second-generation mobile communication technology (2G), or third-generation mobile communication technology (3G), or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0227] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0228] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of an electronic device 800 to perform the above-described method.

[0229] Figure 10 A schematic diagram of another electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 10 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0230] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TMApple's graphical user interface-based operating system (Mac OSX) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0231] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0232] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0233] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0234] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0235] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0236] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0237] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0238] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0239] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0240] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0241] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An object re-identification method, characterized in that, The method includes: Identify the image to be recognized, which includes the target object; Determine an image set comprising at least one candidate image, each candidate image comprising an object; The image to be identified and the image set are input into the re-identification network to obtain the re-identification result. If there is a target candidate image in the image set, the re-identification result includes the target candidate image, and the object included in the target candidate image matches the target object. The re-identification network is trained in two stages. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on the at least one sample image, a pseudo label for each sample image, and a first category label. The pseudo label for each sample image is determined based on the re-identification network after the first stage training process is completed. The first category label represents the category of the corresponding image.

2. The method according to claim 1, characterized in that, Each candidate image has a corresponding second category label, which represents the category of the object in the corresponding image; The method further includes: The second category label corresponding to the target candidate image is determined as the second category label of the image to be identified.

3. The method according to claim 1, characterized in that, The training process of the re-identification network includes: Determine at least one preset image including an object, each preset image having at least one image frame for labeling the area where the object is located, and a first category label corresponding to each image frame; Determine at least one sample image corresponding to each preset image based on at least one corresponding image frame; The re-identification network is trained in the first stage based on the sample images and the corresponding first category labels; The pseudo-labels of the sample images are determined based on the re-identification network after the completion of the first phase of training; The re-identification network obtained after the first stage of training is trained in the second stage based on the sample images, the corresponding first category labels, and pseudo labels.

4. The method according to claim 3, characterized in that, The determination of at least one preset image including an object includes: Randomly sample from a preset image set to obtain at least one preset image containing an object.

5. The method according to claim 3, characterized in that, The step of determining at least one sample image corresponding to each preset image based on at least one corresponding image frame includes: Each preset image is subjected to at least one data augmentation, and after each data augmentation, at least one region within the image frame is cropped as a sample image.

6. The method according to claim 5, characterized in that, Before performing data augmentation on each preset image, the preset images are preprocessed.

7. The method according to any one of claims 3-6, characterized in that, The first-stage training of the re-identification network based on the sample image and the corresponding first category label includes: The first category label corresponding to each of the sample images is determined as the second category label; Each of the sample images is input into the re-identification network, and the first predicted category corresponding to the sample image is output. A first network loss is determined based on the first category label, the second category label, and the first predicted category corresponding to each sample image, and the re-identification network is adjusted based on the first network loss.

8. The method according to claim 7, characterized in that, The step of determining a first network loss based on a first category label, a second category label, and a first predicted category corresponding to each sample image, and adjusting the re-identification network based on the first network loss, includes: A first loss is determined based on the first category label and the first predicted category corresponding to each sample image; The second loss is determined based on the second category label and the first predicted category corresponding to each sample image; A first network loss is determined based on the first loss and the second loss, and the re-identification network is adjusted based on the first network loss.

9. The method according to any one of claims 3-6, characterized in that, The step of determining the pseudo-labels of the sample images based on the re-identification network after the first stage of training includes: Each of the sample images is input into the re-identification network after the first stage of training is completed, to obtain the feature vector after feature extraction of each of the sample images; Cluster the feature vectors of each sample image and determine the unique identifier information corresponding to each cluster obtained after clustering; The identifier information corresponding to each cluster is used as a pseudo-label for the sample image corresponding to each feature vector included therein.

10. The method according to claim 9, characterized in that, The clustering is implemented based on the k-means clustering algorithm.

11. The method according to claim 8, characterized in that, The second stage training of the re-identification network obtained after the first stage training, based on the sample image, the corresponding first category label, and the pseudo label, includes: Each of the sample images is input into the re-identification network obtained after the first stage of training, and the corresponding second predicted category is output. A second network loss is determined based on the first category label, pseudo label, and second predicted category corresponding to each sample image, and the re-identification network is adjusted based on the second network loss.

12. The method according to claim 11, characterized in that, The step of determining a second network loss based on a first category label, a pseudo label, and a second predicted category corresponding to each sample image, and adjusting the re-identification network based on the second network loss, includes: The third loss is determined based on the first category label and the second predicted category corresponding to each of the sample images; The fourth loss is determined based on the pseudo-label and the second predicted category corresponding to each of the sample images; The second network loss is determined based on the third loss and the fourth loss, and the re-identification network is adjusted based on the second network loss.

13. The method according to claim 12, characterized in that, The first loss and / or the third loss are triplet losses, and the second loss and / or the fourth loss are cross-entropy classification losses.

14. The method according to any one of claims 1-6, characterized in that, The step of inputting the image to be identified and the image set into the re-identification network to obtain the re-identification result includes: The image to be identified and the image set are input into the re-identification network, and the target object features of the image to be identified and the candidate object features of each candidate image are extracted by the re-identification network. The similarity between each candidate image and the image to be identified is determined based on the target object features and each candidate object feature; In response to the similarity between the candidate image and the image to be identified meeting a preset condition, it is determined that the object in the candidate image matches the target object, and the candidate image is used as the target candidate image to obtain the re-identification result.

15. The method according to claim 14, characterized in that, The preset condition is that the similarity value is the largest and greater than the similarity threshold.

16. An object re-identification device, characterized in that, The device includes: The image determination module is used to determine the image to be identified, which includes the target object. A set determination module is used to determine an image set including at least one candidate image, each candidate image including an object; The re-identification module is used to input the image to be identified and the image set into the re-identification network to obtain the re-identification result. If there is a target candidate image in the image set, the re-identification result includes the target candidate image, and the object included in the target candidate image matches the target object. The re-identification network is trained in two stages. The first stage training process is implemented based on at least one sample image and a first category label for each sample image. The second stage training process is implemented based on the at least one sample image, a pseudo label for each sample image, and a first category label. The pseudo label for each sample image is determined based on the re-identification network after the first stage training process is completed. The first category label represents the category of the corresponding image.

17. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 15.

Images