Pedestrian re-identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a pedestrian re-identification method and device, electronic equipment and a storage medium, which are used for solving the technical problem that a conventional pedestrian re-identification mode excessively pays attention to local information and lacks the capability of paying attention to the whole, so that the pedestrian re-identification effect is relatively poor. The method comprises the following steps: acquiring pedestrian images and various prompt words corresponding to a plurality of preset types of human bodies respectively; based on the pedestrian image, training cue words by adopting an improved CLIP algorithm to obtain text cue words; training the image network by adopting the text cue word to obtain a target image network; the loss function of the image network comprises a comparison loss function of image features and text features, an identity loss function and a triple loss function; inputting the collected human body image to be re-recognized into the target image network to obtain a multi-dimensional feature; matching a target image from the candidate image set according to the multi-dimensional features; and comparing the target image with the to-be-re-identified image to obtain an identification result.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, and in particular to a pedestrian re-identification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Person re-identification, also known as pedestrian re-identification, is a technique that uses computer vision to determine the presence of a specific pedestrian in an image or video sequence. It is widely considered a sub-problem of image retrieval. Given a surveillance image of a pedestrian, it retrieves images of that pedestrian across different devices. It aims to overcome the visual limitations of fixed cameras and can be combined with pedestrian detection / tracking technologies, finding wide application in fields such as intelligent video surveillance and intelligent security.

[0003] Conventional pedestrian re-identification methods typically use convolutional neural networks (CNNs) as the backbone network, which can easily lead to an overemphasis on local information and a lack of ability to consider the overall picture. Summary of the Invention

[0004] This invention provides a pedestrian re-identification method, apparatus, electronic device, and storage medium to address the technical problem that conventional pedestrian re-identification methods focus too much on local information and lack the ability to consider the overall picture, resulting in poor pedestrian re-identification performance.

[0005] This invention provides a pedestrian re-identification method, comprising:

[0006] Acquire pedestrian images and various prompt words corresponding to multiple preset types of human bodies;

[0007] Based on the pedestrian image, the improved CLIP algorithm is used to train the prompt words to obtain text prompt words;

[0008] The image network is trained using the text prompts to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function.

[0009] The acquired human images to be re-identified are input into the target image network to obtain multi-dimensional features;

[0010] Match the target image from the candidate image set based on the multidimensional features;

[0011] The target image and the image to be re-identified are compared to obtain the recognition result.

[0012] Optionally, the step of training the prompt words using the improved CLIP algorithm based on the pedestrian image to obtain text prompt words includes:

[0013] The pedestrian image is input into an image encoder to obtain image features;

[0014] The image features are input into a visual enhancement network to obtain enhanced prompts;

[0015] By combining the aforementioned prompt word and the enhanced prompt word, a combined prompt word is obtained;

[0016] The combined prompt words are input into a text encoder to obtain text features;

[0017] A contrastive loss function is generated based on the text features and the image features;

[0018] Based on the value of the contrast loss function, adjust the prompt word parameters until the value of the contrast loss function is minimized, and output the current prompt word parameters as the text prompt word.

[0019] Optionally, the step of generating a contrastive loss function based on the text features and the image features includes:

[0020] Obtain the category corresponding to each human body image;

[0021] Generate a first contrast loss function for text-to-image and a second contrast loss function for image-to-text based on the category, the text features, and the image features;

[0022] By combining the first contrastive loss function and the second contrastive loss function, the contrastive loss function is obtained.

[0023] Optionally, the prompt words include human ID prompt words, modal prompt words, and platform prompt words; the step of adjusting the prompt word parameters according to the value of the contrast loss function until the value of the contrast loss function is minimized, and outputting the current prompt word parameters as the text prompt words, includes:

[0024] The prompting parameters of the modal prompting word and the platform prompting word are fixed, and the prompting parameters of the human ID prompting word are adjusted according to the value of the contrast loss function until the value of the contrast loss function is minimized;

[0025] The prompt parameters of the human ID prompt are fixed, and a new contrast loss function is calculated. The prompt parameters of the modal prompt and the platform prompt are adjusted according to the value of the new contrast loss function until the value of the contrast loss function is minimized. The current human ID prompt, the current modal prompt, and the current platform prompt are then output.

[0026] Optionally, the candidate image set includes multiple candidate images; the step of matching the target image from the candidate image set based on the multidimensional features includes:

[0027] Calculate the cosine similarity between the multidimensional feature and each of the candidate features;

[0028] The candidate image with the highest cosine similarity is selected as the target image.

[0029] Optionally, the step of comparing the target image and the image to be re-identified to obtain the recognition result includes:

[0030] Determine whether the human ID in the target image is the same as the human ID in the image to be re-identified;

[0031] If so, the image to be re-identified is determined to have been successfully re-identified;

[0032] If not, the re-identification of the image to be re-identified is deemed to have failed.

[0033] The present invention also provides a pedestrian re-identification device, comprising:

[0034] The pedestrian image and multi-category prompt word acquisition module is used to acquire pedestrian images and multi-category prompt words corresponding to multiple preset types of human bodies;

[0035] The text prompt word generation module is used to train the prompt words based on the pedestrian image using an improved CLIP algorithm to obtain the text prompt words;

[0036] The target image network training module is used to train the image network using the text prompt words to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function;

[0037] A multidimensional feature generation module is used to input the acquired human image to be re-identified into the target image network to obtain multidimensional features;

[0038] The matching module is used to match the target image from the candidate image set based on the multidimensional features;

[0039] The comparison module is used to compare the target image and the image to be re-identified to obtain the recognition result.

[0040] Optionally, the text prompt word generation module includes:

[0041] An image feature generation submodule is used to input the pedestrian image into an image encoder to obtain image features;

[0042] An enhanced prompt word generation submodule is used to input the image features into a visual enhancement network to obtain enhanced prompt words;

[0043] A combined prompt word generation submodule is used to combine the prompt word and the enhanced prompt word to obtain a combined prompt word;

[0044] The text feature generation submodule is used to input the combined prompt words into the text encoder to obtain text features;

[0045] A contrast loss function generation submodule is used to generate a contrast loss function based on the text features and the image features;

[0046] The text prompt output submodule is used to adjust the prompt parameters according to the value of the contrast loss function until the value of the contrast loss function is minimized, and then output the current prompt parameters as the text prompt.

[0047] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0048] The memory is used to store program code and transmit the program code to the processor;

[0049] The processor is configured to execute the pedestrian re-identification method as described above, according to instructions in the program code.

[0050] The present invention also provides a computer-readable storage medium for storing program code for performing the pedestrian re-identification method as described in any of the preceding claims.

[0051] As can be seen from the above technical solutions, the present invention has the following advantages: The present invention provides a pedestrian re-identification method, specifically disclosing: acquiring pedestrian images and multiple types of prompt words corresponding to various preset human body types; training prompt words using an improved CLIP algorithm based on the pedestrian images to obtain text prompt words; training an image network using the text prompt words to obtain a target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function; inputting the acquired human body image to be re-identified into the target image network to obtain multi-dimensional features; matching the target image from the candidate image set based on the multi-dimensional features; comparing the target image and the image to be re-identified to obtain the recognition result. The present invention increases the number and dimension of the trained prompt words by adding prompt words, and then trains the image network by combining the prompt words with multiple loss functions, thereby performing pedestrian re-identification analysis through the image network, thus improving the pedestrian re-identification effect. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the steps of a pedestrian re-identification method provided in this embodiment of the invention;

[0054] Figure 2 A flowchart illustrating the steps of a pedestrian re-identification method according to another embodiment of the present invention;

[0055] Figure 3 A schematic diagram illustrating the training process of text prompt words provided in an embodiment of the present invention;

[0056] Figure 4 This is a structural block diagram of a pedestrian re-identification device provided in an embodiment of the present invention. Detailed Implementation

[0057] This invention provides a pedestrian re-identification method, apparatus, electronic device, and storage medium to address the technical problem that conventional pedestrian re-identification methods focus too much on local information and lack the ability to consider the overall picture, resulting in poor pedestrian re-identification performance.

[0058] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Please see Figure 1 , Figure 1 A flowchart illustrating the steps of a pedestrian re-identification method provided in an embodiment of the present invention.

[0060] The present invention provides a pedestrian re-identification method, which may specifically include the following steps:

[0061] Step 101: Obtain pedestrian images and prompts corresponding to multiple preset types of human bodies;

[0062] Step 102: Based on pedestrian images, train prompt words using the improved CLIP algorithm to obtain text prompt words;

[0063] CLIP (Contrastive Language–Image Pretraining) is a pre-trained image-text model within a contrastive learning framework. It is trained using large-scale image-text pairs to learn the alignment relationship between vision and language.

[0064] The main improvement of the CLIP algorithm in this embodiment of the invention is the increase in the types of prompt words.

[0065] In this embodiment of the invention, pedestrian images corresponding to multiple preset types of human bodies and multiple types of prompt words can be obtained, and the prompt words can be trained by improving the CLIP algorithm to obtain text prompt words. Here, each human body represents one type; for example, if there are 751 people, there are 751 types of human bodies.

[0066] Step 103: Train the image network using text prompts to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function;

[0067] After obtaining the text prompts through training, the image network can be trained using the text prompts to obtain the target image network. The loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function.

[0068] Step 104: Input the acquired human image to be re-identified into the target image network to obtain multi-dimensional features;

[0069] In this embodiment of the invention, after training the target image network, the collected human image to be identified can be input into the target image network to obtain multidimensional features.

[0070] Step 105: Match the target image from the candidate image set based on multidimensional features;

[0071] Step 106: Compare the target image and the image to be re-identified to obtain the recognition result.

[0072] After obtaining multidimensional features, the target image can be matched from the candidate image set, and the re-identification judgment can be made by comparing the image to be re-identified and the target image.

[0073] This invention increases the number and dimensionality of trained prompt words by adding prompt words, and then trains an image network by combining the prompt words with multiple loss functions, thereby performing pedestrian re-identification analysis through the image network and improving the pedestrian re-identification effect.

[0074] Please see Figure 2 , Figure 2A flowchart illustrating the steps of a pedestrian re-identification method according to another embodiment of the present invention. Specifically, it may include the following steps:

[0075] Step 201: Obtain pedestrian images and prompts corresponding to multiple preset types of human bodies;

[0076] Step 202: Input the pedestrian image into the image encoder to obtain image features;

[0077] Step 203: Input the image features into the visual enhancement network to obtain enhanced prompt words;

[0078] Step 204: Combine the prompt words and the enhanced prompt words to obtain the combined prompt words;

[0079] Step 205: Input the combined prompt words into the text encoder to obtain text features;

[0080] Step 206: Generate a contrastive loss function based on text features and image features;

[0081] In this embodiment of the invention, the prompt words may include human ID prompt words (general prompt words), modal prompt words (including thermal imaging prompt words, infrared prompt words, and RGB prompt words), and platform prompt words (including surveillance camera prompt words and drone prompt words). Human ID prompt words include prompt words for all different people; for example, if there are 1000 different people, then there are 1000 features for human ID prompt words. Modal prompt words refer to different ambient light conditions, such as infrared modality and RGB image modality. Platform prompt words include, for example, CCTV closed-circuit television monitoring systems and the UVA human body weight recognition research platform. All three types of prompt words are 3-dimensional. For example, a human ID prompt word has a shape of (751, 8, 512), where 751 represents the number of human body types. This prompt word is essentially a string of numbers with a dimension of (751, 8, 512).

[0082] Human ID prompts, modal prompts, and platform prompts can be represented as X, P, and M, respectively. The initial values ​​for all three types of prompts can be implemented using PyTorch's `nn.Parameter` class. For example, the human ID prompt can be set using `nn.Parameter(torch.empty(num_class, n_generic_ctx, ctx_dim, dtype=dtype))`. The other two types of prompts are similar, except that the 0th dimension of modal and platform prompts is always 2, while the 0th dimension of the ID prompt is the number of human types. Additionally, the first dimension can also be different. The three types of prompts can be represented as follows:

[0083] t i1 (a) = X1(a)X2(a)...XM (a);

[0084] t i2 (a) = P1(a)P2(a)...P R (a);

[0085] t i3 (a) = M1(a)M2(a)...M B (a), person;

[0086] Where a is a human image, t i1 (a) is a human ID prompt word, t i2 (a) is a modal cue word, t i3 (a) is a platform prompt word.

[0087] In embodiments of the present invention, such as Figure 3 As shown, the training process for text prompts includes two paths. One path starts with the pedestrian image, passes through an image encoder to obtain image features, and then uses a visual enhancement network to obtain visually enhanced prompts. Each type of visually enhanced prompt is added to one of the three types of text prompts. The other path starts with the combined prompts, which are initially generated from random numbers and can be trained and learned. The three types of prompts are concatenated to obtain a tensor. This tensor, along with the text "human body" converted to numbers using a text tokenizer, is input into the encoder to obtain text features. These features have the same dimension as the image features. Then, the similarity between the image features and the text features is calculated, typically by direct multiplication. This similarity is used as a loss function, and the prompt parameters are adjusted using this loss function until the loss function is minimized. This is the entire training process.

[0088] In one example, the steps for generating a contrastive loss function based on text features and image features include:

[0089] S61, obtain the category corresponding to each human body image;

[0090] S62, Generate a first contrastive loss function from text to image and a second contrastive loss function from image to text based on category, text features and image features;

[0091] S63, combining the first contrastive loss function and the second contrastive loss function, the contrastive loss function is obtained.

[0092] In the specific implementation, the prompt words are first converted into text features (textfeatures) using a textencoder network, and the images are converted into image features (imagefeatures) using an image network. The shapes of the two types of features are (batchsizeT, featuredim) and (batchsizeM, featuredim), respectively, and the corresponding class labels are labelT and labelM, respectively. Then, labelT and labelM are broadcast to obtain two matrices of the same size (batchsizeT, batchsizeM). Assuming one vector is [1,2,3] and the other vector is [10,20,3,4], then the two matrices with shape (3,4) are obtained as follows:

[0093] First vector matrix:

[0094] ;

[0095] The second vector matrix:

[0096] ;

[0097] Set the value to 1 for the positions where the two matrices have the same value, and set the value to 0 for the positions where the values ​​are different, thus obtaining a mask matrix:

[0098] ;

[0099] A value of 1 indicates that the corresponding text and image belong to the same human body category. Then, the textfeatures matrix and the imagefeatures matrix are multiplied to obtain a matrix of shape (batchsizeT, batchsizeM). The maximum value of each row in this matrix is ​​found, and then the maximum value of each row is subtracted from the value of that row. This is done to prevent the exponent of each value from being too large when calculating the exponent of each value later. The exponent of each value is calculated as e raised to the power of x, and then the exponents of each row are summed to obtain a sum, denoted as m. Each row's exponent value is then divided by this sum m, resulting in a softmax vector for each row. For each row's softmax vector, the previously calculated mask matrix is ​​used. The softmax vector values ​​corresponding to the positions where each row of the mask has a value of 1 are summed and divided by the number of 1 values ​​in that row of the mask matrix. The average of these batchsize values ​​is then multiplied by (-1). This value is used as the loss calculated from textfeatures to imagefeatures, denoted as loss_t2i. The loss calculated from imagefeatures to textfeatures, i.e., loss_i2t, is calculated in the same way. The final loss is loss_i2t + loss_t2i.

[0100] The first contrastive loss function for image-to-text conversion, loss_i2t, is:

[0101] ;

[0102] The second contrastive loss function for text-to-image conversion, loss_t2i, is:

[0103] ;

[0104] Where B represents the batch size. Let i be the category of the person corresponding to the i-th person's image. Refers to the category as Let 'a' represent the set of all pedestrian human bodies in the current batch, 'p' represent the set of all samples in the batch, and 'p' represent the number of iterations. Positive samples in the data, where u represents the same platform. This represents the feature vector of the i-th human body image. For text features that match the i-th image sample, This refers to the cross-platform text features corresponding to the i-th human image, that is, the text features of the same human image i on different platforms. For the first Text features corresponding to an individual's human image For the first Cross-platform text features corresponding to individual body images, that is, the same person... Text features across different platforms This represents the image feature vector of a positive sample, where a positive sample is represented by a vector containing the same features as the positive sample. A collection of indexes of all samples of the same category. This represents the image feature vector of all samples. This represents the image feature vectors on the same platform. This represents the text features of each sample within a batch across different platforms. represents the text features matched for each sample within the batch, and s(*,*) is the cosine similarity function used to calculate the similarity between two feature vectors.

[0105] Step 207: Adjust the prompt word parameters according to the value of the contrast loss function until the value of the contrast loss function is minimized, and output the current prompt word parameters as the text prompt word;

[0106] In this embodiment of the invention, step 207 may include the following sub-steps:

[0107] S71, the prompt parameters of fixed modal prompts and platform prompts are adjusted according to the value of the contrast loss function until the value of the contrast loss function is minimized;

[0108] S72, fix the prompt parameters of the human ID prompt, calculate the new contrast loss function, adjust the prompt parameters of the modal prompt and the platform prompt according to the value of the new contrast loss function until the value of the contrast loss function is minimized, and output the current human ID prompt, the current modal prompt and the current platform prompt.

[0109] In practical implementation, after determining the loss function, when training the text prompt words, we can first train the human ID prompt words while fixing the parameters of the other two types of prompt words. The second step is to train the other two types of prompt words, fixing the parameters of the previously trained human ID prompt words. The loss function for all three types of prompt words is a contrastive loss function. The prompt word parameters are adjusted with the goal of minimizing the contrastive loss function value, thus obtaining the text prompt words.

[0110] Step 208: Train the image network using text prompts to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function;

[0111] In this embodiment of the invention, the image network employs a lightweight ReID re-identification backbone network, as shown in Table 1 below:

[0112] Table 1

[0113]

[0114] For the lightweight ReID re-identification backbone network described above, there are three types of loss functions. The first type is IDloss, which is a cross-entropy loss that uses the person's category as supervision information. The second type is triplet loss, which selects three people at a time: two from the same category and one from a different category. The similarity of the features of the two people from the different category is calculated, and then the similarity of the features of the two people from the same category is subtracted. This difference is used as the loss. The functional forms of the first and second types of loss functions are as follows:

[0115] ID loss formula:

[0116] ;

[0117] in, These are the model weight parameters. For the output of the previous layer of the model, Here, represents the model weight parameters, i represents the i-th class, j represents all classes, including class i and non-class i, and p represents the confidence score.

[0118] Triplet loss formula:

[0119] ;

[0120] Where d represents the cosine similarity of the 128-dimensional features extracted from two human images, (a,p) represents the cosine similarity of the same person, and (a,n) represents the cosine similarity of different people. 'a' is a manually set threshold whose core function is to force the model to maintain a difference of at least 'a' between the distance of positive sample pairs and the distance of negative sample pairs, thus preventing model training degradation and gradient vanishing, while also controlling the strength of the distance constraint between positive and negative sample pairs.

[0121] The third type of loss function is calculated as follows: First, the labels of the number of human body categories are converted into a two-dimensional tensor (classesnum, featdim), denoted as n, using the text encoder. Then, batches of images are converted into a two-dimensional tensor (batchsize, featdim), denoted as m, using the image network. Multiplying m by the transpose of n yields a tensor with shape (batchsize, classesnum). Since the second dimension of the resulting tensor is always the number of human body categories (classesnum), the cross-entropy loss function can be applied using the corresponding image category (target). This loss function aims to make the features obtained from each image through the image network increasingly closer to the corresponding three types of prompts. During this training phase, the weights of the text encoder and the three types of prompts remain fixed.

[0122] The calculation process of the third type of loss function includes:

[0123] 1. Calculate the similarity matrix: Logits = (B * C) ij :

[0124] Where B is batchsize, C is classesnum, and the dimension is B*C, representing the similarity score (logtic) between the i-th image feature and the j-th prompt word feature.

[0125] 2. Calculate the cross-entropy loss:

[0126]

[0127] Among them, y i is the true class label (Target) of the i-th sample, C represents the batch size, i.e. the number of samples, and C represents classesnum.

[0128] Step 209: Input the acquired human image to be re-identified into the target image network to obtain multi-dimensional features;

[0129] After training the target image network, the acquired human images to be re-identified can be input into the target image network to obtain multi-dimensional features.

[0130] Step 210: Match the target image from the candidate image set based on multidimensional features;

[0131] In practice, a target image can be matched from a set of candidate images based on multidimensional features.

[0132] In one example, step 210 may include the following sub-steps:

[0133] S101, calculate the cosine similarity between the multidimensional feature and each candidate feature;

[0134] S102, the candidate image with the highest cosine similarity is determined as the target image.

[0135] For two 128-dimensional feature vectors and The cosine similarity can be calculated using the following formula:

[0136] ;

[0137] Before calculating cosine similarity, vectors are generally... and Perform 2-norm normalization, therefore , Therefore, the formula for calculating the cosine distance is:

[0138]

[0139] Since the range of cosine similarity is [-1, 1], the range of cosine distance is [0, 2]. When the cosine similarity is 1, that is, the cosine distance is 0, it means that the two vectors are completely identical; when the cosine similarity is 0, that is, the cosine distance is 1, it means that the two vectors are orthogonal and independent; when the cosine similarity is -1, that is, the cosine distance is 2, it means that the two vectors are completely opposite.

[0140] Step 211: Compare the target image and the image to be re-identified to obtain the recognition result.

[0141] In this embodiment of the invention, after obtaining the target image, the target image and the image to be re-identified can be compared to obtain the recognition result.

[0142] In one example, step 211 may include the following sub-steps:

[0143] S111, determine whether the human body ID in the target image is the same as the human body ID in the image to be re-identified;

[0144] S112, if so, determine that the image to be re-identified has been successfully re-identified;

[0145] S113, if not, determine that the image to be re-identified has failed to be re-identified.

[0146] In the specific implementation, it can be determined whether the human ID in the target image is the same as the human ID in the image to be re-identified. If they are the same, the re-identification of the image to be re-identified is determined to be successful; otherwise, the re-identification is determined to be unsuccessful.

[0147] This invention increases the number and dimensionality of trained prompt words by adding prompt words, and then trains an image network by combining the prompt words with multiple loss functions, thereby performing pedestrian re-identification analysis through the image network and improving the pedestrian re-identification effect.

[0148] Please see Figure 4 , Figure 4 This is a structural block diagram of a pedestrian re-identification device provided in an embodiment of the present invention.

[0149] This invention provides a pedestrian re-identification device, comprising:

[0150] The pedestrian image and multiple prompt word acquisition module 401 is used to acquire pedestrian images and multiple prompt words corresponding to multiple preset types of human bodies;

[0151] The text prompt word generation module 402 is used to train prompt words based on pedestrian images using an improved CLIP algorithm to obtain text prompt words;

[0152] The target image network training module 403 is used to train the image network using text prompts to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function;

[0153] The multidimensional feature generation module 404 is used to input the acquired human image to be re-identified into the target image network to obtain multidimensional features;

[0154] Matching module 405 is used to match target images from a candidate image set based on multidimensional features;

[0155] The comparison module 406 is used to compare the target image and the image to be re-identified to obtain the recognition result.

[0156] In this embodiment of the invention, the text prompt word generation module 402 includes:

[0157] The image feature generation submodule is used to input pedestrian images into the image encoder to obtain image features;

[0158] The enhanced prompt word generation submodule is used to input image features into the visual enhancement network to obtain enhanced prompt words;

[0159] The combined prompt word generation submodule is used to combine prompt words and enhanced prompt words to obtain combined prompt words;

[0160] The text feature generation submodule is used to input combined prompt words into the text encoder to obtain text features;

[0161] The contrastive loss function generation submodule is used to generate a contrastive loss function based on text features and image features;

[0162] The text prompt output submodule is used to adjust the prompt parameters based on the value of the contrast loss function until the value of the contrast loss function is minimized, and then output the current prompt parameters as the text prompt.

[0163] In this embodiment of the invention, the comparison loss function generation submodule includes:

[0164] The category acquisition unit is used to acquire the category corresponding to each human body image;

[0165] The first contrast loss function and the second contrast loss function generation unit are used to generate a first contrast loss function from text to image and a second contrast loss function from image to text based on category, text features and image features.

[0166] The contrast loss function generation unit is used to combine the first contrast loss function and the second contrast loss function to obtain the contrast loss function.

[0167] In this embodiment of the invention, the prompt words include human ID prompt words, modal prompt words, and platform prompt words; the text prompt word output submodule includes:

[0168] The Human ID Prompt Training Unit is used to fix the prompt parameters of modal prompts and platform prompts, and adjust the prompt parameters of the Human ID Prompts according to the value of the contrast loss function until the value of the contrast loss function is minimized.

[0169] The reincarnation prompt word and platform prompt word training unit is used to fix the prompt word parameters of the human ID prompt word and calculate a new contrast loss function. Based on the value of the new contrast loss function, the prompt word parameters of the modal prompt word and platform prompt word are adjusted until the value of the contrast loss function is minimized, and the current human ID prompt word, current modal prompt word and current platform prompt word are output.

[0170] In this embodiment of the invention, the matching module 405 includes:

[0171] The cosine similarity calculation submodule is used to calculate the cosine similarity between multidimensional features and each candidate feature;

[0172] The target image determination submodule is used to determine the candidate image with the highest cosine similarity as the target image.

[0173] In this embodiment of the invention, the comparison module 406 includes:

[0174] The Human ID Determination Submodule is used to determine whether the human ID in the target image is the same as the human ID in the image to be re-identified.

[0175] The re-identification success determination submodule is used to determine if the image to be re-identified has been successfully re-identified.

[0176] The re-identification failure determination submodule is used to determine if the re-identification of the image to be re-identified has failed.

[0177] This invention also provides an electronic device, which includes a processor and a memory:

[0178] The memory is used to store program code and transfer the program code to the processor;

[0179] The processor is used to execute the pedestrian re-identification method of this invention according to the instructions in the program code.

[0180] This invention also provides a computer-readable storage medium for storing program code for executing the pedestrian re-identification method of this invention.

[0181] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0182] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0183] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0187] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0189] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0190] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian re-identification method, characterized in that, include: Acquire pedestrian images and various prompt words corresponding to multiple preset types of human bodies; Based on the pedestrian image, the improved CLIP algorithm is used to train the prompt words to obtain text prompt words; The image network is trained using the text prompts to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function. The acquired human images to be re-identified are input into the target image network to obtain multi-dimensional features; Match the target image from the candidate image set based on the multidimensional features; The target image and the image to be re-identified are compared to obtain the recognition result.

2. The method according to claim 1, characterized in that, The step of training the prompt words using the improved CLIP algorithm based on the pedestrian image to obtain text prompt words includes: The pedestrian image is input into an image encoder to obtain image features; The image features are input into a visual enhancement network to obtain enhanced prompts; By combining the aforementioned prompt word and the enhanced prompt word, a combined prompt word is obtained; The combined prompt words are input into a text encoder to obtain text features; A contrastive loss function is generated based on the text features and the image features; Based on the value of the contrast loss function, adjust the prompt word parameters until the value of the contrast loss function is minimized, and output the current prompt word parameters as the text prompt word.

3. The method according to claim 2, characterized in that, The step of generating a contrast loss function based on the text features and the image features includes: Obtain the category corresponding to each human body image; Generate a first contrast loss function for text-to-image and a second contrast loss function for image-to-text based on the category, the text features, and the image features; By combining the first contrastive loss function and the second contrastive loss function, the contrastive loss function is obtained.

4. The method according to claim 2, characterized in that, The prompt words include human ID prompt words, modal prompt words, and platform prompt words; the step of adjusting the prompt word parameters according to the value of the contrast loss function until the value of the contrast loss function is minimized, and outputting the current prompt word parameters as the text prompt words, includes: The prompting parameters of the modal prompting word and the platform prompting word are fixed, and the prompting parameters of the human ID prompting word are adjusted according to the value of the contrast loss function until the value of the contrast loss function is minimized; The prompt parameters of the human ID prompt are fixed, and a new contrast loss function is calculated. The prompt parameters of the modal prompt and the platform prompt are adjusted according to the value of the new contrast loss function until the value of the contrast loss function is minimized. The current human ID prompt, the current modal prompt, and the current platform prompt are then output.

5. The method according to claim 1, characterized in that, The candidate image set includes multiple candidate images; the step of matching the target image from the candidate image set based on the multidimensional features includes: Calculate the cosine similarity between the multidimensional feature and each of the candidate features; The candidate image with the highest cosine similarity is selected as the target image.

6. The method according to claim 1, characterized in that, The step of comparing the target image and the image to be re-identified to obtain the recognition result includes: Determine whether the human ID in the target image is the same as the human ID in the image to be re-identified; If so, the image to be re-identified is determined to have been successfully re-identified; If not, the re-identification of the image to be re-identified is deemed to have failed.

7. A pedestrian re-identification device, characterized in that, include: The pedestrian image and multi-category prompt word acquisition module is used to acquire pedestrian images and multi-category prompt words corresponding to multiple preset types of human bodies; The text prompt word generation module is used to train the prompt words based on the pedestrian image using an improved CLIP algorithm to obtain the text prompt words; The target image network training module is used to train the image network using the text prompt words to obtain the target image network; the loss function of the image network includes a contrast loss function between image features and text features, an identity loss function, and a triplet loss function; A multidimensional feature generation module is used to input the acquired human image to be re-identified into the target image network to obtain multidimensional features; The matching module is used to match the target image from the candidate image set based on the multidimensional features; The comparison module is used to compare the target image and the image to be re-identified to obtain the recognition result.

8. The apparatus according to claim 7, characterized in that, The text prompt word generation module includes: An image feature generation submodule is used to input the pedestrian image into an image encoder to obtain image features; An enhanced prompt word generation submodule is used to input the image features into a visual enhancement network to obtain enhanced prompt words; A combined prompt word generation submodule is used to combine the prompt word and the enhanced prompt word to obtain a combined prompt word; The text feature generation submodule is used to input the combined prompt words into the text encoder to obtain text features; A contrast loss function generation submodule is used to generate a contrast loss function based on the text features and the image features; The text prompt output submodule is used to adjust the prompt parameters according to the value of the contrast loss function until the value of the contrast loss function is minimized, and then output the current prompt parameters as the text prompt.

9. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the pedestrian re-identification method according to any one of claims 1-6 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the pedestrian re-identification method according to any one of claims 1-6.

Images