Pedestrian re-identification method and device, electronic device and storage medium

Abstract

The present application relates to a method and device for pedestrian re-identification, an electronic device and a storage medium. The method includes: obtaining an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian; generating a target description sentence corresponding to the image to be identified; obtaining a target feature vector corresponding to the target description sentence; segmenting the image to be identified to obtain multiple segmented images, and obtaining a target convolution vector corresponding to the segmented images; inputting the target feature vector and the target convolution vector into a preset neural network to obtain a pedestrian feature vector of the target pedestrian. The method in this embodiment can make the features ultimately used to determine the pedestrian feature vector include image features and text features, so that by obtaining features of different modalities, it is possible to focus on more fine-grained features of pedestrians, making the recognition of difficult samples more accurate.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a pedestrian re-identification method and device, an electronic device, and a storage medium. Background Art

[0002] In recent years, with the rapid development of Internet technology, smart hardware has also flourished. With the vigorous strengthening of social security and the construction of smart cities, every corner of the city is covered by cameras. The various forms of security risks faced by humans in society are also diverse, such as lost children and the arrest of criminal suspects. The required technical means also need a certain form of update. With the emergence of convolutional neural networks, tasks such as target detection and target classification have developed rapidly. Pedestrian re-identification aims to find specific pedestrians across cameras, which is of great significance for finding lost people, arresting suspects, and building a harmonious society.

[0003] However, in the related art, feature extractors (such as the ResNet class) are used for feature extraction, which only focuses on global features and ignores local features. There is also a method that uses the Vision Transformer (VIT) model as a network structure to extract features. The operation of this method is to first divide the pedestrian image into blocks, and then input the attention between the image blocks learned in the Transformer module. Some details of the pedestrians cannot be observed through each image block, so the information obtained is incomplete. Therefore, the related art only focuses on the global features or local features of pedestrians, and fails to pay attention to obtaining more fine-grained features of pedestrians, resulting in technical problems such as inaccurate recognition.

[0004] There is currently no effective solution to the technical problem that related technologies are unable to focus on obtaining more fine-grained features of pedestrians, resulting in inaccurate recognition. Summary of the Invention

[0005] In order to solve the above-mentioned technical problem of not being able to focus on obtaining more fine-grained features of pedestrians, resulting in inaccurate recognition, the present application provides a pedestrian re-identification method and device, electronic device and storage medium.

[0006] In a first aspect, an embodiment of the present application provides a method for person re-identification, comprising:

[0007] Acquire an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian;

[0008] Generate a target description sentence corresponding to the image to be identified; obtain a target feature vector corresponding to the target description sentence; segment the image to be identified to obtain multiple segmented images, and obtain target convolution vectors corresponding to the segmented images;

[0009] The target feature vector and the target convolution vector are input into a preset neural network to obtain a pedestrian feature vector of the target pedestrian.

[0010] Optionally, as in the aforementioned method, generating a target description sentence corresponding to the image to be recognized includes:

[0011] Inputting the image to be recognized into a first convolutional neural network to obtain visual features corresponding to the image to be recognized;

[0012] Decoding the visual features to obtain an output queue, wherein the output queue includes at least one decoded feature;

[0013] Obtain the probability corresponding to each word through the decoded features;

[0014] Based on the probability corresponding to each word, a target description sentence corresponding to the image to be recognized is obtained.

[0015] Optionally, as in the aforementioned method, segmenting the image to be identified to obtain a plurality of segmented images, and obtaining target convolution vectors corresponding to the segmented images, includes:

[0016] Segmenting the image to be identified according to a preset segmentation strategy to obtain the multiple segmented images;

[0017] A convolution operation is performed on each of the segmented images to obtain the convolution vector corresponding to each of the segmented images.

[0018] Optionally, as in the aforementioned method, obtaining the target feature vector corresponding to the target description sentence includes:

[0019] Inputting the image to be recognized into a second convolutional neural network to obtain global image features corresponding to the target pedestrian;

[0020] Inputting the target description sentence into a preset language representation model to obtain a sentence vector for indicating the semantics of the target description sentence;

[0021] Performing comparative learning on the global image features and the sentence vectors, and calculating feature similarity between the global image features and the sentence vectors;

[0022] The target description text is processed by the preset language representation model and according to the feature similarity to obtain the target feature vector, wherein the target feature vector includes a feature vector corresponding to each character one by one, and the target description sentence includes at least one of the characters.

[0023] Optionally, as in the aforementioned method, before inputting the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, the method further includes:

[0024] Determining a feature vector corresponding to each character in the target feature vector, and a first positional relationship between the characters in the target description sentence, wherein the target description sentence includes at least one of the characters; and

[0025] determining a second positional relationship between the segmented images;

[0026] A position vector is obtained according to the first position relationship and the second position relationship.

[0027] Optionally, as in the aforementioned method, inputting the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian includes:

[0028] Inputting a preset classification head, the target feature vector, the target convolution vector, and the position vector into the preset neural network, wherein the preset classification head is a classification vector pre-trained by the preset neural network;

[0029] The preset neural network performs feature processing on the target feature vector and the target convolution vector according to the first position relationship and the second position relationship indicated by the position vector to obtain the pedestrian feature vector.

[0030] Optionally, as in the aforementioned method, after inputting the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, the method further includes:

[0031] Matching the pedestrian feature vector with all historical pedestrian feature vectors to obtain a matching result;

[0032] When the matching result indicates that a designated pedestrian feature vector identical to the pedestrian feature vector exists in all the historical pedestrian feature vectors, the target pedestrian is determined to be the designated pedestrian corresponding to the designated pedestrian feature vector.

[0033] In a second aspect, an embodiment of the present application provides a pedestrian re-identification device, comprising:

[0034] An acquisition module is used to acquire an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian;

[0035] a processing module, configured to generate a target description statement corresponding to the image to be identified; obtain a target feature vector corresponding to the target description statement; segment the image to be identified to obtain a plurality of segmented images, and obtain a target convolution vector corresponding to the segmented images;

[0036] An obtaining module is used to input the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0038] The memory is used to store computer programs;

[0039] The processor is configured to implement any of the aforementioned methods when executing the computer program.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium includes a stored program, wherein the program executes the method as described in any of the preceding items when running.

[0041] This solution can be applied to prediction and optimization in the field of marketing intelligence technology. The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method in this embodiment obtains a target feature vector corresponding to the target description statement, and at the same time segments the image to be identified to obtain multiple segmented images, and obtains a target convolution vector corresponding to the segmented image, so that the features ultimately used to determine the pedestrian feature vector can include image features and text features, thereby obtaining features of different modalities, and then focusing on more fine-grained features of pedestrians, making the recognition of difficult samples more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of a pedestrian re-identification method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a pedestrian re-identification method provided in another embodiment of the present application;

[0046] Figure 3 A flowchart of a pedestrian re-identification method provided in another embodiment of the present application;

[0047] Figure 4 A flowchart of a pedestrian re-identification method provided in an application example of this application;

[0048] Figure 5 A block diagram of a pedestrian re-identification device provided in another embodiment of the present application;

[0049] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] According to one aspect of an embodiment of the present application, a method for pedestrian re-identification is provided. Optionally, in this embodiment, the above-mentioned method for pedestrian re-identification can be applied to a hardware environment consisting of a terminal and a server. The server is connected to the terminal via a network and can be used to provide services (such as data processing services, data analysis services, etc.) to the terminal or a client installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for the server.

[0052] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, or a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The terminal may be, but is not limited to, a PC, a mobile phone, a tablet computer, or the like.

[0053] The pedestrian re-identification method of the embodiment of the present application can be executed by a server, a terminal, or a server and a terminal. The pedestrian re-identification method of the embodiment of the present application can be executed by a terminal or a client installed thereon.

[0054] Taking the method of pedestrian re-identification in this embodiment performed by the server as an example, Figure 1 A pedestrian re-identification method provided in an embodiment of the present application includes the following steps:

[0055] Step S101 : obtaining an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian.

[0056] The person re-identification method of this embodiment can be applied to scenarios where it is necessary to continuously track pedestrians in surveillance videos, or to scenarios where it is necessary to identify the person to be found in surveillance videos. The above-mentioned person re-identification method is also applicable to other types of scenarios, provided there is no conflict.

[0057] The image to be identified can be a frame of an image in a surveillance video, or a partial image of a frame that only includes the target pedestrian.

[0058] For example, a preset camera can be used to capture video of the target monitoring area to obtain a target video, and then a frame of the target video can be extracted to obtain one of the frames of the image to be processed. Human body recognition can then be performed in the image to be processed to determine the image to be processed that requires pedestrian re-identification.

[0059] Step S102, generating a target description sentence corresponding to the image to be identified; obtaining a target feature vector corresponding to the target description sentence; segmenting the image to be identified to obtain multiple segmented images, and obtaining target convolution vectors corresponding to the segmented images.

[0060] After obtaining the image to be recognized, a picture description can be performed on the image to be recognized, thereby obtaining a target description sentence corresponding to the image to be recognized. For example, the target description sentence can be the following sentence: a woman wearing glasses and a white T-shirt.

[0061] After obtaining the target description sentence, the target description sentence can be subjected to feature extraction to obtain the target feature vector corresponding to the description sentence. Since the target description sentence is the text information obtained by describing the image to be recognized, the obtained target feature vector is a vector used to indicate global features.

[0062] After obtaining the image to be recognized, in order to obtain local features of the image to be recognized, the image to be recognized may be segmented to obtain a plurality of segmented images.

[0063] For example, the aspect ratio of the feature to be identified can be determined. When it is determined that the length is greater than the width (for example, when the target pedestrian is in a standing posture), the image to be identified can be horizontally segmented to obtain multiple segmented images. When the width of the image to be identified is greater than the length (for example, when the target pedestrian is in a lying posture), the image to be identified can be vertically segmented to obtain multiple segmented images. When the size of the original image is larger than the preset pixels, the original image can be processed to obtain the image to be identified within the preset pixels.

[0064] After all the segmented images are obtained, convolution processing may be performed on one or more or all of the segmented images to obtain target convolution vectors corresponding to the segmented images.

[0065] Step S103: Input the target feature vector and the target convolution vector into a preset neural network to obtain a pedestrian feature vector of the target pedestrian.

[0066] After obtaining the target feature vector and the target convolution vector, feature extraction can be performed based on the target feature vector and the target convolution vector.

[0067] By inputting the target feature vector and the target convolution vector into the preset neural network, a pedestrian feature vector that focuses on both global and local features can be obtained.

[0068] The method in this embodiment obtains a target feature vector corresponding to the target description sentence, and at the same time segments the image to be identified to obtain multiple segmented images, and obtains a target convolution vector corresponding to the segmented images, so that the features ultimately used to determine the pedestrian feature vector can include image features and text features. By obtaining features of different modalities, it is possible to focus on more fine-grained features of pedestrians, making the recognition of difficult samples more accurate.

[0069] As an optional implementation, as in the above method, generating a target description sentence corresponding to the image to be recognized in step S102 includes the following steps:

[0070] Step S201: input the image to be recognized into the first convolutional neural network to obtain visual features corresponding to the image to be recognized.

[0071] Step S202: Decode the visual features to obtain an output queue, wherein the output queue includes at least one decoded feature.

[0072] Step S203: Obtain the probability corresponding to each word through the decoded features. Step S204: Obtain the target description sentence corresponding to the image to be recognized based on the probability corresponding to each word.

[0073] After obtaining the image to be recognized, the image to be recognized can be input into a pre-trained CNN network (i.e., a first convolutional neural network) to extract corresponding visual features. After obtaining the video features, a decoder can be used to decode the visual features into an output queue; and the output queue includes at least one decoded feature.

[0074] After obtaining the output queue, the word corresponding to each decoded feature can be determined. Optionally, each decoded feature can be sequentially input into a separate LSTM (Long Short-Term Memory) module, with each LSTM module having a corresponding dataset (a dataset with a corresponding vocabulary (e.g., a vocabulary of 1,000 words)). For each LSTM module, taking the example of inputting a decoded feature into the LSTM module, it is determined that the LSTM module has a corresponding dataset. The corresponding dataset is then input into the LSTM module to determine the probability of each decoded feature corresponding to the dataset. If the dataset contains 1,000 words, the output will be a probability of dimension 1,000. Finally, the loss is calculated for each label. The probability of each word obtained through softmax in the training phase is such that the greater the probability, the greater the probability that the decoded feature corresponds to the word. The probability of each word is directly output in the prediction phase. Finally, the words can be connected into sentences by obtaining a preset number of words with the highest probabilities.

[0075] After obtaining the probabilities corresponding to all words in the word list, the target description sentence can be obtained by selecting the N target words with the highest probabilities, or target words with probabilities higher than a preset lower threshold, and then combining all target words.

[0076] Through the method in this embodiment, a target description sentence corresponding to the object to be identified can be obtained, so that the pedestrian feature vector of the target pedestrian can be determined based on text features in the later stage; thereby achieving the purpose of determining pedestrian features based on multimodal features.

[0077] like Figure 2 As shown, as an optional implementation, as in the above method, the step S102 of segmenting the image to be identified to obtain multiple segmented images and obtaining the target convolution vector corresponding to the segmented images includes the following steps:

[0078] Step S301 : Segment the image to be recognized according to a preset segmentation strategy to obtain a plurality of segmented images.

[0079] After acquiring the image to be recognized, in order to determine the local features of the image to be recognized, the image to be recognized may be segmented.

[0080] Optionally, a preset segmentation strategy may be predetermined, so as to segment the image to be identified according to the preset segmentation strategy and obtain a segmented image.

[0081] The preset segmentation strategy can be a predetermined strategy for segmenting the image to be identified, for example, segmenting the image from top to bottom into P segmented images, where P is any integer greater than 1; or segmenting according to preset pixels so that the height of each segmented image is a preset pixel.

[0082] Step S302: Perform a convolution operation on each segmented image to obtain a convolution vector corresponding to each segmented image.

[0083] After all the segmented images are obtained, a convolution operation can be performed on each of the segmented images to obtain a convolution vector corresponding to the segmented image.

[0084] For example, each segmented image can be input into the projector convolutional network, and then a convolution operation can be performed on each segmented image to obtain a convolution vector corresponding to each segmented image.

[0085] By using the method in this embodiment, the convolution vector corresponding to each segmented image can be quickly determined, so that the pedestrian feature vector corresponding to the target pedestrian can be determined based on the convolution vector in the later stage.

[0086] like Figure 3 As shown, as an optional implementation, as in the above method, obtaining the target feature vector corresponding to the target description sentence in step S102 includes the following steps:

[0087] Step S401: Input the image to be identified into the second convolutional neural network to obtain the global image features corresponding to the target pedestrian.

[0088] After obtaining the image to be identified, the image to be identified can be input into the ResNet network, and the features of the layer before the classification layer of the ResNet network can be extracted and used as the global features, that is, the global features of the image corresponding to the target pedestrian can be obtained.

[0089] In the process of ResNet network using convolutional neural network to extract image features, the local connection and weight sharing features of convolution are finally obtained, and the high-dimensional abstract features will contain the attribute information of multiple targets (for example, the color of the hat, whether wearing glasses, etc.).

[0090] Step S402: Input the target description sentence into a preset language representation model to obtain a sentence vector for indicating the semantics of the target description sentence.

[0091] After obtaining a target description sentence for describing the image to be recognized, a sentence vector corresponding to the target description sentence can be used.

[0092] For example, the target description sentence is fed into the pre-trained language representation model BERT, and processed to obtain the first classification head CLS (i.e., sentence vector).

[0093] Step S403: performing comparative learning on the global image features and the sentence vectors to calculate the feature similarity between the global image features and the sentence vectors;

[0094] In step S404, the target description text is processed according to feature similarity using a preset language representation model to obtain a target feature vector, wherein the target feature vector includes a feature vector corresponding to each character, and the target description sentence includes at least one character.

[0095] After obtaining the global image features and sentence vectors, we can compare the global image features with the first classification head (CLS) output by BERT and calculate the feature similarity between the global image features and the first classification head (CLS). This calculation aims to make the feature representation of the image to be recognized similar to that of the text, thus narrowing the gap between the two features. This also constrains the feature vector (i.e., the first classification head (CLS)) output by the BERT model.

[0096] Therefore, after obtaining the feature similarity and the target feature vector, the preset language representation model can be used to extract features from the target description problem based on the feature similarity as a constraint condition, so that the feature vector corresponding to each character can be close to the global feature of the image.

[0097] Through the method in this embodiment, the final pedestrian feature vector can be obtained not only based on local features, but also with reference to the global features of the image to be identified, so that the obtained information is more comprehensive, and thus the technical problem existing in the related technology of only focusing on the global features or local features of pedestrians, resulting in incomplete information, is solved, thereby achieving the purpose of combining the global features and local features of pedestrians to improve the accuracy of recognition.

[0098] As an optional embodiment, as in the aforementioned method, before inputting the target feature vector and the target convolution vector into the preset neural network in step S103 to obtain the pedestrian feature vector of the target pedestrian, the method further includes the following steps:

[0099] Step S501, determining a feature vector corresponding to each character in a target feature vector, and a first positional relationship between the characters in a target description sentence, wherein the target description sentence includes at least one character; and

[0100] Step S502, determining a second positional relationship between each segmented image;

[0101] Step S503: Obtain a position vector according to the first position relationship and the second position relationship.

[0102] After obtaining the target feature vector, the feature vector corresponding to each character can be determined; and in the target description sentence, there is a sequence between the characters, so the first position relationship between the characters in the target description sentence can be obtained; for example, when the target description sentence is "woman wearing glasses", the first position relationship can be (wearing, eyes, glasses, of, woman, person) to indicate the position relationship between the characters. Since the segmented image is obtained by segmenting the image to be processed according to the preset segmentation strategy, the second position relationship between the segmented images can be determined. For example: when the image to be processed A is segmented from top to bottom to obtain segmented image a1, segmented image a2, segmented image a3, segmented image a4, segmented image a5, segmented image a6, the second position relationship can be (segmented image a1, segmented image a2, segmented image a3, segmented image a4, segmented image a5, segmented image a6) to indicate the upper and lower relationship of the segmented images.

[0103] After obtaining the first positional relationship and the second positional relationship, a position vector can be obtained, and then the position vector is used to indicate the order between the feature vectors input into the preset neural network.

[0104] Through the method in this embodiment, the position vector can be determined based on the first position relationship and the second position relationship, so that when the pedestrian feature vector of the target pedestrian is determined later based on the feature vector and the target convolution vector, the first position relationship between each character and the second position relationship between each segmented image can be referred to.

[0105] As an optional implementation, as in the aforementioned method, step S103 inputs the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, including the following steps:

[0106] Step S601: input a preset classification head, a target feature vector, a target convolution vector, and a position vector into a preset neural network, wherein the preset classification head is a classification vector obtained by pre-training the preset neural network;

[0107] Step S602 : performing feature processing on the target feature vector and the target convolution vector according to the first position relationship and the second position relationship indicated by the position vector through a preset neural network to obtain a pedestrian feature vector.

[0108] After obtaining the target features, target convolution vectors and position vectors, the preset classification head, target feature vectors, target convolution vectors and position vectors can be input into the preset neural network. Optionally, when the preset neural network is a transformer neural network, the preset classification head is the second classification head CLS preset by the transformer neural network. The target features, target convolution vectors, position vectors and second classification head CLS are input into the transformer neural network, so that the preset neural network can perform feature processing on the target feature vectors and target convolution vectors according to the first position relationship and the second position relationship indicated by the position vector to obtain the pedestrian feature vector.

[0109] Optionally, during the training phase, in order to strengthen the constraints of multimodal features, the output image vectors (i.e., the vector obtained after the target convolution vector is processed by the preset neural network) can be integrated together and then the first classification is performed, the output character vectors (i.e., the vector obtained after the target feature vector is processed by the preset neural network) can be integrated together and then the second classification is performed, and the classification vector CLS output by the Transformer module (i.e., the pedestrian feature vector obtained after the preset classification head is processed by the preset neural network) is subjected to the third classification. Moreover, the results of the third classification can be constrained by the results of the first and second classifications to narrow the distance between the classification results. Since the Transformer module is a self-attention module, the classification vector CLS (i.e., the pedestrian feature vector) output by the Transformer module is fused with the relevant features of the target feature vector and the target convolution vector. Therefore, during use, the classification vector CLS output by the Transformer module can be directly used as the pedestrian feature vector.

[0110] Through the method in this embodiment, the output pedestrian feature vector can be integrated with the relevant features of the target feature vector and the target convolution vector, thereby achieving the purpose of the component multimodal pedestrian re-identification method.

[0111] As an optional embodiment, as in the aforementioned method, after inputting the target feature vector and the target convolution vector into the preset neural network in step S103 to obtain the pedestrian feature vector of the target pedestrian, the method further includes the following steps:

[0112] Step S701 : Match the pedestrian feature vector with all historical pedestrian feature vectors to obtain a matching result.

[0113] Step S702 : When the matching result indicates that a designated pedestrian feature vector identical to the pedestrian feature vector exists in all historical pedestrian feature vectors, the target pedestrian is determined to be the designated pedestrian corresponding to the designated pedestrian feature vector.

[0114] In order to realize pedestrian re-identification, the pedestrian feature vector can be matched with all historical pedestrian feature vectors to obtain a matching result.

[0115] The historical pedestrian feature vector may be a pedestrian feature vector obtained before the current pedestrian feature vector is obtained.

[0116] Optionally, matching can be performed by calculating the cosine value similarity between the pedestrian feature vector and each historical pedestrian feature vector, and a minimum lower limit threshold can be preset. When the cosine value is greater than or equal to the minimum lower limit threshold, the historical pedestrian feature vector is determined as the designated pedestrian feature vector, and the target pedestrian is determined as the designated pedestrian corresponding to the designated pedestrian feature vector.

[0117] Through the method in this embodiment, pedestrian tracking and identification can be performed using the pedestrian feature vectors obtained in the aforementioned embodiments, which can be widely used in searching for designated persons and can effectively improve the efficiency of searching for persons.

[0118] like Figure 4 As shown, an application example of applying any of the above embodiments is provided:

[0119] A pedestrian image is passed through an image caption module to generate a corresponding description text (i.e., the target description sentence). This description text is then fed into the pre-trained language representation model BERT (i.e., the preset language representation model). The image is then input into a ResNet network (i.e., the second convolutional neural network) to extract features. The resulting global image features are compared with the classification head CLS (i.e., sentence vector) output by BERT, and the feature similarity between the two features is calculated. This feature similarity calculation aims to make the feature representation of the pedestrian image similar to that of the text, thus narrowing the gap between the two features. This also constrains the feature vector output by the BERT model.

[0120] The pedestrian image is divided into 6 segments from top to bottom. After certain processing, these 6 segments are input into the Transformer module (i.e., the preset neural network) together with the feature vector of each character output by the BERT model. In order to strengthen the constraints of multimodal features, the output image vectors are integrated together and then the first classification is performed; the output character vectors are integrated together and then the second classification is performed; the classification vector CLS output by the Transformer is subjected to the third classification to obtain the pedestrian feature vector.

[0121] like Figure 5 According to another embodiment of the present application, a pedestrian re-identification device is provided, including:

[0122] Acquisition module 1, used to acquire an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian;

[0123] Processing module 2 is used to generate a target description sentence corresponding to the image to be identified; obtain a target feature vector corresponding to the target description sentence; segment the image to be identified to obtain multiple segmented images, and obtain target convolution vectors corresponding to the segmented images;

[0124] Obtaining module 3, which is used to input the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian.

[0125] Specifically, the specific process of each module in the device of the embodiment of the present invention realizing its function can be referred to the relevant description in the method embodiment, which will not be repeated here.

[0126] According to another embodiment of the present application, there is also provided an electronic device, including: Figure 6 As shown, the electronic device may include: a processor 1501 , a communication interface 1502 , a memory 1503 and a communication bus 1504 , wherein the processor 1501 , the communication interface 1502 , and the memory 1503 communicate with each other via the communication bus 1504 .

[0127] Memory 1503, used for storing computer programs;

[0128] The processor 1501 is configured to implement the steps of the above method embodiment when executing the program stored in the memory 1503 .

[0129] The bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0130] The communication interface is used for communication between the above electronic device and other devices.

[0131] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0132] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0133] An embodiment of the present application further provides a computer-readable storage medium, the storage medium including a stored program, wherein the method steps of the above method embodiment are executed when the program is run.

[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0135] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A pedestrian re-identification method, characterized in that: include: Acquire an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian; Generate a target description sentence corresponding to the image to be identified; obtain a target feature vector corresponding to the target description sentence; segment the image to be identified to obtain multiple segmented images, and obtain target convolution vectors corresponding to the segmented images; Determining a feature vector corresponding to each character in the target feature vector, and a first positional relationship between the characters in the target description sentence, wherein the target description sentence includes at least one character; and determining a second positional relationship between the segmented images; and obtaining a position vector based on the first positional relationship and the second positional relationship; The target feature vector and the target convolution vector are input into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, including: inputting a preset classification head, the target feature vector, the target convolution vector and the position vector into the preset neural network, wherein the preset classification head is a classification vector pre-trained by the preset neural network; and performing feature processing on the target feature vector and the target convolution vector according to the first position relationship and the second position relationship indicated by the position vector through the preset neural network to obtain the pedestrian feature vector.

2. The method according to claim 1, characterized in that The generating of a target description sentence corresponding to the image to be recognized includes: Inputting the image to be recognized into a first convolutional neural network to obtain visual features corresponding to the image to be recognized; Decoding the visual features to obtain an output queue, wherein the output queue includes at least one decoded feature; Obtain the probability corresponding to each word through the decoded features; Based on the probability corresponding to each word, a target description sentence corresponding to the image to be recognized is obtained.

3. The method according to claim 1, characterized in that The step of segmenting the image to be recognized to obtain a plurality of segmented images and obtaining target convolution vectors corresponding to the segmented images includes: Segmenting the image to be identified according to a preset segmentation strategy to obtain the multiple segmented images; A convolution operation is performed on each of the segmented images to obtain the convolution vector corresponding to each of the segmented images.

4. The method according to claim 1, wherein The obtaining of the target feature vector corresponding to the target description sentence includes: Inputting the image to be recognized into a second convolutional neural network to obtain global image features corresponding to the target pedestrian; Inputting the target description sentence into a preset language representation model to obtain a sentence vector for indicating the semantics of the target description sentence; Performing comparative learning on the global image features and the sentence vector, and calculating feature similarity between the global image features and the sentence vector; The target description text is processed by the preset language representation model and according to the feature similarity to obtain the target feature vector, wherein the target feature vector includes a feature vector corresponding to each character one by one, and the target description sentence includes at least one of the characters.

5. The method according to any one of claims 1 to 4, characterized in that After inputting the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, the method further includes: Matching the pedestrian feature vector with all historical pedestrian feature vectors to obtain a matching result; When the matching result indicates that a designated pedestrian feature vector identical to the pedestrian feature vector exists in all the historical pedestrian feature vectors, the target pedestrian is determined to be the designated pedestrian corresponding to the designated pedestrian feature vector.

6. A pedestrian re-identification device, characterized in that: include: An acquisition module is used to acquire an image to be identified that requires pedestrian re-identification, wherein the image to be identified includes a target pedestrian; a processing module, configured to generate a target description statement corresponding to the image to be identified; obtain a target feature vector corresponding to the target description statement; segment the image to be identified to obtain a plurality of segmented images, and obtain a target convolution vector corresponding to the segmented images; The system further includes a module for determining a feature vector corresponding to each character in the target feature vector, and a first positional relationship between the characters in the target description sentence, wherein the target description sentence includes at least one character; and determining a second positional relationship between the segmented images; and obtaining a position vector based on the first positional relationship and the second positional relationship. An obtaining module is used to input the target feature vector and the target convolution vector into a preset neural network to obtain the pedestrian feature vector of the target pedestrian, including: inputting a preset classification head, the target feature vector, the target convolution vector and the position vector into the preset neural network, wherein the preset classification head is a classification vector pre-trained by the preset neural network; and performing feature processing on the target feature vector and the target convolution vector according to the first position relationship and the second position relationship indicated by the position vector through the preset neural network to obtain the pedestrian feature vector.

7. An electronic device, characterized in that: include: A processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 5 when executed on a processor.

Images