Pedestrian re-identification method, device, equipment and storage medium based on deep learning

Abstract

The present invention discloses a pedestrian re-identification method, device, equipment and storage medium based on deep learning, the method comprising: performing human target detection on source image data to determine the target pedestrian; performing attribute detection on the target pedestrian to determine the local area attribute information of the target pedestrian; performing feature extraction on the local area of ​​the target pedestrian to obtain the features of the local area and weighted sum the features to determine the features of the target pedestrian; determining the similarity between the features of the target pedestrian and each feature in the pedestrian feature library; and determining the pedestrian re-identification result according to the similarity. The present invention helps to improve the recognition accuracy and timeliness of recognition, and can be widely used in the field of deep learning technology.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a deep learning-based pedestrian re-identification method, device, equipment and storage medium. Background Art

[0002] Person Re-identification (also known as pedestrian re-identification, abbreviated as ReID) is a technology that uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence. In video surveillance scenarios, pedestrians are an important component, and pedestrian re-identification technology is used to identify pedestrians and track their trajectories. Pedestrian re-identification technology mainly includes two parts: pedestrian feature extraction and pedestrian feature comparison. Pedestrian feature extraction refers to the extraction of representative feature vectors from pedestrians detected by snapshots; pedestrian feature comparison refers to the calculation of distance similarity between the extracted features and the pedestrian features in the feature library. The higher the similarity, the greater the probability that they are the same person.

[0003] Among them, the existing feature representation-based methods generally represent pedestrians by extracting more robust identification features. Although the computational complexity is relatively low, the accuracy is also relatively low.

[0004] The distance metric-based method generally learns a discriminative distance metric function to make the distance between images of the same person smaller than the distance between images of different pedestrians. This requires a complex learning process and cannot meet the timeliness and accuracy requirements in practical applications.

[0005] In real-world scenarios, the situation is complex and changeable. Due to the huge changes in pedestrian appearance caused by different cameras, including pedestrian scale, lighting changes, target occlusion, and viewing angle, pedestrian re-identification methods face huge challenges in practical applications. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a pedestrian re-identification method, device, equipment and storage medium based on deep learning, thereby improving the recognition accuracy and timeliness of recognition.

[0007] One aspect of the present invention provides a method for pedestrian re-identification based on deep learning, comprising:

[0008] Perform human target detection on the source image data to determine the target pedestrian;

[0009] Performing attribute detection on the target pedestrian to determine local area attribute information of the target pedestrian;

[0010] Extracting features from a local area of ​​the target pedestrian to obtain features of the local area and performing weighted summation on the features to determine the features of the target pedestrian;

[0011] Determine the similarity between the feature of the target pedestrian and each feature in the pedestrian feature library;

[0012] A pedestrian re-identification result is determined according to the similarity.

[0013] Optionally, performing human target detection on the source image data to determine the target pedestrian includes:

[0014] Acquire real-time video sequences to obtain multiple source image data;

[0015] Human target detection is performed on each of the source image data through the YOLOv5 network to determine multiple target pedestrians in each of the source image data.

[0016] Optionally, the target pedestrian is subjected to attribute detection to determine local area attribute information of the target pedestrian.

[0017] include:

[0018] The target pedestrian is subjected to attribute detection through the YOLOv5 network to determine the local area attribute information of the target pedestrian; wherein the attribute information includes the pedestrian's whole body area, the pedestrian's head area, the pedestrian's upper body area, the pedestrian's lower body area, the pedestrian category and the pedestrian confidence.

[0019] Optionally, extracting features from a local area of ​​the target pedestrian to obtain features of the local area and performing weighted summation on the features to determine the features of the target pedestrian includes:

[0020] Inputting the attribute information of the target pedestrian into a feature extraction network, and extracting the target pedestrian's features by the feature extraction network;

[0021] The step of extracting the target pedestrian's features by the feature extraction network includes:

[0022] Processing the attribute information by sharing the CNN to obtain a first processing result;

[0023] Differentiate the first processing results by labels to obtain second processing results of different types;

[0024] Inputting the different types of second processing results into an independent network for feature extraction to obtain a third processing result;

[0025] Perform feature weighted summation processing on the third processing result obtained by the independent network processing to obtain a fourth processing result;

[0026] Output the features of the target pedestrian.

[0027] Optionally, the method further comprises the step of constructing a pedestrian feature library.

[0028] Optionally, determining the similarity between the feature of the target pedestrian and each feature in a pedestrian feature library includes:

[0029] Acquiring attribute information of the target pedestrian, and filtering features in the pedestrian feature library according to the attribute information;

[0030] Extracting the candidate features in the filtered pedestrian feature library, calculating the vector distance between the candidate features and the features of the target pedestrian, and determining the similarity between the features of the target pedestrian and each feature in the pedestrian feature library;

[0031] The filtering conditions include age group conditions, gender conditions and hair length conditions.

[0032] Optionally, the inference step of the feature extraction network includes:

[0033] Convert the trained data into a TensorRT inference model;

[0034] Perform Int8 quantization on the TensorRT inference model;

[0035] The result of the quantization process is inferred by TensorRT to obtain the characteristics of the target pedestrian.

[0036] The embodiment of the present invention further provides a pedestrian re-identification device based on deep learning, comprising:

[0037] The target detection module is used to perform human target detection on the source image data and determine the target pedestrian;

[0038] An attribute detection module, used to perform attribute detection on the target pedestrian and determine local area attribute information of the target pedestrian;

[0039] A feature extraction module is used to extract features from a local area of ​​the target pedestrian, obtain features of the local area and perform weighted summation on the features to determine the features of the target pedestrian;

[0040] A similarity calculation module, used to determine the similarity between the features of the target pedestrian and each feature in the pedestrian feature library;

[0041] The recognition module is used to determine the pedestrian re-recognition result according to the similarity.

[0042] An embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0043] The memory is used to store programs;

[0044] The processor executes the program to implement the method described above.

[0045] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0046] An embodiment of the present invention also discloses a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0047] The embodiment of the present invention performs human target detection on the source image data to determine the target pedestrian; then performs attribute detection on the target pedestrian to determine the local area attribute information of the target pedestrian; then performs feature extraction on the local area of ​​the target pedestrian to obtain the features of the local area and performs weighted summation on the features to determine the features of the target pedestrian; then determines the similarity between the features of the target pedestrian and each feature in the pedestrian feature library; finally, determines the pedestrian re-identification result based on the similarity. The embodiment of the present invention helps to improve the recognition accuracy and timeliness of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 An overall step flow chart provided for an embodiment of the present invention;

[0050] Figure 2 A feature extraction network flow chart provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] In view of the problems existing in the prior art, the present invention provides a method, device, equipment and storage medium for pedestrian re-identification based on deep learning. The overall steps of the method may include: 1. Detecting a single pedestrian based on the whole image. 2. Detecting the human body module of a single pedestrian to obtain four regions of the pedestrian's whole body, head, upper body and lower body. 3. Extracting the features of each block through a feature network. 4. Conditional filtering of the pedestrian feature library to reduce the features to be compared. 5. Calculating the similarity of feature vector similarity distance to obtain similarity.

[0053] Specifically, Figure 1 As shown, the method of the present invention comprises the following steps:

[0054] Step 1: In actual application scenarios, the data source comes from camera capture or real-time video sequences. Most of them are full images, and each image generally has multiple pedestrian targets. The present invention needs to perform pedestrian target detection on the full image to detect a single pedestrian. This system uses but is not limited to the YOLOv5 network to perform pedestrian target detection and obtain a single pedestrian. The YOLOv5 network training steps are as follows:

[0055] 1) Organize the labeled data and divide it into training set, validation set and test set;

[0056] 2) Convert the data format and convert the xml tag file into a TXT file in yolo format;

[0057] 3) Select the yolo5s model, modify the configuration file for model training, and output the trained model file.

[0058] Step 2: Attribute detection of a single pedestrian is performed to obtain four regions of the pedestrian's body, head, upper body, and lower body, and these regions are used as inputs to the feature extraction model. Pedestrian attribute detection is a necessary step in extracting pedestrian structured information. This system uses but is not limited to the YOLOv5 network to perform pedestrian attribute detection to obtain various regions of the pedestrian's torso.

[0059] Step 3: Pedestrian feature extraction is to extract discriminative features of pedestrians. Figure 2 As shown in the figure, the images of each area of ​​the pedestrian's torso are input into the network, passed through the shared CNN, distinguished by labels, and then enter the corresponding independent network to extract the features of each part. After global pooling, the dimension is reduced to a 512-dimensional vector, and then the features are weighted and summed (implemented using formula 1) as the output. The backbone feature extraction part of the network, that is, the shared CNN part, is a ResNet50 network. At the cov5 layer, it is modified to add a sub-network to extract features from each part separately, and then pool the output feature values. The weighted summation formula of the features is as follows:

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] Among them, the vector They are the head region feature vector, the upper body region feature vector, the lower body region feature vector and the whole body region feature vector. is the weighted sum of the feature vectors, P is the confidence of the detection of each region, and n is the vector dimension, which is 512.

[0066] Step 4: The pedestrian feature library is established by extracting features from the network in step 3. The present invention first extracts the structured information of pedestrians. When comparing features, the structured information of pedestrians is used to filter the feature library to reduce the number of features to be compared, thereby reducing time consumption and improving timeliness. The filtering conditions used are: age group, gender, hair length, etc. After the training is completed, the weight model is obtained, quantized by INT8, and converted to int8 type. During inference, the feature vector of the specified layer is obtained to calculate the similarity and reduce the time consumption of forward calculation.

[0067] Step 5: Calculate the vector distance between the extracted features and the features in the feature library, and sort them according to the calculated distance. The higher the ranking, the higher the similarity. The distance calculation uses the conventional cosine distance to calculate the distance of the feature vector. The smaller the distance, the higher the similarity. The re-sorting method is used to optimize the distance calculation.

[0068] Step 6: All model files in the embodiment of the present invention are converted from trained weight files into TensorRT inference models, and then quantized to Int8, and then inferenced using TensorRT to improve the inference speed of the model. Among them, TensorRT processes the trained feature extraction network model in three steps:

[0069] First: TensorRT parses the trained network model, eliminates useless output layers, and reduces calculations.

[0070] Second: vertical integration of the network, that is, integrating the three layers of conv, BN, and Relu into one layer.

[0071] Third: Network horizontal combination, fusing together layers that take the same tensor as input and perform the same operation.

[0072] In summary, the present invention has the following advantages:

[0073] 1. In practical applications, the present invention can be used as a main function in pedestrian structured products, effectively utilizing pedestrian structured information, providing more conditions for pedestrian re-identification, and improving the timeliness and accuracy of pedestrian retrieval.

[0074] 2. The present invention extracts local features and then performs feature fusion to construct global and local fine-grained features, making pedestrian features more representative and improving the accuracy of pedestrian retrieval in practical applications.

[0075] 3. The present invention can improve the timeliness in practical applications by quantifying the extracted features and reducing the reasoning time of feature extraction.

[0076] The embodiment of the present invention further provides a pedestrian re-identification device based on deep learning, comprising:

[0077] The target detection module is used to perform human target detection on the source image data and determine the target pedestrian;

[0078] An attribute detection module, used to perform attribute detection on the target pedestrian and determine local area attribute information of the target pedestrian;

[0079] A feature extraction module is used to extract features from a local area of ​​the target pedestrian, obtain features of the local area and perform weighted summation on the features to determine the features of the target pedestrian;

[0080] A similarity calculation module, used to determine the similarity between the features of the target pedestrian and each feature in the pedestrian feature library;

[0081] The recognition module is used to determine the pedestrian re-recognition result according to the similarity.

[0082] An embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0083] The memory is used to store programs;

[0084] The processor executes the program to implement the method described above.

[0085] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0086] An embodiment of the present invention also discloses a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0087] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0088] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0089] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0090] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0091] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0092] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0093] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0094] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0095] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A pedestrian re-identification method based on deep learning, characterized in that: include: Perform human target detection on the source image data to determine the target pedestrian; Performing attribute detection on the target pedestrian to determine local area attribute information of the target pedestrian; Extracting features from a local area of ​​the target pedestrian to obtain features of the local area and performing weighted summation on the features to determine the features of the target pedestrian; Determine the similarity between the feature of the target pedestrian and each feature in the pedestrian feature library; Determine a person re-identification result according to the similarity; The extracting features of the local area of ​​the target pedestrian, obtaining features of the local area and performing weighted summation on the features to determine the features of the target pedestrian includes: Inputting the attribute information of the target pedestrian into a feature extraction network, and extracting the target pedestrian's features by the feature extraction network; The step of extracting the target pedestrian's features by the feature extraction network includes: Processing the attribute information by sharing the CNN to obtain a first processing result; Differentiate the first processing results by labels to obtain second processing results of different types; Inputting the different types of second processing results into an independent network for feature extraction to obtain a third processing result; Perform feature weighted summation processing on the third processing result obtained by the independent network processing to obtain a fourth processing result; Outputting the characteristics of the target pedestrian; The inference steps of the feature extraction network include: Convert the trained data into a TensorRT inference model; Perform Int8 quantization on the TensorRT inference model; Inferring the result of the quantization processing through TensorRT to obtain the characteristics of the target pedestrian; The attribute information includes the pedestrian's whole body area, the pedestrian's head area, the pedestrian's upper body area, the pedestrian's lower body area, the pedestrian's category and the pedestrian's confidence.

2. The method for pedestrian re-identification based on deep learning according to claim 1, characterized in that: The performing human target detection on the source image data to determine the target pedestrian includes: Acquire real-time video sequences to obtain multiple source image data; Human target detection is performed on each of the source image data through the YOLOv5 network to determine multiple target pedestrians in each of the source image data.

3. The method for pedestrian re-identification based on deep learning according to claim 1, characterized in that: The performing attribute detection on the target pedestrian to determine the local area attribute information of the target pedestrian includes: The attribute of the target pedestrian is detected through the YOLOv5 network to determine the local area attribute information of the target pedestrian.

4. The method for pedestrian re-identification based on deep learning according to claim 1, characterized in that: The method also includes the step of constructing a pedestrian feature library.

5. The method for pedestrian re-identification based on deep learning according to claim 1, characterized in that: The determining of the similarity between the feature of the target pedestrian and each feature in the pedestrian feature library includes: Acquire attribute information of the target pedestrian, and filter features in the pedestrian feature library according to the attribute information; Extracting the candidate features in the filtered pedestrian feature library, calculating the vector distance between the candidate features and the features of the target pedestrian, and determining the similarity between the features of the target pedestrian and each feature in the pedestrian feature library; The filtering conditions include age group conditions, gender conditions and hair length conditions.

6. A pedestrian re-identification device based on deep learning, characterized in that: include: The target detection module is used to perform human target detection on the source image data and determine the target pedestrian; An attribute detection module, used to perform attribute detection on the target pedestrian and determine local area attribute information of the target pedestrian; A feature extraction module is used to extract features from a local area of ​​the target pedestrian, obtain features of the local area and perform weighted summation on the features to determine the features of the target pedestrian; A similarity calculation module, used to determine the similarity between the features of the target pedestrian and each feature in the pedestrian feature library; A recognition module, used to determine a pedestrian re-recognition result according to the similarity; The extracting features of the local area of ​​the target pedestrian, obtaining features of the local area and performing weighted summation on the features to determine the features of the target pedestrian includes: Inputting the attribute information of the target pedestrian into a feature extraction network, and extracting the target pedestrian's features by the feature extraction network; The step of extracting the target pedestrian's features by the feature extraction network includes: Processing the attribute information by sharing the CNN to obtain a first processing result; Differentiate the first processing results by labels to obtain second processing results of different types; Inputting the different types of second processing results into an independent network for feature extraction to obtain a third processing result; Perform feature weighted summation processing on the third processing result obtained by the independent network processing to obtain a fourth processing result; Outputting the characteristics of the target pedestrian; The inference steps of the feature extraction network include: Convert the trained data into a TensorRT inference model; Perform Int8 quantization on the TensorRT inference model; Inferring the result of the quantization processing through TensorRT to obtain the characteristics of the target pedestrian; The attribute information includes the pedestrian's whole body area, the pedestrian's head area, the pedestrian's upper body area, the pedestrian's lower body area, the pedestrian's category and the pedestrian's confidence.

7. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 5.

Images