Pedestrian re-identification method and device based on image optimization, equipment and storage medium

Abstract

The invention relates to a pedestrian re-identification method and device based on image optimization, equipment and a storage medium, and the method comprises the steps: obtaining a monitoring video, and extracting a video frame image from the monitoring video; detecting a key point of each target pedestrian in the video frame image, and performing image screening based on a detection result of each key point to obtain a target image; constructing a training image set and a test image set based on the target image; optimizing a feature extraction layer, a camera classifier and an identity classifier in the pedestrian re-recognition model through the training image set to obtain a target pedestrian re-recognition model; and obtaining a to-be-retrieved image frame, and retrieving the to-be-retrieved image frame through the target pedestrian re-recognition model to obtain a pedestrian retrieval result. The method solves a problem that the training image quality of a pedestrian re-recognition model is insufficient, improves the accuracy of pedestrian re-recognition, and can be suitable for complex and changeable scenes.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for pedestrian re-identification based on image optimization. Background Art

[0002] Person re-identification is an important computer vision technology that aims to retrieve pedestrians with the same identity from cameras with non-overlapping surveillance scenes. It extracts the appearance features of pedestrians, such as body shape, posture, clothing, etc., and performs feature matching to associate pedestrian identities between different cameras. This technology is widely used in fields such as intelligent transportation, security monitoring, and smart parks. It can effectively improve the efficiency of urban management and play an important role in maintaining public safety.

[0003] With the successful application of deep learning in many fields, pedestrian re-identification based on deep learning has also achieved certain research results. The main research idea is to use deep learning methods to extract feature vectors, then use metric learning to discriminate feature vectors, and use a large amount of labeled data to iterate the model training.

[0004] However, in the training data of the pedestrian re-identification model, there are common problems such as the same pedestrian image having a single angle, blurred pedestrian parts, and severe occlusion. In addition, pedestrian re-identification is a cross-camera matching task, and the brightness, contrast, and background of pedestrian images taken by different cameras will be different. The model only considers extracting the features of the entire pedestrian image, and does not focus on removing information related to the camera scene. In this way, it is difficult to extract more robust discriminant features from pedestrian images of different cameras, resulting in low accuracy of the pedestrian re-identification system, and it is easy to overfit the image of a certain scene, making it difficult to apply to complex and changeable real-world scenes. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a pedestrian re-identification method, device, equipment and storage medium based on image optimization to solve the problem of insufficient image quality for training pedestrian re-identification models, improve the accuracy of pedestrian re-identification, and be applicable to complex and changeable scenarios.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a pedestrian re-identification method based on image optimization, including:

[0007] Acquire surveillance video, and extract video frame images from the surveillance video;

[0008] Detecting the key points of each target pedestrian in the video frame image to obtain a key point detection result, and performing image screening based on each key point detection result to obtain a target image;

[0009] Constructing a training image set and a test image set based on the target image;

[0010] Optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model;

[0011] The image frame to be retrieved is obtained, and the image frame to be retrieved is retrieved by using the target pedestrian re-identification model to obtain a pedestrian retrieval result.

[0012] In order to solve the above technical problems, the embodiment of the present application provides a pedestrian re-identification device based on image optimization, comprising:

[0013] An image extraction unit, used to obtain a surveillance video and extract a video frame image from the surveillance video;

[0014] A key point detection unit, used to detect the key point of each target pedestrian in the video frame image to obtain a key point detection result, and perform image screening based on each key point detection result to obtain a target image;

[0015] An image set construction unit, used to construct a training image set and a test image set based on the target image;

[0016] A re-identification model training unit, used to optimize the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model;

[0017] The model inference unit is used to obtain the image frame to be retrieved, retrieve the image frame to be retrieved through the target pedestrian re-identification model, and obtain the pedestrian retrieval result.

[0018] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that the one or more processors can implement any one of the above-mentioned image-based pedestrian re-identification methods.

[0019] In order to solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned image-based pedestrian re-identification methods is implemented.

[0020] The embodiment of the present invention provides a method, device, equipment and storage medium for pedestrian re-identification based on image optimization. The method includes: obtaining a surveillance video and extracting a video frame image from the surveillance video; detecting the key points of each target pedestrian in the video frame image to obtain a key point detection result, and performing image screening based on each key point detection result to obtain a target image; constructing a training image set and a test image set based on the target image; optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model; obtaining an image frame to be retrieved, and retrieving the image frame to be retrieved through the target pedestrian re-identification model to obtain a pedestrian retrieval result. The embodiment of the present invention performs key point detection on video frame images and can screen images according to the key point detection results, thereby solving the problem of insufficient image quality for training pedestrian re-identification models. At the same time, by optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model, a target pedestrian re-identification model is generated, and pedestrian recognition is performed through the target pedestrian re-identification model, which can effectively improve the accuracy of pedestrian re-identification and can be applied to complex and changeable scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 is a flowchart of the implementation process of the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the overall implementation process of the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0024] Figure 3 is a flowchart for implementing the first sub-process in the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of the preferred implementation process of the exemplary image provided in the embodiment of the present application;

[0026] Figure 5 is a flowchart for implementing the second sub-process in the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0027] Figure 6is a flowchart for implementing the third sub-process in the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0028] Figure 7 is a flowchart for implementing the fourth sub-process in the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0029] Figure 8 is a flowchart for implementing the fifth sub-process in the pedestrian re-identification method based on image optimization provided in an embodiment of the present application;

[0030] Fig. 9 is a schematic diagram of a pedestrian re-identification device based on image optimization provided in an embodiment of the present application;

[0031] Fig.10 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0033] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the pedestrian re-identification method based on image optimization provided in the embodiment of the present application is generally executed by a server, and accordingly, the pedestrian re-identification device based on image optimization is generally configured in the server.

[0037] See also Figure 1 and Figure 2 , Figure 1 A specific implementation of the pedestrian re-identification method based on image optimization is shown. Figure 2 It is a schematic diagram of the overall implementation process of the pedestrian re-identification method based on image optimization provided in an embodiment of the present application.

[0038] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 The process sequence shown is limited to the following steps:

[0039] S1: Acquire a surveillance video, and extract a video frame image from the surveillance video.

[0040] Specifically, the surveillance video is a video in which pedestrians exist, and video frame images are extracted from the surveillance video. The video frame images are used to screen training images for the pedestrian re-identification model.

[0041] S2: Detect key points of each target pedestrian in the video frame image to obtain key point detection results, and perform image screening based on each key point detection result to obtain a target image.

[0042] Specifically, the key points of each target pedestrian in the video frame image are detected to obtain the key point detection results, and the image is screened based on each key point detection result to obtain the target image. The target image can be used as training data for the subsequent pedestrian re-identification model.

[0043] See also Figure 3 and Figure 4 , Figure 3 A specific implementation of step S2 is shown. Figure 4 This is a schematic diagram of the preferred implementation process of the exemplary image provided in the embodiment of the present application, which is described in detail as follows:

[0044] S21: performing pedestrian detection on the video frame image using a preset target detection algorithm to generate a pedestrian detection frame of a target pedestrian.

[0045] S22: Tracking the trajectory of the pedestrian detection frame using a multi-target tracking algorithm to obtain a continuous pedestrian image sequence.

[0046] S23: Perform key point detection on the pedestrian image sequence of each target pedestrian to obtain the key point detection result.

[0047] Specifically, the target detection algorithm (such as YOLOv8, YOLOv5, Fast RCNN, etc.) is used to analyze the video frames and generate multiple pedestrian detection frames of target pedestrians. Each pedestrian detection frame contains the location information, category label and confidence score of the target pedestrian. Then, the target trajectory tracking process for the pedestrian detection frame is to track multiple pedestrians in the video through a multi-target tracking algorithm, and record their motion trajectories and identity information to obtain a continuous pedestrian image sequence. Finally, the key point detection is performed on the pedestrian image sequence of each target pedestrian to obtain the key point detection result.

[0048] See also Figure 5 , Figure 5 A specific implementation of step S23 is shown, which is described in detail as follows:

[0049] S231: performing key point recognition on the pedestrian image sequence of each target pedestrian using a skeleton recognition model to obtain a plurality of initial key points;

[0050] S232: Composing the head, torso, legs and feet of the target pedestrian using the initial key points;

[0051] S233: constructing the visibility parameters of the initial key points by presetting the key point score threshold to obtain the key point detection result.

[0052] Specifically, the skeleton recognition model can be a structure such as HRNet, and the specific process of human key point detection includes: selecting 17 key points from the output of the skeleton recognition model as important key points, among which the nose, left ear, right ear, left eye, and right eye constitute the head, the left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, and right hip constitute the torso, the left knee, right knee, left ankle, and right ankle constitute the legs and feet, and setting key point visibility parameters to evaluate the integrity of the human body.

[0053] By setting the key point score threshold, we set its visibility parameter V and analyze the relative position relationship between different important key points. V is in binary form. Taking the nose as an example, if the nose can be predicted, V = 1, otherwise V = 0. Set the key point score threshold a = 0.6, assuming that the predicted score of the nose is s n , then the nose visibility V n The calculation expression is as follows:

[0054]

[0055] The visibility parameters of other body parts can also be calculated using the above calculation formula. The key point detection results include key points of different body parts of the target pedestrian, key point visibility parameters, body parts composed of key points, and visibility parameters of body parts.

[0056] S24: performing image screening on the video frame image based on the key point detection result to obtain a target image.

[0057] Specifically, an image optimization strategy is executed according to the key point detection results, and the best image after optimization is saved for each target pedestrian, thereby obtaining a target image.

[0058] See also Figure 6 , Figure 6 A specific implementation of step S24 is shown, which is described in detail as follows:

[0059] S241: determining the orientation of pedestrians in the image sequence corresponding to the target pedestrian by using the key point detection result, and storing the images corresponding to the same orientation of the pedestrians in the same list, wherein the orientation of the pedestrians includes front orientation, side orientation and back orientation;

[0060] S242: Classifying the images in the list into different levels according to the visibility parameter to obtain images of different levels;

[0061] S243: extracting a preset number of images from the images of different levels in the multiple lists to obtain the target image.

[0062] Specifically, first determine the orientation of the pedestrian in the image sequence corresponding to the target pedestrian. The specific process is as follows: If the height of the pedestrian detection frame is within the preset range, such as greater than or equal to 128 and the width of the pedestrian detection frame is greater than or equal to 64, then: If the cosine value of the angle between the left shoulder key point and the right shoulder key point is also within the preset range, for example, the cosine value is between 0.6 and 1, and the nose key point is at the edge or outside the endpoint of the line segment connecting the left shoulder and the right shoulder, and the difference between the horizontal coordinates of the left shoulder and the right shoulder is less than one-quarter of the width of the pedestrian detection frame, then the pedestrian orientation can be determined to be sideways. If the horizontal coordinates of the left shoulder, left elbow, and left wrist key points are greater than the horizontal coordinates of the right shoulder, right elbow, and right wrist key points, then the pedestrian orientation can be determined to be frontal; otherwise, the pedestrian orientation is back.

[0063] Then, the images that meet the different orientation judgment conditions are temporarily stored in three lists. Then, in each list, three priority levels are set from high to low according to the visibility parameters of the 17 key points, namely: all important key points of the whole body are visible, the head and torso are visible, and the torso and legs are visible. The set number of images is selected from each of the three lists to ensure that each target pedestrian retains images of different angles and with high integrity of the pedestrian area.

[0064] It should be noted that the preset number is set according to actual conditions and is not limited this time.

[0065] S3: constructing a training image set and a test image set based on the target image.

[0066] Specifically, the target image is preprocessed to obtain a preprocessed image, and the preprocessed image is divided according to a preset division ratio to generate a training image set and a test image set. The image preprocessing may include image cropping and image enhancement.

[0067] S4: Optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model.

[0068] Specifically, the pedestrian recognition model is trained using images in the training image set, so as to optimize the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model to obtain the target pedestrian re-identification model.

[0069] See also Figure 7 , Figure 5 A specific implementation of step S7 is shown, which is described in detail as follows:

[0070] S41: extracting features from the images in the training image set through the feature extraction layer to obtain pedestrian features, calculating triplet loss based on the pedestrian features, and optimizing the feature extraction layer through the triplet loss.

[0071] Specifically, an existing deep learning architecture is selected as the feature extraction layer, and the deep learning architecture can be a convolutional neural network CNN, Transformer and its variant structure, etc., for extracting pedestrian features in the training image. After the pedestrian features are extracted, the triplet loss is calculated according to the pedestrian features, and the feature extraction layer is optimized by the triplet loss.

[0072] The basic idea of ​​the triplet loss function is that the distance between positive sample pairs is smaller than the distance between negative sample pairs by a predefined threshold. i , a positive sample x of the same identity j and a negative sample x of a different identity k The calculation expression is as follows:

[0073]

[0074] Among them, ρ is the set distance threshold, d ij Denotes the Euclidean distance of the positive sample pair, d ik Represents the Euclidean distance of negative sample pairs.

[0075] S42: Performing classification prediction according to the pedestrian features by the camera classifier to obtain a classification prediction probability, calculating a camera confusion loss based on the classification prediction probability, and optimizing the camera classifier according to the camera confusion loss.

[0076] Specifically, a camera classifier consisting of a gradient reversal layer and a fully connected layer is constructed after the feature extraction layer to filter the information related to the camera scene in the pedestrian features. The main function of the gradient reversal layer (GRL) is to reverse the gradient back-propagated by the camera classifier when it is passed to the feature extraction layer during the model training process, and work together with the camera confusion loss. To achieve the adversarial effect between the feature extraction layer and the camera classifier, the cross entropy calculation based on gradient reversal is used to continuously encourage the feature extraction layer to reduce the accuracy of the camera classifier, thereby achieving confusion in the camera field, but the camera classifier is sometimes unstable during the training process.

[0077] In order to strengthen the degree of adversarial force, so that the camera classifier can determine its own prediction, whether the current camera category is right or wrong, the feature extraction layer is always kept on the opposite side of the camera classifier during training. Therefore, the camera confusion loss is calculated as:

[0078]

[0079] in, is the camera confusion loss, K represents the number of images in each batch, and N c represents the number of camera categories, c j,i is the classification prediction probability of the camera classifier about the camera, y i For a given label, is the classification probability of the given label. The gradient of the camera confusion loss is inverted by the gradient reversal layer GRL and then back-propagated through the feature extraction layer, aiming to make it difficult for the camera classifier to distinguish the camera source of the pedestrian features, and gradually guide the feature extraction layer to pay more attention to the features of the pedestrian itself through adversarial action.

[0080] S43: The identity classifier is used to classify the pedestrian features to obtain an identity label, and an identity loss is calculated based on the identity label through a cross entropy function, and the identity classifier is optimized according to the identity loss.

[0081] Specifically, an identity classifier consisting of a fully connected layer is built after the feature extraction layer to accurately predict the category of pedestrian features. The calculation expression of the identity loss is:

[0082]

[0083] Where N represents the number of training samples in each batch, and the input image x i The given label is y i , q(k) is a distribution function. When the predicted identity and the target identity are the same, the value of q(k) is 1, otherwise it is 0. The identity loss function is calculated by cross entropy.

[0084] S44: When the feature extraction layer, the camera classifier and the identity classifier are optimized, the target pedestrian re-identification model is obtained.

[0085] Specifically, when the feature extraction layer, the camera classifier and the identity classifier are optimized, the pedestrian re-identification model has been trained, thereby obtaining the target pedestrian re-identification model.

[0086] Furthermore, the target person re-identification model is tested and verified using images in the test image set. During the test and verification, only the feature extraction layer in the target person re-identification model is used, and the extracted pedestrian features are used to calculate the similarity, and the pedestrian retrieval results are obtained according to the similarity sorting.

[0087] S5: Obtain the image frame to be retrieved, and retrieve the image frame to be retrieved using the target pedestrian re-identification model to obtain a pedestrian retrieval result.

[0088] Specifically, after the target pedestrian re-identification model is trained and generated, in the inference stage or the actual application stage, the image frame to be retrieved is obtained, and the image frame to be retrieved is retrieved through the target pedestrian re-identification model to obtain the pedestrian retrieval result.

[0089] See also Figure 8 , Figure 8 A specific implementation of step S5 is shown, which is described in detail as follows:

[0090] S51: Acquire the image frame to be retrieved, extract features of the image frame to be retrieved through the target pedestrian re-identification model, and obtain features of the target pedestrian.

[0091] S52: performing similarity calculation according to the target pedestrian features to obtain similarity values, and sorting the similarity values ​​to obtain sorting results.

[0092] S53: Generate the pedestrian retrieval result based on the sorting result.

[0093] Specifically, in an actual pedestrian retrieval scenario, the image frame to be retrieved is obtained, and the target pedestrian re-identification model is used to extract the features of the image frame to be retrieved to obtain the target pedestrian features. Similarity calculation is performed based on the target pedestrian features to obtain similarity values, and the similarity values ​​are sorted to obtain sorting results. Finally, based on selecting the top similarity values ​​from the sorting results, the pedestrian results corresponding to the top similarities are used as the final pedestrian retrieval results.

[0094] In an embodiment of the present application, a surveillance video is obtained, and a video frame image is extracted from the surveillance video; the key points of each target pedestrian in the video frame image are detected to obtain a key point detection result, and an image is screened based on each of the key point detection results to obtain a target image; a training image set and a test image set are constructed based on the target image; the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model are optimized through the training image set to obtain a target pedestrian re-identification model; an image frame to be retrieved is obtained, and the image frame to be retrieved is retrieved through the target pedestrian re-identification model to obtain a pedestrian retrieval result. The embodiment of the present invention can screen images according to the key point detection results by performing key point detection on the video frame image, thereby solving the problem of insufficient image quality for training the pedestrian re-identification model; at the same time, by optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model, a target pedestrian re-identification model is generated, and pedestrian recognition is performed through the target pedestrian re-identification model, which can effectively improve the accuracy of pedestrian re-identification and can be applied to complex and changeable scenes.

[0095] Please refer to Fig. 9 , as a response to the above Figure 1 The present application provides an embodiment of a pedestrian re-identification device based on image optimization, and the device embodiment is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0096] like Fig. 9 As shown, the pedestrian re-identification device based on image optimization of this embodiment includes: a to-be-processed image acquisition module 61, a face key point detection module 62, a face cropping image generation module 63, a correction image generation module 64 and a target repair image generation module 65, wherein:

[0097] An image extraction unit 61 is used to obtain a surveillance video and extract a video frame image from the surveillance video;

[0098] A key point detection unit 62 is used to detect the key point of each target pedestrian in the video frame image to obtain a key point detection result, and perform image screening based on each key point detection result to obtain a target image;

[0099] An image set construction unit 63, configured to construct a training image set and a test image set based on the target image;

[0100] A re-identification model training unit 64 is used to optimize the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model;

[0101] The model inference unit 65 is used to obtain the image frame to be retrieved, and retrieve the image frame to be retrieved through the target pedestrian re-identification model to obtain the pedestrian retrieval result.

[0102] Further, the key point detection unit 62 includes:

[0103] A pedestrian detection unit, used to perform pedestrian detection on the video frame image by using a preset target detection algorithm to generate a pedestrian detection frame of a target pedestrian;

[0104] A trajectory tracking unit, used for tracking the trajectory of the pedestrian detection frame by a multi-target tracking algorithm to obtain a continuous pedestrian image sequence;

[0105] A key point detection unit, used for performing key point detection on the pedestrian image sequence of each target pedestrian to obtain the key point detection result;

[0106] An image screening unit is used to screen the video frame image based on the key point detection result to obtain a target image.

[0107] Furthermore, the key point detection unit includes:

[0108] An initial key point recognition unit, used for performing key point recognition on the pedestrian image sequence of each target pedestrian through a skeleton recognition model to obtain a plurality of initial key points;

[0109] A pedestrian part composition unit, used to compose the head, torso, legs and feet of the target pedestrian through the initial key points;

[0110] The detection result generating unit is used to construct the visibility parameter of the initial key point by presetting the key point score threshold to obtain the key point detection result.

[0111] Furthermore, the image screening unit comprises:

[0112] A pedestrian orientation recognition unit, used to determine the orientation of pedestrians in the image sequence corresponding to the target pedestrian through the key point detection result, and store the images corresponding to the same pedestrian orientation in the same list, wherein the pedestrian orientation includes front orientation, side orientation and back orientation;

[0113] An image level classification unit, configured to classify the images in the list into different levels according to the visibility parameter, to obtain images of different levels;

[0114] The image extraction unit is used to extract a preset number of images from the images of different levels in a plurality of lists to obtain the target image.

[0115] Furthermore, the re-identification model training unit 64 includes:

[0116] A feature extraction layer optimization unit, configured to extract features from images in the training image set through the feature extraction layer to obtain pedestrian features, calculate triplet losses based on the pedestrian features, and optimize the feature extraction layer through the triplet losses;

[0117] A camera classifier optimization unit, configured to perform classification prediction according to the pedestrian features through the camera classifier to obtain a classification prediction probability, calculate a camera confusion loss based on the classification prediction probability, and optimize the camera classifier according to the camera confusion loss;

[0118] An identity classifier optimization unit, configured to perform identity classification using the pedestrian features of the identity classifier to obtain an identity label, calculate an identity loss based on the identity label using a cross entropy function, and optimize the identity classifier according to the identity loss;

[0119] The optimization completion unit is used to obtain the target pedestrian re-identification model when the feature extraction layer, the camera classifier and the identity classifier are optimized.

[0120] Furthermore, the camera confusion loss is calculated as follows:

[0121]

[0122] in, is the camera confusion loss, K represents the number of images in each batch, and N c represents the number of camera categories, c j,i is the classification prediction probability of the camera classifier about the camera, y i For a given label, c yi,i is the classification probability of the given label.

[0123] Furthermore, the model reasoning unit 65 includes:

[0124] A target pedestrian feature extraction unit is used to obtain the image frame to be retrieved, extract the image frame to be retrieved through the target pedestrian re-identification model to perform feature extraction, and obtain the target pedestrian feature;

[0125] A similarity calculation unit, used to perform similarity calculation according to the target pedestrian feature to obtain similarity values, and to sort the similarity values ​​to obtain sorting results;

[0126] A result generating unit is used to generate the pedestrian retrieval result based on the sorting result.

[0127] In an embodiment of the present application, a surveillance video is obtained, and a video frame image is extracted from the surveillance video; the key points of each target pedestrian in the video frame image are detected to obtain a key point detection result, and an image is screened based on each of the key point detection results to obtain a target image; a training image set and a test image set are constructed based on the target image; the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model are optimized through the training image set to obtain a target pedestrian re-identification model; an image frame to be retrieved is obtained, and the image frame to be retrieved is retrieved through the target pedestrian re-identification model to obtain a pedestrian retrieval result. The embodiment of the present invention can screen images according to the key point detection results by performing key point detection on the video frame image, thereby solving the problem of insufficient image quality for training the pedestrian re-identification model; at the same time, by optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model, a target pedestrian re-identification model is generated, and pedestrian recognition is performed through the target pedestrian re-identification model, which can effectively improve the accuracy of pedestrian re-identification and can be applied to complex and changeable scenes.

[0128] In the embodiment of the present application, the human key point visible judgment rules and specific strategies for image optimization involved in pedestrian image optimization based on human key point detection avoid the high-cost and low-efficiency manual screening process. The camera confusion loss involved in the pedestrian re-identification method based on camera confusion loss improves the adaptability of the pedestrian re-identification model to different cameras without using an image segmentation model.

[0129] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0130] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 which are interconnected and communicated with each other through a system bus. It should be noted that: Fig.10Only a computer device 7 having three components, memory 71, processor 72, and network interface 73, is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0131] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0132] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 7. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as the program code of the pedestrian re-identification method based on image optimization, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0133] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run the program code stored in the memory 71 or process data, such as running the program code of the above-mentioned image-based person re-identification method to implement various embodiments of the image-based person re-identification method.

[0134] The network interface 73 may include a wireless network interface or a wired network interface, and the network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0135] The present application also provides another implementation, namely, providing a computer-readable storage medium, which stores a computer program, and the computer program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned image-based pedestrian re-identification method.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0137] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A pedestrian re-identification method based on image optimization, characterized in that: include: Acquire surveillance video, and extract video frame images from the surveillance video; Detecting the key points of each target pedestrian in the video frame image to obtain a key point detection result, and performing image screening based on each key point detection result to obtain a target image; Constructing a training image set and a test image set based on the target image; Optimizing the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model; The image frame to be retrieved is obtained, and the image frame to be retrieved is retrieved by using the target pedestrian re-identification model to obtain a pedestrian retrieval result.

2. The method for person re-identification based on image optimization according to claim 1, characterized in that: The detecting of the key points of each target pedestrian in the video frame image to obtain a key point detection result, and performing image screening based on each key point detection result to obtain a target image, includes: Performing pedestrian detection on the video frame image using a preset target detection algorithm to generate a pedestrian detection frame of the target pedestrian; Tracking the trajectory of the pedestrian detection frame by a multi-target tracking algorithm to obtain a continuous pedestrian image sequence; Performing key point detection on the pedestrian image sequence of each target pedestrian to obtain the key point detection result; The video frame image is screened based on the key point detection result to obtain a target image.

3. The method for person re-identification based on image optimization according to claim 2, characterized in that: The performing key point detection on the pedestrian image sequence of each target pedestrian to obtain the key point detection result includes: Performing key point recognition on the pedestrian image sequence of each target pedestrian using a skeleton recognition model to obtain a plurality of initial key points; The head, torso, legs and feet of the target pedestrian are formed by the initial key points; The visibility parameters of the initial key points are constructed by presetting the key point score threshold to obtain the key point detection result.

4. The method for person re-identification based on image optimization according to claim 3, characterized in that: The step of performing image screening on the video frame image based on the key point detection result to obtain a target image includes: Determine the orientation of the pedestrian in the image sequence corresponding to the target pedestrian by using the key point detection result, and store the images corresponding to the same orientation of the pedestrian in the same list, wherein the orientation of the pedestrian includes front orientation, side orientation and back orientation; Classifying the images in the list into different levels according to the visibility parameter to obtain images of different levels; A preset number of images are extracted from the images of different levels in a plurality of lists to obtain the target image.

5. The method for person re-identification based on image optimization according to claim 1, characterized in that: The method of optimizing the feature extraction layer, the camera classifier and the identity classifier in the pedestrian re-identification model by using the training image set to obtain a target pedestrian re-identification model includes: Extracting features from the images in the training image set through the feature extraction layer to obtain pedestrian features, calculating triplet loss based on the pedestrian features, and optimizing the feature extraction layer through the triplet loss; Performing classification prediction according to the pedestrian features by the camera classifier to obtain a classification prediction probability, calculating a camera confusion loss based on the classification prediction probability, and optimizing the camera classifier according to the camera confusion loss; Performing identity classification on the pedestrian features by the identity classifier to obtain an identity label, calculating an identity loss based on the identity label by a cross entropy function, and optimizing the identity classifier according to the identity loss; When the feature extraction layer, the camera classifier and the identity classifier are optimized, the target pedestrian re-identification model is obtained.

6. The method for person re-identification based on image optimization according to claim 5, characterized in that: The camera confusion loss is calculated as: in, is the camera confusion loss, K represents the number of images in each batch, and N c represents the number of camera categories, c j,i is the classification prediction probability of the camera classifier about the camera, y i For a given label, c yi,i is the classification probability of the given label.

7. The method for person re-identification based on image optimization according to any one of claims 1 to 6, characterized in that: The step of acquiring the image frame to be retrieved and retrieving the image frame to be retrieved by using the target pedestrian re-identification model to obtain a pedestrian retrieval result includes: Acquire the image frame to be retrieved, extract the image frame to be retrieved by using the target pedestrian re-identification model to perform feature extraction, and obtain the target pedestrian feature; Calculating similarity based on the target pedestrian features to obtain similarity values, and sorting the similarity values ​​to obtain sorting results; The pedestrian retrieval result is generated based on the ranking result.

8. A pedestrian re-identification device based on image optimization, characterized in that: include: An image extraction unit, used to obtain a surveillance video and extract a video frame image from the surveillance video; A key point detection unit, used to detect the key point of each target pedestrian in the video frame image to obtain a key point detection result, and perform image screening based on each key point detection result to obtain a target image; An image set construction unit, used to construct a training image set and a test image set based on the target image; A re-identification model training unit, used to optimize the feature extraction layer, camera classifier and identity classifier in the pedestrian re-identification model through the training image set to obtain a target pedestrian re-identification model; The model inference unit is used to obtain the image frame to be retrieved, retrieve the image frame to be retrieved through the target pedestrian re-identification model, and obtain the pedestrian retrieval result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the pedestrian re-identification method based on image optimization as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for pedestrian re-identification based on image optimization according to any one of claims 1 to 7 is implemented.

Images