Method and system for re-identifying pedestrians changing clothes based on spatial consistency

Abstract

The present invention provides a method, system, electronic device and storage medium for re-identifying pedestrians changing clothes based on spatial consistency, which belongs to the field of computer vision technology. By generating new pedestrian images wearing the same clothes in different spaces, a pedestrian re-identification system for changing clothes based on spatial consistency can preliminarily meet the requirements of spatial consistency; a view label generation module is then used to generate a view label for each image, and combined with clothing labels, joint learning based on clothing and view angle adversarial loss is used to make the model pay more attention to features unrelated to clothing and view angle through a penalty mechanism, so that more robust and discriminative features suitable for clothing changing scenes can be extracted; the present invention makes full use of features unrelated to clothing and view angle, and realizes re-identification of pedestrians changing clothes in various scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method for re-identifying pedestrians who change clothes based on spatial consistency. Background Art

[0002] With the development of machine learning technology and deep learning models in the field of computer vision, human recognition based on deep learning methods has also been increasingly widely used. Among them, pedestrian re-identification (Person ReID) is a representative technology of human recognition and is widely used in the fields of smart security, target association, and smart retail. It can also be used as an effective alternative technology when face recognition fails. The purpose of the pedestrian re-identification task is to solve the problem of cross-camera retrieval of target pedestrians. Specifically, it is to determine whether there is a specific pedestrian in the images or video sequences taken by different cameras based on computer vision technology.

[0003] Person re-identification mainly solves the problem of matching pedestrians across cameras and scenes, and then retrieves pedestrians with the same identity from different camera images. However, in actual application scenarios, objective factors such as image resolution, viewing angle, lighting, and posture changes hinder the person re-identification algorithm from extracting effective identification features, making pedestrian re-identification a challenging task.

[0004] At present, pedestrian re-identification algorithms are mainly based on convolutional neural networks to extract image features, and then perform classification and feature matching; among them, identity recognition and gait recognition by changing clothes need to rely on multimodal input or learn through decomposition representation; however, there are the following disadvantages: 1) Multimodal input requires additional models or devices to capture multimodal information, and learning disentangled representation is time-consuming; 2) The original RGB modality contains rich information, including both clothing-related information and clothing-irrelevant information, but the feature maps learned by the existing identity recognition by changing clothes can only focus on some simple features unrelated to clothes, such as faces; while other key features unrelated to clothes are ignored; 3) The existing pedestrian re-identification algorithms are mainly suitable for scenes with short time spans. It is assumed that a pedestrian wears the same clothes under different cameras, that is, the pedestrian's clothes are used as an important feature for model extraction; resulting in pedestrians being unable to be recognized in the following scenarios; for example, pedestrians should change clothes for scene changes, criminals deliberately change their dressing style to escape pursuit, and participants in collective activities wear uniforms.

[0005] Therefore, there is an urgent need for a method and system for re-identifying pedestrians who change clothes based on spatial consistency. Summary of the invention

[0006] The present invention provides a method, system, electronic device and storage medium for re-identifying pedestrians changing clothes based on spatial consistency, so as to overcome at least one technical problem existing in the prior art.

[0007] To achieve the above object, the present invention provides a method for re-identifying pedestrians who change clothes based on spatial consistency, the method comprising:

[0008] Obtaining an original pedestrian image of a pedestrian to be identified;

[0009] Based on the original image of the pedestrian to be identified, obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture;

[0010] Inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified;

[0011] Performing similarity matching between the features to be retrieved and the features of pedestrian images in the retrieval library, and sorting the features of pedestrian images in descending order according to the similarity scores;

[0012] According to the preset rules, the pedestrian images corresponding to the pedestrian image features in the set order are used as the output results of the clothing-changing pedestrian re-recognition.

[0013] Further, preferably, based on the original image of the pedestrian to be identified, the method for obtaining the pedestrian image of the pedestrian to be identified in different clothes in the same scene and the same posture includes:

[0014] Based on the original image of the pedestrian to be identified, a clothing region mask image and coordinates of key points of the human skeleton of the pedestrian to be identified are obtained; wherein the method for obtaining the clothing region mask image comprises: annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be identified; locating the position of the clothing region in the original image by using the clothing label information of the human body semantic segmentation map, and distinguishing the clothing region from the region other than the clothing by using binarization to obtain a clothing region mask image;

[0015] Obtain an image of a pedestrian to be identified whose clothes are different from those in the original image, locate the clothing area of ​​the pedestrian image whose clothes are different from those in the original image by using the coordinates of key points of the human skeleton, and generate a perspective transformation matrix;

[0016] By using the perspective transformation matrix and the clothing region mask image, pedestrian images of the pedestrian to be identified in different clothing in the same scene and the same posture are obtained.

[0017] Further, preferably, the training method of the clothing-changing pedestrian re-identification model includes:

[0018] Obtaining the original pedestrian images to be processed and the corresponding image labels in the pedestrian image dataset; wherein the image labels include clothing labels, identity labels and camera view labels;

[0019] Obtain a clothing area mask image and coordinates of key points of the human skeleton of a pedestrian to be processed; wherein the method for obtaining the clothing area mask image of the pedestrian to be processed comprises obtaining another pedestrian image with the same identity label as the original pedestrian image to be processed, annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be processed; and obtaining a clothing area mask image based on the human body semantic segmentation map;

[0020] Based on the coordinates of the key points of the human skeleton, coordinates corresponding to the perspective transformation and coordinates of the human body perspective are obtained; using the coordinates corresponding to the perspective transformation and coordinates of the human body perspective and the clothing area mask image, clothes in two original pedestrian images with the same identity label are exchanged to generate a new pedestrian image; and based on the coordinates of the human body perspective, a perspective label corresponding to the new pedestrian image is obtained;

[0021] Based on the original pedestrian image and the corresponding image label, as well as the new pedestrian image and the corresponding view label, the original feature representation is obtained through a feature extraction network; the features that are independent of clothes and view are mined from the original pedestrian image using an adversarial loss based on clothes and view;

[0022] The clothing-changing pedestrian re-identification model is trained based on the original feature representation and features that are independent of clothing and viewing angle, and a penalty mechanism is used to make the clothing-changing pedestrian re-identification model pay more attention to features that are independent of clothing and viewing angle.

[0023] Further, preferably, the loss function corresponding to the clothing-changing pedestrian re-identification model is composed of a first-stage loss function and a second-stage loss function;

[0024] The first stage loss function is used to optimize the clothing classifier by minimizing the clothing classification loss and the clothing label; and to optimize the view classifier by minimizing the view classification loss and the view label;

[0025] The second-stage loss function incorporates a penalty mechanism to penalize features related to clothing and viewpoint, and the degree of penalty is positively correlated with the classification efficiency of the clothing classifier and the viewpoint classifier.

[0026] Further, preferably, the first stage loss function is implemented by the following formula:

[0027]

[0028] in, Used to predict clothes; Used to predict perspective; Indicates clothing label; Indicates the viewpoint label; , N represents the batch size, N C Represents the number of clothing categories in the training set; Indicates samples with the same identity and the same clothes. represents samples with the same identity, , N V Represents the number of view categories in the training set; Represents samples with the same identity and the same perspective. is the temperature parameter. The second stage loss function is implemented by the following formula:

[0029]

[0030] in, Used to predict clothes; Used to predict perspective; Used to predict identity; They represent identity tags, clothing tags, and perspective tags respectively; L ID Indicates identity loss, L CA Indicates loss of clothing, L VA Indicates the perspective loss.

[0031] Further, preferably, the clothing-changing pedestrian re-identification model training further includes:

[0032] The server assigns an initial value to each selected feature. During the federated learning training process, the features corresponding to the deducted initial values ​​are removed from the federated learning to ensure training efficiency.

[0033] Further, preferably, the method for obtaining the viewpoint label corresponding to the new pedestrian image based on the coordinates of the human viewpoint includes:

[0034] Based on the coordinates of the key points of the human skeleton, the coordinate points of the nose, left eye, right eye, left ear, right ear, left foot and right foot in the original pedestrian image are obtained respectively;

[0035] Determine the center point of the body according to the average value of the coordinate points of the nose, left eye, right eye, left ear, and right ear; and then determine the vector between the feet according to the difference between the coordinate points of the left foot and the right foot;

[0036] Obtaining a transformed vector according to a vector between the center point of the body and the foot;

[0037] Calculate the radian angle between the transformed vector and the center point using a function;

[0038] A viewing angle label is determined according to the arc angle.

[0039] In order to solve the above problems, the present invention also provides a pedestrian re-identification system based on spatial consistency, comprising:

[0040] An acquisition unit, used for acquiring an original pedestrian image of a pedestrian to be identified;

[0041] A retrieval feature determination unit is used to obtain pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture based on the original image of the pedestrian to be identified; the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture are input into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified;

[0042] The recognition unit is used to perform similarity matching between the features to be retrieved and the pedestrian image features in the retrieval library, and to sort the pedestrian image features in descending order according to the similarity scores; according to preset rules, the pedestrian images corresponding to the pedestrian image features in the set order are used as the output results of the clothing-changing pedestrian re-recognition.

[0043] In order to solve the above problems, the present invention also provides an electronic device, which includes: a memory storing at least one instruction; and a processor executing the instructions stored in the memory to implement the steps in the above-mentioned method for re-identifying pedestrians who change clothes based on spatial consistency.

[0044] The present invention also protects a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for re-identifying pedestrians who change clothes based on spatial consistency.

[0045] The present invention provides a method, system, electronic device and storage medium for re-identifying pedestrians changing clothes based on spatial consistency, which has the following beneficial effects:

[0046] 1) The present invention mines features related to clothes and perspective from RGB images through an adversarial loss based on perspective and clothes, and uses a penalty mechanism similar to that of the GAN network to make the clothing-changing pedestrian re-identification model pay more attention to features unrelated to clothes and perspective; it makes full use of features unrelated to clothes and perspective; it can extract more robust and discriminative features in various clothing-changing scenarios, and thus realizes clothing-changing pedestrian re-identification in various scenarios; 2) Compared with multi-modal input, the present invention only needs to input RGB images, and does not need to rely on additional models or equipment to capture multi-modal information, and can efficiently adapt to various clothing-changing scenarios; 3) The present invention has achieved excellent results in the test of relevant clothing-changing pedestrian re-identification data sets, and has the technical effects of higher recognition performance, higher stability, lower testing cost, and higher testing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 2 is a flow chart of a method for re-identifying pedestrians who have changed clothes based on spatial consistency according to an embodiment of the present invention;

[0048] Figure 2 2 is a schematic diagram showing the principle of a method for re-identifying pedestrians who have changed clothes based on spatial consistency according to an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of output results of a method for re-identifying pedestrians who have changed clothes based on spatial consistency according to an embodiment of the present invention;

[0050] Figure 4 is a logical structure block diagram of a system for re-identifying pedestrians changing clothes based on spatial consistency according to an embodiment of the present invention;

[0051] Figure 5 Schematic diagram of the internal structure of an electronic device for implementing a method for re-identifying a pedestrian who has changed clothes based on spatial consistency according to an embodiment of the present invention.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0054] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology and computer vision technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0055] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0056] Specifically, as an example, Figure 1 A flowchart of a method for re-identifying a pedestrian who has changed clothes based on spatial consistency provided by an embodiment of the present invention. Figure 1 As shown, the present invention provides a method for re-identifying a pedestrian changing clothes based on spatial consistency, which can be performed by a device, and the device can be implemented by software and / or hardware. The method for re-identifying a pedestrian changing clothes based on spatial consistency includes steps S110 to S140.

[0057] Specifically, S110, obtaining an original pedestrian image of a pedestrian to be identified; S120, based on the original pedestrian image of the pedestrian to be identified, obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture; S130, inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified; S140, performing similarity matching between the features to be retrieved and the pedestrian image features in the retrieval library, and sorting the pedestrian image features in descending order according to the similarity scores; according to preset rules, taking the pedestrian images corresponding to the pedestrian image features in the set order as the output result of the clothing-changing pedestrian re-identification.

[0058] Figure 2 FIG. 1 is a schematic diagram showing the principle of a method for re-identifying a pedestrian who has changed clothes based on spatial consistency according to an embodiment of the present invention; Figure 2 As shown, the process of building a clothing-changing pedestrian re-identification model in the clothing-changing pedestrian re-identification method based on spatial consistency is divided into a jigsaw clothing-changing module, a viewpoint label generation module, a feature extraction backbone network, a clothing-based adversarial learning module, and a viewpoint-based adversarial learning module according to the functions. That is to say, for the clothing-changing pedestrian re-identification model of the present invention, in the first part, pedestrian images wearing different clothes in the same scene and the same posture are first obtained according to the original pedestrian image and the image label of the original pedestrian image, and the pedestrian images wearing different clothes are exchanged with each other using the jigsaw clothing-changing module; and the new pedestrian image generated after the clothing-changing is determined; and then the viewpoint label generation module is used to obtain the corresponding viewpoint label for the new pedestrian image. In the second part, the original pedestrian image and its corresponding image label, and the new pedestrian image obtains its corresponding viewpoint label and inputs them into the feature extraction backbone network for feature extraction. These features include features related to clothes and viewpoints, and also include features unrelated to clothes and viewpoints. For features unrelated to clothes and viewpoints, low-level features such as edges, textures, shapes, etc. can be included, and more advanced semantic information, such as the outline and posture of pedestrians, can also be included. In the third part, the loss function is used to train the clothing-changing pedestrian re-identification model based on the extracted features. Specifically, in addition to the identity loss, perspective loss and clothing loss, the loss function includes; the clothing-changing pedestrian re-identification method based on spatial consistency of the present invention also mines features related to clothing and perspective from RGB images by using an adversarial loss based on perspective and clothing; through a penalty mechanism similar to the GAN network, the model pays more attention to features that are not related to clothing and perspective, and the degree of penalty is positively correlated with the efficiency of the clothing classifier and the perspective classifier. Specifically, the jigsaw clothing-changing module is used to generate images wearing the same clothes in different spaces, so that the clothing-changing pedestrian re-identification system based on spatial consistency can initially meet the requirements of spatial consistency.

[0059] Specifically, the training method of the clothing-changing pedestrian re-identification model includes steps S131-S135.

[0060] S131. Obtaining original pedestrian images to be processed and corresponding image labels in a pedestrian image dataset; wherein the image labels include clothing labels, identity labels, and camera view labels.

[0061] S132. Obtain a clothing area mask image and coordinates of key points of the human skeleton of the pedestrian to be processed; wherein, the method for obtaining the clothing area mask image of the pedestrian to be processed includes obtaining another pedestrian image with the same identity label as the original pedestrian image to be processed, annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be processed; and obtaining the clothing area mask image based on the human body semantic segmentation map.

[0062] Specifically, the data of the pedestrian image dataset is processed in small batches through the pre-trained human semantic parsing model, and the semantic information of each part of the human body is annotated to obtain the corresponding human semantic segmentation map. It should be noted that the human semantic parsing model can be but is not limited to the Self-Correction for Human Parsing (SCHP) model. Human parsing is a sophisticated semantic segmentation task, and its purpose is to assign each image pixel in the human body to a semantic category, such as arms, legs, skirts, skirts, etc. It should be noted that in order to obtain more fine-grained segmentation semantic annotations, the pedestrian image is divided into "background, head, hair, sunglasses, tops, shirts, pants, skirts, belts, left shoes, right shoes, faces, left legs, right legs, left arms, right arms, backpacks, scarves" for a total of 18 semantic annotation parts, combined with specific task conditions, such as Figure 2 As shown, 18 semantic parts are used in combination.

[0063] The method for acquiring a clothing region mask image comprises the following steps: using a human body semantic segmentation map with semantic labels, locating a local position of clothing according to clothing label information of the semantic segmentation; distinguishing clothing regions from other regions by performing a binarization process; wherein the pixel portion of the human body semantic segmentation map belonging to the clothing region is set to 1, and the pixel portions of other regions are set to 0; and the binarized image after the binarization process is used as a clothing region mask image.

[0064] S133. Based on the coordinates of the key points of the human skeleton, the coordinates corresponding to the perspective transformation and the coordinates of the human body's viewing angle are obtained; the coordinates corresponding to the perspective transformation and the coordinates of the human body's viewing angle and the clothing area mask image are used to exchange the clothes in two original pedestrian images with the same identity label to generate a new pedestrian image; and the viewing angle label corresponding to the new pedestrian image is obtained based on the coordinates of the human body's viewing angle.

[0065] In a specific embodiment, a method for obtaining a viewpoint label corresponding to a new pedestrian image based on the coordinates of the human viewpoint includes:

[0066] S1331. Based on the coordinates of the key points of the human skeleton, the coordinate points of the nose, left eye, right eye, left ear, right ear, left foot and right foot areas in the original pedestrian image are respectively obtained.

[0067] Specifically, further, the coordinates used for perspective transformation in the key points of the human skeleton and the coordinate acquisition method for calculating the human body's viewing angle include obtaining 17 skeleton nodes of the human body with the help of a pre-trained Alpha-Pose model. Specifically, the pedestrian image data set is processed in small batches through the pre-processed human posture framework (Alpha-Pose), and each joint of the human body is annotated to obtain the corresponding coordinates of the human skeleton joints. It should be noted that the human posture framework will generate 17 human skeleton key point coordinates including left shoulder, right shoulder, left hip, right hip, nose, left eye, right eye, left ear, right ear, left foot, right foot, etc., and save the coordinates in a .json file.

[0068] According to the built-in labels in the Alpha-Pose model, the coordinates of the four points of left shoulder, right shoulder, left hip and right hip are obtained to prepare for the subsequent perspective transformation; according to the built-in labels in the Alpha-Pose model, the coordinates of the seven points of nose, left eye, right eye, left ear, right ear, left foot and right foot are obtained to prepare for the subsequent perspective label generation module.

[0069] S1332. Find another image with the same identity label as the original input image, obtain the top mask of the original input image through the human body parsing model, and obtain the left shoulder, right shoulder, left hip, and right hip coordinates of the pair of images through the human body posture estimation framework.

[0070] S1333. Calculate the perspective transformation matrix through the 4*2 key points, and implement the puzzle clothing change through the top mask and the perspective transformation matrix.

[0071] That is to say, we randomly find another image with the same identity, locate the area of ​​the person's upper clothes through the coordinates of the left shoulder, right shoulder, left hip, and right hip, and generate a perspective transformation matrix. We use the perspective transformation matrix and the clothing area mask to obtain an image of the same identity changing clothes. By generating images of people wearing the same clothes in different spaces, the spatial consistency-based clothing-changing person re-identification system can initially meet the spatial consistency requirements.

[0072] In the specific implementation process, the clothing area mask image is used to locate the clothing area of ​​the corresponding original pedestrian image, and the four points of the left shoulder, right shoulder, left hip and right hip in the human skeleton key points are used to locate the position of the clothing on the upper part of the human body. Since the perspective transformation matrix (M) is defined by 8 constants (degrees of freedom), in order to find the matrix, it is first necessary to select the four points of the left shoulder, right shoulder, left hip and right hip in each of the two input images, respectively defined as a_corners and b_corners, and then use cv2. getPerspectiveTransform (a_corners, b_corners) to calculate the perspective transformation matrix M. The original input image a and the perspective transformation matrix M are used to perform the first perspective transformation, and the output is defined as a_transformed. Then, the upper clothing mask of the original input image a and the perspective matrix M are used to perform the second perspective transformation, and the output is defined as a_mask_transformed. The two are multiplied to obtain the shape of the clothes in the original image a that will be worn after the image b. Then, since the top mask is in binary form, the mask of image a excluding the top is obtained through 1-a_mask_transformed, and it is multiplied by image b to obtain the area of ​​image b excluding the top. Finally, the two are added a_transformed * a_mask_transformed + (1 - a_mask_transformed) * image_b to obtain the image after changing clothes in the same area, so as to achieve spatial consistency.

[0073] S1334, then obtain the coordinate points of the nose, left eye, right eye, left ear, right ear, left foot, and right foot areas of the image through the human posture estimation framework, obtain the human perspective by calculation and convert it into a new perspective label. After obtaining the coordinate points of the nose, left eye, right eye, left ear, right ear, and left foot and right foot according to the .json file containing the coordinates of the key points of the human skeleton described in step S1331; determine the center point of the body according to the average value of the coordinate points of the nose, left eye, right eye, left ear, and right ear; then determine the vector between the feet according to the difference between the coordinate points of the left foot and the right foot; obtain the converted vector according to the center point of the body and the vector between the feet; specifically, subtract the coordinates of the center point of the body from the vector between the feet to obtain the converted vector so as to realize the conversion of the vector to the coordinate system of the center point of the body. Use a function to calculate the radian angle of the converted vector and the center point; determine the perspective label according to the radian angle.

[0074] Specifically, the np.arctan2() function is used to calculate the radian angle of the converted vector, that is, to calculate the angle (radian) between the vector and the center point. The np.degrees() function is then used to convert the radian angle to a degree, and finally the direction is determined according to the range of the angle. If the angle is between -45 and 45, 0 is returned, if the angle is between 45 and 135, 1 is returned, if the angle is between -135 and -45, 2 is returned, otherwise 3 is returned. Labels related to the viewing angle are generated in sequence.

[0075] Specifically, the puzzle-changing clothing module is used to generate images of people wearing the same clothes in different spaces, so that the clothing-changing pedestrian re-identification system based on spatial consistency can initially meet the requirements of spatial consistency.

[0076] S134. Based on the original pedestrian image and the corresponding image label, as well as the new pedestrian image and the corresponding view label, obtain the original feature representation through the feature extraction network; and use the adversarial loss based on clothing and view to mine the features of the original pedestrian image that are independent of clothing and view.

[0077] S135. Train the clothing-changing pedestrian re-identification model based on the original feature representation and the features unrelated to clothing and viewpoint, and make the clothing-changing pedestrian re-identification model pay more attention to the features unrelated to clothing and viewpoint through a penalty mechanism. That is, the adversarial loss is essentially a multi-positive classification loss, in which all viewpoint classes and clothing classes of the same identity are mutually positive. Use the adversarial loss based on viewpoint and clothing to mine the features related to clothing and viewpoint from the RGB image, and use a penalty mechanism similar to the GAN network to make the model pay more attention to the features unrelated to clothing and viewpoint.

[0078] In the specific implementation process, the training of the clothing-changing pedestrian re-identification model also includes that the server assigns an initial value to each screened feature, and during the federated learning training process, the features corresponding to the deducted initial values ​​are removed from the federated learning to ensure training efficiency.

[0079] In the specific implementation process, the loss function corresponding to the clothing-changing pedestrian re-identification model consists of a first-stage loss function and a second-stage loss function; the first-stage loss function is used to optimize the clothing classifier by minimizing the clothing classification loss and clothing labels; and to optimize the perspective classifier by minimizing the perspective classification loss and perspective labels; the second-stage loss function incorporates a penalty mechanism to penalize features related to clothing and perspective, and the degree of penalty is positively correlated with the classification efficiency of the clothing classifier and the perspective classifier.

[0080] Further, preferably, the first stage loss function is implemented by the following formula:

[0081]

[0082] in, Used to predict clothes; Used to predict perspective; Indicates clothing label; Indicates the viewpoint label; , N represents the batch size, N C Represents the number of clothing categories in the training set; Indicates samples with the same identity and the same clothes. represents samples with the same identity, , N V Represents the number of view categories in the training set; Represents samples with the same identity and the same viewpoint. is the temperature parameter.

[0083] Compared with the first-stage loss function, the second-stage loss function incorporates a corresponding penalty mechanism to penalize features related to clothing and viewpoint. The degree of penalty is positively correlated with the classification efficiency of clothing classifier and viewpoint classifier. The overall loss function consists of three parts. It aims to make the model learn features that are unrelated to clothing and viewpoint. To this end, it is necessary to penalize the prediction ability of the clothing-changing pedestrian re-identification model. L VA and L V as well as L CA and L C are two pairs of opposite functions,L VA and L CA is a multi-positive classification loss, where all view classes and clothing classes belonging to the same identity are mutually positive.

[0084] In general, the second stage loss function is implemented by the following formula:

[0085]

[0086] in, Used to predict clothes; Used to predict perspective; Used to predict identity; They represent identity tags, clothing tags, and perspective tags respectively; L ID Indicates identity loss, L CA Indicates loss of clothing, L VA Indicates the perspective loss.

[0087]

[0088]

[0089] in, ; ; and is the set of clothing classes and viewpoint classes that have the same identity (different identity) as fi; K is The number of classes in and is the weight of the cross entropy loss between the cth clothing class and the vth view class. The positive class with the same clothing (view) and the positive class with different clothing (view) have the same weight, that is, . . Clothing labels that indicate the same identity, Indicates view labels with the same identity. Represents an identity tag.

[0090] In summary, the loss function includes two stages. The purpose of the loss function in the first stage is to optimize the clothing classifier by minimizing the clothing classification loss and clothing labels, and to optimize the view classifier by minimizing the view classification loss and view labels; the loss function in the second stage aims to make the model learn features that are independent of clothing and view; the former is the basis of the latter, and the latter is the development of the former. In other words, through the joint training of the jigsaw clothing-changing module, the view label generation module, the backbone network, and the clothing- and view-based adversarial loss, the penalty mechanism makes the model pay more attention to features that are independent of clothing and view, and can extract more robust and discriminative features that are suitable for clothing-changing scenarios, thereby obtaining a trained clothing-changing pedestrian re-identification model.

[0091] In a specific implementation process, the method for re-identifying pedestrians who change clothes based on spatial consistency includes steps S110 to S140.

[0092] S110: Obtain an original pedestrian image of a pedestrian to be identified.

[0093] It should be noted that, unlike the clothing-changing pedestrian re-identification method in the prior art, the pedestrian image of the present invention can have different viewing angles, that is, the original pedestrian image of the pedestrian to be identified and the pedestrian image in the retrieval library are taken by different cameras.

[0094] Specifically, in scenes where it is difficult to capture facial images, such as those with occlusions, side angles, and long distances, the pedestrian image to be retrieved is the image of the pedestrian to be identified. The acquisition device can be, but is not limited to, a digital camera, a surveillance camera, a tablet computer, a mobile phone, and the like.

[0095] S120. Based on the original image of the pedestrian to be identified, obtain pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture.

[0096] A method for obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture based on the original image of the pedestrian to be identified, comprising: S1201, obtaining a clothing area mask image and coordinates of key points of the human skeleton of the pedestrian to be identified based on the original image of the pedestrian to be identified; wherein the method for obtaining the clothing area mask image comprises: annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be identified; locating the position of the clothing area in the original image using the clothing label information of the human body semantic segmentation map, and distinguishing the clothing area from the area other than the clothes using binarization to obtain a clothing area mask image.

[0097] S1202, obtaining an image of a pedestrian to be identified whose clothes are different from those in the original image, locating the clothing area of ​​the pedestrian image that is different from those in the original image through the coordinates of key points of the human skeleton, and generating a perspective transformation matrix.

[0098] S1203: Using the perspective transformation matrix and the clothing region mask image, obtain pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture.

[0099] S130, inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified.

[0100] The original feature representation is obtained from the original pedestrian image through the feature extraction backbone network in the clothing-changing pedestrian re-identification model. Specifically, the ResNet50 pre-trained on ImageNet is used as the feature extraction backbone network, and the original pedestrian image is input into ResNet50 to obtain the original feature representation. .

[0101] S140, performing similarity matching between the features to be retrieved and the features of pedestrian images in the retrieval library, and sorting the features of pedestrian images in descending order according to the similarity scores; according to preset rules, taking the pedestrian images corresponding to the pedestrian image features in the set order as the output result of the clothing-changing pedestrian re-identification.

[0102] Specifically, the pedestrian image to be retrieved is input into the trained clothing-changing pedestrian re-identification model to obtain a feature vector that can represent the pedestrian image to be retrieved, the feature of the pedestrian image to be retrieved is matched with the features of the pedestrian images in the retrieval library for similarity, the retrieval results are sorted in descending order according to the similarity score, and the sorted results are output as the re-identification result, and finally the pedestrian image with the highest ranking is returned as the retrieval result. In other words, the pedestrian image to be retrieved is input into the trained clothing-changing pedestrian re-identification model to obtain a feature vector that can represent the pedestrian image to be retrieved, the feature of the pedestrian image to be retrieved is matched with the features of the pedestrian images in the retrieval library for similarity, the retrieval results are sorted in descending order according to the similarity score, and the sorted results are output as the re-identification result, and finally the pedestrian image with the highest ranking is returned as the retrieval result. The similarity measurement between features is completed by calculating the Euclidean distance of the normalized features.

[0103] In the specific implementation process, given a pedestrian image, other pedestrian images of the same identity are retrieved in the test set, and a sorted list is returned. The retrieval and matching process is as follows: the feature vector representation of all images in the test set is extracted through the trained model, the similarity of the given retrieval image and all images in the test set is calculated respectively, the retrieval results of the test set are sorted in descending order according to the similarity score, and the retrieval result list is returned in order; it should be noted that the pedestrian images corresponding to the pedestrian image features in the set order are used as the output results of the clothing-changing pedestrian re-identification according to the preset rules; the preset rules can be to select the top ten retrieval results for display, or the top five retrieval results for display, which can be set according to the actual application scenario and is not specifically limited here.

[0104] Figure 3 Schematic diagram of the effect before and after the clothes-changing method of pedestrian re-identification based on spatial consistency according to an embodiment of the present invention; Figure 3 As shown, in order to verify the effectiveness of the clothing-changing pedestrian re-identification method based on spatial consistency of the present invention, as shown in Figure 3 As shown in the figure, the retrieval results of the model on the LTCC and PRCC datasets are shown; each row is a retrieval example. The retrieval process is to calculate the similarity distance between all images in the retrieval library and the query graph given a query graph, and return the top ten most similar images. All returned results may be correct or incorrect, and they are confirmed based on the labels. In the visualization results, the correct retrieval results are marked with T, and the incorrect retrieval results are marked with F. It can be seen that the correct results retrieved by the model can be ranked at the top of the retrieval list, proving that the clothing-changing pedestrian re-identification method based on spatial consistency is very useful for clothing-changing pedestrian re-identification, and the extracted features are effective and robust.

[0105] In summary, the method for re-identifying pedestrians who change clothes based on spatial consistency of the present invention establishes a model for re-identifying pedestrians who change clothes, implements the preliminary work of spatial consistency through the jigsaw clothing-changing module, and makes the spatial consistency more perfect by using the perspective label module and the perspective-based adversarial loss, and then performs joint training in combination with the clothing-based adversarial loss. The penalty mechanism makes the model pay more attention to features that are not related to clothes and perspective, and can extract more robust and discriminative features suitable for clothing-changing scenes. The method for re-identifying pedestrians who change clothes based on spatial consistency of the present invention has lower testing costs and higher efficiency, can efficiently cope with clothing-changing scenes, and has systematic, scientific, robust and generalized technical effects.

[0106] Corresponding to the above-mentioned method for re-identifying pedestrians changing clothes based on spatial consistency, the present invention also provides a system for re-identifying pedestrians changing clothes based on spatial consistency. Figure 4The functional modules of the system for re-identifying pedestrians changing clothes based on spatial consistency according to an embodiment of the present invention are shown.

[0107] like Figure 4 As shown, the spatial consistency-based pedestrian re-identification system 400 provided by the present invention can be installed in an electronic device. According to the functions to be implemented, the spatial consistency-based pedestrian re-identification system 400 can include a data acquisition unit 410, a retrieval feature determination unit 420, and an identification unit 430. The unit described in the present invention can also be called a module, which refers to a series of computer program segments that can be executed by a processor of an electronic device and can complete a fixed function, which is stored in the memory of the electronic device.

[0108] In this embodiment, the functions of each module / unit are as follows:

[0109] An acquisition unit 410 is used to acquire an original pedestrian image of a pedestrian to be identified;

[0110] The retrieval feature determination unit 420 is used to obtain pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture based on the original image of the pedestrian to be identified; input the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified;

[0111] The recognition unit 430 is used to perform similarity matching between the features to be retrieved and the pedestrian image features in the retrieval library, and to sort the pedestrian image features in descending order according to the similarity scores; according to a preset rule, the pedestrian image corresponding to the pedestrian image features in the set order is used as the output result of the clothing-changing pedestrian re-recognition.

[0112] More specific implementations of the above-mentioned system for re-identifying pedestrians changing clothes based on spatial consistency provided by the present invention can all be described with reference to the above-mentioned embodiments of the method for re-identifying pedestrians changing clothes based on spatial consistency, and will not be listed one by one here.

[0113] The above-mentioned clothing-changing pedestrian re-identification system based on spatial consistency provided by the present invention, through the joint training of the jigsaw clothing-changing module, the view label generation module, the feature extraction backbone network, and the adversarial loss based on clothing and view, makes the model pay more attention to the features that are not related to clothing and view. Specifically, the jigsaw clothing-changing module is used to generate images wearing the same clothing in different spaces, so that the clothing-changing pedestrian re-identification system based on spatial consistency can initially meet the requirements of spatial consistency, and then the view label generation module is used to generate the view label of each image, combined with the clothing label, and the joint learning based on the adversarial loss of clothing and view, through the penalty mechanism, the model pays more attention to the features that are not related to clothing and view, and can extract more robust and discriminative features that are suitable for the clothing-changing scene. The clothing-changing pedestrian re-identification method based on spatial consistency of the present invention achieves excellent results in related clothing-changing pedestrian re-identification data sets, such as LTCC and PRCC, and has higher recognition performance and stability.

[0114] like Figure 5 As shown, the present invention provides an electronic device 5 for a method of re-identifying pedestrians who change clothes based on spatial consistency.

[0115] The electronic device 5 may include a processor 50, a memory 51 and a bus, and may also include a computer program stored in the memory 51 and executable on the processor 50, such as a program 52 for re-identifying pedestrians changing clothes based on spatial consistency.

[0116] The memory 51 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a mobile hard disk of the electronic device 5. In other embodiments, the memory 51 may also be an external storage device of the electronic device 5, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 may not only be used to store application software and various types of data installed in the electronic device 5, such as the code of the pedestrian re-identification program for changing clothes based on spatial consistency, but also be used to temporarily store data that has been output or is to be output.

[0117] The processor 50 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips, etc. The processor 50 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 51 (such as a program for re-identifying pedestrians who change clothes based on spatial consistency, etc.), and calls data stored in the memory 51 to execute various functions of the electronic device 5 and process data.

[0118] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 51 and at least one processor 50, etc.

[0119] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device 5, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0120] For example, although not shown, the electronic device 5 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 50 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 5 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0121] Furthermore, the electronic device 5 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the electronic device 5 and other electronic devices.

[0122] Optionally, the electronic device 5 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 5 and to display a visual user interface.

[0123] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0124] The spatial consistency-based clothing-changing pedestrian re-identification program 52 stored in the memory 51 in the electronic device 5 is a combination of multiple instructions. When running in the processor 50, it can achieve: S110, obtaining the original pedestrian image of the pedestrian to be identified; S120, based on the original image of the pedestrian to be identified, obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture; S130, inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified; S140, matching the features to be retrieved with the pedestrian image features in the retrieval library for similarity, and sorting the pedestrian image features in descending order according to the similarity scores; according to preset rules, using the pedestrian images corresponding to the pedestrian image features in the set order as the output result of clothing-changing pedestrian re-identification.

[0125] Specifically, the specific implementation method of the processor 50 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiment is not repeated here. It should be emphasized that in order to further ensure the privacy and security of the above-mentioned pedestrian re-identification program based on spatial consistency, the above-mentioned pedestrian re-identification program based on spatial consistency is stored in the node of the blockchain where the server cluster is located.

[0126] Furthermore, if the module / unit integrated in the electronic device 5 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0127] The embodiment of the present invention also provides a computer-readable storage medium, which may be non-volatile or volatile, and stores a computer program, which, when executed by a processor, implements: S110, obtaining an original pedestrian image of a pedestrian to be identified; S120, based on the original image of the pedestrian to be identified, obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture; S130, inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine a retrieval feature vector of the pedestrian to be identified; S140, performing similarity matching between the features to be retrieved and the pedestrian image features in the retrieval library, and sorting the pedestrian image features in descending order according to the similarity scores; according to preset rules, using the pedestrian images corresponding to the pedestrian image features in the set order as the output result of clothing-changing pedestrian re-identification.

[0128] Specifically, the specific implementation method when the computer program is executed by the processor can refer to the description of the relevant steps in the method for re-identifying pedestrians changing clothes based on spatial consistency in the embodiment, and will not be repeated here.

[0129] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0130] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0132] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0133] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0134] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0135] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for re-identifying pedestrians who change clothes based on spatial consistency, characterized in that: include: Obtaining an original pedestrian image of a pedestrian to be identified; Based on the original pedestrian image of the pedestrian to be identified, obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture; Inputting the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified; Performing similarity matching between the retrieval feature vector and the pedestrian image features in the retrieval library, and sorting the pedestrian image features in descending order according to the similarity scores; According to the preset rules, the pedestrian images corresponding to the pedestrian image features in the set order are used as the output results of the clothing-changing pedestrian re-recognition; The method of obtaining pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture based on the original image of the pedestrian to be identified includes: Based on the original image of the pedestrian to be identified, a clothing region mask image and coordinates of key points of the human skeleton of the pedestrian to be identified are obtained; wherein the method for obtaining the clothing region mask image comprises: annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be identified; locating the position of the clothing region in the original image by using the clothing label information of the human body semantic segmentation map, and distinguishing the clothing region from the region other than the clothing by using binarization to obtain a clothing region mask image; Obtain an image of a pedestrian to be identified whose clothes are different from those in the original image, locate the clothing area of ​​the pedestrian image whose clothes are different from those in the original image by using the coordinates of key points of the human skeleton, and generate a perspective transformation matrix; By using the perspective transformation matrix and the clothing region mask image, pedestrian images of the pedestrian to be identified in different clothing in the same scene and the same posture are obtained.

2. The method for re-identifying pedestrians changing clothes based on spatial consistency as claimed in claim 1, characterized in that: The training method of the clothing-changing pedestrian re-identification model includes: Obtaining the original pedestrian images to be processed and the corresponding image labels in the pedestrian image dataset; wherein the image labels include clothing labels, identity labels and camera view labels; Obtain a clothing area mask image and coordinates of key points of the human skeleton of a pedestrian to be processed; wherein the method for obtaining the clothing area mask image of the pedestrian to be processed comprises obtaining another pedestrian image with the same identity label as the original pedestrian image to be processed, annotating the original image of the pedestrian to be identified with semantic information of human body parts through a preset human semantic parsing model to obtain a human body semantic segmentation map of the pedestrian to be processed; and obtaining a clothing area mask image based on the human body semantic segmentation map; Based on the coordinates of the key points of the human skeleton, coordinates corresponding to the perspective transformation and coordinates of the human body perspective are obtained; using the coordinates corresponding to the perspective transformation and coordinates of the human body perspective and the clothing area mask image, clothes in two original pedestrian images with the same identity label are exchanged to generate a new pedestrian image; and based on the coordinates of the human body perspective, a perspective label corresponding to the new pedestrian image is obtained; Based on the original pedestrian image and the corresponding image label, as well as the new pedestrian image and the corresponding view label, the original feature representation is obtained through a feature extraction network; the features that are independent of clothes and view are mined from the original pedestrian image using an adversarial loss based on clothes and view; The clothing-changing pedestrian re-identification model is trained based on the original feature representation and features that are independent of clothing and viewing angle, and a penalty mechanism is used to make the clothing-changing pedestrian re-identification model pay more attention to features that are independent of clothing and viewing angle.

3. The method for re-identifying pedestrians changing clothes based on spatial consistency as claimed in claim 2, characterized in that: The loss function corresponding to the clothing-changing pedestrian re-identification model is composed of the first-stage loss function and the second-stage loss function; The first stage loss function is used to optimize the clothing classifier by minimizing the clothing classification loss and the clothing label; and to optimize the view classifier by minimizing the view classification loss and the view label; The second-stage loss function incorporates a penalty mechanism to penalize features related to clothing and viewpoint, and the degree of penalty is positively correlated with the classification efficiency of the clothing classifier and the viewpoint classifier.

4. The method for re-identifying pedestrians changing clothes based on spatial consistency as claimed in claim 3, characterized in that: The clothing-changing pedestrian re-identification model training also includes: The server assigns an initial value to each selected feature. During the federated learning training process, the features corresponding to the deducted initial values ​​are removed from the federated learning to ensure training efficiency.

5. The method for re-identifying pedestrians changing clothes based on spatial consistency as claimed in claim 4, characterized in that: A method for obtaining a viewpoint label corresponding to a new pedestrian image based on the coordinates of the human viewpoint, include, Based on the coordinates of the key points of the human skeleton, the coordinate points of the nose, left eye, right eye, left ear, right ear, left foot and right foot in the original pedestrian image are obtained respectively; Determine the center point of the body according to the average value of the coordinate points of the nose, left eye, right eye, left ear, and right ear; and then determine the vector between the feet according to the difference between the coordinate points of the left foot and the right foot; Obtaining a transformed vector according to a vector between the center point of the body and the foot; Calculate the radian angle between the transformed vector and the center point using a function; A viewing angle label is determined according to the arc angle.

6. A system for re-identifying pedestrians changing clothes based on spatial consistency, which executes the method for re-identifying pedestrians changing clothes based on spatial consistency as claimed in any one of claims 1 to 5, characterized in that: include: An acquisition unit, used for acquiring an original pedestrian image of a pedestrian to be identified; A retrieval feature determination unit is used to obtain pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture based on the original pedestrian image of the pedestrian to be identified; the original pedestrian image and the pedestrian images of the pedestrian to be identified in different clothes in the same scene and the same posture are input into a preset clothing-changing pedestrian re-identification model to determine the retrieval feature vector of the pedestrian to be identified; an identification unit, used for performing similarity matching between the retrieval feature vector and the pedestrian image features in the retrieval library, and sorting the pedestrian image features in descending order according to the similarity scores; According to the preset rules, the pedestrian images corresponding to the pedestrian image features in the set order are used as the output results of the clothing-changing pedestrian re-recognition.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps in the method for re-identifying pedestrians changing clothes based on spatial consistency as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for re-identifying pedestrians who change clothes based on spatial consistency as described in any one of claims 1 to 5 is implemented.

Images