Procedures for discriminating against dress codes, procedures for training a person identification model and device

Abstract

The present application provides a method for discriminating against dress codes, a method for training a person re-identification model, and a device.The procedure for discriminating against dress codes comprises: performing human body detection on a target image to obtain an initial human body detection image of a target individual within the target image; performing human body area partitioning on the initial human body detection image to obtain a target human body area image of the target individual; using a person identification model to extract features from the target human body area image of the target individual to obtain an initial feature vector; comparing the initial feature vector with a second feature vector of a target human body area in a dress code template image to obtain a comparison result; and determining whether the target human body area clothing of the target individual complies with a dress code based on the comparison result.

Description

AREA OF TECHNOLOGY

[0001] Embodiments of the present application relate to the field of image processing technologies and in particular to a method for discriminating against dress codes, a method for training a person identification model and a device. STATE OF THE ART

[0002] With the development of science and technology, computer vision technology has become increasingly widespread in life and production. In factories, bank lobbies, and other places, it is necessary to regulate employee dress codes. Pedestrian re-identification technology in computer vision can compare the similarity between a real person image and a given dress code pattern image, thus helping to determine whether the employee's clothing meets requirements.

[0003] However, current pedestrian re-identification technology cannot accurately identify the clothing requirements for every part of the human body. SUMMARY

[0004] Embodiments of the present application provide a method for discriminating against dress codes, a method for training a person identification model, and a device that can solve the problem that the pedestrian re-identification technology in the related technique cannot accurately identify the dress code of each part of the human body.

[0005] To solve the aforementioned technical problems, the present application will be implemented as follows.

[0006] According to a first aspect, an embodiment of the present application provides a method for discriminating against dress codes, which includes: Performing human body detection on an image to be identified in order to obtain an initial human body detection image of a target person in the image to be identified; Performing a human body area division on the initial human body detection image to obtain a target human body area image of the target person; Using a person identification model to extract features from the target person's human body area image to obtain an initial feature vector; Comparing the first feature vector with a second feature vector of a target human body region in a clothing regulation pattern image to obtain a comparison result; and Determine whether the clothing worn by the target human body area of ​​the target person complies with a clothing regulation according to the comparison result.

[0007] Optionally, performing a human body area division on the first human body detection image includes:

[0008] Using a preprocessing model to perform a human body keypoint extraction on the first human body detection image to obtain an initial human body keypoint;

[0009] Dividing the first human body detection image into human body areas according to the first human body key point.

[0010] Optionally, the procedure further includes the following before comparing the first feature vector with the second feature vector of the target human body area in the clothing regulation pattern image:

[0011] Performing a human body detection on the clothing regulations pattern image to obtain a second human body detection image;

[0012] Performing a human body area subdivision on the second human body detection image to obtain a target human body area image of the clothing regulation pattern image;

[0013] Using the person identification model to extract features from the target human body area image of the clothing regulation pattern image to obtain a second feature vector.

[0014] Optionally, this includes comparing the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image to obtain a comparison result:

[0015] Calculating the cosine similarity of the first feature vector and the second feature vector of the target human body area in the clothing regulation pattern image to obtain similarity information as the comparison result.

[0016] Optionally, determining whether the clothing worn by the target human body area of ​​the target person complies with a clothing regulation according to the comparison result includes: For one target human body area of ​​the target person, if a comparison result of N frames of the image to be identified, which includes the target person, indicates that a similarity information between the first feature vector of the target human body area in at least M frames of the image to be identified and the second feature vector of the target human body area in the dress code pattern image does not reach a preset threshold, determine that the clothing of the target human body area does not comply with the dress code; where N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to 1 and less than N.

[0017] According to a second aspect, an embodiment of the present application provides a method for training a person identification model, which includes:

[0018] Determine several training image pairs, where each of the training image pairs includes at least two training images;

[0019] Performing a human body detection on the training image in the training image pair to obtain a third human body detection image of the training image;

[0020] Performing a human body area subdivision on the third human body detection image to obtain a target human body area image;

[0021] Using a person identification model to be trained to perform feature extraction on the target human body area image in order to obtain a third feature vector;

[0022] Comparing the third feature vectors of the target human body area of ​​different training images in the training image pair to obtain a comparison result;

[0023] Optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model.

[0024] Optionally, performing a human body area division on the third human body detection image includes:

[0025] Using a preprocessing model to perform a human body keypoint extraction on the third human body detection image to obtain a third human body keypoint;

[0026] Dividing the third human body detection image into human body areas according to the third human body key point.

[0027] Optionally, determining multiple training image pairs includes:

[0028] Using the preprocessing model to extract human body attribute information from candidate images in order to obtain human body attribute information of the candidate images;

[0029] Selecting training images from the candidate images to form the training image pairs according to the human body attribute information.

[0030] Optionally, the human body attribute information includes a human body orientation, and selecting training images from the candidate images to form the training image pairs according to the human body attribute information includes: selecting training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair; and / or The human body attribute information includes a human body orientation and a clothing color. Selecting training images from the candidate images to form the training image pairs according to the human body attribute information includes: selecting training images of different people wearing clothing of the same color and facing in the same orientation from the candidate images as the training images in the training image pair.

[0031] Optionally, this includes selecting training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair: For the one training image, from several candidate images that include the same person, selecting a first image of a first difficulty level with a first probability, selecting a second image of a second difficulty level with a second probability, and selecting a third image of a third difficulty level with a third probability, than the training images in the training image pair; where the first level of difficulty means that: the person in one of the training image and the first image is facing forward, and the person in the other of the training image and the first image is facing backward; or, the person in one of the training image and the first image is facing left, and the person in the other of the training image and the first image is facing right; The second level of difficulty means that: the person in one of the training images and the second image is facing forward, and the person in the other of the training image and the second image is facing left or right; or, the person in one of the training image and the first image is facing backward, and the person in the other of the training image and the second image is facing left or right; The third level of difficulty means that the person in the training image and the third image is facing in the same direction.

[0032] Optionally, selecting training images of different people wearing the same color clothing and facing the same direction from the candidate images includes the training images in the training image pair: For the one training image, selecting candidate images that include a person other than the training image; Calculating similarity information between the training image and the candidate images, wherein the similarity information is determined by at least one of the following: color of clothing, wearing of a hat, and orientation of the person in the training image and the candidate images; Dividing the candidate images into several sets according to the similarity information of the candidate images; Selecting candidate images from different sets as training images in the training image pair.

[0033] Optionally, determining multiple training image pairs includes: For the one training image, select a candidate image that includes a different person than the training image; Performing a human body detection on the candidate image to obtain a fourth human body detection image of the candidate image; Using a preprocessing model to perform a human key point extraction from the fourth human body detection image to obtain second human key points; According to the second human body key points, perform a human body area division on the fourth human body detection image to obtain a target human body area image of the fourth human body detection image; Determining the mean and variance of the target human body area image of the fourth human body detection image on three RGB channels; Converting the target human body area image of the fourth human body detection image into an HSV space, and calculating the mean and variance on three HSV channels after conversion to the HSV space; according to the mean and variance on the three RGB channels and the mean and variance on the three HSV channels, preserving image features after dimensionality reduction; Clustering of image features after dimensionality reduction of the multiple candidate images and dividing the multiple candidate images into different clusters; Selecting candidate images from different clusters as training images in the training image pair.

[0034] According to a third aspect, an embodiment of the present application comprises a device for discriminating against dress codes, which includes: a first human body detection module used to perform human body detection on an image to be identified, in order to obtain a first human body detection image of a target person in the image to be identified; a first human body area partitioning module that is used to perform a human body area partitioning on the first human body detection image in order to obtain a target human body area image of the target person; a first feature extraction module that is used to employ a person identification model to extract features from the target person's human body area image in order to obtain a first feature vector; a first comparison module used to compare the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image to obtain a comparison result; a determination module used to determine whether the clothing of the target human body area of ​​the target person complies with a clothing regulation according to the comparison result.

[0035] According to a fourth aspect, an embodiment of the present application provides a device for training a person identification model, comprising: a determination module used to determine multiple training image pairs, each of which includes at least two training images; a third human body detection module, which is used to perform a human body detection on the training image in the training image pair in order to obtain a third human body detection image in the training image; a third human body area partitioning module, which is used to perform a human body area partitioning on the third human body detection image in order to obtain a target human body area image; a third feature extraction module that is used to employ a person identification model to be trained to perform feature extraction on the target human body area image in order to obtain a third feature vector; a second comparison module used to compare the third feature vectors of target human body areas of different training images in the training image pair in order to obtain a comparison result; an optimization module that is used to optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model.

[0036] According to a fifth aspect, an embodiment of the present application provides an electronic device comprising: a processor, a memory, and a program stored in the memory and executable on the processor; wherein, when the program is executed by the processor, the steps of the method for discriminating against dress codes according to the first aspect or the second aspect are implemented.

[0037] According to a sixth aspect, an embodiment of the present application provides a non-volatile, computer-readable storage medium comprising a computer program stored thereon; wherein, when the program is executed by the processor, the steps of the method for discriminating against dress codes according to the first aspect or the second aspect are implemented.

[0038] In the embodiment of the present application, prior to using the pedestrian re-identification module to identify the clothing of the target person in the image to be identified, the human body detection image detected in the image to be identified is divided into human body areas to obtain human body area images of the target person, and the pedestrian re-identification module is used to identify each human body area image separately, thereby accurately identifying the clothing requirement of each part of the human body and improving the recognition accuracy. BRIEF DESCRIPTION OF THE FIGURES

[0039] Several other advantages and benefits will become apparent to the person skilled in the art upon reading the detailed description of the following preferred embodiments. The accompanying drawings serve only to illustrate the preferred embodiments and are not to be considered as limiting the present application. Furthermore, the same reference symbols are used in the accompanying drawings to represent the same components. In the accompanying drawings: Fig. Figure 1 is a schematic flowchart of a procedure for discriminating against dress codes according to an embodiment of the present application; Fig. Figure 2 is a schematic flowchart of a method for training a person identification model according to an embodiment of the present application; Fig. Figure 3 is a schematic diagram of a method for training a person identification model according to an embodiment of the present application; Fig. Figure 4 is a schematic diagram of a preprocessing model according to an embodiment of the present application; Fig. Figure 5 is a schematic diagram of a device for discriminating against dress codes according to an embodiment of the present application; Fig. Figure 6 is a schematic diagram of a device for training a person identification model according to an embodiment of the present application; and Fig. Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE REVELATION

[0040] The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments represent only a subset of the embodiments of the present application and not all embodiments. Based on the embodiments of the present application, all other embodiments that can be achieved by those skilled in the art without any creative work fall within the scope of protection of the present application.

[0041] With reference to Fig. 1 provides an embodiment of the present application providing a method for discriminating against dress codes, comprising:

[0042] Step 11: Perform a human body detection on an image to be identified in order to obtain an initial human body detection image of a target person in the image to be identified.

[0043] In this embodiment of the present application, various algorithms can be used to perform human body detection on the image to be identified. For example, a target detection algorithm such as the yolo-v5 algorithm is used to detect the target person, and a target tracking algorithm such as the Sort algorithm is used to track the target.

[0044] In the embodiment of the present application, the image to be identified can be an image in a surveillance video stream recorded by a camera device at a predetermined location (such as a factory, a bank lobby, etc.).

[0045] In the embodiment of the present application, the number of target persons in the image to be identified can be one or more. Each target person can be assigned a unique tracking ID.

[0046] In the embodiment of the present application, a specific pedestrian can optionally be detected in the image to be identified. For example, for an image in a surveillance video stream of a bank lobby, only bank employees appearing in the image can be detected. In this case, a target person database must be created beforehand. The target person database stores facial images of one or more target persons. When performing human body detection on the image to be identified, facial recognition is first carried out on the detected pedestrian according to the target person database. If the pedestrian in the image to be identified is identified as a pedestrian in the target person database, the subsequent steps are executed. If the pedestrian in the image to be identified is not identified as a pedestrian in the target person database, the process is terminated.

[0047] In the embodiment of the present application, optionally all pedestrians in the image to be identified can be taken as target persons in order to carry out a subsequent clothing regulation identification, e.g. in a factory or other place to which outsiders generally do not have access.

[0048] Step 12: Perform a human body area division on the first human body detection image to obtain a target human body area image of the target person.

[0049] In the embodiment of the present application, the target human area can be one or more, comprising, for example, a head area, an upper body area, and a lower body area. That is, the first human body detection image can be divided into one or more target human area images.

[0050] Step 13: Use a person identification model to extract features from the target person's human body area image to obtain an initial feature vector.

[0051] In the embodiment of the present application, if the first human body detection image corresponds to several target human body area images, the multiple target human body area images can optionally be spliced ​​first to obtain a spliced ​​image and input it into the person identification model. Of course, multiple target human body area image pages can also be entered individually into the person identification model. Alternatively, multiple target human body area images can also be entered separately into the person identification model instead of being spliced.

[0052] Step 14: Compare the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image to obtain a comparison result.

[0053] In the embodiment of the present application, if the first human body detection image corresponds to several target human body area images, each target human body area image of the first human body detection image can be compared with the corresponding target human body area image in the clothing regulation pattern image.

[0054] For example, it is assumed that the target human body area image comprises: a head area image, an upper body area image, and a lower body area image, and at this point, a first feature vector of the head area image in the first human body detection image can be compared with a second feature vector of a head area in the clothing regulation pattern image; a first feature vector of the upper body area image in the first human body detection image can be compared with a second feature vector of an upper body area in the clothing regulation pattern image; and a first feature vector of the lower body area image in the first human body detection image can be compared with a second feature vector of a lower body area in the clothing regulation pattern image to obtain three comparison results.

[0055] Step 15: Determine whether the clothing of the target human body area of ​​the target person complies with the clothing regulation according to the comparison result.

[0056] In the embodiment of the present application, prior to using the pedestrian re-identification module to identify the clothing of the target person in the image to be identified, the human body detection image detected in the image to be identified is divided into human body areas to obtain human body area images of the target person, and the pedestrian re-identification module is used to identify each human body area image separately, thereby accurately identifying the clothing requirement of each part of the human body and improving the recognition accuracy.

[0057] In the embodiment of the present application, the optional procedure includes performing a human body area division on the first human body detection image:

[0058] Step 121: Using a preprocessing model to perform a human body keypoint extraction on the first human body detection image to obtain an initial human body keypoint.

[0059] In the embodiment of the present application, the key points of the human body may, for example, include the upper part of the head, the neck, the limbs and other key points of the human body.

[0060] Step 122: Dividing the first human body detection image into human body areas according to the first human body key point.

[0061] In the embodiment of the present application, the method optionally includes, prior to comparing the first feature vector with the second feature vector of the target human body area in the clothing regulation pattern image:

[0062] Step 01: Perform a human body detection on the clothing regulations pattern image to obtain a second human body detection image;

[0063] Step 02: Perform a human body area division on the second human body detection image to obtain a target human body area image of the clothing regulation pattern image;

[0064] Step 03: Using the person identification model to extract features from the target human body area image of the clothing regulation pattern image to obtain a second feature vector.

[0065] In the embodiment of the present application, before using the person identification model to perform dress code identification on the target person in the image to be identified, the dress code pattern image is identified, and different dress code pattern images are identified in advance for different locations. If the application location changes, no new training images need to be collected to train the person identification model; only the dress code pattern image needs to be replaced.

[0066] In the embodiment of the present application, optionally the comparison of the first feature vector with the second feature vector of the target human body area in a clothing regulation pattern image to obtain a comparison result includes: calculating the cosine similarity of the first feature vector and the second feature vector of the target human body area in the clothing regulation pattern image to obtain similarity information as the comparison result.

[0067] Of course, in some other embodiments of the present application, the similarity calculation is not limited to the use of cosine similarity, and other methods for calculating similarity may also be used.

[0068] In the embodiment of the present application, optionally determining whether the clothing of the target human body area of ​​the target person complies with the clothing regulation according to the comparison result includes: for one target human body area of ​​the target person, if the comparison result of N frames of the image to be identified, which include the target person, indicates that a similarity information between the first feature vector of the target human body area in at least M frames of the image to be identified and the second feature vector of the target human body area in the clothing regulation template image does not reach a preset threshold, determining that the clothing of the target human body area does not comply with the clothing regulation; wherein N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to 1 and less than N.

[0069] For example, 5 (i.e., N) frames of the image to be identified, encompassing the target individual, can be identified, and each target human body region (e.g., head, torso, and lower body) in each frame of the image to be identified is compared to the corresponding target human body region in the dress code pattern image. It is assumed that the similarity information from 3 (i.e., M) frames of the head region image does not reach a preset threshold (e.g., 0.45). It is determined that the target individual's head region clothing does not comply with the dress code.

[0070] In the embodiment of the present application, the method optionally includes, after it has been determined whether the clothing of the target human body area of ​​the target person complies with the dress code according to the comparison result, issuing a warning message if the comparison result indicates that the clothing of the target human body area does not comply with the dress code. For example, the issued warning message may read: “Zhang San’s hat is not being worn in accordance with the dress code.”

[0071] The following describes a training procedure for the person identification model in the above embodiment.

[0072] With reference to Fig. 2 further provides an embodiment of the present application, a method for training a person identification model, comprising: Step 21: Determine several training image pairs, where each training image pair includes at least two training images; Step 22: Perform a human body detection on the training image in the training image pair to obtain a third human body detection image in the training image; Step 23: Perform a human body area subdivision on the third human body detection image to obtain a target human body area image; Step 24: Using a person identification model to be trained to perform feature extraction on the target human body area image to obtain a third feature vector; Step 25: Compare the third feature vectors of the target human body area of ​​different training images in the training image pair to obtain a comparison result; Step 26: Optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model.

[0073] With reference to Fig. 3 is Fig. 3 A schematic diagram of a method for training a person identification model according to an embodiment of the present application. The input to the person re-identification model is a spliced ​​image (i.e., a whole-body image) of various target human body area images corresponding to the training image (i.e., a whole-body image). In the embodiment of the present application, there are three target human body area images comprising a head area image, an upper-body area image, and a lower-body area image, where Feature1, Feature2, and Feature3 are feature vectors extracted from the head area image, the upper-body area image, and the lower-body area image, respectively. FC_Total refers to a fully connected network layer that outputs a whole-body feature vector extracted from the whole-body image.

[0074] In the embodiment of the present application, triplet loss and softmax loss can optionally be used for joint training of the person re-identification model. Joint training involves: calculating a weighted average of multiple losses, calculating a derivative of the weighted average with respect to the input, and using the gradient descent method to update the network parameters for training.

[0075] In the embodiment of the present application, performing a human body area partitioning on the third human body detection image to obtain a target human body area image optionally includes: Step 231: Using a preprocessing model to perform a human body keypoint extraction on the third human body detection image to obtain a third human body keypoint.

[0076] In the embodiment of the present application, human body key points can, for example, include the upper part of the head, the neck, the limbs, and other key points of the human body. The number of human body key points can be adjusted as required, for example, 21 human body key points.

[0077] Step 232: Dividing the third human body detection image into human body areas according to the third human body key point.

[0078] For example, the human body detection image can be divided into three areas according to the human body key points: head, upper body and lower body.

[0079] Using the head as an example, the key points of the left and right ears are selected as a reference to extract an area [center_x-d, center_x+d, center_y-d, center_y+d] from the entire image. Center_x and center_y are the horizontal and vertical coordinates of the midpoint of a line connecting the two key points; and d is a distance between the two key points. The extracted image is rotated so that the line connecting the key points of the left and right ears is horizontal, and the rotated image is modified (scaled) to a predefined size (such as 128x128) as the head area image.

[0080] The same method can be used to process the upper and lower body areas to obtain three images of predefined sizes. These three images are then spliced ​​together to create a full-body image (e.g., 384x128 pixels) as input for the person identification model.

[0081] In the embodiment of the present application, human body key points are extracted by the preprocessing model prior to training the person identification model; the human body detection image is divided into areas according to the human body key points, and the divided human body area images are used to train the person identification model, thereby obtaining a person identification model for each human body area and improving the accuracy of the person identification model. Simultaneously, clothing information from different parts of the human body is obtained, which improves the person identification model's ability to recognize detailed information.

[0082] In the embodiment of the present application, determining multiple training image pairs optionally includes: Step 211: Using the preprocessing model to extract human body attribute information from candidate images to obtain human body attribute information of the candidate images.

[0083] In the embodiment of the present application, the human body attribute information may optionally include, for example, at least one of the following: color of upper body clothing, color of lower body clothing, human orientation, whether a hat is worn, whether it is blocked, etc.

[0084] Step 212: Selecting training images from the candidate images to form the training image pairs according to the human body attribute information.

[0085] In the embodiment of the present application, prior to training the person identification model, the human body attribute information of the candidate images is extracted by the preprocessing model, and a training image is selected based on the human body attribute information of the candidate images, so that the required training images can be obtained according to different requirements, such as performing detection of difficult patterns to improve the accuracy of the person identification model.

[0086] It should be noted that the preprocessing model for extracting human body key points and the preprocessing model for extracting human body attribute information may be different models or may be integrated into the same model in the embodiment of the present application.

[0087] With reference to Fig. 4 is Fig. Figure 4 shows a schematic diagram of a preprocessing model according to an embodiment of the present application. The preprocessing model comprises a convolutional neural network (Conv). The Conv is the backbone of the preprocessing model and is used to extract feature vectors from an input image. The Conv can select structures such as resnet50 or mobilenetv2. Two network branches in the dashed box are, respectively, a human body key point extraction network and a human body attribute information extraction network. The human body key point extraction network extracts human body key points from the feature vectors extracted by the Conv. Optionally, the last layer of the human body key point extraction network can be a fully connected layer with N1 (e.g., 42) dimensions that outputs horizontal and vertical coordinates of N1 / 2 (e.g., 21) human body key points.The human body attribute information extraction network extracts human body attribute information from the feature vectors extracted by the Conv. Optionally, the last layer of the human body attribute information extraction network can be a fully connected layer with N2 dimensions, and each dimension is a binary classification result of a specific human body attribute, such as whether the upper body clothing is red, whether the upper body clothing is green, whether the human body is facing forward, whether the human body is facing backward, etc. N2 is, for example, 24. The binary classification result can include: 8 colors for upper body clothing, 8 colors for lower body clothing, 4 human body orientations, whether a hat is worn, whether the head is blocked, whether the upper body is blocked, and whether the lower body is blocked.

[0088] The advantage of integrating the human body key point extraction network and the human body attribute information extraction network into one model is that the human body key point extraction network and the human body attribute information extraction network can share the same Conv.

[0089] A procedure for selecting training images based on human body attribute information from candidate images is explained in detail below.

[0090] In the embodiment of the present application, the training image pair can optionally be a triple, for example [img, img+, img-], where img+ is another image that includes the same person as img, and img- is another image that includes a different person than img. Of course, in some other embodiments of the present application, the training image pair is not limited to a triple.

[0091] In the embodiment of the present application, the human body attribute information optionally includes a human body orientation, and the selection of training images from the candidate images to form the training image pairs according to the human body attribute information includes: selecting training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair.

[0092] In general, image pairs of the same person with different body orientations can be used as difficult patterns. In the embodiment of the present application, the two most difficult directions are forward and backward, and left and right; the four second most difficult directions are forward and left, forward and right, backward and left, and backward and right; and image pairs of the same direction have the lowest difficulty level. For a given training image img, images from the most difficult, second most difficult, and least difficult directions are extracted to form positive image pairs img for training.

[0093] This means that selecting training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair includes: For a training image, from several candidate images that include the same person, selecting a first image of a first difficulty level with a first probability, selecting a second image of a second difficulty level with a second probability, and selecting a third image of a third difficulty level with a third probability, than the training images in the training image pair.

[0094] Optionally, the first probability > the second probability > the third probability. For example, the first probability is 50%, the second probability is 30%, and the third probability is 20%.

[0095] The first level of difficulty means that: the person in one of the training images and the first image is facing forward, and the person in the other of the training image and the first image is facing backward; or, the person in one of the training images and the first image is facing left, and the person in the other of the training image and the first image is facing right.

[0096] The second level of difficulty means that: the person in one of the training image and the second image is facing forward, and the person in the other of the training image and the second image is facing left or right; or, the person in one of the training image and the first image is facing backward, and the person in the other of the training image and the second image is facing left or right.

[0097] The third difficulty level means that the person in the training image and the third image is facing in the same direction.

[0098] In some other embodiments of the present application, the first probability, the second probability and the third probability may be equal or partially equal.

[0099] In general, a pair of pedestrian images of different people wearing clothing of the same color and facing the same orientation can be used as difficult patterns, and color has a greater impact on identification difficulties than orientation. In the embodiment of the present application, the human body attribute information includes a human body orientation and clothing color, the selection of training images from the candidate images to form the training image pairs according to the human body attribute information, optionally: the selection of training images of different people wearing clothing of the same color and facing the same orientation from the candidate images as the training images in the training image pair.

[0100] In the embodiment of the present application, selecting training images of different persons wearing clothing of the same color and having the same orientation from the candidate images as the training images in the training image pair optionally includes: for a training image, selecting candidate images that include a person other than the training image; Calculating similarity information between the training image and the candidate images, wherein the similarity information is determined by at least one of the following: color of clothing, wearing of a hat, and orientation of the person in the training image and the candidate images; Dividing the candidate images into several sets according to the similarity information of the candidate images; Selecting candidate images from different sets as training images in the training image pair. Optionally, candidate images from different sets with different probabilities are selected as training images in the training image pair. Optionally, the similarity information between the training image img and the candidate images, which include different people, can be calculated using the following formula: score=3.5*sameupper+3.5*samedown+2*samehat+samedirection

[0101] Whereby same upper indicates whether the color of the upper body clothing is the same (0 means different, 1 means the same); same down indicates whether the color of the lower body clothing is the same (0 means different, 1 means the same); same hat indicates whether wearing a hat is the same (0 means different, 1 means the same), and same direction indicates whether pedestrians are facing the same direction (0 means different, 1 means the same).

[0102] The candidate images, which include different people, can be split into multiple sets according to the similarity information, and candidate images are extracted from different sets with a probability of score / 10 to form negative image pairs using img for training.

[0103] In the embodiment of the present application, the preprocessing model cannot be used for mining difficult patterns. As an alternative to the method of mining difficult patterns involving different people, since the color of the clothing has the most significant influence on the difficulty of the patterns, in this embodiment, only color information is considered to improve training speed for mining difficult patterns involving different people.

[0104] This means that, optionally, determining multiple training image pairs includes: For a training image, select a candidate image that includes a different person than the training image; Performing a human body detection on the candidate image to obtain a fourth human body detection image of the candidate image; Using a preprocessing model to perform a human key point extraction from the fourth human body detection image to obtain second human key points; According to the second human body key points, perform a human body area division on the fourth human body detection image to obtain a target human body area image of the fourth human body detection image; Determining the mean and variance of the target human body area image of the fourth human body detection image on three RGB channels; Converting the target human body area image of the fourth human body detection image into HSV space, and calculating the mean and variance on the three HSV channels after conversion into HSV space; that is, a total of 12-dimensional features are obtained; According to the mean and variance on the three RGB channels and the mean and variance on the three HSV channels, image features are preserved after dimensional reduction; optionally, the t-SNE method can be used to reduce the 12-dimensional features to 2 dimensions; Clustering of image features after dimensional reduction of multiple candidate images and division of the multiple candidate images into different clusters; optionally, the DBSCAN method can be used to cluster the image features after dimensional reduction; Selecting candidate images from different clusters as training images in the training image pair.

[0105] Optionally, candidate images from different clusters are selected with different probabilities as training images in the training image pair. For example, images belonging to different clusters than the training image and to the same cluster as the training image are selected with probabilities of 80% and 20%, respectively, to form negative pattern pairs for training.

[0106] In the embodiment of the present application, the method may optionally further include: training the preprocessing model.

[0107] In the embodiment of the present application, if the human body key point extraction network and the human body attribute information extraction network are different preprocessing models, the two preprocessing models are trained separately. If the human body key point extraction network and the human body attribute information extraction network are integrated into a single preprocessing model, the human body key point extraction network and the human body attribute information extraction network are trained separately to obtain losses from both, and the losses from the two are combined (added or weighted addition, etc.), and then the gradient is calculated and the parameters of the conversion are updated.

[0108] In the embodiment of the present application, wing loss is optionally used to train the human body key point extraction network in the preprocessing model.

[0109] In the embodiment of the present application, BCE loss is optionally used to train the human body attribute information extraction network in the preprocessing model.

[0110] In the above embodiments of the present application, in addition to the use of human body key points for dividing the human body image into areas, a segmentation model can also be used to divide the human body image into areas.

[0111] With reference to Fig. 5 further provides an embodiment of the present application, a device 50 for discriminating against dress codes, comprising: a first human body detection module 51, which is used to perform a human body detection on an image to be identified in order to obtain a first human body detection image of a target person in the image to be identified; a first human body area partitioning module 52, which is used to perform a human body area partitioning on the first human body detection image in order to obtain a target human body area image of the target person; a first feature extraction module 53, which is used to employ a person identification model to extract features from the target human body area image of the target person in order to obtain a first feature vector; a first comparison module 54, which is used to compare the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image in order to obtain a comparison result; a determination module 55, which is used to determine whether the clothing of the target human body area of ​​the target person complies with the clothing regulation according to the comparison result.

[0112] Optionally, the first human body area partitioning module 52 is used to employ a preprocessing model to perform a human body keypoint extraction on the first human body detection image to obtain a first human body keypoint; and to partition the first human body detection image into human body areas according to the first human body keypoint.

[0113] Optionally, the device 50 for discriminating against dress codes also includes: a second human body detection module used to perform a human body detection on the clothing regulation pattern image in order to obtain a second human body detection image; a second human body area partitioning module, which is used to perform a human body area partitioning on the second human body detection image in order to obtain a target human body area image in the clothing regulation pattern image; a second feature extraction module that is used to utilize the person identification model to extract features from the target human body area image of the clothing regulation pattern image to obtain a first feature vector.

[0114] Optionally, the first comparison module 54 is used to calculate a cosine similarity of the first feature vector and the second feature vector of the target human body area in the clothing regulation pattern image in order to obtain similarity information as the comparison result.

[0115] Optionally, determination module 55 is used for a target human body area of ​​the target person if the comparison result of N frames of the image to be identified, which includes the target person, indicates that the similarity information between the first feature vector of the target human body area in at least M frames of the image to be identified and the second feature vector of the target human body area in the clothing regulation pattern image does not reach a predetermined threshold to determine that the clothing of the target human body area does not comply with the clothing regulation; where N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to 1 and less than N.

[0116] With reference to Fig. 6 further provides an embodiment of the present application comprising a device 60 for training a person identification model, comprising: a determination module 61, which is used to determine several training image pairs, wherein each of the training image pairs comprises at least two training images; a third human body detection module 62, which is used to perform a human body detection on the training image in the training image pair in order to obtain a third human body detection image in the training image; a third human body area partitioning module 63, which is used to perform a human body area partitioning on the third human body detection image in order to obtain a target human body area image; a third feature extraction module 64, which is used to employ a person identification model to be trained to perform a feature extraction on the target human body area image in order to obtain a third feature vector; a second comparison module 65, which is used to compare the third feature vectors of target human body areas of different training images in the training image pair in order to obtain a comparison result; an optimization module 66, which is used to optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model.

[0117] Optionally, the third human body area partitioning module 63 is used to employ a preprocessing model to perform a human body keypoint extraction on the third human body detection image to obtain a third human body keypoint; and to partition the third human body detection image into human body areas according to the third human body keypoint.

[0118] Optionally, the determination module 61 is used to use the preprocessing model to extract human body attribute information from candidate images in order to obtain human body attribute information of the candidate images; and to select training images from the candidate images in order to form the training image pairs according to the human body attribute information.

[0119] Optionally, the human body attribute information includes a human body orientation, and the determination module 61 is used to select training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair.

[0120] Optionally, the human body attribute information includes a human body orientation and a clothing color, and the determination module 61 is used to select training images of different people wearing clothing of the same color and facing in the same orientation from the candidate images as the training images in the training image pair.

[0121] Optionally, the determination module 61 is used to select, for a training image, from several candidate images comprising the same person as the training images in the training image pair, a first image of a first difficulty level with a first probability, a second image of a second difficulty level with a second probability, and a third image of a third difficulty level with a third probability; where the first difficulty level means that: the person in one of the training image and the first image is facing forward, and the person in the other of the training image and the first image is facing backward, or, the person in one of the training image and the first image is facing left, and the person in the other of the training image and the first image is facing right;The second level of difficulty means that: the person in one of the training images and the second image is facing forward, and the person in the other of the training image and the second image is facing left or right; or, the person in one of the training image and the first image is facing backward, and the person in the other of the training image and the second image is facing left or right; the third level of difficulty means that the person in the training image and the third image is facing in the same direction.

[0122] Optionally, the determination module 61 is used to select candidate images for a training image that include a different person than the training image; to calculate similarity information between the training image and the candidate images, where the similarity information is determined by at least one of the following: color of clothing, wearing of a hat, and orientation of the person in the training image and the candidate images; to divide the candidate images into several sets according to the similarity information of the candidate images; and to select candidate images from different sets as training images in the training image pair.

[0123] Optionally, the determination module 61 is used to: select a candidate image for a training image that includes a different person than the training image; perform human body detection on the candidate image to obtain a fourth human body detection image of the candidate image; use a preprocessing model to perform human key point extraction from the fourth human body detection image to obtain second human key points; based on the second human body key points, perform a human body area split on the fourth human body detection image to obtain a target human body area image of the fourth human body detection image; and determine the mean and variance of the target human body area image of the fourth human body detection image on three RGB channels.to convert the target human body area image of the fourth human body detection image into an HSV space, and to calculate the mean and variance on three HSV channels after conversion to the HSV space; to obtain image features after dimensional reduction based on the mean and variance on the three RGB channels and the mean and variance on the three HSV channels; to cluster image features after dimensional reduction of the multiple candidate images and to divide the multiple candidate images into different clusters; to select candidate images from different clusters as training images in the training image pair.

[0124] With reference to Fig.Figure 7 further provides an embodiment of the present application comprising an electronic device 70, which includes a processor 71, a memory 72, and a computer program stored in the memory 72 and executable on the processor 71. When the computer program is executed by the processor 71, each process of the above method for discriminating against dress codes or the method for training a person identification model is implemented in the embodiment, and the same technical effect can be achieved, which is not described here to avoid repetition.

[0125] One embodiment of the present application further provides a non-volatile, computer-readable storage medium comprising a computer program stored thereon. When the computer program is executed by the processor, each process of the above method for discriminating against dress codes or the method for training a person identification model is implemented in the embodiment, and the same technical effect can be achieved, which is not described here to avoid repetition. The non-volatile, computer-readable storage medium is, for example, a read-only memory (ROM), a random-access memory (RAM), a magnetic disk, or an optical disk.

[0126] It should be noted that in this disclosure, the terms "comprise," "comprehensive," or other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a number of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such a process, method, article, or device. Unless further limitations are specified, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0127] From the description of the embodiments above, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a required general hardware platform, and of course by means of hardware, but in many cases the former is a better implementation method. Based on such an understanding, the essence of the technical solution of the present application, or the part that contributes to the prior art, can be implemented in the form of a software product. The software product is stored on a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk) and comprises a variety of instructions that enable an end device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0128] The embodiments of this application are described above with reference to the accompanying drawings. However, this application is not limited to the specific implementations described above. The specific implementations described above are merely illustrative and not limiting. A person skilled in the art may derive various forms from this application without departing from the spirit of this application and the scope claimed in the claims, all of which are protected by this application.

Claims

[1] Procedures for discrimination based on dress codes, including: Performing human body detection on an image to be identified in order to obtain an initial human body detection image of a target person in the image to be identified; Performing a human body area division on the initial human body detection image to obtain a target human body area image of the target person; Using a person identification model to extract features from the target person's human body area image to obtain an initial feature vector; Comparing the first feature vector with a second feature vector of a target human body region in a clothing regulation pattern image to obtain a comparison result; and Determine whether the clothing worn by the target human body area of ​​the target person complies with a clothing regulation according to the comparison result. [2] Method according to claim 1, wherein performing a human body area division on the first human body detection image comprises: Using a preprocessing model to perform a human body keypoint extraction on the first human body detection image to obtain an initial human body keypoint; Dividing the first human body detection image into human body areas according to the first human body key point. [3] Method according to claim 1, wherein the method further comprises, prior to comparing the first feature vector with the second feature vector of the target human body area in the clothing regulation pattern image: Performing a human body detection on the clothing regulations pattern image to obtain a second human body detection image; Performing a human body area subdivision on the second human body detection image to obtain a target human body area image of the clothing regulation pattern image; Using the person identification model to extract features from the target human body area image of the clothing regulation pattern image to obtain a second feature vector. [4] The method of claim 1, wherein comparing the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image to obtain a comparison result comprises: Calculating the cosine similarity of the first feature vector and the second feature vector of the target human body area in the clothing regulation pattern image to obtain similarity information as the comparison result. [5] The method of claim 1, wherein determining whether the clothing of the target human body area of ​​the target person complies with a clothing regulation according to the comparison result comprises: For one target human body area of ​​the target person, if a comparison result of N individual images of the image to be identified, which include the target person, indicates that a similarity information between the first feature vector of the target human body area in at least M individual images of the image to be identified and the second feature vector of the target human body area in the clothing regulation pattern image does not reach a preset threshold, it is determined that the clothing of the target human body area does not comply with the clothing regulation; where N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to 1 and less than N. [6] Methods for training a person identification model, comprising: Determine several training image pairs, where each of the training image pairs includes at least two training images; Performing a human body detection on the training image in the training image pair to obtain a third human body detection image of the training image; Performing a human body area subdivision on the third human body detection image to obtain a target human body area image; Using a person identification model to be trained to perform feature extraction on the target human body area image in order to obtain a third feature vector; Comparing the third feature vectors of the target human body area of ​​different training images in the training image pair to obtain a comparison result; Optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model. [7] Method according to claim 6, wherein performing a human body area division on the third human body detection image comprises: Using a preprocessing model to perform a human body keypoint extraction on the third human body detection image to obtain a third human body keypoint; Dividing the third human body detection image into human body areas according to the third human body key point. [8] The method of claim 6, wherein determining multiple training image pairs comprises: Using the preprocessing model to extract human body attribute information from candidate images in order to obtain human body attribute information of the candidate images; Selecting training images from the candidate images to form the training image pairs according to the human body attribute information. [9] The method of claim 8, wherein the human body attribute information comprises a human body orientation, and the selection of training images from the candidate images to form the training image pairs according to the human body attribute information comprises: selecting training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair; and / or The human body attribute information includes a human body orientation and a clothing color. Selecting training images from the candidate images to form the training image pairs according to the human body attribute information includes: selecting training images of different people wearing clothing of the same color and facing in the same orientation from the candidate images as the training images in the training image pair. [10] Method according to claim 9, wherein the selection of training images of the same person with the same and / or different orientations from the candidate images as training images in the training image pair comprises: For the one training image, from several candidate images that include the same person, selecting a first image of a first difficulty level with a first probability, selecting a second image of a second difficulty level with a second probability, and selecting a third image of a third difficulty level with a third probability, than the training images in the training image pair; where the first level of difficulty means that the person in one of the training image and the first image is facing forward, and the person in the other of the training image and the first image is facing backward; or, the person in one of the training image and the first image is facing left, and the person in the other of the training image and the first image is facing right; The second level of difficulty means that the person in one of the training images and the second image is facing forward, and the person in the other of the training image and the second image is facing left or right; or, the person in one of the training image and the first image is facing backward, and the person in the other of the training image and the second image is facing left or right; The third level of difficulty means that the person in the training image and the third image is facing in the same direction. [11] The method of claim 9, wherein the selection of training images of different persons wearing clothing of the same color and facing in the same orientation from the candidate images as the training images in the training image pair comprises: For the one training image, selecting candidate images that include a person other than the training image; Calculating similarity information between the training image and the candidate images, wherein the similarity information is determined by at least one of the following: clothing color, wearing a hat, and orientation of the person in the training image and the candidate images; Dividing the candidate images into several sets according to the similarity information of the candidate images; Selecting candidate images from different sets as training images in the training image pair. [12] The method of claim 6, comprising determining multiple training image pairs: For the one training image, select a candidate image that includes a different person than the training image; Performing a human body detection on the candidate image to obtain a fourth human body detection image of the candidate image; Using a preprocessing model to perform a human key point extraction from the fourth human body detection image to obtain second human key points; According to the second human body key points, perform a human body area division on the fourth human body detection image to obtain a target human body area image of the fourth human body detection image; Determining the mean and variance of the target human body area image of the fourth human body detection image on three RGB channels; Converting the target human body area image of the fourth human body detection image into an HSV space, and calculating the mean and variance on three HSV channels after conversion to the HSV space; according to the mean and variance on the three RGB channels and the mean and variance on the three HSV channels, preserving image features after dimensional reduction; Clustering of image features after dimensional reduction of the multiple candidate images and dividing the multiple candidate images into different clusters; Selecting candidate images from different clusters as training images in the training image pair. [13] Device for discriminating against dress codes, comprising: a first human body detection module used to perform human body detection on an image to be identified, in order to obtain a first human body detection image of a target person in the image to be identified; a first human body area partitioning module that is used to perform a human body area partitioning on the first human body detection image in order to obtain a target human body area image of the target person; a first feature extraction module that is used to employ a person identification model to extract features from the target person's human body area image in order to obtain a first feature vector; a first comparison module used to compare the first feature vector with a second feature vector of a target human body area in a clothing regulation pattern image to obtain a comparison result; a determination module used to determine whether the clothing of the target human body area of ​​the target person complies with a clothing regulation according to the comparison result. [14] Device for training a person identification model, comprising: a determination module used to determine multiple training image pairs, each of which includes at least two training images; a third human body detection module, which is used to perform a human body detection on the training image in the training image pair in order to obtain a third human body detection image in the training image; a third human body area partitioning module, which is used to perform a human body area partitioning on the third human body detection image in order to obtain a target human body area image; a third feature extraction module, which is used to employ a person identification model to be trained to perform feature extraction on the target human body area image in order to obtain a third feature vector; a second comparison module used to compare the third feature vectors of target human body areas of different training images in the training image pair in order to obtain a comparison result; an optimization module that is used to optimize the person identification model to be trained according to the comparison result in order to obtain a trained person identification model. [15] Electronic device comprising: a processor, a memory, and a program stored in the memory and executable on the processor; wherein, when the program is executed by the processor, the steps of the method for discriminating against dress codes according to any one of claims 1 to 5 are implemented, or, when the program is executed by the processor, the steps of the method for training a person identification model according to any one of claims 6 to 12 are implemented. [16] Non-volatile, computer-readable storage medium comprising a computer program stored thereon; wherein, when the program is executed by the processor, the steps of the method for discriminating against dress codes according to any one of claims 1 to 5 are implemented, or, when the program is executed by the processor, the steps of the method for training a person identification model according to any one of claims 6 to 12 are implemented.

Images