A re-identification method, system, device, identification equipment and storage medium

Abstract

This application provides a re-identification method, system, apparatus, identification device, and storage medium, relating to the field of computer vision technology. The method includes: acquiring an image to be identified; determining the pose of a person to be identified contained in the image to be identified from various preset poses, as a preset pose to be utilized; and extracting image features of the region of the image to be identified occupied by the person to be identified, as image features to be identified; comparing the image features to be identified with reference image features of each reference person in the preset pose to be utilized; wherein, the reference image features of any reference person in a preset pose are obtained by fusing previously identified image features of the reference person in that preset pose; and determining the target reference person corresponding to the reference image features that match the image features to be identified as the re-identification result of the person to be identified. This improves the efficiency of re-identification.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a re-identification method, system, apparatus, identification device, and storage medium. Background Technology

[0002] Re-Identification (Re-ID) is a technique that uses computer vision to associate the same person in different images.

[0003] To improve recognition accuracy, for a given reference person, the feature database records image features of that person in different dimensions. These different dimensions include image features of the person in a distant view, a mid-range view, and a close-up view. Then, the image features to be identified are compared with the image features of each reference person in the feature database to determine the person to whom the image features belong.

[0004] However, the above method involves a large number of feature comparisons, resulting in low efficiency in re-identification. Summary of the Invention

[0005] The purpose of this application is to provide a re-identification method, system, apparatus, identification device, and storage medium to improve the efficiency of re-identification. The specific technical solution is as follows:

[0006] A first aspect of this application provides a re-identification method, the method comprising:

[0007] Acquire the image to be recognized;

[0008] The pose of the person to be identified contained in the image to be identified is determined from each preset pose and used as the preset pose to be utilized; and the image features of the region of the image to be identified occupied by the person to be identified are extracted and used as the image features to be identified.

[0009] The image features to be identified are compared with the reference image features of each reference person in the preset pose to be used; wherein, the reference image features of any reference person in a preset pose are obtained by fusing the image features of the reference person in the preset pose that have been identified in the past.

[0010] The target reference person whose features match the features of the image to be identified is determined as the re-identification result of the person to be identified.

[0011] In an optional embodiment, determining the pose of the person to be identified contained in the image to be identified from various preset poses, as the preset pose to be utilized, includes:

[0012] The image to be identified is input into a pre-trained human key point recognition model to obtain the positions of each specified human key point of the person to be identified.

[0013] The yaw angle of the person to be identified is calculated based on the position of each designated key point on the human body, and is used as the yaw angle to be utilized.

[0014] A preset attitude containing the yaw angle to be used is determined in the corresponding yaw angle range, and is used as the preset attitude to be used for the person to be identified in the image to be identified.

[0015] In one alternative embodiment, the method is applied to an identification device;

[0016] After determining the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified, the method further includes:

[0017] An update task is sent to an update device, which, based on the update task, fuses the image features to be identified and the reference image features of the target reference person in the preset pose to be utilized, provided that the image features to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person; wherein, the update task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the image features to be identified;

[0018] or,

[0019] When the image features to be identified meet the preset merging conditions, the image features to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain new reference image features of the target reference person.

[0020] The preset merging conditions include at least one of the following:

[0021] The confidence level of each specified human key point of the person to be identified output by the human key point recognition model is greater than the preset confidence threshold.

[0022] The angular deviation between the yaw angle to be utilized and the designated center yaw angle is less than a preset angle threshold; the designated center yaw angle represents the midpoint of the yaw angle interval corresponding to the preset attitude to be utilized;

[0023] In the image area to be identified, the proportion of a specified human body part of the person to be identified that is obscured is no greater than a preset ratio.

[0024] In an alternative embodiment, the update task further includes determining the moment when the re-identification result of the person to be identified is determined;

[0025] The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain new reference image features of the target reference person, including:

[0026] By adjusting the parameters, the fusion weights used in the previous update of the reference image features of the target reference figure in the preset pose to be utilized are adjusted to obtain new fusion weights;

[0027] The adjustment parameters include the available time and / or the angle deviation; the available time is the time between the time when the re-identification result of the person to be identified is determined and the time of the last update; when the adjustment parameters include the available time, the new fusion weight is positively correlated with the available time; when the adjustment parameters include the angle deviation, the new fusion weight is negatively correlated with the angle deviation.

[0028] The new fusion weights are used as the weights when fusing the features of the image to be identified. The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain the new reference image features of the target reference person.

[0029] In an optional embodiment, when the adjustment parameter includes the duration to be utilized, the new fusion weight is expressed as:

[0030]

[0031] This indicates the new fusion weights; Indicates the fusion weights used in the last update; Represents a coefficient greater than 0; This indicates the duration to be utilized; This represents a function with the desired duration as the independent variable, and is positively correlated with the desired duration; when the desired duration is less than a preset duration threshold, Greater than 0 and less than 1; The maximum value is the preset coefficient threshold.

[0032] In an optional embodiment, before determining the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified, the method further includes:

[0033] For each reference person, if the reference person does not have reference image features in the preset pose to be used, the image features to be identified are compared with the reference image features of the reference person in the specified preset pose.

[0034] In one optional embodiment, each preset posture includes: a frontal posture, a side posture, and a rear posture; the specified preset posture represents the frontal posture.

[0035] A second aspect of this application provides a re-identification device, the device comprising:

[0036] The image acquisition module is used to acquire the image to be recognized;

[0037] The recognition module is used to determine the pose of the person to be recognized contained in the image to be recognized from various preset poses, as the preset pose to be used; and to extract the image features of the area of ​​the image to be recognized occupied by the person to be recognized, as the image features to be recognized.

[0038] The first comparison module is used to compare the image features to be identified with the reference image features of each reference person in the preset pose to be used; wherein, the reference image features of any reference person in a preset pose are obtained by fusing the image features of the reference person in the preset pose that have been identified in the past.

[0039] The determination module is used to determine the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified.

[0040] In an optional embodiment, the recognition module is specifically used to input the image to be recognized into a pre-trained human key point recognition model to obtain the positions of each specified human key point of the person to be recognized;

[0041] The yaw angle of the person to be identified is calculated based on the position of each designated key point on the human body, and is used as the yaw angle to be utilized.

[0042] A preset attitude containing the yaw angle to be used is determined in the corresponding yaw angle range, and is used as the preset attitude to be used for the person to be identified in the image to be identified.

[0043] In one alternative embodiment, the device is applied to an identification device;

[0044] The device further includes:

[0045] The sending module is used to send an update task to the updating device, so that the updating device, based on the update task, fuses the image features to be identified and the reference image features of the target reference person in the preset pose to be utilized, when the image features to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person; wherein, the update task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the image features to be identified;

[0046] or,

[0047] The fusion module is used to fuse the features of the image to be identified and the reference image features of the target reference person in the preset pose to be used, when the features of the image to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person.

[0048] The preset merging conditions include at least one of the following:

[0049] The confidence level of each specified human key point of the person to be identified output by the human key point recognition model is greater than the preset confidence threshold.

[0050] The angular deviation between the yaw angle to be utilized and the designated center yaw angle is less than a preset angle threshold; the designated center yaw angle represents the midpoint of the yaw angle interval corresponding to the preset attitude to be utilized;

[0051] In the image area to be identified, the proportion of a specified human body part of the person to be identified that is obscured is no greater than a preset ratio.

[0052] In an alternative embodiment, the update task further includes determining the moment when the re-identification result of the person to be identified is determined;

[0053] The fusion module is specifically used to adjust the fusion weights used in the previous update of the reference image features of the target reference person in the preset pose to be used by adjusting the parameters, so as to obtain new fusion weights.

[0054] The adjustment parameters include the available time and / or the angle deviation; the available time is the time between the time when the re-identification result of the person to be identified is determined and the time of the last update; when the adjustment parameters include the available time, the new fusion weight is positively correlated with the available time; when the adjustment parameters include the angle deviation, the new fusion weight is negatively correlated with the angle deviation.

[0055] The new fusion weights are used as the weights when fusing the features of the image to be identified. The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain the new reference image features of the target reference person.

[0056] In an optional embodiment, when the adjustment parameter includes the duration to be utilized, the new fusion weight is expressed as:

[0057]

[0058] This indicates the new fusion weights; Indicates the fusion weights used in the last update; Represents a coefficient greater than 0; This indicates the duration to be utilized; This represents a function with the desired duration as the independent variable, and is positively correlated with the desired duration; when the desired duration is less than a preset duration threshold, Greater than 0 and less than 1; The maximum value is the preset coefficient threshold.

[0059] In an optional embodiment, the apparatus further includes:

[0060] The second comparison module is used to compare the image features to be identified with the reference image features of the reference person in the specified preset posture before determining the target reference person corresponding to the reference image features that match the image features to be identified as the re-identification result of the reference person.

[0061] In one optional embodiment, each preset posture includes: a frontal posture, a side posture, and a rear posture; the specified preset posture represents the frontal posture.

[0062] A third aspect of this application provides a re-identification system, the system comprising: an identification device and an updating device;

[0063] The identification device is used to perform any of the methods described in the first aspect above;

[0064] The updating device is configured to receive an updating task sent by the recognition device, and, when the features of the image to be recognized meet the preset merging conditions, fuse the features of the image to be recognized and the reference image features of the target reference person in the preset pose to be utilized, to obtain new reference image features of the target reference person; wherein, the updating task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the features of the image to be recognized.

[0065] A fourth aspect of this application provides an identification device, comprising:

[0066] Memory, used to store computer programs;

[0067] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0068] The fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described in the first aspect above.

[0069] The sixth aspect of this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described in the first aspect above.

[0070] Beneficial effects of the embodiments in this application:

[0071] The re-identification method provided in this application can acquire an image to be identified, determine the pose of the person to be identified contained in the image to be identified from various preset poses, and use this as a preset pose to be utilized; and extract the image features of the region of the image to be identified occupied by the person to be identified. Then, the image features to be identified are compared with the reference image features of each reference person in the preset pose to be utilized; the reference image features of any reference person in a preset pose are obtained by fusing the image features of the already identified reference person in that preset pose. The target reference person corresponding to the reference image features that match the image features to be identified is determined as the re-identification result of the person to be identified.

[0072] Based on the above processing, for any reference person, the reference image features of the reference person in the preset pose to be used are compared with the image features to be identified only once, which can reduce the number of feature comparisons and improve the efficiency of re-identification.

[0073] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0075] Figure 1 A flowchart illustrating a re-identification method provided in an embodiment of this application;

[0076] Figure 2 This is a schematic diagram of the overall process of re-identification provided in an embodiment of this application;

[0077] Figure 3 A structural diagram of a re-identification device provided in an embodiment of this application;

[0078] Figure 4This is a structural diagram of an identification device provided in an embodiment of this application. Detailed Implementation

[0079] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0080] To address the technical problem of low efficiency in re-identification, embodiments of this application provide a re-identification method, system, apparatus, identification device, and storage medium.

[0081] The re-identification method provided in the embodiments of this application will be described first. In some embodiments, the method is applied to an identification device that provides computing services, such as a server or other device used for re-identification of images.

[0082] See Figure 1 , Figure 1 A flowchart of a re-identification method provided in this application embodiment, the method including the following steps:

[0083] S101: Obtain the image to be recognized.

[0084] S102: Determine the pose of the person to be identified contained in the image to be identified from each preset pose, and use it as the preset pose to be used; and extract the image features of the area of ​​the image to be identified occupied by the person to be identified, and use it as the image features to be identified.

[0085] S103: Compare the features of the image to be identified with the features of the reference image of each reference person in the preset pose to be used.

[0086] The reference image features of any reference figure in a preset pose are obtained by fusing the image features of the reference figure in the preset pose that have been identified in the past.

[0087] S104: The target reference person whose features match the features of the image to be identified is determined as the re-identification result of the person to be identified.

[0088] Based on the re-identification method provided in the embodiments of this application, for any reference person, the reference image features of the reference person in the preset pose to be used are compared with the image features to be identified only once, which can reduce the number of feature comparisons and improve the efficiency of re-identification.

[0089] For step S101, the image to be identified is the image of the person in which the identification is needed.

[0090] For example, when dealing with a video captured by an image acquisition device, the people in each video frame can be identified sequentially according to their timestamps, and the people in each video frame can be associated to determine the same person in each video frame. In this case, the video frame currently acquired by the device, according to the timestamp order, is the image to be identified.

[0091] For example, when dealing with multiple videos captured by different image acquisition devices, the people in each video can be associated, that is, the same person can be identified in each video. In this case, the recognition device can extract a video frame from each video as the current image to be recognized.

[0092] For step S102, the number of preset poses and the specific pose types can be determined based on the scene of the image to be recognized. For example, if the person's pose in the scene of the image to be recognized is a standing pose or a squatting pose, then each preset pose includes a standing pose and a squatting pose.

[0093] Alternatively, if the person in the scene of the image to be recognized is in a frontal, side, or back posture, then the preset postures include frontal, side, and back postures. A frontal posture means that the person in the image to be recognized is facing the image acquisition device from the front; a side posture means that the person in the image to be recognized is facing the image acquisition device from the side; and a back posture means that the person in the image to be recognized is facing the image acquisition device from the back.

[0094] In some embodiments, the image to be recognized can be input into a pre-trained pose recognition model to obtain the probability that the pose of the person in the image is a preset pose, and the preset pose with the highest probability is determined as the preset pose to be used. For example, a deep learning-based classification model can be trained using a first sample image and its corresponding label to obtain a pose recognition model. The person in the first sample image is a preset pose, and the label corresponding to the first sample image represents the pose of the person in the first sample image.

[0095] In some embodiments, the method of determining the attitude to be utilized in step S102 may include the following steps:

[0096] Step 1: Input the image to be recognized into the pre-trained human key point recognition model to obtain the position of each specified human key point of the person to be recognized.

[0097] Step 2: Calculate the yaw angle of the person to be identified based on the position of each specified key point on the human body, and use it as the yaw angle to be utilized.

[0098] Step 3: Determine the preset attitude that includes the yaw angle to be used in the corresponding yaw angle range, and use it as the preset attitude to be used for the person to be identified in the image to be identified.

[0099] In this embodiment of the application, the identified key points of the human body are used to determine the yaw angle of the human body. Therefore, the key points of the human body can be set in combination with the specific algorithm for determining the yaw angle of the human body and business requirements.

[0100] Human body keypoint recognition models can be Open Pose, Alpha Pose, or YOLO-Pose, etc. The human body yaw angle represents the rotation angle of the human body relative to the image plane in an image.

[0101] For example, if a high level of recognition accuracy is required, a number of human body key points can be set as designated human body key points. In this case, the designated human body key points can include: shoulder key points, hip key points, and ear key points; accordingly, the human body yaw angle is calculated based on the shoulder key points, hip key points, and ear key points.

[0102] If the accuracy requirement for recognition is not high, or to reduce the requirements for device performance and computational load, fewer human body key points can be set as designated human body key points to improve recognition efficiency. In this case, designated human body key points can include: left key point and right key point; correspondingly, the human body yaw angle is calculated based on the pixel coordinates of the left key point and right key point.

[0103] A preset posture corresponds to a yaw angle range. If the yaw angle of a person in an image belongs to this yaw angle range, then the posture of the person in the image is the preset posture.

[0104] For example, each preset pose includes a frontal pose, a side pose, and a back pose. If the person faces the image acquisition device with their front end completely perpendicular to the acquisition direction, the person's yaw angle is 0°. A positive yaw angle is defined as the person's front end yawing to the right of the image, and a negative yaw angle is defined as the person's front end yawing to the left of the image. Then, the yaw angle range corresponding to the frontal pose can be [-45°, 45°]; the yaw angle range corresponding to the side pose can be (45°, 135°) and [-135°, -45°]; and the yaw angle range corresponding to the back pose can be (-180°, -135°) and (135°, 180°).

[0105] Based on the above processing, according to the key points of the human body, the preset pose of the person in the image can be accurately determined, thus improving the accuracy of re-identification.

[0106] In some embodiments, the recognition device may utilize an image feature extraction network to extract features from the image to be recognized. For example, the image feature extraction network may be a convolutional neural network (CNN) or a visual geometry group (VGG) network.

[0107] Regarding step S103 above, the image features of the image region occupied by the person (i.e., the reference person) in the currently re-identified image can be recorded, which are the reference image features. It is understood that since the same person may have different poses in different images, for the same person, if the person has multiple preset poses in multiple re-identified images, then the reference image features of the person under each of these preset poses are recorded. If, for a preset pose, the person does not exist in the re-identified image, then the reference image features of the person under that preset pose are not currently recorded.

[0108] In addition, since the same person may have the same preset pose in multiple images that have been re-identified, the image features of the image regions occupied by the person in the multiple images can be fused to obtain the reference image features of the person in the preset pose.

[0109] Since the features of the same person differ in different poses, after determining the pose of the person to be identified in the image to be identified (i.e., the preset pose to be used), the features of the image to be identified are compared with the features of a reference image of a reference person in the same pose (i.e., the preset pose to be used) (called the comparison image features). This improves the accuracy of the comparison results, and thus improves the accuracy of re-identification. It can be understood that for a person to be identified, it is only necessary to compare the features of that person's image to be identified with the reference image features of each reference person in one pose. In this way, the number of feature comparisons can be reduced, improving the efficiency of re-identification.

[0110] For example, the feature similarity between the features of the image to be identified and the features of each image to be compared can be calculated as the comparison result.

[0111] For step S104, the image feature with the highest feature similarity is matched with the image feature to be identified. The reference person to which the image feature belongs is the target reference person, which is the re-identification result of the person to be identified.

[0112] In some embodiments, if the maximum value among the calculated feature similarities is less than a preset threshold, it indicates that there are no reference image features that match the features of the image to be identified. That is, the person to be identified is not among the reference persons, and the person to be identified does not appear in the image that has already been re-identified. For example, the preset threshold can be 98% or 95%. In this case, the person to be identified can be used as a new reference person, and the features of the image to be identified can be used as the reference image features of the new reference person in the preset pose to be utilized.

[0113] As described in the preceding embodiments, for a reference person, the reference image features of that reference person in the preset pose to be utilized may not currently be recorded. For example, the reference person in the image that has already undergone re-identification may not be in the preset pose to be utilized. In this case, the features of the image to be identified are compared with the reference image features of the reference person in the specified preset pose, and the comparison result is also used to determine whether the person to be identified is the reference person. In this way, missed detections can be avoided to a certain extent.

[0114] In the examples above, where the preset poses include frontal, side, and back poses, since the features of the person in the image are richer when in a frontal pose, specifying a frontal pose as the preset pose can improve the accuracy of re-identification. Furthermore, in this case, when a new person is identified, if the new person is not in the specified preset pose in the image, then the new person is not considered a reference person; conversely, if the new person is in the specified preset pose in the image, then the new person is considered a reference person, and its image features are recorded as reference image features.

[0115] In some embodiments, after identifying the target reference person, the recognition device can fuse the image features to be recognized and the reference image features of the target reference person in a preset pose to be utilized. The fused image features are the new reference image features of the target reference person.

[0116] In some embodiments, the recognition device may further detect whether the image features to be recognized meet preset merging conditions. If the preset merging conditions are met, the image features to be recognized and the reference image features of the target reference person in the preset pose to be utilized are fused. If the preset merging conditions are not met, the image features to be recognized and the reference image features of the target reference person in the preset pose to be utilized are not fused; that is, the reference image features of the target reference person in the preset pose to be utilized are not updated.

[0117] The default merge criteria include at least one of the following:

[0118] Condition 1: The confidence level of each specified human key point of the person to be identified output by the human key point recognition model is greater than the preset confidence threshold.

[0119] When using a human keypoint recognition model to detect specified human keypoints of a person to be identified, the model also outputs the confidence score for each specified human keypoint. The confidence score represents the probability that the identified specified human keypoint is a true specified human keypoint.

[0120] In one approach, condition 1 is determined to be satisfied when the confidence scores of all specified human key points output by the human key point recognition model are greater than a preset confidence threshold.

[0121] In another approach, the mean confidence score of each specified human keypoint output by the human keypoint recognition model is calculated. If the mean score is greater than a preset confidence threshold, condition 1 is considered satisfied. For example, the following formula can be used to determine whether condition 1 is satisfied.

[0122]

[0123] This represents the mean confidence score of each specified human key point output by the human key point recognition model. Indicates the number of key points on the specified human body; This indicates a preset confidence threshold; Indicates the first The confidence level of a specified human key point. For example, the default confidence threshold is 0.9 or 0.95.

[0124] Based on condition 1, when the confidence level of each specified human key point is high, the reference image features of the target reference person in the preset pose to be used are updated using the features of the image to be identified. This avoids the reference image features from being contaminated to a certain extent, improves the accuracy of the reference image features, and further improves the accuracy of re-identification.

[0125] Condition 2: The angular deviation between the yaw angle to be utilized and the specified center yaw angle is less than the preset angle threshold; the specified center yaw angle represents the midpoint of the yaw angle interval corresponding to the preset attitude to be utilized.

[0126] For example, in the above example, the yaw angle range corresponding to the frontal attitude is [-45°, 45°]; the yaw angle range corresponding to the side attitude is (45°, 135°) and [-135°, -45°]; and the yaw angle range corresponding to the rear attitude is (-180°, -135°) and (135°, 180°). The midpoint of the yaw angle range corresponding to the frontal attitude is 0°, the midpoint of the yaw angle range corresponding to the side attitude is 90° and -90°, and the midpoint of the yaw angle range corresponding to the rear attitude is 180°. In this case, the preset angle threshold can be 30° or 25°.

[0127] As described in the preceding embodiments, each preset posture corresponds to a yaw angle range. If the angular deviation between the yaw angle to be utilized and the specified center yaw angle is less than a preset angle threshold, it indicates that the preset posture to be utilized can accurately reflect the current posture of the person to be identified. In this case, the reference image features of the target reference person under the preset posture to be utilized are updated using the features of the image to be identified. This avoids the reference image features from being contaminated to a certain extent, improves the accuracy of the reference image features, and further improves the accuracy of re-identification.

[0128] Condition 3: In the image area to be identified, the proportion of the specified human body parts of the person to be identified that are occluded is not greater than a preset ratio.

[0129] The designated body parts can be determined based on business requirements. In one implementation, the designated body parts represent the core torso of a person. For example, the designated body parts may include the head, torso, and legs; or, the designated body parts may include the head, torso, legs, and arms.

[0130] In some embodiments, the image region to be identified can be input into a pre-trained occlusion detection model to obtain occlusion detection results. The occlusion detection results indicate the parts of a person that are occluded. For example, using a second sample image and its corresponding label, a deep learning-based classification model can be trained to obtain an occlusion detection model. Different parts of a person in the second sample image are occluded, and the label corresponding to the second sample image represents the occluded parts of the person in the second sample image.

[0131] For example, if the specified human body part contains 4 parts, the preset ratio is 1 / 2, and the occlusion detection result of the image area to be identified indicates that 3 parts are occluded, then it is determined that condition 3 is not met.

[0132] Based on the above processing, if the proportion of the specified human body part of the person to be identified being occluded is not greater than the preset ratio, it indicates that the image features of the person to be identified have a high degree of integrity. In this case, the reference image features of the target reference person under the preset pose to be used are updated using the image features to be identified, which avoids the reference image features from being contaminated to a certain extent, improves the accuracy of the reference image features, and further improves the accuracy of re-identification.

[0133] In some embodiments, after identifying the target reference person, the identification device itself does not update the reference image features of the target reference person in the preset pose to be utilized, but instead sends an update task to the updating device. The update task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the image features to be identified.

[0134] Accordingly, the updating device updates the reference image features of the target reference person in the preset pose to be utilized based on the updating task. It can be understood that if the updating device also determines whether the features of the image to be recognized meet the preset merging conditions before updating, and updates the reference image features of the target reference person in the preset pose to be utilized, then the updating task may also include: determining the time of the re-recognition result of the person to be recognized, the yaw angle to be utilized, the confidence level of each specified human key point, and the proportion of the specified human body parts of the person to be recognized that are occluded.

[0135] The method by which the updating device updates the reference image features can refer to the method by which the recognition device updates the reference image features in the above embodiment. That is to say, in this application, whether the recognition device updates the image features itself or the updating device updates the image features through an update task, the updating logic is the same. For example, both update the reference image features of the target reference person in the preset pose to be used when it is determined that the image features to be recognized meet the preset merging conditions.

[0136] In some embodiments, the recognition device can be a front-end device for re-recognition, and the updating device can be a back-end device for updating image features.

[0137] In one implementation, reference image features are recorded in a database. The recognition device pushes update tasks to a distributed task queue. Correspondingly, an independent thread pool of the update device continuously monitors the distributed task queue, retrieves update tasks from the queue, performs image feature fusion, and, after fusion, writes the data to the database silently. The distributed task queue can be a Redis Queue or a thread pool queue.

[0138] Based on the above processing, an asynchronous architecture for front-end recognition and back-end updates is implemented to achieve non-blocking recognition and achieve the "zero latency" effect of front-end recognition.

[0139] In some embodiments, the features of the image to be identified and the features of the reference image of the target reference person in the preset pose to be utilized can be fused using any of the following methods.

[0140] Method 1: The features of the image to be identified and the features of the reference image of the target person in the preset pose are fused with equal weights. This can be understood as the weight of the features of the image to be identified and the reference image features being 1.

[0141] Method 2: By adjusting the parameters, the fusion weights used in the last update of the reference image features of the target reference person in the preset pose to be used are adjusted to obtain new fusion weights. Then, the new fusion weights are used as the weights when fusing the features of the image to be recognized, and the features of the image to be recognized and the reference image features of the target reference person in the preset pose to be used are fused.

[0142] It is understandable that as the re-identification process continues, the reference image features of the target reference person in the preset pose to be utilized will be updated multiple times. Therefore, the fusion weights used each time the reference image features of the target reference person in the preset pose to be utilized are updated can be recorded.

[0143] When it is necessary to update the reference image features of the target reference person in the preset pose to be utilized, a new fusion weight is determined based on the fusion weight used in the previous step. Then, this new fusion weight is used as the weight of the new image features of the target reference person identified this time, and fused with the existing reference image features of the target reference person in the preset pose to be utilized. During each fusion, the sum of the weights of the existing reference image features of the target reference person in the preset pose to be utilized and the weights of the new image features of the target reference person identified this time is 1.

[0144] In some embodiments, in order to unify the data and improve computational efficiency, the fusion result can be normalized and used as the updated reference image feature.

[0145] For example, the features of the image to be identified and the features of the reference image can be fused using the following formula.

[0146]

[0147] Indicates the updated reference image features. express Normalization processing This represents the features of an existing target reference figure in a preset pose to be utilized. Indicates the features of the image to be identified. This indicates the fusion weight.

[0148] Since the waiting time is the time between determining the re-identification result of the person to be identified and the last update of the reference image features of the target reference person in the waiting preset pose, a longer waiting time means a longer period since the last update of the reference image features of the target reference person in the waiting preset pose, and a greater possibility that the features of the target reference person have changed. To make the fused reference image features closer to the current true state of the target reference person, a larger fusion weight is set. Therefore, when adjusting parameters including the waiting time, the new fusion weight is positively correlated with the waiting time.

[0149] This application does not limit the method for determining new fusion weights based on the duration of use, as long as the determined new fusion weights are positively correlated with the duration of use. In this way, when the time span is large, the appearance or environment representing the person has undergone objective changes, and the new features are given greater weights, which accelerates the "drift" of the reference image features in the library to the current state, thereby adapting to the scenario of the person changing clothes seasonally (such as changing from a down jacket to a shirt).

[0150] Since a smaller angular deviation between the yaw angle to be utilized and the specified center yaw angle indicates that the preset posture to be utilized better reflects the current true posture of the person to be identified, a larger fusion weight is set to make the features of the fused reference image closer to the true state of the target reference person when using the preset posture. Therefore, when adjusting parameters including angular deviation, the new fusion weight is negatively correlated with the angular deviation. This application does not limit the method of determining the new fusion weight based on the angular deviation, as long as the determined new fusion weight is negatively correlated with the angular deviation.

[0151] In some embodiments, where the adjustment parameters include the duration to be utilized, the new fusion weight is expressed as:

[0152]

[0153] This indicates the new fusion weights; Indicates the fusion weights used in the last update; Represents a coefficient greater than 0; Indicates the duration to be utilized; This represents a function with the available usage time as the independent variable, and it is positively correlated with the available usage time; when the available usage time is less than a preset time threshold, Greater than 0 and less than 1; The maximum value is a preset coefficient threshold. For example, , Indicates the yaw angle to be utilized. Indicates the specified center yaw angle.

[0154] If the usage time is less than a preset time threshold (e.g., one day), Minimal, to prevent overfitting of image features of the same scene within a short period. When the utilization time exceeds a preset time threshold, The coefficient threshold increases linearly with the time remaining until it reaches an upper limit (i.e., a preset coefficient threshold), preventing excessive drift and loss of historical image features. The preset coefficient threshold can be 1.5 times the fusion weight used in the last update, or 1.6 times the fusion weight used in the last update.

[0155] See Figure 2 , Figure 2 A schematic diagram of the overall process of re-identification provided in this application embodiment includes the following steps:

[0156] S201: Obtain the image to be recognized.

[0157] S202: Determine the preset poses to be used for the person to be identified contained in the image to be identified from each preset pose; extract the features of the image to be identified of the area occupied by the person to be identified.

[0158] S203: If the preset pose to be used is a frontal pose, compare the features of the image to be identified with the features of the reference image of the reference person in the frontal pose.

[0159] S204: If the preset pose to be used is a side pose, compare the features of the image to be identified with the features of the reference image of the reference person in the side pose.

[0160] For any reference figure, if the reference figure does not have reference image features in a side view, the image features to be identified are compared with the reference image features of the reference figure in a front view.

[0161] S205: If the preset pose to be used is the back pose, compare the features of the image to be identified with the features of the reference image of the reference person in the back pose.

[0162] For any reference figure, if the reference figure does not have reference image features in a back view, the image features to be identified are compared with the reference image features of the reference figure in a front view.

[0163] S206: Based on the comparison results, determine the re-identification result of the person to be identified, generate an update task, and push it to the distributed task queue.

[0164] The entity performing the above steps S201-S206 is the identification device.

[0165] S207: Extract update tasks from the distributed task queue.

[0166] S208: Determine whether the features of the image to be identified meet the preset merging conditions. If they do, proceed to S209 to S2011; if they do not, proceed to S2012.

[0167] S209: By adjusting the parameters, the fusion weights used in the last update of the reference image features of the target reference figure in the preset pose to be utilized are adjusted to obtain new fusion weights.

[0168] S2010: Using new fusion features, the features of the image to be identified and the features of the reference image of the target reference person in the preset pose to be used are fused.

[0169] S2011: Update the features of the reference image of the target reference person in the database under the preset pose to be used.

[0170] S2012: Discard update tasks.

[0171] The main body responsible for executing the above steps S207-S2012 is the updating device.

[0172] Based on the same inventive concept, this application also provides a re-identification device, see [link to relevant documentation]. Figure 3 The device includes:

[0173] Image acquisition module 301 is used to acquire the image to be recognized;

[0174] The recognition module 302 is used to determine the posture of the person to be recognized contained in the image to be recognized from each preset posture, as the preset posture to be used; and to extract the image features of the area of ​​the image to be recognized occupied by the person to be recognized, as the image features to be recognized.

[0175] The first comparison module 303 is used to compare the image features to be identified with the reference image features of each reference person in the preset posture to be used; wherein, the reference image features of any reference person in a preset posture are obtained by fusing the image features of the reference person in the preset posture that have been identified in the past.

[0176] The determining module 304 is used to determine the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified.

[0177] In an optional embodiment, the recognition module 302 is specifically used to input the image to be recognized into a pre-trained human key point recognition model to obtain the positions of each specified human key point of the person to be recognized;

[0178] The yaw angle of the person to be identified is calculated based on the position of each designated key point on the human body, and is used as the yaw angle to be utilized.

[0179] A preset attitude containing the yaw angle to be used is determined in the corresponding yaw angle range, and is used as the preset attitude to be used for the person to be identified in the image to be identified.

[0180] In one alternative embodiment, the device is applied to an identification device;

[0181] The device further includes:

[0182] The sending module is used to send an update task to the updating device, so that the updating device, based on the update task, fuses the image features to be identified and the reference image features of the target reference person in the preset pose to be utilized, when the image features to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person; wherein, the update task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the image features to be identified;

[0183] or,

[0184] The fusion module is used to fuse the features of the image to be identified and the reference image features of the target reference person in the preset pose to be used, when the features of the image to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person.

[0185] The preset merging conditions include at least one of the following:

[0186] The confidence level of each specified human key point of the person to be identified output by the human key point recognition model is greater than the preset confidence threshold.

[0187] The angular deviation between the yaw angle to be utilized and the designated center yaw angle is less than a preset angle threshold; the designated center yaw angle represents the midpoint of the yaw angle interval corresponding to the preset attitude to be utilized;

[0188] In the image area to be identified, the proportion of a specified human body part of the person to be identified that is obscured is no greater than a preset ratio.

[0189] In an alternative embodiment, the update task further includes determining the moment when the re-identification result of the person to be identified is determined;

[0190] The fusion module is specifically used to adjust the fusion weights used in the previous update of the reference image features of the target reference person in the preset pose to be used by adjusting the parameters, so as to obtain new fusion weights.

[0191] The adjustment parameters include the available time and / or the angle deviation; the available time is the time between the time when the re-identification result of the person to be identified is determined and the time of the last update; when the adjustment parameters include the available time, the new fusion weight is positively correlated with the available time; when the adjustment parameters include the angle deviation, the new fusion weight is negatively correlated with the angle deviation.

[0192] The new fusion weights are used as the weights when fusing the features of the image to be identified. The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain the new reference image features of the target reference person.

[0193] In an optional embodiment, when the adjustment parameter includes the duration to be utilized, the new fusion weight is expressed as:

[0194]

[0195] This indicates the new fusion weights; Indicates the fusion weights used in the last update; Represents a coefficient greater than 0; This indicates the duration to be utilized; This represents a function with the desired duration as the independent variable, and is positively correlated with the desired duration; when the desired duration is less than a preset duration threshold, Greater than 0 and less than 1; The maximum value is the preset coefficient threshold.

[0196] In an optional embodiment, the apparatus further includes:

[0197] The second comparison module is used to compare the image features to be identified with the reference image features of the reference person in the specified preset posture before determining the target reference person corresponding to the reference image features that match the image features to be identified as the re-identification result of the reference person.

[0198] In one optional embodiment, each preset posture includes: a frontal posture, a side posture, and a rear posture; the specified preset posture represents the frontal posture.

[0199] This application embodiment also provides a re-identification system, the system comprising: an identification device and an update device;

[0200] The identification device is used to perform any of the identification methods described in the above embodiments.

[0201] The updating device is configured to receive an updating task sent by the recognition device, and, when the features of the image to be recognized meet the preset merging conditions, fuse the features of the image to be recognized and the reference image features of the target reference person in the preset pose to be utilized, to obtain new reference image features of the target reference person; wherein, the updating task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the features of the image to be recognized.

[0202] In the technical solution of this application, the operations of acquiring, storing, using, processing, transmitting, providing and disclosing user images are all carried out with the user's authorization.

[0203] It should be noted that the images containing people in this embodiment are from a publicly available dataset.

[0204] This application also provides an identification device, such as... Figure 4 As shown, it includes:

[0205] Memory 401 is used to store computer programs;

[0206] When processor 402 executes the program stored in memory 401, it performs the following steps:

[0207] Acquire the image to be recognized;

[0208] The pose of the person to be identified contained in the image to be identified is determined from each preset pose and used as the preset pose to be utilized; and the image features of the region of the image to be identified occupied by the person to be identified are extracted and used as the image features to be identified.

[0209] The image features to be identified are compared with the reference image features of each reference person in the preset pose to be used; wherein, the reference image features of any reference person in a preset pose are obtained by fusing the image features of the reference person in the preset pose that have been identified in the past.

[0210] The target reference person whose features match the features of the image to be identified is determined as the re-identification result of the person to be identified.

[0211] Furthermore, the aforementioned identification device may also include a communication bus and / or a communication interface, with the processor 402, communication interface, and memory 401 communicating with each other via the communication bus.

[0212] The communication bus mentioned in the aforementioned identification device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0213] The communication interface is used for communication between the aforementioned identification device and other devices.

[0214] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0215] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0216] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described multiple identification methods.

[0217] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the re-identification methods described above.

[0218] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0219] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0220] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, apparatus, identification device, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0221] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A re-identification method, characterized in that, The method includes: Acquire the image to be recognized; The pose of the person to be identified contained in the image to be identified is determined from each preset pose and used as the preset pose to be utilized; and the image features of the region of the image to be identified occupied by the person to be identified are extracted and used as the image features to be identified. The image features to be identified are compared with the reference image features of each reference person in the preset pose to be used; wherein, the reference image features of any reference person in a preset pose are obtained by fusing the image features of the reference person in the preset pose that have been identified in the past. The target reference person whose features match the features of the image to be identified is determined as the re-identification result of the person to be identified.

2. The method according to claim 1, characterized in that, The step of determining the pose of the person to be identified contained in the image to be identified from various preset poses, as the preset pose to be utilized, includes: The image to be identified is input into a pre-trained human key point recognition model to obtain the positions of each specified human key point of the person to be identified. The yaw angle of the person to be identified is calculated based on the position of each designated key point on the human body, and is used as the yaw angle to be utilized. A preset attitude containing the yaw angle to be used is determined in the corresponding yaw angle range, and is used as the preset attitude to be used for the person to be identified in the image to be identified.

3. The method according to claim 2, characterized in that, The method is applied to identification devices; After determining the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified, the method further includes: An update task is sent to an update device, which, based on the update task, fuses the image features to be identified and the reference image features of the target reference person in the preset pose to be utilized, provided that the image features to be identified meet the preset merging conditions, to obtain new reference image features of the target reference person; wherein, the update task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the image features to be identified; or, When the image features to be identified meet the preset merging conditions, the image features to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain new reference image features of the target reference person. The preset merging conditions include at least one of the following: The confidence level of each specified human key point of the person to be identified output by the human key point recognition model is greater than the preset confidence threshold. The angular deviation between the yaw angle to be utilized and the designated center yaw angle is less than a preset angle threshold; the designated center yaw angle represents the midpoint of the yaw angle interval corresponding to the preset attitude to be utilized; In the image area to be identified, the proportion of a specified human body part of the person to be identified that is obscured is no greater than a preset ratio.

4. The method according to claim 3, characterized in that, The update task also includes determining the moment when the re-identification result of the person to be identified is determined; The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain new reference image features of the target reference person, including: By adjusting the parameters, the fusion weights used in the previous update of the reference image features of the target reference figure in the preset pose to be utilized are adjusted to obtain new fusion weights; The adjustment parameters include the available time and / or the angle deviation; the available time is the time between the time when the re-identification result of the person to be identified is determined and the time of the last update; when the adjustment parameters include the available time, the new fusion weight is positively correlated with the available time; when the adjustment parameters include the angle deviation, the new fusion weight is negatively correlated with the angle deviation. The new fusion weights are used as the weights when fusing the features of the image to be identified. The features of the image to be identified and the reference image features of the target reference person in the preset pose to be used are fused to obtain the new reference image features of the target reference person.

5. The method according to claim 4, characterized in that, When the adjustment parameters include the duration to be utilized, the new fusion weight is expressed as: ; This indicates the new fusion weights; Indicates the fusion weights used in the last update; Represents a coefficient greater than 0; This indicates the duration to be utilized; This represents a function with the desired duration as the independent variable, and is positively correlated with the desired duration; when the desired duration is less than a preset duration threshold, Greater than 0 and less than 1; The maximum value is the preset coefficient threshold.

6. The method according to any one of claims 1-5, characterized in that, Before determining the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified, the method further includes: For each reference person, if the reference person does not have reference image features in the preset pose to be used, the image features to be identified are compared with the reference image features of the reference person in the specified preset pose.

7. The method according to claim 6, characterized in that, Each preset posture includes: frontal posture, side posture, and back posture; the specified preset posture represents the frontal posture.

8. A re-identification system, characterized in that, The system includes: an identification device and an update device; The identification device is used to perform the method of any one of claims 1-7; The updating device is configured to receive an updating task sent by the recognition device, and, when the features of the image to be recognized meet the preset merging conditions, fuse the features of the image to be recognized and the reference image features of the target reference person in the preset pose to be utilized, to obtain new reference image features of the target reference person; wherein, the updating task includes: the person identifier of the target reference person, the pose identifier of the preset pose to be utilized, and the features of the image to be recognized.

9. A re-identification device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be recognized; The recognition module is used to determine the pose of the person to be recognized contained in the image to be recognized from various preset poses, as the preset pose to be utilized; and to extract the image features of the area of ​​the image to be recognized occupied by the person to be recognized, as the image features to be recognized. The comparison module is used to compare the image features to be identified with the reference image features of each reference person in the preset pose to be used; wherein, the reference image features of any reference person in a preset pose are obtained by fusing the image features of the reference person in the preset pose that have been identified in the past. The determination module is used to determine the target reference person corresponding to the reference image features that match the features of the image to be identified as the re-identification result of the person to be identified.

10. An identification device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images