A half-body occluded pedestrian re-identification method and related device
Patent Information
- Application Number
- CN202110976012.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-24
- Publication Date
- 2026-03-03
- Estimated Expiration
- 2041-08-24
Smart Images
Figure CN113887279B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of depth recognition technology, and in particular to a method and related apparatus for re-identifying pedestrians whose bodies are partially obscured. Background Technology
[0002] With the development of deep learning technology, pedestrian re-identification technology has also been widely applied and developed. However, the current pedestrian re-identification technology suffers a significant drop in recognition rate when the target is occluded, especially in different locations or in front of different cameras. Pedestrians are often obscured by cars, luggage, non-motorized vehicles, or other pedestrians, making it difficult to see the lower half of the pedestrian's body in other scenes or in front of other cameras. This makes it difficult to match pedestrians with occluded lower bodies with pedestrians with full bodies. Summary of the Invention
[0003] This application provides a method and related apparatus for re-identifying pedestrians whose bodies are partially obscured, which solves the problem of re-identifying pedestrians whose bodies are partially obscured and has a high recognition accuracy.
[0004] The first aspect of this application provides a method for pedestrian re-identification when half of the body is occluded, including:
[0005] Obtain an initial image set of the target person to be identified, wherein the initial image set contains at least one image;
[0006] Obtain the dataset to be identified based on the initial set of images to be identified;
[0007] The dataset to be identified is used to generate identification results through a target recognition model, which is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians.
[0008] Based on the recognition results, it is determined whether the similarity between the target individuals in the initial image set to be identified is greater than a first preset value. If so, it is confirmed that the target individuals in the initial image set to be identified are the same person.
[0009] Optionally, before generating recognition results from the dataset to be identified using a target recognition model trained from multiple full-body data samples, half-body data samples, and mixed full-body and half-body data samples of pedestrians, the method further includes:
[0010] Obtain a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian;
[0011] The training sample dataset is obtained from the training sample image set, and the training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset.
[0012] The initial model is trained based on the training sample dataset to obtain the target recognition model, which is a model based on a deep neural network.
[0013] Optionally, the initial model is trained based on the training sample dataset to obtain the target recognition model, wherein the initial model is a model built based on a deep neural network, including:
[0014] Feature vectors of the training sample dataset are extracted using an initial model, which is a model built based on a deep neural network.
[0015] The total loss value is calculated based on the feature vector;
[0016] Determine whether the total loss value is less than a second preset value; if so, determine the initial model as the target recognition model.
[0017] If not, adjust the parameters of the initial model according to the total loss value, and re-input the training sample dataset into the updated initial model.
[0018] Optionally, the step of calculating the total loss value based on the feature vector includes:
[0019] The loss values of different categories of sample data in the feature vector are calculated by using the Id Loss loss function and the Triplet Loss loss function respectively, resulting in three sets of Triplet Loss loss values and three sets of Id Loss loss values. The Id Loss loss function is used to distinguish different categories of sample data in the feature vector, and the Triplet Loss loss function is used to train sample data with small differences.
[0020] The total loss value is calculated based on the Triplet Loss value and the Id Loss value.
[0021] Optionally, after determining whether the similarity of the target individuals in the initial image set to be identified is greater than a first preset value based on the recognition result, the method further includes:
[0022] If not, then it is confirmed that the target person in the initial image set to be identified is not the same person.
[0023] Optionally, obtaining the initial image set of the target person to be identified includes:
[0024] An initial set of images of a target person is obtained through a target localization network, which is used to detect target objects in the images and to pinpoint the position of the target objects in the images.
[0025] Optionally, obtaining the dataset to be identified based on the initial set of images to be identified includes:
[0026] The initial image set to be identified is detected by a detection model, which is used to detect the body contour of the target person and mark the contour coordinates.
[0027] Based on the contour coordinates, crop out the full-body image set and half-body image set of the target person;
[0028] The full-body image set and the half-body image set are classified to obtain the dataset to be identified. The dataset to be identified includes the full-body dataset to be identified, the half-body dataset to be identified, and the mixed full-body and half-body dataset to be identified.
[0029] A second aspect of this application provides a pedestrian re-identification device for pedestrians whose upper body is partially obscured, comprising:
[0030] The first acquisition unit is used to acquire an initial image set of the target person to be identified, wherein the initial image set to be identified contains at least one image;
[0031] The second acquisition unit is used to acquire the dataset to be identified based on the initial image set to be identified;
[0032] The generation unit is used to generate recognition results from the dataset to be identified through the target recognition model, which is trained by multiple full-body data samples, half-body data samples, and mixed full-body and half-body data samples of pedestrians.
[0033] The judgment unit is used to determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0034] The confirmation unit is used to confirm that the target people in the initial image set to be identified are the same person after the judgment unit determines that the similarity between the target people in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0035] Optionally, prior to the generating unit, the apparatus further includes:
[0036] The third acquisition unit is used to acquire a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian.
[0037] The fourth acquisition unit is used to obtain a training sample dataset based on the training sample image set. The training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset.
[0038] The training unit is used to train the initial model based on the training sample dataset to obtain the target recognition model, wherein the initial model is a model built based on a deep neural network.
[0039] Optionally, the training unit includes:
[0040] An extraction module is used to extract feature vectors from the training sample dataset using an initial model, wherein the initial model is a model built based on a deep neural network.
[0041] The calculation module is used to calculate the total loss value based on the feature vector;
[0042] The judgment module is used to determine whether the total loss value is less than a second preset value;
[0043] The determining module is used to determine the initial model as the target recognition model after the judgment module determines, based on the recognition result, that the similarity between the target person in the initial image set to be identified is greater than a first preset value.
[0044] The adjustment module is used to adjust the parameters of the initial model according to the total loss value after the judgment module determines that the similarity of the target person in the initial image set to be identified is less than or equal to a first preset value based on the recognition result, and to re-input the training sample dataset into the initial model after updating the parameters.
[0045] A third aspect of this application provides a pedestrian re-identification device for pedestrians whose bodies are partially obscured, comprising:
[0046] Processor, memory, input / output units, and bus;
[0047] The processor is connected to the memory, input / output unit, and bus;
[0048] The processor performs the following operations:
[0049] Obtain an initial image set of the target person to be identified, wherein the initial image set contains at least one image;
[0050] Obtain the dataset to be identified based on the initial set of images to be identified;
[0051] The dataset to be identified is used to generate identification results through a target recognition model, which is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians.
[0052] Based on the recognition results, it is determined whether the similarity between the target individuals in the initial image set to be identified is greater than a first preset value. If so, it is confirmed that the target individuals in the initial image set to be identified are the same person.
[0053] This application provides a computer-readable storage medium storing a program, which, when executed on a computer, performs any of the pedestrian re-identification methods described in the first aspect.
[0054] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0055] In this application, when it is necessary to confirm whether a pedestrian whose half-body is obscured under different shots is the same person as a pedestrian whose whole body is obscured, it is only necessary to obtain the initial image set of the target person to be identified, obtain the dataset to be identified based on the initial image set, generate the recognition result through the target recognition model, and confirm whether the pedestrian whose half-body is obscured matches the pedestrian whose whole body is obscured based on the recognition result. This solves the problem of re-identification of pedestrians whose half-body is obscured, and the recognition accuracy is high. Attached Figure Description
[0056] Figure 1 This is a flowchart illustrating an embodiment of the pedestrian re-identification method with half of the body obscured in this application.
[0057] Figure 2-1 This is a flowchart illustrating another embodiment of the pedestrian re-identification method with half of the body obscured in this application.
[0058] Figure 2-2 This is a flowchart illustrating another embodiment of the pedestrian re-identification method with half of the body obscured in this application.
[0059] Figure 3 This is a schematic flowchart of an embodiment of the pedestrian re-identification device with half of the body obscured in this application.
[0060] Figure 4 This is a schematic flowchart of another embodiment of the pedestrian re-identification device with half of the body obscured in this application.
[0061] Figure 5 This is a schematic flowchart of another embodiment of the pedestrian re-identification device with half of the body obscured in this application. Detailed Implementation
[0062] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.
[0063] This application provides a method and related apparatus for re-identifying pedestrians whose bodies are partially obscured, which solves the problem of re-identifying pedestrians whose bodies are partially obscured and has a high recognition accuracy.
[0064] Please see Figure 1 One embodiment of the pedestrian re-identification method with half of the body obscured in this application includes:
[0065] 101. Obtain an initial image set of the target person to be identified, wherein the initial image set to be identified contains at least one image;
[0066] It should be noted that when it is necessary to confirm a person's whereabouts and other information, the footage is obtained from multiple surveillance videos. However, the target object is captured from different angles in different cameras. If there are obstacles around, the target object may be obscured to varying degrees. For example, in the first camera, the target object can be captured in its entirety, but in the second camera, due to obstruction, only the upper or lower body can be seen, making it difficult to match the target object with half of its body obscured with the target object with its full body.
[0067] To improve the algorithm's recognition rate, in this embodiment, multiple initial images of the target person taken from different angles are acquired to form an initial image set for recognition, which serves as the basis for recognition. It should be noted that the initial image to be recognized can be a full-body image or a half-body image; there is no specific limitation, and images that can be acquired based on the actual situation can be used as the initial image for recognition.
[0068] It should also be noted that, in this embodiment of the application, the initial image set to be identified can be obtained from the monitoring video collected by the camera device, or it can be obtained by the user uploading to the terminal or entering it on the terminal. The specific method of obtaining the initial image set to be identified is not limited in this embodiment of the application.
[0069] 102. Obtain the dataset to be identified based on the initial set of images to be identified;
[0070] It should be noted that, since the initial image to be identified is an image that contains the target person and may also contain other unrelated objects, in this embodiment of the application, after obtaining the initial image set to be identified, it is also necessary to search the initial image set to be identified, identify the target person in the image, and perform processing operations such as coordinate calibration on the full body or half body of the target person to finally obtain the dataset to be identified.
[0071] 103. Generate recognition results from the dataset to be identified using a target recognition model, wherein the target recognition model is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians;
[0072] It should be noted that in this embodiment of the application, the dataset to be identified is input into the target recognition model, which will automatically perform operations such as loss value calculation on the dataset to be identified according to the pre-trained network structure and loss function, and generate recognition results.
[0073] It should be noted that the training method and recognition method of the target recognition model distance will be described in detail in the following embodiments, and will not be described in the embodiments of this application.
[0074] 104. Based on the recognition result, determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value. If so, proceed to step 105.
[0075] It should be noted that in this embodiment of the application, the recognition result generated by the target recognition model displays the position coordinates of the target person in each image of the initial image set to be recognized, and the similarity of the target person in each image. The similarity is determined to be greater than a preset threshold to confirm whether the target person in the initial image set to be recognized is the same person.
[0076] In this embodiment of the application, if the similarity is greater than the first preset value, then step 105 is executed.
[0077] 105. Confirm that the target person in the initial image set to be identified is the same person.
[0078] It should be noted that, in this embodiment of the application, the similarity of the target persons in the initial image set to be identified reaches a preset value, as determined by the identification results of the target recognition model described above. This indicates that all the target persons in the initial image set to be identified are the same person.
[0079] In this embodiment, when it is necessary to confirm whether a pedestrian whose half-body is obscured under different lenses is the same person as a pedestrian whose whole body is obscured, it is only necessary to obtain the initial image set of the target person to be identified, obtain the dataset to be identified based on the initial image set, generate the recognition result through the target recognition model, and confirm whether the pedestrian whose half-body is obscured matches the pedestrian whose whole body is obscured based on the recognition result. This solves the problem of re-identification of pedestrians whose half-body is obscured, and the recognition accuracy is high.
[0080] The above provides a general overview of pedestrian re-identification methods when half of the body is obscured. The following section will provide a detailed introduction to pedestrian re-identification methods when half of the body is obscured.
[0081] Please see Figure 2-1 and Figure 2-2 Another embodiment of the pedestrian re-identification method with half of the body obscured in this application includes:
[0082] 201. Obtain an initial image set of a target person to be identified through a target localization network, wherein the target localization network is used to detect the target object in the image and mark the position of the target object in the image, and the initial image set to be identified contains at least one image;
[0083] In this embodiment of the application, a target object in a surveillance video or image is detected by a target positioning network, and the coordinate position of the target object is marked so as to locate the whole body, upper body or lower body of the target person, and obtain an initial image set containing the target person to be identified.
[0084] 202. The initial image set to be identified is detected by a detection model, wherein the detection model is used to detect the body contour of the target person and mark the contour coordinates;
[0085] It should be noted that, in order to detect the whole body of the target person in this embodiment, it is necessary to train a detection model in advance to detect the body contour of the target person and mark the contour coordinates. This detection model has a whole body detection model and a half body detection model. Since the datasets used normally are basically full-body photos and are not usually occluded, this embodiment mainly uses the whole body detection model to detect the whole body of pedestrians.
[0086] 203. Based on the contour coordinates, crop out the full-body image set and half-body image set of the target person;
[0087] It should be noted that, in this embodiment of the application, after the body outline of the target person is detected by the detection model, the detected target person will be cropped into an upper half and a lower half according to the coordinates of the detection box.
[0088] It should be noted that, since the main issue in practice is addressing the occlusion of the lower half of the body, this embodiment requires pairing a full-body image and an upper-body image. Because the upper-body image is obtained by cropping the full-body image, the recognition method in this application does not require it. For most solutions to pedestrian re-identification involving half-body occlusion that rely solely on collecting half-body pedestrian images, this application can use both full-body and upper-body images to solve the problem of pedestrian re-identification with half-body occlusion.
[0089] 204. Classify the full-body image set and the half-body image set to obtain a dataset to be identified, wherein the dataset to be identified includes a full-body dataset to be identified, a half-body dataset to be identified, and a mixed full-body and half-body dataset to be identified;
[0090] It should be noted that, in this embodiment of the application, after the detected target person is cropped into an upper and lower half, the cropped image is classified and labeled according to category (e.g., No. 1), ensuring that the half-body image and the full-body image are consistent. Figure 1 Thus, the dataset to be identified was changed from a full-body dataset to three types: a full-body dataset, a half-body dataset, and a mixed full-body and half-body dataset.
[0091] 205. Obtain a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian;
[0092] In order to obtain a recognition model with a high recognition rate, this embodiment of the application requires a large number of training samples for model training.
[0093] It should be noted that, in order to improve the accuracy of pedestrian re-identification, the selection of the training sample image set should be diverse, such as including diversity in pedestrian poses, pedestrian scales, and backgrounds. This application does not limit the method of obtaining the image set; for example, it can be obtained from surveillance video captured by a camera, or it can be obtained directly from an online database based on images uploaded by users. The specific method can be determined as needed.
[0094] 206. Obtain a training sample dataset based on the training sample image set, wherein the training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset;
[0095] It should be noted that, in this embodiment of the application, after obtaining the training sample image set, a series of preprocessing steps such as detection, cropping, and classification are performed through detection models to obtain the training sample dataset. The training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset.
[0096] 207. Extract feature vectors from the training sample dataset using the initial model, wherein the initial model is a model built based on a deep neural network;
[0097] It should be noted that, in the embodiments of this application, before training the target recognition model, a better initial backbone network can be selected for training, such as ResNet, MobileNet, VGGNet, ibnResNet and other commonly used neural network structures. The specific embodiments of this application are not limited.
[0098] In addition, features of the training sample dataset are extracted through the initial model. These features can specifically include pedestrians' clothing color, accessories, body shape, hairstyle, posture, and surrounding buildings.
[0099] 208. The loss values of sample data of different categories in the feature vector are calculated by using the Id Loss loss function and the Triplet Loss loss function respectively, to obtain three sets of Triplet Loss loss values and three sets of Id Loss loss values. The Id Loss loss function is used to distinguish sample data of different categories in the feature vector, and the Triplet Loss loss function is used to train sample data with small differences.
[0100] It should be noted that in this embodiment, the Id Loss loss function and the Triplet Loss loss function are used to jointly supervise training and calculate the loss value of the training sample data. Id Loss is mainly used to distinguish between categories, and its calculation method is the same as that of Softmax. Triplet Loss is mainly used to reduce the distance between sample pairs of the same category and increase the distance between sample pairs of different categories during training. It is very effective for datasets with mixed half-body and full-body images.
[0101] The specific formula for calculating the loss is L triplet =d p -d n +a, where d p d is the normalized Euclidean distance between samples of the same class in a training batch. n It is a normalized Euclidean distance between samples of different classes in a training batch, and 'a' is a distance penalty term, which is usually set to a floating-point decimal between 0 and 1.
[0102] In this embodiment, three datasets—the full-body training dataset, the half-body training dataset, and the mixed full-body and half-body training dataset—are simultaneously fed into a predetermined initial network structure (i.e., the initial model). The three datasets share a single network structure, but the initial model calculates the loss for each dataset based on the number of frames used in each dataset. Therefore, three Triplet Loss and three Id Loss are obtained. The calculation methods for the three Triplet Loss and three Id Loss are shown in the following formulas (1) and (2):
[0103] L triplet =a1*L triplet1 +a2*L triplet2 +a3*L triplet3 Formula (1)
[0104] L id =b1*L id1 +b2*L id2 +b3*L id3 Formula (2)
[0105] Finally, two total Loss values are output. Among them, a1, a2, a3, b1, b2, and b3 are weights between 0 and 1. In this application, more attention is paid to the half-body image, so a2, a3, b2, and b3 can be set to larger values.
[0106] 209. Calculate the total loss value based on the Triplet Loss value and the Id Loss value;
[0107] It should be noted that, in this embodiment of the application, after outputting two total Loss values, the result is then calculated based on L. triplet and L id Calculate the total loss value.
[0108] It should be noted that different datasets have different loss calculation methods and different numbers of samples selected for a single training session. For example, in this embodiment, the batch size for the whole-body training dataset can be set to 48, the batch size for the half-body training dataset can be set to 96, and the batch size for the mixed dataset is 96. The batch size affects the optimization degree and speed of the model. It also directly affects the usage of GPU memory. If your GPU memory is limited, it is best to set this value to a smaller value.
[0109] In addition, in this embodiment of the application, the learning rate can be set to 0.01, and the learning rate is multiplied by 0.1 for every 100,000 training iterations, until the learning rate drops to 0.00001, at which point training ends.
[0110] 210. Determine whether the total loss value is less than the second preset value. If yes, proceed to step 211; if no, proceed to step 212.
[0111] It should be noted that, in this embodiment of the application, in order to confirm whether the detection results of the initial model on the training sample dataset are accurate, it is also necessary to determine whether the total loss value calculated in the aforementioned steps is greater than a preset threshold, i.e., the second preset value. By determining whether the preset threshold is met, the accuracy of the output results of the initial model can be confirmed.
[0112] In this embodiment of the application, if the total loss value is greater than the second preset value, then step 211 is executed; if it is less than or equal to the preset value, then step 212 is executed.
[0113] 211. The initial model is determined as the target recognition model;
[0114] It should be noted that, in this embodiment of the application, if the total loss value is greater than the second preset value, the initial model is determined as the target recognition model.
[0115] 212. If not, adjust the parameters of the initial model according to the total loss value, and re-input the training sample dataset into the updated initial model;
[0116] It should be noted that in this embodiment of the application, if the total loss value is less than or equal to the second preset value, the parameters of the initial model are adjusted according to the total loss value, and the training sample dataset is re-input into the initial model after updating the parameters, and training is performed again until the total loss value is less than the second preset value, and then step 213 is executed.
[0117] 213. Generate recognition results from the dataset to be identified using a target recognition model, wherein the target recognition model is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians;
[0118] It should be noted that step 213 in this embodiment is similar to step 103 in the previous embodiment, and will not be repeated here.
[0119] 214. Based on the recognition result, determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value. If yes, proceed to step 215; otherwise, proceed to step 216.
[0120] It should be noted that in this embodiment of the application, the recognition result generated by the target recognition model displays the position coordinates of the target person in each image of the initial image set to be recognized, and the similarity of the target person in each image. The similarity is determined to be greater than a preset threshold to confirm whether the target person in the initial image set to be recognized is the same person.
[0121] In this embodiment of the application, if the similarity is greater than the first preset value, then step 215 is executed; if it is less than or equal to the first preset value, then step 216 is executed.
[0122] 215. Confirm that the target person in the initial image set to be identified is the same person.
[0123] It should be noted that step 215 in this embodiment is similar to step 105 in the previous embodiment, and will not be described again here.
[0124] 216. Confirm that the target person in the initial image set to be identified is not the same person.
[0125] It should be noted that in this embodiment of the application, if the similarity is less than or equal to the first preset value, it is confirmed that the target person in the initial image set to be identified is not the same person.
[0126] In this embodiment of the application, a target recognition model is specifically trained to address the problem of low re-identification accuracy caused by pedestrian occlusion. This target recognition model does not require separate collection of sample images of occluded half-body for training. Instead, it can be trained by directly cropping half-body images from full-body images. This method is simple, convenient, and the target recognition model trained in this way has a high recognition rate.
[0127] The above describes the method for re-identifying pedestrians whose bodies are partially obscured. The following will explain the re-identification device for pedestrians whose bodies are partially obscured.
[0128] Please see Figure 3 One embodiment of the pedestrian re-identification device with half of the body obscured in this application includes:
[0129] The first acquisition unit 301 is used to acquire an initial image set of the target person to be identified, wherein the initial image set to be identified contains at least one image;
[0130] The second acquisition unit 302 is used to acquire the dataset to be identified based on the initial image set to be identified;
[0131] The generation unit 303 is used to generate recognition results from the dataset to be identified through the target recognition model, wherein the target recognition model is trained from multiple full-body data samples, half-body data samples and mixed full-body and half-body data samples of pedestrians;
[0132] The judgment unit 304 is used to determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0133] The confirmation unit 305 is used to confirm that the target people in the initial image set to be identified are the same person after the judgment unit 304 determines that the similarity between the target people in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0134] In this embodiment, when it is necessary to confirm whether a pedestrian whose half-body is obscured under different lenses is the same person as a pedestrian whose whole body is obscured, it is only necessary to obtain the initial image set of the target person to be identified by the first acquisition unit 301, and obtain the dataset to be identified by the second acquisition unit 302 based on the initial image set to be identified, and then generate the recognition result by the generation unit 303 based on the target recognition model. Based on the recognition result, it is confirmed whether the pedestrian whose half-body is obscured matches the pedestrian whose whole body is obscured, thus solving the problem of re-identification of pedestrians whose half-body is obscured, and the recognition accuracy is high.
[0135] The above provides a general description of the functions of each unit in the pedestrian re-identification device with half of the body obscured. The following section will provide a detailed description of the functions of each unit in the pedestrian re-identification device with half of the body obscured.
[0136] Please see Figure 4 In this application, another embodiment of the pedestrian re-identification device with half of the body obscured includes:
[0137] The first acquisition unit 401 is used to acquire an initial image set of the target person to be identified, wherein the initial image set to be identified contains at least one image;
[0138] The second acquisition unit 402 is used to acquire the dataset to be identified based on the initial image set to be identified;
[0139] The third acquisition unit 403 is used to acquire a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian.
[0140] The fourth acquisition unit 404 is used to obtain a training sample dataset based on the training sample image set. The training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset.
[0141] The training unit 405 is used to train the initial model based on the training sample dataset to obtain the target recognition model, wherein the initial model is a model built based on a deep neural network.
[0142] Optionally, training unit 405 may further include:
[0143] The extraction module 4051 is used to extract feature vectors from the training sample dataset through an initial model, wherein the initial model is a model built based on a deep neural network.
[0144] Calculation module 4052 is used to calculate the total loss value based on the feature vector;
[0145] The judgment module 4053 is used to determine whether the total loss value is less than a second preset value;
[0146] The determining module 4054 is used to determine the initial model as the target recognition model after the judging module 4053 determines, based on the recognition result, that the similarity between the target person in the initial image set to be recognized is greater than a first preset value.
[0147] The adjustment module 4055 is used to adjust the parameters of the initial model according to the total loss value after the judgment module 4053 determines that the similarity of the target person in the initial image set to be identified is less than or equal to a first preset value based on the recognition result, and re-inputs the training sample dataset into the initial model after updating the parameters.
[0148] The generation unit 406 is used to generate recognition results from the dataset to be identified through the target recognition model, wherein the target recognition model is trained from multiple full-body data samples, half-body data samples and mixed full-body and half-body data samples of pedestrians;
[0149] The judgment unit 407 is used to determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0150] The confirmation unit 408 is used to confirm that the target people in the initial image set to be identified are the same person after the judgment unit 407 determines that the similarity between the target people in the initial image set to be identified is greater than a first preset value based on the recognition result.
[0151] In this embodiment, the functions of each unit module are the same as those described above. Figures 1 to 2-1 and Figure 2-2 The steps in the illustrated embodiments are the same and will not be repeated here.
[0152] Please see Figure 5 Another embodiment of the pedestrian re-identification device with half of the body obscured in this application includes:
[0153] Processor 501, memory 502, input / output unit 503, and bus 504;
[0154] The processor 501 is connected to the memory 502, the input / output unit 503, and the bus 504;
[0155] Processor 501 performs the following operations:
[0156] Obtain an initial image set of the target person to be identified, wherein the initial image set contains at least one image;
[0157] Obtain the dataset to be identified based on the initial set of images to be identified;
[0158] The dataset to be identified is used to generate identification results through a target recognition model, which is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians.
[0159] Based on the recognition results, it is determined whether the similarity between the target individuals in the initial image set to be identified is greater than a first preset value. If so, it is confirmed that the target individuals in the initial image set to be identified are the same person.
[0160] In this embodiment, the function of processor 501 is the same as described above. Figures 1 to 2-1 and Figure 2-2 The steps in the illustrated embodiments are the same and will not be repeated here.
[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0162] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A method for pedestrian re-identification when half of the body is obscured, characterized in that, include: Obtain an initial image set of the target person to be identified, wherein the initial image set contains at least one image; The initial image set to be identified is detected by a detection model, which is used to detect the body contour of the target person and mark the contour coordinates; based on the contour coordinates, a full-body image set and a half-body image set of the target person are cropped; the full-body image set and the half-body image set are classified and labeled to obtain the dataset to be identified, which includes a full-body dataset to be identified, a half-body dataset to be identified, and a mixed full-body and half-body dataset to be identified; Obtain a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian; The training sample dataset is obtained from the training sample image set, and the training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset. Feature vectors are extracted from the training sample dataset using an initial model, which is a model built based on a deep neural network. Loss values are calculated for different categories of sample data in the feature vector using the Id Loss and Triplet Loss functions, resulting in three sets of Triplet Loss values and three sets of Id Loss values. The Id Loss function is used to distinguish between different categories of sample data in the feature vector, while the Triplet Loss function is used to train on sample data with relatively small differences. A total loss value is calculated based on the Triplet Loss and Id Loss values. It is then determined whether the total loss value is less than a second preset value. If so, the initial model is identified as a target recognition model; otherwise, the parameters of the initial model are adjusted based on the total loss value, and the training sample dataset is re-input into the updated initial model. The dataset to be identified is used to generate identification results through a target recognition model, which is trained from full-body data samples, half-body data samples, and mixed full-body and half-body data samples of multiple pedestrians. Based on the recognition results, it is determined whether the similarity between the target individuals in the initial image set to be identified is greater than a first preset value. If so, it is confirmed that the target individuals in the initial image set to be identified are the same person.
2. The pedestrian re-identification method according to claim 1, characterized in that, After determining whether the similarity of the target individuals in the initial image set to be identified is greater than a first preset value based on the recognition result, the method further includes: If not, then it is confirmed that the target person in the initial image set to be identified is not the same person.
3. The pedestrian re-identification method according to any one of claims 1 to 2, characterized in that, The acquisition of the initial image set of the target person to be identified includes: An initial set of images of a target person is obtained through a target localization network, which is used to detect target objects in the images and to pinpoint the position of the target objects in the images.
4. A pedestrian re-identification device with half of the body obscured, characterized in that, include: The first acquisition unit is used to acquire an initial image set of the target person to be identified, wherein the initial image set to be identified contains at least one image; The second acquisition unit is used to detect the initial image set to be identified using a detection model. The detection model is used to detect the body contour of the target person and mark the contour coordinates. Based on the contour coordinates, a full-body image set and a half-body image set of the target person are cropped. The full-body image set and the half-body image set are classified and labeled to obtain the dataset to be identified. The dataset to be identified includes a full-body dataset to be identified, a half-body dataset to be identified, and a mixed full-body and half-body dataset to be identified. The third acquisition unit is used to acquire a training sample image set, wherein the training sample image set contains at least one training sample, and the training sample is a full-body image and / or half-body image of a pedestrian. The fourth acquisition unit is used to obtain a training sample dataset based on the training sample image set. The training sample dataset includes a full-body training sample dataset, a half-body training sample dataset, and a mixed full-body and half-body training sample dataset. An extraction module is used to extract feature vectors from the training sample dataset using an initial model, wherein the initial model is a model built based on a deep neural network. The calculation module is used to calculate the loss values of sample data of different categories in the feature vector using the Id Loss loss function and the Triplet Loss loss function, respectively, to obtain three sets of Triplet Loss loss values and three sets of Id Loss loss values. The Id Loss loss function is used to distinguish sample data of different categories in the feature vector, and the Triplet Loss loss function is used to train sample data with small differences. The total loss value is calculated based on the Triplet Loss loss values and the Id Loss loss values. The judgment module is used to determine whether the total loss value is less than a second preset value; The determination module is used to determine the initial model as the target recognition model after the judgment module determines that the total loss value is less than the second preset value; An adjustment module is used to adjust the parameters of the initial model according to the total loss value after the judgment module determines that the total loss value is greater than or equal to the second preset value, and to re-input the training sample dataset into the initial model with updated parameters. The generation unit is used to generate recognition results from the dataset to be identified through the target recognition model, which is trained by multiple full-body data samples, half-body data samples, and mixed full-body and half-body data samples of pedestrians. The judgment unit is used to determine whether the similarity of the target person in the initial image set to be identified is greater than a first preset value based on the recognition result. The confirmation unit is used to confirm that the target people in the initial image set to be identified are the same person after the judgment unit determines that the similarity between the target people in the initial image set to be identified is greater than a first preset value based on the recognition result.
Citation Information
Patent Citations
Local target re-recognition method, device and system
CN109214366A