Target pedestrian tracking method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a target pedestrian tracking method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original pedestrian image, and determining a target region of the original pedestrian image; marking the different target areas, and generating an analyzed image; executing an image embedding operation based on the original pedestrian image and the analyzed image, and constructing a garment enhancement sample; extracting original features of the original pedestrian image, and obtaining body shape features through the original features; obtaining garment enhancement features based on the garment enhancement sample pair; generating a tracking model based on the original features, the body shape features and the garment enhancement features; when the target pedestrian image is obtained, the tracking result for the target pedestrian is generated based on the target pedestrian image through the tracking model, and the performance of the pedestrian re-identification model is effectively improved through a multi-feature fusion mode.

Description

Technical Field

[0001] The present invention relates to the technical field of target pedestrian tracking, and in particular to a target pedestrian tracking method, a target pedestrian tracking device, an electronic device and a computer-readable storage medium. Background Art

[0002] Person re-identification technology aims to retrieve specific target pedestrians from multiple non-overlapping camera views. This technology assumes that different pedestrians appear at different times, locations, and cameras. With the rapid development of deep learning, pedestrian re-identification technology plays an increasingly important role in intelligent monitoring, cross-camera tracking, and smart transportation.

[0003] However, there is a common problem in related technologies when it comes to pedestrian re-identification: they rely too much on the appearance of pedestrians, especially clothing information, to perform identity recognition. This method works well in the short term, but as time goes by and the clothing of pedestrians changes, the accuracy of the model will drop significantly. This is because traditional models are easily misled by people with similar clothing, resulting in incorrect recognition results. Summary of the invention

[0004] The embodiments of the present invention provide a target pedestrian tracking method, device, electronic device and computer-readable storage medium to overcome the above problems or at least partially solve the above problems.

[0005] The embodiment of the present invention discloses a target pedestrian tracking method, comprising:

[0006] Acquire an original pedestrian image, and determine a target area of ​​the original pedestrian image;

[0007] Annotating different target areas to generate parsed images;

[0008] Performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample;

[0009] Extracting original features of the original pedestrian image, and obtaining body shape features through the original features;

[0010] Acquire clothing enhancement features based on the clothing enhancement sample pairs;

[0011] generating a tracking model based on the original features, the body shape features and the clothing enhancement features;

[0012] When the target pedestrian image is acquired, a tracking result for the target pedestrian is generated based on the target pedestrian image by the tracking model.

[0013] Optionally, the original pedestrian image includes a first training target and a second training target, the parsed image includes a first target region of the first training target and a second target region of the second training target, the first target region is an upper garment region or a lower garment region of the first training target, the first target region and the second target region have a corresponding relationship, and the step of performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample includes:

[0014] The image of the first target area is embedded into the second target area to generate an initial clothing enhancement sample for the second training target, and the initial clothing enhancement samples of all training targets are used to form the clothing enhancement sample.

[0015] Optionally, the step of extracting original features of the original pedestrian image and acquiring body shape features through the original features includes:

[0016] Determine the backbone network and body shape extraction module;

[0017] Inputting the original pedestrian image into formula 1 to generate original features;

[0018] The formula 1 is:

[0019] f i =θ(x a )

[0020] Among them, x a represents the original pedestrian image, θ represents the backbone network, f i represents the original features obtained by performing feature extraction operation on the backbone network;

[0021] Inputting the original features into Formula 2 to generate body shape features;

[0022] The formula 2 is:

[0023] f s =M(θ(x a ))

[0024] Wherein, M represents the body shape extraction module, f s Represents the body shape features obtained after passing through the body shape extraction module.

[0025] Optionally, it also includes:

[0026] Parsing the original pedestrian image to generate a pedestrian body shape map;

[0027] Determine a first pixel value of the pedestrian body shape map;

[0028] Determine a second pixel value, a length, a width, and a weight update sample quantity of a feature map including the body shape feature;

[0029] The first pixel value, the second pixel value, the length, the width and the weight update sample quantity are used to calculate a body consistency loss coefficient, and the pedestrian body shape map and the body shape feature are kept consistent based on the body consistency loss coefficient.

[0030] Optionally, the step of calculating the body shape consistency loss coefficient by using the first pixel value, the second pixel value, the length, the width, and the weight update sample quantity includes:

[0031] Input the first pixel value, the second pixel value, the length, the width, and the number of weight update samples into Formula 3 to generate a body shape consistency loss coefficient;

[0032] The formula three is:

[0033]

[0034] Among them, I s (h,w) represents the first pixel value, f s (h, w) represents the second pixel value, H represents the length of the feature map, W represents the width of the feature map, and B represents the number of weight update samples.

[0035] Optionally, the step of acquiring clothing enhancement features based on the clothing enhancement sample pair includes:

[0036] Inputting the clothing enhancement sample into formula 4 to generate clothing enhancement features;

[0037] The formula 4 is:

[0038] f c =θ(x a ′)

[0039] Among them, f c The clothing enhancement features obtained by performing feature extraction operations on the backbone network are x a ' represents the clothing enhancement sample.

[0040] Optionally, the step of generating a tracking model based on the original features, the body shape features and the clothing enhancement features comprises:

[0041] Calculating the cross entropy loss, the difficult triplet loss and the clothing contrast loss of the original features, the body shape features and the clothing enhanced features;

[0042] Determine a balance coefficient, and perform weighting of the cross entropy loss, the difficulty triplet loss, and the clothing contrast loss by the balance coefficient to generate a loss value;

[0043] determining a clothing enhancement contrast loss of the original features;

[0044] Calculating an overall loss for the tracking model based on the clothing enhancement contrast loss and the loss value;

[0045] The tracking model is updated based on the overall loss.

[0046] The embodiment of the present invention further discloses a target pedestrian tracking device, comprising:

[0047] An original pedestrian image acquisition module is used to acquire an original pedestrian image and determine a target area of ​​the original pedestrian image;

[0048] A target area labeling module, used to label different target areas and generate parsed images;

[0049] A clothing enhancement sample construction module, used to perform an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample;

[0050] A body shape feature acquisition module, used to extract original features of the original pedestrian image and acquire body shape features through the original features;

[0051] A clothing enhancement feature acquisition module, used for acquiring clothing enhancement features based on the clothing enhancement sample pairs;

[0052] A tracking model generation module, used to generate a tracking model based on the original features, the body shape features and the clothing enhancement features;

[0053] The tracking result generating module is used to generate a tracking result for the target pedestrian based on the target pedestrian image by using the tracking model when the target pedestrian image is acquired.

[0054] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0055] The memory is used to store computer programs;

[0056] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0057] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0058] The embodiments of the present invention include the following advantages:

[0059] The embodiment of the present invention obtains an original pedestrian image and determines the target area of ​​the original pedestrian image; annotates different target areas to generate a parsed image; performs image embedding operations based on the original pedestrian image and the parsed image to construct a clothing enhancement sample; extracts the original features of the original pedestrian image and obtains body features through the original features; obtains clothing enhancement features based on the clothing enhancement sample; generates a tracking model based on the original features, the body features and the clothing enhancement features; when the target pedestrian image is obtained, the tracking result for the target pedestrian is generated based on the target pedestrian image through the tracking model, thereby achieving the following beneficial effects: by means of multi-feature fusion, the performance of the pedestrian re-identification model is effectively improved. In addition, the scheme also enhances the robustness of the model to clothing changes and improves the generalization ability of the model. This method is of great significance for solving the problem of pedestrian re-identification in actual scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flowchart of a target pedestrian tracking method provided in an embodiment of the present invention;

[0061] Figure 2 is a flow chart of a target pedestrian tracking method provided in an embodiment of the present invention;

[0062] Figure 3 is a schematic flow chart of another target pedestrian tracking method provided in an embodiment of the present invention;

[0063] Figure 4 is a structural block diagram of a target pedestrian tracking device provided in an embodiment of the present invention;

[0064] Figure 5 is a hardware structure block diagram of an electronic device provided in an embodiment of the present invention;

[0065] Figure 6 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Reference Figure 1, shows a flowchart of a target pedestrian tracking method provided in an embodiment of the present invention, which may specifically include the following steps:

[0068] Step 101, obtaining an original pedestrian image, and determining a target area of ​​the original pedestrian image;

[0069] Step 102, marking different target areas to generate parsed images;

[0070] Step 103, performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample;

[0071] Step 104, extracting original features of the original pedestrian image, and obtaining body shape features through the original features;

[0072] Step 105, obtaining clothing enhancement features based on the clothing enhancement sample pairs;

[0073] Step 106, generating a tracking model based on the original features, the body shape features and the clothing enhancement features;

[0074] Step 107 : When the target pedestrian image is acquired, a tracking result for the target pedestrian is generated based on the target pedestrian image by using the tracking model.

[0075] In practical applications, the embodiments of the present invention can be applied to a camera management system when recording film and television programs. Since the recording of a program requires multiple cameras and actors often need to change clothes, the tracking model of the embodiments of the present invention can be used to track the target pedestrians to improve the tracking efficiency of pedestrians.

[0076] In a specific implementation, the embodiment of the present invention can obtain an original pedestrian image.

[0077] The original pedestrian image is a training set. Optionally, the following preprocessing steps may be performed on the training set:

[0078] Image resizing:

[0079] Purpose: Ensure that all images have the same size when input to the model, making it easier for the model to process.

[0080] Method: You can adjust the image to a uniform size by scaling, cropping, etc.

[0081] Data Augmentation:

[0082] Purpose: Increase the diversity of the data set and prevent model overfitting.

[0083] Methods: You can randomly rotate, flip, scale, crop the image, or add noise, etc.

[0084] Common data enhancement methods:

[0085] Random cropping, Random flipping, Random rotation, Color jittering, Gaussian noise, etc.

[0086] Image normalization:

[0087] Purpose: Scale image pixel values ​​to a specific range, usually between 0-1 or -1 to 1.

[0088] Method: Min-Max normalization, Z-score standardization and other methods can be used.

[0089] Remove redundant information:

[0090] Purpose: Remove background information or noise that is irrelevant to the task.

[0091] Method: Redundant information can be removed through methods such as cutout and edge detection.

[0092] Data balance:

[0093] Purpose: Ensure that the number of samples in different categories is balanced to prevent the model from being biased towards the category with a larger number.

[0094] Method: Data can be balanced by oversampling, undersampling, etc.

[0095] Special preprocessing for pedestrian images:

[0096] Human key point detection: For some tasks, such as human posture estimation, human key point detection can be performed first, and the key point information can be extracted as additional features.

[0097] Human attribute annotation: Pedestrians’ gender, age, clothing and other attributes can be annotated as additional supervision information.

[0098] Occlusion handling: If there are pedestrians occluded in the dataset, you can consider using some methods to deal with the occlusion problem, such as inpainting or generating adversarial networks.

[0099] Note: First, we need to obtain a clear image of a pedestrian. Then, in this image, we need to clearly point out the specific area we want to focus on, such as the face, body, etc.

[0100] Beneficial effects: Provides basic data for subsequent image analysis.

[0101] The embodiment of the present invention can also determine the target area of ​​the original pedestrian image.

[0102] In a specific implementation, the embodiment of the present invention can use the existing human body analysis model to perform human body analysis on the original data samples in the training set, divide the target areas such as the head, upper body, upper limbs, lower body, lower limbs and feet of the pedestrian in the original pedestrian image, and obtain the analyzed pedestrian image with the above target areas marked.

[0103] In a specific implementation, the embodiment of the present invention can mark different target areas and generate a parsed image.

[0104] Different human body parts (such as head, torso, limbs) in the original image are annotated and parsed to generate a new image. In the parsed image, each part has a specific color or number to mark the target area.

[0105] Beneficial effects: It can more accurately analyze the characteristics of various parts of the human body and provide more fine-grained information for subsequent feature extraction.

[0106] In a specific implementation, the embodiment of the present invention can perform an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample.

[0107] For example, based on the original data and the parsed pedestrian image, the pedestrian clothing data is enhanced. For samples x under two different identities i ∈y i and x j ∈y j , select x i The partial clothing images of the top or bottom in the sample x are embedded into j In the corresponding upper or lower installation, we get x j Sample x′ after clothing enhancement j Similarly, for each sample x in the training set a The corresponding clothing enhanced sample x′ is obtained in the same way a Used for subsequent model training.

[0108] Of course, the above examples are only for illustration, and those skilled in the art may construct clothing enhancement samples in other ways, for example, by nesting other target areas except pedestrian clothing in the original pedestrian image, and the embodiments of the present invention are not limited to this.

[0109] The embodiment of the present invention extracts original features of the original pedestrian image and obtains body shape features through the original features, thereby extracting some key features from the original image, such as the height, weight, body shape, etc. of the person.

[0110] In practical applications, body shape features are one of the important bases for identifying people and can improve the accuracy of recognition.

[0111] The embodiment of the present invention can also obtain clothing enhancement features based on the clothing enhancement sample pairs, and extract new features from the clothing enhanced image, so that the model can better learn the impact of clothing changes on human body characteristics and improve the model's recognition ability in different clothing scenes.

[0112] Then, the embodiment of the present invention can also generate a tracking model based on the original features, the body shape features and the clothing enhancement features, and when the target pedestrian image is acquired, generate a tracking result for the target pedestrian based on the target pedestrian image through the tracking model.

[0113] For example, when it is necessary to track a target person, the image of the target person is input into the trained model, and the model will output a tracking result, which may include the current position of the target person and some related prediction information.

[0114] It should be noted that the camera management system of the embodiment of the present invention acquires image data with the knowledge and consent of the user and complies with relevant regulations.

[0115] The embodiment of the present invention obtains an original pedestrian image and determines a target area of ​​the original pedestrian image; annotates different target areas to generate a parsed image; performs an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample; extracts original features of the original pedestrian image and obtains body features through the original features; obtains clothing enhancement features based on the clothing enhancement sample; generates a tracking model based on the original features, the body features and the clothing enhancement features; and when a target pedestrian image is obtained, generates a tracking result for the target pedestrian based on the target pedestrian image through the tracking model, thereby achieving the following beneficial effects:

[0116] Enhance model robustness:

[0117] Original feature stream: Learn features that are highly relevant to pedestrian identity directly from the original image, such as facial features, body shape, etc. These features have a strong ability to distinguish pedestrian identities.

[0118] Clothing-enhanced feature flow: By transforming clothing, the model can learn more robust features, that is, features that are not affected by clothing changes. In this way, even if the pedestrian changes clothes, the model can accurately identify his identity.

[0119] Body shape feature flow: Body shape is one of the important bases for distinguishing different pedestrians. By learning the body shape differences between different pedestrians, the model can better capture the individual characteristics of pedestrians and improve recognition accuracy.

[0120] Improve model accuracy:

[0121] Multi-feature fusion: By fusing original features, clothing-enhanced features, and body shape features, the model can learn information about pedestrians from multiple angles, thereby improving the ability to discriminate pedestrian identities.

[0122] Reduce the impact of clothing changes: The introduction of clothing enhancement feature flow effectively reduces the impact of clothing changes on model performance and improves the robustness of the model in practical application scenarios.

[0123] Enhance the generalization ability of the model:

[0124] Data enhancement: Clothing enhancement is equivalent to expanding the training data to a certain extent, increasing the diversity of the data, thereby improving the generalization ability of the model and enabling it to better adapt to pedestrian re-identification tasks in different scenarios.

[0125] In summary, this solution effectively improves the performance of the person re-identification model by fusing multiple features. In addition, this solution also enhances the model's robustness to clothing changes and improves the model's generalization ability. This method is of great significance for solving the problem of person re-identification in practical scenarios.

[0126] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0127] refer to Figure 2 and Figure 3 , Figure 2 is a flow chart of a target pedestrian tracking method provided in an embodiment of the present invention; Figure 3 is a schematic flow chart of another target pedestrian tracking method provided in an embodiment of the present invention;

[0128] Optionally, the original pedestrian image includes a first training target and a second training target, the parsed image includes a first target region of the first training target and a second target region of the second training target, the first target region is an upper garment region or a lower garment region of the first training target, the first target region and the second target region have a corresponding relationship, and the step of performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample includes:

[0129] The image of the first target area is embedded into the second target area to generate an initial clothing enhancement sample for the second training target, and the initial clothing enhancement samples of all training targets are used to form the clothing enhancement sample.

[0130] For example, based on the original data and the parsed pedestrian image, the pedestrian clothing data is enhanced. For samples x under two different identities i ∈y i and x j ∈y j , select x i The partial clothing images of the top or bottom in the sample x are embedded into j In the corresponding upper or lower installation, we get x j Sample x′ after clothing enhancement j Similarly, for each sample x in the training set a The corresponding clothing enhanced sample x′ is obtained in the same way a Used for subsequent model training.

[0131] Through the above method, the clothing of pedestrians in the original pedestrian images can be rearranged and combined to increase the diversity of training data.

[0132] Specifically:

[0133] Select samples of different identities: Select two pedestrian images of different identities from the training set, denoted as x i and x j .

[0134] Partial clothing replacement: From sample x i Extract the local area of ​​the top or bottom (for example, a T-shirt or a pair of pants) from the image, and then replace this local area with the sample x j In this way, we get a new sample x′ j , this sample is overall similar to x j Similar, but with changes in clothing.

[0135] Generate diverse training data: Repeat the above process for each sample in the training set to generate a large number of clothing augmentation samples.

[0136] For example, suppose there are two samples:

[0137] Sample x i : A man wearing a blue T-shirt and jeans;

[0138] Sample x j : Men wearing white shirts and black pants;

[0139] We can take the sample x i Replace the blue T-shirt with sample x j On the white shirt, we get a new sample x′ j : A man wearing a blue T-shirt and black pants.

[0140] In this way, the generalization ability of the model can be improved: by increasing the diversity of clothing styles in the training data, the model can better learn the characteristics of pedestrians in different clothing styles, thereby improving the generalization ability of the model in practical applications.

[0141] Alleviating overfitting: More training data can effectively alleviate the problem of model overfitting, allowing the model to better capture the essential characteristics of the data.

[0142] Enhanced robustness to occlusion and incomplete information: Clothing replacement simulates occlusion or incomplete information that may occur in real scenes, allowing the model to better handle these situations.

[0143] Optionally, the step of extracting original features of the original pedestrian image and acquiring body shape features through the original features includes:

[0144] Determine the backbone network and body shape extraction module;

[0145] Inputting the original pedestrian image into formula 1 to generate original features;

[0146] The formula 1 is:

[0147] f i =θ(x a )

[0148] Among them, x a represents the original pedestrian image, θ represents the backbone network, f i represents the original features obtained by performing feature extraction operation on the backbone network;

[0149] Inputting the original features into Formula 2 to generate body shape features;

[0150] The formula 2 is:

[0151] f s =M(θ(x a ))

[0152] Wherein, M represents the body shape extraction module, f s Represents the body shape features obtained after passing through the body shape extraction module.

[0153] Optionally, it also includes:

[0154] Parsing the original pedestrian image to generate a pedestrian body shape map;

[0155] Determine a first pixel value of the pedestrian body shape map;

[0156] Determine a second pixel value, a length, a width, and a weight update sample quantity of a feature map including the body shape feature;

[0157] The first pixel value, the second pixel value, the length, the width and the weight update sample quantity are used to calculate a body consistency loss coefficient, and the pedestrian body shape map and the body shape feature are kept consistent based on the body consistency loss coefficient.

[0158] Optionally, the step of calculating the body shape consistency loss coefficient by using the first pixel value, the second pixel value, the length, the width, and the weight update sample quantity includes:

[0159] Input the first pixel value, the second pixel value, the length, the width, and the number of weight update samples into Formula 3 to generate a body shape consistency loss coefficient;

[0160] The formula three is:

[0161]

[0162] Among them, I s (h,w) represents the first pixel value, f s (h,w) represents the second pixel value, H represents the length of the feature map, W represents the width of the feature map, B represents the number of weight update samples, and Batch Size refers to the number of samples used each time the model weight is updated. In deep learning, usually all training data is not used at once to update the model weights, but the data is divided into several batches, each batch containing a certain number of samples. In this way, the model can update the weights on each batch, thereby gradually optimizing its performance.

[0163] From the given formula 3, we can see that the body shape consistency loss function attempts to minimize the body shape map I s and body shape characteristics s The difference between.

[0164] Specifically, it calculates the true body shape label I at all pixels in all images s and predict body shape characteristics f s The cross entropy loss between .

[0165] Through the above method, the following beneficial effects can be achieved:

[0166] Enhance the expressiveness of body shape features: By minimizing the body shape consistency loss, the body shape features f learned by the model are forced to s It can better reflect the human body shape information in the image. This helps the model to more accurately capture the shape, size, and position of different parts of the human body.

[0167] Improving the accuracy of human body analysis: body shape features s Figure I s The consistency of the model means that the model is more accurate in segmenting human body parts. This is of great significance for subsequent tasks such as posture estimation and behavior recognition.

[0168] Improve the performance of downstream tasks: Accurate human parsing results can provide more reliable input for various subsequent visual tasks. For example, in the pedestrian re-identification task, accurate human parsing can help the model better focus on the discriminative body parts and improve the recognition accuracy.

[0169] Supervised learning: By introducing body shape graph I s As supervisory information, the human body parsing task can be transformed into a supervised learning problem, thereby utilizing a large amount of labeled data to train the model.

[0170] End-to-end training: The human body parsing network and the body shape feature extraction module can be trained end-to-end together, so that the model can better learn the relationship between body shape features and human body parsing.

[0171] Interpretability: Body shape consistency loss can provide a way to explain the model's predictions. By analyzing the differences between body shape features and body shape maps, we can understand where the model's predictions are good and where the predictions are poor.

[0172] Body shape consistency loss is an effective technical means to improve the performance of human body parsing models. It enhances the model's ability to understand human body structure by forcing the body shape features learned by the model to be consistent with the real human body shape. This method has broad application prospects in the field of computer vision, especially in human-related tasks.

[0173] Optionally, the step of acquiring clothing enhancement features based on the clothing enhancement sample pair includes:

[0174] Inputting the clothing enhancement sample into formula 4 to generate clothing enhancement features;

[0175] The formula 4 is:

[0176] f c =θ(x a ′)

[0177] Among them, fc The clothing enhancement features obtained by performing feature extraction operations on the backbone network are x a ' represents the clothing enhancement sample.

[0178] Optionally, the step of generating a tracking model based on the original features, the body shape features and the clothing enhancement features comprises:

[0179] Calculating the cross entropy loss, the difficult triplet loss and the clothing contrast loss of the original features, the body shape features and the clothing enhanced features;

[0180] Determine a balance coefficient, and perform weighting of the cross entropy loss, the difficulty triplet loss, and the clothing contrast loss by the balance coefficient to generate a loss value;

[0181] determining a clothing enhancement contrast loss of the original features;

[0182] Calculating an overall loss for the tracking model based on the clothing enhancement contrast loss and the loss value;

[0183] The tracking model is updated based on the overall loss.

[0184] The original features, the body shape features and the clothing enhancement features are input into Formula 5, and the feature vectors obtained by the classifier are respectively calculated for their cross entropy losses in the backbone network;

[0185] Formula 5 is:

[0186]

[0187] Among them, p(x i ) represents the identity probability predicted by the model, y i represents the true label of the sample, B represents the Batch size, represents the cross entropy loss of the original features, represents the cross entropy loss of body shape features, Represents the cross entropy loss of clothing enhancement features. Batch Size refers to the number of samples used in each model weight update. In deep learning, usually all training data is not used at once to update the model weights. Instead, the data is divided into several batches, each containing a certain number of samples. In this way, the model can update the weights on each batch, thereby gradually optimizing its performance.

[0188] Input the original feature, the body shape feature and the clothing enhancement feature into Formula 6 to calculate the difficult triplet loss of the original feature, the body shape feature and the clothing enhancement feature;

[0189] Formula 6 is:

[0190]

[0191] Among them, m represents the edge coefficient, represents the feature distance between the anchor box sample and the difficult positive sample, represents the feature distance between the anchor box sample and the difficult negative sample, The hard triplet loss representing the original features, The difficult triplet loss representing body shape features, Difficult triplet loss for representing clothing enhanced features.

[0192] Input the original feature, the body shape feature and the clothing enhancement feature into Formula 7 to calculate the clothing contrast loss of the sample under the original feature, the body shape feature and the clothing enhancement feature;

[0193] Formula 7:

[0194]

[0195] Among them, f j represents the characteristics of pedestrians with different identities similar to the sample clothing, f p Represents the feature information of different clothes from the same identity as the sample, N negative represents the number of negative samples, γ clothes represents the temperature parameter, represents the clothing contrast loss under the original features, represents the clothing contrast loss under body shape features, Represents the clothing contrast loss under clothing enhancement features.

[0196] Determine a balance coefficient, and weight the cross entropy loss, the difficulty triplet loss, and the clothing contrast loss by the balance coefficient to generate a loss value:

[0197]

[0198] Among them, λ1, λ2, α1, α2, β1 and β2 represent balance coefficients.

[0199] The original feature f i The feature f obtained by inputting into the fully connected layer i ″ and clothing enhancement featuresf c Compute the clothing enhancement contrast loss between the two features:

[0200]

[0201] Among them, ||*|| represents the L2 normal form, f″ iThe feature vector representing the output of the fully connected layer.

[0202] The overall loss of the final tracking model training is:

[0203]

[0204] Among them, η1 and η2 represent the balance coefficients.

[0205] The tracking model is updated through the above overall loss. After the model training reaches convergence, the backbone network θ is used in the target pedestrian tracking task to calculate the feature similarity between images, perform pedestrian identity inference, and generate tracking results for the target pedestrian based on the inference results.

[0206] The embodiment of the present invention, through the above method, proposes a target pedestrian tracking method to solve the problem that clothing changes occur in the traditional pedestrian tracking process, resulting in performance degradation. The method effectively improves the robustness of the model in the clothing change scenario by designing a network structure of multi-feature fusion.

[0207] Specifically, through data enhancement and feature learning, the original image is segmented using human body analysis technology, and diversified training data is generated by exchanging local clothing areas of different pedestrian images. This data enhancement strategy helps the model learn more robust and clothing-independent features.

[0208] Three feature extraction modules are designed to realize multi-feature fusion, including:

[0209] The original feature stream extracts features containing rich identity information, such as face, body shape, etc., directly from the original image.

[0210] Clothing-enhanced feature flow: Feature extraction is performed on images that have undergone clothing transformation, focusing on learning the underlying features that are independent of clothing and can better reflect the identity of pedestrians.

[0211] Body shape feature stream: Through the human body analysis results, features related to body shape are extracted to further enhance the model’s ability to perceive individual differences of pedestrians.

[0212] Loss function design:

[0213] Body shape consistency loss: ensures that the extracted body shape features are highly consistent with the human body parsing results, thereby improving the reliability of body shape features.

[0214] Clothing enhancement contrast loss: It forces the original feature stream and the clothing enhancement feature stream to learn similar features, thereby improving the model’s robustness to clothing changes.

[0215] Through the above design, the following beneficial effects can be achieved:

[0216] Enhance the model's robustness to clothing changes: By learning clothing-independent features, the model can better cope with the challenges brought by pedestrian clothing changes.

[0217] Improve the generalization ability of the model: Diverse training data and multi-feature fusion strategies help the model learn more generalizable feature representations.

[0218] Improve the accuracy of the model: The complementarity of different feature flows helps the model identify pedestrians more accurately.

[0219] The embodiment of the present invention generates diversified training data by means of clothing exchange, which effectively solves the impact of clothing changes on model performance. The original features, clothing enhancement features and body shape features are comprehensively utilized to improve the robustness and accuracy of the model. The introduction of body shape consistency loss and clothing enhancement contrast loss further improves the performance of the model.

[0220] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0221] Reference Figure 4 , shows a structural block diagram of a target pedestrian tracking device provided in an embodiment of the present invention, which may specifically include the following modules:

[0222] The original pedestrian image acquisition module 401 is used to acquire the original pedestrian image and determine the target area of ​​the original pedestrian image;

[0223] A target region labeling module 402 is used to label different target regions and generate a parsed image;

[0224] A clothing enhancement sample construction module 403 is used to perform an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample;

[0225] A body shape feature acquisition module 404 is used to extract original features of the original pedestrian image and acquire body shape features through the original features;

[0226] A clothing enhancement feature acquisition module 405, configured to acquire clothing enhancement features based on the clothing enhancement sample pairs;

[0227] A tracking model generation module 406, for generating a tracking model based on the original features, the body shape features and the clothing enhancement features;

[0228] The tracking result generating module 407 is used to generate a tracking result for the target pedestrian based on the target pedestrian image by using the tracking model when the target pedestrian image is acquired.

[0229] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0230] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0231] Memory 503, used for storing computer programs;

[0232] The processor 501 is used to implement the target pedestrian tracking method described in any one of the above embodiments when executing the program stored in the memory 503:

[0233] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0234] The communication interface is used for communication between the above terminal and other devices.

[0235] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0236] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0237] like Figure 6 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 601 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the target pedestrian tracking method described in the above embodiment.

[0238] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0239] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0240] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0241] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0242] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0243] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0244] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0245] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A target pedestrian tracking method, characterized in that: include: Acquire an original pedestrian image, and determine a target area of ​​the original pedestrian image; Annotating different target areas to generate parsed images; Performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample; Extracting original features of the original pedestrian image, and obtaining body shape features through the original features; Acquire clothing enhancement features based on the clothing enhancement sample pairs; generating a tracking model based on the original features, the body shape features and the clothing enhancement features; When the target pedestrian image is acquired, a tracking result for the target pedestrian is generated based on the target pedestrian image by the tracking model.

2. The method according to claim 1, characterized in that: The original pedestrian image includes a first training target and a second training target, the parsed image includes a first target area of ​​the first training target and a second target area of ​​the second training target, the first target area is an upper clothing area or a lower clothing area of ​​the first training target, the first target area and the second target area have a corresponding relationship, and the step of performing an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample includes: The image of the first target area is embedded into the second target area to generate an initial clothing enhancement sample for the second training target, and the initial clothing enhancement samples of all training targets are used to form the clothing enhancement sample.

3. The method according to claim 1, characterized in that The step of extracting original features of the original pedestrian image and obtaining body shape features through the original features comprises: Determine the backbone network and body shape extraction module; Inputting the original pedestrian image into formula 1 to generate original features; The formula 1 is: f i =θ(x a ) Among them, x a represents the original pedestrian image, θ represents the backbone network, f i represents the original features obtained by performing feature extraction operation on the backbone network; Inputting the original features into Formula 2 to generate body shape features; The formula 2 is: f s =M(θ(x a )) Wherein, M represents the body shape extraction module, f s Represents the body shape features obtained after passing through the body shape extraction module.

4. The method according to claim 3, characterized in that Also includes: Parsing the original pedestrian image to generate a pedestrian body shape map; Determine a first pixel value of the pedestrian body shape map; Determine a second pixel value, a length, a width, and a weight update sample quantity of a feature map including the body shape feature; The first pixel value, the second pixel value, the length, the width and the weight update sample quantity are used to calculate a body consistency loss coefficient, and the pedestrian body shape map and the body shape feature are kept consistent based on the body consistency loss coefficient.

5. The method according to claim 4, characterized in that The step of calculating the body shape consistency loss coefficient by using the first pixel value, the second pixel value, the length, the width and the weight update sample quantity comprises: Input the first pixel value, the second pixel value, the length, the width, and the number of weight update samples into Formula 3 to generate a body shape consistency loss coefficient; The formula three is: Among them, I s (h,w) represents the first pixel value, f s (h, w) represents the second pixel value, H represents the length of the feature map, W represents the width of the feature map, and B represents the number of weight update samples.

6. The method according to claim 5, characterized in that The step of acquiring clothing enhancement features based on the clothing enhancement sample pair comprises: Inputting the clothing enhancement sample into formula 4 to generate clothing enhancement features; The formula 4 is: f c =θ(x a ′) Among them, f c The clothing enhancement features obtained by performing feature extraction operation on the backbone network are x a ' represents the clothing enhancement sample.

7. The method according to claim 6, characterized in that The step of generating a tracking model based on the original features, the body shape features and the clothing enhancement features comprises: Calculating the cross entropy loss, the difficult triplet loss and the clothing contrast loss of the original features, the body shape features and the clothing enhanced features; Determine a balance coefficient, and perform weighting of the cross entropy loss, the difficulty triplet loss, and the clothing contrast loss by the balance coefficient to generate a loss value; determining a clothing enhancement contrast loss of the original features; Calculating an overall loss for the tracking model based on the clothing enhancement contrast loss and the loss value; The tracking model is updated based on the overall loss.

8. A target pedestrian tracking device, characterized in that: include: An original pedestrian image acquisition module is used to acquire an original pedestrian image and determine a target area of ​​the original pedestrian image; A target area labeling module, used to label different target areas and generate parsed images; A clothing enhancement sample construction module, used to perform an image embedding operation based on the original pedestrian image and the parsed image to construct a clothing enhancement sample; A body shape feature acquisition module, used to extract original features of the original pedestrian image and acquire body shape features through the original features; A clothing enhancement feature acquisition module, used for acquiring clothing enhancement features based on the clothing enhancement sample pairs; A tracking model generation module, used to generate a tracking model based on the original features, the body shape features and the clothing enhancement features; The tracking result generating module is used to generate a tracking result for the target pedestrian based on the target pedestrian image by using the tracking model when the target pedestrian image is acquired.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images