Shielding pedestrian retrieval method based on human body semantic perception and recovery

Abstract

The invention discloses an occluded pedestrian retrieval method based on human body semantic perception and recovery, and belongs to the technical field of computer vision. According to the method, a multi-branch enhancement strategy is introduced, and the strategy adopts a parallel data enhancement mode and covers reference enhancement, amplification enhancement, reduction enhancement, coverage enhancement and other methods, so that training samples in a shielding scene are greatly enriched, and the model can better adapt to different types of shielding conditions; the robustness and the recognition capability in a complex shielding environment are improved; according to the method, the information extraction capability of Transform is utilized to ensure that the model can sense missing and shielded areas; according to the method, a human body semantic perception method is used for positioning a missing semantic part, then semantic information of a similar positive sample corresponding region in a training set is used for complementing the missing part, the human body semantic recovery capability is learned in a training stage, a model can directly predict a missing region in a testing stage, and the accuracy of human body semantic recovery is improved. And the method does not depend on any additional auxiliary data.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for retrieving occluded pedestrians based on human semantic perception and restoration. Background Art

[0002] Person retrieval, also known as person re-identification (ReID), is a visual task that aims to identify and match the same person based on images of the same person taken by different cameras. In recent years, deep learning technology has significantly improved the overall performance of ReID. However, due to factors such as camera viewing angle, lighting conditions, and posture changes, the ReID task still faces many challenges, among which occlusion is particularly prominent. When a pedestrian is partially obscured by other objects (such as vehicles, luggage, or people), part of the image information is lost, making it difficult to extract complete and accurate visual features. This puts new demands on traditional ReID methods. To meet this challenge, more and more research has begun to focus on the ReID problem in occluded scenarios, aiming to design ReID models that can remain robust under occlusion conditions.

[0003] Data collection in occluded scenarios is inherently challenging. Traditional data collection methods often fail to capture all types and degrees of occlusion, resulting in insufficient model generalization in occluded environments. This makes it difficult for models to cope with the diverse partial and full-body occlusions found in the real world. To address this issue, many researchers have increased the diversity of training data by synthesizing occluded scenes or artificially augmenting existing data, thereby improving the model's adaptability to occluded environments.

[0004] Furthermore, image information in occluded areas is often unhelpful for identity recognition and may even introduce interference information unrelated to identity, thus affecting overall feature extraction. If a deep learning model "mislearns" these noisy features, it may reduce the model's ability to discern differences in pedestrian identities. To overcome this problem, researchers have proposed various methods to reduce the interference introduced by occlusion and ensure that the model can focus on valid features related to identity.

[0005] Furthermore, when key parts of a pedestrian's body are occluded, the feature information captured by the model is often incomplete, resulting in a loss of overall pedestrian representation and, in severe cases, recognition errors. To compensate for this missing feature, many researchers have explored restoring a complete pedestrian representation by reconstructing or inferring occluded areas to improve recognition accuracy.

[0006] Existing technologies have made some progress in retrieval of occluded pedestrians. However, these methods still have some shortcomings, such as overly simplistic preprocessing methods and insufficient consideration of domain differences between the training and test sets. The training set contains fewer occlusion samples, most of which are simple, while the types and degrees of occlusion in the test set are often more complex and diverse. This results in insufficient performance when faced with complex occlusions in real-world scenarios.

[0007] Semantic information about pedestrians plays a crucial role in feature representation, helping models better understand and distinguish different pedestrians. However, existing Transformer-based methods have yet to fully exploit this information. While Transformer models excel at capturing global dependencies, they still lack understanding of the semantic details and contextual information of pedestrians. This is particularly true in the case of occlusion, where missing regions and information often compromise the final recognition performance.

[0008] To solve the above problems, the present invention proposes an occluded pedestrian retrieval method based on human semantic perception and restoration. Summary of the Invention

[0009] The technical problem to be solved by the present invention is: how to effectively deal with the occlusion problem, restore the missing key features, and capture the fine-grained semantic information in pedestrian images, and provide an occluded pedestrian retrieval method based on human semantic perception and restoration.

[0010] The present invention solves the above technical problems through the following technical solutions, which include the following steps:

[0011] S1: Image preprocessing

[0012] The input pedestrian image is preprocessed using a multi-branch data enhancement strategy to obtain an image quadruple;

[0013] S2: Feature Extraction

[0014] The image quadruple is input into a parameter-sharing backbone network TransReID for feature extraction;

[0015] S3: Acquiring true semantic labels

[0016] Use the human body semantic parsing model to obtain the real semantic labels of the human body;

[0017] S4: Human Semantic Perception

[0018] Obtain a semantic mask for each body part, use the semantic mask to obtain a semantic perception score, and extract the semantic features of each body part;

[0019] S5: Human Semantic Restoration

[0020] Based on the semantic perception score, the reconstructed semantic features are obtained to achieve feature reconstruction;

[0021] S6: Model Training

[0022] The model is trained using multiple loss functions to obtain a trained pedestrian retrieval model.

[0023] S7: Pedestrian retrieval

[0024] The query image in an occluded state is input into the trained pedestrian retrieval model for retrieval, and the retrieval results are output to complete the occluded pedestrian retrieval task.

[0025] Furthermore, in step S1, the method of obtaining the image quadruple using the multi-branch data enhancement strategy is as follows:

[0026] I ba =τ ba (I)

[0027] I amp =τ amp (I)

[0028] I shr =τ shr (I)

[0029] I cov =τ cov (I)

[0030] Among them, τ ba , τ amp , τ shr , τ cov They represent baseline enhancement, magnification enhancement, reduction enhancement, and coverage enhancement, respectively. ba Denotes the baseline enhanced image, I amp Represents the magnified image, I shr Represents the image after reduction, I cov represents the covered image, I represents the input pedestrian image, that is, the original image;

[0031] Multi-branch data enhancement methods include baseline enhancement, magnification enhancement, reduction enhancement and coverage enhancement. Among them, baseline enhancement is used to perform basic preprocessing operations on the original image, including resizing and normalization. Magnification enhancement is used to randomly amplify the local area of ​​the original image. Reduction enhancement is used to reduce the original image and synthesize it with a partial area of ​​another image in the test set. Coverage enhancement is used to use a partial area of ​​an image in the training set and randomly cover it on the original image.

[0032] Furthermore, in step S2, the specific processing formula is as follows:

[0033]

[0034] Among them, φ represents the feature extractor, namely the backbone network TransReID, represents the image features after baseline enhancement, Represents the image features after magnification, Represents the image features after reduction, Represents the image features after coverage.

[0035] Furthermore, in step S3, the specific processing process is as follows:

[0036] M l =R(SCHP(I ba ))

[0037] in, R(·) represents resizing, SCHP(·) represents the human semantic parsing model, P represents the number of channels, and M p Represents the true semantic label M of the human body l The feature of each channel is the true semantic label of the p-th body part, p∈{1,2,...,P}.

[0038] Furthermore, in step S4, the specific processing is as follows:

[0039] S41: N patch features f t Reshape into the original patch space position feature Then the original patch spatial position feature Send it to a convolutional layer Ψ(·) to obtain the semantic mask M of each body part;

[0040] S42: Use semantic mask to obtain the visibility score v of each body part i , and spliced ​​together to form a vector to obtain the visible score v of all body parts, that is, the semantic perception score, where i∈{1,2,...,P};

[0041] S43: semantic mask M p and local features f t* Perform element-wise multiplication to obtain the semantic features of each body part

[0042] Furthermore, in step S41, the formula for obtaining the semantic mask M is as follows:

[0043] M=Ψ(f t* )

[0044] Where M=[M1,...,M P] represents the semantic mask, M p represents the semantic mask of the p-th body part, p∈{1,2,...,P}.

[0045] Furthermore, in step S42, the visibility score v of each body part is i The calculation formula is as follows:

[0046]

[0047] in, i represents the i-th body part.

[0048] Furthermore, in step S43, the semantic features of each body part The calculation formula is as follows:

[0049]

[0050] Here, GAP(·) represents the global average pooling operation.

[0051] Furthermore, in step S5, the semantic perception score v is used as a weight to calculate the semantic features of the restored sample. That is, the reconstructed semantic features of the original sample The formula is as follows:

[0052]

[0053] Among them, v represents the visible score, represents the complement of v, is the semantic feature of the most similar positive sample of the original sample.

[0054] Furthermore, in step S6, the multiple loss functions include the classification loss L ID , contrast loss L Trip , segmentation loss L Seg , reconstruction loss L Rec 、KL loss L KL , the loss calculation formulas are as follows:

[0055]

[0056] Among them, L id represents ID loss, L trip represents triplet loss, L1 represents L1 loss, Cls(·) represents classification layer, Represents the global features obtained after different data enhancement methods, i∈{ba,amp,shr,cov}; Represent the semantic features of each body part, p∈{1,…,P}; Represents the reconstructed semantic features of the original sample; M p represents semantic mask; CE represents cross entropy loss; represents the true semantic label obtained from the human semantic parsing model SCHP; KL(·) represents the KL divergence; y represents the true label.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] 1. The present invention proposes a multi-branch enhancement strategy. Through this multi-branch strategy, multiple sets of image pairs can be generated as input models, which greatly enriches the training samples under occlusion scenes and enhances the model's adaptability to various occlusion situations.

[0059] 2. This paper proposes a human semantic perception method that uses the powerful information extraction capability of Transformer to ensure that the model can perceive missing and occluded areas.

[0060] 3. The present invention also proposes a semantic restoration method, which uses the proposed human semantic perception method to locate the missing semantic part, and then uses the semantic information of the corresponding area of ​​similar positive samples in the training set to complete the missing part; therefore, the human semantic restoration ability is learned in the training phase, so that the model can directly predict the missing area in the testing phase without relying on any additional auxiliary data. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 1 is a flow chart of an occluded pedestrian retrieval method based on human semantic perception and restoration in an embodiment of the present invention;

[0062] Figure 2 Schematic diagram of pedestrian retrieval in an occlusion scene according to an embodiment of the present invention;

[0063] Figure 3 is a schematic diagram of reduction and enhancement in an embodiment of the present invention;

[0064] Figure 4 Schematic diagram of coverage enhancement in an embodiment of the present invention, where (a) shows a person blocking another person, and (b) shows an object blocking a person;

[0065] Figure 5 1 is a diagram illustrating the overall method architecture of an occluded pedestrian retrieval method based on human semantic perception and restoration in an embodiment of the present invention;

[0066] Figure 6 2. It is a schematic diagram of the output result of the human body semantic parsing model according to an embodiment of the present invention;

[0067] Figure 7 2 is a schematic diagram of label merging of a human body semantic parsing model in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment.

[0069] Example 1

[0070] This embodiment provides a technical solution: a method for retrieval of occluded pedestrians based on human semantic perception and restoration, aiming to make better use of training data. The specific solution process is as follows: Figure 1 As shown in the figure, a multi-branch enhancement method is used to generate more training samples with occlusions, thereby improving the model's robustness to complex occlusion scenarios. A semantic perception method is then used to extract multi-granularity regional features of the human body from these occluded samples. A semantic restoration mechanism is then used to align and reconstruct the semantic features of the occluded pedestrian with the closest human semantic features, preserving a more comprehensive semantic expression and enhancing retrieval performance.

[0071] The backbone network used in the model of this embodiment is TransReID. After passing through the backbone network, a global feature f g ∈R B×d Followed by a local feature f t ∈R B×N×d , the detailed steps are as follows:

[0072] Step 1: Preprocess the input pedestrian image, that is, use the multi-branch data enhancement strategy to obtain an image quadruple.

[0073] I ba =τ ba (I)

[0074] I amp =τ amp (I)

[0075] I shr =τ shr (I)

[0076] I cov =τ cov (I)

[0077] Among them, τ ba , τ amp , τ shr , τ cov They represent baseline enhancement, magnification enhancement, reduction enhancement, and coverage enhancement, respectively. ba Denotes the baseline enhanced image, I amp Represents the magnified image, I shr Represents the image after reduction, I covRepresents the covered image, and I represents the input pedestrian image, that is, the original image.

[0078] Step 2: Input this set of images into a parameter-sharing backbone network TransReID, which is expressed as follows:

[0079]

[0080] Among them, φ represents the feature extractor, namely the backbone network TransReID, represents the image features after baseline enhancement, Represents the image features after magnification, Represents the image features after reduction, Represents the image features after coverage.

[0081] Step 3: Use the human body semantic parsing model to obtain the real semantic label of the human body. The formula is as follows:

[0082] M l =R(SCHP(I ba ))

[0083]

[0084] Among them, R(·) represents resizing, SCHP(·) represents the human semantic parsing model, P represents the number of channels, l represents the local, and M is used later. l To optimize the human semantic mask, Represents the true semantic label M of the human body l Characteristics of each channel.

[0085] Step 4: In order to obtain the semantic information of P different parts, we need to first transform N patch features f t ∈R B×N×d Reshape into the original patch space position feature f t* ∈R B×h×w×d , that is, converting the patch feature from a one-dimensional sequence representation to a two-dimensional space representation. Then f t* It is sent to a convolutional layer Ψ(·) to obtain the semantic mask of each body part, which is expressed as follows:

[0086] M=Ψ(f t* )

[0087] Where M=[M1,...,M P ] represents the semantic mask, M p ∈R h*w(p∈{1,2,...,P}) represents the semantic mask of the pth part. In the model learning stage, the knowledge distillation method is used to make the output M of the semantic perception method as close as possible to the output of the human semantic parsing model SCHP.

[0088] Step 5: We can use these semantic masks to obtain the semantic perception score v. This process consists of two steps. First, the semantic mask of each body part is used to perform semantic segmentation of pedestrians. The formula is as follows:

[0089]

[0090] We determine the body part to which the corresponding pixel belongs by obtaining the maximum index of all semantic masks. Secondly, for each body part, we determine whether it is visible by judging whether the part is in the semantic segmentation result Q. Through this operation, we can obtain the visibility score v of each body part i , and concatenate them (concat operation) together to form a vector, thereby obtaining the visibility scores v of all body parts.

[0091]

[0092] Step 6: In order to further obtain semantic features, we perform semantic mask M p (p∈{1,2,...,P}) and local features f t* Perform element-by-element multiplication, the formula is as follows:

[0093]

[0094] Among them, GAP(·) represents the global average pooling operation, Represents the semantic features of each part.

[0095] Step 7: After the above operations, we can obtain the semantic features of the sample And by calculating the feature similarity between samples, the semantic features of the most similar positive sample are obtained. Using the visible score v obtained from the human semantic perception method as a weight, the semantic features of the restored sample can be obtained The formula is as follows:

[0096]

[0097] Among them, v represents the visible score, represents the complement of v, and They represent the semantic features of the original sample, the semantic features of the most similar positive sample, and the reconstructed semantic features of the original sample respectively.

[0098] Step 8: Use classification loss L ID , contrast loss L Trip , segmentation loss L Seg , reconstruction loss L Rec 、KL loss L KL To optimize the model.

[0099]

[0100]

[0101] Among them, L id represents ID loss, L trip denotes triplet loss, L1 denotes L1 loss, and Cls(·) denotes classification layer. It is obtained from the multi-branch enhancement method and represents the global features obtained after different data enhancements. It is obtained from the human body semantic perception method and represents the semantic features of each part. It is obtained from the human semantic restoration method and represents the restored semantic features. represents the semantic mask. CE represents the cross entropy loss. represents the true semantic label obtained from the auxiliary human semantic parsing model SCHP. KL(·) represents the KL divergence. y represents the true label.

[0102] Step 9. Repeat steps 1-8 above until the model converges or the termination condition is reached.

[0103] The present invention has the following advantages over the prior art:

[0104] 1. Multi-branch data enhancement strategy

[0105] Existing methods generally use simple data enhancement methods, such as random magnification, horizontal flipping, and random erasing. This preprocessing method is too simple and does not fully consider the domain differences between the training set and the test set. Therefore, the present invention greatly enriches the types of occlusion samples in the training data by introducing a multi-branch data enhancement strategy, including baseline enhancement, magnification enhancement, reduction enhancement, and coverage enhancement operations. During the training stage, the model can be exposed to various complex occlusion situations, such as local occlusion, full-body occlusion, etc., thereby enhancing the model's adaptability to different forms of occlusion. Through diversified training samples, the robustness of the model has been significantly improved, especially in real occlusion environments, the recognition performance has been effectively improved.

[0106] 2. Transformer-based human semantic perception method

[0107] Some existing semantic perception methods are mainly based on CNN networks and fail to fully utilize the powerful Transformer network. In contrast, the present invention combines the context perception capabilities of Transformer and uses the human semantic parsing model SCHP to provide real semantic labels for each image, and uses the knowledge distillation method to effectively enhance the semantic perception capabilities of the Transformer model. The human semantic perception method helps the model better understand the appearance characteristics of pedestrians, especially in occluded parts, the model can effectively extract effective features related to the pedestrian's identity. This mechanism reduces the impact of irrelevant interference information on identity recognition, thereby improving the recognition accuracy in occluded environments. By optimizing the extraction and utilization of semantic information, the present invention significantly improves the performance of the model in retrieval of occluded pedestrians.

[0108] 3. Transformer-based human semantic restoration method

[0109] While some existing image restoration methods are implemented during the testing phase and are overly reliant on the test set, the human semantic restoration method designed in this paper is implemented during the training phase, leveraging the semantic information of the same person across different images to effectively complete the missing portions of occluded areas. As a result, the restored features are more accurate and complete, especially in complex occlusion environments. The model can effectively restore features of occluded areas, thereby improving the accuracy and reliability of pedestrian retrieval.

[0110] 4. Improved overall performance and greater adaptability

[0111] The method of the present invention does not require additional annotated data or complex model adjustments, and has high flexibility and adaptability. Through multi-branch data enhancement and Transformer-based semantic perception recovery, the model can maintain high recognition performance in complex occluded environments. The method can also be seamlessly integrated into the existing ReID framework, reducing computational overhead and improving computational efficiency. Therefore, the present invention not only improves the accuracy of occluded pedestrian retrieval, but also enhances the practicality and scalability of the model.

[0112] Example 2

[0113] This embodiment further illustrates the technical solution in the first embodiment.

[0114] The main difficulty of occluded pedestrian retrieval task is how to extract accurate pedestrian features under occlusion. Traditional pedestrian retrieval methods usually rely on complete visual information, while occluded pedestrian retrieval is Figure 2 As shown in Figure 3, occlusion can cause the loss of key information in the image, making it difficult for the model to accurately identify pedestrians.

[0115] Pedestrian occlusion problems include full-body occlusion and partial occlusion. Full-body occlusion is often caused by objects, groups of pedestrians, or vehicles, while partial occlusion usually occurs when clothing, backpacks, or other body parts partially block the view.

[0116] Most of the current mainstream methods rely on simple data augmentation methods. Although they can alleviate the occlusion problem to a certain extent, they still perform poorly when dealing with complex occlusion situations. To this end, we simulate three different occlusion methods: enlargement enhancement, reduction enhancement (such as Figure 3 as shown) and coverage enhancement (as Figure 4 It aims to enrich the diversity of training data and thus improve the robustness of the model.

[0117] The method structure diagram of the present invention is as follows Figure 5 As shown in Figure 3, our method consists of three main components: a multi-branch enhancement strategy, human semantic perception, and human semantic restoration. The multi-branch enhancement strategy provides our method with richer training data. Human semantic perception enables the model to mine fine-grained human semantic information, while human semantic restoration further refines and refines the extracted human semantic information, thereby improving the accuracy and completeness of the overall representation.

[0118] It should be noted that the Transformer is a deep learning model based on the self-attention mechanism, which demonstrates superior global information modeling capabilities when processing sequential data. In pedestrian retrieval, the Transformer model can effectively capture long-range dependencies and global contextual information in images, which is crucial for recovering occluded areas in pedestrian images. This paper leverages the Transformer's global modeling capabilities and introduces human semantic perception and semantic restoration methods to effectively solve the problem of feature extraction of occluded pedestrians.

[0119] like Figure 6 As shown in the output diagram of the human semantic parsing model, through deep learning and computer vision technology, the human semantic parsing model SCHP can analyze the input image, identify and segment the human body objects in the image, and obtain 20 classification labels about the human body information, thereby extracting high-level semantic information related to the human body. However, in order to adapt to our task, we roughly merge the labels into 6 types, such as Figure 7 This is shown in the label merging diagram. This merging method yields merged labeled images of human bodies. These labeled images are then resized to obtain the true semantic labels for the human body required by our model. This information helps our model understand the specific features of the human body in the image, such as distinguishing areas like the head, upper limbs, and lower limbs, and even recognizes human movements, postures, and relationships with other objects.

[0120] The technical solution of this invention is widely applicable to urban surveillance, public security, traffic monitoring, and other fields. It can effectively improve the accuracy and robustness of pedestrian retrieval, especially in complex obstructed environments (such as crowded areas and during rush hour). With the advancement of urbanization and the popularization of intelligent monitoring systems, the technical solution of this invention will greatly enhance the reliability and efficiency of intelligent security systems and has broad commercial application prospects.

[0121] In summary, the above-mentioned embodiment proposes a multi-branch enhancement strategy for occluded pedestrian retrieval based on human semantic perception and restoration. It includes a baseline enhancement and three different occlusion enhancements. The baseline enhancement only includes basic preprocessing operations (such as image resizing and normalization), while the three occlusion enhancements are magnification, reduction, and coverage. The magnification enhancement randomly enlarges local areas of the original image; the reduction enhancement reduces the original image and synthesizes it with a partial area of ​​another image in the test set; and the coverage enhancement uses a partial area of ​​an image in the training set and randomly overlays it on the original image to simulate diverse occlusion patterns. This multi-branch strategy ultimately generates multiple sets of image pairs for the input model, greatly enriching the training samples in occlusion scenarios and enhancing the model's adaptability to various occlusion situations. The present invention proposes a human semantic perception method that leverages the powerful information extraction capabilities of the Transformer and combines it with knowledge distillation to ensure that the model can perceive missing and occluded areas. The present invention also proposes a semantic restoration method that uses the proposed human semantic perception method to locate missing semantic parts and then uses semantic information from the corresponding areas of similar positive samples in the training set to complete the missing parts. Therefore, the ability to recover human semantics is learned during the training phase, enabling the model to directly predict missing regions during the testing phase without relying on any additional auxiliary data.

[0122] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for retrieval of occluded pedestrians based on human semantic perception and restoration, characterized by: The following steps are involved: S1: Image preprocessing The input pedestrian image is preprocessed using a multi-branch data enhancement strategy to obtain an image quadruple; S2: Feature Extraction The image quadruple is input into a parameter-sharing backbone network TransReID for feature extraction; S3: Acquisition of true semantic labels Use the human body semantic parsing model to obtain the real semantic labels of the human body; S4: Human Semantic Perception Obtain a semantic mask for each body part, use the semantic mask to obtain a semantic perception score, and extract the semantic features of each body part; S5: Human Semantic Restoration Based on the semantic perception score, the reconstructed semantic features are obtained to achieve feature reconstruction; S6: Model Training The model is trained using multiple loss functions to obtain a trained pedestrian retrieval model. S7: Pedestrian retrieval The query image in an occluded state is input into the trained pedestrian retrieval model for retrieval, and the retrieval results are output to complete the occluded pedestrian retrieval task.

2. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 1, characterized in that: In step S1, the method of obtaining the image quadruple using the multi-branch data enhancement strategy is as follows: I ba =τ ba (I) I amp =τ amp (I) I shr =τ shr (I) I cov =τ cov (I) Among them, τ ba , τ amp , τ shr , τ cov They represent baseline enhancement, magnification enhancement, reduction enhancement, and coverage enhancement, respectively. ba Denotes the baseline enhanced image, I amp Represents the magnified image, I shr Represents the image after reduction, I cov represents the covered image, I represents the input pedestrian image, that is, the original image; Multi-branch data enhancement methods include baseline enhancement, magnification enhancement, reduction enhancement and coverage enhancement. Among them, baseline enhancement is used to perform basic preprocessing operations on the original image, including resizing and normalization. Magnification enhancement is used to randomly amplify the local area of ​​the original image. Reduction enhancement is used to reduce the original image and synthesize it with a partial area of ​​another image in the test set. Coverage enhancement is used to use a partial area of ​​an image in the training set and randomly cover it on the original image.

3. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 2 is characterized in that: In step S2, the specific processing formula is as follows: Among them, φ represents the feature extractor, namely the backbone network TransReID, represents the image features after baseline enhancement, Represents the features of the enlarged image, Represents the image features after reduction, Represents the image features after coverage.

4. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 3 is characterized in that: In step S3, the specific processing process is as follows: M l =R(SHCP(I ba )) in, R(·) represents resizing, SCHP(·) represents the human semantic parsing model, P represents the number of channels, and M p Represents the true semantic label M of the human body l The feature of each channel is the true semantic label of the p-th body part, p∈{1,2,...,P}.

5. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 4 is characterized in that: In step S4, the specific processing is as follows: S41: N patch features f t Reshape into the original patch space position feature f t* , and then the original patch spatial position feature f t* Send it to a convolutional layer Ψ(·) to obtain the semantic mask M of each body part; S42: Use semantic mask to obtain the visibility score v of each body part i , and spliced ​​together to form a vector to obtain the visible score v of all body parts, that is, the semantic perception score, where i∈{1,2,...,P}; S43: semantic mask M p and local features f t* Perform element-wise multiplication to obtain the semantic features of each body part 6. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 5, characterized in that: In step S41, the formula for obtaining the semantic mask M is as follows: M=ψ(f t* ) Where M=[M1,...,M P ] represents the semantic mask, M p represents the semantic mask of the p-th body part, p∈{1,2,...,P}.

7. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 6, characterized in that: In step S42, the visibility score v of each body part is i The calculation formula is as follows: in, i represents the i-th body part.

8. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 7, characterized in that: In step S43, the semantic features of each body part The calculation formula is as follows: Here, GAP(·) represents the global average pooling operation.

9. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 8, characterized in that: In step S5, the semantic perception score v is used as a weight to calculate the semantic features of the restored sample. That is, the reconstructed semantic features of the original sample The formula is as follows: Among them, v represents the visible score, represents the complement of v, is the semantic feature of the most similar positive sample of the original sample.

10. The method for retrieval of occluded pedestrians based on human semantic perception and restoration according to claim 1, characterized in that: In step S6, multiple loss functions include classification loss L ID , contrast loss L Trip , segmentation loss L Seg , reconstruction loss L Rec 、KL loss L KL , the loss calculation formulas are as follows: Among them, L id represents ID loss, L trip represents triplet loss, L1 represents L1 loss, Cls(·) represents classification layer, and f i g Represents the global features obtained after different data enhancement methods, i∈{ba,amp,shr,cov}; Represent the semantic features of each body part, p∈{1,…,P}; Represents the reconstructed semantic features of the original sample; M p represents semantic mask; CE represents cross entropy loss; represents the true semantic label obtained from the human semantic parsing model SCHP; KL(·) represents the KL divergence; y represents the true label.

Images