Pedestrian image search method based on semantic segmentation and visual text attribute alignment

By adopting the visual text attribute alignment method of semantic division in pedestrian image search, using joint constraint training of global and local features, the problem of insufficient local similarity and fine-grained division in the existing technology is solved, and the accuracy of pedestrian image search is improved Rate.

CN114036336BActive Publication Date: 2024-11-08SHANGHAI JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111344497.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-15
Publication Date
2024-11-08
Estimated Expiration
2041-11-15

AI Technical Summary

Technical Problem

The prior art fails to fully utilize the local similarity and interaction between images and text in pedestrian image search, and lacks fine-grained text semantic division, resulting in poor cross-modal feature alignment.

Method used

Using the visual text attribute alignment method based on semantic division, global and local features are extracted through image and text feature extraction networks, and embedded features are transformed in the embedding network, combined with global alignment, local alignment and segmentation loss functions for joint constraint training. Achieve finer-grained local feature alignment.

Benefits of technology

The performance of pedestrian image search is improved, and the global branches are assisted in extracting more aligned cross-modal embedded features through local feature alignment, which improves the accuracy of pedestrian image search under natural language description.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114036336B_ABST
    Figure CN114036336B_ABST
Patent Text Reader

Abstract

The present invention provides a pedestrian image search method based on semantic division and visual text attribute alignment, including: processing the original data in the image modality and the text modality to obtain the global image and global and local text data sets; using the image feature extraction network and the text feature extraction network to extract features of the data sets respectively, and obtain the global and local features in the image and text single modality; converting the global and local features in the single modality into the embedding features of the corresponding modality in the embedding network; and training the model under the joint constraints of multiple loss functions. The present invention divides the local features into finer granularity, makes full use of the corresponding relationship between the local features, and assists the backbone network to extract more aligned global cross-modal embedding features; through the training of the overall loss joint constraint network, the model is prompted to converge in the optimal direction, and the performance of pedestrian image search based on natural language description is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cross-modal alignment, and in particular to a pedestrian image search method based on semantic segmentation and visual text attribute alignment. Background Art

[0002] Human identification technology plays an extremely important role in the field of intelligent video surveillance. Pedestrian cross-image and text modality retrieval aims to match the most suitable pedestrian image in the image dataset based on the given natural language description of the target pedestrian. Compared with the pedestrian image search method based on images or attributes, this person image search method based on natural language description can handle complex situations where there is a lack of known images of the person to be searched, and has the advantages of being more flexible, convenient and friendly in application, but at the cost of being more complex and challenging in model processing.

[0003] First, for the same image, its text description forms are rich, so the text encoding network needs to be able to handle diverse data. Third, the sample data comes from two modalities, image and text, and there is a heterogeneous gap in data composition and a semantic gap in features. Only by overcoming the differences between modalities can the similarity between feature vectors from the two modalities be effectively measured and ranked.

[0004] At present, there are two main approaches: joint embedding learning and similarity learning. The former refers to learning a mapping method to project the features of images and texts into a common subspace, and directly matching the embedded features of images and texts in this common subspace; the latter refers to designing a similarity measurement network.

[0005] Joint embedding learning is the current mainstream method. Its process is to first extract separate features from the two modalities, then map the features to the common feature subspace (high-level network) shared by the visual text, and maximize the correlation of different modal representations of the same sample in the common subspace. There are currently three types of pedestrian image search methods based on natural language descriptions:

[0006] The first is to embed the image and text into a shared subspace based on global features (see Zhang Y, Lu H. Deep cross-modal projection learning for image-text matching [C] / / Proceedings of the European Conference on Computer Vision (ECCV). 2018: 686-701.; Zheng Z, Zheng L, Garrett M, et al. Dual-path convolutional image-text embeddings with instance loss [J]. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2020, 16 (2): 1-23.), extract their overall features, and use the similarity between global features to constrain the network training. However, this method does not take into account the local similarity and interaction between images and texts.

[0007] The second is based on the attention mechanism (see Li S, Xiao T, Li H, et al. Person search with natural language description [C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1970-1979.; Li S, Xiao T, Li H, et al. Identity-aware textual-visual matching with latent co-attention [C] / / Proceedings of the IEEE International Conference on Computer Vision. 2017: 1890-1899.; Chen D, Li H, Liu X, et al. Improving deep visual representation for person re-identification by global and local image-language association [C] / / Proceedings of the European conference on computer vision (ECCV). 2018: 54-70.). This method focuses on the local similarities between images and texts. Generally, the image is divided into grids and the text is divided into single words or phrases. Then, an attention mechanism is added to the network to obtain the similarity between parts, parts and the whole, and the whole and the whole. However, the defect of this method is that it only divides the image and text into grids, and does not align the meaningful semantic parts of the two modalities.The third is to fully explore the correlation between visual and textual semantic components (see Niu K, Huang Y, Ouyang W, et al. Improving description-based person re-identification by multi-granularity image-text alignments [J]. IEEE Transactions on Image Processing, 2020, 29: 5542-5556.; Jing Y, Si C, Wang J, et al. Pose-guided joint global and attentive local matching network for text-based person search [J]. Association for the Advance of Artificial Intelligence (AAAI), 2020.; Wang, Zhe, et al. "Vitaa: Visual-textual attributes alignment in person search by natural language." European Conference on Computer Vision. Springer, Cham, 2020.), and clearly locate and align the meaningful semantic parts in the image and text description. By segmenting the image based on human body parts and then dividing the text accordingly, the alignment between the local features of the image and the text is promoted. However, this method is only based on images in the division of local features and lacks a more fine-grained division of text semantics. Summary of the invention

[0008] In view of the defects in the prior art, the object of the present invention is to provide a pedestrian image search method based on semantic segmentation and visual text attribute alignment.

[0009] According to one aspect of the present invention, a pedestrian image search method based on semantic segmentation and visual text attribute alignment is provided, comprising:

[0010] Process the original data of the image modality and the text modality to obtain the global image and global and local text data sets in the corresponding modality;

[0011] Using an image feature extraction network and a text feature extraction network to extract features from the data set, respectively, to obtain global and local features within the corresponding single modality;

[0012] Using an embedding network to transform the global and local features in the single modality into embedded features of the corresponding modality;

[0013] Train the model under the joint constraint of the overall loss of the model;

[0014] Use the trained model to perform pedestrian image search.

[0015] Preferably, the processing of the original data in the image modality and the text modality to obtain the global image and global and local text data sets includes:

[0016] Obtaining a global image representation of the human body I global , the overall text indicates T global ;

[0017] The raw data in the image and text modalities are divided based on body parts and word parts of speech, including:

[0018] The existing human body segmentation network is used to generate image segmentation masks based on the five parts of the human body. The image with normalized size is input into the human body segmentation network to generate image segmentation masks based on the five parts of the human body. local-label ;

[0019] Use the word-body part correspondence table, combine the correspondence table with the existing natural language processing tool library NLTK to obtain the text representation of each body part and

[0020] Preferably, the human body parts include: head, upper body, lower body, shoes and backpack;

[0021] The word parts of speech include: noun and adjective;

[0022] The word-body part correspondence table artificially designed for the present invention;

[0023] Preferably, the image feature extraction network is F visual , which are the first three blocks of Resnet50, and the output dimension is set to 1024;

[0024] The text feature extraction network is F textual , which is a Bi-LSTM, and the output dimension is set to 512 dimensions.

[0025] Preferably, the image feature extraction network F visual And the text feature extraction network F textualThe global image data, global and local text data are respectively input into the feature extraction network of the corresponding modality to extract the corresponding single modality features. global After the image feature extraction network F visual Then form the global image feature v global The overall text T global , Local description of text based on noun semantic division and local description of text based on adjective semantic division Independently through F textual The global feature t of the corresponding text can be obtained global , Local features of text based on noun semantic division Local features of text based on adjective semantic segmentation

[0026] Preferably, converting the global and local features in the single modality into embedding features of the corresponding modality in the embedding network includes:

[0027] Construct feature embedding network, including: image global feature embedding network E visual-global , Image local feature embedding network corresponding to noun semantics Image local feature embedding network corresponding to adjective semantics Text global feature embedding network E textual-global , Text local feature embedding network corresponding to noun semantics Image local feature embedding network corresponding to adjective semantics

[0028] Get global and local embedding features, including:

[0029] Global image features v global After the local features of the image are embedded in the network E visual-global Get the global embedding feature v global-embed ;

[0030] Global image features v global Image local feature embedding network based on noun semantic correspondence Get the local embedding feature v of the image noun-embed ;

[0031] Global image features v global Image local feature embedding network based on adjective semantic correspondence Get the local embedding feature v of the image adj-embed ;

[0032] Global features of text globalAfter the text global feature embedding network E textual-global Get the global embedding feature t of the text global-embed ;

[0033] Local features of text based on noun semantic segmentation The local feature embedding network of the text corresponding to the noun semantics Get the local embedding feature b of the text noun-embed ;

[0034] Local features of text based on adjective semantic segmentation Image local feature embedding network based on adjective semantic correspondence Get the local embedding feature t of the text adj-embed ;

[0035] The local embedding feature v of the image noun-embed and v adj-embed After deconvolution operation, the category prediction x of the local features of the image is obtained noun and x adj .

[0036] Preferably, for each modality, each sub-embedding network is independent of each other and does not share parameters; the first link of all image feature embedding networks is the fourth group of large blocks (block4) of Resnet50 with different parameters. In this way, the global and local features of the image are obtained by processing the original image through the complete Resnet50. The difference is that the fourth block is independent of each other and generates corresponding features under the constraints of the corresponding function; the first three blocks are shared by global features and local features. This part of the shared network enables the alignment of local features to constrain the extraction of overall features during the back propagation process.

[0037] Preferably, the overall loss includes: global alignment loss, local alignment loss and segmentation loss;

[0038] Preferably, the function of the overall loss is:

[0039] Among them, L global-align is the global alignment loss, which is used to constrain the similarity of global embedding features between modalities; and is the local alignment loss, which constrains the similarity of the local embedding features corresponding to the semantics of nouns and adjectives between modalities; and is the segmentation loss, which is used to ensure that the extracted local embedded features of the image correspond to the five body parts of the human body; λ1 and λ2 represent the weights of the corresponding loss components.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention divides the local features at a finer granularity, fully utilizes the correspondence between local features, and assists the global branch (composed of a global feature extraction network and a global feature embedding network) to extract more aligned global cross-modal embedding features; through the training of multiple loss functions and joint constraint networks, the model is prompted to converge in the optimal direction, thereby improving the performance of pedestrian image search based on natural language description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0043] Figure 1 A flowchart of a pedestrian image search method based on semantic segmentation and visual text attribute alignment according to an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the principle of a pedestrian image search method based on semantic division and visual text attribute alignment according to an embodiment of the present invention;

[0045] Figure 3 The figure is a performance comparison diagram of an application embodiment of the present invention with different algorithms on the data set CUHK-PEDES. DETAILED DESCRIPTION

[0046] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0047] like Figure 1 FIG. 1 is a flowchart of a pedestrian image search method based on semantic segmentation and visual text attribute alignment according to an embodiment of the present invention, comprising:

[0048] S1, processing the original data of the image modality and the text modality to obtain the global image and global and local text data sets in the corresponding modality;

[0049] S2, build image feature extraction network and text feature extraction network;

[0050] S3, extract the data set using the image feature extraction network and text features to obtain global and local features within the image and text single modality;

[0051] S4, constructing the embedding network;

[0052] S5, in the embedding network, the global and local features within a single modality are converted into embedded features of the corresponding modality;

[0053] S6, model training is performed under the joint constraint of the overall loss.

[0054] In order to better perform data preprocessing, an embodiment is provided to perform S1. In the data preprocessing of this embodiment, a global image representation of a human body is first obtained. global , the overall text indicates T global Then, data is divided based on human body parts and word parts of speech. Specifically, the existing human body segmentation network is used to obtain image segmentation masks based on human body parts. local-label ; Combine the word-part correspondence table Table-voc designed for the present invention and the text segmentation model to obtain the text representation of the local part and

[0055] As a preferred embodiment, S1 is performed, including:

[0056] S101, normalize the size of the whole image to obtain a whole image representation I with a size of 384*128 global ;

[0057] S102, the human body is divided into five parts, namely: head, upper body, lower body, shoes and backpack. The image segmentation masks of the five parts of the human body are generated using the existing human body segmentation model:

[0058] S103, manually collect the noun table Table-voc (see Table 1) corresponding to the five body parts of the human body;

[0059] S104, using the NLTK library to obtain the part of speech of each word in the original sentence and its coordinates in the sentence, and then looking up each noun in Table-voc to determine its corresponding body part;

[0060] Specifically, in this embodiment, it is assumed that the adjectives between the i-1th noun and the i-th noun are used to modify the i-th noun, so the body part corresponding to each adjective can be determined. Thus, the adjectives and nouns modifying each body part in a text description can be determined. In this way, local text information based on semantics and human body parts is obtained;

[0061] S105, one-hot encoding is performed on the overall and local texts to form global and local text descriptions. The length of the global text description is unified to 100, and the length of the local text description is 15. The overall text description is recorded as T. The local divisions corresponding to each text description generated in the end have two groups. One group is the local representation corresponding to the central word noun of the phrase representing the type of thing: The other group is the local expressions corresponding to adjectives that describe the nature and state attributes of things.

[0062] S106, converting global and local text into vector representations through word embedding.

[0063] Table 1 Table-voc

[0064]

[0065]

[0066] Based on the above S1, S2 is executed to construct an image feature extraction network and a text feature extraction network. In order to better extract image features and text features, a preferred embodiment is provided. In this embodiment, the selected image feature extraction network F visual The first three blocks of Resnet50, the output dimension is set to 1024; the text feature extraction network F textual It is Bi-LSTM, and the output dimension is set to 512. Resnet can solve the problem of vanishing gradients in deep networks and is the mainstream network in the current image processing field. Bi-LSTM is a bidirectional long short-term memory model that can capture and model contextual information.

[0067] Based on the above S2, execute S3, use the constructed image feature extraction network and text feature extraction network to generate a single-modal feature vector, and obtain the global feature v of the image global , the global features of the text global , Local features of text based on noun semantic division Local features of text based on adjective semantic segmentation All feature extraction networks are shared globally and locally. This part only obtains global image features, global text features, and local text features, that is, only global features are extracted in the image modality, and global and local embedded features are subsequently generated from the global features.

[0068] Specifically, as a preferred embodiment, executing S3 includes:

[0069] S301, the global image I obtained by processing in S1 global Input to the image feature extraction network F visualThe global feature v of the generated image global :

[0070] v global =F visual (I global )

[0071] S302, the text data T obtained in S1 is processed global , and Input to the text feature extraction network F textual Generate global features of text t global , local features of text corresponding to noun semantics Local features of text corresponding to adjective semantics Specifically, the output is:

[0072] t global =F textual (T global )

[0073]

[0074]

[0075] Based on the above S3, execute S4 to build a feature embedding network. In multimodal tasks, model performance depends largely on the quality of embedded features. Because this process is a crucial part of the entire network, it aims to generate richer embedded representations based on the previously extracted unimodal feature vectors.

[0076] The feature embedding network includes: image global feature embedding network E visual-global , Image local feature embedding network corresponding to noun semantics Image local feature embedding network corresponding to adjective semantics Text global feature embedding network E textual-global , Text local feature embedding network corresponding to noun semantics Text local feature embedding network corresponding to adjective semantics The six sub-networks of the feature embedding network are independent and unconnected.

[0077] As a preferred embodiment, the global and local embedding networks are not shared, and global features can only be input into the global feature embedding network of the corresponding modality, and local features can only be input into the local feature embedding network of the corresponding modality. The first link of all image feature embedding networks is the fourth group of large blocks (block4) of Resnet50 with different parameters. In this way, the global and local features of the image are obtained by processing the original image with a complete Resnet50. The difference is that the fourth block is independent of each other and generates corresponding features under the constraints of the corresponding function; the first three blocks are shared by global features and local features. This part of the shared network enables the alignment of local features to constrain the extraction of overall features during the back propagation process.

[0078] Based on the above S4, S5 is executed to generate embedded features. Specifically, as a preferred embodiment, S5 is executed, including:

[0079] S501, the global features of the image in the individual modality obtained in S3 are respectively input into the feature embedding network E in S4 visual-global , and In the above example, we get the global embedding feature v of the image. global-embed , local embedding features v of the image noun-embed and v adj-embed The input and output relationship is:

[0080] v global-embed =E visual-global (v global )

[0081] =Linear visual-global (Avgpool(block 4-global (v global )))

[0082]

[0083]

[0084] v noun-embed and v adj-embed After deconvolution operation, the category prediction x of the local features of the image is obtained noun and x adj . The following will be through x noun With x adj The segmentation loss is calculated between the segmentation mask obtained by the previous human segmentation network and

[0085] S502, input the global and local features of the text in the single modality obtained in S3 into the feature embedding network E in S4 textual-global , and In the above example, we get the global embedding feature t of the text. global-embed , local embedding features of text t noun-embed and t adj-embed The input-output relationship is:

[0086] t=E textual-global (t global )=Linear textual-global (t global )

[0087]

[0088]

[0089] In the input and output formula of the above embedding network, block4 represents the fourth block of Resnet50, partname represents the corresponding human body part (head\upperbody\lowerbody\shoes\backpack), Avgpool represents the average pooling operation, Maxpool represents the maximum pooling operation, and Linear represents the fully connected layer operation. The dimensions of the global and local embedding features are both 256.

[0090] This embodiment makes more effective use of text information in the division of local features. Based on the existing technology of dividing local features according to body parts, the local information is divided into two groups corresponding to noun and adjective semantics according to the different semantics represented by words of different parts of speech, thereby generating more fine-grained local features based on different body parts and different semantics. Since the basic network of global features and local features is shared, the alignment of local features can assist the global branch in extracting more aligned cross-modal global embedding features through backpropagation optimization iterations.

[0091] Based on the above S5, execute S6 and use the sum of losses L for training.

[0092] Specifically, as a preferred embodiment, executing S6 includes:

[0093] The overall loss function is:

[0094] Among them, L global-align Used to constrain the similarity of global embedding features between modalities, and Constrain the similarity of the local embedding features corresponding to the semantics of nouns and adjectives between modalities, respectively. and It is used to ensure that the extracted local embedded features of the image correspond to the five body parts of the human body. λ1 and λ2 represent the weights of the corresponding loss components. In this embodiment, λ1 is 0.80 and λ2 is 0.65.

[0095] The expressions of each alignment loss component are as follows:

[0096]

[0097]

[0098]

[0099]

[0100]

[0101] Wherein, N represents the batch_size, which is 64 in this embodiment; τ p and τ n They are the temperature parameters for adjusting the gradient slope of positive and negative samples. In this embodiment, all τ p Take 10 for all τ n Take 40; S i Represents the dot product of the embedded features from the two modalities, the superscript "+" represents the positive sample pair, and the superscript "-" represents the negative sample pair:

[0102]

[0103]

[0104]

[0105] Through the joint constraints of these five loss functions, the constraints on the model are strengthened, prompting the network to converge in the optimal direction.

[0106] The present invention provides a specific application embodiment for searching pedestrian images. Figure 2 As shown, it is a schematic diagram of the principle of a pedestrian image search method based on semantic segmentation and visual text attribute alignment in this embodiment. The CUHK-PEDES dataset used contains 40206 images (34054 in the training set, 3078 in the validation set, and 3074 in the test set) and 80440 natural language text descriptions (68126 in the training set, 6158 in the validation set, and 6156 in the test set). On average, each image corresponds to about two natural language text descriptions.

[0107] Experiments have shown that the method in this embodiment can extract more aligned global features through the division of fine-grained local features, thereby improving the accuracy of cross-modal retrieval. Figure 3 The performance of this embodiment is compared with different algorithms on CUHK-PEDES. Figure 3 Taking the cumulative matching characteristic (CMC) curve as a representation, the nine curves from top to bottom represent the performance of the method of this embodiment and other methods.

[0108] Table 2

[0109]

[0110] It can be seen that the results obtained by this embodiment have greatly improved the performance of cross-modal pedestrian retrieval.

[0111] In summary, the embodiment method makes fuller use of the semantic information in the text and generates more fine-grained local features based on the existing division of local features according to human body parts. Make full use of the correspondence between local features to assist the global branch (composed of a global feature extraction network and a global feature embedding network) to extract more aligned global cross-modal embedding features. Through the alignment constraints of local features, the auxiliary network learns more aligned overall features; through the joint constraints of global alignment loss, local alignment loss and segmentation loss, the model converges to the optimal direction, improving the retrieval performance of searching for pedestrian images based on natural language descriptions.

[0112] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A pedestrian image search method based on semantic segmentation and visual text attribute alignment, characterized in that: include: Process the raw data in the image modality and text modality to obtain global image and global and local text data sets; Using an image feature extraction network and a text feature extraction network to extract features from the data set, respectively, to obtain global and local features within the corresponding single modality; Using an embedding network to transform the global and local features in the single modality into embedded features of the corresponding modality; Train the model under the joint constraint of the overall loss of the model; Use the trained model to search for pedestrian images; The processing of the original data in the image modality and the text modality to obtain the global image and global and local text data sets includes: Obtaining a global image representation of the human body I global , the overall text indicates T global ; The raw data in the image and text modalities are divided based on body parts and word parts of speech, including: Use the existing human body segmentation network to generate image segmentation masks based on human body parts I local-label ; Use the word-body part correspondence table, combine the correspondence table with the existing natural language processing tool library NLTK to obtain the text representation of each body part and The feature embedding network includes six sub-networks: Image global feature embedding network E visual-global , Image local feature embedding network corresponding to noun semantics Image local feature embedding network corresponding to adjective semantics Text global feature embedding network E textual-global , Text local feature embedding network corresponding to noun semantics Image local feature embedding network corresponding to adjective semantics The global and local multimodal embedding features include: Global embedding feature v global-embed , local embedding features v of the image noun-embed and v adj-embed , the global embedding feature of the text t global-embe , local embedding features of text t noun-embed and t adj-embed ; The method of using an embedding network to convert the global and local features in the single modality into embedded features of the corresponding modality includes: Global image features v global After the local features of the image are embedded in the network E visual-globa Get the global embedding feature v global-embe ; Global image features v global Image local feature embedding network based on noun semantic correspondence Get the local embedding feature v of the image noun-embe ; Global image features v global Image local feature embedding network based on adjective semantic correspondence Get the local embedding feature v of the image adj-embed ; Global features of text global After the text global feature embedding network E textual-glob Get the global embedding feature t of the text global-emb ; Local features of text based on noun semantic segmentation The local feature embedding network of the text corresponding to the noun semantics Get the local embedding feature t of the text noun-embed ; Local features of text based on adjective semantic segmentation Image local feature embedding network based on adjective semantic correspondence Get the local embedding feature t of the text adj-embed ; The local embedding feature v of the image noun-embed and v adj-embed After deconvolution operation, the category prediction x of the local features of the image is obtained noun and x adj ; The function of the overall loss is: Among them, L global-ali is the global alignment loss, which is used to constrain the similarity of global embedding features between modalities; and is the local alignment loss, which constrains the similarity of the local embedding features corresponding to the semantics of nouns and adjectives between modalities; and is the segmentation loss, which is used to ensure that the extracted local embedded features of the image correspond to the five body parts of the human body; λ1 and λ2 represent the weights of the corresponding loss components.

2. The pedestrian image search method based on semantic segmentation and visual text attribute alignment according to claim 1 is characterized in that: The human body parts include: head, upper body, lower body, shoes and backpack; The word parts of speech include: noun and adjective.

3. The pedestrian image search method based on semantic segmentation and visual text attribute alignment according to claim 1 is characterized in that: The image feature extraction network is F visual , which are the first three blocks of Resnet50, and the output dimension is set to 1024; The text feature extraction network is F textual , which is a Bi-LSTM, and the output dimension is set to 512; The image feature extraction network F visual And the text feature extraction network F textual It is shared by the global feature and the local feature.

4. The pedestrian image search method based on semantic segmentation and visual text attribute alignment according to claim 3 is characterized in that: The method of extracting the data set using an image feature extraction network and a text feature extraction network to obtain global and local features in a corresponding single modality includes: The global image data, global and local text data are respectively input into the feature extraction network of the corresponding modality to extract the corresponding single modality features, including: The global image I global After the image feature extraction network F visual Then form the global image feature v global ; The overall text T global , Local description of text based on noun semantic division and local description of text based on adjective semantic division Independently through F textual The global feature t of the corresponding text can be obtained global , Local features of text based on noun semantic division Local features of text based on adjective semantic segmentation 5. The pedestrian image search method based on semantic segmentation and visual text attribute alignment according to claim 1 is characterized in that: For each modality, the six sub-networks of the feature embedding network are independent of each other and do not share parameters; Among them, the first link of all image feature embedding networks is the fourth group of large blocks of Resnet50 with different parameters.

6. The pedestrian image search method based on semantic segmentation and visual text attribute alignment according to claim 5 is characterized in that: The overall loss includes: global alignment loss, local alignment loss and segmentation loss.