Multi-granularity text pedestrian search method based on attribute learning and feature screening

Abstract

The invention provides a multi-granularity text pedestrian search method based on attribute learning and feature screening, and the method comprises the steps: carrying out the coarse-granularity feature learning of an image-text pair, filtering noise data through a probability model, and optimizing the cross-modal alignment through a self-supervision cost function, so as to obtain a global feature; tokens of the image and the text are screened based on the self-attention weight output by the encoder, and medium-granularity feature representation is generated; performing random masking on pedestrian attribute vocabularies conforming to adjective-noun combination in the text, fusing features and predicting masked vocabularies through a multi-modal interaction encoder, and realizing fine-grained attribute alignment; and combining the coarse-grained loss, the medium-grained loss and the fine-grained loss, iteratively training the model, storing an optimal model according to verification accuracy, and outputting a pedestrian search result.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and natural language processing technology, specifically relating to a multi-granularity text pedestrian search method based on attribute learning and feature selection. Background Technology

[0002] With the deepening of smart city construction globally, video surveillance systems have become a core infrastructure of modern security systems. Especially in the field of public safety, massive camera networks play a crucial role in tasks such as tracking suspects and finding missing persons. However, an increasingly acute contradiction has arisen between the sheer volume of video data and human processing capabilities—facing hundreds of millions of real-time video streams, relying on human labor to locate target pedestrians from complex and varied images is not only inefficient but also fails to meet the timeliness requirements for emergency response.

[0003] In recent years, pedestrian search technology based on computer vision has made significant progress, especially in the field of pedestrian re-identification. This type of technology analyzes pedestrian appearance features, such as clothing, posture, and carried items, to achieve target tracking across multiple cameras. However, this method has a fundamental limitation: its operational logic is highly dependent on image input as the query condition. When only text descriptions are available in real-world scenarios (e.g., a family member reporting a missing person's condition as "red shirt, dark trousers, and a black backpack"), existing systems cannot respond effectively. This deficiency is particularly prominent in security practices, where numerous critical tasks are hindered by the lack of target images.

[0004] To overcome this limitation, text-based pedestrian search technology has emerged. This technology aims to associate natural language descriptions with visual content, achieving cross-modal retrieval of images using text. Early attempts mainly used joint embedding models to map images and text to a common feature space for similarity matching. Although such methods are effective in simple scenarios, they are always hampered by the modal gap between images and text: images, as continuous signals, contain a large amount of low-level details, while text, as discrete symbols, carries abstract semantics. The two have inherent differences in representation, information density, and granularity. More specifically, text descriptions usually focus on combinations of local attributes (such as "wearing glasses" or "holding an umbrella"), while traditional visual models tend to extract global features, leading to the failure of fine-grained alignment. In addition, unavoidable noise interference in practical applications—such as partial image-text pair description deviations and labeling errors—further weakens the robustness of the model.

[0005] Existing solutions remain inadequate in addressing these challenges. On one hand, global feature matching struggles to capture attribute-level relationships, making the model insufficiently sensitive to specific descriptions such as "blue jeans." On the other hand, annotation noise in training data easily leads to model overfitting, reducing its adaptability to the complexity of real-world scenes. More critically, existing methods lack hierarchical information processing mechanisms, failing to leverage the complementary nature of images and text at different granularities—such as macroscopic semantics, local regions, and attribute details—resulting in retrieval accuracy that falls short of practical requirements.

[0006] These technological bottlenecks profoundly reveal the industry's core demand: there is an urgent need for a cross-modal search framework that can penetrate modal barriers, accurately analyze fine-grained semantics, and has noise resistance capabilities, in order to bridge the cognitive gap between text descriptions and visual content. Summary of the Invention

[0007] To address the shortcomings and deficiencies of existing technologies, this invention provides a multi-granularity text pedestrian search method based on attribute learning and feature selection. Through a hierarchical, three-level processing framework, it significantly improves cross-modal alignment accuracy and noise robustness. Its core innovative design lies in:

[0008] 1. Coarse-grained noise filtering and cross-modal optimization

[0009] A probabilistic model is employed to automatically distinguish noisy data from valid samples, and a cross-modal alignment mechanism is constructed by combining a self-supervised cost function. The training set is dynamically partitioned using a two-component Gaussian mixture model, which significantly reduces the interference of annotation bias. At the same time, optimal transport theory is used to model the global similarity between images and text, alleviating the modal gap problem and providing high-purity input for subsequent processing.

[0010] 2. Adaptive Filtering Based on Medium-Grained Features

[0011] Based on Transformer self-attention weights, key image regions and text tags are dynamically identified, eliminating redundant information interference. By focusing on the correlation between local and global tokens, the visual segments and semantic units with the highest information content are selected and retained. Combined with the residual optimization module to enhance feature representation, accurate compression and alignment of cross-modal information are achieved.

[0012] 3. Fine-grained attribute-level interactive learning

[0013] This system pioneers an attribute-oriented masking prediction mechanism: it randomly masks only pedestrian attribute words that match adjective-noun combinations (such as "blue backpack" and "striped shirt"), and then fuses visual and textual features through a multimodal interactive encoder. The encoder employs a layer-normalized multi-head cross-attention mechanism to establish fine-grained associations between local image regions and textual attributes during the reconstruction of masked words, significantly improving attribute-level semantic alignment accuracy.

[0014] 4. Multi-granularity collaborative training mechanism

[0015] An innovative three-level loss joint optimization strategy is designed: coarse-grained transport loss ensures global consistency, medium-grained similarity loss strengthens local matching, and fine-grained cross-entropy loss optimizes attribute reconstruction. A dynamic model saving mechanism is introduced during training, automatically selecting the optimal model parameters based on the validation accuracy, and finally outputting retrieval results that integrate multi-level complementary information.

[0016] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0017] A multi-granularity text-based pedestrian search method based on attribute learning and feature selection:

[0018] We perform coarse-grained feature learning on image-text pairs, use a probabilistic model to filter noisy data, and optimize cross-modal alignment through a self-supervised cost function to obtain global features.

[0019] Tokens from images and text are filtered based on the self-attention weights output by the encoder to generate medium-granularity feature representations;

[0020] Randomly mask pedestrian attribute words in the text that match adjective-noun combinations, fuse features through a multimodal interactive encoder and predict the masked words to achieve fine-grained attribute alignment;

[0021] The model is trained iteratively by combining coarse-grained loss, medium-grained loss, and fine-grained loss, and the optimal model is saved based on the validation accuracy. The pedestrian search results are then output.

[0022] Furthermore, the probability model is a two-component Gaussian mixture model.

[0023] Furthermore, the self-supervised cost function is implemented using optimal transport theory.

[0024] Furthermore, the representation of the global features is achieved by segmenting the image into N non-overlapping blocks, generating a marker sequence, and extracting the text by converting it into a marker sequence containing start and end characters.

[0025] Furthermore, the self-attention weights are extracted from the self-attention map of the last Transformer block of the encoder.

[0026] Furthermore, the selected token is L2 regularized, processed by a multilayer perceptron and a fully connected layer, and then max pooling is performed.

[0027] Furthermore, in the step of predicting the masked words, the loss function is cross-entropy loss.

[0028] Furthermore, the multimodal interactive encoder is implemented using a multi-head cross-attention mechanism, which performs layer normalization on the query, key, and value before executing multi-head cross-attention.

[0029] Furthermore, the overall loss function is a weighted sum of coarse, medium, and fine-grained losses.

[0030] And, a multi-granularity text pedestrian search system based on attribute learning and feature selection, comprising:

[0031] The coarse-grained processing module is used to learn features from image-text pairs, filter noisy data using a probabilistic model, and optimize cross-modal alignment through a self-supervised cost function to obtain global features.

[0032] The medium-granularity processing module is used to filter tokens of images and text based on the self-attention weights output by the encoder, and generate medium-granularity feature representations.

[0033] The fine-grained processing module is used to randomly mask pedestrian attribute words in the text that match adjective-noun combinations. It achieves attribute alignment by fusing features and predicting the masked words through a multimodal interactive encoder.

[0034] The training output module is used to combine multi-granularity loss functions to iteratively train the model, save the optimal model based on the validation accuracy, and output the search results.

[0035] And a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0036] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0037] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0038] 1. Multi-granularity collaborative optimization breaks through the modal gap

[0039] Through a three-tiered framework of coarse-grained noise filtering, medium-grained feature selection, and fine-grained attribute alignment, hierarchical collaborative learning of cross-modal information is achieved for the first time. At the coarse-grained level, noisy data is automatically isolated and global matching is optimized; at the medium-grained level, key visual regions and text markers are dynamically focused; and at the fine-grained level, attribute-level semantics are aligned at the pixel level, significantly improving the accuracy of cross-modal mapping between images and text.

[0040] 2. A leap in fine-grained semantic parsing capabilities

[0041] The innovative attribute-oriented masking mechanism (limited adjective-noun combination) combined with layer normalized multimodal fusion enables the model to accurately capture combined attribute features such as "blue backpack" and "striped shirt", maintaining sensitivity to pedestrian details in complex scenes and solving the problem of insufficient response to local attribute descriptions in traditional methods.

[0042] 3. Noise robustness is significantly enhanced.

[0043] Dynamic data cleaning mechanisms based on probabilistic models (such as two-component Gaussian mixture models) can effectively identify and filter label biases and mismatched pairs in the training set, enabling the model to maintain stable performance even under partially mismatched data conditions in real-world scenarios.

[0044] 4. Optimization of training efficiency and generalization ability

[0045] A multi-granularity loss combination strategy (coarse / medium / fine loss weighting) combined with a dynamic model saving mechanism automatically selects the optimal parameters by verifying accuracy, avoiding the risk of overfitting and improving the model's generalization ability to unknown scenarios.

[0046] 5. Efficient utilization of computing resources

[0047] Self-attention weight-driven feature selection significantly compresses redundant information, while the residual optimization module ensures effective representation of key features, maintaining high-precision retrieval requirements while reducing computational complexity. Attached Figure Description

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0049] Figure 1 This is a flowchart illustrating the principle and implementation of an embodiment of the present invention. Detailed Implementation

[0050] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0051] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0052] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0053] like Figure 1 As shown, this embodiment provides a multi-granularity text-based pedestrian search method based on attribute learning and feature selection, aiming to improve cross-modal alignment and retrieval accuracy between images and text. First, image and text pairs are acquired as input. Feature learning is performed at coarse-grained, medium-grained, and fine-grained levels using a multi-granularity framework network. At the coarse-grained level, positive and negative pair perception learning strategies are combined to optimize image-text matching. Then, a feature selection strategy is employed at the medium-grained level to focus on the most informative image regions and text markers, reducing redundant information and enhancing cross-modal alignment. Next, an attribute-based mask language modeling method is used to accurately learn the attribute features of images and text at the fine-grained level, and a multi-modal interactive encoder is used for modal fusion to further improve the attribute alignment capability of images and text. Finally, by iteratively training the model and selecting the optimal model on the validation set, high-precision text-based pedestrian search results are obtained. This method requires no additional manual annotation, effectively handles some mismatched image-text pairs, and has high robustness and accuracy. Its implementation process specifically includes the following steps:

[0054] Step S1: Obtain the text-based pedestrian search dataset, input it into the coarse-grained feature learning module, use a two-component Gaussian mixture model to filter noisy data so that the model can effectively learn positive samples, then randomly shuffle part of the training data so that the model can effectively learn negative samples, and combine the optimal transport theory to calculate the loss of the coarse-grained feature learning module.

[0055] Step S2: Input the image-text pairs from the dataset into the medium-granularity feature learning module, and then... The encoder obtains global features, through the last... The self-attention map of the block can obtain the local values ​​of each part. With the global The weights between them are determined, and the items to be retained are selected based on the weights. To filter important features;

[0056] Step S3: Input the image-text pairs from the dataset into the fine-grained feature learning module. Randomly mask words in the text related to pedestrian attributes, and then... The encoder obtains the image and text features containing masking information, and then inputs them into... Predict the pedestrian attribute words that are initially masked in the head, and calculate the cross-entropy loss of the fine-grained feature module as the loss of the fine-grained module.

[0057] Step S4: Iteratively train according to the specified training parameters, update the model parameters by optimizing the combined loss, continuously save the optimal model based on the validation accuracy, make full use of the complementarity of multi-granularity information, and finally obtain a combination of multi-granularity prediction results to output text-based pedestrian search results.

[0058] As a preferred embodiment, step S1 specifically includes the following steps:

[0059] Step S11: Use publicly available text-based pedestrian search datasets to perform data preprocessing and complete label extraction;

[0060] Step S12: Process the input image and text pairs using an image encoder and a text encoder, respectively, to extract global feature representations. For the input image... ,use pre-trained Perform feature extraction and divide the image into segments. A number of non-overlapping small blocks. , and These represent the height, width, and number of channels of the input image, respectively. This indicates the size of each small block. This represents the total number of blocks in the image. Each block is transformed into a one-dimensional label sequence through a trainable linear mapping. Then add a position code and a... Labels are used to form a label sequence: . Indicates the first Feature representation of an image patch yes The labeled feature representation is used to represent the global features of the entire image. The labeled sequence is fed into a layer with a depth of... Layer A network is used to capture the relationships between image patches. Finally, a linear projection is used to... mark Mapping to a shared image-text embedding space yields a global image representation. For the input text ,use A text encoder, this encoder is a modified version. Structure. Using a vocabulary of size 49152, the text is converted into a token sequence through byte-pair encoding, resulting in a sequence containing start characters. and end symbol The labeled sequence: Finally, linear projection is used to... mark Mapping to a shared image-text embedding space yields a global text representation. .in, and These represent the global feature representations of the image and text, respectively. Indicates the start symbol of a text sequence. The symbol that indicates the end of a text sequence. Indicates the first in the text The feature representation of each label.

[0061] Step S13: Next, calculate the loss value for each image-text pair:

[0062]

[0063] in, Representation model, Represents image-text pairs, It's the loss function, specifically the triple alignment loss. Applied to image-text pairs The calculated loss value is then input into the two-component Gaussian mixture model (...). Modeling is performed to distinguish between clean and noisy data. Posterior probability. The calculation formula is as follows:

[0064]

[0065] in, Indicates the first The probability that an image-text pair belongs to clean or noisy data. Indicates whether the data is clean or noisy. This is the loss value for that sample. Then, the set threshold is used. To divide the dataset into clean datasets and noisy datasets All satisfied The samples were divided into clean datasets. The remaining samples were then divided into a noisy dataset. The specific division formula is as follows:

[0066] ,

[0067] ,

[0068] All other modules will use a clean dataset. It can be used as training data for further optimization.

[0069] Step S14: To prevent overfitting from training solely with positive alignment, a cost function is automatically learned using a self-supervised learning strategy, incorporating optimal transfer. The cost function is calculated as follows:

[0070]

[0071] in, This represents the parameters of the network. It is an ideal matching matrix. and For the input image-text pair, and This is the image-text similarity matrix. It is a feedforward network used to predict transmission costs, and finally the cost function is optimized through optimal transmission loss.

[0072] As a preferred embodiment, step S2 specifically includes the following steps:

[0073] Step S21: Construct a medium-granularity feature learning module. First, obtain a clean global image representation according to step S13. With global text representation ;

[0074] Step S22: Next in and The last one Obtain global attention from the self-attention graph of the block and each part The correlation between them reflects the input (or )indivual The correlation between them. Given an image Output self-attention map global and local The correlation weights between them are , Representing a local area of ​​an image Quantity. Similarly, for text. The relevant weights are ,in It is text The output self-attention map, Representing a part of the text Quantity. Finally, based on the selection rate. Determine what needs to be filtered quantity.

[0075] Step S23: The high score obtained after using step S22 Then, the selected... Optimize for both images and text. The optimizations are as follows:

[0076]

[0077]

[0078] in, Representing the image The optimized result Representing text The optimized result Representing an image The value after regularization is similar. Representing text The value after regularization This indicates a max pooling operation. This represents a multilayer perceptron. This indicates a fully connected layer.

[0079] Step S24: Using the image-text pair features generated after step S23, calculate the cosine similarity between the two to measure the image... and text Cross-modal matching between them, the similarity calculation formula is as follows: , Representing an image and text The cosine similarity between them is used in both training and inference processes.

[0080] As a preferred embodiment, step S3 specifically includes the following steps:

[0081] Step S31: Construct a fine-grained feature learning module. Before performing step S12, the text in the image-text pair is first randomly masked. Specifically, this is done by first using a natural language processing toolkit. Part-of-speech tagging is performed, specifying phrases matching the pattern "[adjective][noun]" as attributes, and using a mask ratio. Randomly replace attribute words with Then, steps S12 and S13 are performed respectively to obtain positive sample image-text pairs. The input encoder yields the final embedded representation. ,in, This is an embedded representation of image-mask text pairs. This is the encoded feature representation of the image-text pair. For the first in the text indivual Feature representation, For text contained quantity, This is text containing a mask.

[0082] Step S32: Construct a multimodal interactive encoder to fully fuse image and text features. Execute step S31 to represent the masked text. As a query Representing the image As a key Sum .in, The length of the masked text. The number of image tokens, Indicates the first character in the masked text The characteristic representation of a token Indicates the first in the image The feature representation of each token is then processed through a multi-head cross-attention mechanism. Module and layer normalization of To compute the fused context representation:

[0083]

[0084] in, This represents the fused context representation. The operation is defined as follows:

[0085]

[0086] in, This is the embedding dimension of the mask marker. For each mask position... Then a multilayer perceptron was used. To predict the probability of each possible word, the formula is as follows:

[0087]

[0088] in, Indicates the first The first mask position corresponds to the th The predicted probability of each word. It refers to the size of the vocabulary list.

[0089] Step S33: Execute step S32 to obtain the prediction results of the masked attribute words, construct the loss function of the fine-grained feature learning module, and use it to optimize the output of the multimodal interactive encoder so that the reconstruction of the masked tags is as close as possible to the real attribute words. The specific formula is as follows:

[0090]

[0091] in, It is a collection of masked text tags. The same applies to vocabulary size. It is The predicted probability distribution of each mask position. This represents the one-hot encoding distribution of the true label. By minimizing... It can effectively learn fine-grained mapping relationships between image and text attributes.

[0092] As a preferred embodiment, step S4 specifically includes the following steps:

[0093] Step S41: Calculate the overall network loss function The calculation formula is as follows:

[0094]

[0095] The overall optimization objective consists of three parts, among which It is the optimal transmission loss, used to reduce the modal differences between the image and the overall text at a coarse-grained level. It is a cross-modal matching between images and text after feature filtering, which can reduce the modal differences between images and text at a medium-granularity level. It is a pedestrian attribute learning loss, which mainly reduces the local modal gap between image patches and text words by reconstructing mask words, thereby improving the fine-grained alignment of the model. The hyperparameters representing the loss of the coarse-grained feature learning module are: The hyperparameters representing the loss of the medium-granularity feature learning module are: This represents the hyperparameter of the loss of the fine-grained feature learning module.

[0096] Step S42: Perform iterative training according to the specified parameters, and calculate the overall loss based on step S41. Continuously update the gradients for iterative training. Specifically, use... The model acts as a modality-specific encoder. To enrich the training data, this embodiment performs random horizontal flipping, random cropping and padding, and random erasure on the images. The input image size is set to... Maximum tag length of text The model was trained using a setting of 77, employing the Adam optimizer and a cosine learning rate decay strategy, for a total of 60 training epochs. The initial learning rate for the CLIP parameters during training is set to... , The network's learning rate The learning rate of the randomly initialized modules is Temperature parameters Set to 0.015, batch size to 64, image and text embedding dimensions to 768 each, sharing latent dimensions. = 256. Attribute-based mask rate Set to 0.8, marker selectivity Set it to 0.3.

[0097] Step S43: During the reasoning process, by combining the prediction results of multiple granularities and making full use of the complementarity of multiple granularities of information, a text pedestrian retrieval function with higher accuracy and better robustness is obtained.

[0098] Step S44: During the training process, the model is validated at certain iteration intervals according to the validation interval flag, and the optimal model is continuously saved. When the number of iterations reaches the preset maximum iteration threshold, the training process ends and the best prediction accuracy of the current specified dataset is returned.

[0099] Compared with the prior art, the solutions provided in the embodiments of the present invention have the following beneficial effects:

[0100] 1. The present invention proposes a multi-granularity text pedestrian search method based on attribute learning and feature selection, which covers three levels of tasks: coarse-grained, medium-grained and fine-grained. By systematically aligning image and text features at multiple granular levels, it effectively improves the performance of existing technologies in text-based pedestrian search tasks. Its effectiveness has been verified through extensive experiments on multiple public benchmarks.

[0101] 2. This invention introduces a Gaussian mixture model based on triplet alignment loss at the coarse-grained level. By filtering out noise pairs in the training set, the model's recognition accuracy is enhanced. At the same time, the learning process is further optimized through optimal transmission loss, avoiding the overfitting phenomenon that may occur when the model is trained using only positive pairs, thereby improving robustness.

[0102] 3. At the medium-granularity level, the feature selection method proposed in this invention reduces redundant information and enhances cross-modal alignment capabilities by focusing on the most informative image regions and text tags. Compared with existing methods, this invention can more efficiently utilize the correlation between images and text, thereby improving matching accuracy.

[0103] 4. At the fine-grained level, this invention employs an attribute-based masking language modeling strategy, performing masking prediction only on words related to character attributes, thereby accurately learning fine-grained attribute features. Compared to traditional methods, this invention can perform modality fusion more accurately, improving the learning effect of attribute-level features.

[0104] This embodiment only uses category labeling, eliminating the need for additional manual labeling such as bounding boxes. Addressing the unique characteristics of text-based pedestrian retrieval tasks, this invention innovatively introduces a multi-granularity framework. The coarse-grained level optimizes the alignment of positive and negative sample pairs through a two-component Gaussian mixture model and optimal transfer method. Particularly when handling partially mismatched image-text pairs, it enhances the model's robustness to noisy data and improves its generalization ability in practical applications. The medium-granularity level uses a feature selection strategy to enable the model to focus on the most informative image regions and text markers. At the fine-grained learning level, an attribute-based masked language modeling strategy is innovatively introduced, focusing on attribute-related vocabulary in the text description and introducing a new multimodal interactive encoder to achieve precise alignment of image and text features. This method not only improves the modal fusion effect of images and text but also makes the retrieval and matching of person attributes more accurate through precise attribute-level feature learning. The multi-granularity text pedestrian search method based on attribute learning and feature selection constructed in this invention can effectively align image and text features systematically, thereby achieving more efficient and accurate cross-modal matching.

[0105] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0106] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0107] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0109] This invention is not limited to the preferred embodiment described above. Anyone inspired by this invention can derive other forms of multi-granularity text pedestrian search methods based on attribute learning and feature filtering. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.

Claims

1. A multi-granularity text pedestrian search method based on attribute learning and feature selection, characterized in that: We perform coarse-grained feature learning on image-text pairs, use a probabilistic model to filter noisy data, and optimize cross-modal alignment through a self-supervised cost function to obtain global features. Tokens from images and text are filtered based on the self-attention weights output by the encoder to generate medium-granularity feature representations; Randomly mask pedestrian attribute words in the text that match adjective-noun combinations, fuse features through a multimodal interactive encoder and predict the masked words to achieve fine-grained attribute alignment; The model is trained iteratively by combining coarse-grained loss, medium-grained loss, and fine-grained loss, and the optimal model is saved based on the validation accuracy. The pedestrian search results are then output.

2. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The probability model is a two-component Gaussian mixture model.

3. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The self-supervised cost function is implemented using optimal transport theory.

4. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The global feature representation is achieved by segmenting the image into N non-overlapping blocks, generating a marker sequence, and extracting the text by converting it into a marker sequence containing start and end characters.

5. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The self-attention weights are extracted from the self-attention map of the last Transformer block of the encoder.

6. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The selected token is L2 regularized, processed by a multilayer perceptron and a fully connected layer, and then max pooling is performed.

7. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: In the step of predicting the masked words, the loss function is cross-entropy loss.

8. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The multimodal interactive encoder is implemented using a multi-head cross-attention mechanism. Before performing multi-head cross-attention, the query, key, and value are subjected to layer normalization.

9. The multi-granularity text pedestrian search method based on attribute learning and feature selection according to claim 1, characterized in that: The overall loss function is a weighted sum of coarse, medium, and fine-grained losses.

10. A multi-granularity text pedestrian search system based on attribute learning and feature selection, characterized in that, include: The coarse-grained processing module is used to learn features from image-text pairs, filter noisy data using a probabilistic model, and optimize cross-modal alignment through a self-supervised cost function to obtain global features. The medium-granularity processing module is used to filter tokens of images and text based on the self-attention weights output by the encoder, and generate medium-granularity feature representations. The fine-grained processing module is used to randomly mask pedestrian attribute words in the text that match adjective-noun combinations. It achieves attribute alignment by fusing features and predicting the masked words through a multimodal interactive encoder. The training output module is used to combine multi-granularity loss functions to iteratively train the model, save the optimal model based on the validation accuracy, and output the search results.

Images