Text-pedestrian retrieval method based on big language model enhancement

Abstract

The invention discloses a text-pedestrian retrieval method based on big language model enhancement, and belongs to the field of computers and the Internet. The method specifically comprises the steps of dividing original text data into a test set and a training set for a combination pair of images and texts in an existing data set; meanwhile, calling a large language model to enhance the text data to generate an enhanced text; then, the enhanced text data are sent into the filtering model, and the text similarity between the enhanced text and the original text is calculated; when the text similarity is larger than a set threshold value, the enhanced text is reserved; otherwise, abandoning the enhanced text or regenerating the text. And finally, the reserved enhanced text data set is used for training or testing a text-pedestrian retrieval model, and the tested text-pedestrian retrieval model is used for personnel identification. According to the method, the retrieval performance of the model is remarkably improved, and more accurate matching is realized.

Description

Technical Field

[0001] The invention belongs to the field of computers and the Internet, and in particular relates to a text-pedestrian retrieval method based on large language model enhancement. Background Art

[0002] Text-person retrieval technology is widely used in personnel identification, for example, to find lost children, assist the police in finding criminals, etc. When a specific text description of a person is given, the text-person retrieval model can find the corresponding image based on the input text to achieve personnel identification.

[0003] For example, in a search scenario, if a user enters the query text: a woman wearing grey shorts, grey shoes and a white wallet hanging around her waist, the text-person search technology can be used to find pictures containing the woman corresponding to the text. This technology greatly improves the speed at which users can find the target person and helps users accurately determine the location of the target person.

[0004] Compared with directly using images for personnel identification, text-based personnel retrieval has a simpler search format, effectively improves the search speed, and can further protect the privacy of relevant personnel.

[0005] However, the datasets of existing related technologies are small in scale, and the styles of manually annotated text information vary greatly, which makes the text-based pedestrian retrieval technology, a cross-modal learning task from text to image, benefit little from large-scale cross-modal training. Summary of the invention

[0006] In order to solve the problems of small data set size, inconsistent annotation style and low quality in the field of text-pedestrian retrieval, the present invention provides a text-pedestrian retrieval method based on large language model enhancement, which significantly improves the retrieval performance of the model and achieves more accurate matching.

[0007] The text-to-person retrieval method based on large language model enhancement has the following specific steps:

[0008] Step 1: For the image and text pairs in the existing data set, the original text data is divided into a test set and a training set; at the same time, a large language model is called to enhance the text data to generate enhanced text;

[0009] Step 2: Send the enhanced text data to the filtering model to calculate the text similarity between the enhanced text and the original text;

[0010] The calculation formula for the similarity between the original text and the enhanced text is:

[0011]

[0012] Ti ori is the original text, T i aug To enhance text, we use the Sentence Transformers model st (·) Encode the original text and the enhanced text respectively to get f st (T i ori ) and f st (T i aug ).

[0013] Step 3: When the text similarity is greater than the set threshold, the enhanced text is retained; otherwise, the enhanced text is discarded or regenerated.

[0014] Step 4: Use the retained enhanced text dataset for training or testing the text-person retrieval model, and use the tested text-person retrieval model for person recognition.

[0015] If used for model training, the enhanced text data is passed through the text encoder to obtain the text feature value f text (T i aug ), the image feature value f obtained by the image encoder corresponding to the text img (V i ) to calculate and obtain the text image similarity matrix s(V i ,T i aug ), and trained by the loss function of the model;

[0016] The specific calculation formula is as follows:

[0017] s(V i ,T i ) = f text (T i )·f img (V i ) T

[0018] V i is the original image, f img (·) is the image encoder, f text (·) is the text encoder;

[0019] If used for model testing, both the enhanced text and the original text are obtained through the text encoder to obtain the corresponding text feature values, and the text-image similarity matrix s(V i ,T i aug ) and s(V i,T i ori ), calculate the two text image similarity matrices by weighted summation, and get the new text image similarity matrix value S:

[0020] S=λ·s(V i ,T i aug )+(1-λ)·s(V i ,T i ori )

[0021] Here, λ is a uniformly distributed random number between [0,1].

[0022] The advantages of the present invention are:

[0023] 1) The present invention provides a text-to-person retrieval method based on large language model enhancement. When training the text-to-person retrieval model, the generalization ability of the retrieval model is enhanced by introducing diversified texts rewritten by the large language model; when testing the text-to-person retrieval model, the original text and the enhanced text are comprehensively tested to enhance the robustness of the retrieval model; and a more comprehensive and accurate basis is provided for subsequent retrieval matching.

[0024] 2) The present invention provides a text-to-person retrieval method based on large language model enhancement, which expands the existing text-based pedestrian retrieval data set by introducing a large language model to enhance text data; the large language model can extract key information of the text content, maintain the integrity of sentence structure and semantics, and unify the text style.

[0025] 3) The present invention provides a text-pedestrian retrieval method based on large language model enhancement, which can effectively remove enhanced texts with low text similarity through filtering models, thereby increasing the diversity and reliability of enhanced texts.

[0026] 4) The present invention provides a text-to-person retrieval method based on large language model enhancement, which can effectively balance the role of the original text and the test text in the testing process of the retrieval model by weighted averaging the text-image similarity matrix between the original text and the enhanced text and the image respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a text-pedestrian retrieval method based on large language model enhancement in this application;

[0028] Figure 2 It is a schematic diagram of the filtering model framework adopted in this application;

[0029] Figure 3 It is a schematic diagram of the training framework of the text-pedestrian retrieval model adopted in this application;

[0030] Figure 4 It is a schematic diagram of the test framework of the text-pedestrian retrieval model adopted in this application. DETAILED DESCRIPTION

[0031] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0032] The present invention discloses a text-to-person retrieval method based on large language model enhancement. First, a large language model is introduced to enhance text data, and a data set is expanded by generating text; at the same time, the large language model can extract key information of the text content and unify the style of the text in the data set. Secondly, the generated text data is sent to the filtering model, the text similarity between the generated text and the original text is calculated, and a threshold is set for screening to alleviate the hallucination problem of the large language model and ensure the style uniformity of the generated text data. When the text similarity is greater than the threshold, the generated text is retained to expand the data set; otherwise, the generated text is considered to be discarded or regenerated; finally, the training framework and test framework of the text-to-person model are trained or tested using the retained generated text, and finally used for personnel recognition. The present invention solves the problem of small data set size and inconsistent annotation style in the field of text-to-person retrieval, significantly improves the retrieval performance of the model, and achieves more accurate matching. This innovative method comprehensively uses large language models for data enhancement, text similarity calculation, threshold screening and other steps to provide a comprehensive and effective solution for text-to-person retrieval tasks.

[0033] The text-to-person retrieval method based on large language model enhancement, such as Figure 1 As shown, the specific steps are as follows:

[0034] Step 1: For the image and text pairs in the existing data set, the original text data is divided into a test set and a training set; at the same time, a large language model is called to enhance the text data to generate enhanced text;

[0035] First, select a suitable large language model for local deployment;

[0036] Then, input the corresponding instructions to call the large language model to generate new text based on the original text of the data set to enhance the data set.

[0037] By introducing a large language model to enhance the text in the dataset, it can capture and maintain consistent semantics and style, thus ensuring that the enhanced text data is more unified; this process can not only enrich the dataset, but also unify the overall style of the text while maintaining semantic consistency. Therefore, by selecting an appropriate large language model and correctly calling its generation function, text data can be enhanced efficiently in a local environment, providing a more powerful and consistent data foundation for subsequent tasks.

[0038] Step 2: Send the enhanced text data to the filtering model to calculate the text similarity between the enhanced text and the original text;

[0039] Hallucination problems are common in large language models, which mainly refer to the fact that large language models occasionally generate content that does not match user input, contradicts previously generated content, or is inconsistent with existing world knowledge.

[0040] By calculating the text similarity score between the original text and the newly generated text of the large language model, setting a score threshold, discarding the newly generated text below the threshold or inputting instructions to make the large language model regenerate new text, the text style generated by the large language model is further unified; this process not only helps to maintain the high quality of text data, but also through the regeneration method, the large language model tends to produce a language expression style that is more consistent with the original text. Therefore, by cleverly setting the threshold and regeneration strategy, the consistency of text style can be better controlled in the text generation stage.

[0041] The calculation formula for the similarity between the original text and the enhanced text is:

[0042]

[0043] T i ori is the original text, T i aug To enhance text, we use the Sentence Transformers model st (·) Encode the original text and the enhanced text respectively to get f st (T i ori ) and f st (T i aug ).

[0044] Step 3: When the text similarity is greater than the set threshold, the enhanced text is retained; otherwise, the enhanced text is discarded or regenerated.

[0045] Step 4: Use the retained enhanced text dataset for training or testing the text-person retrieval model, and use the tested text-person retrieval model for person recognition.

[0046] If used for model training, the text data is first input into the large language model for processing, new text data is generated according to the instruction requirements, and the newly generated text data is sent to the filtering model for screening; the newly generated text data serves as a supplement to the original text data, expanding the data set.

[0047] After that, the enhanced text data is passed through the text encoder to obtain the corresponding text feature value f text (T i aug );

[0048] At the same time, the image corresponding to the text in the dataset is obtained through the image encoder to obtain the image feature value f img (V i ).

[0049] The text feature value f text (T i aug ) and image eigenvalue f img (V i ) to calculate and obtain the text image similarity matrix s(V i ,T i aug ):

[0050] The specific calculation formula is as follows:

[0051] s(V i ,T i ) = f text (T i )·f img (V i ) T

[0052] V i is the original image, f img (·) is the image encoder, f text (·) is the text encoder;

[0053] Finally, according to the corresponding loss function, the corresponding model is trained.

[0054] This process aims to optimize the performance of the model by training it with the help of the generated text data and image data. The key to the entire process is to make full use of the text generated by the large language model and provide a richer and more targeted data foundation for model training through intelligent screening of the filtering model.

[0055] If used for model testing, first, the text data is input into the large language model for processing, new text data is generated according to the instruction requirements, and the newly generated text data is sent to the filtering model for screening. The newly generated text data is sent to the text encoder to obtain text feature values; the original text data is sent to a consistent text encoder to obtain another set of text feature values.

[0056] Then, the corresponding image is passed through the image encoder to obtain a set of image feature values.

[0057] Next, a set of text-image similarity matrices s(V i ,T i ori ); At the same time, another set of text-image similarity matrix s(V i ,T i aug ).

[0058] Perform weighted sum calculation on the two text-image similarity matrices to obtain a new text-image similarity matrix value S:

[0059] S=λ·s(V i ,T i aug )+(1-λ)·s(V i ,T i ori )

[0060] Here, λ is a uniformly distributed random number between [0,1].

[0061] Finally, the retrieval task is performed according to the feature score corresponding to the matrix value S to obtain the matching result.

[0062] This composite process is based on the newly generated text data and comprehensively considers the relationship between text and image through the text-image similarity matrix, achieving more accurate and comprehensive retrieval results.

[0063] When using text for pedestrian retrieval, existing technologies usually consider changing the network structure or optimizing the target, or building a larger data set, adding attribute annotations, etc. to improve the performance of the model. These methods are relatively complex and costly to implement. At the same time, in terms of data enhancement, image processing is generally given priority. Few researchers have explored ways to enhance text data, and even fewer have used large language models for data enhancement.

[0064] First, some terms involved in the embodiments of the present application are explained below.

[0065] Large language models refer to machine learning models with a large number of parameters and computing resources. These models require a large amount of data and computing power during training and have millions to billions of parameters.

[0066] Data enhancement means making limited data produce the equivalent value of more data without substantially increasing the data.

[0067] Text-image similarity matrix: the scalar product of the image feature values ​​output by the image encoder and the text feature values ​​output by the text encoder.

[0068] Preset threshold: a value set by the inventor based on implementation data and experience.

[0069] Image encoder: used to map images to feature space.

[0070] Text encoder: used to map text into the same feature space.

[0071] Example:

[0072] There are three commonly used existing datasets in the field of text-pedestrian retrieval: CUHK-PEDES, RSTPReid, and ICFG-PEDES. The formats of these datasets are all image and text pairs, so text data can be enhanced for different datasets. At the same time, different types of data can be enhanced according to the task. For example, if you want to use the data enhancement training framework, you only need to enhance the text data in the training set; if you want to use the data enhancement test framework, you only need to enhance the text data in the test set; if you want to use text data enhancement in both the training and test stages, you need to enhance the text data in both the test and training sets.

[0073] CUHK-PEDES is a pedestrian dataset with caption annotations. It contains 40,206 images with more than 13,003 persons. The images are collected from five existing person retrieval datasets (CUHK03, Market-1501, SSM, VIPER, and CUHK01), and each image is annotated with 2 text descriptions by crowd workers. The sentences contain rich details about the person's appearance, actions, and postures.

[0074] RSTPReid contains 20,505 images of 4,101 persons from 15 cameras. Each person has 5 corresponding images taken by different cameras. These images have complex indoor and outdoor scene transitions and backgrounds of different time periods. Each image is accompanied by 2 text descriptions.

[0075] ICFG-PEDES: A large-scale database for text-to-image person re-identification, i.e., text-based person retrieval. It contains 54,522 images of 4,102 identities, each with only one corresponding text description. The dataset is divided into a training set and a test set, the former containing 34,674 image-text pairs of 3,102 identities, and the latter containing 19,848 image-text pairs of the remaining 1,000 identities. The average length of the description is 37.2 words.

[0076] Sentence Transformers: is a Python library for sentence, text, and image embedding. It can compute text embeddings for more than 100 languages ​​and easily use them for common tasks such as semantic text similarity, semantic search, and synonym mining.

[0077] The CUHK-PEDES data set given in this embodiment contains 80,540 text data, the training data set R contains 68,126 text data, the verification data set V contains 6,158 text data, and the test data set U contains 6,156 text data. After the training data set R is enhanced, the new training data set R' contains 136,252 text data, and after the test data set u is enhanced, the new test data set u' contains 12,312 text data. It should be emphasized that the data sets to which the method proposed in the present invention can be applied include but are not limited to the above three data sets.

[0078] Then, this embodiment selects Vicuna as a large language model for text data enhancement.

[0079] Following Stanford Alpaca, scholars from UC Berkeley, CMU, Stanford and other institutions jointly released the latest open source large language model Vicuna, which contains 7B and 13B parameters. Among them, the 13B parameter model has a training cost of only $300, reaching more than 90% of ChatGPT's capabilities. Specifically, Vicuna can generate new text data through the instruction: "Please rewrite this sentence while keeping the semantics unchanged."

[0080] Secondly, a filtering model is proposed to alleviate the hallucination problem of large language models.

[0081] A common problem among many large language models is the hallucination problem. When a large language model is instructed to generate new text data, in some cases it will output some non-text symbols or sentences that are irrelevant to the semantics of the original text. The existence of these elements is noise compared to the original data set, which will have a certain impact on the training of subsequent models.

[0082] Therefore, the present invention proposes a filtering model to alleviate this problem by setting a certain threshold to remove low-quality generated texts. The threshold is a hyperparameter, and its selection is related to the data set and the model, and is generally determined based on experience. After the threshold is determined, the data set is screened, and the corresponding generated texts with text similarity lower than the threshold will be deleted or the large language model will be instructed to regenerate new texts according to instructions. The corresponding generated texts with text similarity higher than the threshold will be retained as a data set used in the training or testing part of the subsequent model.

[0083] Then, a training framework is proposed for text data augmentation for model training.

[0084] After the text data in the training set is fed into the large language model, the large language model outputs the corresponding text data according to the instructions.

[0085] The newly generated text data is used as a supplement to the text data in the original data set. The appropriate threshold parameters are determined according to the model and data set, and then the text data is sent to the filtering model. After the filtering model filters the text data according to the threshold, the text data in the training set has a more unified style, which is more convenient for model learning.

[0086] The above is an enhancement operation on the text data in the training set. After that, the text data in the training set is sent to the text encoder to obtain the text feature value, and the image data in the training set is sent to the corresponding image encoder to obtain the image feature value.

[0087] Note that there is no restriction on the image encoder and text encoder here. Different models may have different network structures, so the image encoder and text encoder cannot be completely consistent.

[0088] The text feature values ​​and image feature values ​​obtained above can be used to train the model according to different loss functions.

[0089] Finally, a testing framework is proposed for text data augmentation for model testing.

[0090] The text data in the test set is sent to the large language model, which generates new text with consistent semantics according to the input instructions. The appropriate threshold parameters are determined according to the model and data set, and then the text data is sent to the filtering model. After the filtering model filters the text data according to the threshold, the text data in the training set has a more uniform style.

[0091] After that, the generated text is fed into the text encoder to obtain a text feature value, and the original text in the test set is fed into the same text encoder to obtain another text feature value. After the image in the test set is fed into the image encoder, an image feature value is obtained. Since both the original text and the newly generated text are used to describe the features of pedestrians in the same image, the two text feature values ​​are respectively matched with the same image feature value to obtain two matching scores.

[0092] The weighted sum of these two matching scores is used as the basis for subsequent model retrieval, and the corresponding retrieval results are finally obtained.

[0093] The technical solution provided by this embodiment has the following beneficial effects:

[0094] The present invention provides an innovative text data enhancement method that fully utilizes the powerful generation capability of a large language model. The method expands the size of the data set by inputting the text data of the original data set into the large language model and generating new text data according to the instructions. At the same time, the large language model can extract key information of the text content during the generation process to achieve the unification of the text style in the data set.

[0095] In the data generation stage, a filtering model is introduced to solve the hallucination problem that may occur in large language models. After the generated text data passes through the filtering model, the quality of the text data is regulated by setting a threshold to ensure the consistency of the style of the generated text. This step aims to improve the reliability and applicability of the generated text, so that the enhanced dataset is more effective for training and testing.

[0096] Furthermore, the enhanced dataset can be used for model training or testing. If used for model training, the text data is passed through the text encoder to obtain text feature values, which are calculated with the image feature values ​​obtained by the image encoder, and trained through the model's loss function. If used for model testing, both the generated text and the original text are passed through the text encoder to obtain corresponding text feature values, and the text-image similarity matrix is ​​calculated with the image feature values. The two text-image similarity matrices are calculated by weighted summation to obtain a new text-image similarity matrix value, which provides a more comprehensive and accurate basis for subsequent retrieval matching.

[0097] In summary, this method generates text by using a large language model and combines it with a filtering model to effectively enhance the data set, while providing a unified data foundation for model training and testing. This innovative method has significant advantages in solving the hallucination problem in text generation by large language models and improving the quality of data sets, and provides a comprehensive and feasible solution for text data processing in the field of deep learning.

[0098] In the second aspect, this embodiment also provides a schematic diagram of a filtering model framework of a text-pedestrian retrieval method based on a large language model enhancement, such as Figure 2 As shown:

[0099] The core components of the filtering model include two key parts: First, the text similarity between the original text and the generated text is calculated. The Python library Sentence-Transformers is used to calculate the text similarity of two sentences by calling its function, where the value range of text similarity is [-1, 1]. The purpose of this step is to quantify the semantic similarity between the original text and the generated text, providing an objective indicator for subsequent text quality screening.

[0100] Secondly, the filtering model removes the generated texts with lower quality by setting a predetermined threshold.

[0101] The choice of threshold as a hyperparameter is related to the characteristics of the specific dataset and model, and usually needs to be determined empirically.

[0102] Once the threshold is determined, the generated texts can be screened, and the corresponding generated texts with text similarity below the threshold can be deleted or the large language model can be re-used to generate new texts, while the generated texts with text similarity above the threshold are retained to expand the data set. The effect of this step is to filter out those generated texts that have a large semantic difference with the original text, thereby improving the quality of the data set.

[0103] Finally, through the processing of the filtering model, a generated text dataset that has been screened by text similarity can be obtained, which is more conducive to the training or testing of subsequent models. The introduction of the filtering model not only improves the purity of the dataset and reduces the interference with model training, but also provides more reliable and high-quality input for subsequent tasks. This innovative method effectively solves the hallucination problem in text generation by large language models and provides a feasible solution to deal with noise interference.

[0104] In a third aspect, the present application embodiment also provides a schematic diagram of a training framework for a text-to-person retrieval method enhanced by a large language model, such as Figure 3 As shown:

[0105] The training set refers to the data set used for training and learning in machine learning models. Figure 3In the training framework shown, by performing enhancement operations on the text data of the training set, richer and more diverse learning samples can be provided for the machine learning model. The text data T in the training set is fed into the large language model, and new text data T′ is generated through instructions, thereby expanding the number of texts in the training set to twice the original amount. The key to this step is to ensure that the newly generated text data is semantically consistent with the original data to maintain the semantic consistency of the overall data set.

[0106] The filtering model is introduced to filter the generated text data by adjusting the appropriate threshold parameters. The purpose of this process is to ensure that the text data in the training set has a more unified style, making it easier for the model to capture the common characteristics of the data during the learning process. In this way, more targeted training data is provided to the model, which helps to improve its learning effect and generalization ability.

[0107] Based on data enhancement, the text data in the training set is fed into the text encoder E t , and obtain the corresponding text feature value t. At the same time, the image data I in the training set is sent to the corresponding image encoder E i , and get the image feature value v.

[0108] By using different loss functions, the text feature values ​​and image feature values ​​are matched to train the model. This step aims to strengthen the connection between text and image, and enable the model to better understand the connection between text description and image content. The design of the entire training framework focuses on the organic combination of data enhancement, feature extraction and model training to improve the model's performance on complex tasks.

[0109] In summary, the innovation of this training framework lies mainly in enhancing the text data of the training set, and through the synergy of the filtering model and the encoder, providing the model with more challenging and diverse training data, thereby improving its generalization performance and task adaptability.

[0110] In a fourth aspect, this embodiment also provides a test framework schematic diagram of a text-to-person retrieval method based on a large language model enhancement, such as Figure 4 As shown:

[0111] The test set is a dataset used to evaluate the final performance of the model. It does not overlap with the training set. The purpose of the test set is to evaluate the performance of the model on data that has not been seen before and to determine whether the model is accurate and robust enough.

[0112] By using the powerful semantic generation capability of the large language model, new text data T′ is generated by inputting instructions, which is semantically consistent with the original text data T. The key to this step is to ensure that the generated text is semantically consistent with the original data, so that the subsequent model testing and matching process can achieve good results. By adding the amount of generated text data to the amount of text in the original test set, the size of the test set is successfully doubled, providing more sufficient data support for subsequent model training.

[0113] Next, the filtering model is introduced, whose goal is to make the text data in the training set present a more uniform style by adjusting the appropriate threshold parameters. The importance of this step is to ensure that the model can have a better understanding of the style and context of the text during retrieval, thereby improving the accuracy and consistency of the retrieval results.

[0114] After generating the text, it is fed into the text encoder E t , and get the corresponding text feature value t. At the same time, the original text in the test set is processed in the same way to get another text feature value t′. These two text feature values ​​are respectively used with the image feature value v of the picture in the test set to calculate the text image similarity matrix s, s′. The specific calculation formula is:

[0115] s=v×t

[0116] s′=v×t′

[0117] Since both the generated text and the original text are descriptions of the features of pedestrians in the same image, the feature matching score reflects the degree of association between the text and the image.

[0118] Finally, by weighted summing the two text-image similarity matrices, we obtain a comprehensive feature matching score S, which becomes the basis for subsequent model retrieval. The specific calculation formula for S is:

[0119] S = λ·s+(1-λ)·s′

[0120] The value of λ varies according to different models and data sets.

[0121] This comprehensive score is designed to more comprehensively reflect the relationship between text and images, providing a more accurate and reliable basis for the generation of retrieval results. The entire process uses meticulous data processing and model design to make the final retrieval results more in line with actual needs and provide strong support for the deep connection between images and text.

[0122] The text data enhancement method proposed in the present invention can be applied to model training or model testing alone, and can also be applied simultaneously in the model training and testing stages, and has a high degree of flexibility.

Claims

1. A text-person retrieval method based on large language model enhancement, characterized in that: The specific steps are as follows: Step 1: For the image and text pairs in the existing data set, the original text data is divided into a test set and a training set; at the same time, a large language model is called to enhance the text data to generate enhanced text; Step 2: Send the enhanced text data to the filtering model to calculate the text similarity between the enhanced text and the original text; Step 3: When the text similarity is greater than the set threshold, the enhanced text is retained; Otherwise, discard the enhanced text or regenerate the text; Step 4: Use the retained enhanced text dataset for training or testing the text-person retrieval model, and use the tested text-person retrieval model for person recognition; If used for model training, the enhanced text data is passed through the text encoder to obtain text feature values The image feature value f obtained by the image encoder for the image corresponding to the text img (V i ) to calculate and obtain the text image similarity matrix And train it through the loss function of the model; The specific calculation formula is as follows: s(V i ,T i )=f text (T i )·f img (V i ) T Vi is the original image, f img (·) is the image encoder, ftext(·) is the text encoder; If used for model testing, both the enhanced text and the original text are obtained through the text encoder to obtain the corresponding text feature values, and the text-image similarity matrix is ​​calculated with the image feature values. and The two text-image similarity matrices are calculated by weighted summation to obtain a new text-image similarity matrix value S: Here, λ is a uniformly distributed random number between [0,1].

2. The text-to-person retrieval method based on large language model enhancement as claimed in claim 1, characterized in that: The calculation formula for the similarity between the original text and the enhanced text is: is the original text, To enhance text, we use the Sentence Transformers model st (·) After encoding the original text and enhanced text respectively, we can get and

Images