Text-image retrieval method based on phrase-level mask and large language model

Abstract

The application discloses a text image re-identification method based on a phrase-level mask and a large language model, which comprises the following steps: collecting paired text descriptions and person images, uniformly preprocessing the paired text descriptions and the person images, and obtaining a person re-identification image dataset; constructing a large language model according to the person re-identification dataset, performing language detection on a user query text, performing quality discrimination on a detection result, and generating normalized text for the input text that passes the quality discrimination; constructing a cross-modal feature joint network based on a multi-element interaction attention mechanism, and obtaining a target person image in which image features and text features are fused; performing network model training optimization by using a joint cross-modal projection correction model, and obtaining an optimal multi-scale information interaction model; and repeatedly performing the above steps on input text to be detected for secondary image matching, and outputting corresponding target person images, so that end-to-end person re-identification in a complex scene is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a text image re-identification method based on phrase-level masking and a large language model. Background Technology

[0002] Text-based image re-identification is a task in intelligent video surveillance used to query for target individuals across multiple cameras. Traditional methods neglect complex or special scenes where pedestrian images are unavailable, such as remote roads or occluded situations. To address this, eyewitness descriptions can be used for searching, i.e., text-based image re-identification. This method ranks images of people in a large image database by comparing the similarity between query text and images, and selects the top-ranked images as matches. Because using text descriptions as queries is simpler and more natural, text-based image re-identification has broad application prospects. Text-based image re-identification of people is a challenging task. During processing, the handling of text descriptions is often affected by differences between Chinese and English languages, non-standard expressions, or ambiguity, making alignment between different languages ​​or modalities more difficult. In such cases, cross-language and cross-modal query matching often results in errors, reducing recognition accuracy. Furthermore, images may suffer from occlusion, background clutter, and pose interference. In recent years, two main methods have been used to narrow the modal gap between text and images: global matching methods and local matching methods. Global matching methods fail to fully exploit local details in images and lack sufficient cross-modal interaction in intermediate layers. Local matching methods are highly complex, can destroy the contextual information of images and text or introduce noise, thus affecting the alignment results of image and text features. At the same time, due to the large amount of computation required, the information interaction in local matching methods inevitably reduces inference efficiency, making them difficult to implement in practical applications. Summary of the Invention

[0003] The present invention proposes a text image re-identification method based on phrase-level masking and large language model, which can at least solve one of the technical problems in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A text image re-identification method based on phrase-level masking and a large language model includes the following steps:

[0006] S100. Collect paired text descriptions and person images and perform unified preprocessing to obtain a person re-identification image dataset.

[0007] S200. Construct a large language model based on the person re-identification dataset, perform language detection on the user query text, judge the quality of the detection results, and generate normalized text for the input text that passes the quality judgment.

[0008] S300: Construct a cross-modal feature joint network based on a multi-interaction attention mechanism to obtain a target person image that is fused with normalized text image features and text features;

[0009] S400: A joint cross-modal projection correction model is used to train and optimize the joint network of cross-modal features to obtain the optimal multi-scale information interaction model.

[0010] S500. Based on the optimal multi-scale information interaction model, repeat steps S100 to S300 to perform secondary image matching on the input text to be detected and output the corresponding target person image.

[0011] Furthermore, the method for language detection of the user query text in step S200 of the present invention includes:

[0012] S210, Input text Perform language testing;

[0013] S220, Call the large language model to process the input text The system performs autoregressive generation and incorporates terminology normalization and length control mechanisms to process the input text. Standardization;

[0014] S230. A comprehensive quality assessment mechanism is constructed by combining multiple dimensions such as grammatical correctness, text fluency, and entity coverage to judge the translation quality and backtrack the standardized input text.

[0015] S240. Normalize the input text that has passed the quality assessment and generate text. And transmit it to the Transformer text encoder;

[0016] The language detection method in step S210 includes:

[0017] S211, Input text Preprocessing is performed, and the preprocessing results are input into the large language model mBERT for language decision-making;

[0018] The large language model mBERT converts the preprocessed results into fixed-dimensional word vector representations. It analyzes the input text using contextual information and outputs the probability of each language.

[0019] Output probability vector ,in, It represents the probability that the input text is in Chinese. It represents the probability that the input text is in English.

[0020] S212. Based on the language discrimination results, determine the subsequent processing decision;

[0021] Define indicator functions The routing indicator used to select Chinese or English:

[0022]

[0023] in, This is an indicator function that returns 1 if the condition is true and 0 otherwise. That is, when the language discrimination result is Chinese... ,otherwise ;

[0024] S213. Introduce a threshold to address situations where subsequent processing decisions are uncertain.

[0025] Introduce a threshold To handle situations involving mixed or ambiguous language; when or When the difference between them is less than the threshold, a preset processing step is initiated.

[0026] The specific discrimination mechanism is as follows:

[0027] when When, it enters the ambiguity handling process; when At that time, according to ;

[0028] Otherwise, proceed to the next processing step.

[0029] Furthermore, the subsequent processing flow described in this invention includes:

[0030] A glossary and hints were constructed based on the Person Re-identification Image Dataset I;

[0031] Text descriptions and annotations are extracted from each image in the person re-identification image dataset. As a preliminary source of terminology;

[0032] Organize the initial terminology sources by category: color category Clothing ; Items carried Posture category ,

[0033] Establish a domain terminology list Define synonymous normalization mappings. :

[0034] ;

[0035] in, Represents a collection of English words. Cosine similarity of word vectors The threshold is defined as follows: the construction includes: further constructing a prompt P, which is used to constrain the large language model to retain key entities and prohibit the addition or deletion of meanings during the generation process, and to unify related terms into the aforementioned domain terminology table S.

[0036] Furthermore, in step S200 of the present invention, a large language model is invoked to process the input text. Autoregressive generation methods include:

[0037] During the translation process, a large language model is first invoked for autoregressive generation;

[0038] Large language models analyze input Chinese text. The process is performed to calculate the unnormalized logit value. :

[0039]

[0040] In the formula, ,in This is the size of the English vocabulary; the logit value reflects the model's predicted probability for each possible word generated.

[0041] The terminology normalization and length control mechanisms include:

[0042] S221. Strengthen terminology and length constraints;

[0043] The system further performs weighted correction on logits to enhance the standardization of terminology during the generation process and the consistency of translation length;

[0044] By introducing a domain terminology table and length control items The generated logits are adjusted to ensure that the generated text not only conforms to domain terminology requirements but also has a reasonable length. The adjusted logit value is... for:

[0045]

[0046] in: Indicates and Consistent dimensional vectors of all 1s Glossary of field terms The indicator function for the vocabulary ensures that the generated vocabulary conforms to the domain terminology requirements; and These represent the minimum and maximum length thresholds for the generated text, respectively. For length shaping, it represents the length generated. Exceeding the specified range If the value is negative, a penalty is applied; otherwise, the value is 0, which is used to control the length of the translation result. and These are weighting parameters used to balance the effects of terms and length, and it is recommended to set them to [value missing]. , ;

[0047] S222, Temperature and Nuclear Sampling;

[0048] The system uses temperature softening and kernel sampling mechanisms to select words. Temperature softening generates smoothness, while kernel sampling limits the generated word set.

[0049]

[0050] Among them: Cat( ) represents the category distribution sampling function, and represents the th The tokens obtained from the generation steps This is a temperature parameter used to control the smoothness of the model output; a recommended range is... ; The smallest subset of the vocabulary is selected using a kernel sampling method, such that the cumulative probability of the smallest subset is greater than or equal to 1. To ensure the diversity and accuracy of the generated results, it is recommended that... ;

[0051] S223, Domain Adaptability and Contextual Understanding;

[0052] The system introduces a context adjustment factor. To dynamically adjust the translation results, the system incorporates the current context information in each generation step. Combined with the model's output logits, the selection of each word is adjusted:

[0053]

[0054] in: Indicates the first The current context information corresponding to each generation step Indicates the first The tokens obtained from the generation steps The context adjustment factor reflects the impact of the current context on word translation; it is generated through the analysis and reasoning of the context to optimize word selection. The weights adjusted for context are recommended to be set to [value]. ;

[0055] S224, Terminology Normalization and Generation Process;

[0056] For each generated token Through synonym normalization mapping Perform terminology normalization; each token after normalization... It meets the pre-defined requirements of the domain.

[0057]

[0058] in, It is based on the domain terminology table A mapping function for synonym normalization is used to ensure that the generated text conforms to domain specifications.

[0059] Furthermore, the method for constructing the comprehensive quality assessment mechanism in step S230 of the present invention includes:

[0060] Mass fraction calculation:

[0061] quality score The evaluation combines three metrics: entity coverage, perplexity regularization, and semantic consistency, as shown in the following formula:

[0062]

[0063] in: These are weighting coefficients used to balance the influence of different quality assessment indicators; This represents entity coverage, measuring whether all important entities in the source text are preserved during translation. It is the level of confusion, which measures the fluency of the translated language; Semantic consistency measures the semantic consistency between the translation and the source text.

[0064] Entity coverage:

[0065] Entity coverage is calculated by comparing the entity sets extracted from the source text and the translated text, using the following formula:

[0066]

[0067] in: A collection of entities in the source text; For the set of entities in the translated text; Indicates bilingual mapping Aligned set of entities;

[0068] Perplexity regularization:

[0069] The perplexity score (ppl) is used to measure the fluency of the generated translation, and the calculation formula is as follows:

[0070]

[0071] in: It is the calculation of each word The probability of; It is the length of the translated text.

[0072] If the perplexity exceeds the predetermined maximum value Perform regularization:

[0073]

[0074] Semantic consistency:

[0075] Semantic consistency is assessed by calculating the similarity between the source and translated texts in the semantic space, as shown in the following formula:

[0076]

[0077] in: This is the sentence vector for the English translation; This represents the Chinese sentence vector projected onto the English space.

[0078] Rollback mechanism and anomaly detection;

[0079] When mass fraction Below the preset threshold The system will revert to a rule-based or dictionary-based translation method for correction, under the following conditions:

[0080]

[0081] Meanwhile, an anomaly detection mechanism is introduced to automatically detect potential errors in the translation and trigger a rollback mechanism for correction.

[0082] Furthermore, the method for constructing the joint cross-modal projection correction model in step S400 of the present invention includes:

[0083] S410. Construct the matching loss function, identity loss function, and diversity loss function respectively, and combine them with the scale consistency regularization to construct the total loss function.

[0084] S420. An adaptive weighting strategy and regularization are introduced to optimize the total loss function;

[0085] The total loss function method in step S410 includes:

[0086] Constructing a cross-modal projection matching loss from text to image :

[0087]

[0088] Where N represents the number of samples in a training batch. Indicates the first The final text representation obtained after a text description passes through a multi-scale information interaction network; Indicates the first The standardized representation of image features corresponding to the image of Zhang; Representing text features Image features The predicted probability of a matching pair; Representing text features Image features The true probability of a match; Indicates whether the two are true matching tags of a matching pair; Indicates temperature parameter; This represents a smoothing parameter to prevent fractions with zero denominators and undefined logarithms.

[0089] Cross-modal projection matching loss from image to text Cross-modal projection matching loss from text to image Adding them together, we get the cross-modal projection matching loss:

[0090]

[0091] The cross-modal projection matching loss This is used to simultaneously constrain the projection consistency of image features onto the text feature space and the projection consistency of text features onto the image feature space, thereby improving the bidirectional alignment capability between image modalities and text modalities.

[0092] Among them, in the total loss function The loss can be taken as the cross-modal projection matching loss mentioned above. ;

[0093] Construct the identity loss function:

[0094] ;

[0095] in, This represents the parameter matrix of the identity classifier. Represents the normalization function. Indicates the first Image feature representation corresponding to each person's image;

[0096] Constructing diversity loss :

[0097]

[0098] in, and They represent the first The and the first Image features, and They represent the first The and the first Text features, Represents the L2 norm;

[0099] Construct scale-consistency regularization;

[0100] ;

[0101] in , For two-wheel fusion output, For the final matching score, For linear projection, corr( ) represents the correlation coefficient function. and These are the weighting coefficients;

[0102] Total loss function:

[0103] .

[0104] Furthermore, the loss function optimization method in step S420 of the present invention includes:

[0105] S421. Introduce scale consistency weight constraints;

[0106] Total loss function:

[0107]

[0108] The weights are redefined using parameterized mapping. Using interval Sigmoid mapping:

[0109]

[0110] For the remaining weights ( Softplus is used to ensure positive values, and normalization is performed to stabilize the scale.

[0111]

[0112]

[0113] in, Indicates the first During the first round of training Learnable parameterized variables corresponding to each loss term; This represents the intermediate weight variables after Softplus transformation; Softplus( () represents a smooth activation function used to ensure that the output is positive;

[0114] This ensures:

[0115]

[0116] The initial weight ratio is set as follows:

[0117]

[0118] And by The corresponding initial value can be obtained by reverse calculation;

[0119] S422. Construct a two-layer optimization framework

[0120] After the total loss is determined, the training process is divided into inner layer parameter update and outer layer weight update.

[0121] (1) Inner layer: Fixed weights, updated network parameters;

[0122] Let the network parameters be In the training set The next step is to descend:

[0123]

[0124] in:

[0125]

[0126] in, Represents the training set, Represents the validation set. This represents the learning rate used to update the inner parameters. This represents the learning rate used to update the outer weights. This indicates the number of inner gradient descent steps;

[0127] (2) Outer layer: fixed The update path involves performing validation set-oriented updates on the weights within the validation set. Above calculation:

[0128]

[0129] And perform meta-gradient updates on the weights:

[0130]

[0131] because pass Parameterization, specifically in the form of:

[0132]

[0133] S423. Introduce gradient consistency-driven adaptive weight enhancement terms;

[0134] After obtaining the updated network parameters and adaptive weights, the consistency of the gradients of each loss term with respect to the main objective is introduced as an auxiliary signal.

[0135] Define the gradient of each sub-loss with respect to the parameters on the training set:

[0136]

[0137] Define the main optimization direction and take the weighted total gradient:

[0138]

[0139] Define gradient consistency score:

[0140]

[0141] To give higher weight to the loss in the main alignment direction and suppress the loss in the conflict direction, a regularization term for the weights is constructed:

[0142]

[0143] The final verification target during the outer layer update becomes:

[0144]

[0145] in, Indicates the gradient consistency threshold. The weight coefficients represent the gradient consistency regularization term. This represents the weight coefficient of the entropy regularization term, which is used to prevent the weights from collapsing into a single loss term.

[0146] S424. Introduce loss scale normalization to avoid different dimensions dominating training.

[0147] After obtaining the enhanced adaptive weights, a sliding statistical normalization is introduced for each loss term to ensure that the weight update of the total loss function, which is composed of differences in the magnitude of different losses, is not distorted.

[0148]

[0149] and use Alternative Sum the results using the total loss function formula.

[0150] In summary, the method of this invention achieves dynamic weight adjustment during multi-loss training through an adaptive weighting strategy and gradient consistency regularization. The system can automatically adjust weights based on the gradient direction of different loss terms during training and the performance on the validation set, ensuring model stability in the early stages of training and in complex scenarios. This adaptive capability is not only reflected in weight updates but also covers multi-dimensional objectives such as text-image feature matching, projection, identity recognition, and scale consistency, achieving end-to-end adaptive optimization. Attached Figure Description

[0151] Figure 1 This is a flowchart of the method of the present invention;

[0152] Figure 2 This is a diagram of the multi-scale information interaction network structure of the present invention. Detailed Implementation

[0153] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0154] like Figure 1 As shown, the text image re-identification method based on phrase-level masking and a large language model described in this embodiment includes the following steps:

[0155] S100. Collect paired text descriptions and person images and perform unified preprocessing to obtain a person re-identification image dataset.

[0156] S200. Construct a large language model based on the person re-identification dataset, perform language detection on the user query text, judge the quality of the detection results, and generate normalized text for the input text that passes the quality judgment.

[0157] S300: Construct a cross-modal feature joint network based on a multi-interaction attention mechanism to obtain a target person image that is fused with normalized text image features and text features;

[0158] S400: A joint cross-modal projection correction model is used to train and optimize the joint network of cross-modal features to obtain the optimal multi-scale information interaction model.

[0159] S500. Based on the optimal multi-scale information interaction model, repeat steps S100 to S300 to perform secondary image matching on the input text to be detected and output the corresponding target person image.

[0160] The following provides a detailed explanation of each step:

[0161] S100. Collect paired text descriptions and person images and perform unified preprocessing to obtain a person re-identification image dataset.

[0162] S110: Collect images of several individuals in different scenes and viewpoints using cameras of different specifications;

[0163] Images of several individuals were acquired using cameras of different specifications, each in different scenes and viewpoints, to obtain a dataset of images for person re-identification. ,in, This represents the nth image of a person. Image dataset for re-identifying people The total number of images of people in the text.

[0164] S120. Generate a person re-identification image dataset I based on person images, and text descriptions corresponding to the person images in I;

[0165] Generate a dataset of images for re-identifying people. The text dataset is obtained by providing text descriptions corresponding to images of people. .

[0166] in, Represents the nth image of a person. The corresponding text description.

[0167] In this embodiment, the text-image people dataset is CUHK-PEDES, which contains 40,206 images and 80,412 text descriptions for 13,003 identities. Following the data segmentation method below, the training set consists of 11,003 identities, 34,054 images, and 68,108 text descriptions. The validation set contains 3,078 images and 6,156 text descriptions, and the test set contains 3,074 images and 6,148 text descriptions.

[0168] S200. Construct a large language model based on the person re-identification dataset, perform language detection on the user query text, judge the quality of the detection results, and generate normalized text for the input text that passes the quality judgment.

[0169] When the input text is received First, language detection is performed, and based on the detection results, it is decided whether to call a large language model for subsequent processing.

[0170] S210, Input text Perform language testing;

[0171] The specific steps are as follows:

[0172] S211, Input text Preprocessing is performed, and the preprocessing results are input into the large language model mBERT for language decision-making;

[0173] The large language model mBERT converts the preprocessed results into fixed-dimensional word vector representations. It then analyzes the input text using contextual information and outputs the probability of each language.

[0174] Output probability vector ,in, It represents the probability that the input text is in Chinese. This represents the probability that the input text is in English. Based on these probabilities, the system determines which language the input text belongs to.

[0175] S212. Based on the language discrimination results, determine the subsequent processing decision;

[0176] Define indicator functions The routing indicator used to select Chinese or English:

[0177]

[0178] in, This is an indicator function; it returns 1 if the condition is true, and 0 otherwise. That is, when the language discrimination result is Chinese, ,otherwise .

[0179] S213. Introduce a threshold to address situations where subsequent processing decisions are uncertain.

[0180] To address situations involving mixed or ambiguous language, a threshold is introduced. .when or When the difference is less than this threshold, a special processing step is initiated, where manual intervention is performed or the default language is selected for processing.

[0181] The specific discrimination mechanism is as follows:

[0182]

[0183] Otherwise, proceed to the next processing step.

[0184] The subsequent processing procedure is as follows:

[0185] A glossary and hints were constructed based on the Person Re-identification Image Dataset I;

[0186] Extracting text descriptions or annotations for each image in a person re-identification image dataset. As a preliminary terminology source. Organize the preliminary terminology sources by category: color category. Clothing ; Items carried Posture category .

[0187] Establish a domain terminology list Define synonymous normalization mappings. :

[0188] ;

[0189] in, This refers to word vector cosine similarity or edit distance similarity. Threshold (recommended) The construction includes: further constructing a hint P, which is used to constrain the large language model to retain key entities and prohibit the addition or deletion of meanings during the generation process, and to unify related terms into the aforementioned domain terminology table S.

[0190] S220, Call the large language model to process the input text The system performs autoregressive generation and incorporates terminology normalization and length control mechanisms to process the input text. Standardization;

[0191] To ensure the accuracy and consistency of the translation, while improving the translation quality, the system further considers domain-specific context and requirements, and ensures that the generated English translation conforms to domain standards by weighted correction of logits and strengthening semantic constraints, thereby optimizing the naturalness and accuracy of the translation.

[0192] The autoregressive generation method is as follows:

[0193] Large language model autoregressive generation

[0194] During the translation process, a large language model is first invoked for autoregressive generation;

[0195] Large language models analyze input Chinese text. The process is performed to calculate the unnormalized logit value. :

[0196]

[0197] In the formula, ,in This is the size of the English vocabulary; the logit value reflects the model's predicted probability for each possible word.

[0198] The terminology normalization and length control mechanisms include:

[0199] S221. Strengthen terminology and length constraints;

[0200] The system further performs weighted corrections on logits to enhance terminology standardization and translation length consistency during the generation process.

[0201] By introducing a domain terminology table and length control items The generated logits are adjusted to ensure that the generated text not only conforms to domain terminology requirements but also has a reasonable length. The adjusted logit value is... for:

[0202]

[0203] in: Glossary The indicator function for the vocabulary ensures that the generated vocabulary conforms to the domain terminology requirements; For length shaping, it represents the length generated. Exceeding the specified range and These represent the minimum and maximum length thresholds for generating text, respectively. If the value is negative, a penalty is applied; otherwise, the value is 0, which is used to control the length of the translation result. and These are weighting parameters used to balance the effects of terms and length, and it is recommended to set them to [value missing]. , .

[0204] S222, Temperature and Nuclear Sampling;

[0205] To generate more natural and diverse translation results, the system employs temperature softening and kernel sampling (top-p) mechanisms to select words.

[0206] Temperature softening generates smoothness, while kernel sampling limits the generated vocabulary set to ensure diversity and accuracy.

[0207]

[0208] Among them: Cat( ) represents the category distribution sampling function, Indicates the first The tokens obtained from the generation steps This is a temperature parameter used to control the smoothness of the model output; a recommended range is... ; The smallest subset of the vocabulary is selected using a kernel sampling method, such that its cumulative probability is greater than or equal to that of the set of words. To ensure the diversity and accuracy of the generated results, it is recommended that... .

[0209] S223, Domain Adaptability and Contextual Understanding;

[0210] To account for the impact of contextual information on translation, the system introduces a context adjustment factor. To dynamically adjust the translation results. In each generation step, the system incorporates the current context information. Combined with the model's output logits, the selection of each word is adjusted:

[0211]

[0212] in: Indicates the first The current context information corresponding to each generation step The context adjustment factor reflects the impact of the current context on vocabulary translation. This factor is generated through context analysis and reasoning to optimize vocabulary selection. The weights adjusted for context are recommended to be set to [value]. .

[0213] S224, Terminology Normalization and Generation Process;

[0214] For each generated token Through synonym normalization mapping Terminology normalization is performed. This process ensures that the generated text conforms to domain standards, avoiding inconsistencies in terminology. Each token is normalized. It will meet the specific requirements of the field and ensure the accuracy of the translation.

[0215]

[0216] in, It is based on the domain terminology table A mapping function for synonym normalization ensures that the generated text conforms to domain specifications, especially when dealing with specific terms, to avoid inconsistencies in terminology.

[0217] S230. A comprehensive quality assessment mechanism is constructed by combining multiple dimensions such as grammatical correctness, text fluency, and entity coverage to judge the translation quality and backtrack the standardized input text.

[0218] To ensure that the generated translated text meets domain requirements and conforms to natural language standards, the system employs a comprehensive quality assessment mechanism. This mechanism not only evaluates the accuracy of the translation but also considers multiple dimensions such as grammatical correctness, text fluency, and entity coverage.

[0219] By calculating the overall quality score The system can judge the quality of the generated translation and decide whether to implement a rollback mechanism based on the quality assessment results to ensure that the translation quality meets the standards.

[0220] The methods for constructing a comprehensive quality assessment mechanism include:

[0221] S231, Calculation of mass fraction;

[0222] quality score It integrates three main evaluation metrics: entity coverage, perplexity regularization, and semantic consistency. The formula is as follows:

[0223]

[0224] in: These are weighting coefficients used to balance the influence of different quality assessment indicators; This represents entity coverage, measuring whether all important entities in the source text are preserved during translation. It is the level of confusion, which measures the fluency of the translated language; Semantic consistency measures the semantic consistency between the translation and the source text.

[0225] S232, Entity coverage rate;

[0226] Entity coverage is calculated by comparing the entity sets extracted from the source text and the translated text. The specific formula is as follows:

[0227]

[0228] in: This represents an entity extraction function, used to extract a set of entities from text. This represents a bilingual entity alignment mapping function, used to map Chinese entities to English entity spaces. A collection of entities in the source text; For the set of entities in the translated text; Indicates bilingual mapping Aligned set of entities.

[0229] S233, Perplexity Regularization;

[0230] Perplexity score (ppl) is used to measure the fluency of the generated translation. The calculation formula is as follows:

[0231]

[0232] in: Based on language model Each word calculated The probability of language model An autoregressive language model for calculating the fluency of translated text, which outputs the conditional probability distribution of the next lexical word based on the generated lexical sequence yt; It is the length of the translated text. Indicates the first The word sequence that was generated before the previous generation step This represents the language model used to compute fluency. This indicates the preset maximum perplexity threshold.

[0233] If the perplexity exceeds the predetermined maximum value Perform regularization:

[0234]

[0235] S234, Semantic consistency;

[0236] Semantic consistency is assessed by calculating the similarity between the source and translated texts in the semantic space. The formula is as follows:

[0237]

[0238] in: This is the sentence vector for the English translation; This is the encoding function for representing English text after projecting Chinese sentence vectors onto the English space. This represents the Chinese text encoding function. This represents a linear projection matrix that projects Chinese semantic vectors onto the English semantic space.

[0239] S235, rollback mechanism and anomaly detection;

[0240] When mass fraction Below the preset threshold The system will revert to a rule-based or dictionary-based translation method for correction. The revert conditions are as follows:

[0241]

[0242] Meanwhile, an anomaly detection mechanism is introduced to automatically detect potential errors in translation (such as inconsistent terminology, grammatical problems, etc.) and trigger a rollback mechanism for correction.

[0243] S240. Normalize the input text that has passed the quality assessment to generate the final normalized English text. And transmit it to the Transformer text encoder;

[0244] After completing the quality assessment and rollback in step S240, the system will verify the qualified final normalized English text. Input to the Transformer text encoder for further processing.

[0245] At this point, the selected English text can be the final normalized English text of the generated translation. Or the translated text after rollback correction :

[0246]

[0247] S300: Construct a cross-modal feature joint network based on a multi-interaction attention mechanism to obtain a target person image that is fused with normalized text image features and text features;

[0248] A novel cross-modal feature joint network is proposed, which combines several new modules, including dual-path extraction of image and text features, an enhanced foreground discriminator module, a phrase-level adaptive masking mechanism, cross-modal self-supervised learning (CMSSL), and an adaptive cross-modal cross-attention mechanism (ACCAM). The detailed steps of each module are shown below:

[0249] S310. Construct a dual-path network model for image and text feature extraction;

[0250] Image and text feature extraction is a crucial step in cross-modal learning. To fully utilize the fine-grained information in images, we employ a dual-path network model to extract features from both images and text separately.

[0251] Image feature extraction:

[0252] Image feature extraction utilizes convolutional neural networks (CNNs) and visual Transformers (ViT) models. In this process, the image is divided into multiple fixed-size blocks, and global and local visual features are extracted through ViT's multi-layer self-attention mechanism.

[0253]

[0254] in, Represents the global features of the image. Indicates the first Local features of a patch It is the first input Images. Visual features are extracted from the images using a ViT encoder, where each image... It is divided into several patches, each patch is represented as These local features are then compared with global features. To combine.

[0255] Text feature extraction:

[0256] Text feature extraction uses a Transformer encoder to encode the input text, thereby extracting global and local features of the text.

[0257]

[0258] in, It is a global feature of the text. It is the first in the text Local features of a word For the input of the first Each text is divided into multiple words or phrases, and global features are calculated for each word or phrase. and local features of each word .

[0259] S320. Construct a multi-scale foreground enhancement and phrase-level adaptive masking mechanism to enhance image and text features.

[0260] To enhance the model's ability to perceive local details in images, we designed a multi-scale foreground enhancement and a phrase-level adaptive masking mechanism.

[0261] S321, Multi-scale Foreground Enhancement Discriminator Mechanism

[0262] The foreground enhancement discriminator mechanism enhances the foreground portion of an image and suppresses background noise through two key steps: spatial guided localization and channel denoising.

[0263] Spatial guided positioning: The spatial weight coefficient of each position is calculated through max pooling and average pooling operations to highlight the foreground area of ​​the figure.

[0264]

[0265] in, It is the Sigmoid activation function. For convolution operations, For splicing operations, and These represent the max pooling and average pooling operations for image features, respectively. Indicates the first Spatial guided weight coefficients corresponding to each image feature.

[0266] Channel denoising: Feature vectors are generated through global pooling, and channel weight coefficients are calculated to further remove noise and enhance image details.

[0267]

[0268] in, The channel denoising weight coefficients represent the nth image feature; This represents the input feature map of the nth image; Indicates the feature map Perform global pooling operations in the spatial dimension; This represents the channel statistics vector obtained from global max pooling; This represents the channel statistics vector obtained from global average pooling; This represents a multilayer perceptron; This represents the Sigmoid activation function.

[0269] The enhanced image features are obtained by multiplying the image features element-wise with the corresponding weight coefficients:

[0270]

[0271] S322, Phrase-level adaptive language masking mechanism:

[0272] An adaptive mask rate is generated for each text phrase based on its importance. The phrase mask rate evaluates phrase weights based on multiple dimensions, including entropy, gradient, and cross-attention.

[0273]

[0274] in, It is a phrase The mask rate is normalized using Softmax. Indicates the first A text phrase; Entropy ) indicates a phrase Information entropy score; Gradient ( ) indicates a phrase Gradient sensitivity scoring of target loss; CrossAttention ( ) indicates a phrase Response intensity in cross-modal attention; The maximum protection mask threshold for key phrases; Mask represents a mask vector composed of the mask rates of each phrase; This function represents the process of masking the input text based on the mask vector. It applies maximum mask protection to key phrases such as color and clothing.

[0275]

[0276] Masks are generated using Gumbel-Softmax or Bernoulli sampling techniques, and the text is masked during the training phase, while the full text is used during the inference phase.

[0277]

[0278] S323. Combine cross-modal self-supervised learning to further align and refine image features and text features;

[0279] Cross-modal self-supervised learning (CMSSL) is introduced to further improve the alignment accuracy of image and text features. This method automatically mines the potential correlations between images and text in an unsupervised manner, thereby enhancing feature alignment accuracy.

[0280] Bidirectional contrastive learning loss:

[0281] To maximize the similarity between image and text features and push away mismatched feature pairs, we introduce a bidirectional contrastive learning loss:

[0282]

[0283] in, This indicates the number of samples in a training batch. and Representing image and text features respectively. This refers to the temperature parameter.

[0284] Cross-modal self-supervised mapping:

[0285] We learn the mapping relationship between images and text through self-supervised learning, and further optimize the model using pseudo-labels:

[0286]

[0287] in, and These are functions for mapping images to text and text to images.

[0288] Dynamic self-supervised loss:

[0289] We introduce a dynamic self-supervised loss to further optimize the alignment of image and text features:

[0290]

[0291] In this way, we optimize the spatial distribution of images and text, enabling them to be effectively aligned within the same space.

[0292] S330. Image and text feature fusion based on cross-modal joint learning network with adaptive multimodal feature optimization;

[0293] S331, Adaptive Multi-Head Attention Mechanism and Optimized Convolutional Networks

[0294] This module optimizes the fusion of image and text features through adaptive convolution and multi-head attention mechanisms, and enhances semantic understanding capabilities through a self-learning attention strategy.

[0295] Image and text queries, key-value vectors:

[0296]

[0297] in, , , These represent the query vector, key vector, and value vector of the nth image feature under the mth attention head, respectively. , , These represent the query vector, key vector, and value vector of the nth text feature under the mth attention head, respectively. This represents the input feature representation of the nth image; This represents the input feature representation of the nth text; , , These represent the query projection parameter matrix, key projection parameter matrix, and value projection parameter matrix corresponding to the m-th attention head, respectively; n represents the input sample number; and m represents the attention head number in the multi-head attention mechanism.

[0298] Adaptive convolutional fusion: Improves the fusion effect by optimizing the interaction between image and text features through convolutional layers.

[0299] S332, Cross-modal information propagation and multi-dimensional optimization;

[0300] This module enhances the complementarity of images and text and achieves deep fusion through cross-attention mechanism and multi-dimensional information flow.

[0301] Cross-attention map generation:

[0302]

[0303] in, This represents the function for calculating cross-attention.

[0304] Final fusion: Further optimization of image and text feature representations through Transformer layers:

[0305]

[0306] Ultimately, the fused features The input is then fed into a subsequent Transformer feedforward network for processing, yielding the final image and text representations.

[0307]

[0308] Among them, Fuse( ) represents the fusion function of image features and text features.

[0309] S400: A joint cross-modal projection correction model is used to train and optimize the joint network of cross-modal features to obtain the optimal multi-scale information interaction model.

[0310] like Figure 2 As shown, the method for constructing a joint cross-modal projection correction model includes:

[0311] S410. Construct the matching loss function, identity loss function, and diversity loss function respectively, and combine them with the scale consistency regularization to construct the total loss function.

[0312] Constructing a cross-modal projection matching loss from text to image :

[0313]

[0314] Where N represents the number of samples in a training batch. Indicates the first The final text representation obtained after a text description passes through a multi-scale information interaction network; Indicates the first The standardized representation of image features corresponding to the image of Zhang; Representing text features Image features The predicted probability of a matching pair; Representing text features Image features The true probability of a match; Indicates whether the two are true matching tags of a matching pair; Indicates temperature parameter; This represents a smoothing parameter to prevent the denominator from being zero or the logarithm from being undefined.

[0315] Cross-modal projection matching loss from image to text Cross-modal projection matching loss from text to image Adding them together, we get the cross-modal projection matching loss:

[0316]

[0317] The cross-modal projection matching loss This is used to simultaneously constrain the consistency of projection of image features onto the text feature space and the consistency of projection of text features onto the image feature space, thereby improving the bidirectional alignment capability between image modalities and text modalities. Among them, the total loss function includes... The loss can be taken as the cross-modal projection matching loss mentioned above. .

[0318] Construct the identity loss function:

[0319] ;

[0320] in, This represents the parameter matrix of the identity classifier. Represents the normalization function. Indicates the first Image feature representation corresponding to each person's image.

[0321] Constructing diversity loss :

[0322]

[0323] in, and They represent the first The and the first Image features, and They represent the first The and the first Text features, This represents the L2 norm.

[0324] Construct scale-consistency regularization;

[0325] ;

[0326] in , For two-wheel fusion output, For the final matching score, For linear projection, corr( ) represents the correlation coefficient function. and These are the weighting coefficients.

[0327] Total loss function:

[0328]

[0329] The process of obtaining the optimal multi-scale information interaction model is as follows:

[0330] S420. An adaptive weighting strategy and regularization are introduced to optimize the total loss function;

[0331] Total loss function optimization methods include:

[0332] S421 introduces a scale consistency weight constraint;

[0333] Total loss function:

[0334]

[0335] To ensure that the weights always satisfy nonnegativity and interval constraints during training, a parameterized mapping is used to redefine the weights.

[0336] right Using interval Sigmoid mapping:

[0337]

[0338] For the remaining weights ( To avoid collapse or excessive fluctuations, Softplus is used to ensure positive values, and normalization is performed to stabilize the scale.

[0339]

[0340]

[0341] in, Indicates the first During the first round of training Learnable parameterized variables corresponding to each loss term; This represents the intermediate weight variables after Softplus transformation; Softplus( ) represents a smooth activation function used to ensure that the output is positive.

[0342] This ensures:

[0343]

[0344] The initial weight ratio can be set as follows:

[0345]

[0346] And by The corresponding initial value can be obtained by reverse calculation.

[0347] S422. Construct a two-layer optimization framework

[0348] After the total loss is determined, in order to realize the meta-learning idea of ​​"dynamically adjusting based on gradient changes and validation set performance", the training process is divided into inner layer parameter update and outer layer weight update.

[0349] (1) Inner layer: fixed weights, update network parameters

[0350] Let the network parameters be In the training set Perform gradient descent in one step (or K steps):

[0351]

[0352] in:

[0353]

[0354] in, Represents the training set, Represents the validation set. This represents the learning rate used to update the inner parameters. This represents the learning rate used to update the outer weights. This indicates the number of gradient descent steps in the inner layer.

[0355] (2) Outer layer: fixed The update path involves performing validation set-guided updates on the weights.

[0356] In the validation set Above calculation:

[0357]

[0358] And perform meta-gradient updates on the weights:

[0359]

[0360] because pass Parameterization (equations (18)(19)) results in a more stable actual update form:

[0361]

[0362] This naturally satisfies constraints such as weight interval / normalization, and avoids [the following issues]. Out of bounds or negative values.

[0363] S423. Introduce gradient consistency-driven adaptive weight enhancement terms.

[0364] After obtaining the updated network parameters and adaptive weights, to mitigate the potential instability of updating weights using the validation set loss in the early stages of the total loss function, the "consistency of gradients of each loss term with respect to the main objective" is introduced as an auxiliary signal. The gradients of each sub-loss with respect to the parameters on the training set are defined as follows:

[0365]

[0366] The "main optimization direction" can be defined as the weighted total gradient:

[0367]

[0368] Define gradient consistency score (cosine similarity):

[0369]

[0370] Based on this, a "stable regularization" term is constructed for the weights, giving higher weight to the loss in the alignment of the principal direction and suppressing the loss in the conflict direction:

[0371]

[0372] The final verification target during the outer layer update becomes:

[0373]

[0374] in, Indicates the gradient consistency threshold. The weight coefficients represent the gradient consistency regularization term. This represents the weight coefficient of the entropy regularization term, which is used to prevent the weights from collapsing into a single loss term.

[0375] S424. Introduce loss scale normalization to avoid different dimensions dominating training.

[0376] After obtaining the enhanced adaptive weights, to avoid distortion in the total loss function weight update caused by differences in the magnitude of different losses, a sliding statistical normalization is introduced for each loss term:

[0377]

[0378] in, This represents the momentum coefficient in sliding statistics.

[0379] and use Alternative The total loss function formula is used to sum the results, making the weight learning process more "fair" and more interpretable.

[0380] After loss scale normalization and adaptive weighted optimization, the final total loss function used for network training is expressed as:

[0381]

[0382] Right now:

[0383]

[0384] in, , , and These represent the cross-modal projection matching loss, identity loss, diversity loss, and scale consistency loss after sliding statistical normalization, respectively. Indicates the first During the first round of training The adaptive weights corresponding to the term loss, and satisfying , , Each normalized loss term is defined as follows:

[0385]

[0386] in, and They represent the first The loss was in the first Sliding mean and sliding variance in round training To prevent smoothing parameters with a denominator of zero.

[0387] Within the two-layer optimization framework, the outer layer of weight parameters is first updated based on validation set performance, gradient consistency regularization, and entropy regularization to obtain the optimal weight parameters. And obtain the optimal adaptive weight vector through parameterized mapping. Then, in the optimal weight vector... Under constraints, the network parameters are optimized to obtain the optimal network parameters. :

[0388]

[0389] Thus, the optimal multi-scale information interaction model is obtained:

[0390]

[0391] in, This represents the optimal multi-scale information interaction model obtained after training and optimization using a joint cross-modal projection correction model. This represents the corresponding optimal network parameters.

[0392] Furthermore, the optimal multi-scale information interaction model This is used to output the final optimized image feature representation and text feature representation for subsequent cross-modal matching and person re-identification tasks.

[0393] S500. Based on the optimal multi-scale information interaction model, repeat steps S100 to S300 to perform secondary image matching on the input text to be detected and output the corresponding target person image.

[0394] In summary, the method of this invention achieves dynamic weight adjustment during multi-loss training through an adaptive weighting strategy and gradient consistency regularization. The system can automatically adjust weights based on the gradient direction of different loss terms during training and the performance on the validation set, ensuring model stability in the early stages of training and in complex scenarios. This adaptive capability is not only reflected in weight updates but also covers multi-dimensional objectives such as text-image feature matching, projection, identity recognition, and scale consistency, achieving end-to-end adaptive optimization.

[0395] Through language detection and thresholding strategies, the system can perform real-time language determination and processing on input text, automatically selecting the appropriate processing path. Although the existing embodiments are designed for Chinese and English, the technical architecture supports expansion to other languages, enabling multilingual text querying and image matching adaptation, thus improving the system's application capabilities in internationalized and multilingual environments.

[0396] By utilizing an optimal multi-scale information interaction model for secondary image matching, and comprehensively considering local and global features as well as multi-level text semantics, refined person re-identification is achieved. Effective feature alignment is performed on images of different scales and in different scenes, improving recognition accuracy across scenes and cameras.

[0397] Gradient consistency regularization ensures stable weight direction and avoids weight fluctuations caused by validation set noise in the early stages of training. Loss normalization ensures that losses of different dimensions are fairly reflected in weight allocation, enhancing the interpretability and controllability of the training process.

[0398] This invention combines advanced techniques such as two-layer optimization, meta-learning strategies, multi-loss adaptive weights, gradient consistency regularization, and loss normalization to achieve end-to-end person re-identification in complex scenarios. It supports large-scale crowd data, cross-camera, and multi-language queries, is engineering-feasible, and can be applied to high-tech scenarios such as security monitoring, intelligent video analytics, and identity verification.

[0399] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0400] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0401] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the text image re-identification methods based on phrase-level masking and large language models in the above embodiments.

[0402] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0403] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0404] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0405] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0406] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text image re-identification method based on phrase-level masking and a large language model, characterized in that, Includes the following steps: S100. Collect paired text descriptions and person images and perform unified preprocessing to obtain a person re-identification image dataset. S200. Construct a large language model based on the person re-identification dataset, perform language detection on the user query text, judge the quality of the detection results, and generate normalized text for the input text that passes the quality judgment. S300: Construct a cross-modal feature joint network based on a multi-interaction attention mechanism to obtain a target person image that is fused with normalized text image features and text features; S400: A joint cross-modal projection correction model is used to train and optimize the joint network of cross-modal features to obtain the optimal multi-scale information interaction model. S500. Based on the optimal multi-scale information interaction model, repeat steps S100 to S300 to perform secondary image matching on the input text to be detected and output the corresponding target person image. The language detection method for the user query text in step S200 includes: S210, Input text Perform language testing; S220, Call the large language model to process the input text The system performs autoregressive generation and incorporates terminology normalization and length control mechanisms to process the input text. Standardization; S230. A comprehensive quality assessment mechanism is constructed by combining multiple dimensions such as grammatical correctness, text fluency, and entity coverage to judge the translation quality and backtrack the standardized input text. S240. Normalize the input text that has passed the quality assessment and generate text. And transmit it to the Transformer text encoder; The language detection method in step S210 includes: S211, Input text Preprocessing is performed, and the preprocessing results are input into the large language model mBERT for language decision-making; The large language model mBERT converts the preprocessed results into fixed-dimensional word vector representations. It analyzes the input text using contextual information and outputs the probability of each language. Output probability vector ,in, It represents the probability that the input text is in Chinese. It represents the probability that the input text is in English. S212. Based on the language discrimination results, determine the subsequent processing decision; Define characteristic function The routing indicator used to select Chinese or English: in, This is an indicator function that returns 1 if the condition is true and 0 otherwise. That is, when the language discrimination result is Chinese... ,otherwise ; S213. Introduce a threshold to address situations where subsequent processing decisions are uncertain. Introduce a threshold To handle situations involving mixed or ambiguous language; when or When the difference between them is less than the threshold, a preset processing step is initiated. The specific discrimination mechanism is as follows: when When, it enters the ambiguity handling process; when At that time, according to ; Otherwise, proceed to the next processing step; The subsequent processing steps include: A glossary and hints were constructed based on the Person Re-identification Image Dataset I; Text descriptions and annotations are extracted from each image in the person re-identification image dataset. As a preliminary source of terminology; Organize the initial terminology sources by category: color category Clothing ; Items carried Posture category , Establish a domain terminology list Define synonymous normalization mappings. : ； in, Represents a collection of English words. Cosine similarity of word vectors The threshold is defined as follows: the construction includes: further constructing a prompt P, which is used to constrain the large language model to retain key entities and prohibit the addition or deletion of meanings during the generation process, and to unify related terms into the aforementioned domain terminology table S; The method for constructing the joint cross-modal projection correction model in step S400 includes: S410. Construct the matching loss function, identity loss function, and diversity loss function respectively, and combine them with the scale consistency regularization to construct the total loss function. S420. An adaptive weighting strategy and regularization are introduced to optimize the total loss function; The total loss function method in step S410 includes: Constructing a cross-modal projection matching loss from text to image : Where N represents the number of samples in a training batch. Indicates the first The final text representation obtained after a text description passes through a multi-scale information interaction network; Indicates the first The standardized representation of image features corresponding to the image of Zhang; Representing text features Image features The predicted probability of a matching pair; Representing text features Image features The true probability of a match; Indicates whether the two are true matching tags of a matching pair; Indicates temperature parameter; This represents a smoothing parameter to prevent fractions with zero denominators and undefined logarithms. Cross-modal projection matching loss from image to text Cross-modal projection matching loss from text to image Adding them together, we get the cross-modal projection matching loss: The cross-modal projection matching loss This is used to simultaneously constrain the projection consistency of image features onto the text feature space and the projection consistency of text features onto the image feature space, thereby improving the bidirectional alignment capability between image modalities and text modalities. Among them, in the total loss function The loss can be taken as the cross-modal projection matching loss mentioned above. ; Construct the identity loss function: ； in, This represents the parameter matrix of the identity classifier. Represents the normalization function. Indicates the first Image feature representation corresponding to each person's image; Constructing diversity loss : in, and They represent the first The and the first Image features, and They represent the first The and the first Text features, Represents the L2 norm; Construct scale-consistency regularization; ； in , For two-wheel fusion output, For the final matching score, For linear projection, corr( ) represents the correlation coefficient function. and These are the weighting coefficients; Total loss function: 。 2. The text image re-identification method based on phrase-level masking and a large language model according to claim 1, characterized in that, In step S200, the large language model is called to process the input text. Autoregressive generation methods include: During the translation process, a large language model is first invoked for autoregressive generation; Large language models analyze input Chinese text. The process is performed to calculate the unnormalized logit value. : In the formula, ,in This is the size of the English vocabulary; the logit value reflects the model's predicted probability for each possible word generated. The terminology normalization and length control mechanisms include: S221. Strengthen terminology and length constraints; The system further performs weighted correction on logits to enhance the standardization of terminology during the generation process and the consistency of translation length; By introducing a domain terminology table and length control items The generated logits are adjusted to ensure that the generated text not only conforms to domain terminology requirements but also has a reasonable length. The adjusted logit value is... for: in: Indicates and A vector of all 1s with the same dimension. Glossary of field terms The indicator function for the vocabulary ensures that the generated vocabulary conforms to the domain terminology requirements; and These represent the minimum and maximum length thresholds for the generated text, respectively. For length shaping, it represents the length generated. Exceeding the specified range If the value is negative, a penalty is applied; otherwise, the value is 0, which is used to control the length of the translation result. and These are weighting parameters used to balance the effects of terms and length, and it is recommended to set them to [value missing]. , ; S222, Temperature and Nuclear Sampling; The system uses temperature softening and kernel sampling mechanisms to select words. Temperature softening generates smoothness, while kernel sampling limits the generated word set. Among them: Cat( ) represents the category distribution sampling function, and represents the th The tokens obtained from the generation steps This is a temperature parameter used to control the smoothness of the model output; a recommended range is... ; The smallest subset of the vocabulary is selected using a kernel sampling method, such that the cumulative probability of the smallest subset is greater than or equal to 1. To ensure the diversity and accuracy of the generated results, it is recommended that... ; S223, Domain Adaptability and Contextual Understanding; The system introduces a context adjustment factor. To dynamically adjust the translation results, the system incorporates the current context information in each generation step. Combined with the model's output logits, the selection of each word is adjusted: in: Indicates the first The current context information corresponding to each generation step Indicates the first The tokens obtained from the generation steps The context adjustment factor reflects the impact of the current context on word translation; it is generated through the analysis and reasoning of the context to optimize word selection. The weights adjusted for context are recommended to be set to [value]. ; S224, Terminology Normalization and Generation Process; For each generated token Through synonym normalization mapping Perform terminology normalization; each token after normalization... It meets the pre-defined requirements of the domain. in, It is based on the domain terminology table A mapping function for synonym normalization is used to ensure that the generated text conforms to domain specifications.

3. The text image re-identification method based on phrase-level masking and a large language model according to claim 2, characterized in that, The method for constructing the comprehensive quality assessment mechanism in step S230 includes: Mass fraction calculation: quality score The evaluation combines three metrics: entity coverage, perplexity regularization, and semantic consistency, as shown in the following formula: in: These are weighting coefficients used to balance the influence of different quality assessment indicators; This represents entity coverage, measuring whether all important entities in the source text are preserved during translation. It is the level of confusion, which measures the fluency of the translated language; Semantic consistency measures the semantic consistency between the translation and the source text. Entity coverage: Entity coverage is calculated by comparing the entity sets extracted from the source text and the translated text, using the following formula: in: A collection of entities in the source text; For the set of entities in the translated text; Indicates bilingual mapping Aligned set of entities; Perplexity regularization: The perplexity score (ppl) is used to measure the fluency of the generated translation, and the calculation formula is as follows: in: It is the calculation of each word The probability of; It is the length of the translated text. If the perplexity exceeds the predetermined maximum value Perform regularization: Semantic consistency: Semantic consistency is assessed by calculating the similarity between the source and translated texts in the semantic space, as shown in the following formula: in: This is the sentence vector for the English translation; This represents the Chinese sentence vector projected onto the English space. Rollback mechanism and anomaly detection; When mass fraction Below the preset threshold The system will revert to a rule-based or dictionary-based translation method for correction, under the following conditions: Meanwhile, an anomaly detection mechanism is introduced to automatically detect potential errors in the translation and trigger a rollback mechanism for correction.

4. The text image re-identification method based on phrase-level masking and a large language model according to claim 1, characterized in that, The loss function optimization method in step S420 includes: S421. Introduce scale consistency weight constraints; Total loss function: The weights are redefined using parameterized mapping. Using interval Sigmoid mapping: For the remaining weights ( Softplus is used to ensure positive values, and normalization is performed to stabilize the scale. in, Indicates the first During the first round of training Learnable parameterized variables corresponding to each loss term; This represents the intermediate weight variables after Softplus transformation; Softplus( () represents a smooth activation function used to ensure that the output is positive; This ensures: The initial weight ratio is set as follows: And by The corresponding initial value can be obtained by reverse calculation; S422. Construct a two-layer optimization framework After the total loss is determined, the training process is divided into inner layer parameter update and outer layer weight update. (1) Inner layer: Fixed weights, updated network parameters; Let the network parameters be In the training set The next step is to descend: in: in, Represents the training set, Represents the validation set. This represents the learning rate used to update the inner parameters. This represents the learning rate used to update the outer weights. This indicates the number of inner gradient descent steps; (2) Outer layer: fixed The update path involves performing validation set-guided updates on the weights. In the validation set Above calculation: And perform meta-gradient updates on the weights: because pass Parameterization, specifically in the form of: S423. Introduce gradient consistency-driven adaptive weight enhancement terms; After obtaining the updated network parameters and adaptive weights, the consistency of the gradients of each loss term with respect to the main objective is introduced as an auxiliary signal. Define the gradient of each sub-loss with respect to the parameters on the training set: Define the main optimization direction and take the weighted total gradient: Define gradient consistency score: To give higher weight to the loss in the main alignment direction and suppress the loss in the conflict direction, a regularization term for the weights is constructed: The final verification target during the outer layer update becomes: in, Indicates the gradient consistency threshold. The weight coefficients of the gradient consistency regularization term are represented. This represents the weight coefficient of the entropy regularization term, which is used to prevent the weights from collapsing into a single loss term. S424. Introduce loss scale normalization to avoid different dimensions dominating training. After obtaining the enhanced adaptive weights, a sliding statistical normalization is introduced for each loss term to ensure that the weight update of the total loss function, which is composed of differences in the magnitude of different losses, is not distorted. and use Alternative Sum the results using the total loss function formula.

Images