Scene text recognition method and device based on multi-modal large language model

Abstract

The invention relates to a scene text recognition method and device based on a multi-modal large language model, and belongs to the technical field of artificial intelligence. The method comprises the steps that a first token set composed of tokens with the correlation with an input image meeting a set requirement in the text generation process is obtained, a second token set composed of tokens with the semantic stability meeting the set requirement in the text generation process is obtained, and semantic anchor paragraphs are formed according to the tokens continuously appearing in an intersection of the first token set and the second token set, recording the sampling temperature of each token position in the text generation process; according to the method, the sampling temperature of each token position in the text generation process is adjusted based on the semantic anchor paragraph, and text generation is performed again according to the adjusted sampling temperature, so that accurate recognition and semantic illusion suppression of a visual text in image text recognition are realized, and the accuracy and robustness of scene text recognition are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a scene text recognition method and device based on a multi-modal large language model. BACKGROUND

[0002] Scene Text Recognition (STR) refers to a technology of detecting and recognizing text content from natural scene images or videos. Unlike traditional document text recognition (such as scanned or printed materials), scene text often faces challenges such as complex background, diverse fonts, light changes, and distortion, and is a core research direction in the fields of computer vision and OCR (Optical Character Recognition).

[0003] In the prior art, traditional scene text processing methods usually adopt a multi-stage processing flow, including text detection, recognition, and language modeling. Although such methods perform well in specific environments, due to the decoupling and error accumulation between processing modules, their generalization ability in complex and open environments is limited.

[0004] With the development of large multi-modal models, such as LLaVA, Qwen2-VL, Mini-Monkey, and other systems, researchers have attempted to introduce them into Optical Character Recognition (OCR) related tasks, and have made significant progress in document question answering, graphical interface analysis, and general OCR frameworks. However, the current large multi-modal models still have deficiencies in the ability to recognize and understand scene text, especially in accurately recognizing image text.

[0005] Research shows that existing large multi-modal models tend to rely on semantic priors for reasoning, i.e., when recognizing image text, they do not strictly base their recognition judgments on visual content, but output results based on common semantic knowledge "guesses" (i.e., the semantic illusion phenomenon). In addition, the model's accuracy significantly decreases when processing text samples that are semantically irrelevant but visually real, indicating that its actual ability to process visual features is limited.

[0006] To address the above problems, although existing methods have improved MLLM (Multimodal Large Language Model) in visual-linguistic tasks through resolution enhancement, Token encoding optimization, and new task design (such as UReader, Ocean-OCR, GOT-OCR), most models still perform poorly when processing semantically disturbed text (i.e., text with grammatical structure but no actual semantics), with a significant decrease in recognition accuracy, exposing their strong dependence on semantics and insufficient adaptation to pure visual features.

[0007] And, the existing improvement method depends on training and structural changes: for example, additional modules, specific pre-training methods, resulting in complex actual deployment and high migration cost. For example, the current method for improving text recognition performance usually needs to add branch networks, resolution enhancement modules, visual cropping mechanisms, etc. on the original model, or rely on additional training processes and specific data set support, resulting in a cumbersome model deployment and migration process and poor universality.

[0008] Therefore, how to achieve accurate recognition of visual text and semantic illusion suppression in image text recognition without changing the original MLLM architecture is a key challenge in current technology. SUMMARY

[0009] In view of the above analysis, the embodiments of the present application aim to provide a scene text recognition method and device based on a multi-modal large language model, to solve the problem of improving the recognition accuracy of a multi-modal large language model in scene text recognition without changing the MLLM architecture.

[0010] On the one hand, the embodiments of the present application provide a scene text recognition method based on a multi-modal large language model, comprising obtaining a first token set constituted by tokens whose relevance to input images in a text generation process meets a set requirement, and obtaining a second token set constituted by tokens whose semantic stability in the text generation process meets a set requirement, forming a semantic anchor point paragraph according to tokens that continuously appear in the intersection of the first token set and the second token set, and recording the sampling temperature of each token position in the text generation process; adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph, and re-performing text generation according to the adjusted sampling temperature.

[0011] The beneficial effects of the above technical solution are as follows: without changing the original MLLM architecture, accurate recognition of visual text and semantic illusion suppression in image text recognition are achieved, and the accuracy and robustness of scene text recognition are improved.

[0012] Based on the further improvement of the above method, the first token set composed of tokens meeting the set requirements in the text generation process in relation to the input image comprises: for the token in the text generation process, extracting the cross-attention vector of the token in the cross-attention mechanism; wherein the cross-attention vector reflects the attention degree of the token to each region of the input image; calculating the cross-attention autocorrelation matrix between the cross-attention vectors of the tokens in the set time window, obtaining the related tokens whose cosine similarity of the cross-attention autocorrelation matrix is greater than the preset similarity threshold, and marking the remaining tokens after the related tokens are screened out from the token sequence output by the multi-modal large language model as the first token set.

[0013] The beneficial effect of the above further improvement scheme is that the removal of noise tokens with no high correlation with the image and attention drift in the construction process of the semantic anchor point paragraph is realized, and the accuracy of the semantic anchor point paragraph acquisition is improved.

[0014] Based on the further improvement of the above method, the second token set composed of tokens meeting the set requirements in the text generation process in terms of semantic stability comprises: for the token in the text generation process, extracting the prediction distribution of the token at each layer of the Transformer, calculating the consistency index of the prediction distribution according to the prediction distribution of the token at each layer of the Transformer, and marking the token whose consistency index of the prediction distribution meets the requirements as the second token set.

[0015] The beneficial effect of the above further improvement scheme is that the removal of tokens with unstable semantics in the construction process of the semantic anchor point paragraph is realized, and the accuracy of the semantic anchor point paragraph acquisition is further improved.

[0016] Based on the further improvement of the above method, the calculation of the consistency index of the prediction distribution according to the prediction distribution of the token at each layer of the Transformer comprises: calculating the KL divergence of the prediction distribution of the token at each two adjacent layers of the Transformer, and determining the token whose average value of the KL divergence is less than the preset threshold as the token whose consistency index of the prediction distribution meets the requirements.

[0017] The beneficial effect of the above further improvement scheme is that the accuracy of the second token set result is improved.

[0018] Based on the further improvement of the above method, the adjustment of the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph comprises: maintaining the sampling temperature of the token position or slightly adjusting the sampling temperature by a preset smaller amplitude for the token position at the semantic anchor point paragraph; and reducing the sampling temperature by a preset larger amplitude for the token position at the non-semantic anchor point paragraph.

[0019] The beneficial effect of the above further improvement scheme is to ensure natural and smooth text and rich semantics and hallucination suppression in text generation.

[0020] Based on the further improvement of the above method, before the text generation is performed again according to the adjusted sampling temperature, the method further comprises: obtaining a low-confidence region of the input image whose cross-modal matching confidence is lower than a preset confidence threshold; and reducing the sampling temperature of the token position corresponding to the low-confidence region and / or edge region of the input image.

[0021] The beneficial effect of the above further improvement scheme is to reduce the probability of generating a risky token, making the generated result more reliable.

[0022] Based on the further improvement of the above method, after the text generation is performed again according to the adjusted sampling temperature, the method further comprises: performing a text-image semantic matching evaluation on the generated text to obtain a matching score; in response to the matching score being lower than a preset score threshold, determining a token region not covered by the semantic anchor point paragraph as a low-confidence region and determining a token region covered by the semantic anchor point paragraph as a high-confidence region; further reducing the sampling temperature of the token position of the low-confidence region, rolling back the decoding state of the low-confidence region, and performing text generation on the low-confidence region again; merging the text generated by the low-confidence region again and the text of the high-confidence region generated in the last text generation process as an integrated generated text, and going to the processing step of obtaining the first token set whose relevance to the input image meets the set requirement; and in response to the matching score being greater than the preset threshold, taking the text generated according to the adjusted sampling temperature as the final output text.

[0023] The beneficial effect of the above further improvement scheme is to further improve the accuracy of text generation.

[0024] Based on a further improvement of the above method, the generated text is subjected to a graphic-text semantic matching evaluation to obtain a matching score, including: vector encoding the generated text to obtain a first vector; vector encoding the input image to obtain a second vector; and obtaining the matching score by calculating the correlation between the first vector and the second vector.

[0025] The beneficial effect of the above further improved solution is that the accuracy of the matching score is improved.

[0026] On the other hand, an embodiment of the present invention provides a scene text recognition device based on a multimodal large language model, which includes: a semantic anchor point recognition module, used to: obtain a first token set consisting of tokens whose correlation with the input image meets the set requirements during the text generation process, and obtain a second token set consisting of tokens whose semantic stability meets the set requirements during the text generation process, form a semantic anchor point paragraph based on the tokens that appear continuously in the intersection of the first token point set and the second token point set, and record the sampling temperature of each token position during the text generation process; a temperature control generation module, used to: adjust the sampling temperature of each token position during the text generation process based on the semantic anchor point paragraph, and regenerate text according to the adjusted sampling temperature.

[0027] The beneficial effects of the above technical solution are as follows: without changing the original MLLM architecture, it can achieve accurate recognition of visual text and suppression of semantic hallucinations in image text recognition, thereby improving the accuracy and robustness of scene text recognition.

[0028] Based on the further improvement of the above-mentioned device, it also includes a local retraction generation module, which is used to: perform a graphic-text semantic matching evaluation on the generated text to obtain a matching score; in response to the matching score being lower than a preset score threshold, determine the token area not covered by the semantic anchor paragraph as a low-confidence area, and determine the token area covered by the semantic anchor paragraph as a high-confidence area; further reduce the sampling temperature of the token position of the low-confidence area, roll back the decoding state of the low-confidence area, and re-generate text for the low-confidence area; merge the text regenerated from the low-confidence area with the text of the high-confidence area generated in the last text generation process as the integrated generated text, and go to the processing step of the semantic anchor recognition module to obtain the first token set consisting of tokens whose correlation with the input image meets the set requirements in the text generation process; in response to the matching score being greater than the preset threshold, use the text obtained by re-generating the text according to the adjusted sampling temperature as the final output text.

[0029] The beneficial effect of adopting the further improved scheme is that the accuracy of text generation is further improved.

[0030] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the scene text recognition method based on the multi-modal large language model according to any one of the above when executing the computer program.

[0031] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the scene text recognition method based on the multi-modal large language model according to any one of the above.

[0032] The application further provides a computer program product, which includes a computer program, and the computer program is executable on a processor to implement the scene text recognition method based on the multi-modal large language model according to any one of the above.

[0033] In the application, the above technical solutions can be combined with each other to achieve more preferred combination solutions. Other features and advantages of the application will be described in the subsequent description, and some advantages will become apparent from the description or be understood by implementing the application. The purposes and other advantages of the application can be achieved and obtained from the contents specifically indicated in the description and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings are included to provide a further understanding of the application and are incorporated herein and constitute a part of the application. The drawings illustrate embodiments of the application and, together with the description, serve to explain the principles of the application. In the drawings: Figure 1 is a flowchart of the scene text recognition method based on the multi-modal large language model provided by the application; Figure 2 is one of structural schematic diagrams of the scene text recognition device based on the multi-modal large language model provided by the application; Figure 3 is another of structural schematic diagrams of the scene text recognition device based on the multi-modal large language model provided by the application; Figure 4 is a flowchart of the scene text recognition of the scene text recognition device based on the multi-modal large language model provided by the application; Figure 5 is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0035] Preferred embodiments of the present application will be described in detail below with reference to the drawings, wherein the drawings constitute a part of this application and are used to explain the principles of the application together with the embodiments of the application, but are not used to limit the scope of the application.

[0036] The present application relates to the cross field of computer vision and natural language processing, and is particularly applied to scene text recognition and understanding, belongs to the multi-modal information processing technology in the field of artificial intelligence, and is widely applied to automatic driving, product intelligent analysis, auxiliary reading equipment and other practical scenes.

[0037] The traditional or current mainstream multi-modal large language model (MLLM) tends to speculate results according to semantic common sense when processing scene text, rather than accurately identifying based on the actual existing text region in the image, especially when facing non-semantic disturbance text (such as spelling errors, word creation, and random codes), it is easy to output incorrect results that "look reasonable" but do not match the visual content.

[0038] It is found through research that the current multi-modal model internal attention mechanism does not pay sufficient attention to the text region in the image, different Transformer layers respond inconsistently to the text region and even attention drift occurs, resulting in loss of text information in the transmission process, further causing recognition errors. Different Transformer layers show different attention distribution and recognition tendency when processing visual text, and some intermediate layers are closer to the true label than the final output layer. This shows that attention drift may be one of the key reasons for semantic illusion. Although the attention mechanism is an important part of the multi-modal model, most current methods treat it as an implicit behavior, lack of systematic regulation and guidance methods, resulting in some visual features not being effectively reflected in the upper layer language output.

[0039] To solve this problem, the present application provides a scene text recognition method based on a multi-modal large language model, which can realize attention-guided semantic illusion suppression, has the advantages of training freedom, high compatibility, etc., and can significantly improve the performance and robustness of MLLM in complex scene text recognition tasks.

[0040] Figure 1 is a flowchart of the scene text recognition method based on the multi-modal large language model provided by the present application. As shown in Figure 1 , the method comprises: Step S1, obtaining a first token set composed of tokens whose relevance to an input image in a text generation process meets a set requirement, and obtaining a second token set composed of tokens whose semantic stability in the text generation process meets a set requirement, forming a semantic anchor paragraph according to tokens that appear continuously in the second token set, and recording the sampling temperature of each token position in the text generation process.

[0041] The multi-modal large language model supports multi-modal input when performing scene text recognition, such as supporting joint input of multiple data types such as text, image, audio, and video (for example: inputting a picture + question, and outputting a text answer). The scene text recognition method provided by the application is mainly applied to image text recognition. The image can be a single picture or a frame of image in a video stream. When performing scene text recognition, the image or the image + prompt text can be used as input. The multi-modal large language model converts the encoded information of multi-modal input (such as image, text, audio, etc.) into human understandable output such as text answer through a decoder.

[0042] In this step, the token paragraph with stable semantics and high correlation with the input image is detected and marked to form a semantic anchor paragraph. The first token set formed by tokens meeting the set requirements of the correlation with the input image in the text generation process is obtained, and the second token set formed by tokens meeting the set requirements of the semantic stability in the text generation process is obtained. The tokens in the intersection of the first token set and the second token set meet the set requirements of the semantic stability and the correlation with the input image at the same time. The first token set formed by tokens meeting the set requirements of the correlation with the input image in the text generation process and the second token set formed by tokens meeting the set requirements of the semantic stability in the text generation process can be dynamically obtained during the text generation process.

[0043] The semantic anchor paragraph is formed by the tokens continuously appearing in the intersection of the first token set and the second token set. The semantic anchor paragraph can be one or more. The tokens in the semantic anchor paragraph are the tokens continuously appearing in the intersection of the first token set and the second token set. The sampling temperature of each token position in the text generation process is recorded. The tokens in the semantic anchor paragraph are highly correlated with the input image, which reduces the dependence of the model on language semantics from the mechanism. In addition, attention drift is an important reason for unstable semantics. By setting the tokens in the semantic anchor paragraph to meet the semantic stability, the attention drift phenomenon can be effectively overcome.

[0044] Step S2, adjust the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph, and perform text generation again according to the adjusted sampling temperature.

[0045] In large language models such as GPT or multimodal large models (MLLM), temperature is an important parameter for controlling the "randomness" or "diversity" of the output content. It mainly plays a role in the process of sampling generation, that is, when the model predicts the next word or token, it does not always choose the one with the highest probability, but "draws" one from multiple candidate words according to a set of probability distributions. At this time, the temperature determines whether the model chooses more "safely" or more "freely".

[0046] In this step, the sampling temperature of each token position in the text generation process is finely adjusted based on the semantic anchor paragraph, realizing the differential control of the diversity and certainty of different regions, and the text is generated again according to the adjusted sampling temperature.

[0047] The scene text recognition method based on the multimodal large language model provided by the application comprises the following steps: obtaining a first token set composed of tokens whose relevance to an input image meets a set requirement in a text generation process, and obtaining a second token set composed of tokens whose semantic stability meets a set requirement in the text generation process; forming a semantic anchor paragraph according to tokens that continuously appear in an intersection of the first token set and the second token set, and recording the sampling temperature of each token position in the text generation process; adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph; and generating the text again according to the adjusted sampling temperature. Without changing the original MLLM architecture, the method realizes accurate recognition of visual text and inhibition of semantic hallucination in image text recognition, and improves the accuracy and robustness of scene text recognition.

[0048] According to the scene text recognition method based on the multimodal large language model provided by the application, the first token set composed of tokens whose relevance to an input image meets a set requirement in a text generation process comprises the following steps: for a token in the text generation process, extracting a cross-attention vector of the token in a cross-attention mechanism; wherein the cross-attention vector reflects the attention degree of the token to each region of the input image; calculating a cross-attention autocorrelation matrix between the cross-attention vectors of the tokens in a set time window, obtaining relevant tokens whose cosine similarity of the cross-attention autocorrelation matrix is greater than a preset similarity threshold, and marking the remaining tokens in a token sequence output by the multimodal large language model after excluding the relevant tokens as the first token set.

[0049] In the text generation process, when the relevance of the input image meets the set requirements token constitutes a first token set, first, for the token in the text generation process, the cross attention vector of the token in the cross attention mechanism of image-text is extracted, and the cross attention vector reflects the attention degree of the token to each region of the input image.

[0050] The cross attention vector is obtained by the Cross-Attention layer in the Transformer, and is represented as: ; Among them, the cross attention vector of the t i token, the Query vector of the t i token generated by the language end decoder (decoder), the key vector set of the input image, the dimension of key / Query. , and n represents the number of input image tokens.

[0051] The vector reflects the attention degree of the current generated token to each region of the image, and the higher the value, the greater the influence of the region on the token generation.

[0052] In a set time window (for example, the last k tokens, k can take a value between 32 and 64), the autocorrelation matrix C∈R k×k between the cross attention vectors of these tokens is calculated to evaluate whether the attention is concentrated and stable over time: ; Among them, the cross attention autocorrelation matrix of the t i token and the t j token.

[0053] If the attention vectors between multiple tokens have high cosine similarity (such as greater than 0.85), it means that these tokens long-term attention to the same region in the image, such as a corner of the image, and cannot pay attention to the whole image, and the image does not have high relevance, and attention drift occurs. Therefore, according to the calculation result of the cross attention autocorrelation matrix, the noise token of attention drift can be screened out, that is, the token whose attention vectors between multiple tokens have high cosine similarity.

[0054] Therefore, according to the calculation result of the cross attention vector of the token, a cross attention self-correlation matrix between the cross attention vectors of the tokens in a set time window is calculated, relevant tokens with a cosine similarity of the cross attention self-correlation matrix greater than a preset similarity threshold are obtained, and remaining tokens after the relevant tokens are filtered out from a token sequence output by the multi-modal large language model are marked as a first token set as semantic anchor point candidates.

[0055] The method for scene text recognition based on a multi-modal large language model provided by the application extracts a cross attention vector of a token in a cross attention mechanism between a text generation process, the cross attention vector reflecting the attention degree of the token to each region of an input image, calculates a cross attention self-correlation matrix between the cross attention vectors of the tokens in a set time window, obtains relevant tokens with a cosine similarity of the cross attention self-correlation matrix greater than a preset similarity threshold, marks remaining tokens after the relevant tokens are filtered out from a token sequence output by the multi-modal large language model as a first token set, and removes noise tokens with no high correlation with an image and attention drift in the construction process of a semantic anchor point paragraph, thereby improving the accuracy of the semantic anchor point paragraph.

[0056] According to the method for scene text recognition based on a multi-modal large language model provided by the application, the second token set composed of tokens with semantic stability meeting a set requirement in a text generation process is obtained, including: for a token in a text generation process, extracting a prediction distribution of the token at each layer of a Transformer, calculating a consistency index of the prediction distribution according to the prediction distribution of the token at each layer of the Transformer, and marking a token with a consistency index meeting a requirement as the second token set.

[0057] The consistency of the prediction distribution of the token at each layer of the Transformer is high, indicating that the semantic output is stable and is not disturbed by the context. The consistency of the prediction distribution of the token at each layer of the Transformer is low, indicating that the token is greatly affected by semantic illusion at a high layer and is a token with unstable semantics.

[0058] The embodiment obtains a second token set as a semantic anchor point candidate. When a second token set composed of tokens that meet the set requirement in terms of semantic stability in the text generation process is obtained, for the tokens in the text generation process, the prediction distribution of the token at each layer of the Transformer is extracted, the consistency index of the prediction distribution is calculated according to the prediction distribution of the token at each layer of the Transformer, and the token that meets the requirement of the consistency index of the prediction distribution is marked as the second token set.

[0059] The application provides a scene text recognition method based on a multi-modal large language model. For tokens in a text generation process, the prediction distribution of the token at each layer of the Transformer is extracted, the consistency index of the prediction distribution is calculated according to the prediction distribution of the token at each layer of the Transformer, and the token that meets the requirement of the consistency index of the prediction distribution is marked as a second token set. The method realizes the removal of tokens with unstable semantics in the semantic anchor point paragraph construction process, and further improves the accuracy of semantic anchor point paragraph acquisition.

[0060] According to the scene text recognition method based on the multi-modal large language model, the consistency index of the prediction distribution is calculated according to the prediction distribution of the token at each layer of the Transformer, including: calculating the KL divergence of the prediction distribution of the token at each two adjacent layers of the Transformer, and determining that the token with an average value of the KL divergence less than a preset threshold value is the token that meets the requirement of the consistency index of the prediction distribution.

[0061] The embodiment evaluates the consistency of the prediction distribution of the token at each layer of the Transformer by calculating the KL divergence. Specifically, by calculating the KL divergence of the prediction distribution of each two adjacent layers, the token with an average value of the KL divergence less than a preset threshold value (such as 0.1) is determined as the token that meets the requirement of the consistency index of the prediction distribution. ; wherein, represents the KL divergence of the prediction distribution of the token at the layer and the layer of the Transformer, represents the prediction distribution of the token at the layer of the Transformer, represents the prediction distribution of the token at the layer of the Transformer, represents the prediction distribution of the token at the a vector of a jth dimension of a predicted distribution of a layer, represents the token in the Transformer a vector of a jth dimension of a predicted distribution of a layer.

[0062] The application provides a scene text recognition method based on a multi-modal large language model. The KL divergence of the prediction distribution of each two adjacent layers of the token in the Transformer is calculated to determine that the token with an average value of the KL divergence less than a preset threshold is a token with a required consistency index of the prediction distribution, thereby improving the accuracy of the second token set result.

[0063] According to the scene text recognition method based on the multi-modal large language model, the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph is adjusted, which comprises: for the token position at the semantic anchor point paragraph, maintaining the sampling temperature of the token position or slightly adjusting the sampling temperature by a preset smaller amplitude; for the token position at the non-semantic anchor point paragraph, reducing the sampling temperature by a preset larger amplitude.

[0064] The slight adjustment of the sampling temperature by the preset smaller amplitude can be adjusting the sampling temperature to 85%-95% of the original sampling temperature, and the reduction of the sampling temperature by the preset larger amplitude can be adjusting the sampling temperature to 40%-60% of the original sampling temperature.

[0065] When the text generation is performed for the first time, a temperature mapping table TokenTempMap needs to be initialized in advance. The temperature mapping table TokenTempMap is an array of structures [T1, T2,..., Tn] for recording the sampling temperature of each token position. The initialization value is Tinit by default, for example, 0.5.

[0066] When the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph is adjusted, for the token position at the semantic anchor point paragraph, the sampling temperature of the token position is maintained or slightly adjusted by a preset smaller amplitude to ensure the natural fluency and semantic richness of the text, for example, the sampling temperature of the token position at the semantic anchor point paragraph is set to T anchor ≈0.5; for the token position at the non-semantic anchor point paragraph, the sampling temperature is reduced by a preset larger amplitude, for example, the temperature is reduced to T non <0.5, for example, 0.3 or 0.2, to reduce the sampling diversity, enhance the generation certainty, and inhibit the hallucination generation.

[0067] The scene text recognition method based on a multimodal large language model provided by the present invention maintains the sampling temperature of the token position at the semantic anchor paragraph or fine-tunes the sampling temperature by a preset smaller amount, and reduces the sampling temperature by a preset larger amount for the token position at the non-semantic anchor paragraph. This ensures the natural fluency and semantic enrichment of the text and suppresses hallucinations in text generation.

[0068] According to a scene text recognition method based on a multimodal large language model provided by the present invention, before regenerating text based on the adjusted sampling temperature, the method further includes: obtaining a low-confidence area of ​​the input image where the cross-modal matching confidence is lower than a preset confidence threshold; and lowering the sampling temperature of the token position corresponding to the low-confidence area and / or edge area of ​​the input image.

[0069] For image edge regions and / or regions with low cross-modal matching confidence (e.g., CLIPScore is below a threshold), a temperature perturbation factor δT is added: ; in, Indicates the sampling temperature before adjustment.

[0070] This additional cooling strategy can reduce the probability of risky tokens being sampled and generated, making the generation more credible.

[0071] The low confidence region can be determined based on the relationship between the cross-modal matching confidence of the image and a preset confidence threshold. The edge region can be determined based on existing methods or customized rules.

[0072] TokenTempMap is used dynamically when sampling token by token. At each token generation stage, the temperature corresponding to the current token is sampled: ; in, Represents a token The probability of generating is the current logits vector, From TokenTempMap; the lower the temperature, the sharper the distribution and the more certain the result.

[0073] During the generation process, if a newly identified semantic anchor point or attention area drifts, TokenTempMap will be dynamically updated so that the sampling strategy evolves with the generation process to ensure both local accuracy and global fluency.

[0074] The scene text recognition method based on the multi-modal large language model provided by the application reduces the sampling temperature of the token position corresponding to the low-confidence region and / or the edge region of the input image, reduces the probability of generating a risk token, and makes the generated result more reliable.

[0075] According to the scene text recognition method based on the multi-modal large language model provided by the application, after the text generation is performed again according to the adjusted sampling temperature, the method further comprises: performing image-text semantic matching evaluation on the generated text to obtain a matching score; in response to the matching score being lower than a preset score threshold, determining a token region not covered by the semantic anchor point paragraph as a low-confidence region and determining a token region covered by the semantic anchor point paragraph as a high-confidence region; further reducing the sampling temperature of the token position of the low-confidence region, rolling back the decoding state of the low-confidence region, and performing text generation on the low-confidence region again; merging the text generated by the low-confidence region again and the text of the high-confidence region generated in the last text generation process to obtain integrated generated text, and proceeding to the processing step of obtaining the first token set meeting the set requirement of the relevance of the input image.

[0076] The image semantic reverse matching and anchor point coverage verification are performed on the generated content to determine the semantic reliability of the generated text, and local rollback and regeneration are triggered when necessary to ensure the semantic fidelity of the generated content. After the text generation is completed, the image-text semantic matching evaluation is performed on the text to identify the low-confidence region and perform local rollback and rewriting, thereby forming a closed-loop enhancement mechanism.

[0077] The image-text semantic matching evaluation is performed on the generated text to obtain a matching score, and if the matching score is lower than a preset score threshold, a token region not covered by the semantic anchor point paragraph is determined as a low-confidence region, and a token region covered by the semantic anchor point paragraph is determined as a high-confidence region. The sampling temperature of the token position of the low-confidence region is further reduced (for example, the sampling temperature is reduced to 0.3 times the previous sampling temperature), the decoding state of the low-confidence region is rolled back to the starting point (or offset point) of the region, and the decoder is used to generate text for the low-confidence region according to the reduced sampling temperature, so as to ensure that the content of the paragraph is more closely related to the image semantic content on the basis of language rationality.

[0078] The text regenerated in the low-trust region is merged with the text of the high-trust region generated in the last text generation process as the integrated generated text, and the process step of obtaining the first token set constituted by the tokens meeting the set requirements in terms of relevance to the input image is entered. That is, based on the full-region text obtained through local rollback, further execution is performed: The first token set constituted by the tokens meeting the set requirements in terms of relevance to the input image in the text generation process and the second token set constituted by the tokens meeting the set requirements in terms of semantic stability in the text generation process are obtained, a semantic anchor paragraph is formed according to the tokens continuously appearing in the intersection of the first token set and the second token set, and the sampling temperature of each token position in the text generation process is recorded; the sampling temperature of each token position in the text generation process is adjusted based on the semantic anchor paragraph, and text generation is performed again according to the adjusted sampling temperature.

[0079] Further, the generated text is evaluated for graphic-text semantic matching to obtain a matching score; in response to the matching score being lower than a preset score threshold, the token region not covered by the semantic anchor paragraph is determined as a low-trust region, and the token region covered by the semantic anchor paragraph is determined as a high-trust region; the sampling temperature of the token position in the low-trust region is further reduced, the decoding state of the low-trust region is rolled back, and text generation is performed again for the low-trust region; wherein the local token rollback mechanism is started for the low-trust region, and the low-trust region is regenerated using a stricter temperature and sampling strategy to ensure faithful restoration of the text to the image semantics; the text regenerated in the low-trust region is merged with the text of the high-trust region generated in the last text generation process as the integrated generated text, and the process step of obtaining the first token set constituted by the tokens meeting the set requirements in terms of relevance to the input image in the text generation process is entered, that is, as long as the graphic-text semantic matching evaluation of the generated text does not meet the requirements, the above process is repeatedly executed to form a complete quality control closed loop. By dynamically adjusting the temperature, the local details and global coherence of the overall generation process are ensured, and the accuracy and stability of the generated text are continuously improved.

[0080] If the matching score obtained by evaluating the generated text for graphic-text semantic matching is greater than the preset threshold, the generated text is taken as the final output text.

[0081] The scene text recognition method based on the multi-modal large language model provided by the application, by evaluating the generated text for graphic-text semantic matching, a matching score is obtained, in response to the matching score being lower than the preset score threshold, determining the token region not covered by the semantic anchor point paragraph as a low trust region, determining the token region covered by the semantic anchor point paragraph as a high trust region, further reducing the sampling temperature of the token position of the low trust region, rolling back the decoding state of the low trust region, re-generating text for the low trust region, merging the text generated by the low trust region and the text generated by the high trust region in the last text generation process as the integrated generated text, and going to the processing step of obtaining the first token set constituted by the token whose relevance to the input image meets the set requirement, in response to the matching score being greater than the preset threshold, the text generated according to the adjusted sampling temperature is taken as the final output text, and the accuracy of text generation is further improved.

[0082] According to the scene text recognition method based on the multi-modal large language model provided by the application, the generated text is vector encoded to obtain a first vector, the input image is vector encoded to obtain a second vector, and the matching score is obtained by calculating the correlation between the first vector and the second vector.

[0083] When the generated text is evaluated for graphic-text semantic matching, the generated text is vector encoded to obtain a first vector, the input image is vector encoded to obtain a second vector, and the matching score is obtained by calculating the correlation between the first vector and the second vector. The matching score is used to evaluate the semantic coupling degree of the generated text and the input image.

[0084] The matching score is expressed as: ; Wherein, The matching score is expressed as: The first vector is expressed as: The second vector is expressed as: If the matching score is lower than the threshold (such as 0.25), it is considered as semantic disengagement. The CLIP or BLIP type graphic-text encoder can be used to obtain the matching score.

[0085] The scene text recognition method based on the multi-modal large language model provided by the application, by vector encoding the generated text to obtain a first vector, vector encoding the input image to obtain a second vector, and calculating the correlation between the first vector and the second vector to obtain the matching score, the accuracy of the matching score is improved.

[0086] The application realizes explicit modeling, sampling control and posterior correction of visual regions in a multi-modal generation task through the synergistic effect of semantic anchor point paragraph recognition, temperature regulation generation and local rollback generation. The technical scheme effectively solves the key problems of semantic prior bias, visual attention drift and complex deployment in the existing method without changing the original model structure, and brings the following beneficial effects: I. Explicitly model the image visual region to effectively suppress semantic prior bias The application dynamically detects semantic anchors during the generation process, that is, token segments that have stable high response on cross-modal attention and consistent prediction distribution in multi-layer Transformers. These anchors reduce the model's dependence on language semantics from a mechanism.

[0087] II. Relieve attention drift and enhance visual region response consistency By calculating the KL divergence across multiple layers, token regions with consistent multi-layer responses can be selected, effectively filtering attention drift or unstable attention. In combination with the local rollback generation mechanism, bidirectional constraints from forward generation to posterior matching are achieved.

[0088] III. Provide a fine-grained sampling regulation mechanism to improve generation stability and accuracy Construct TokenTempMap to implement a dynamic temperature regulation mechanism at the token level. The sampling temperature of non-anchor regions is lowered to effectively suppress "hallucination" generation, and the temperature of anchor regions is fine-tuned to ensure semantic coherence and language fluency.

[0089] IV. Construct a dynamic feedback loop to significantly improve visual fidelity Introduce reverse semantic matching and local token rollback mechanisms to revise "low matching regions" in the later stage of text generation, ensuring that the final text accurately represents the information contained in the image, and achieving "forward anchoring + process regulation + posterior correction" of the whole link optimization.

[0090] In summary, the application effectively addresses the problems of heavy semantic dependence, weak visual modeling and high structural complexity in current multi-modal image-text understanding tasks through semantic anchor point recognition, dynamic temperature sampling and posterior verification mechanisms. It has the advantages of high precision, strong robustness, friendly deployment, and is suitable for various application scenarios such as image-text recognition, image-text question answering and OCR-assisted understanding.

[0091] The following describes the scene text recognition device based on a multi-modal large language model provided by the application. The scene text recognition device based on a multi-modal large language model described below can be mutually referenced with the scene text recognition method based on a multi-modal large language model described above.

[0092] Figure 2 is one of the structural schematic diagrams of the scene text recognition device based on a multi-modal large language model provided by the application. As shown in Figure 2 The device comprises a semantic anchor point recognition module 10 and a temperature regulation generation module 20, wherein: the semantic anchor point recognition module 10 is used to: acquire a first token set constituted by tokens whose relevance to an input image in a text generation process meets a set requirement, and acquire a second token set constituted by tokens whose semantic stability in the text generation process meets a set requirement, form a semantic anchor point paragraph according to tokens that continuously appear in the intersection of the first token set and the second token set, and record the sampling temperature of each token position in the text generation process; the temperature regulation generation module 20 is used to: adjust the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph, and perform text generation again according to the adjusted sampling temperature.

[0093] The scene text recognition device based on a multi-modal large language model provided by the application, by acquiring a first token set constituted by tokens whose relevance to an input image in a text generation process meets a set requirement, and acquiring a second token set constituted by tokens whose semantic stability in the text generation process meets a set requirement, forming a semantic anchor point paragraph according to tokens that continuously appear in the intersection of the first token set and the second token set, and recording the sampling temperature of each token position in the text generation process, adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph, and performing text generation again according to the adjusted sampling temperature, realizes accurate recognition of visual text and inhibition of semantic hallucination in image text recognition without changing the original MLLM architecture, and improves the accuracy and robustness of scene text recognition.

[0094] Figure 3 is the second structural schematic diagram of the scene text recognition device based on a multi-modal large language model provided by the application. As shown in Figure 3As shown, the device further comprises a local rollback generation module 30, which is configured to: perform text-image semantic matching evaluation on the generated text to obtain a matching score; in response to the matching score being lower than a preset score threshold, determine a token region not covered by the semantic anchor paragraph as a low-confidence region and a token region covered by the semantic anchor paragraph as a high-confidence region; further reduce the sampling temperature of the token position in the low-confidence region, rollback the decoding state of the low-confidence region, and re-perform text generation on the low-confidence region; merge the re-generated text of the low-confidence region with the text of the high-confidence region generated in the last text generation process as the integrated generated text, and proceed to the processing step of the first token set constituted by the tokens whose relevance to the input image meets the set requirement in the text generation process of the semantic anchor recognition module; in response to the matching score being greater than the preset threshold, the text generated by re-performing text generation according to the adjusted sampling temperature is taken as the final output text.

[0095] The scene text recognition device based on the multi-modal large language model provided by the present application further improves the accuracy of text generation by performing text-image semantic matching evaluation on the generated text to obtain a matching score, determining a token region not covered by the semantic anchor paragraph as a low-confidence region and a token region covered by the semantic anchor paragraph as a high-confidence region in response to the matching score being lower than a preset score threshold, further reducing the sampling temperature of the token position in the low-confidence region, rolling back the decoding state of the low-confidence region, re-performing text generation on the low-confidence region, merging the re-generated text of the low-confidence region with the text of the high-confidence region generated in the last text generation process as the integrated generated text, and proceeding to the processing step of the first token set constituted by the tokens whose relevance to the input image meets the set requirement in the text generation process of the semantic anchor recognition module, and taking the text generated by re-performing text generation according to the adjusted sampling temperature as the final output text in response to the matching score being greater than the preset threshold.

[0096] Figure 4 is a process schematic diagram of the scene text recognition device based on the multi-modal large language model provided by the present application. As shown in Figure 4 , the process includes: inputting an image + initial text prompt to the multi-modal large language model; The semantic anchor point identification module is used to obtain a first token set composed of tokens meeting a set requirement in terms of relevance to the input image in a text generation process, and a second token set composed of tokens meeting a set requirement in terms of semantic stability in the text generation process, form a semantic anchor point paragraph based on tokens continuously appearing in an intersection of the first token set and the second token set, and record sampling temperatures of token positions in the text generation process; The temperature regulation generation module is used to adjust the sampling temperatures of the token positions in the text generation process based on the semantic anchor point paragraph, and re-perform text generation according to the adjusted sampling temperatures. The local rollback generation module is used to perform text-image semantic matching evaluation on the generated text to obtain a matching score, determine a token region not covered by the semantic anchor point paragraph as a low-confidence region and a token region covered by the semantic anchor point paragraph as a high-confidence region in response to the matching score being lower than a preset score threshold, further reduce the sampling temperatures of token positions in the low-confidence region, rollback the decoding state of the low-confidence region, and re-perform text generation on the low-confidence region, merge the text generated by the low-confidence region and the text of the high-confidence region generated in the last text generation process as integrated generated text, and go to the processing step of obtaining the first token set composed of tokens meeting a set requirement in terms of relevance to the input image in a text generation process by the semantic anchor point identification module, and in response to the matching score being greater than a preset threshold, take the text generated according to the adjusted sampling temperatures as the final output text.

[0097] Figure 5 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 5As shown, the electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 complete mutual communication through the communications bus 540. The processor 510 can invoke a logical instruction in the memory 530 to execute a scene text recognition method based on a multi-modal large language model, the method including: obtaining a first token set constituted by tokens whose relevance to an input image in a text generation process meets a set requirement, and obtaining a second token set constituted by tokens whose semantic stability in the text generation process meets a set requirement, forming a semantic anchor point paragraph according to tokens that continuously appear in an intersection of the first token set and the second token set, and recording a sampling temperature of each token position in the text generation process; adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor point paragraph, and re-performing text generation according to the adjusted sampling temperature.

[0098] In addition, the logical instructions in the memory 530 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0099] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer-readable storage medium, and the computer program is executable by a processor to enable the computer to perform the scene text recognition method based on a multi-modal large language model provided by the above method, which comprises: obtaining a first token set constituted by tokens whose relevance to an input image meets a set requirement in a text generation process, and obtaining a second token set constituted by tokens whose semantic stability meets a set requirement in the text generation process, forming a semantic anchor paragraph according to tokens that continuously appear in an intersection of the first token set and the second token set, and recording a sampling temperature of each token position in the text generation process; adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph, and re-performing text generation according to the adjusted sampling temperature.

[0100] In another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is executable by a processor to implement the scene text recognition method based on a multi-modal large language model provided by the above method, which comprises: obtaining a first token set constituted by tokens whose relevance to an input image meets a set requirement in a text generation process, and obtaining a second token set constituted by tokens whose semantic stability meets a set requirement in the text generation process, forming a semantic anchor paragraph according to tokens that continuously appear in an intersection of the first token set and the second token set, and recording a sampling temperature of each token position in the text generation process; adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph, and re-performing text generation according to the adjusted sampling temperature.

[0101] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium is a disk, an optical disk, a read-only memory or a random access memory, etc.

[0102] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0103] Those skilled in the art can clearly understand the implementation of the embodiments by the description of the above embodiments, and the embodiments can be implemented by means of software and necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some parts of the embodiment.

[0104] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0105] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A scene text recognition method based on a multimodal large language model, characterized in that: include: Obtaining a first token set consisting of tokens whose correlation with the input image meets set requirements during the text generation process, and obtaining a second token set consisting of tokens whose semantic stability meets set requirements during the text generation process, forming semantic anchor paragraphs based on tokens that appear continuously in the intersection of the first token set and the second token set, and recording the sampling temperature of each token position during the text generation process; The sampling temperature of each token position in the text generation process is adjusted based on the semantic anchor paragraph, and the text is regenerated according to the adjusted sampling temperature.

2. The scene text recognition method based on a multimodal large language model according to claim 1, characterized in that: The first token set consisting of tokens whose correlation with the input image during the text generation process meets the set requirements includes: For a token in the text generation process, extract the cross-attention vector of the token in the image-text cross-attention mechanism; wherein the cross-attention vector reflects the degree of attention of the token to each area of ​​the input image; Calculate the cross-attention autocorrelation matrix between the cross-attention vectors of the tokens within the set time window, obtain the related tokens whose cosine similarity of the cross-attention autocorrelation matrix is ​​greater than the preset similarity threshold, and mark the remaining tokens after filtering out the related tokens in the token sequence output by the multimodal large language model as the first token set.

3. The scene text recognition method based on a multimodal large language model according to claim 1, characterized in that: The second token set consisting of tokens whose semantic stability meets the set requirements during the text generation process is obtained, including: For the token in the text generation process, the predicted distribution of the token in each layer of the Transformer is extracted, the consistency index of the predicted distribution is calculated based on the predicted distribution of the token in each layer of the Transformer, and the tokens whose consistency index of the predicted distribution meets the requirements are marked as the second token set.

4. The scene text recognition method based on a multimodal large language model according to claim 3 is characterized in that: Calculating the consistency index of the predicted distribution based on the predicted distribution of the token at each layer of the Transformer includes: Calculate the KL divergence of the predicted distribution of the token in two adjacent layers of the Transformer, and determine the token whose average value of the KL divergence is less than a preset threshold as the token that meets the requirement of the consistency index of the predicted distribution.

5. The scene text recognition method based on a multimodal large language model according to claim 1, characterized in that: The adjusting the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph includes: For the token position at the semantic anchor paragraph, maintain the sampling temperature of the token position or fine-tune the sampling temperature by a preset small amplitude; For token positions that are not in the semantic anchor paragraph, the sampling temperature is lowered by a preset large margin.

6. The scene text recognition method based on a multimodal large language model according to claim 1, characterized in that: Before regenerating the text according to the adjusted sampling temperature, the method further includes: Acquire a low-confidence region of the input image where the cross-modal matching confidence is lower than a preset confidence threshold; Lowering the sampling temperature of the token position corresponding to the low confidence area and / or edge area of ​​the input image.

7. The scene text recognition method based on a multimodal large language model according to claim 1, characterized in that: After regenerating the text according to the adjusted sampling temperature, the method further includes: Perform image-text semantic matching evaluation on the generated text to obtain a matching score; In response to the matching score being lower than a preset score threshold, determining a token region not covered by the semantic anchor paragraph as a low-confidence region, and determining a token region covered by the semantic anchor paragraph as a high-confidence region; Further reducing the sampling temperature of the token position in the low-confidence area, reverting the decoding state of the low-confidence area, and regenerating text for the low-confidence area; Merging the text regenerated from the low-confidence region with the text generated from the previous text generation process in the high-confidence region as the integrated generated text, and proceeding to the step of obtaining a first token set consisting of tokens whose correlation with the input image meets the set requirements in the text generation process; In response to the matching score being greater than a preset threshold, the text obtained by regenerating the text according to the adjusted sampling temperature is used as the final output text.

8. The scene text recognition method based on a multimodal large language model according to claim 7, characterized in that: The generated text is subjected to a semantic matching evaluation of the image and text to obtain a matching score, including: Performing vector encoding on the generated text to obtain a first vector; Performing vector encoding on the input image to obtain a second vector; The matching score is obtained by calculating the correlation between the first vector and the second vector.

9. A scene text recognition device based on a multimodal large language model, characterized in that: include: A semantic anchor point identification module is configured to: obtain a first token set consisting of tokens whose correlation with the input image meets set requirements during the text generation process, and obtain a second token set consisting of tokens whose semantic stability meets set requirements during the text generation process; form semantic anchor paragraphs based on tokens that appear continuously in the intersection of the first token set and the second token set; and record the sampling temperature of each token position during the text generation process; The temperature control generation module is used to adjust the sampling temperature of each token position in the text generation process based on the semantic anchor paragraph, and regenerate the text according to the adjusted sampling temperature.

10. The scene text recognition device based on a multimodal large language model according to claim 9, characterized in that: It also includes a local retracement generation module; The local retracement generation module is used to: Perform image-text semantic matching evaluation on the generated text to obtain a matching score; In response to the matching score being lower than a preset score threshold, determining a token region not covered by the semantic anchor paragraph as a low-confidence region, and determining a token region covered by the semantic anchor paragraph as a high-confidence region; Further reducing the sampling temperature of the token position in the low-confidence area, reverting the decoding state of the low-confidence area, and regenerating text for the low-confidence area; Merging the text regenerated from the low-confidence region with the text generated from the previous text generation process in the high-confidence region as the integrated generated text, and transferring the step of obtaining a first token set consisting of tokens whose correlation with the input image satisfies set requirements in the text generation process of the semantic anchor recognition module; In response to the matching score being greater than a preset threshold, the text obtained by regenerating the text according to the adjusted sampling temperature is used as the final output text.

Images