Text processing method and device

Abstract

The embodiment of the invention provides a text processing method and device, and the method comprises the steps: determining a to-be-processed text, and obtaining a to-be-processed image related to the to-be-processed text; and determining at least two types of model input contents based on the to-be-processed text and the to-be-processed image, and respectively inputting the at least two types of model input contents into a content processing model to obtain at least two groups of model prediction texts, thereby improving the text processing efficiency and accuracy. And constructing a target text corresponding to the to-be-processed text based on the at least two groups of model prediction texts, and generating a text data pair based on the to-be-processed text and the target text, thereby rapidly generating a high-quality text data pair.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a text processing method and device. BACKGROUND

[0002] The rise of large language models and multi-modal large models has reached an unprecedented level in understanding, generating, and processing complex information. When training a large model, a high-quality dataset needs to be constructed. Specifically, given an input content, the model needs to give an output content. The input content and the output content together form a data pair. However, constructing a high-quality data pair generally has a high cost.

[0003] In the prior art, a data pair is usually generated by using a generative model to synthesize data, that is, using the powerful understanding, reasoning, and generation capabilities of an external pre-trained large model to process, supplement, transform, or generate new data and labels, and then constructing a data pair. However, the quality of the generated data pair is difficult to guarantee. Therefore, there is an urgent need for an effective text processing method to solve the above problems. SUMMARY Therefore, one or more embodiments of the present specification provide a text processing method. One or more embodiments of the present specification also provide a text processing device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects in the prior art.

[0004] According to a first aspect of an embodiment of the present specification, a text processing method is provided, comprising: determining a to-be-processed text, and obtaining a to-be-processed image associated with the to-be-processed text; determining at least two types of model input content based on the to-be-processed text and the to-be-processed image, and inputting the at least two types of model input content into a content processing model respectively to obtain at least two groups of model predicted texts; constructing a target text corresponding to the to-be-processed text based on the at least two groups of model predicted texts, and generating a text data pair based on the to-be-processed text and the target text.

[0005] Optionally, the obtaining the to-be-processed image associated with the to-be-processed text comprises: determining an original image associated with the to-be-processed text; determining a text region corresponding to the to-be-processed text in the original image; extracting the text region in the original image to obtain the to-be-processed image.

[0006] Optionally, the determining the at least two types of model input contents based on the to-be-processed text and the to-be-processed image comprises: taking the to-be-processed text as a text type model input content and taking the to-be-processed image as an image type model input content; combining the to-be-processed text and the to-be-processed image to obtain a text-image mixed type model input content.

[0007] Optionally, before the inputting the at least two types of model input contents into the content processing model, the method further comprises: determining a large language model matched with the text type model input content and determining a multi-modal large model matched with the image type model input content and the text-image mixed type model input content; taking the large language model and the multi-modal large model as the content processing model.

[0008] Optionally, the inputting the at least two types of model input contents into the content processing model to obtain at least two groups of model predicted texts comprises: inputting the text type model input content into the large language model to obtain a first model predicted text; inputting the image type model input content and the text-image mixed type model input content into the multi-modal large model respectively to obtain a second model predicted text corresponding to the image type model input content and a third model predicted text corresponding to the text-image mixed type model input content; taking the first model predicted text, the second model predicted text and the third model predicted text as the at least two groups of model predicted texts.

[0009] Optionally, the constructing the target text corresponding to the to-be-processed text based on the at least two groups of model predicted texts comprises: judging whether the at least two groups of model predicted texts match or not; if yes, taking the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text; if no, selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

[0010] Optionally, the selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text comprises: calling an evaluation model to evaluate the at least two groups of model predicted texts respectively according to preset evaluation dimensions to obtain text evaluation information corresponding to each group of model predicted texts; According to the text evaluation information corresponding to each group of model predicted texts, at least one group of model predicted texts is selected from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

[0011] Optionally, the selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text according to the text evaluation information corresponding to each group of model predicted texts comprises: determining a preset score threshold, and extracting an evaluation score from the text evaluation information corresponding to each group of model predicted texts; comparing the evaluation score corresponding to each group of model predicted texts with the preset score threshold, and selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

[0012] According to a second aspect of the embodiments of the present specification, a text processing apparatus is provided, comprising: a determination module configured to determine a to-be-processed text, and acquire a to-be-processed image associated with the to-be-processed text; an input module configured to determine at least two types of model input contents based on the to-be-processed text and the to-be-processed image, and input the at least two types of model input contents into a content processing model respectively to obtain at least two groups of model predicted texts; a generation module configured to construct a target text corresponding to the to-be-processed text based on the at least two groups of model predicted texts, and generate a text data pair based on the to-be-processed text and the target text.

[0013] According to a third aspect of the embodiments of the present specification, a computing device is provided, comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the above-mentioned text processing method.

[0014] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions, when executed by a processor, implement the steps of the above-mentioned text processing method.

[0015] According to a fifth aspect of the embodiments of the present specification, a computer program product is provided, comprising a computer program or instructions, and the computer program or instructions, when executed by a processor, implement the steps of the above-mentioned text processing method.

[0016] One embodiment of the present specification provides a text processing method, which comprises determining a to-be-processed text, and obtaining a to-be-processed image associated with the to-be-processed text. At least two types of model input contents are determined based on the to-be-processed text and the to-be-processed image, and the at least two types of model input contents are respectively input into a content processing model to obtain at least two groups of model predicted texts, thereby improving the text processing efficiency and accuracy. The target text corresponding to the to-be-processed text is constructed based on the at least two groups of model predicted texts, and the text data pair is generated based on the to-be-processed text and the target text, so as to quickly generate high-quality text data pairs. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of a text processing method provided by one embodiment of the present specification; Figure 2 is a text screening schematic diagram of a text processing method provided by one embodiment of the present specification; Figure 3 is a processing process flowchart of a text processing method provided by one embodiment of the present specification; Figure 4 is a text processing flowchart of a text processing method provided by one embodiment of the present specification; Figure 5 is a structural schematic diagram of a text processing device provided by one embodiment of the present specification; Figure 6 is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0018] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than those described herein, and it is understood that the present specification can be practiced with yet other embodiments that are apparent in light of, and tailored to, the overall scope of the present specification and the following detailed description. Therefore, the specific details disclosed in the present specification, including the dependent claims, are not to be interpreted as limiting.

[0019] The terms used in one or more embodiments of the present specification are merely for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present specification. The singular forms "a", "an" and "the" used in one or more embodiments of the present specification and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present specification means and includes any or all possible combinations of one or more associated listed items.

[0020] It should be understood that, although the terms first, second, etc. can be employed in describing various information in one or more embodiments of the present specification, the information should not be limited to such terms. These terms are only used to distinguish one category of information from another category of information. For example, without departing from the scope of one or more embodiments of the present specification, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to ascertaining".

[0021] In addition, it should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0022] First, the nomenclature involved in one or more embodiments of the present specification is explained.

[0023] Large Models: Refers to neural network models with a large number of parameters and complex structures. They are usually trained on massive amounts of data, capable of learning very rich patterns and knowledge, and exhibiting excellent performance in a variety of tasks. According to the type of data they process and the application field, large models can be further subdivided into large language models and multimodal large models.

[0024] Large Language Models (LLMs): are large models that specialize in processing and generating human language. They are based on the Transformer architecture, learning massive amounts of text data (such as books, articles, web pages, etc.), mastering language grammar, semantics, context understanding and generation capabilities. LLMs can complete various natural language processing tasks, such as text generation, question and answer systems, machine translation, abstracts and sentiment analysis.

[0025] Multimodal Large Models: are large models that can understand and process multiple types of data (or "modalities") simultaneously. In addition to text, they can also process images, audio, video and other information. By fusing data from different modalities, multimodal large models can better understand the world and perform more complex tasks. For example, image-text understanding and generation, video analysis, speech recognition and synthesis, and cross-modal retrieval.

[0026] Distillation: In the field of machine learning, it is a model compression and optimization technique. Its core idea is to use the knowledge of a high-performance "teacher model" (usually a large and complex model) to train a smaller and more efficient "student model. In this invention, the powerful capabilities of large models are mainly used to "produce" or "enhance" training data for future model learning, especially when the original data resources are limited or higher quality data is needed.

[0027] Problem-solving data: refers to a collection of problems and their corresponding solutions and processes to achieve these solutions, which are used to teach artificial intelligence models how to solve similar problems.

[0028] In this specification, a text processing method is provided, and this specification also relates to a text processing device, a computing device, a computer readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0029] Referring to Figure 1 , Figure 1 A flowchart of a text processing method according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0030] Step 102: Determine the text to be processed, and obtain the image to be processed associated with the text to be processed.

[0031] Specifically, the text to be processed can be a text content composed of question text and answer text corresponding to the question text, and the question text includes but is not limited to natural language question text, subject question text (mathematics questions, Chinese questions, etc.). In the case of subject question text, the text to be processed includes question text and answer text corresponding to the question text, and the question type corresponding to the subject question text can be any one of the question types such as multiple choice questions, fill-in-the-blank questions, true-or-false questions, and answer questions. The image to be processed corresponding to the text to be processed can be an image obtained by image acquisition on a paper file recording the text to be processed, and the image to be processed contains all the contents of the text to be processed. The paper file recording the text to be processed can be a paper test paper containing the text to be processed. The image to be processed can also be an electronic document recording the text to be processed, and the image to be processed can be an image containing only the text to be processed by screenshot and the like.

[0032] In practical applications, after determining the text to be processed, the image to be processed associated with the text to be processed can be obtained, and the image to be processed can display the text to be processed completely. The text to be processed in the image to be processed can be the text to be processed completed by the question.

[0033] Further, it is considered that the to-be-processed text can be a subject text, and the document recording the subject text can be a test paper, i.e., a to-be-answered test paper or a test paper with completed subject answering. The test paper can be used as the original image from which the to-be-processed image is extracted, and then the to-be-processed image is extracted from the original image. Specifically, in implementation: determining an original image associated with the to-be-processed text, determining a text region corresponding to the to-be-processed text in the original image, and extracting the text region in the original image to obtain the to-be-processed image.

[0034] Specifically, the original image can be an image recording the to-be-processed text and text content other than the to-be-processed text. In the case where the to-be-processed text is a subject text, the original image can be a test paper image. In the case where the to-be-processed text is a natural language question text, the original image can be a question and answer page containing the natural language question text and the answer text. The text region is an image region containing all content of the to-be-processed text in the original image.

[0035] Based on this, the original image associated with the to-be-processed text is determined, the text region where the to-be-processed text is located in the original image is determined by image recognition, and the text region is extracted from the original image by image interception or image clipping to obtain the to-be-processed image.

[0036] For example, in the scenario of constructing a high-quality data set, a text data containing a question text, an answer text, and a question analysis text can be constructed. The to-be-processed text can be the stem of a Chinese selection question, and the original image can be a Chinese test paper image recording the to-be-processed text. Image extraction is performed on the to-be-processed text recorded in the Chinese test paper image to obtain the to-be-processed image.

[0037] As described above, the text region is extracted from the original image to obtain the to-be-processed text in the form of an image, which can intuitively display the to-be-processed text and enable the to-be-processed image to display more text details.

[0038] Step 104: determining at least two types of model input content based on the to-be-processed text and the to-be-processed image, inputting the at least two types of model input content into a content processing model respectively, and obtaining at least two groups of model predicted texts.

[0039] Specifically, after determining the to-be-processed text and obtaining the to-be-processed image associated with the to-be-processed text, at least two types of model input contents can be determined based on the to-be-processed text and the to-be-processed image, and the at least two types of model input contents are input into the content processing model respectively to obtain at least two groups of model predicted texts. The at least two types of model input contents can be text type model input contents, image type model input contents, or mixed text and image type model input contents. The content processing model can be a large language model or a multi-modal large model. For the large language model, the input content can be text. For the multi-modal large model, the input content can be text, picture, or mixed text and picture content. The model predicted text is the output content of the content processing model, which can be generated by answering the question corresponding to the to-be-processed text, and can be a problem solving process text or a question analysis text.

[0040] Based on this, after determining the to-be-processed text and obtaining the to-be-processed image associated with the to-be-processed text, at least two types of model input contents can be determined based on the to-be-processed text and the to-be-processed image, and the at least two types of model input contents are input into the content processing model respectively to obtain at least two groups of model predicted texts. The at least two types of model input contents can be text type model input contents, image type model input contents, or mixed text and image type model input contents. The content processing model can be a large language model or a multi-modal large model. For the large language model, the input content can be text. For the multi-modal large model, the input content can be text, picture, or mixed text and picture content. The model predicted text is the output content of the content processing model, which can be generated by answering the question corresponding to the to-be-processed text, and can be a problem solving process text or a question analysis text.

[0041] Further, after determining the to-be-processed text and the to-be-processed image, at least two types of model input contents can be determined based on the to-be-processed text and the to-be-processed image. In specific implementation: The to-be-processed text is used as text type model input content, and the to-be-processed image is used as image type model input content. The to-be-processed text and the to-be-processed image are combined to obtain mixed text and image type model input content.

[0042] Based on this, the to-be-processed text is used as text type model input content, and the to-be-processed image is used as image type model input content. The to-be-processed text and the to-be-processed image are combined to obtain mixed text and image type model input content. At least two types of model input contents can be selected for subsequent data processing.

[0043] Following the above example, after determining the to-be-processed text and the to-be-processed image corresponding to the to-be-processed text, the to-be-processed text, the to-be-processed image, and the mixed text and image content composed of the to-be-processed text and the to-be-processed image can be used as model input contents.

[0044] In summary, the to-be-processed text is used as text type model input content, and the to-be-processed image is used as image type model input content. The to-be-processed text and the to-be-processed image are combined to obtain mixed text and image type model input content, which improves the richness of model input content and enriches the data set of model input.

[0045] Further, considering that the model input content is of text type, image type, and mixed text and image type, etc., different types of data processing capabilities are required to process the data using large models with different types of data processing capabilities. In specific implementation: determine a large language model matching the text type model input content, and determine a multi-modal large model matching the image type model input content and the mixed text and image type model input content; and use the large language model and the multi-modal large model as the content processing model.

[0046] Based on this, the large language model has text processing capability, so the large language model matching the text type model input content is determined, and the large language model is used as the content processing model for processing the text type model input content. The multi-modal large model has image and mixed text and image content processing capability, so the multi-modal large model matching the image type model input content and the mixed text and image type model input content is determined, and the multi-modal large model is used as the content processing model.

[0047] Further, the text type model input content is input into the large language model to obtain a first model predicted text; the image type model input content and the mixed text and image type model input content are input into the multi-modal large model respectively to obtain a second model predicted text corresponding to the image type model input content and a third model predicted text corresponding to the mixed text and image type model input content; and the first model predicted text, the second model predicted text and the third model predicted text are used as the at least two groups of model predicted texts.

[0048] Specifically, the first model predicted text refers to the topic analysis and answer content obtained by using the large language model to analyze the text type model input content, i.e., the to-be-processed text, which is the analysis and answer text corresponding to the to-be-processed problem obtained by prediction. The second model predicted text refers to the text content obtained by using the multi-modal large model to process the image type model input content, i.e., the to-be-processed image, which is the text content generated by analyzing and answering the topic in the to-be-processed image. The third model predicted text refers to the text content obtained by using the multi-modal large model to process the mixed text and image type model input content, i.e., the content composed of the to-be-processed image and the to-be-processed text, which is the text content obtained by analyzing and answering the topic.

[0049] Based on this, the text type model input content is input into the large language model, and the large language model analyzes and answers the topic based on the text type model input content, i.e., the to-be-processed text, to obtain the first model predicted text. The image type model input content and the image-text mixed type model input content are input into the multi-modal large model at the same time, to obtain the second model predicted text corresponding to the image type model input content, i.e., the text content generated by the multi-modal large model based on the image type model input content, and the third model predicted text corresponding to the image-text mixed type model input content, i.e., the text content generated by analyzing and answering the topic based on the combination of the to-be-processed text and the to-be-processed image. The first model predicted text, the second model predicted text and the third model predicted text are used as at least two groups of model predicted texts. In actual application, at least two types of model input content can be selected from the three types of model input content, and the corresponding large model is used to generate the topic analysis and answer text, i.e., the model predicted text.

[0050] In the above example, the to-be-processed text can be "1. The following statements about the origin of mankind are correct ( ) A. Mankind is naturally produced; B. Mankind is evolved from chimpanzees; C. Mankind is evolved from modern apes; D. Mankind is evolved from forest apes". The to-be-processed image is an image containing only the to-be-processed text extracted from an image containing the to-be-processed text and other text. The large language model is used to analyze and answer the topic based on the to-be-processed text, and the multi-modal large model is used to analyze and answer the topic based on the to-be-processed image. The to-be-processed text and the to-be-processed image can also be combined into image-text mixed type model input content, which is processed by the multi-modal large model to output the analysis and answer text of the question.

[0051] In summary, at least two types of model input content are selected from the three types of model input content, and the corresponding large model is used to generate the topic analysis and answer text, i.e., the model predicted text, to improve the richness of the model predicted text.

[0052] Step 106: Based on the at least two groups of model predicted texts, the target text corresponding to the to-be-processed text is constructed, and the text data pair is generated based on the to-be-processed text and the target text.

[0053] Specifically, after determining the at least two types of model input contents based on the to-be-processed text and the to-be-processed image, inputting the at least two types of model input contents into the content processing model respectively, and obtaining the at least two groups of model predicted texts, the target text corresponding to the to-be-processed text can be constructed based on the at least two groups of model predicted texts, and the text data pair can be generated based on the to-be-processed text and the target text, where the target text corresponding to the to-be-processed text is the question analysis and answer text with higher correctness determined based on the at least two groups of model predicted texts. The text data pair contains model input content and model output content, and can be used to train the large model, so that the large model has the ability to analyze and answer questions.

[0054] Based on this, after determining the at least two types of model input contents based on the to-be-processed text and the to-be-processed image, inputting the at least two types of model input contents into the content processing model respectively, and obtaining the at least two groups of model predicted texts, the target text corresponding to the to-be-processed text can be constructed based on the at least two groups of model predicted texts, and the text data pair can be generated based on the to-be-processed text and the target text, where the target text corresponding to the to-be-processed text is the question analysis and answer text with higher correctness determined based on the at least two groups of model predicted texts. The text data pair contains model input content and model output content, and can be used to train the large model, so that the large model has the ability to analyze and answer questions.

[0055] Further, after obtaining the at least two groups of model predicted texts, the target text can be constructed based on the at least two groups of model predicted texts, and in specific implementation: determining whether the at least two groups of model predicted texts match; if yes, taking the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text; if no, selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

[0056] Based on this, whether the at least two groups of model predicted texts match can be determined by judging whether the analysis content and the answer content corresponding to the at least two groups of model predicted texts are the same. If the analysis content and the answer content corresponding to the at least two groups of model predicted texts are the same, the at least two groups of model predicted texts can be taken as the target text corresponding to the to-be-processed text; if the analysis content and the answer content corresponding to the at least two groups of model predicted texts are different, at least one group of model predicted texts can be selected from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

[0057] In the above example, if the text to be processed is "1. The following statements about the origin of mankind are correct ( ) A. Mankind is naturally produced; B. Mankind is evolved from chimpanzees; C. Mankind is evolved from modern anthropoids; D. Mankind is evolved from forest apes", the large language model and the multi-modal large model can output their respective analysis and answer content, i.e. the model predicts the text: "### Analysis of option A - Mankind is not naturally produced, but has evolved through a long process, A is wrong. ### Analysis of option B - Chimpanzees are a kind of modern anthropoids, and humans and chimpanzees have a common ancestor, not that humans evolved from chimpanzees, B is wrong. ### Analysis of option C - Modern anthropoids and humans have a common ancestor, not that humans evolved from modern anthropoids, C is wrong. ### Analysis of option D - The common ancestor of modern anthropoids and humans is forest apes, and humans evolved from forest apes, D is correct. The answer is D". By comparing at least two sets of model prediction texts, comparing the option analysis content and the answer content, it is determined whether the at least two sets of model prediction texts match, i.e. whether the analysis content and the answer content are the same. If they are the same, the at least two sets of model prediction texts can form the target text; if there are differences, at least one set of model prediction texts can be selected from the at least two sets of model prediction texts as the target text.

[0058] In summary, based on at least two sets of model prediction texts, the target text is constructed, the accuracy of the target text is improved, and the matching degree between the target text and the text to be processed is improved.

[0059] Further, in the case where the at least two sets of model prediction texts do not match, at least one set of model prediction texts can be selected from the at least two sets of model prediction texts as the target text corresponding to the text to be processed, and in specific implementation: An evaluation model is called to evaluate the at least two sets of model prediction texts according to preset evaluation dimensions, to obtain text evaluation information corresponding to each set of model prediction texts; and at least one set of model prediction texts is selected from the at least two sets of model prediction texts as the target text corresponding to the text to be processed according to the text evaluation information corresponding to each set of model prediction texts.

[0060] Specifically, the evaluation model can be a multi-modal large model. The preset evaluation dimensions include but are not limited to the accuracy dimension, the completeness dimension and the logic dimension of the model prediction text. The text evaluation information can be an evaluation result given by evaluating the at least two sets of model prediction texts respectively, and the text evaluation information can include evaluation content and a comprehensive score, which can be obtained by weighted calculation according to the accuracy score, the completeness score and the logic clarity score.

[0061] Based on this, an evaluation model is invoked to evaluate at least two sets of model-predicted texts according to preset evaluation dimensions, obtaining text evaluation information for each set of model-predicted texts in terms of text accuracy, completeness, and logical clarity. Based on the text evaluation information for each set of model-predicted texts, at least one set of model-predicted texts is selected as the target text for processing from the at least two sets of model-predicted texts according to text accuracy, completeness, and logical clarity.

[0062] Continuing with the previous example, such as Figure 2 As shown, when at least two sets of model-predicted text include "Image 1 + Reply A", "Image 1 + Reply B", and "Text 1 + Reply C", we can first determine whether the three sets of model-predicted text are the same. If they are the same, all of them are retained as target texts. If they are different, a scoring model is used to comprehensively score the three sets of model-predicted text in terms of accuracy, completeness, and logical clarity, obtaining scores of 0.9, 0.8, and 0.5 respectively. Then, based on the scores, the model-predicted text "Reply A" with the higher score is selected as the target text.

[0063] In summary, based on the text evaluation information corresponding to each set of model-predicted texts, at least one set of model-predicted texts is selected as the target text corresponding to the text to be processed from at least two sets of model-predicted texts according to text accuracy, completeness, and logical clarity, thereby improving the accuracy of the target text.

[0064] Furthermore, when selecting the target text from at least two sets of model-predicted texts based on the text evaluation information corresponding to each set of model-predicted texts, a threshold comparison can be used to select at least one set of model-predicted texts as the target text corresponding to the text to be processed. Specifically: A preset scoring threshold is determined, and an evaluation score is extracted from the text evaluation information corresponding to each set of model-predicted texts. By comparing the evaluation score corresponding to each set of model-predicted texts with the preset scoring threshold, at least one set of model-predicted texts is selected from the at least two sets of model-predicted texts as the target text corresponding to the text to be processed.

[0065] Therefore, the preset scoring threshold can be a score threshold set according to actual needs, and this embodiment does not impose any limitations on it. The preset scoring threshold is determined, and the evaluation score is extracted from the text evaluation information corresponding to each group of model-predicted texts. By comparing the evaluation score corresponding to each group of model-predicted texts with the preset scoring threshold, at least one group of model-predicted texts with an evaluation score greater than the preset scoring threshold is selected as the target text corresponding to the text to be processed from at least two groups of model-predicted texts.

[0066] Taking the above example, the scoring model is used to score the three groups of model predicted texts in terms of accuracy, completeness, and logical clarity, and scores of 0.9, 0.8, and 0.5 are obtained respectively. In the case of a preset scoring threshold of 0.7, scores of 0.9 and 0.8 greater than 0.7 are selected, that is, "reply A" and "reply B" are selected as the target text.

[0067] In summary, by comparing the evaluation scores of each group of model predicted texts with the preset scoring threshold, at least one group of model predicted texts is selected from at least two groups of model predicted texts as the target text corresponding to the to-be-processed text, ensuring a high matching degree between the target text and the to-be-processed text. One embodiment of the present specification provides a text processing method. The to-be-processed text is determined, and the to-be-processed image associated with the to-be-processed text is obtained. At least two types of model input contents are determined based on the to-be-processed text and the to-be-processed image, and the at least two types of model input contents are input into a content processing model respectively to obtain at least two groups of model predicted texts, thereby improving the text processing efficiency and accuracy. The target text corresponding to the to-be-processed text is constructed based on the at least two groups of model predicted texts, and the text data pair is generated based on the to-be-processed text and the target text, thereby quickly generating high-quality text data pairs.

[0068] The following describes the text processing method provided by the present specification in conjunction with the accompanying Figure 3 The text processing method provided by the present specification is further described by taking the application of the text processing method in the title data pair generation as an example. Among them, Figure 3 FIG. 1 shows a process flow diagram of a text processing method provided by one embodiment of the present specification, which specifically includes the following steps.

[0069] Step 302: Determine the to-be-processed text, and obtain the to-be-processed image associated with the to-be-processed text.

[0070] The text processing method provided by one embodiment of the present specification can be applied to the training data set construction of a large model. The training data set includes a plurality of data pairs. Given an input content of a large model, the large model needs to give an output content. The input content and the output content together constitute a data pair. However, the construction of high-quality data pairs generally has a high cost. One embodiment of the present specification introduces the concept of "multiple large model collaborative distillation", which more efficiently and comprehensively solves key problems such as high cost, poor quality, and limited diversity in data construction.

[0071] A data pair in the training data set can be obtained by data processing on the to-be-processed text and the to-be-processed image associated with the to-be-processed text. In the case where the constructed training data set is a question and answer data set, the to-be-processed text can be a question text. The question text includes but is not limited to a subject title text and a natural language question text. In the case where the question text is a mathematical question, the to-be-processed text is a mathematical question text "When Xiaohong uses a protractor to measure the angle, one side of the angle is aligned with the outer circle scale of the protractor, and the other side is aligned with the inner circle scale. What is the angle ( ). A. 20°; B. 50°; C. 80°; D. 100°". The to-be-processed image can be an image obtained by image acquisition on a test paper recording the to-be-processed text, and the image only contains the mathematical question text.

[0072] Step 304: The to-be-processed text is used as text model input content, the to-be-processed image is used as image model input content, and the to-be-processed text and the to-be-processed image are combined to obtain mixed text and image model input content.

[0073] Step 306: Determine a large language model matched with the text model input content, and determine a multi-modal large model matched with the image model input content and the mixed text and image model input content.

[0074] Step 308: Input the text model input content into the large language model to obtain first model predicted text.

[0075] In actual application, multiple large models can be used to solve mathematical problems. For a large language model, the input can only be text; for a multi-modal large model, the input can be text, picture, or mixed text and picture. Regardless of the form of the output, the output is in the form of text.

[0076] The first model predicted text obtained by inputting the text model input content into the large language model can be "### Determine the relationship between the outer circle scale and the inner circle scale of the protractor: The degrees of the outer circle scale and the inner circle scale of the protractor are opposite numbers, that is, the sum of the outer circle scale 50° and the inner circle scale 30° is 50°+30°=80°. ### Calculate the angle: Because one side of the angle is aligned with the outer circle scale 50° and the other side is aligned with the inner circle scale 30°, the angle is 80°. The answer is C." Step 310: Input the image model input content and the mixed text and image model input content into the multi-modal large model respectively to obtain second model predicted text corresponding to the image model input content and third model predicted text corresponding to the mixed text and image model input content.

[0077] Input the image-based model input and the image-text hybrid model input respectively into the multimodal large model to obtain the second model prediction text corresponding to the image-based model input: "### Clarify the relationship between the inner and outer scales of the protractor: The sum of the readings of the same scale on the inner and outer scales of the protractor is \(180^{\circ}\). For example, the outer scale reading corresponding to the inner scale \(30^{\circ}\) is \(180 - 30=150^{\circ}\). ### Calculate the degree of the angle: Given that one side is aligned with the outer scale \(50^{\circ}\) and the other side is aligned with the inner scale \(30^{\circ}\), then the outer scale reading corresponding to the other side is \(180 - 30 = 150^{\circ}\). The degree of the angle is the difference between the two sides' readings on the outer scale, i.e. \(150 - 50 = 100^{\circ}\). The answer is D”, and the third model prediction text corresponding to the input content of the image-text hybrid model is “### Characteristics of the protractor scale: The sum of the inner and outer scales of the protractor is \(180^{\circ}\). ### Calculate the degree of an angle: Given that one side of the angle is aligned with the outer scale \(50^{\circ}\) and the other side is aligned with the inner scale \(30^{\circ}\), then the degree of the angle is the outer scale minus the inner scale, i.e. \(50^{\circ}-30^{\circ}=20^{\circ}\). The answer is A”.

[0078] In practical applications, such as Figure 4 As shown, based on the given text and image to be processed, at least three types of model input content can be determined. Taking the image to be processed as input to the image-type model (multimodal large model) yields response A from model A. Taking the text to be processed as input to the text-type model (large language model) yields response B from model B (predicted text by the second model). Combining the text and image to be processed yields the image-text hybrid model input "image + text," which is then input to model C (multimodal large model) to obtain response C from model C (predicted text by the third model).

[0079] Step 312: If there is a match between the text predicted by the first model, the text predicted by the second model, and the text predicted by the third model, then the text predicted by the first model, the text predicted by the second model, and the text predicted by the third model are taken as the target text corresponding to the text to be processed.

[0080] Collaborative filtering is performed on responses A, B, and C. It determines whether responses A, B, and C are identical. If they are all identical, they are all retained as target text. The target text and the text to be processed form a data pair.

[0081] Step 314: In the case of no match between the first model predicted text, the second model predicted text, and the third model predicted text, select at least one set of model predicted text from the first model predicted text, the second model predicted text, and the third model predicted text as the target text corresponding to the to-be-processed text.

[0082] In the case of determining that the reply A, the reply B, and the reply C are not the same, then the reply A, the reply B, and the reply C are submitted to a trained scoring model for evaluation. The scoring model scores each reply according to preset evaluation criteria (such as correctness, logic, and completeness), and finally selects a reply with a score greater than a certain threshold (such as 0.6) as the target text.

[0083] In practical applications, the scoring model can be a multi-modal large language model, and a system (instruction) is set: you are playing a strict teacher, please objectively evaluate the following text reply from correctness, logic, and completeness. The ranges are 0-1 points respectively.

[0084] 1. Correctness: This dimension measures the factual accuracy of the reply content. The higher the score, the more reliable the content.

[0085] 0.8-1.0 points: All information in the reply is completely correct, without errors or misleading statements.

[0086] 0.5-0.7 points: Most of the information in the reply is correct, but there may be some minor errors that do not affect the core understanding, or the expression is not precise enough.

[0087] 0.0-0.4 points: The reply contains serious factual errors, or the information provided is completely inconsistent with the question.

[0088] 2. Logic: This dimension focuses on whether the structure of the reply is clear, the argument is reasonable, and the expression is smooth.

[0089] 0.8-1.0 points: The organization of the reply is clear, the logic is rigorous, the argument is well-organized, and the language is fluent and easy to understand.

[0090] 0.5-0.7 points: The logic of the reply is basically clear, but there may be some inconsistencies, or the language expression is slightly harsh, and the reader needs to think a little bit.

[0091] 0.0-0.4 points: The reply logic is chaotic, the sentence is not smooth, the argument is contradictory, and it is difficult to understand the core idea.

[0092] 3. Completeness: This dimension evaluates whether the reply fully addresses the user's question and provides sufficient information.

[0093] 0.8-1.0 points: The reply not only directly answers the question, but also considers potential follow-up questions, providing comprehensive information or solutions.

[0094] 0.5-0.7 points: The reply provides basic information and addresses the main problem, but is not comprehensive, missing some important details or supplementary explanations.

[0095] 0.0-0.4 points: The reply is too brief, providing only a simple answer and failing to adequately address the problem or omitting key information.

[0096] The processed image and reply A are input into the scoring model, which can give scores in three dimensions and the final score "correctness: 0.6; logic: 0.8; completeness: 0.7; final score: (0.6+0.8+0.7) / 3=0.7". After scoring model scoring reply A, reply B and reply C respectively, it is determined that reply A is 0.7 points, reply B is 0.5 points, and reply C is 0.3 points. Finally, it is determined that reply A is the target text, which forms a data pair with the processed text.

[0097] In summary, the embodiment adopts a multi-model collaborative verification method, and multiple large models independently generate problem-solving replies based on their respective knowledge systems from different angles. The multi-modal input enriches the data types, allowing the input of various modalities such as text and images, making the constructed dataset not limited to single text form and covering a wider range of application scenarios, improving the model's generalization ability to handle complex and multi-source information. The data quality of the data pair is improved, and the efficiency and accuracy of generating data pairs are also improved.

[0098] Corresponding to the method embodiments described above, the present specification also provides text processing device embodiments, Figure 5 A structural diagram of a text processing device according to an embodiment of the present specification is shown. As shown in the figure, Figure 5 The device includes: A determination module 502 configured to determine a processed text and obtain a processed image associated with the processed text; An input module 504 configured to determine at least two types of model input content based on the processed text and the processed image, and input the at least two types of model input content into a content processing model respectively to obtain at least two groups of model predicted texts; A generation module 506 configured to construct a target text corresponding to the processed text based on the at least two groups of model predicted texts, and generate a text data pair based on the processed text and the target text.

[0099] In an optional embodiment, the determining module 502 is further configured to: determine an original image associated with the text to be processed; determine a text region corresponding to the text to be processed in the original image; extract the text region in the original image to obtain the image to be processed.

[0100] In an optional embodiment, the input module 504 is further configured to: input the text to be processed as text model input content, and input the image to be processed as image model input content; combine the text to be processed and the image to be processed to obtain mixed text and image model input content.

[0101] In an optional embodiment, the input module 504 is further configured to: determine a large language model matched with the text model input content, and determine a multi-modal large model matched with the image model input content and the mixed text and image model input content; use the large language model and the multi-modal large model as the content processing model.

[0102] In an optional embodiment, the input module 504 is further configured to: input the text model input content into the large language model to obtain first model predicted text; input the image model input content and the mixed text and image model input content into the multi-modal large model respectively to obtain second model predicted text corresponding to the image model input content and third model predicted text corresponding to the mixed text and image model input content; use the first model predicted text, the second model predicted text, and the third model predicted text as the at least two groups of model predicted text.

[0103] In an optional embodiment, the generation module 506 is further configured to: determine whether the at least two groups of model predicted text match; if yes, use the at least two groups of model predicted text as the target text corresponding to the text to be processed; if no, select at least one group of model predicted text from the at least two groups of model predicted text as the target text corresponding to the text to be processed.

[0104] In an optional embodiment, the generation module 506 is further configured to: call the evaluation model to evaluate the at least two groups of model predicted texts respectively according to preset evaluation dimensions, to obtain text evaluation information corresponding to each group of model predicted texts; select at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text according to the text evaluation information corresponding to each group of model predicted texts.

[0105] An optional embodiment, the generation module 506 is further configured to: determine a preset score threshold, and extract an evaluation score from the text evaluation information corresponding to each group of model predicted texts; select at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text by comparing the evaluation score corresponding to each group of model predicted texts with the preset score threshold.

[0106] An embodiment of the present specification provides a text processing apparatus. The text processing apparatus determines a to-be-processed text, and obtains a to-be-processed image associated with the to-be-processed text. At least two types of model input contents are determined based on the to-be-processed text and the to-be-processed image, and the at least two types of model input contents are input into a content processing model respectively to obtain at least two groups of model predicted texts, thereby improving the text processing efficiency and accuracy. A target text corresponding to the to-be-processed text is constructed based on the at least two groups of model predicted texts, and a text data pair is generated based on the to-be-processed text and the target text, thereby quickly generating a high-quality text data pair.

[0107] The above is a schematic scheme of a text processing apparatus according to an embodiment of the present specification. It should be noted that the technical scheme of the text processing apparatus belongs to the same concept as the technical scheme of the text processing method described above, and the details of the technical scheme of the text processing apparatus that are not described in detail can be referred to the description of the technical scheme of the text processing method.

[0108] Figure 6 A structural block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 through a bus 630, and a database 650 is used to save data.

[0109] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or combinations of such networks, such as the Internet. The access device 640 can include one or more of any type of network interface (for example, a network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC).

[0110] In one embodiment of the present specification, the above-mentioned components of the computing device 600 and other components not shown in the Figure 6 may be connected to each other, such as through a bus. It should be understood that Figure 6 The computing device structure diagram shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Those skilled in the art can add or replace other components as needed.

[0111] The computing device 600 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (for example, a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, and the like), a mobile phone (for example, a smartphone), a wearable computing device (for example, a smart watch, smart glasses, and the like), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 can also be a mobile or stationary server.

[0112] The processor 620 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the text processing method described above.

[0113] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the text processing method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the text processing method.

[0114] An embodiment of the present specification further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions realize the steps of the text processing method when executed by a processor.

[0115] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the text processing method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the text processing method.

[0116] An embodiment of the present specification further provides a computer program product, which includes a computer program or instructions, and the computer program or instructions realize the steps of the text processing method when executed by a processor.

[0117] The above is a schematic scheme of the computer program product of the embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the text processing method described above belong to the same concept, and the details of the technical scheme of the computer program product that are not described in detail can be referred to the description of the technical scheme of the text processing method.

[0118] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order described in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0119] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of patent practice. For example, according to the patent practice in some regions, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0120] It should be noted that, for the foregoing method embodiments, in order to facilitate description, each is described as a combination of a series of acts, but those skilled in the art should appreciate that the embodiments of the present specification are not limited by the order of the described acts, because according to the embodiments of the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should appreciate that the embodiments described in the specification are all preferred embodiments, and the acts and modules involved are not necessarily essential to the embodiments of the present specification.

[0121] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0122] The preferred embodiments of the present specification disclosed above are only used to help explain the present specification. The alternative embodiments do not describe all the details and limit the invention to the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and use the present specification. The present specification is limited only by the claims and their full scope and equivalents.

Claims

1. A text processing method characterized by, include: Identify the text to be processed and obtain the image associated with the text to be processed; Based on the text to be processed and the image to be processed, at least two types of model input content are determined, and the at least two types of model input content are respectively input into the content processing model to obtain at least two sets of model predicted text; Based on the predicted text from the at least two sets of models, the target text corresponding to the text to be processed is constructed, and text data pairs are generated based on the text to be processed and the target text.

2. The text processing method according to claim 1, characterized by, The step of obtaining the image to be processed associated with the text to be processed includes: Determine the original image associated with the text to be processed; Determine the text region corresponding to the text to be processed in the original image; The text region is extracted from the original image to obtain the image to be processed.

3. The text processing method of claim 1, wherein, The process of determining at least two types of model input content based on the text to be processed and the image to be processed includes: The text to be processed is used as the input content of the text class model, and the image to be processed is used as the input content of the image class model; The text to be processed and the image to be processed are combined to obtain the input content of the image-text hybrid model.

4. The text processing method according to claim 3, characterized by, Before inputting the at least two types of model input content into the content processing model, the method further includes: Determine a large language model that matches the input content of the text-based model, and determine a multimodal large model that matches the input content of the image-based model and the input content of the image-text hybrid model; The large language model and the multimodal large model are used as the content processing model.

5. The text processing method of claim 4, wherein, The step of inputting the at least two types of model input content into the content processing model to obtain at least two sets of model predicted text includes: The text class model input content is input into the large language model to obtain the first model predicted text; The input content of the image class model and the input content of the image-text hybrid class model are respectively input into the multimodal large model to obtain the second model predicted text corresponding to the input content of the image class model and the third model predicted text corresponding to the input content of the image-text hybrid model. The first model-predicted text, the second model-predicted text, and the third model-predicted text are used as the at least two sets of model-predicted text.

6. The text processing method of claim 1, wherein, The construction of the target text corresponding to the text to be processed based on the predicted text of the at least two sets of models includes: Determine whether the predicted texts from at least two sets of models match; If so, the at least two sets of model-predicted texts shall be used as the target text corresponding to the text to be processed; If not, select at least one set of model-predicted texts from the at least two sets of model-predicted texts as the target text corresponding to the text to be processed.

7. The text processing method of claim 6, wherein, The step of selecting at least one set of model-predicted texts as the target text corresponding to the text to be processed from the at least two sets of model-predicted texts includes: The evaluation model is invoked, and the at least two sets of model-predicted texts are evaluated according to the preset evaluation dimensions to obtain the text evaluation information corresponding to each set of model-predicted texts. According to the text evaluation information corresponding to each group of model predicted texts, at least one group of model predicted texts is selected from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

8. The text processing method of claim 7, wherein, The selecting at least one group of model predicted texts from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text according to the text evaluation information corresponding to each group of model predicted texts comprises: A preset score threshold is determined, and an evaluation score is extracted from the text evaluation information corresponding to each group of model predicted texts; By comparing the evaluation score corresponding to each group of model predicted texts with the preset score threshold, at least one group of model predicted texts is selected from the at least two groups of model predicted texts as the target text corresponding to the to-be-processed text.

9. A text processing apparatus characterized by comprising: Comprise: A determination module is configured to determine a to-be-processed text and obtain a to-be-processed image associated with the to-be-processed text; An input module is configured to determine at least two types of model input contents based on the to-be-processed text and the to-be-processed image, and input the at least two types of model input contents into a content processing model respectively to obtain at least two groups of model predicted texts; A generation module is configured to construct a target text corresponding to the to-be-processed text based on the at least two groups of model predicted texts, and generate a text data pair based on the to-be-processed text and the target text.

10. A computing device, comprising: Comprise: A memory and a processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, and the computer executable instructions realize the steps of the text processing method in any one of claims 1 to 8 when executed by the processor.

11. A computer readable storage medium characterized by, It has computer executable instructions, and the computer executable instructions realize the steps of the text processing method in any one of claims 1 to 8 when executed by the processor.

12. A computer program product, characterised in that, It comprises computer programs or instructions, and the computer programs or instructions realize the steps of the text processing method in any one of claims 1 to 8 when executed by the processor.

Images