Text processing method and text processing model training method

Abstract

Embodiments of the present specification provide a text processing method and a text processing model training method, wherein the text processing method comprises: obtaining a to-be-processed text, and determining language feature information of an associated target language. The language feature information is used to identify the target language and control the language type of the generated target text, so as to ensure that the generated target text corresponding to the to-be-processed text matches the target language. Language control information of the target language is generated based on the language feature information and the to-be-processed text. The target text corresponding to the to-be-processed text is generated based on the control guidance of the to-be-processed text and the language control information. Based on the control guidance of the language control information, it can be ensured that the generated target text is the content of the target language, and the target text meets the reading and understanding of the target text by a user whose main language is the target language. In a multi-language scenario, it has controllable generation capability to generate multi-language content.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to text processing methods and text processing model training methods. Background Technology

[0002] With the acceleration of globalization and the rapid development of internet technology, the demand for cross-language communication is increasing. Traditional text processing methods and tools are often limited to mechanical processing of texts word by word and mechanical interpretation of user-provided text content, which makes it difficult to meet users' needs for generating high-quality, language-specific content.

[0003] While some existing language processing models based on machine learning or deep learning can control the language of generated content, these models often lack sufficient flexibility and controllability when handling multilingual scenarios. They frequently fail to accurately generate text content tailored to the specific language, hindering reading and impacting the user experience. Therefore, a more effective text processing method is urgently needed to address these issues. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a text processing method. One or more embodiments of this specification also relate to a text processing apparatus, a text processing model training method, a text processing model training apparatus, a question-answering task processing method, a question-answering task processing apparatus, a model training platform, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a text processing method is provided, comprising:

[0006] Obtain the text to be processed and determine the linguistic features associated with the target language;

[0007] Based on the language feature information and the text to be processed, language control information for the target language is generated;

[0008] Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated.

[0009] According to a second aspect of the embodiments of this specification, a text processing apparatus is provided, comprising:

[0010] The acquisition module is configured to acquire the text to be processed and determine the language feature information associated with the target language;

[0011] The first generation module is configured to generate language control information of the target language based on the language feature information and the text to be processed;

[0012] The second generation module is configured to generate the target text corresponding to the text to be processed based on the control guidance of the text to be processed and the language control information.

[0013] According to a third aspect of the embodiments of this specification, a text processing model training method is provided, comprising:

[0014] Determine an initial text processing model, and training samples for fine-tuning the initial text processing model, and determine the parameter training samples and feature metric samples included in the training samples;

[0015] The target features associated with the target language are labeled in the target model layer of the initial text processing model using the feature measurement samples, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples;

[0016] The initial text processing model is updated to a new text processing model based on the target features and the language control parameters.

[0017] According to a fourth aspect of the embodiments of this specification, a text processing model training apparatus is provided, comprising:

[0018] Determine an initial text processing model, and training samples for fine-tuning the initial text processing model, and determine the parameter training samples and feature metric samples included in the training samples;

[0019] The target features associated with the target language are labeled in the target model layer of the initial text processing model using the feature measurement samples, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples;

[0020] The initial text processing model is updated to a new text processing model based on the target features and the language control parameters.

[0021] According to a fifth aspect of the embodiments of this specification, a question-answering task processing method is provided, including:

[0022] Obtain the target question and determine the linguistic features associated with the target language;

[0023] Based on the language feature information and the target question, language control information for the target language is generated;

[0024] Based on the target question and the control guidance of the language control information, the target answer corresponding to the target question is generated.

[0025] According to a sixth aspect of the embodiments of this specification, a question-answering task processing apparatus is provided, comprising:

[0026] Obtain the target question and determine the linguistic features associated with the target language;

[0027] Based on the language feature information and the target question, language control information for the target language is generated;

[0028] Based on the target question and the control guidance of the language control information, the target answer corresponding to the target question is generated.

[0029] According to a seventh aspect of the embodiments of this specification, a model training platform is provided, including a request interface unit, a model training unit, and a response unit; the request interface unit is used to receive a model training request, wherein the model training request includes model information of an initial text processing model;

[0030] The model training unit is used to determine the initial text processing model based on the model attribute information, and to train the initial text processing model to obtain a text processing model, wherein the target model layer of the text processing model is labeled with target features and language control parameters;

[0031] The response unit is used to output the target model.

[0032] According to an eighth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0033] Memory and processor;

[0034] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described text processing method.

[0035] According to a ninth aspect of an embodiment of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the text processing method described above.

[0036] According to a tenth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described text processing method.

[0037] This specification provides a text processing method according to one embodiment, which acquires the text to be processed and determines the language feature information associated with the target language. The language feature information is used to identify the target language and control the language type of the generated target text, ensuring that the generated target text matches the target language of the text to be processed. Language control information for the target language is generated based on the language feature information and the text to be processed. Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated. The control guidance based on the language control information ensures that the generated target text is in the target language, conforming to the reading and understanding requirements of users whose primary language is the target language. In multilingual scenarios, it has the controllable generation capability of multilingual content. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a text processing method provided in one embodiment of this specification;

[0039] Figure 2 This is a flowchart illustrating the processing procedure of a question-and-answer method provided in one embodiment of this specification;

[0040] Figure 3 This is a flowchart illustrating a text processing method provided in one embodiment of this specification.

[0041] Figure 4 This is a schematic diagram of the structure of a text processing device provided in one embodiment of this specification;

[0042] Figure 5 This is a flowchart illustrating a text processing model training method provided in one embodiment of this specification;

[0043] Figure 6 This is a schematic diagram of the structure of a text processing model training device provided in one embodiment of this specification;

[0044] Figure 7 This is a flowchart illustrating a question-and-answer task processing method provided in one embodiment of this specification;

[0045] Figure 8 This is a schematic diagram of the structure of a question-and-answer task processing device provided in one embodiment of this specification;

[0046] Figure 9 This is a schematic diagram of the structure of a model training platform provided in one embodiment of this specification;

[0047] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0048] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0049] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0051] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0052] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0053] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0054] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0055] Transformer: A model framework. The Transformer model is a multi-layer model. In a general sense, a Transformer layer contains two sub-layers, which are mainly composed of a Self-Attention Network (SAN) and a Feedforward Network (FFN), and combine residual connections and layer normalization.

[0056] Sparse Autoencoders (SAEs) are a special type of autoencoder designed to decompose the intermediate representations within large language models into high-dimensional sparse representations, which are represented by linear combinations of learned feature directions.

[0057] Steering Vector: A steering vector can be viewed as a specific direction of an intermediate representation within the text processing model. Specifically, we first obtain the steering vector from the residual stream of some prompts, and then add these steering vectors to the residual stream of a second prompt. This gives the output of the second forward pass properties inherited from the first forward pass, thus achieving our specific purpose.

[0058] Current research on the multilingual capabilities of large language models mainly focuses on explaining their performance, while relatively little work has been done to study the control of their multilingual capabilities from the perspective of internal mechanisms. Given the extreme importance of the multilingual capabilities of large language models, existing methods primarily enhance the model's output control across all languages ​​by analyzing end-to-end results and modifying the proportion of data corresponding to different languages. However, this approach lacks a solid theoretical foundation and struggles to provide intuitive and easily understandable justifications. Furthermore, simultaneously optimizing multiple languages ​​makes it difficult to improve the performance of all languages ​​at the same time. Therefore, this specification proposes a text processing method in one embodiment to address the aforementioned problems.

[0059] This specification provides a text processing method, and also relates to a text processing apparatus, a text processing model training method, a text processing model training apparatus, a question-answering task processing method, a question-answering task processing apparatus, a model training platform, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0060] See Figure 1 , Figure 1 A flowchart of a text processing method according to an embodiment of this specification is shown, which specifically includes the following steps:

[0061] Step 102: Obtain the text to be processed and determine the language feature information associated with the target language.

[0062] The target language refers to the language type of the text to be processed, i.e., the language itself. The text to be processed refers to the text content that needs to be translated, answered, explained, or otherwise processed. Language feature information is associated with the target language and is used to guide the language type of the generated target text.

[0063] Language feature information includes target features and language control parameters. Target features refer to the language-related features existing in the target model layer, serving to identify the target language. Target features exhibit monolingualism, being the features with the highest relevance to the target language among multiple features included in the target model layer. Language control parameters are the pre-calculated average activation values ​​of features specific to the target language, used to guide the generation of target text in the target language and achieve language type control of the target text.

[0064] When processing the text to be processed, it is necessary to first determine the target features included in the language feature information and the pre-calculated language control parameters. The target features are used to identify the target language. During the processing of the text to be processed, the target features act as control signals to determine whether to use the language control parameters, thereby outputting the target text in the target language. The language control parameters are used to determine the language type of the generated content. Before generating the target text, the language type of the target text is detected. If the target text to be output matches the target language, it means that the target text is content in the language type corresponding to the target language, so the language control parameters do not need to be used. If the target text to be output does not match the target language, it means that the target text is not the answer in the language type corresponding to the target language, so the language control parameters can be used to control the target language of the target text. Subsequently, the target text corresponding to the text to be processed can be generated based on the target features and the language control parameters.

[0065] Step 104: Generate language control information for the target language based on the language feature information and the text to be processed.

[0066] After obtaining the text to be processed and determining the language feature information of the associated target language, the text to be processed can be processed based on the language control information to generate language control information. The language type of the generated target text can be determined based on the feature parameters contained in the language feature information. By modifying the feature parameters, that is, by superimposing the language control information on the feature parameters, the target text corresponding to the target language can be generated.

[0067] After determining the linguistic features of the target language, language control information can be generated based on these features and the text to be processed. This information controls the language type of the generated target text to match the target language. Feature parameters associated with the target features of the text to be processed are calculated. Based on these parameters, it is determined whether the target text to be generated matches the target language. If a match is found, the target text can be directly generated based on the feature parameters. If a mismatch is found, language control parameters are superimposed on the feature parameters to obtain language control information. Finally, the target text is generated based on this language control information.

[0068] Step 106: Based on the control guidance of the text to be processed and the language control information, generate the target text corresponding to the text to be processed.

[0069] For example, the text to be processed can be text content that needs interpretation. If the user providing the text's primary language is Chinese, they typically expect to obtain an interpretation text in Chinese, i.e., the target text. Therefore, when processing the text, it is known that the user's primary language is Chinese, so Chinese needs to be used as the target language. Furthermore, language feature information is determined for language intervention in the target text corresponding to the text to be processed. Based on the language feature information and the text to be processed, language control information for the corresponding target language is calculated. Then, based on the control guidance of the language control information, the target text in the target language is generated.

[0070] In this embodiment, the text to be processed is acquired, and the language feature information of the associated target language is determined. The language feature information is used to identify the target language and control the language type of the generated target text, ensuring that the generated target text matches the target language. Language control information for the target language is generated based on the language feature information and the text to be processed. Guided by the control information, the target text corresponding to the text to be processed is generated. This control guidance ensures that the generated target text is in the target language, conforming to the reading and understanding requirements of users whose primary language is the target language. In multilingual scenarios, it possesses the controllable generation capability for multilingual content.

[0071] The language feature information includes target features and language control parameters. Target features serve to identify the target language. Language control parameters are used to guide the generation of target text in the target language, thereby controlling the language type of the target text.

[0072] In one optional embodiment of this specification, generating language control information for the target language based on the language feature information and the text to be processed includes:

[0073] Based on the text to be processed and the target features in the language feature information, feature parameters of the target language are generated; if the feature parameters are greater than a parameter threshold, language control information is generated based on the feature parameters and language control parameters in the language feature information; if the feature parameters are less than or equal to the parameter threshold, the feature parameters are used as the language control information.

[0074] Language feature information includes target features and language control parameters. Target features refer to the features related to the target language existing in the target model layer, and they serve to identify the target language within this layer. Target features exhibit monolingualism, meaning they are the features with the highest relevance to the target language among the multiple features included in the target model layer. Language control parameters are the pre-calculated average activation values ​​of features specific to the target language, used to guide the generation of target text in the target language and achieve language type control of the target text.

[0075] Based on the target features in the text to be processed and the language feature information, feature parameters for the target language are generated. If the feature parameters are greater than a threshold, it indicates that the language type of the target text to be generated is not the target language, and the language type of the generated text needs to be controlled and adjusted to the target language. That is, language control information is generated based on the feature parameters and language control parameters, which allows the target text to be generated in the target language. If the feature parameters are less than or equal to the threshold, it indicates that the language type of the target text to be generated is the target language, and there is no need to control the language type of the generated text. The feature parameters can be used as language control information to directly generate the target text, and the generated target text is the answer corresponding to the target language.

[0076] For example, the parameter threshold can be 0, or any value that can be used to judge the feature parameters. The parameter threshold can be set according to the actual calculation needs and question-answering scenario. After calculating the feature activation value of the language A associated with the text to be processed, the feature activation value is used as the feature parameter. It is judged whether the feature activation value is greater than 0. If the feature activation value is greater than 0, the feature activation value of language B occupies the main control position in the generation of the target text. This means that under the influence of the feature activation value of language B, the generated target text may be content of the language type corresponding to language B. At this time, it is necessary to intervene in the feature activation value of language A. Language control information, namely, SteeringVector, is added to the feature activation value of language A to generate language control information. Then, under the influence of the language control information, the text processing model generates target text that matches the target language corresponding to language A. If the feature activation value is less than 0, the feature activation value of language A occupies the main control position in the generation of the target text. This means that under the influence of the feature activation value of language A, the generated target text may be content that matches the target language corresponding to language A. At this time, it is not necessary to intervene in the feature activation value of language A, and the target text can be generated directly. The generated target text is the target text that matches the target language.

[0077] In summary, by using parameter thresholds to determine the language type control capability of feature parameters, it is possible to determine whether the feature parameters need to be modified, thereby controlling the language type of the generated target text.

[0078] In practical applications, text processing models can be used when processing text.

[0079] In one optional embodiment of this specification, obtaining the text to be processed and determining the language feature information of the associated target language includes: inputting the text to be processed into a text processing model, and determining the language feature information of the associated target language in the text processing model.

[0080] Language control information for the target language is generated based on language feature information and the text to be processed, including: using the target model layer of the text processing model to generate language control information for the target language based on language feature information and the text to be processed.

[0081] Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated, including: using the output layer of the text processing model, based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated.

[0082] When processing the text to be processed to generate the target text, the text processing work can be delegated to a text processing model. The text to be processed is input into the text processing model, and after processing, the target text is generated. The text to be processed is input into the text processing model, and the linguistic feature information associated with the target language is determined within the text processing model. Using the target model layer of the text processing model, language control information for the target language is generated based on the linguistic feature information and the text to be processed. Using the output layer of the text processing model, guided by the control information of the text to be processed and the linguistic control information, the target text corresponding to the text to be processed is generated.

[0083] Text processing models can answer target questions by encoding, understanding, analyzing, responding to, and decoding them. The target model layer can be a model layer in the text processing model that includes language-related features. Language control information consists of the average activation values ​​of features pre-calculated for the text to be processed. This information guides the text processing model to output the target text, thus controlling the language type of the target text.

[0084] When processing text, the target language text is input into the text processing model. Based on the target language, language feature information associated with the target model layer in the text processing model is determined. This language feature information includes target features and pre-calculated language control parameters. Target features are used to identify the text to be processed. During the text processing model's processing of the text, the target features act as control signals to determine whether to use language control parameters, thus controlling the text processing model to output target text in the target language. Language control parameters control the model's output language type. Before the text processing model outputs the target text, it detects the language type of the target text. If the target text to be output matches the target language, it indicates that the target text is the answer corresponding to the target language's language type, and therefore, language control parameters are not needed. If the target text to be output does not match the target language, it indicates that the target text is not the answer corresponding to the target language's language type, and therefore, language control parameters can be used to control the target language of the target text. Subsequently, based on the target features contained in the language control information and the control parameters, the target text corresponding to the text to be processed can be generated, ensuring that the target text corresponds to the target language.

[0085] By fine-tuning the text processing model, language control parameters can be added to control the language type of the output content. Furthermore, target features with high relevance to the target language can be identified and labeled in the target model layer of the text processing model, thereby enabling control over the language type of the output content of the text processing model.

[0086] In one optional embodiment of this specification, fine-tuning training of the text processing model includes:

[0087] The training samples include parameter training samples and feature measurement samples; the feature measurement samples are used to label the target features associated with the target language in the target model layer of the initial text processing model, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples; the initial text processing model is updated to a text processing model based on the target features and language control parameters.

[0088] Training samples are used to fine-tune the initial text processing model, parameter training samples are used to calculate language control parameters corresponding to the target language in the manner of model training, and feature measurement samples are used to label target features related to the target language in the target model layer by means of relevance comparison.

[0089] The training samples include parameter training samples and feature measurement samples. Using the feature measurement samples, target features associated with the target language are labeled among multiple features in the target model layer of the initial text processing model through relevance calculation; that is, target features with high relevance to the target language are identified. The parameter training samples are then used to calculate the language control parameters corresponding to the target language in the target model layer. Based on the target features and language control parameters, the initial text processing model is updated to a new text processing model. The language control parameters are associated with the text processing model to control the language type of the output content of the text processing model. Target features are then labeled in the target model layer of the initial text processing model to identify the specific target language.

[0090] In practical applications, besides modifying a single layer (the target model layer), multiple model layers of the text processing model can be modified simultaneously. The target model layer can contain at least one model layer from the initial text processing model. When the target model layer contains at least two model layers, the aforementioned model fine-tuning operations only need to be performed on each model layer. Since single-layer operations can reduce the feature activation values ​​of the target language for most tokens to 0 in most cases, multi-layer operations will only affect a small number of tokens and will not significantly alter the normal distribution of activation values. More importantly, this adaptive dynamic adjustment can further improve the effectiveness of language control in the text processing model.

[0091] In summary, by using feature measurement samples to label target features associated with the target language in the target model layer of the initial text processing model, and by using parameter training samples to calculate the language control parameters corresponding to the target model layer, the text processing model can be fine-tuned. Based on the target features and language control parameters, the language type of the output content of the text processing model can be controlled.

[0092] When selecting target features in the feature set associated with the target model layer of the initial text processing model, the target features can be determined by calculating the features in the feature set and at least one language sample corresponding to the feature measurement sample.

[0093] In one optional embodiment of this specification, the target features associated with the target language are labeled in the target model layer of the initial text processing model using feature measurement samples, including:

[0094] In the target model layer of the initial text processing model, a feature set is determined; based on the feature set and the feature measurement sample, feature-related information corresponding to at least one language sample corresponding to the feature measurement sample is calculated; target feature information corresponding to the target language is determined based on at least two feature-related information, and target features are determined based on the target feature information.

[0095] A feature set refers to a collection of at least one sub-feature associated with the target model layer, where each sub-feature corresponds to a language type with high relevance. The feature measurement sample contains language samples of at least one language type, and feature-related information refers to the relevance between each sub-feature in the feature set and each language sample. Target feature information refers to the relevance between the target language and each sub-feature in the feature set.

[0096] In the target model layer of the initial text processing model, a feature set is determined. The feature set contains at least one sub-feature. Based on the feature set and feature measurement samples, feature-related information corresponding to at least one language sample corresponding to the feature measurement sample is calculated to obtain the relevant information between each sub-feature and each language sample. Target feature information with high relevance to the target language is determined based on at least two feature-related information sets, and the features corresponding to the target feature information are used as target features.

[0097] For example, a sparse autoencoder can be used to analyze the multilingual capabilities of the initial text processing model, selecting the target feature with the highest relevance to the target language from multiple features (feature sets) included in the target model layer. The target feature is a specific linguistic feature of the target language, serving to identify the target language. Marking the target feature in the target model layer allows it to be used subsequently to assist the text processing model in controlling the language type of the output content.

[0098] In summary, by determining the target feature information corresponding to the target language based on at least two feature-related information, and by determining the target feature based on the target feature information, the target feature can be selected from the feature set that is highly relevant to the target language. This can then be used to identify the target language when using the text processing model.

[0099] Furthermore, when determining the characteristic information corresponding to at least one language sample, we can first determine the monolingual features in the feature set, and then calculate the feature-related information in combination with at least one language sample. The specific implementation is as follows:

[0100] In one optional embodiment of this specification, calculating feature-related information corresponding to at least one language sample corresponding to the feature metric sample based on the feature set and the feature metric sample includes:

[0101] Determine at least one sub-feature in the feature set, and determine at least one language sample containing the target language based on the feature measurement sample; input the at least one language sample into the target model layer to obtain at least one language parameter; calculate the feature-related information corresponding to the at least one language sample based on the at least one sub-feature and the at least one language parameter.

[0102] The feature measurement samples include language samples of the target language, and may also include language samples corresponding to multiple other languages. Language parameters refer to the language activation values ​​corresponding to the language samples.

[0103] At least one sub-feature is determined from the feature set, and at least one language sample containing the target language is determined based on the feature measurement sample. The at least one language sample is input into the target model layer to obtain at least one language parameter, that is, the language activation value corresponding to each language sample is calculated. Based on the at least one sub-feature and at least one language parameter, the feature-related information corresponding to the at least one language sample is calculated to determine the correlation between each single language feature and its corresponding specific language. The feature-related information can be calculated using the following formulas (1), (2) and (3), where the following formulas (1), (2) and (3) use the sub-feature s in the feature set and the language parameter D in the at least one language parameter. U and D I For example, calculate the feature-related information of the target language U.

[0104]

[0105] When analyzing the multilingual capabilities of an initial text processing model using a sparse autoencoder, the monolinguality of sub-features in the feature set can be measured to identify a target feature with high relevance to the target language. When selecting a target feature from the feature set using feature measurement samples, the sample must first be input into the initial text processing model to calculate the language activation value for each language sample, resulting in a language activation value set, i.e., a parameter set D = {D...}, consisting of at least one language parameter. I ,…,D J For each sub-feature in the feature set, each sub-feature is denoted as sub-feature s. The feature-related information of the sub-feature is calculated using the formulas (1), (2), and (3) above. The feature-related information represents the difference in the average activation values ​​of sub-feature s in the target language U (language sample) and other languages, and can represent the correlation between sub-feature s and the target language U (language sample). s (x) represents the feature activation value of sub-feature s; D represents the set of language activation values; D U Represents all language activation values ​​for language U; This represents the average language activation value of feature s in language U; This represents the average language activation value of feature s in other languages; The difference between the average language activation values ​​of feature s in a specific language U and other languages ​​actually represents the correlation between language U and feature s. By calculating the feature-related information between at least one sub-feature and at least one language parameter, a sub-feature with high correlation to each language sample can be determined, and finally, the target feature corresponding to the target language can be determined from the feature set.

[0106] In summary, by calculating feature-related information for at least one language sample based on monolingual features, we can further determine the relevant information between monolingual features and each language sample based on the determined monolingual features.

[0107] When calculating the language control parameters corresponding to the target model layer of a text processing model, a large number of samples can be used to calculate the average activation value, which can then be used as the language control layer parameter.

[0108] In one optional embodiment of this specification, the language control parameters corresponding to the target model layer are calculated using parameter training samples, including:

[0109] In the parameter training samples, determine the target language sample corresponding to the target language and the initial language sample corresponding to the initial language; calculate the target language control parameters of the target language sample in the target model layer and the initial language control parameters of the initial language sample in the target model layer; calculate the language control parameters based on the target language control parameters and the initial language control parameters.

[0110] The target language sample refers to the corpus containing the target language, while the initial language sample refers to the corpus containing the initial language. The target language and the initial language are different language types. The target language control parameters are the average activation values ​​of the target language corresponding to the target language, calculated based on the target language samples. The initial language control parameters are the average activation values ​​of the initial language corresponding to the initial language, calculated based on the initial language samples.

[0111] In the parameter training samples, the target language samples corresponding to the target language and the initial language samples corresponding to the initial language are determined. The activation value of each target language subsample in the target language samples is calculated, and the average activation value of the target language corresponding to the target language samples is calculated based on the number of target language subsamples. The average activation value of the target language is used as the target language control parameter of the target model layer. Similarly, the activation value of each initial language subsample in the initial language samples is calculated, and the average activation value of the initial language corresponding to the initial language samples is calculated based on the number of initial language subsamples. The average activation value of the initial language is used as the initial language control parameter of the target model layer. The language control parameter is calculated based on the target language control parameter and the initial language control parameter. That is, the target language control parameter is used as the subtrahend and the initial language control parameter is used as the minuend to calculate the language control parameter. The calculation formula of the language control layer parameter is as follows (4):

[0112]

[0113] Where sv represents the language control parameter, i.e., the Steering Vector; Represents the initial language sample; Represents a target language sample; a L (x) represents the average activation value of the training sample x for parameters in the target model layer L.

[0114] In summary, calculating language control parameters based on target language control parameters and initial language control parameters facilitates the control of the language type, i.e., the language, of the text processing model's output content when using the text processing model subsequently.

[0115] When calculating the feature parameters of the target features associated with the text to be processed, the feature parameter function can be used for calculation.

[0116] In one optional embodiment of this specification, generating feature parameters of the target language based on target features in the text to be processed and language feature information includes:

[0117] Determine the feature weights and feature thresholds of the target features; using the target model layer, calculate the feature parameters of the target features associated with the text to be processed based on the feature parameter function of the associated feature weights and feature thresholds.

[0118] Feature parameters can be calculated using a sparse autoencoder. The feature weight of the target feature refers to the weight assigned to the target feature when calculating activation values. Feature weights typically refer to the strength or importance of edges connecting the input layer and hidden layers, and between the hidden layer and the output layer. The feature threshold is the bias of the target feature; the bias term allows the model to adapt to any potential shifts in the data. For each neuron in the hidden layer, there is a corresponding bias value, which is added to the weighted sum of the neuron's inputs. The feature parameter function is used to calculate feature activation values ​​based on the target feature, its feature weights, and the feature threshold. The calculated feature activation values ​​are the feature parameters.

[0119] Identify the target features in the linguistic feature information, and determine the feature weights and thresholds of the target features. Using the target model layer, calculate the feature parameters of the target features associated with the text to be processed based on the feature parameter function of the associated feature weights and thresholds. The feature parameters are the feature activation values ​​of the target features, which are used to determine the language type of the output content of the text processing model.

[0120] For example, when the text to be processed is "explain the word 'red'", the target language is Chinese. The text can be input into a text processing model, where the target model layer determines the target features matching Chinese, as well as the pre-calculated language control parameters for Chinese, i.e., the feature average activation value, or Steering Vector. When calculating the feature parameters of the target features, the text processing model can use the feature parameter function f(x·) to calculate the feature activation values ​​of the text to be processed, i.e., the feature parameters.

[0121] The characteristic parameter function f(x·) is given by the following formula (5):

[0122] f(x·) := ReLU(W enc x·+b enc (5)

[0123] Among them, W enc b represents the feature weights; enc represents the feature threshold, i.e., the bias. RELU represents the calculation of the activation value; x· represents the text to be processed. The feature parameters can be calculated using the above formula (5).

[0124] In summary, the feature parameter function based on the associated feature weights and feature thresholds calculates the feature parameters of the target features associated with the text to be processed, enabling the detection of the language type of the content to be output before the target text is output, thus facilitating language type control.

[0125] This specification provides a text processing method according to one embodiment, which acquires the text to be processed and determines the language feature information associated with the target language. The language feature information is used to identify the target language and control the language type of the generated target text, ensuring that the generated target text matches the target language of the text to be processed. Language control information for the target language is generated based on the language feature information and the text to be processed. Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated. The control guidance based on the language control information ensures that the generated target text is in the target language, conforming to the reading and understanding requirements of users whose primary language is the target language. In multilingual scenarios, it has the controllable generation capability of multilingual content.

[0126] The following is in conjunction with the appendix Figure 2 Taking the question-answering method provided in this manual as an example of its application in fine-tuning training of a large language model, the question-answering method will be further explained. Figure 2 The flowchart of a question-and-answer method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0127] Step 202: Determine the parameter training samples and feature measurement samples included in the large language model training samples.

[0128] During fine-tuning training of the initial large language model, parameter training samples are used to calculate language control parameters, i.e., the Steering Vector. Feature metric samples are used to measure the monolinguality of features in the target model layer of the initial large language model, and are used to determine a target feature with high relevance to the target language from the feature set in the target model layer.

[0129] Step 204: Determine the feature set in the target model layer of the initial large language model, determine at least one sub-feature in the feature set, and determine at least one language sample containing the target language based on the feature measurement sample.

[0130] Step 206: Input at least one language sample into the target model layer to obtain at least one language parameter, and calculate the feature-related information corresponding to at least one language sample based on at least one sub-feature and at least one language parameter.

[0131] Step 208: Determine the target features corresponding to the target language based on at least two feature-related information.

[0132] Before measuring feature metrics, each language sample is first input into an initial large language model for processing to determine the language parameters of each language sample, i.e., the language activation value, ultimately obtaining a set of language activation values. This set of language activation values ​​corresponds to multiple languages ​​including the target language. At this point, it is necessary to calculate the difference between a specific language (the target language) and all other languages ​​on any single feature; this is the feature-related information. Feature-related information represents the relevance between a sub-feature in the feature set and at least one language sample. Based on the feature-related information, a target feature with a high relevance to the target language can be identified within the feature set. By labeling the target feature, the target language can be identified. Then, when using the large language model obtained through model fine-tuning for question answering, the target language can be identified based on the target feature.

[0133] Step 210: Determine the target language sample corresponding to the target language and the initial language sample corresponding to the initial language in the parameter training samples.

[0134] Step 212: Calculate the target language control parameters of the target language sample at the target model layer, and calculate the initial language control parameters of the initial language sample at the target model layer.

[0135] Step 214: Calculate the language control parameters based on the target language control parameters and the initial language control parameters.

[0136] When calculating the language control parameters, the target language samples corresponding to the target language and the initial language samples corresponding to the initial language are determined in the parameter training samples. If the output of the large language model needs to be switched from the initial language to the target language, the initial language samples need to be input into the initial large language model to calculate the average activation value of the initial language in the target model layer, i.e., the initial language control parameters; and the target language samples need to be input into the initial large language model to calculate the average activation value of the target language in the target model layer, i.e., the target language control parameters. The target language average activation value is used as the minuend, and the initial language average activation value is used as the subtrahend to calculate the language control parameters, the Steering Vector.

[0137] It should be noted that the target feature determination process in steps 204-208 and the language control parameter calculation process in steps 210-210 do not have a specific execution order; they can be executed in parallel or sequentially.

[0138] Step 216: Update the initial large language model into a large language model based on the target features and language control parameters.

[0139] In the target model layer of the initial large language model, target features are labeled and language control parameters are recorded to update the initial large language model to the target large language model.

[0140] Step 218: Input the target question for the corresponding target language into the large language model.

[0141] In practical applications, when using a large language model to solve a target problem, the target question in the target language can be directly input into the large language model for processing. For example... Figure 3 As shown, when processing the target question, the output content of the French text needs to be changed to the output content of the Japanese text, so that the language type of the target answer output by the large language model is French.

[0142] Step 220: Determine the target features and language control parameters associated with the target model layer in the large language model based on the target language.

[0143] Among them, target features are used to identify the target language, and language control parameters are used to control the language type output by the model.

[0144] Step 222: Calculate the feature parameters of the target problem associated with the target features using the target model layer, and generate the answer control parameters based on the feature parameters and language control parameters.

[0145] Sparse autoencoders are used to determine French features, with French characteristics serving as the target features. For the target question, the feature activation values ​​(i.e., feature parameters) are calculated along the target feature dimension. Answer control parameters are determined by testing these feature parameters. Specifically, the system checks if the feature activation value is greater than 0. If it is, the language control parameters are added to the activation value to obtain the answer control parameters. If the activation value is not greater than 0, the activation value is directly used as the answer control parameters, and subsequent processing is performed.

[0146] Step 224: After generating the answer features corresponding to the target question in the processing layer of the large language model, the answer features are converted into the target answer in the corresponding target language through the output layer of the large language model according to the answer control parameters.

[0147] Before the large language model outputs the target answer, the language type of the target answer can be controlled using answer control parameters. The processing layer of the large language model includes a Transformer module. Answer features are obtained through a feedforward neural network and a multi-head attention mechanism within the Transformer module. Then, if the feature activation values ​​are determined to be greater than 0, language control parameters are added to these activation values ​​to control the language type of the large language model's output content. For example, if the original output was in Japanese, it might be changed to French. The language control parameters can control whether the language type of the large language model's output content is French or Japanese.

[0148] Furthermore, besides modifying a single layer (the target model layer), multiple model layers of a large language model can be modified simultaneously. This requires only performing the aforementioned model fine-tuning operations on each model layer. Since single-layer operations can reduce the feature activation values ​​of the target language for most tokens to 0 in most cases, multi-layer operations will only affect a small number of tokens and will not significantly alter the normal distribution of activation values. More importantly, this adaptive dynamic adjustment can further improve the effectiveness of language control in a large language model.

[0149] In summary, a sparse autoencoder is used to determine specific features (target features) corresponding to the target language in the target model layer. Simultaneously, the Steering Vector is improved by using specific features as control signals to achieve automatic selection (use or disuse) of the Steering Vector and complementary combination of multiple Steering Vector layers. It eliminates the need to explicitly specify the output language type in the prompt corresponding to the large language model; instead, it achieves language control by modifying the feature activation values ​​(feature parameters) within the large model, while also avoiding language mixing in the model. This improves the performance of the large language model, making the target answer output by the large language model easier to understand.

[0150] Corresponding to the above method embodiments, this specification also provides embodiments of a question-and-answer device. Figure 4 A schematic diagram of the structure of a text processing apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes:

[0151] The acquisition module 402 is configured to acquire the text to be processed and determine the language feature information associated with the target language;

[0152] The first generation module 404 is configured to generate language control information of the target language based on the language feature information and the text to be processed.

[0153] The second generation module 406 is configured to generate target text corresponding to the text to be processed based on the control guidance of the text to be processed and the language control information.

[0154] In an optional embodiment, the first generation module 404 is further configured to:

[0155] Generate feature parameters of the target language based on the target features in the text to be processed and the language feature information;

[0156] If the feature parameter is greater than the parameter threshold, the language control information is generated based on the feature parameter and the language control parameter in the language feature information;

[0157] If the feature parameter is less than or equal to the parameter threshold, the feature parameter is used as the language control information.

[0158] In an optional embodiment, the second generation module 406 is further configured to:

[0159] The text to be processed is input into a text processing model, and the language feature information associated with the target language is determined in the text processing model;

[0160] The process of generating language control information for the target language based on the language feature information and the text to be processed includes:

[0161] Using the target model layer of the text processing model, language control information of the target language is generated based on the language feature information and the text to be processed;

[0162] The control guidance based on the text to be processed and the language control information generates the target text corresponding to the text to be processed, including:

[0163] Using the output layer of the text processing model, the target text corresponding to the text to be processed is generated based on the control guidance of the text to be processed and the language control information.

[0164] In an optional embodiment, the second generation module 406 is further configured to:

[0165] Determine the parameter training samples and feature measurement samples included in the training samples;

[0166] The target features associated with the target language are labeled in the target model layer of the initial text processing model using the feature measurement samples, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples.

[0167] The initial text processing model is updated to the text processing model based on the target features and the language control parameters.

[0168] In an optional embodiment, the second generation module 406 is further configured to:

[0169] Determine the feature set in the target model layer of the initial text processing model;

[0170] Based on the feature set and the feature measurement sample, calculate the feature-related information corresponding to at least one language sample corresponding to the feature measurement sample;

[0171] The target feature information corresponding to the target language is determined based on at least two feature-related information, and the target feature is determined based on the target feature information.

[0172] In an optional embodiment, the second generation module 406 is further configured to:

[0173] At least one sub-feature is determined from the feature set, and at least one language sample containing the target language is determined based on the feature measurement sample;

[0174] The at least one language sample is input into the target model layer to obtain at least one language parameter;

[0175] Calculate the feature-related information corresponding to the at least one language sample based on the at least one sub-feature and the at least one language parameter.

[0176] In an optional embodiment, the second generation module 406 is further configured to:

[0177] In the parameter training samples, determine the target language sample corresponding to the target language, and the initial language sample corresponding to the initial language;

[0178] Calculate the target language control parameters of the target language sample in the target model layer, and calculate the initial language control parameters of the initial language sample in the target model layer;

[0179] The language control parameters are calculated based on the target language control parameters and the initial language control parameters.

[0180] In an optional embodiment, the first generation module 404 is further configured to:

[0181] The target feature in the language feature information is determined, and the feature weight and feature threshold of the target feature are determined;

[0182] The feature parameters associated with the target features of the text to be processed are calculated based on the feature parameter function that associates the feature weights and the feature thresholds.

[0183] This specification provides a text processing method according to one embodiment, which acquires the text to be processed and determines the language feature information associated with the target language. The language feature information is used to identify the target language and control the language type of the generated target text, ensuring that the generated target text matches the target language of the text to be processed. Language control information for the target language is generated based on the language feature information and the text to be processed. Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated. The control guidance based on the language control information ensures that the generated target text is in the target language, conforming to the reading and understanding requirements of users whose primary language is the target language. In multilingual scenarios, it has the controllable generation capability of multilingual content.

[0184] The above is an illustrative scheme of a text processing device according to this embodiment. It should be noted that the technical solution of this text processing device and the technical solution of the above-described text processing method belong to the same concept. For details not described in detail in the technical solution of the text processing device, please refer to the description of the technical solution of the above-described text processing method.

[0185] See Figure 5 , Figure 5 A flowchart of a text processing model training method according to an embodiment of this specification is shown, which specifically includes the following steps:

[0186] Step 502: Determine the initial text processing model and the training samples for fine-tuning the initial text processing model, and determine the parameter training samples and feature measurement samples included in the training samples;

[0187] Step 504: Using the feature measurement samples, label the target features associated with the target language in the target model layer of the initial text processing model, and use the parameter training samples to calculate the language control parameters corresponding to the target model layer;

[0188] Step 506: Update the initial text processing model to a text processing model based on the target features and the language control parameters.

[0189] In practical applications, training samples are used to fine-tune the initial text processing model, parameter training samples are used to calculate language control parameters corresponding to the target language in a model training manner, and feature measurement samples are used to label target features related to the target language in the target model layer by comparing relevance. The training samples include parameter training samples and feature measurement samples. Using feature measurement samples, target features related to the target language are labeled among multiple features contained in the target model layer of the initial text processing model by calculating relevance; that is, target features with high relevance to the target language are identified. The parameter training samples are used to calculate the language control parameters corresponding to the target language in the target model layer. Based on the target features and language control parameters, the initial text processing model is updated to a new text processing model. The language control parameters are associated with the text processing model to control the language type of the output content of the text processing model. Target features are labeled in the target model layer of the initial text processing model to identify specific target languages. Without configuring prompts for the text processing model, it is ensured that the target answer output by the text processing model is in the target language, which is suitable for users whose primary language is the target language to read and understand the target answer. In multilingual scenarios, text processing models have the ability to controllably generate multilingual content.

[0190] In practical applications, besides modifying a single layer (the target model layer), multiple model layers of the text processing model can be modified simultaneously. The target model layer can contain at least one model layer from the initial text processing model. When the target model layer contains at least two model layers, the aforementioned model fine-tuning operations only need to be performed on each model layer. Since single-layer operations can reduce the feature activation values ​​of the target language for most tokens to 0 in most cases, multi-layer operations will only affect a small number of tokens and will not significantly alter the normal distribution of activation values. More importantly, this adaptive dynamic adjustment can further improve the effectiveness of language control in the text processing model.

[0191] In one optional embodiment of this specification, after updating the initial text processing model to a text processing model based on target features and language control parameters, the method further includes:

[0192] Determine the target text obtained by processing the user-input text using the text processing model;

[0193] The system receives feedback from the user regarding the target text and fine-tunes the text processing model based on the feedback.

[0194] In practical applications, when a user inputs the text to be processed into the text processing model and obtains the target text output by the text processing model, the user can generate feedback information based on the target text output by the text processing model and use the feedback information to further fine-tune the text processing model.

[0195] In one optional embodiment of this specification, fine-tuning the text processing model based on feedback information includes:

[0196] If the feedback information is evaluation information for the target text, the text processing model is fine-tuned based on the evaluation information;

[0197] If the feedback information is reference information for the target text, the text processing model is fine-tuned based on the reference information.

[0198] In practical applications, when a user inputs the text to be processed into a text processing model and obtains the target text output by the model, the user can generate evaluation information based on the target text output. This evaluation assesses the text processing model's ability to process the text and uses the feedback information to further fine-tune the model. Users can also provide reference information—texts with a high degree of similarity to the text to be processed—after determining the target text output by the model. This reference information is then used as feedback to further fine-tune the text processing model, improving its text processing capabilities.

[0199] Corresponding to the above method embodiments, this specification also provides embodiments of a question-and-answer device. Figure 6 A schematic diagram of a text processing model training device according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:

[0200] The determination module 602 is configured to determine an initial text processing model, training samples for fine-tuning the initial text processing model, and to determine the parameter training samples and feature measurement samples included in the training samples.

[0201] The labeling module 604 is configured to label the target features associated with the target language in the target model layer of the initial text processing model using the feature measurement samples, and to calculate the language control parameters corresponding to the target model layer using the parameter training samples;

[0202] The update module 606 is configured to update the initial text processing model to a text processing model based on the target features and the language control parameters.

[0203] In an optional embodiment, the update module 606 is further configured to:

[0204] Determine the target text obtained by processing the user-input text using the text processing model;

[0205] The system receives feedback from the user regarding the target text and fine-tunes the text processing model based on the feedback.

[0206] In an optional embodiment, the update module 606 is further configured to:

[0207] If the feedback information is evaluation information for the target text, the text processing model is fine-tuned based on the evaluation information;

[0208] If the feedback information is reference information for the target text, the text processing model is fine-tuned based on the reference information.

[0209] In summary, the training samples consist of parameter training samples and feature measurement samples. Using the feature measurement samples, target features associated with the target language are labeled among multiple features in the target model layer of the initial text processing model through relevance calculation; that is, target features with high relevance to the target language are identified. Furthermore, language control parameters corresponding to the target language are calculated in the target model layer using the parameter training samples. Based on the target features and language control parameters, the initial text processing model is updated to a new text processing model. The language control parameters are associated with the text processing model to control the language type of the output content. Target features are labeled in the target model layer of the initial text processing model to identify specific target languages. Without configuring prompts for the text processing model, it is possible to ensure that the target answer output by the text processing model is in the target language, which is suitable for users whose primary language is the target language to read and understand the target answer. In multilingual scenarios, the text processing model has the controllable generation capability to output multilingual content.

[0210] The above is an illustrative scheme of a text processing model training device according to this embodiment. It should be noted that the technical solution of this text processing model training device and the technical solution of the text processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the text processing model training device, please refer to the description of the technical solution of the text processing model training method described above.

[0211] See Figure 7 , Figure 7 A flowchart of a question-answering task processing method according to an embodiment of this specification is shown, which specifically includes the following steps:

[0212] Step 702: Obtain the target question and determine the linguistic features associated with the target language;

[0213] Step 704: Generate language control information for the target language based on the language feature information and the target question;

[0214] Step 706: Based on the control guidance of the target question and the language control information, generate the target answer corresponding to the target question.

[0215] In practical applications, the target language refers to the language type of the target question, i.e., the language, used to specify that the target question should be answered in the target language. When processing the target question, it is necessary to first determine the target features contained in the language feature information and the pre-calculated language control parameters. The target features are used to identify the target language. During the processing of the target question, the target features act as control signals to determine whether to use the language control parameters, thereby outputting the target answer in the target language. The language control parameters are used to determine the language type of the generated content. Before generating the target answer, they are used to detect the language type of the target answer. If the target answer to be output matches the target language, it means that the target answer is content in the language type corresponding to the target language, and therefore the language control parameters do not need to be used. If the target answer to be output does not match the target language, it means that the target answer is not an answer in the language type corresponding to the target language, and therefore the language control parameters can be used to control the target language of the target answer. After determining the language feature information associated with the target language, language control information can be generated based on the language feature information and the target question to control the language type of the generated target answer to be the target language. The system calculates the feature parameters associated with the target features of the target question, and determines whether the target answer to be generated matches the target language based on the feature parameters. If they match, the target answer can be generated directly based on the feature parameters. If they do not match, language control parameters can be superimposed on the feature parameters to obtain language control information, and then the target answer can be generated based on the language control information.

[0216] In summary, the process involves acquiring the target question and determining the linguistic features associated with the target language. These linguistic features are used to identify the target language and control the language type of the generated target answer, ensuring that the generated answer matches the target language. Based on the linguistic features and the target question, language control information for the target language is generated. Guided by the target question and the language control information, the target answer corresponding to the target question is generated. This guidance ensures that the generated answer is in the target language, facilitating reading and comprehension for users whose primary language is the target language. In multilingual scenarios, the system possesses controllable generation capabilities for multilingual content.

[0217] Corresponding to the above method embodiments, this specification also provides embodiments of a question-answering task processing device. Figure 8 A schematic diagram of a question-and-answer task processing apparatus according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0218] The acquisition module 802 is configured to acquire the target question and determine the language feature information associated with the target language;

[0219] The generation module 804 is configured to generate language control information for the target language based on the language feature information and the target question;

[0220] The control module 806 is configured to generate a target answer corresponding to the target question based on the control guidance of the target question and the language control information.

[0221] In summary, the process involves acquiring the target question and determining the linguistic features associated with the target language. These linguistic features are used to identify the target language and control the language type of the generated target answer, ensuring that the generated answer matches the target language. Based on the linguistic features and the target question, language control information for the target language is generated. Guided by the target question and the language control information, the target answer corresponding to the target question is generated. This guidance ensures that the generated answer is in the target language, facilitating reading and comprehension for users whose primary language is the target language. In multilingual scenarios, the system possesses controllable generation capabilities for multilingual content.

[0222] The above is an illustrative scheme of a question-and-answer task processing device according to this embodiment. It should be noted that the technical solution of this question-and-answer task processing device and the technical solution of the question-and-answer task processing method described above belong to the same concept. For details not described in detail in the technical solution of the question-and-answer task processing device, please refer to the description of the technical solution of the question-and-answer task processing method described above.

[0223] Figure 9 A schematic diagram of a model training platform according to an embodiment of this specification is shown. The model training platform includes a request interface unit, a model training unit, and a response unit. The request interface unit is used to receive a model training request, wherein the model training request includes model information of an initial text processing model.

[0224] The model training unit is used to determine the initial text processing model based on the model attribute information, and to train the initial text processing model to obtain a text processing model, wherein the target model layer of the text processing model is labeled with target features and language control parameters;

[0225] The response unit is used to output the target model.

[0226] Furthermore, the model training platform also includes a data receiving unit.

[0227] The data receiving unit is used to receive training samples input by the user and send the training samples to the model training unit;

[0228] The model training unit is further configured to train the initial text processing model based on the training samples to obtain the trained text processing model.

[0229] Furthermore, the model training platform also includes a model library, wherein the model library stores multiple candidate text processing models;

[0230] The model training unit is also used to determine the initial text processing model from the model library based on the model attribute information.

[0231] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0232] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0233] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0234] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0235] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above method.

[0236] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method.

[0237] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0238] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above method.

[0239] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0240] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to in the description of the technical solution of the above method.

[0241] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0242] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0243] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0244] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0245] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A text processing method, comprising: Obtain the text to be processed and determine the linguistic features associated with the target language; Based on the language feature information and the text to be processed, language control information for the target language is generated; Based on the control guidance of the text to be processed and the language control information, the target text corresponding to the text to be processed is generated.

2. The text processing method according to claim 1, wherein generating language control information of the target language based on the language feature information and the text to be processed includes: Generate feature parameters of the target language based on the target features in the text to be processed and the language feature information; If the feature parameter is greater than the parameter threshold, the language control information is generated based on the feature parameter and the language control parameter in the language feature information; If the feature parameter is less than or equal to the parameter threshold, the feature parameter is used as the language control information.

3. The text processing method according to claim 1, wherein obtaining the text to be processed and determining the language feature information associated with the target language includes: The text to be processed is input into a text processing model, and the language feature information associated with the target language is determined in the text processing model; The process of generating language control information for the target language based on the language feature information and the text to be processed includes: Using the target model layer of the text processing model, language control information of the target language is generated based on the language feature information and the text to be processed; The control guidance based on the text to be processed and the language control information generates the target text corresponding to the text to be processed, including: Using the output layer of the text processing model, the target text corresponding to the text to be processed is generated based on the control guidance of the text to be processed and the language control information.

4. The text processing method according to claim 3, wherein the fine-tuning training of the text processing model includes: Determine the parameter training samples and feature measurement samples included in the training samples; The target features associated with the target language are labeled in the target model layer of the initial text processing model using the feature measurement samples, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples. The initial text processing model is updated to the text processing model based on the target features and the language control parameters.

5. The text processing method according to claim 4, wherein labeling the target features associated with the target language in the target model layer of the initial text processing model using the feature measurement samples comprises: Determine the feature set in the target model layer of the initial text processing model; Based on the feature set and the feature measurement sample, calculate the feature-related information corresponding to at least one language sample corresponding to the feature measurement sample; The target feature information corresponding to the target language is determined based on at least two feature-related information, and the target feature is determined based on the target feature information.

6. The text processing method according to claim 5, wherein calculating feature-related information corresponding to at least one language sample corresponding to the feature measurement sample based on the feature set and the feature measurement sample includes: At least one sub-feature is determined from the feature set, and at least one language sample containing the target language is determined based on the feature measurement sample; The at least one language sample is input into the target model layer to obtain at least one language parameter; Calculate the feature-related information corresponding to the at least one language sample based on the at least one sub-feature and the at least one language parameter.

7. The text processing method according to claim 4, wherein calculating the language control parameters corresponding to the target model layer using the parameter training samples includes: In the parameter training samples, determine the target language sample corresponding to the target language, and the initial language sample corresponding to the initial language; Calculate the target language control parameters of the target language sample in the target model layer, and calculate the initial language control parameters of the initial language sample in the target model layer; The language control parameters are calculated based on the target language control parameters and the initial language control parameters.

8. The text processing method according to claim 2, wherein generating feature parameters of the target language based on the text to be processed and the target features in the language feature information comprises: The target feature in the language feature information is determined, and the feature weight and feature threshold of the target feature are determined; The feature parameters associated with the target features of the text to be processed are calculated based on the feature parameter function that associates the feature weights and the feature thresholds.

9. A method for training a text processing model, comprising: Determine an initial text processing model, and training samples for fine-tuning the initial text processing model, and determine the parameter training samples and feature metric samples included in the training samples; The target features associated with the target language are labeled in the target model layer of the initial text processing model using the feature measurement samples, and the language control parameters corresponding to the target model layer are calculated using the parameter training samples; The initial text processing model is updated to a new text processing model based on the target features and the language control parameters.

10. The text processing model training method according to claim 9, further comprising, after updating the initial text processing model to a text processing model based on the target features and the language control parameters: Determine the target text obtained by processing the user-input text using the text processing model; The system receives feedback from the user regarding the target text and fine-tunes the text processing model based on the feedback.

11. The text processing model training method according to claim 10, wherein fine-tuning the text processing model based on the feedback information includes: If the feedback information is evaluation information for the target text, the text processing model is fine-tuned based on the evaluation information; If the feedback information is reference information for the target text, the text processing model is fine-tuned based on the reference information.

12. A question-answering task processing method, comprising: Obtain the target question and determine the linguistic features associated with the target language; Based on the language feature information and the target question, language control information for the target language is generated; Based on the target question and the control guidance of the language control information, the target answer corresponding to the target question is generated.

13. A model training platform, comprising a request interface unit, a model training unit, and a response unit; The request interface unit is used to receive model training requests, wherein... The model training request includes model information for the initial text processing model; The model training unit is used to determine the initial text processing model based on the model attribute information, and to train the initial text processing model to obtain a text processing model, wherein the target model layer of the text processing model is labeled with target features and language control parameters; The response unit is used to output the target model.

14. The model training platform according to claim 13 further includes a data receiving unit. The data receiving unit is used to receive training samples input by the user and send the training samples to the model training unit; The model training unit is further configured to train the initial text processing model based on the training samples to obtain the trained text processing model.

15. The model training platform according to claim 13 further includes a model library, wherein, The model library stores multiple candidate text processing models; The model training unit is also used to determine the initial text processing model from the model library based on the model attribute information.

16. A computing device, comprising: Memory and processor; The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 12.

17. A computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

18. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images