Model training method, data processing method, system, electronic device, storage medium, and computer program product

Abstract

The present disclosure relates to the fields of large model technology and data query technology, and provides a model training method, a data processing method, a system, an electronic device, a storage medium, and a computer program product. The model training method comprises: acquiring a first training dataset and a second training dataset, wherein the first training dataset is monolingual instruction data described in a source language, and the second training dataset is monolingual instruction data described in a target language and cross-lingual instruction data described in the source language and the target language, respectively; using the first training dataset to train an initial language model to generate an intermediate language model; and using the second training dataset to train the intermediate language model to generate a target language model. The present disclosure solves the technical problem that a large language model trained on the basis of a source language exhibits poor performance when applied to a target language.

Description

Model training methods, data processing methods, systems, electronic devices, storage media, and computer program products Technical Field

[0001] This disclosure relates to the fields of large model technology and data query technology, and more specifically, to a model training method, data processing method, system, electronic device, storage medium, and computer program product. Background Technology

[0002] Because training data is primarily in English, Large Language Models (LLMs) face significant challenges when handling non-English languages. To address this, related technologies propose improving LLM performance on non-English languages ​​through extensive pre-training on large datasets or supervised fine-tuning (SFT) on instruction-response data pairs. However, these methods for training LLMs on non-English languages ​​require not only substantial amounts of non-English training data and computational resources but also significant manual labor costs.

[0003] Therefore, how to train LLMs to improve their performance on non-English languages ​​at a lower cost has become one of the important technical problems in related fields. Currently, no effective solution has been proposed to address these issues. Summary of the Invention

[0004] This disclosure provides a model training method, data processing method, system, electronic device, storage medium, and computer program product to at least solve the technical problem that large language models trained based on the original language have poor performance in the target language.

[0005] According to one aspect of the present disclosure, a model training method is provided, comprising: acquiring a first training dataset and a second training dataset, wherein the first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in the original language and the target language, respectively; training an initial language model using the first training dataset to generate an intermediate language model; and training the intermediate language model using the second training dataset to generate a target language model, wherein the target language model is used to perform query parsing on query data in the target language to generate a target response in the target language.

[0006] According to one aspect of the present disclosure, a data processing method is also provided, comprising: acquiring query data in a target language; using a target language model to perform query parsing on the query data in the target language to obtain a target response in the target language; wherein the target language model is generated according to the model training method of any one of the above.

[0007] According to one aspect of the present disclosure, a data processing method is also provided, comprising: acquiring Chinese astronomical knowledge query data; using a target language model to query and parse the Chinese astronomical knowledge query data to obtain Chinese astronomical knowledge query responses; wherein the target language model is generated according to the model training method of any one of the above, and the target language model is natively English.

[0008] According to one aspect of the embodiments of this disclosure, a data processing method is also provided, comprising: obtaining a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes: query data in a target language; and returning a data processing response through a second application programming interface, wherein the response data carried in the data processing response includes: a target response in a target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to the model training method of any one of the above.

[0009] According to one aspect of the embodiments of this disclosure, a data processing method is also provided, comprising: acquiring a currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: query data in a target language; responding to the data processing dialogue request, returning a data processing dialogue response, wherein the information carried in the data processing dialogue response includes: a target response in the target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the model training methods described above; and displaying the target response in the target language within a graphical user interface.

[0010] According to one aspect of the present disclosure, a data processing method is also provided, comprising: displaying query data in a target language on the operation interface in response to an input command applied to the operation interface; and displaying a target response in the target language on the operation interface in response to a processing command applied to the operation interface; wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to the model training method of any one of the above.

[0011] According to one aspect of the present disclosure, a data processing system is also provided, comprising: a client configured to send query data in a target language; a server connected to the client and configured to use a target language model to perform query parsing on the query data in the target language to obtain a target response in the target language; the client is further configured to output the target response in the target language; wherein the target language model is generated according to the model training method of any of the above.

[0012] According to one aspect of the present disclosure, an electronic device is also provided, including: a memory storing an executable program; and a processor configured to run the program, wherein the program executes the model training method or the data processing method of any of the above-mentioned methods during runtime.

[0013] According to one aspect of the present disclosure, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute the model training method or the data processing method of any of the above-mentioned methods.

[0014] According to one aspect of the present disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the model training method or the data processing method of any of the above-described embodiments.

[0015] In this embodiment of the disclosure, a first training dataset and a second training dataset are obtained. The first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages. An initial language model is trained using the first training dataset to generate an intermediate language model. The intermediate language model is then trained using the second training dataset to generate a target language model. The target language model is used to parse query data in the target language to generate a target response in the target language.

[0016] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing higher-performance query services in the target language. In other words, this embodiment achieves the goal of training a target language model for processing query data in the target language, thus achieving the technical effect of training a target language model for processing query instructions in the target language at a lower cost. This solves the technical problem of poor performance of large language models trained based on the original language in the target language.

[0017] It is worth noting that the above general description and the following detailed description are merely for illustrative and explanatory purposes and do not constitute a limitation thereof. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0019] Figure 1 is a schematic diagram of an application scenario of a model training method and a data processing method according to an embodiment of the present disclosure;

[0020] Figure 2 is a flowchart of a model training method according to an embodiment of the present disclosure;

[0021] Figure 3 is a schematic diagram of an optional model training process according to an embodiment of the present disclosure;

[0022] Figure 4 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0023] Figure 5 is a flowchart of another data processing method according to an embodiment of the present disclosure;

[0024] Figure 6 is a flowchart of another data processing method according to an embodiment of the present disclosure;

[0025] Figure 7 is a flowchart of another data processing method according to an embodiment of the present disclosure;

[0026] Figure 8 is a flowchart of another data processing method according to an embodiment of the present disclosure;

[0027] Figure 9 is a schematic diagram of a model training device according to an embodiment of the present disclosure;

[0028] Figure 10 is a schematic diagram of the structure of a data processing apparatus according to an embodiment of the present disclosure;

[0029] Figure 11 is a schematic diagram of another data processing apparatus according to an embodiment of the present disclosure;

[0030] Figure 12 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of the present disclosure;

[0031] Figure 13 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of the present disclosure;

[0032] Figure 14 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of the present disclosure;

[0033] Figure 15 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] The technical solution disclosed herein is primarily implemented using large-scale model technology. Here, "large-scale model" refers to a deep learning model with a massive number of parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of parameters. Large-scale models, also known as foundation models, are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. These models are adaptable to a wide range of downstream tasks and exhibit good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0037] It's important to note that in practical applications, large models can be fine-tuned using a small number of samples after pre-training, allowing them to be applied to various tasks. For example, large models can be widely used in Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios for large models include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0038] First, some of the nouns or terms that appear in the description of the embodiments of this disclosure are to be interpreted as follows.

[0039] Second language learning (also known as "second language acquisition") refers to the process by which an individual learns a language other than their native language. Specifically, in this embodiment of the disclosure, second language learning can refer to the process by which an LLM learns another language (such as a non-English) based on their native language (such as English).

[0040] Principle-Parameter Theory (PPT) is a theoretical framework used in linguistics and computational modeling. In PPT, principles refer to shared rules or universal constraints across multiple human languages, such as syntactic structure and grammatical functions. Parameters refer to specific settings allowed based on differences between languages. For example, different languages ​​may have different choices regarding the order of subject and verb; English uses a subject-verb-object order, while Japanese uses a subject-object-verb order. By setting parameters, specific grammatical features in the computational model can be determined. PPT can be used to explain the language learning process, similar to children learning and mastering complex language structures. It can also assist LLM (Language Learning Model) in second language learning. For instance, after learning and mastering basic language principles on predominantly English data, LLMs can adjust parameters to adapt to languages ​​other than English.

[0041] Instruction fine-tuning data: This data is used to fine-tune the parameters of the LLM, enabling it to better conform to the requirements of specific instructions or tasks. The dataset for this instruction fine-tuning data can be multi-task-based, showing the inputs and expected outputs for different instructions, helping the LLM learn how to produce the correct responses in specific contexts.

[0042] Supervised Fine-Tuning (SFT) refers to a method of supervising the parameter fine-tuning of an LLM using instruction fine-tuning data.

[0043] Self-Paced Learning (SPL) is a learning method in which learners actively arrange their learning content and time according to their own learning progress and needs.

[0044] Cross-language transfer: refers to the method of leveraging high-resource capabilities to enhance the capabilities of LLM in other languages, that is, transferring the high-resource language capabilities of LLM to other languages.

[0045] According to embodiments of this disclosure, a model training method and a data processing method are provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] Considering the large number of model parameters in the large model and the limited computing resources of mobile terminals, the method provided in this disclosure can be applied to the application scenario shown in Figure 1, but is not limited thereto. In the operating environment corresponding to the application scenario shown in Figure 1, the large model is deployed in server 10. Server 10 can connect to one or more client devices 20 through a local area network (LAN), a wide area network (WAN), an internet connection, or other types of data networks. The client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. The client devices 20 can interact with users through a graphical user interface to invoke the large model, thereby implementing the method provided in this disclosure.

[0047] In this embodiment, the system comprising a client device and a server can perform the following steps: The client device uploads a first training dataset and a second training dataset to the server, wherein the first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages; the server uses the first training dataset to train an initial language model to generate an intermediate language model; the server uses the second training dataset to train the intermediate language model to generate a target language model, wherein the target language model is used to parse query data in the target language to generate a target response in the target language. Further, the server returns the target language model to the client device, or the server returns the calling interface of the target language model to the client device.

[0048] It should be noted that with the rapid development of high-performance computing units, the methods provided in this disclosure can also be applied to integrated model machines in other application scenarios. In one optional embodiment, the integrated model machine has multiple built-in models. Users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the integrated model machine can then directly call the adjusted model to execute the methods provided in this disclosure. In another optional embodiment, the large integrated model machine has a pre-trained model built-in. The high-performance computing unit built into the integrated model machine can then directly call this model to execute the methods provided in this disclosure.

[0049] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.

[0050] Under the above operating environment, this disclosure provides a model training method as shown in Figure 2. Figure 2 is a flowchart of a model training method according to an embodiment of this disclosure. As shown in Figure 2, the method may include the following steps S21 to S23.

[0051] Step S21: Obtain the first training dataset and the second training dataset. The first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in the original language and the target language, respectively.

[0052] The first training dataset mentioned above can be an instruction dataset described in the original language. This instruction dataset includes multiple data pairs, each pair including an instruction described in the original language and the corresponding correct response. This first training dataset is used for initial training of the model to enable it to process instructions in the context of the original language.

[0053] The second training dataset mentioned above includes the following two parts: monolingual instruction data and cross-lingual instruction data.

[0054] The monolingual instruction data described above is in the target language and is used to directly train the model's instruction processing ability in the target language context. The monolingual instruction data can also include multiple data pairs, each pair containing an instruction described in the target language and the corresponding correct response. This monolingual instruction data can help the model improve its performance in independently understanding and directly responding to instructions described in the target language.

[0055] The aforementioned cross-linguistic instruction data is used to simulate the early stages of second language learning. The model first understands and responds to instructions in the source language, then converts the responses into the target language for output. In this cross-linguistic instruction data, each instruction corresponds to an instruction in the source language, a correct response in the source language, and a correct response in the target language. The data for each instruction is organized into a two-round dialogue: the first round is the response to the instruction, and the second round is the transfer (translation) from the source language to the target language. This cross-linguistic instruction data helps the model maintain consistent comprehension across multiple languages.

[0056] Step S22: Train the initial language model using the first training dataset to generate an intermediate language model.

[0057] The initial language model described above can be a pre-built deep learning model used to process natural language instructions. This initial language model serves as an untrained initial model.

[0058] Using the first training dataset, an initial language model is trained, and its parameters are adjusted to obtain the intermediate language model described above. During this training process, the model learns from data in the first training dataset described in the original language, improving its instruction processing performance in the original language context. In other words, the intermediate language model possesses strong original language instruction processing capabilities. This intermediate language model can serve as the foundation for subsequent cross-language transfer training.

[0059] Step S23: Train the intermediate language model using the second training dataset to generate the target language model. The target language model is used to parse the query data in the target language to generate the target response in the target language.

[0060] Using the second training dataset mentioned above, the intermediate language model is trained, and its parameters are adjusted to obtain the target language model. Since the second training dataset includes monolingual instruction data and cross-lingual instruction data, the training process for the intermediate language model includes: model training for the target language based on monolingual instruction data, and cross-lingual transfer training from the source language to the target language based on cross-lingual instruction data.

[0061] By training on the second training dataset, the aforementioned target language model exhibits strong adaptability and responsiveness to commands in the target language. That is, the target language model can parse query data (including query commands) in the target language and then generate a target response in the target language.

[0062] For example, the original language is English, and the target language is Chinese. Using the model training method provided in this embodiment, an English training dataset is constructed as the first training dataset, a Chinese training dataset and a cross-language instruction dataset combining Chinese and English are constructed as the second training dataset. Furthermore, through preliminary training based on the first training dataset and secondary training based on the second training dataset, a target language model is generated. This target language model can directly process Chinese query instructions and generate Chinese answers, achieving end-to-end Chinese instruction processing without necessarily relying on English as a language intermediary.

[0063] Through steps S21 to S23 described above, this embodiment of the disclosure obtains a first training dataset and a second training dataset. The first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages. The first training dataset is used to train an initial language model to generate an intermediate language model. The second training dataset is used to train the intermediate language model to generate a target language model. The target language model is used to query and parse query data in the target language to generate a target response in the target language.

[0064] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing higher-performance query services in the target language. In other words, this embodiment achieves the goal of training a target language model for processing query data in the target language, thus achieving the technical effect of training a target language model for processing query instructions in the target language at a lower cost. This solves the technical problem of poor performance of large language models trained based on the original language in the target language.

[0065] In one optional embodiment, the model training method further includes the following method steps:

[0066] Step S24: Obtain the first prompt text, wherein the first prompt text is used to prompt a response to the training instructions in the original language;

[0067] Step S25: Construct the first training dataset based on the first prompt text.

[0068] The aforementioned first prompt text is described in natural language in the original language and is used to prompt the model to respond to the training instructions in the original language. A first training dataset is constructed based on the aforementioned first prompt text. Specifically, multiple data pairs in the first training dataset are constructed based on the training instructions corresponding to the aforementioned first prompt text and the correct responses to those training instructions.

[0069] The aforementioned first prompt text includes auxiliary information to guide the model in understanding and responding to specific training instructions. In embodiments of this disclosure, the first prompt text is used to guide model learning within the context of the original language. The first prompt text typically includes background information, contextual descriptions, or domain-specific knowledge related to the training instructions to help the model understand the intent of the instructions.

[0070] In some exemplary application scenarios, during the acquisition of the initial prompt text, the original language instruction data is analyzed to extract key information points related to the instructions. Then, based on these key information points, initial prompt texts that can effectively guide the model are constructed. These initial prompt texts are then used as part of each instruction during subsequent training, inputting them into the model along with the instructions to enhance the model's understanding and response accuracy.

[0071] Furthermore, during the construction of the first training dataset, the acquired first prompt text is paired with the corresponding training instructions to form a first training dataset containing the first prompt text and the original language instructions. This first training dataset will serve as the main input for model learning in the subsequent SFT process, used to train the initial language model's ability to understand instructions and generate responses in the context of the original language.

[0072] For example, if the original language is English, the first prompt text could be: "Please respond to the user's instruction in English. Instruction: Categorize the given input as either an animal or a vegetable: Mushroom.", which means: "Please respond to the user's instruction in English. For example, categorize the given input as either an animal or a vegetable: Mushroom." Correspondingly, based on this first prompt text, the correct response would be "Mushroom is a vegetable.", which means: "Mushroom is a vegetable."

[0073] It should be noted that when constructing the first training dataset, certain measures are typically taken to ensure the diversity of the initial prompt texts, covering different task scenarios and semantic requirements, so that the model can comprehensively understand and process various types of native language instructions. Simultaneously, the correct response to each native language instruction is also included in the first training dataset as a learning objective for the model, thereby guiding the model during training to learn how to correctly understand and generate responses given prompts.

[0074] Through steps S24 to S25 described above, this embodiment of the present disclosure constructs a high-quality first training dataset. This first training dataset, by introducing first cue text, significantly enhances the model's contextual understanding and task execution capabilities during instruction learning. Furthermore, introducing the first cue text helps the model converge faster, reducing uncertainty in the model's understanding and response processes, thereby accelerating the training process and improving training efficiency.

[0075] In one optional embodiment, the model training method further includes the following method steps:

[0076] Step S26: Obtain the second prompt text and the third prompt text, wherein the second prompt text is used to prompt a response to the training instructions in the target language, and the third prompt text is used to prompt cross-language understanding and conversion between the training instructions in the original language and the training instructions in the target language;

[0077] Step S27: Construct monolingual instruction data describing the target language based on the second prompt text, and construct cross-language instruction data based on the third prompt text;

[0078] Step S28: Determine the second training dataset using monolingual instruction data described in the target language and cross-lingual instruction data.

[0079] The aforementioned second prompt text can be a prompt text specific to the target language, used to guide the model in understanding and responding to training instructions expressed in the target language. When designing the second prompt text, it is necessary to consider the grammatical characteristics, lexical characteristics, and cultural background of the target language that differ from the original language, to ensure that the model can accurately understand and generate responses that conform to the language habits of the target language.

[0080] For example, the above second prompt text can be: "Please respond to the user's instruction in their language. Instruction: List five foods rich in vitamin C." The Chinese interpretation is: "Please respond to the user's instruction in their language, for example, List five foods rich in vitamin C." Correspondingly, the correct response to the above second prompt text is: "Oranges, strawberries, kiwis, bell peppers, and broccoli."

[0081] The above third prompt text can be: "You are a multilingual expert, and your role is to assist users in their native language. But your native language is English, so you'll need to use English for your initial response first. Each user interaction consists of a two-turn conversation. At first, please interpret the instruction in English, and then respond in English. Instruction: Categorize the given input as either an animal or a vegetable: Mushroom." The Chinese interpretation is: "You are a multilingual expert, and your role is to respond to the user in their language. But since your native language is English, you need to make an initial response in English first. Each user interaction consists of two rounds of conversation. First, please interpret the instruction in English, and then respond in English. For example: Categorize the given input as either an animal or a vegetable: Mushroom." Correspondingly, the correct response is: "English instruction: Categorize the given input as either an animal or a vegetable: Mushroom. English Response: Mushroom is a vegetable." The Chinese interpretation is: "English instruction: Categorize the given input as either an animal or a vegetable: Mushroom. English Response: Mushroom is a plant."

[0082] The third prompt text mentioned above also includes: "Now, please respond to the user's instruction in their language." The correct response (in Chinese) would be: "A mushroom is a plant."

[0083] The aforementioned third prompt text facilitates the model's cross-language understanding and conversion between the source and target languages. Through this third prompt text, the model not only needs to understand the source language instructions (i.e., the training instructions described in the source language), but also needs to convert those instructions into the target language instructions (i.e., the training instructions described in the target language), or conversely, the model needs to not only understand the target language instructions but also convert them back into the source language instructions. This enables the model to achieve cross-language instruction following capability.

[0084] In the process of constructing monolingual instruction data in the target language based on the second prompt text, the target language instruction is paired with the corresponding second prompt text to generate monolingual instruction data. This monolingual instruction data is used to directly train the model's ability to understand instructions and generate responses in the target language context, without needing the original language as an intermediary, thus achieving more natural and direct target language instruction processing.

[0085] In constructing cross-language formal instruction data based on third-party prompt text, a dataset containing parallel original language instructions and target language instructions, along with the correct response for each original language instruction and target language instruction, is created. Each instruction in the cross-language formal instruction data consists of two parts: one part represented in the original language and the other part represented in the target language. Through this cross-language formal instruction data, the model can learn how to understand and convert instructions between two languages, enhancing the model's cross-language adaptability and cross-language instruction following ability.

[0086] Furthermore, the monolingual instruction data and cross-lingual instruction data are integrated to obtain a second training dataset. This second training dataset is used to train the intermediate language model to generate the target language model.

[0087] Through steps S26 to S28 described above, this embodiment of the disclosure constructs a second training dataset to help improve the model's instruction understanding and response capabilities in the target language during training. In particular, the training dataset includes cross-lingual instruction data, which helps enhance the model's cross-lingual instruction following ability during subsequent training, improving the model's adaptability in multilingual environments. Furthermore, compared to traditional target language data-intensive training methods, the second training dataset constructed in this embodiment reduces the model's dependence on large amounts of target language training data, lowering training costs and resource consumption.

[0088] In an optional embodiment, in step S27, constructing cross-language instruction data based on the third prompt text includes the following method steps:

[0089] Step S271: Based on the third prompt text, control the intermediate language model to learn language by simulating a multi-turn dialogue process and construct cross-language instruction data.

[0090] The aforementioned multi-turn dialogue process involves multiple rounds of interaction between the intermediate language model and the user. In the Cross-lingual Continuous Instruction Tuning (X-CIT) process, this multi-turn dialogue process is used to simulate the cognitive pattern of second language learning. Specifically, in the cognitive pattern of second language learning, the model first understands and answers questions in the source language, and then switches to the target language to respond.

[0091] The aforementioned third prompt text is used to prompt cross-lingual understanding and conversion of training instructions in the original language and training instructions in the target language. Based on this, the intermediate language model can be controlled to simulate language learning in a multi-turn dialogue process, and cross-lingual chat-instruction dataset can be constructed using the dialogue data generated in the multi-turn dialogue process.

[0092] In the fine-tuning stage of the intermediate language model, third-party prompt texts are used to transform parallel source and target language instruction data into dialogue format, enabling the intermediate language model to learn cross-language understanding and conversion during simulated multi-turn dialogues. Guided and controlled by the third-party prompt texts, a dialogue-format instruction dataset can be constructed, containing target language instruction input, parallel source language instruction input, source language responses, and target language responses—that is, cross-language instruction data. Thus, the model continuously processes and compares the similarities and differences between instructions in different languages ​​during language learning, thereby accelerating the model's adaptation and learning process in the target language environment.

[0093] Through the above steps S271, this embodiment of the present disclosure effectively promotes the model's cross-language understanding and conversion capabilities by simulating multi-turn dialogue processes and using third-party prompt text, providing a training strategy for improving the accuracy and naturalness of target language instruction processing, thereby achieving more efficient and natural performance improvement of the model in the target language environment.

[0094] In an optional embodiment, the multi-turn dialogue process includes: a first-turn dialogue and a second-turn dialogue. In step S271, cross-language instruction data is constructed based on the third prompt text, including the following method steps:

[0095] Step S272: Based on the third prompt text control intermediate language model, the first round of dialogue is simulated to perform language understanding and answer the training instructions in the target language, and obtain the training instructions and instruction responses in the original language.

[0096] Step S273: Based on the third prompt text control intermediate language model, the original language instruction response is converted into the target language instruction response by simulating the second round of dialogue.

[0097] Step S274: Construct cross-language instruction data using training instructions in the target language, training instructions in the original language, instruction responses in the original language, and instruction responses in the target language.

[0098] In the first round of dialogue, the model uses the original language to understand and respond to instructions in the target language; in the second round of dialogue, the model uses the target language to generate the final instruction response.

[0099] In application scenarios, training instructions in the target language refer to instructions expressed in the target language. These instructions guide the model to learn and adapt to the target language's linguistic environment during training. Training instructions in the original language refer to instructions expressed in the original language, which have the same semantics as the training instructions in the target language. These original language training instructions serve as reference information for the intermediate language model to perform cross-language understanding and translation.

[0100] When the intermediate language model, controlled by third-party prompt text, simulates the first round of dialogue, it performs language understanding and responses to the training instructions in the target language. During the language understanding process, third-party prompt text is used to assist in understanding the training instructions in the target language, resulting in the training instructions in the original language. For example, the semantic information corresponding to the training instructions in the target language is understood by referring to the third-party prompt text, and based on this semantic information, the training instructions in the target language are converted into training instructions in the original language. Furthermore, referring to the third-party prompt text, the training instructions in the original language are responded to, generating a response in the original language.

[0101] The intermediate language model, based on third-party prompt text control, transforms the original language instruction response into the target language instruction response during simulated second-round dialogues, thereby further enhancing the model's cross-language conversion capabilities.

[0102] It should be noted that the intermediate language model simulates the process of the first round of dialogue, which is the early stage of human second language learning when they first try to understand non-native language situations using their native language; the intermediate language model simulates the process of the second round of dialogue, which is the later stage of human second language learning when they gradually master the use of non-native languages ​​for expression and communication.

[0103] In application scenarios, after multiple rounds of dialogue, the model integrates the data generated during the dialogue to obtain instruction data in multiple language forms, resulting in the aforementioned cross-lingual instruction data. This cross-lingual instruction data includes training instructions in the target language, training instructions in the original language, instruction responses in the original language, and instruction responses in the target language. This cross-lingual instruction data not only demonstrates the parallel translation of instructions and responses but also includes the intermediate steps of language understanding and conversion, providing the model with rich language alignment signals and helping it learn cross-lingual instruction processing methods.

[0104] Through steps S272 to S274 above, in this embodiment of the disclosure, through the first round of dialogue, the model learns how to use third prompt text to assist in understanding and generating instruction responses in the original language when inputting instructions in the target language, thereby enhancing the model's instruction comprehension ability in the target language environment; through the second round of dialogue, the model is able to accurately convert instruction responses in the original language into instruction responses in the target language, improving the model's cross-language conversion ability; by integrating the various language forms of instructions and responses obtained in the simulation process of the first and second rounds of dialogue, a high-quality cross-language instruction dataset is constructed, providing more comprehensive and in-depth cross-language learning materials for the subsequent training stage of the model; simulating the multi-round dialogue process not only strengthens the model's cross-language instruction processing ability, but also accelerates the model's adaptability in the target language environment through the guidance of third prompt text, enabling the model to understand and respond to target language instructions more quickly.

[0105] In an optional embodiment, in the above model training method, the third prompt text includes: a scene description part and an instruction part, wherein the scene description part is used to define the role of the intermediate language model as a multilingual role with the original language as its native language and to assign role responsibilities to the intermediate language model, and the instruction part is used to guide the intermediate language model to construct cross-lingual instruction data.

[0106] In the above optional embodiments, the scene description section includes role definition information and role responsibility information. Inputting the scene description section into the intermediate language model allows the role definition information to define the intermediate language model's roles as multilingual roles with the original language as their native language; the role responsibility information is used to assign role responsibilities to the intermediate language model. The instruction section guides the intermediate language model in constructing cross-language instruction data.

[0107] In the optional embodiments described above, the scenario description section is used to set a context or situation, defining the role of the intermediate language model as a multilingual expert whose native language is the original language (e.g., English). Through the scenario description section, the model is placed in a specific communication environment, such as simulating a conversation with users speaking different languages. The scenario description section not only clarifies the model's "identity" (i.e., the model is expected to handle instructions in multiple languages), but also specifies the model's "role responsibilities" (i.e., task performance requirements in different language environments). These role responsibilities may include, but are not limited to, understanding non-English instructions, generating initial responses in English format, and converting English responses into non-English languages ​​while maintaining consistency in understanding.

[0108] The instruction section directly contains commands or guidelines that instruct the model to perform specific operations. This section guides the intermediate language model on how to construct cross-lingual instruction data. Specifically, the instruction section might require the intermediate language model to first convert non-English instructions into English instructions and generate English responses; in the second step, it might require the model to convert the English responses back into non-English language instructions. Thus, the instruction section ensures that the intermediate language model can accurately complete the task of cross-lingual understanding and conversion, while also providing the intermediate language model with specific steps for learning and executing instructions.

[0109] In an exemplary application scenario, the role definition information in the above scenario description part (presented in English) can be "You are a multilingual expert, and your role is to assist users in their native language. But your native language is English, so you'll need to use English for your initial response first. Each user interaction consists of a two-turn conversation.", and correspondingly, the Chinese interpretation is "你是一位多语言专家,你的角色是用用户的母语为他们提供帮助。但你的母语是英文,因此你需要先用英文进行初始回应。每次用户互动包括两轮对话。"; The role responsibility information in the scenario description part can be "At first, please interpret the instruction in English, and then respond in English.", and correspondingly, the Chinese interpretation is "首先,请用英文解释指令,然后用英文回应。". The above instruction part can be "Instruction: Categorize the given input as either an animal or a vegetable: Mushroom.", and correspondingly, the Chinese interpretation is "指令:将给定的输入分类为动物或植物:蘑菇。".

[0110] Through the scenario description part and the instruction part set in the above third prompt text, the embodiments of the present disclosure not only set a clear role positioning and role responsibilities for the intermediate language model, but also guide the intermediate language model to conduct construction learning of cross-language form instruction data, thereby improving the instruction processing ability of the model in the target language environment as a whole, enabling the model to more effectively adapt to and process multilingual instructions, and providing a more accurate and natural service experience for users in the target language.

[0111] In an optional embodiment, in step S23, the intermediate language model is trained using the second training dataset to generate a target language model, including the following method steps:

[0112] Step S231, using the second training dataset to control the intermediate language model to conduct curriculum learning to generate a target language model.

[0113] The second training dataset includes monolingual instruction data and cross-lingual instruction data in the target language. Using this second training dataset, the intermediate language model is controlled for Self-Paced Learning (SPL), enabling more efficient and targeted training to ultimately generate the target language model.

[0114] The aforementioned learning process refers to an adaptive training method. During this process, the intermediate language model learns from the samples in the second training dataset in order of increasing complexity, based on its own learning progress and capabilities. For example, in SPL, the intermediate language model can be instructed to prioritize learning and processing less difficult samples. As training progresses and the model's capabilities improve, the intermediate language model's training will gradually introduce more complex and challenging samples until it can handle all samples in the second training dataset, at which point the target language model is obtained. This learning process helps the intermediate language model learn new knowledge more efficiently and robustly, avoiding increased training difficulty or decreased model performance caused by introducing complex samples too early.

[0115] Through the above steps S231, the second stage of model training using X-CIT in this embodiment of the present disclosure utilizes the second training dataset for course learning, effectively improving the intermediate language model's instruction processing capability in the target language. The generated target language model can not only process target language instructions more efficiently and accurately, but also generate more natural and fluent target instruction responses, thereby providing a more powerful and flexible solution for natural language processing applications in multilingual environments.

[0116] In an optional embodiment, in step S231, the intermediate language model is controlled to learn the course using the second training dataset to generate the target language model, including the following method steps:

[0117] Step S232: Select training samples to be learned from each batch of samples in the second training dataset according to the preset screening conditions.

[0118] Step S233: For each batch of samples in the second training dataset, use the training samples to be learned to control the intermediate language model to learn the course of the target training rounds until all the training samples selected from each batch of samples in the second training dataset have been learned, and the target language model is generated.

[0119] The aforementioned preset screening criteria are used to determine the difficulty level of each batch of samples in the second training dataset during course learning. That is, based on the preset screening criteria, simple samples and complex samples can be distinguished from multiple samples in the second training dataset. For example, the aforementioned preset screening criteria can be set through sample parameters, which may include, but are not limited to: sample loss value, instruction length, syntactic complexity, semantic complexity, contextual dependence of instructions, instruction category, etc.

[0120] The aforementioned second training dataset includes multiple sample batches. Based on preset filtering conditions, training samples for the current training round can be selected from each sample batch. Furthermore, for each sample batch in the second training dataset, the intermediate language model is controlled to learn a course using the training samples to be learned. The target training round number corresponding to the course learning corresponds to the number of training samples selected from each sample batch of the second training dataset. Finally, when the intermediate language model has learned all the selected training samples, the target language model is obtained.

[0121] Through steps S232 to S233 above, this embodiment of the disclosure effectively controls the learning process of the intermediate language model by using preset screening conditions, ensuring that the model can gradually improve its instruction processing capabilities in the target language environment. The final generated target language model not only has more efficient learning capabilities, but also provides more natural and accurate target language instruction responses.

[0122] In an optional embodiment, the preset screening criteria are defined by the sample loss value. In step S232, training samples to be learned are selected from each batch of samples in the second training dataset according to the preset screening criteria, including the following method steps:

[0123] Step S2321: Based on the comparison results between the sample loss value and the course learning loss threshold, select training samples to be learned from each sample batch of the second training dataset. The sample loss value is used to determine the learning difficulty of multiple samples contained in each sample batch of the second training dataset.

[0124] In the above optional embodiments, preset screening conditions are determined based on sample loss values. Specifically, the sample loss values ​​corresponding to multiple samples in each batch of the second training dataset are compared with the loss threshold of the course learning to obtain comparison results. Based on the comparison results, target samples are selected from each batch to obtain training samples to be learned.

[0125] It should be noted that the sample loss value for a particular sample is used to characterize the difficulty of the intermediate language model learning from that sample. The aforementioned loss threshold can be a threshold pre-set for course learning, or it can be a threshold selection rule set for each learning step in the course learning. For example, as the course learning progresses, the loss threshold corresponding to the current learning step can gradually increase, and correspondingly, the intermediate language model can gradually learn more difficult training samples.

[0126] For example, in the training process of natural language processing, the sample loss value is a quantitative metric used to measure the difference between the model's predicted output and the actual output when processing a specific sample (such as instruction data). A lower sample loss value indicates a more accurate prediction by the model on that sample, and a more "simple" sample; a higher sample loss value indicates a lower prediction accuracy by the model on that sample, and a more "complex" sample. Based on this, the loss threshold for the above-mentioned course learning is used to distinguish between "simple" and "complex" samples in the second training dataset. Depending on the current training stage, the loss threshold can be dynamically adjusted to ensure that the model can learn from easily processed samples and gradually transition to more complex samples.

[0127] Through step S2321 above, this embodiment of the disclosure controls the intermediate language model to process samples with low loss values ​​(i.e., relatively "simple") in the early stages of course learning by comparing sample loss values ​​with a loss threshold. As training progresses and the model's capabilities improve, the loss threshold can be gradually increased, controlling the intermediate language model to begin processing samples with higher loss values ​​(i.e., relatively "complex"), thus achieving a progressive increase in learning difficulty during course learning. This progressive course learning helps improve the model's learning efficiency, avoiding the low training efficiency caused by facing overly complex samples in the early stages of training. It also ensures that the model has stronger basic capabilities when processing more complex samples and enhances the model's robustness and adaptability in the target language environment. The final generated target language model can provide more accurate and natural target language instruction responses, offering a more powerful and flexible solution for natural language processing applications in multilingual environments.

[0128] In one optional embodiment, the model training method further includes the following method steps:

[0129] Step S291: Obtain the initial loss for the first batch of samples in the second training dataset;

[0130] Step S292: Calculate the mean and standard deviation of the initial loss respectively;

[0131] Step S293: Initialize the loss threshold based on the average value and the target number of training rounds to obtain the initial threshold;

[0132] Step S294: Based on the range of values ​​of the standard deviation, update the initial threshold to initialize the loss threshold with the updated threshold, or update the initial iteration coefficients to obtain the updated iteration coefficients, wherein the updated iteration coefficients are used to gradually adjust the loss threshold in multiple training rounds corresponding to the target training round number.

[0133] The first sample batch refers to the first sample batch selected from the second training dataset at the beginning of the course learning. This first sample batch is used to evaluate the initial state and capabilities of the model.

[0134] The initial loss described above characterizes the degree of difference between the model's predicted output and the actual expected output on the first batch of samples. This initial loss is used to quantify the difficulty level of the model when processing the first batch of samples.

[0135] The mean measures the average initial loss of the model on the first batch of samples. The standard deviation reflects the volatility of the initial loss across multiple samples in the first batch, that is, it characterizes the distribution of sample difficulty in the first batch.

[0136] The aforementioned loss threshold serves as the boundary between "simple" and "complex" samples during the learning process. Samples with a loss value less than the threshold are classified as "simple," while those with a loss value greater than or equal to the threshold are classified as "complex." The model will prioritize learning from "simple" samples, and the loss threshold will be gradually adjusted as training progresses.

[0137] The loss threshold for the above-mentioned course can be adjusted dynamically. Specifically, the initial loss of the first batch of samples in the second training dataset is obtained. This initial loss can be preset by technical personnel or other program threads. Based on the multiple initial losses corresponding to multiple samples in the first batch, the average and standard deviation of the multiple initial losses are calculated. The loss threshold is initialized according to the average value of the first batch and the target training epoch, thus obtaining the initial threshold for the first batch. Furthermore, the initial threshold is updated according to the range of values ​​of the standard deviation.

[0138] Through steps S291 to S294 above, this embodiment of the disclosure provides a progressive path for the model's learning by dynamically initializing and adjusting the loss threshold, optimizing learning efficiency and the model's processing capabilities. In particular, in terms of target language instruction processing, the generated target language model will be able to understand and respond to different instructions from target language users more naturally and accurately, providing a more powerful and flexible solution for natural language processing applications in multilingual environments.

[0139] The model training method provided in this disclosure is explained by way of example in conjunction with application scenarios.

[0140] During model training, principles and parameter theory can be applied. This means that while universal principles can be considered shared across different languages, parameters differ. When controlling the model for second language learning, the model starts with parameters corresponding to the native language (i.e., the original language used for training data) and continuously adjusts these parameters during the learning process, enabling the adjusted model to adapt to non-native language (i.e., target language) environments. This ensures the continued effectiveness of universal principles when the model handles tasks in different languages, while parameters corresponding to the native language are transferred to the second language. Based on this, this disclosure provides a two-stage cross-language continuous guidance fine-tuning (X-CIT) scheme, as shown in Figure 3.

[0141] As shown in Figure 3, English is used as the source language and Chinese as the target language. In the X-CIT scheme, the first stage is supervised fine-tuning based on English training data, and the second stage is supervised fine-tuning based on Chinese floating chain data and cross-language training data. Through the first and second stages, a target language model with good Chinese response capabilities can be obtained. In the third stage, Chinese-based model inference can be performed to generate Chinese responses.

[0142] Specifically, in the first stage, the English training data can include the initial prompt text and the corresponding correct response. For example, if the initial prompt text is "Please respond to the user's instruction in English. Instruction: Categorize the given input as either an animal or a vegetable: Mushroom.", the correct response would be "Mushroom is a vegetable." Based on this English training data, the initial language model is fine-tuned under English supervision to obtain an intermediate language model. This intermediate language model can internalize common rules shared by both English and Chinese, and thus initially possesses cross-language capabilities.

[0143] In the second stage, SFT is continuously performed based on the intermediate language model to adapt the model to the Chinese environment. Specifically, the continuous SFT process is implemented using monolingual Chinese instruction data and dialogue-style instruction data. The construction of the dialogue-style instruction data involves parallel instruction fine-tuning datasets, where "parallel" means that the Chinese and English data are semantically identical. The dialogue-style instruction data includes Chinese input instructions, English input instructions (parallel to Chinese input instructions), English responses, and Chinese responses (parallel to English responses). Based on the above monolingual Chinese instruction data and dialogue-style instruction data, the intermediate language model is trained to obtain the target language model. The target language model is defined as a "multilingual expert whose native language is English," and its responsibility is to "first understand and answer questions in its native language (English in this example) and then translate them into the user's language (Chinese in this example) to respond."

[0144] In the second stage, during the model's learning process, Chinese instruction data and dialogue-style instruction data are input into the intermediate language model in batches. Before optimizing the intermediate language model using data from each batch (containing multiple training samples), the current training sample to be learned can be determined based on the learning difficulty of the multiple training samples. Specifically, the learning difficulty of the training sample can be judged using the sample loss value.

[0145] In one implementation, it is believed that the intermediate language model needs to learn from simpler training samples in the early stage of training. A low value can be initialized as the loss threshold. As the training rounds increase, the loss threshold is increased by a certain proportion to include more difficult training samples in the current training, until the intermediate language model has learned all the samples.

[0146] In one implementation, an automated threshold initialization algorithm can be used to dynamically adjust the loss threshold during training. Based on this algorithm, the second-stage model training process can be described as follows. The inputs to the training process include: a large language model Men tuned to native language (English in this example), a target language (Chinese in this example) instruction dataset DL, a cross-language dialogue instruction dataset D, batch size B, and number of training epochs N. The outputs of the training process include: a fine-tuned target language model ML.

[0147] The training process based on the automated threshold initialization algorithm is as follows: Initialize the number of iterations n to 0; in each training round, sample batch data {B} from datasets DL and D; if it is the first training round (n=0), calculate the initial loss corresponding to the batch data {B} according to the preset loss function; calculate the average loss Lavg and the standard deviation Lstd based on this initial loss; determine the initial loss threshold λ based on the average loss Lavg and the number of training rounds N; if Lstd is less than 1, update the loss threshold λ based on the average loss Lavg and the number of training rounds N; determine the iteration coefficient k based on the number of training rounds N. After determining the loss threshold λ, for each training sample b in the batch data {B}, calculate the sample loss value Lb corresponding to that training sample b. If Lb is less than the loss threshold λ, add the training sample b to the training sample set S to be learned in the current step. Based on this, use the training samples to be learned to train and optimize the intermediate language model in the current step. As training progresses (i.e., n = n + 1), the loss threshold for the next round is updated using the product of the current round's loss threshold λ and the iteration coefficient k (e.g., λ = λ × k).

[0148] It is noteworthy that, compared to existing model training methods, this disclosure implements SFT based on translated parallel instruction data from both training and data construction perspectives. In this disclosure, firstly, a two-stage training method is used to fully simulate the human second language learning process. Secondly, a dialogue-based dataset is constructed to simulate the human second language learning thought process. Finally, a self-regulated learning (SPL) method is employed to deeply simulate the process of humans learning new things from easy to difficult. Compared to the most similar PLUG method, our method can achieve end-to-end responses to instructions from non-English speaking users, thanks to the two-turn dialogue-based dataset and the non-English monolingual instruction data used in the second stage.

[0149] In the aforementioned operating environment, this disclosure also provides a data processing method as shown in FIG4. FIG4 is a flowchart of a data processing method according to an embodiment of this disclosure. As shown in FIG4, the data processing method includes:

[0150] Step S41: Obtain query data in the target language;

[0151] Step S42: Use the target language model to query and parse the query data in the target language to obtain the target response in the target language.

[0152] The target language model is generated according to any of the above model training methods.

[0153] The query data in the target language mentioned above can be query data input by the user in their native language. This native language may be different from the training language of the target language model; for example, the training language might be English, while the user's native language is Chinese. The target language model generated using the aforementioned model training method can parse the query data and obtain the target response in the target language. The model training method for the aforementioned target language model can be found in the relevant descriptions above.

[0154] In this embodiment of the disclosure, query data in the target language is obtained; the query data in the target language is parsed using a target language model to obtain the target response in the target language, wherein the target language model is generated according to any of the model training methods described above.

[0155] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing a higher-performance query service in the target language. In other words, the target language model used in this embodiment exhibits good performance in the target language environment. Therefore, this embodiment achieves the technical effect of providing target language query services at a lower cost, thus solving the technical problem that large language models trained based on the original language cannot provide high-performance target language query services.

[0156] It should be noted that the preferred embodiments of steps S41 to S42 described above can be found in the foregoing description, and will not be repeated here.

[0157] In the aforementioned operating environment, this disclosure also provides another data processing method as shown in FIG5. FIG5 is a flowchart of another data processing method according to an embodiment of this disclosure. As shown in FIG5, the data processing method includes:

[0158] Step S51: Obtain Chinese astronomical knowledge query data;

[0159] Step S52: Use the target language model to query and parse the Chinese astronomical knowledge query data to obtain the Chinese astronomical knowledge query response.

[0160] The target language model is generated according to any of the above model training methods, and the target language model uses English as its native language.

[0161] In this embodiment of the disclosure, Chinese astronomical knowledge query data is obtained; a target language model is used to query and parse the Chinese astronomical knowledge query data to obtain Chinese astronomical knowledge query responses. The target language model is generated according to any of the above-mentioned model training methods, and the target language model is natively English.

[0162] The data processing method provided in this disclosure can be a data processing scheme specifically implemented in the field of astronomical knowledge query. The target language model can be trained using the aforementioned model training method and training data in the field of astronomical knowledge. This target language model can provide Chinese astronomical knowledge query services.

[0163] It is noteworthy that the target language model used in the embodiments of this disclosure can perform well in the Chinese environment. Therefore, the embodiments of this disclosure can achieve the technical effect of providing Chinese astronomical knowledge query services at a lower cost, thereby solving the technical problem that large language models trained in English are difficult to provide high-performance Chinese query services.

[0164] It should be noted that the preferred embodiments of steps S51 to S52 described above can be found in the foregoing description, and will not be repeated here.

[0165] In the aforementioned operating environment, this disclosure also provides another data processing method as shown in FIG6. FIG6 is a flowchart of another data processing method according to an embodiment of this disclosure. As shown in FIG6, the data processing method includes:

[0166] Step S61: Obtain a data processing request through the first application programming interface, wherein the request data carried in the data processing request includes: query data in the target language;

[0167] Step S62: Return a data processing response through the second application programming interface. The response data carried in the data processing response includes: the target response in the target language, which is obtained by parsing the query data in the target language using the target language model.

[0168] The target language model is generated according to any of the model training methods mentioned above.

[0169] The data processing method described in this embodiment can run on a cloud server to provide cloud data processing services to clients. The client sends a data processing request by calling a first application programming interface (API). After obtaining the data processing request through the first API, the cloud server generates a target response in the target language according to the data processing method and further returns a data processing response to the client through a second API.

[0170] It should be noted that the target language model used in generating the target response in the target language can be obtained according to any of the aforementioned model training methods.

[0171] The first and second application programming interfaces (APIs) mentioned above can be the same or different APIs. In one optional embodiment, the interface parameters in the first and second APIs may include, but are not limited to: a global interface identifier, an interface signing key, an interface timestamp, an interface request identifier, and a system call credential identifier. The first API can use a GET request or a POST request as the interface request method to obtain a file processing request. The second API can use a lightweight data exchange format (such as JavaScript Object Notation, or JSON for short) to return a file processing response.

[0172] In this embodiment of the disclosure, a data processing request is obtained through a first application programming interface (API), wherein the request data carried in the data processing request includes query data in the target language; a data processing response is returned through a second API, wherein the response data carried in the data processing response includes a target response in the target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-mentioned model training methods.

[0173] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing a higher-performance query service in the target language. In other words, the target language model used in this embodiment exhibits good performance in the target language environment. Therefore, this embodiment achieves the technical effect of providing target language query services at a lower cost, thus solving the technical problem that large language models trained based on the original language cannot provide high-performance target language query services.

[0174] It should be noted that the preferred embodiments of steps S61 to S62 described above can be found in the foregoing description, and will not be repeated here.

[0175] In the aforementioned operating environment, this disclosure also provides another data processing method as shown in FIG7. FIG7 is a flowchart of another data processing method according to an embodiment of this disclosure. As shown in FIG7, the data processing method includes:

[0176] Step S71: Obtain the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: query data in the target language;

[0177] Step S72: In response to the data processing dialogue request, a data processing dialogue response is returned. The information carried in the data processing dialogue response includes: the target response in the target language, which is obtained by parsing the query data in the target language using the target language model.

[0178] Step S73: Display the target response in the target language within the graphical user interface.

[0179] The target language model described above is generated using any of the model training methods mentioned above.

[0180] The data processing method provided in this embodiment can utilize the aforementioned target language model to implement a visualized data processing scheme, facilitating human-computer interaction. The user inputs a data processing dialogue request by triggering an input command on the operation interface. Further, in response to the data processing dialogue request, the system is triggered to generate a target response in the target language according to the aforementioned data processing method. Furthermore, the system displays the target response in the target language within the operation interface for the user.

[0181] The target language model used in generating the target response in the aforementioned target language can be obtained using any of the aforementioned model training methods.

[0182] In this embodiment of the disclosure, a data processing request is obtained through a first application programming interface (API), wherein the request data carried in the data processing request includes query data in the target language; a data processing response is returned through a second API, wherein the response data carried in the data processing response includes a target response in the target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-mentioned model training methods.

[0183] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing a higher-performance query service in the target language. In other words, the target language model used in this embodiment exhibits good performance in the target language environment. Therefore, this embodiment achieves the technical effect of providing target language query services at a lower cost, thus solving the technical problem that large language models trained based on the original language cannot provide high-performance target language query services.

[0184] It should be noted that the preferred embodiments of steps S71 to S73 described above can be found in the foregoing description, and will not be repeated here.

[0185] In the aforementioned operating environment, this disclosure also provides another data processing method as shown in FIG8. FIG8 is a flowchart of another data processing method according to an embodiment of this disclosure. As shown in FIG8, the data processing method includes:

[0186] Step S81: In response to the input command applied to the operation interface, display the query data in the target language on the operation interface;

[0187] Step S82: In response to the processing command applied to the operation interface, display the target response in the target language on the operation interface.

[0188] The target response in the target language is obtained by parsing the query data in the target language using the target language model, which is generated according to any of the above model training methods.

[0189] Based on the above method steps, a visualization scheme for data processing functions is provided. The terminal device provides a graphical user interface (i.e., the operation phase), which displays at least a data processing scenario. The graphical user interface also includes input components (such as text input boxes, voice input controls, etc.) and display components (such as text display windows, image display windows, etc.). The user triggers input commands through the input components to display query data in the target language on the operation interface. Furthermore, the user can trigger processing commands through the operation interface. Upon detecting the processing command, a target response in the target language is generated according to the aforementioned data processing method and displayed on the operation interface.

[0190] In this embodiment of the disclosure, a data processing request is obtained through a first application programming interface (API), wherein the request data carried in the data processing request includes query data in the target language; a data processing response is returned through a second API, wherein the response data carried in the data processing response includes a target response in the target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-mentioned model training methods.

[0191] It is noteworthy that in this embodiment, a preliminary language model is trained using monolingual instruction data described in the original language to obtain an intermediate language model. This intermediate language model is then further trained using monolingual instruction data described in the target language and cross-lingual instruction data described in both the original and target languages ​​to obtain the aforementioned target language model. This enables the target language model to directly process query data in the target language and directly generate target responses in the target language, thereby providing a higher-performance query service in the target language. In other words, the target language model used in this embodiment exhibits good performance in the target language environment. Therefore, this embodiment achieves the technical effect of providing target language query services at a lower cost, thus solving the technical problem that large language models trained based on the original language cannot provide high-performance target language query services.

[0192] It should be noted that the preferred embodiments of steps S81 to S82 described above can be found in the foregoing description, and will not be repeated here.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0194] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0195] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0196] According to an embodiment of this disclosure, an apparatus embodiment for implementing the above-described model training method is also provided. Figure 9 is a schematic diagram of the structure of a model training apparatus according to an embodiment of this disclosure. As shown in Figure 9, the apparatus includes: an acquisition module 901 configured to acquire a first training dataset and a second training dataset, wherein the first training dataset consists of monolingual instruction data described in the original language, and the second training dataset consists of monolingual instruction data described in the target language and cross-lingual instruction data described in the original language and the target language, respectively; a first training module 902 configured to train an initial language model using the first training dataset to generate an intermediate language model; and a second training module 903 configured to train the intermediate language model using the second training dataset to generate a target language model, wherein the target language model is used to query and parse query data in the target language to generate a target response in the target language.

[0197] Optionally, in addition to all the modules mentioned above, the model training device also includes a construction module (not shown in the figure), which is configured to: obtain a first prompt text, wherein the first prompt text is used to prompt a response to the training instructions in the original language; and construct a first training dataset based on the first prompt text.

[0198] Optionally, in addition to all the modules mentioned above, the model training device also includes a determination module (not shown in the figure), configured to: acquire a second prompt text and a third prompt text, wherein the second prompt text is used to prompt responses to training instructions in the target language, and the third prompt text is used to prompt cross-language understanding and conversion between training instructions in the original language and training instructions in the target language; construct monolingual instruction data described in the target language based on the second prompt text, and construct cross-language formal instruction data based on the third prompt text; and determine a second training dataset using the monolingual instruction data described in the target language and the cross-language formal instruction data.

[0199] Optionally, the aforementioned determining module is further configured to: based on the third prompt text, control the intermediate language model to perform language learning by simulating a multi-turn dialogue process, and construct cross-language instruction data.

[0200] Optionally, the multi-turn dialogue process includes: a first-turn dialogue and a second-turn dialogue. The aforementioned determining module is further configured to: control the intermediate language model based on the third prompt text to perform language understanding and answering of the training instructions in the target language by simulating the first-turn dialogue, thereby obtaining the training instructions and instruction responses in the original language; control the intermediate language model based on the third prompt text to perform language conversion of the instruction responses in the original language by simulating the second-turn dialogue, thereby obtaining the instruction responses in the target language; and construct cross-language instruction data using the training instructions in the target language, the training instructions in the original language, the instruction responses in the original language, and the instruction responses in the target language.

[0201] Optionally, in the above-mentioned model training device, the third prompt text includes: a scene description part and an instruction part, wherein the scene description part is used to define the role of the intermediate language model as a multilingual role with the original language as its native language and to assign role responsibilities to the intermediate language model, and the instruction part is used to guide the intermediate language model to construct cross-lingual instruction data.

[0202] Optionally, the second training module 903 is further configured to: use the second training dataset to control the intermediate language model to learn the course and generate the target language model.

[0203] Optionally, the second training module 903 is further configured to: select training samples to be learned from each batch of samples in the second training dataset according to preset screening conditions; and, for each batch of samples in the second training dataset, use the training samples to be learned to control the intermediate language model to learn the course for the target training rounds until all the training samples selected from each batch of samples in the second training dataset have been learned, thereby generating the target language model.

[0204] Optionally, the preset screening conditions are defined by the sample loss value, and the second training module 903 is further configured to: select training samples to be learned from each sample batch of the second training dataset according to the comparison result between the sample loss value and the loss threshold of course learning, wherein the sample loss value is used to determine the learning difficulty of multiple samples contained in each sample batch of the second training dataset.

[0205] Optionally, in addition to all the modules mentioned above, the model training device also includes an initialization module (not shown in the figure), which is configured to: obtain the initial loss of the first batch of samples in the second training dataset; calculate the mean and standard deviation of the initial loss respectively; initialize the loss threshold based on the mean and the target training epoch to obtain the initial threshold; update the initial threshold based on the range of the standard deviation to initialize the loss threshold with the updated threshold, or update the initial iteration coefficients to obtain the updated iteration coefficients, wherein the updated iteration coefficients are used to gradually adjust the loss threshold in multiple training epochs corresponding to the target training epoch.

[0206] It should be noted that the above-mentioned acquisition module 901, first training module 902 and second training module 903 correspond to steps S21 to S23 in the embodiment. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0207] According to an embodiment of this disclosure, an apparatus embodiment for implementing the above-described data processing method is also provided. Figure 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of this disclosure. As shown in Figure 10, the apparatus includes: an acquisition module 1001 configured to acquire query data in a target language; and a query module 1002 configured to use a target language model to perform query parsing on the query data in the target language to obtain a target response in the target language; wherein the target language model is generated according to any of the model training methods described above.

[0208] It should be noted that the above-mentioned acquisition module 1001 and query module 1002 correspond to steps S41 to S42 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0209] According to embodiments of this disclosure, another apparatus embodiment for implementing the above-described data processing method is also provided. Figure 11 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of this disclosure. As shown in Figure 11, the apparatus includes: an acquisition module 1101 configured to acquire Chinese astronomical knowledge query data; and a query module 1102 configured to use a target language model to perform query parsing on the Chinese astronomical knowledge query data to obtain a Chinese astronomical knowledge query response; wherein the target language model is generated according to the model training method of any one of the above, and the target language model uses English as its native language.

[0210] It should be noted that the above-mentioned acquisition module 1101 and query module 1102 correspond to steps S51 to S52 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0211] According to embodiments of this disclosure, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 12 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of this disclosure. As shown in Figure 12, the apparatus includes: an acquisition module 1201 configured to acquire a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes query data in a target language; and a response module 1202 configured to return a data processing response through a second application programming interface, wherein the response data carried in the data processing response includes a target response in a target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-described model training methods.

[0212] It should be noted that the above-mentioned acquisition module 1201 and response module 1202 correspond to steps S61 to S62 in the embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0213] According to embodiments of this disclosure, another apparatus embodiment for implementing the above-described data processing method is also provided. Figure 13 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of this disclosure. As shown in Figure 13, the apparatus includes: an acquisition module 1301 configured to acquire a currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes query data in the target language; a return module 1302 configured to respond to the data processing dialogue request and return a data processing dialogue response, wherein the information carried in the data processing dialogue response includes a target response in the target language, wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-described model training methods; and a display module 1303 configured to display the target response in the target language within a graphical user interface.

[0214] It should be noted that the above-mentioned acquisition module 1301, return module 1302 and display module 1303 correspond to steps S71 to S73 in the embodiment. The three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiment.

[0215] According to embodiments of this disclosure, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 14 is a schematic diagram of the structure of another data processing apparatus according to an embodiment of this disclosure. As shown in Figure 14, the apparatus includes: a first response module 1401 configured to respond to an input command applied to an operation interface and display query data in the target language on the operation interface; a second response module 1402 configured to respond to a processing command applied to the operation interface and display a target response in the target language on the operation interface; wherein the target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to any of the above-described model training methods.

[0216] It should be noted here that the first response module 1401 and the second response module 1402 mentioned above correspond to steps S81 to S82 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the foregoing embodiments.

[0217] It should be noted that the above-mentioned modules or units may be hardware or software components stored in memory and processed by one or more processors. The above-mentioned modules may also be part of a device and run in a computer terminal.

[0218] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0219] Embodiments of this disclosure may provide a data processing system, including: a client configured to send query data in a target language; a server connected to the client and configured to use a target language model to perform query parsing on the query data in the target language to obtain a target response in the target language; the client is further configured to output the target response in the target language; wherein the target language model is generated according to the model training method of any of the above.

[0220] Embodiments of this disclosure may provide an electronic device, including: a memory storing an executable program; and a processor configured to run the program, wherein the program executes either the model training method or the data processing method of any of the foregoing embodiments during runtime.

[0221] Figure 15 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in Figure 15, the electronic device 150 may include: one or more (only one is shown in the figure) processors 152, memory 154, memory controller, and peripheral interfaces.

[0222] The aforementioned electronic device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, personal computers (PCs), all-in-one model machines, etc., and the electronic device may have the model described in the above embodiments of this disclosure pre-installed.

[0223] Specifically, this electronic device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this electronic device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this electronic device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this electronic device can also create applications based on models, providing Application Programming Interface (API) invocation capabilities. Models can be invoked into the created applications through the API interface, and application management tools are provided to manage and monitor the applications.

[0224] Furthermore, the electronic device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing rich training resources to help users learn and master artificial intelligence (AI) technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0225] The memory can be used to store software programs and modules, such as the model training method or data processing method and related device program instructions / modules in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the model training method or data processing method in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0226] The processor can invoke the executable program stored in the memory through the transmission device to execute any of the model training methods or data processing methods described in the above embodiments.

[0227] Those skilled in the art will understand that the structure shown in FIG15 is merely illustrative, and the electronic device may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, or a mobile internet device (MID), etc. FIG15 does not limit the structure of the above-described electronic device. For example, the electronic device 150 may also include more or fewer components (such as a network interface, a display device, etc.) than shown in FIG15, or have a different configuration than shown in FIG15.

[0228] Those skilled in the art will understand that all or part of the steps in the various model training methods or data processing methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0229] Embodiments of this disclosure also provide a computer-readable storage medium, which includes a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute the model training method or the data processing method of any of the foregoing.

[0230] Optionally, in this embodiment, the storage medium may be located in an electronic device.

[0231] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute any of the model training methods or data processing methods described in the above embodiments.

[0232] Embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the model training method or the data processing method of any of the foregoing embodiments.

[0233] Embodiments of this disclosure also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the model training method or data processing method provided in the above embodiments.

[0234] Embodiments of this disclosure also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the model training method or data processing method provided in the above embodiments.

[0235] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0236] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0237] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0238] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0239] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the model training method or data processing method described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, ROM, RAM, portable hard drives, magnetic disks, or optical disks.

[0240] The above description is only a preferred embodiment of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.

Claims

1. A model training method, comprising: Obtain a first training dataset and a second training dataset, wherein the first training dataset is monolingual instruction data described in the original language, and the second training dataset is monolingual instruction data described in the target language and cross-lingual instruction data described in the original language and the target language respectively; The initial language model is trained using the first training dataset to generate an intermediate language model; The intermediate language model is trained using the second training dataset to generate a target language model, wherein the target language model is used to parse query data in the target language to generate a target response in the target language.

2. The model training method according to claim 1, wherein, The model training method also includes: Obtain the first prompt text, wherein the first prompt text is used to prompt a response to the training instructions in the original language; The first training dataset is constructed based on the first prompt text.

3. The model training method according to claim 1, wherein, The model training method also includes: Obtain a second prompt text and a third prompt text, wherein the second prompt text is used to prompt a response to the training instructions in the target language, and the third prompt text is used to prompt cross-language understanding and conversion between the training instructions in the original language and the training instructions in the target language; Based on the second prompt text, monolingual instruction data describing the target language is constructed, and based on the third prompt text, cross-language instruction data is constructed. The second training dataset is determined using the monolingual instruction data described in the target language and the cross-lingual formal instruction data.

4. The model training method according to claim 3, wherein, Constructing the cross-language instruction data based on the third prompt text includes: Based on the third prompt text, the intermediate language model is controlled to learn language by simulating a multi-turn dialogue process, and the cross-language instruction data is constructed.

5. The model training method according to claim 4, wherein, The multi-turn dialogue process includes: a first-turn dialogue and a second-turn dialogue. Constructing the cross-language instruction data based on the third prompt text includes: Based on the third prompt text, the intermediate language model is controlled to perform language understanding and answering of the training instructions in the target language by simulating the first round of dialogue, so as to obtain the training instructions in the original language and the instruction responses in the original language. Based on the third prompt text, the intermediate language model simulates the second round of dialogue to perform language conversion on the instruction response in the original language, thereby obtaining the instruction response in the target language. The cross-language instruction data is constructed using the training instructions in the target language, the training instructions in the original language, the instruction responses in the original language, and the instruction responses in the target language.

6. The model training method according to claim 3, wherein, The third prompt text includes a scene description section and an instruction section. The scene description section is used to define the role of the intermediate language model as a multilingual role with the original language as its native language and to assign role responsibilities to the intermediate language model. The instruction section is used to guide the intermediate language model to construct the cross-language instruction data.

7. The model training method according to any one of claims 1 to 6, wherein, Training the intermediate language model using the second training dataset to generate the target language model includes: The intermediate language model is controlled to learn the course using the second training dataset to generate the target language model.

8. The model training method according to claim 7, wherein, Using the second training dataset to control the intermediate language model for course learning, generating the target language model includes: Training samples to be learned are selected from each batch of samples in the second training dataset according to the preset screening criteria. For each batch of samples in the second training dataset, the intermediate language model is controlled by the training samples to be learned to perform course learning for the target number of training rounds until all the training samples selected from each batch of samples in the second training dataset have been learned, thereby generating the target language model.

9. The model training method according to claim 8, wherein, The preset screening criteria are defined by the sample loss value. Selecting the training samples to be learned from each batch of samples in the second training dataset according to the preset screening criteria includes: Based on the comparison between the sample loss value and the course learning loss threshold, the training samples to be learned are selected from each sample batch of the second training dataset, wherein the sample loss value is used to determine the learning difficulty of multiple samples contained in each sample batch of the second training dataset.

10. The model training method according to claim 9, wherein, The model training method also includes: Obtain the initial loss for the first batch of samples in the second training dataset; Calculate the mean and standard deviation of the initial loss respectively; The loss threshold is initialized based on the average value and the target number of training rounds to obtain an initial threshold; Based on the range of values ​​of the standard deviation, the initial threshold is updated to initialize the loss threshold with the updated threshold, or the initial iteration coefficients are updated to obtain updated iteration coefficients, wherein the updated iteration coefficients are used to gradually adjust the loss threshold in multiple training rounds corresponding to the target training round number.

11. The model training method according to claim 8, wherein, The preset filtering conditions are defined by sample parameters, which include at least one of the following: sample loss value, instruction length, syntactic complexity, semantic complexity, contextual dependence of the instruction, and instruction category.

12. A data processing method, comprising: Obtain query data in the target language; The target language model is used to parse the query data in the target language to obtain the target response in the target language; The target language model is generated according to the model training method described in any one of claims 1 to 11.

13. A data processing method, comprising: Obtain Chinese astronomical knowledge query data; The target language model is used to query and parse the Chinese astronomical knowledge query data to obtain the Chinese astronomical knowledge query response; The target language model is generated according to the model training method described in any one of claims 1 to 11, and the target language model uses English as its native language.

14. A data processing method, comprising: A data processing request is obtained through a first application programming interface, wherein the request data carried in the data processing request includes: query data in the target language; The data processing response is returned through the second application programming interface, wherein the response data carried in the data processing response includes: the target response in the target language, which is obtained by querying and parsing the query data in the target language using a target language model, and the target language model is generated according to the model training method described in any one of claims 1 to 11.

15. A data processing method, comprising: Obtain the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: query data in the target language; In response to the data processing dialogue request, a data processing dialogue response is returned, wherein the information carried in the data processing dialogue response includes: a target response in the target language, which is obtained by querying and parsing the query data in the target language using a target language model, and the target language model is generated according to the model training method described in any one of claims 1 to 11; The target response in the target language is displayed within the graphical user interface.

16. A data processing method, comprising: In response to input commands applied to the user interface, query data in the target language is displayed on the user interface. In response to the processing instructions applied to the operation interface, the target response in the target language is displayed on the operation interface; The target response in the target language is obtained by parsing the query data in the target language using a target language model, and the target language model is generated according to the model training method described in any one of claims 1 to 11.

17. A data processing system, comprising: The client is configured to send query data in the target language; The server, connected to the client, is configured to use a target language model to query and parse the query data in the target language to obtain the target response in the target language; The client is also configured to output a target response in the target language; The target language model is generated according to the model training method described in any one of claims 1 to 11.

18. An electronic device comprising: Memory, which stores executable programs; A processor is configured to run the program, wherein the program, when running, performs the model training method according to any one of claims 1 to 11 or the data processing method according to any one of claims 12 to 16.

19. A computer-readable storage medium comprising a stored executable program, wherein, When the executable program is executed, it controls the device containing the computer-readable storage medium to perform the model training method according to any one of claims 1 to 11 or the data processing method according to any one of claims 12 to 16.

20. A computer program product comprising a computer program that, when executed by a processor, implements the model training method of any one of claims 1 to 11 or the data processing method of any one of claims 12 to 16.

Images