Sample data generation method and device based on artificial intelligence, equipment and medium
The method addresses the complexity and cost of traditional translation and style transfer by using AI to generate sample data efficiently, optimizing the model through low-rank adaptation and direct preference optimization, ensuring accuracy and reliability.
Patent Information
- Application Number
- CN202411256512.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-27
AI Technical Summary
Traditional translation and style transfer models require separate training processes, increasing complexity and cost, and rely heavily on high-quality parallel corpus samples, which are difficult to obtain, especially for small languages or specific fields.
A method and device for generating sample data using artificial intelligence, which includes extracting source and target styles from a request, preprocessing text data, constructing prompt words, and using a large language model to generate parallel corpus pairs, followed by grammar and spelling verification, and optimizing the style transfer model through low-rank adaptation and direct preference optimization.
Reduces the workload of collecting training samples and improves efficiency in generating sample data for style transfer models, ensuring accuracy and reliability through intelligent data generation and optimization.
Smart Images

Figure CN119203928A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to artificial intelligence-based sample data generation methods, devices, computer equipment and storage media. Background Art
[0002] In the field of natural language processing (NLP), translation and style transfer, as two important research directions, have long attracted widespread attention and research. At present, translation models are increasingly used in the field of financial technology. At present, the translation models used in financial companies often focus on improving the accuracy and fluency of translation, and train the models through large-scale parallel corpora to achieve accurate conversion from source language to target language. Similarly, the style transfer task also relies on a large amount of annotated data to learn and convert the style of the text, such as converting the text from a formal style to a colloquial style.
[0003] However, traditional translation model training strategies encounter significant challenges when dealing with the joint task of translation and style transfer. Specifically, such methods usually require that the translation task and style transfer task be handled separately as two independent subtasks, that is, first training a translation model to achieve conversion between languages, and then training a style transfer model to adjust the style of the translation results. This cascade processing method not only increases the complexity and training cost of the model, but more importantly, it relies heavily on high-quality parallel corpus samples, which are often difficult to obtain and costly in reality, especially for translation and style transfer tasks in small languages or specific fields. Therefore, in the construction process of existing style transfer models, there is a large workload in collecting parallel corpus samples. Summary of the invention
[0004] The purpose of the embodiments of the present application is to propose a sample data generation method, device, computer equipment and storage medium based on artificial intelligence to solve the technical problem that the collection of parallel corpus samples in the construction process of the existing style transfer model has a large workload.
[0005] In order to solve the above technical problems, the embodiment of the present application provides a sample data generation method based on artificial intelligence, which adopts the following technical solutions:
[0006] Receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language;
[0007] Determining whether target style data matching the target migration style is stored;
[0008] If the target style data is not stored, collecting text data corresponding to the source language, and preprocessing the text data to obtain the corresponding first source language text;
[0009] Constructing a corresponding first prompt word based on the target transfer style, and constructing a corresponding second prompt word based on the general style;
[0010] Inputting the first source language text, the first prompt word, and the second prompt word into a preset large language model, obtaining a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and obtaining a second translation text corresponding to the first source language text output by the large language model according to the second prompt word;
[0011] Building a parallel corpus pair based on the first source language text, the first translated text, and the second translated text;
[0012] First sample data corresponding to a preset style transfer model is generated based on the parallel corpus pair.
[0013] Furthermore, the step of generating first sample data corresponding to a preset style transfer model based on the parallel corpus pair specifically includes:
[0014] Call the preset spelling checker and grammar checker;
[0015] Performing spelling verification on the parallel corpus pair based on the spelling checker;
[0016] If the parallel corpus pair passes the spelling verification, performing grammar verification on the parallel corpus pair based on the grammar checker;
[0017] If the parallel corpus pair passes the grammar verification, the parallel corpus pair is used as the first sample data of the style transfer model.
[0018] Furthermore, the step of preprocessing the text data to obtain the corresponding first source language text specifically includes:
[0019] Performing word segmentation processing on the text data to obtain corresponding first text data;
[0020] Remove stop words from the first text data to obtain corresponding second text data;
[0021] Performing format normalization processing on the second text data to obtain corresponding third text data;
[0022] The third text data is used as the first source language text.
[0023] Furthermore, after the step of determining whether target style data matching the target migration style is stored, the method further includes:
[0024] If target style data matching the target transfer style is stored, calling the large language model;
[0025] Based on the source language, back-translate the target style data using the large language model to obtain a corresponding second source language text;
[0026] Based on the common style, using the large language model to translate the second source language text to obtain a corresponding third translated text;
[0027] Second sample data of the style transfer model is generated based on the target style data, the second source language text, and the third translated text.
[0028] Furthermore, after the step of generating first sample data corresponding to a preset style transfer model based on the parallel corpus pair, the method further includes:
[0029] Dividing the first sample data into a training data set and a test data set based on a preset division value;
[0030] Training and fine-tuning the style transfer model based on the training data set and a preset low-rank adaptation algorithm to obtain a corresponding first style transfer model;
[0031] Optimizing the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model;
[0032] Performing an effect evaluation on the second style transfer model based on the test data set;
[0033] If the second style transfer model passes the effect evaluation, the second style transfer model is used as a target translation model corresponding to the translation style transfer task, and the target translation model is stored.
[0034] Furthermore, the step of optimizing the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model specifically includes:
[0035] Backing up the first style transfer model to obtain a corresponding designated model;
[0036] Adjusting the specified model based on a preset comparison model construction strategy to obtain a corresponding comparison model;
[0037] Optimizing the first style transfer model based on the comparison model and a preset direct preference optimization algorithm to obtain an optimized first style transfer model;
[0038] The optimized first style transfer model is used as the second style transfer model.
[0039] Furthermore, the step of evaluating the effect of the second style transfer model based on the test data set specifically includes:
[0040] Inputting the test data set into the second style transfer model for performance testing, and obtaining a performance indicator value of the second style transfer model corresponding to a preset performance indicator; wherein the number of the performance indicators includes a plurality;
[0041] Determine whether all the performance indicator values are within the corresponding preset value range;
[0042] If yes, it is determined that the second style transfer model passes the effect evaluation;
[0043] If not, it is determined that the second style transfer model fails the effect evaluation.
[0044] In order to solve the above technical problems, the embodiment of the present application further provides a sample data generation device based on artificial intelligence, which adopts the following technical solution:
[0045] An extraction module is used to receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language;
[0046] A judgment module, used to judge whether target style data matching the target migration style is stored;
[0047] a processing module, configured to collect text data corresponding to the source language if the target style data is not stored, and preprocess the text data to obtain a corresponding first source language text;
[0048] A first construction module is used to construct a corresponding first prompt word based on the target transfer style, and to construct a corresponding second prompt word based on the general style;
[0049] an acquisition module, configured to input the first source language text, the first prompt word, and the second prompt word into a preset large language model, acquire a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and acquire a second translation text corresponding to the first source language text output by the large language model according to the second prompt word;
[0050] A second construction module, configured to construct a parallel corpus pair based on the first source language text, the first translated text, and the second translated text;
[0051] The first generating module is used to generate first sample data corresponding to a preset style transfer model based on the parallel corpus pair.
[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:
[0053] Receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language;
[0054] Determining whether target style data matching the target migration style is stored;
[0055] If the target style data is not stored, collecting text data corresponding to the source language, and preprocessing the text data to obtain the corresponding first source language text;
[0056] Constructing a corresponding first prompt word based on the target transfer style, and constructing a corresponding second prompt word based on the general style;
[0057] Inputting the first source language text, the first prompt word, and the second prompt word into a preset large language model, obtaining a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and obtaining a second translation text corresponding to the first source language text output by the large language model according to the second prompt word;
[0058] Building a parallel corpus pair based on the first source language text, the first translated text, and the second translated text;
[0059] First sample data corresponding to a preset style transfer model is generated based on the parallel corpus pair.
[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:
[0061] Receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language;
[0062] Determining whether target style data matching the target migration style is stored;
[0063] If the target style data is not stored, collecting text data corresponding to the source language, and preprocessing the text data to obtain the corresponding first source language text;
[0064] Constructing a corresponding first prompt word based on the target transfer style, and constructing a corresponding second prompt word based on the general style;
[0065] Inputting the first source language text, the first prompt word, and the second prompt word into a preset large language model, obtaining a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and obtaining a second translation text corresponding to the first source language text output by the large language model according to the second prompt word;
[0066] Building a parallel corpus pair based on the first source language text, the first translated text, and the second translated text;
[0067] First sample data corresponding to a preset style transfer model is generated based on the parallel corpus pair.
[0068] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0069] The present application first receives a sample generation request corresponding to a translation style transfer task input by a user, and extracts a source language, a target transfer style, and a general style from the sample generation request; then determines whether target style data matching the target transfer style is stored; if the target style data is not stored, collects text data corresponding to the source language, and preprocesses the text data to obtain a corresponding first source language text; then constructs a corresponding first prompt word based on the target transfer style, and constructs a corresponding second prompt word based on the general style; subsequently, the first source language text, the first prompt word, and the second prompt word are input into a preset large language model, and a first translation text corresponding to the first source language text output by the large language model according to the first prompt word is obtained, and a second translation text corresponding to the first source language text output by the large language model according to the second prompt word is obtained; further a parallel corpus pair is constructed based on the first source language text, the first translation text, and the second translation text; finally, a first sample data corresponding to a preset style transfer model is generated based on the parallel corpus pair. The present application obtains the source language, target transfer style and general style from the sample generation request corresponding to the translation style transfer task input by the user, and then based on the prompt word guidance mechanism of the large language model, can quickly and intelligently generate sample data of the style transfer model according to the translation samples output by the large language model and the collected source language text, effectively reducing the workload of collecting training samples in the process of building the style transfer model and improving the efficiency of sample data generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0071] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0072] Figure 2 A flowchart of an embodiment of a method for generating sample data based on artificial intelligence according to the present application;
[0073] Figure 3 is a structural schematic diagram of an embodiment of a sample data generation device based on artificial intelligence according to the present application;
[0074] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION
[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0076] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0077] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0078] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0079] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.
[0080] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and the like.
[0081] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0082] It should be noted that the artificial intelligence-based sample data generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based sample data generation device is generally set in the server / terminal device.
[0083] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0084] Continue to refer Figure 2 , showing a flow chart of an embodiment of a sample data generation method based on artificial intelligence according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The sample data generation method based on artificial intelligence provided in the embodiment of the present application can be applied to any scenario where sample data generation is required, and the sample data generation method based on artificial intelligence can be applied to products in these scenarios, for example, sample data generation processing corresponding to translation style transfer tasks in the financial insurance field. The sample data generation method based on artificial intelligence comprises the following steps:
[0085] Step S201, receiving a sample generation request corresponding to a translation style transfer task input by a user, and extracting a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language.
[0086] In this embodiment, the sample data generation method based on artificial intelligence is executed on the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the sample generation request through a wired connection mode or a wireless connection mode. It should be pointed out that the above-mentioned wireless connection mode may include but is not limited to 3G / 4G / 5G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (ultrawi deband) connection, and other wireless connection modes known now or developed in the future. In the process of performing the translation style transfer task, the user can customize the translation style and transfer style target required for the translation style transfer task according to his own needs, and trigger the sample generation request carrying the source language, as well as the target transfer style and general style corresponding to the target language, which effectively improves the user's experience. Exemplarily, the above-mentioned translation style is Chinese to English, the above-mentioned source language can be Chinese, the above-mentioned target language can be English, and the above-mentioned transfer style target is transferred from the "general" style to the "written" style, that is, the above-mentioned target transfer style is a written style, and the above-mentioned general style can also be called the original style.
[0087] Step S202: determine whether target style data matching the target migration style is stored.
[0088] In this embodiment, in the process of performing the translation style transfer task, there may be a case where target style data matching the target transfer style is pre-collected and stored, or there may be a case where target style data matching the target transfer style is not pre-collected and stored.
[0089] Step S203: if the target style data is not stored, then collect text data corresponding to the source language, and pre-process the text data to obtain the corresponding first source language text.
[0090] In this embodiment, the specific implementation process of preprocessing the text data to obtain the corresponding first source language text will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.
[0091] Step S204: constructing a corresponding first prompt word based on the target transfer style, and constructing a corresponding second prompt word based on the general style.
[0092] In this embodiment, there is no specific limitation on the construction method of the first prompt word and the second prompt word. It is only necessary to ensure that the large language model can be guided to generate a translation result in a style corresponding to the prompt word. For example, if the target language is English, the target migration style is the written style, and the universal style is the original style, the corresponding first prompt word may be "Translate the following Chinese into written English:", and the corresponding second prompt word may be "Translate the following Chinese into English:". By using the first prompt word and the second prompt word to guide the large language model, the large language model can generate a written style English translation text and a universal style English translation text corresponding to the first source language text.
[0093] Step S205: input the first source language text, the first prompt word, and the second prompt word into a preset large language model, obtain a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and obtain a second translation text corresponding to the first source language text output by the large language model according to the second prompt word.
[0094] In this embodiment, the first source language text, the first prompt word and the second prompt word are input into a preset large language model, and for each first source language text, the first prompt word and the second prompt word are used for translation through the large language model to obtain an English translation text in a written style (i.e., the first translation text) and an English translation text in a general style (i.e., the second translation text) corresponding to the first source language text.
[0095] Step S206: construct a parallel corpus pair based on the first source language text, the first translated text, and the second translated text.
[0096] In this embodiment, the first source language text, the first translation text and the second translation text are arranged into a parallel prediction pair. Specifically, the parallel prediction pair includes a first source language text and two corresponding style translation texts (i.e., a first translation text corresponding to the target transfer style and a second translation text corresponding to the universal style).
[0097] Step S207: generating first sample data corresponding to a preset style transfer model based on the parallel corpus pair.
[0098] In this embodiment, the style transfer model can specifically adopt the large language model. The specific implementation process of generating the first sample data corresponding to the preset style transfer model based on the parallel corpus will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated here.
[0099] The present application obtains the source language, target transfer style and general style from the sample generation request corresponding to the translation style transfer task input by the user, and then based on the prompt word guidance mechanism of the large language model, can quickly and intelligently generate sample data of the style transfer model according to the translation samples output by the large language model and the collected source language text, effectively reducing the workload of collecting training samples in the process of building the style transfer model and improving the efficiency of sample data generation.
[0100] In some optional implementations, step S207 includes the following steps:
[0101] Invoke the default spelling checker and grammar checker.
[0102] In this embodiment, the spell checker is a pre-built automated detection tool with a detection function that can quickly detect whether there are potential spelling errors in the text. The grammar checker is a pre-built automated detection tool with a detection function that can quickly detect whether there are potential grammar errors in the text.
[0103] Performing spelling verification on the parallel corpus pair based on the spelling checker.
[0104] In this embodiment, the first parallel corpus is input into the spell checker for spelling verification. If the spell checker does not detect any spelling error in the first parallel corpus, it is determined that the parallel corpus pair has passed the spelling verification; otherwise, it is determined that the parallel corpus pair has not passed the spelling verification.
[0105] If the parallel corpus pair passes the spelling verification, then grammar verification is performed on the parallel corpus pair based on the grammar checker.
[0106] In this embodiment, the first parallel corpus is input into the grammar checker for grammar verification. If the grammar checker does not detect any grammatical error in the first parallel corpus, it is determined that the parallel corpus pair has passed the grammar verification; otherwise, it is determined that the parallel corpus pair has not passed the grammar verification.
[0107] If the parallel corpus pair passes the grammar verification, the parallel corpus pair is used as the first sample data of the style transfer model.
[0108] In this embodiment, only when it is determined that the parallel corpus pair has passed the verification at the same time, it indicates that the parallel corpus pair has passed the quality verification, and the parallel corpus pair can be directly used as the first sample data of the style transfer model. If the parallel corpus pair does not pass the quality verification, that is, it is found that the first parallel corpus has obvious semantic errors, grammatical errors or inconsistent styles, further manual correction or screening is required to ensure the accuracy and reliability of the generated parallel corpus pair.
[0109] This application calls a preset spelling checker and grammar checker; then performs spelling verification on the parallel corpus pair based on the spelling checker; if the parallel corpus pair passes the spelling verification, then performs grammar verification on the parallel corpus pair based on the grammar checker; if the parallel corpus pair passes the grammar verification, then the parallel corpus pair is used as the first sample data of the style transfer model. After completing the construction of the parallel corpus pair, this application will also intelligently use the spelling checker and grammar checker to perform quality verification on the parallel corpus pair. Only when it is determined that the parallel corpus pair has passed the verification at the same time, it indicates that the parallel corpus pair has passed the quality verification, so that the parallel corpus pair is subsequently directly used as the first sample data of the style transfer model, effectively achieving quality control processing for the generated parallel corpus pair, and ensuring the accuracy and reliability of the generated first sample data.
[0110] In some optional implementations of this embodiment, the preprocessing of the text data to obtain the corresponding first source language text in step S203 includes the following steps:
[0111] The text data is segmented to obtain corresponding first text data.
[0112] In this embodiment, a general word segmentation tool may be used to perform word segmentation processing on the text data, thereby obtaining corresponding first text data.
[0113] Stop words are removed from the first text data to obtain corresponding second text data.
[0114] In this embodiment, the first text data may be detected and processed by using a preset stop word list to find matching designated stop words from the first text data, and then the designated stop words may be deleted from the first text data to obtain corresponding second text data.
[0115] The second text data is subjected to format normalization processing to obtain corresponding third text data.
[0116] In this embodiment, the format normalization processing refers to performing punctuation normalization processing on the second text data, thereby obtaining corresponding third text data.
[0117] The third text data is used as the first source language text.
[0118] The present application can quickly and accurately complete the preprocessing of the text data by performing word segmentation, stop word removal and format normalization on the text data, thereby improving the processing efficiency and processing intelligence of the preprocessing of the text data, ensuring the data accuracy of the first source language text obtained, and facilitating improving the translation quality of the translation text obtained by subsequent translation processing of the first source language text.
[0119] In some optional implementations, after step S202, the electronic device may further perform the following steps:
[0120] If target style data matching the target transfer style is stored, the large language model is called.
[0121] In this embodiment, the above-mentioned large language model can specifically adopt a large model that supports multi-language translation and has good performance, such as the GPT series, T5, BART and other large models, which usually have powerful back-translation capabilities.
[0122] Based on the source language, the target style data is back-translated using the large language model to obtain a corresponding second source language text.
[0123] In this embodiment, if the target language is English and the source language is Chinese, the target style data in English can be back-translated into a text in Chinese, i.e., a second source language text, by using the large language model. By back-translating the target style data using the large language model, a text having a similar meaning to the target style data but in a different language can be generated.
[0124] Based on the common style, the second source language text is translated using the large language model to obtain a corresponding third translated text.
[0125] In this embodiment, the second source language text is translated back into the target language corresponding to the general style through the large language model, i.e., the third translation text. This translation process is performed without adding style hints to obtain a translation result that retains the general style features as much as possible.
[0126] Second sample data of the style transfer model is generated based on the target style data, the second source language text, and the third translated text.
[0127] In this embodiment, the target style data, the second source language text, and the third translation text are organized into a specified parallel expected pair, thereby obtaining the corresponding second sample data. The specified parallel expected pair includes a source language text and two style translation results corresponding thereto (i.e., the target style data corresponding to the target transfer style, and the third translation text corresponding to the universal style).
[0128] If the present application detects that target style data matching the target transfer style is stored, the large language model is called; then based on the source language, the target style data is back-translated using the large language model to obtain a corresponding second source language text; then based on the general style, the second source language text is translated using the large language model to obtain a corresponding third translated text; subsequently, the second sample data of the style transfer model is generated based on the target style data, the second source language text, and the third translated text. In the case where the present application stores target style data matching the target transfer style, the target style data can be back-translated based on the use of a large language model to obtain a second source language text, and the second source language text can be translated using the large language model to obtain a third translated text corresponding to the general style, and then the second sample data of the style transfer model can be quickly and intelligently generated based on the target style data, the second source language text, and the third translated text, effectively reducing the workload of collecting training samples during the construction of the style transfer model and improving the efficiency of generating the second sample data.
[0129] In some optional implementations, after step S207, the electronic device may further perform the following steps:
[0130] The first sample data is divided into a training data set and a test data set based on a preset division value.
[0131] In this embodiment, there is no specific limitation on the numerical value selection of the above division ratio, which can be set according to the actual model training requirements, for example, it can be set to 7:3.
[0132] The style transfer model is trained and fine-tuned based on the training data set and a preset low-rank adaptation algorithm to obtain a corresponding first style transfer model.
[0133] In the present embodiment, the above-mentioned style transfer model can specifically adopt the above-mentioned large language model. Since the number of parameters of the pre-trained large language model is huge, directly fine-tuning all parameters of the large language model is not only computationally expensive, but also easily leads to overfitting. Therefore, in the process of training the style transfer model using the training data set, the style transfer model is fine-tuned using the low-rank adaptation (LoRA) algorithm to obtain the corresponding first style transfer model. The LoRA algorithm approximates parameter changes by introducing a low-rank matrix, thereby greatly reducing the number of parameters that need to be optimized. Specifically, the LoRA algorithm simulates parameter changes by adding a low-rank matrix to the original large matrix, and only optimizes this low-rank matrix, while keeping the original large matrix unchanged.
[0134] The first style transfer model is optimized based on a preset optimization strategy to obtain a corresponding second style transfer model.
[0135] In this embodiment, the above-mentioned specific implementation process of optimizing the first style transfer model based on the preset optimization strategy to obtain the corresponding second style transfer model will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated in detail here.
[0136] An effect evaluation is performed on the second style transfer model based on the test data set.
[0137] In this embodiment, the specific implementation process of evaluating the effect of the second style transfer model based on the test data set will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.
[0138] If the second style transfer model passes the effect evaluation, the second style transfer model is used as a target translation model corresponding to the translation style transfer task, and the target translation model is stored.
[0139] In this embodiment, if the second style transfer model passes the effect evaluation, it indicates that the model effect of the generated second style transfer model has achieved the expected performance effect, and the second style transfer model will be directly used as the target translation model corresponding to the translation style transfer task. There is no specific limitation on the storage method of the above target translation model, for example, blockchain storage, local database storage, cloud server storage, etc. can be used.
[0140] The present application divides the first sample data into a training data set and a test data set based on a preset division value; then the style transfer model is trained and fine-tuned based on the training data set and a preset low-rank adaptation algorithm to obtain a corresponding first style transfer model; then the first style transfer model is optimized based on a preset optimization strategy to obtain a corresponding second style transfer model; the second style transfer model is subsequently evaluated based on the test data set; if the second style transfer model passes the effect evaluation, the second style transfer model is used as the target translation model corresponding to the translation style transfer task, and the target translation model is stored. In the process of constructing the target translation model, the present application uses a low-rank adaptation algorithm to fine-tune the model, which can effectively save the resources required for the construction of the target translation model, thereby improving the construction efficiency of the target translation model. In addition, by using an optimization strategy to optimize the model, the target translation model can be more inclined to generate text that conforms to the target transfer style, thereby improving the translation ability of the target translation model to the target transfer style. In addition, after the construction of the target translation model is completed, the target translation model can be stored to ensure the security of the target translation model.
[0141] In some optional implementations of this embodiment, the step of optimizing the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model includes the following steps:
[0142] The first style transfer model is backed up to obtain a corresponding designated model.
[0143] In this embodiment, the above-mentioned designated model is a backup of the first style transfer model, and is used to provide comparison during the training process of the first style transfer model.
[0144] The specified model is adjusted based on a preset comparison model construction strategy to obtain a corresponding comparison model.
[0145] In this embodiment, the above-mentioned contrast model construction strategy refers to connecting a classifier after the output of the specified model to distinguish the styles of the translation results, and training based on a preset training goal to obtain a contrast model. Among them, the classifier can be a simple fully connected layer or a more complex neural network. The above-mentioned training goal is to output 0 when a translation text sample corresponding to the general style is input, and output 1 when a translation text sample corresponding to the target transfer style is input.
[0146] The first style transfer model is optimized based on the comparison model and a preset direct preference optimization algorithm to obtain an optimized first style transfer model.
[0147] In this embodiment, Direct Preference Optimization (DPO) is an innovative language model training method that directly uses human preference data to optimize the model through contrastive learning, making it more inclined to generate text that conforms to the target transfer style. Further using DPO technology on the basis of LoRA fine-tuning can more accurately adjust the model's preference for the target transfer style and improve the model's ability to generate target transfer style text. Specifically, after fine-tuning the style transfer model using a low-rank adaptation algorithm to obtain a first style transfer model, the DPO optimization algorithm is used to perform contrastive learning fine-tuning on the corpus Y0 (translation text samples corresponding to the general style) and Y1 (translation text samples corresponding to the target transfer style) contained in the training data set, where the loss function L is defined as:
[0148] L=-log{S i gmo id[log(M(Y1) / M0(Y1))-log((M(Y0)) / M0(Y1))]}.
[0149] Wherein, M is the current model, that is, the model output of the first style transfer model, and M0 is the model output of the comparison model. Through this loss function L, the first style transfer model is optimized in the direction of translating the target transfer style text and avoiding translating the general style text. By using the above loss function L to calculate the loss value, the model parameters of the first style transfer model are updated using the gradient descent method or other optimization algorithms to minimize the above loss function L until the preset training rounds are reached, thereby obtaining the above second style transfer model. The numerical selection of the above training rounds can be set according to the actual model training requirements.
[0150] The optimized first style transfer model is used as the second style transfer model.
[0151] The present application performs a backup process on the first style transfer model to obtain a corresponding designated model; then adjusts the designated model based on a preset comparison model construction strategy to obtain a corresponding comparison model; then optimizes the first style transfer model based on the comparison model and a preset direct preference optimization algorithm to obtain an optimized first style transfer model; and subsequently uses the optimized first style transfer model as the second style transfer model. The present application adjusts the designated model obtained by backing up the first style transfer model using a comparison model construction strategy to obtain a comparison model, and then optimizes the first style transfer model based on the comparison model and a direct preference optimization algorithm, so as to achieve fast and accurate optimization of the first style transfer model and obtain the corresponding second style transfer model, thereby improving the processing efficiency of the model optimization process. And by using a direct preference optimization algorithm to optimize the model, the generated second style transfer model can be more inclined to generate text that conforms to the target transfer style, thereby effectively improving the translation ability of the second style transfer model for the target transfer style.
[0152] In some optional implementations of this embodiment, the performing effect evaluation on the second style transfer model based on the test data set includes the following steps:
[0153] The test data set is input into the second style transfer model for performance testing, and a performance indicator value of the second style transfer model corresponding to a preset performance indicator is obtained; wherein the number of the performance indicators includes multiple ones.
[0154] In this embodiment, the test data set is input into the second style transfer model for performance testing, and after the performance test of the second style transfer model is completed, the performance indicator value corresponding to the preset performance indicator is recorded. The above performance indicator specifically includes the accuracy indicator of the translation result and the success rate indicator of the style transfer. Correspondingly, the above performance indicator value includes the accuracy value of the translation result and the success rate value of the style transfer.
[0155] Determine whether all of the performance indicator values are within the corresponding preset value range.
[0156] In this embodiment, according to the actual model performance test requirements, a preset numerical range that meets normal performance expectations is pre-constructed for each performance indicator. Specifically, an accuracy numerical range that matches the accuracy indicator of the translation result and a success rate numerical range that matches the success rate indicator of the style transfer are pre-constructed.
[0157] If so, it is determined that the second style transfer model passes the effect evaluation.
[0158] In this embodiment, if all of the performance indicator values are within the corresponding preset numerical range, it indicates that the model effect of the generated second style transfer model (the accuracy of the translation result and the success rate of the style transfer) has achieved the expected effect, and it will be determined that the second style transfer model has passed the effect evaluation.
[0159] If not, it is determined that the second style transfer model fails the effect evaluation.
[0160] In this embodiment, if at least one performance indicator value among all the performance indicator values is not within the corresponding preset numerical range, it is determined that the second style transfer model fails the effect evaluation.
[0161] The present application inputs the test data set into the second style transfer model, obtains the performance indicator value of the second style transfer model corresponding to the preset performance indicator, and then performs an effect evaluation on the obtained performance indicator value of the second style transfer model corresponding to the performance indicator based on the use of a preset numerical range, thereby being able to accurately generate an effect evaluation result corresponding to the second style transfer model, thereby ensuring the accuracy of the obtained effect evaluation result.
[0162] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0163] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned first sample data, the above-mentioned first sample data can also be stored in a node of a blockchain.
[0164] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0165] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0166] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0168] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0169] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a sample data generation device based on artificial intelligence, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.
[0170] like Figure 3 As shown, the sample data generating device 300 based on artificial intelligence described in this embodiment includes: an extraction module 301, a judgment module 302, a processing module 303, a first construction module 304, an acquisition module 305, a second construction module 306 and a first generation module 307. Among them:
[0171] The extraction module 301 is used to receive a sample generation request corresponding to a translation style transfer task input by a user, and extract the source language, the target transfer style, and the common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language;
[0172] A determination module 302 is used to determine whether target style data matching the target migration style is stored;
[0173] The processing module 303 is used for collecting text data corresponding to the source language if the target style data is not stored, and preprocessing the text data to obtain the corresponding first source language text;
[0174] A first construction module 304 is used to construct a corresponding first prompt word based on the target transfer style, and to construct a corresponding second prompt word based on the general style;
[0175] An acquisition module 305 is configured to input the first source language text, the first prompt word, and the second prompt word into a preset large language model, acquire a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and acquire a second translation text corresponding to the first source language text output by the large language model according to the second prompt word;
[0176] A second construction module 306, configured to construct a parallel corpus pair based on the first source language text, the first translated text, and the second translated text;
[0177] The first generating module 307 is used to generate first sample data corresponding to a preset style transfer model based on the parallel corpus pair.
[0178] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based sample data generation method in the aforementioned implementation mode, and will not be repeated here.
[0179] In some optional implementations of this embodiment, the first generating module 307 includes:
[0180] Call submodule, used to call the preset spelling checker and grammar checker;
[0181] A first verification submodule, configured to perform spelling verification on the parallel corpus pair based on the spelling checker;
[0182] A second verification submodule, configured to perform grammar verification on the parallel corpus pair based on the grammar checker if the parallel corpus pair passes the spelling verification;
[0183] The first determination submodule is configured to use the parallel corpus pair as first sample data of the style transfer model if the parallel corpus pair passes grammar verification.
[0184] In some optional implementations of this embodiment, the processing module 303 includes:
[0185] A first processing submodule, used for performing word segmentation processing on the text data to obtain corresponding first text data;
[0186] A second processing submodule, configured to remove stop words from the first text data to obtain corresponding second text data;
[0187] A third processing submodule, configured to perform format normalization processing on the second text data to obtain corresponding third text data;
[0188] The second determining submodule is configured to use the third text data as the first source language text.
[0189] In some optional implementations of this embodiment, the sample data generating device based on artificial intelligence further includes:
[0190] A calling module, configured to call the large language model if target style data matching the target migration style is stored;
[0191] A back-translation module, configured to perform back-translation processing on the target style data based on the source language using the large language model to obtain a corresponding second source language text;
[0192] A translation module, configured to translate the second source language text based on the general style using the large language model to obtain a corresponding third translated text;
[0193] The second generating module is used to generate second sample data of the style transfer model based on the target style data, the second source language text, and the third translated text.
[0194] In some optional implementations of this embodiment, the sample data generating device based on artificial intelligence further includes:
[0195] A training module, used to train and fine-tune the style transfer model based on the training data set and a preset low-rank adaptation algorithm to obtain a corresponding first style transfer model;
[0196] An optimization module, configured to optimize the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model;
[0197] An evaluation module, used to evaluate the effect of the second style transfer model based on the test data set;
[0198] A determination module is used to use the second style transfer model as a target translation model corresponding to the translation style transfer task if the second style transfer model passes the effect evaluation, and store the target translation model.
[0199] In some optional implementations of this embodiment, the optimization module includes:
[0200] A backup submodule, used to perform backup processing on the first style transfer model to obtain a corresponding designated model;
[0201] An adjustment submodule, used to adjust the specified model based on a preset comparison model construction strategy to obtain a corresponding comparison model;
[0202] an optimization submodule, configured to optimize the first style transfer model based on the comparison model and a preset direct preference optimization algorithm to obtain an optimized first style transfer model;
[0203] The third determining submodule is configured to use the optimized first style transfer model as the second style transfer model.
[0204] In some optional implementations of this embodiment, the evaluation module includes:
[0205] A testing submodule, configured to input the test data set into the second style transfer model for performance testing, and obtain a performance indicator value of the second style transfer model corresponding to a preset performance indicator; wherein the number of the performance indicators includes a plurality;
[0206] A judgment submodule, used to judge whether all the performance indicator values are within the corresponding preset value range;
[0207] A first determination submodule, configured to determine that if so, the second style transfer model passes the effect evaluation;
[0208] The second determination submodule is used to determine if the second style transfer model fails the effect evaluation.
[0209] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0210] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.
[0211] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0212] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a sample data generation method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.
[0213] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the artificial intelligence-based sample data generation method.
[0214] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.
[0215] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based sample data generation method.
[0216] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0217] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A sample data generation method based on artificial intelligence, characterized in that: The steps include: Receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language; Determining whether target style data matching the target migration style is stored; If the target style data is not stored, collecting text data corresponding to the source language, and preprocessing the text data to obtain the corresponding first source language text; Constructing a corresponding first prompt word based on the target transfer style, and constructing a corresponding second prompt word based on the general style; Inputting the first source language text, the first prompt word, and the second prompt word into a preset large language model, obtaining a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and obtaining a second translation text corresponding to the first source language text output by the large language model according to the second prompt word; Building a parallel corpus pair based on the first source language text, the first translated text, and the second translated text; First sample data corresponding to a preset style transfer model is generated based on the parallel corpus pair.
2. The method for generating sample data based on artificial intelligence according to claim 1, characterized in that: The step of generating first sample data corresponding to a preset style transfer model based on the parallel corpus pair specifically includes: Call the preset spelling checker and grammar checker; Performing spelling verification on the parallel corpus pair based on the spelling checker; If the parallel corpus pair passes the spelling verification, performing grammar verification on the parallel corpus pair based on the grammar checker; If the parallel corpus pair passes the grammar verification, the parallel corpus pair is used as the first sample data of the style transfer model.
3. The method for generating sample data based on artificial intelligence according to claim 1, characterized in that: The step of preprocessing the text data to obtain the corresponding first source language text specifically includes: Performing word segmentation processing on the text data to obtain corresponding first text data; Remove stop words from the first text data to obtain corresponding second text data; Performing format normalization processing on the second text data to obtain corresponding third text data; The third text data is used as the first source language text.
4. The method for generating sample data based on artificial intelligence according to claim 1, characterized in that: After the step of determining whether target style data matching the target migration style is stored, the method further includes: If target style data matching the target transfer style is stored, calling the large language model; Based on the source language, back-translate the target style data using the large language model to obtain a corresponding second source language text; Based on the common style, using the large language model to translate the second source language text to obtain a corresponding third translated text; Second sample data of the style transfer model is generated based on the target style data, the second source language text, and the third translated text.
5. The method for generating sample data based on artificial intelligence according to claim 1, characterized in that: After the step of generating first sample data corresponding to a preset style transfer model based on the parallel corpus pair, the method further includes: Dividing the first sample data into a training data set and a test data set based on a preset division value; Training and fine-tuning the style transfer model based on the training data set and a preset low-rank adaptation algorithm to obtain a corresponding first style transfer model; Optimizing the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model; Performing an effect evaluation on the second style transfer model based on the test data set; If the second style transfer model passes the effect evaluation, the second style transfer model is used as a target translation model corresponding to the translation style transfer task, and the target translation model is stored.
6. The method for generating sample data based on artificial intelligence according to claim 5, characterized in that: The step of optimizing the first style transfer model based on a preset optimization strategy to obtain a corresponding second style transfer model specifically includes: Backing up the first style transfer model to obtain a corresponding designated model; Adjusting the specified model based on a preset comparison model construction strategy to obtain a corresponding comparison model; Optimizing the first style transfer model based on the comparison model and a preset direct preference optimization algorithm to obtain an optimized first style transfer model; The optimized first style transfer model is used as the second style transfer model.
7. The method for generating sample data based on artificial intelligence according to claim 5, characterized in that: The step of evaluating the effect of the second style transfer model based on the test data set specifically includes: Inputting the test data set into the second style transfer model for performance testing, and obtaining a performance indicator value of the second style transfer model corresponding to a preset performance indicator; wherein the number of the performance indicators includes a plurality; Determine whether all the performance indicator values are within the corresponding preset value range; If yes, it is determined that the second style transfer model passes the effect evaluation; If not, it is determined that the second style transfer model fails the effect evaluation.
8. A sample data generation device based on artificial intelligence, characterized in that: include: An extraction module is used to receive a sample generation request corresponding to a translation style transfer task input by a user, and extract a source language, a target transfer style, and a common style from the sample generation request; wherein the sample generation request carries the source language, and the target transfer style and the common style corresponding to the target language; A judgment module, used to judge whether target style data matching the target migration style is stored; a processing module, configured to collect text data corresponding to the source language if the target style data is not stored, and preprocess the text data to obtain a corresponding first source language text; A first construction module is used to construct a corresponding first prompt word based on the target transfer style, and to construct a corresponding second prompt word based on the general style; an acquisition module, configured to input the first source language text, the first prompt word, and the second prompt word into a preset large language model, acquire a first translation text corresponding to the first source language text output by the large language model according to the first prompt word, and acquire a second translation text corresponding to the first source language text output by the large language model according to the second prompt word; A second construction module, configured to construct a parallel corpus pair based on the first source language text, the first translated text, and the second translated text; The first generating module is used to generate first sample data corresponding to a preset style transfer model based on the parallel corpus pair.
9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the sample data generation method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the artificial intelligence-based sample data generation method as described in any one of claims 1 to 7 are implemented.
