Training data management method, training data management device, and readable storage medium

Abstract

This application discloses a training data management method, a training data management device, and a computer-readable storage medium. The method includes: determining a target benchmark model based on a first selection operation received on a model selection interface, and obtaining a training dataset corresponding to the target benchmark model; determining a target training dataset corresponding to the target benchmark model based on a second selection operation received on a training dataset selection interface; determining a test dataset based on the target training dataset; training the target benchmark model based on the target training dataset; and testing the trained target benchmark model based on the test dataset. This solves the technical problem that test data determined using conventional methods lacks objectivity, leading to inaccurate test results, and achieves the acquisition of high-quality test data for accurate evaluation of the technical performance of translation models.

Description

Technical Field

[0001] This application relates to the field of translation model technology, and in particular to a training data management method, a training data management device, and a computer-readable storage medium. Background Technology

[0002] The construction of a translation model generally involves two main steps: training and testing. The training step involves learning statistical knowledge from the corpus and training the parameters. The testing step involves using selected test data to test the accuracy of the trained translation model in translating the original text into the test data.

[0003] In related technologies, the performance of a translation model typically depends on its ability to generalize to new samples. When evaluating a translation model, engineers need to determine test data to test it. Researchers usually obtain a batch of corpus data and then divide it into two parts according to a random ratio: one part for training and the other for testing.

[0004] However, test data determined using conventional methods lacks objectivity, leading to inaccurate test results. Summary of the Invention

[0005] This application provides a training data management method, a training data management device, and a computer-readable storage medium, which solves the technical problem in related technologies where the test data determined by conventional methods lacks objectivity, leading to inaccurate test results. It achieves the acquisition of higher-quality test data and accurate evaluation of the translation model's technical performance.

[0006] This application provides a training data management method, which includes:

[0007] Upon receiving a training task creation instruction, the system outputs the model selection interface corresponding to the training task.

[0008] Based on the first selection operation received from the model selection interface, the target benchmark model is determined, and the training dataset corresponding to the target benchmark model is obtained.

[0009] Output the selection interface for the training dataset, and determine the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface;

[0010] Determine the test dataset based on the target training dataset;

[0011] The target benchmark model is trained based on the target training dataset, and the trained target benchmark model is tested based on the test dataset.

[0012] Optionally, before the step of determining the test dataset based on the target training dataset, the method further includes:

[0013] Upon receiving the association processing instruction, the selected interface of the test dataset is output;

[0014] Based on the third selection operation received from the selection interface of the test dataset, determine the preset quantity and the name of the test dataset;

[0015] The step of determining the test dataset based on the target training dataset includes:

[0016] In the target training dataset, a preset number of training data are randomly selected to form the test dataset, and the test dataset is named according to the name to determine the test dataset;

[0017] After the step of determining the test dataset based on the target training dataset, the method further includes:

[0018] The test dataset is stored in association with the target training dataset.

[0019] Optionally, before the step of determining the test dataset based on the target training dataset, the method further includes:

[0020] When it is detected that the target training dataset does not have an associated test dataset, the association processing instruction is received, and the step of outputting the selected interface of the test dataset when the association processing instruction is received is executed.

[0021] When the target training dataset is detected to have an associated test dataset, the step of determining the test dataset based on the target training dataset includes:

[0022] Obtain the test dataset associated with the target training dataset.

[0023] Optionally, the step of randomly selecting the preset number of training data includes:

[0024] Determine the domain type and sentence pair length corresponding to the association processing instruction;

[0025] According to the preset quantity, training data that matches the domain type and the sentence pair length are randomly selected from the target training dataset.

[0026] Optionally, the step of outputting the selection interface for the training dataset and determining the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface includes:

[0027] Output the selection interface for the training dataset, wherein the training dataset is in the same language as the target benchmark model;

[0028] Upon receiving the second selection operation, determine the domain type and training set name corresponding to the second selection operation;

[0029] The target training dataset is determined based on the domain type and the training set name.

[0030] Optionally, the step of determining the target benchmark model based on the first selection operation received from the model selection interface and obtaining the training dataset corresponding to the target benchmark model includes:

[0031] Determine the model name, language type, and domain type corresponding to the first selected operation;

[0032] Identify the target benchmark model in the model selection interface that matches the model name, language type, and domain type.

[0033] Optionally, after the steps of training the target benchmark model based on the target training dataset and testing the trained target benchmark model based on the test dataset, the method further includes:

[0034] Output the target benchmark model after testing, and store the target benchmark model in the memory bank after naming it;

[0035] Upon receiving a query command for translation data, the corresponding translation data selection interface is output.

[0036] Based on the fourth selection operation received from the translation data selection interface, the target translation data is determined, and the details interface of the target translation data is output.

[0037] Based on the operation instructions received from the details interface, corresponding data processing operations are performed on the target translation data.

[0038] Optionally, the step of performing corresponding data processing operations on the target translation data based on the operation instructions received from the details interface includes at least one of the following:

[0039] Upon receiving a data insertion instruction, determine the base sentence pair corresponding to the data insertion instruction and the content to be inserted, and insert the content to be inserted after the base sentence pair;

[0040] Upon receiving a data merging instruction, the target sentence pair corresponding to the data merging instruction is determined, and the target sentence pair is merged into a single sentence pair.

[0041] Upon receiving a data import instruction, the system determines the data file to be imported corresponding to the data import instruction and updates the target translation data based on the data file to be imported.

[0042] In addition, this application also proposes a training data management device, which includes a memory, a processor, and a training data management program stored in the memory and executable on the processor. When the processor executes the training data management program, it implements the steps of the training data management method described above.

[0043] Furthermore, this application also proposes a computer-readable storage medium storing a training data management program, which, when executed by a processor, implements the steps of the training data management method described above.

[0044] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0045] 1. By employing a method that determines the target benchmark model based on a first selection operation received on the model selection interface and obtains the training dataset corresponding to the target benchmark model; determines the target training dataset corresponding to the target benchmark model based on a second selection operation received on the training dataset selection interface; determines the test dataset based on the target training dataset; trains the target benchmark model based on the target training dataset; and tests the trained target benchmark model based on the test dataset, the method effectively solves the technical problem of inaccurate test results due to the lack of objectivity of test data determined by conventional methods. This method achieves the acquisition of high-quality test data and accurate evaluation of the technical performance of the translation model.

[0046] 2. By employing a third selection operation received from the selection interface based on the test dataset, a preset quantity and name of the test dataset are determined. From the target training dataset, the preset quantity of training data is randomly selected to form the test dataset, which is then named according to the specified name. The test dataset is then associated and stored with the target training dataset. Therefore, this effectively solves the technical problem of inaccurate test results due to the lack of objectivity in test data determined using conventional methods, achieving the acquisition of high-quality test data and accurately evaluating the technical performance of the translation model. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating an embodiment of the training data management method of this application;

[0048] Figure 2 This is a flowchart illustrating Embodiment 2 of the training data management method of this application;

[0049] Figure 3 This is a flowchart illustrating Embodiment 3 of the training data management method of this application;

[0050] Figure 4 This is a schematic diagram of the hardware structure involved in the embodiment of the training data management device of this application. Detailed Implementation

[0051] In related technologies, researchers typically divide a batch of corpus data into two parts according to a random ratio after acquisition: one part as training data and the other as test data. However, the test data determined by this conventional method lacks objectivity, leading to inaccurate test results. The main technical solution adopted in this application is as follows: based on a first selection operation received on the model selection interface, a target benchmark model is determined, and the training dataset corresponding to the target benchmark model is obtained; based on a second selection operation received on the training dataset selection interface, a target training dataset corresponding to the target benchmark model is determined; a test dataset is determined based on the target training dataset; the target benchmark model is trained based on the target training dataset, and the trained target benchmark model is tested based on the test dataset. This achieves the acquisition of high-quality test data and accurate evaluation of the technical performance of the translation model.

[0052] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0053] Example 1

[0054] Embodiment 1 of this application discloses a training data management method, which is referred to... Figure 1 The training data management method includes:

[0055] Step S10: Upon receiving the instruction to create a training task, output the model selection interface corresponding to the training task.

[0056] Step S20: Based on the first selection operation received by the model selection interface, determine the target benchmark model and obtain the training dataset corresponding to the target benchmark model;

[0057] In this embodiment, the creation instruction is a click instruction on the "Create Task" button received by the client interface; the model selection interface is a selection window for the benchmark model, and the first selection operation is a selection instruction on the benchmark model attribute filtering received by the model selection interface; the domain type of the training dataset is the same as the domain type of the benchmark model.

[0058] Optionally, step S20 includes:

[0059] Step S21: Determine the model name, language type, and domain type corresponding to the first selected operation;

[0060] Step S22: Determine the target benchmark model in the model selection interface that matches the model name, the language type, and the domain type.

[0061] As an optional implementation, when a training task creation instruction is received on the client's homepage, a corresponding model selection interface is displayed, which shows a list of benchmark models. Based on the first selection operation received on the model selection interface, a target benchmark model is determined, and the training dataset corresponding to the target benchmark model is obtained.

[0062] For example, after receiving the "Create New Training Task" button, the client jumps to the model selection page, which displays different benchmark models. Based on the received selection operation, the client can filter out benchmark models that meet the conditions corresponding to the selected operation, according to the model name, source language, target language, and domain type. After receiving the instruction that a benchmark model has been selected, the client can filter out training datasets with the same language type based on the language type of the selected target benchmark model.

[0063] Step S30: Output the selection interface for the training dataset, and determine the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface.

[0064] In this embodiment, the training dataset selection interface displays datasets that can be used for training. The target training dataset is a training dataset determined according to the selected operation, which is used to train the target benchmark model.

[0065] Optionally, step S30 includes:

[0066] Step S31: Output the selection interface for the training dataset, wherein the training dataset is in the same language as the target benchmark model;

[0067] Step S32: Upon receiving the second selection operation, determine the domain type and training set name corresponding to the second selection operation;

[0068] Step S33: Determine the target training dataset based on the domain type and the training set name.

[0069] In this embodiment, the language type is divided into source language and target language. Same language type means that both the source language and the target language are the same. Domain type refers to the domain in which the training data or benchmark model is applied, such as general domain, biological domain, chemical domain, mathematical domain, news domain, etc., and is not specifically limited here.

[0070] As an optional implementation, a training dataset with the same language type as the target benchmark model is determined, and a corresponding selection interface is output based on the determined training dataset; when a second selection operation is received, the domain type and training set name corresponding to the second selection operation are determined; training datasets that meet the conditions are filtered according to the limited domain type and training set name, and the target training dataset is determined from the filtered training datasets according to the received click instruction.

[0071] For example, after determining the target benchmark model, training datasets with the same source language and the same target language are determined, and the user is redirected to the selection interface of the training dataset; training datasets that meet the conditions are selected according to the training set name and domain type filtering conditions corresponding to the second selection operation; and the corresponding target training dataset is determined according to the click instruction received from the selection interface after filtering.

[0072] Step S40: Determine the test dataset based on the target training dataset;

[0073] Step S50: Train the target benchmark model based on the target training dataset, and test the trained target benchmark model based on the test dataset.

[0074] In this embodiment, the test dataset is used to test the benchmark model after training. The test dataset can be a test dataset associated with the target training dataset, or it can be a test dataset selected according to the selection operation received from the test dataset selection interface.

[0075] As an optional implementation, after determining the target training dataset, when the "Use the test set associated with the training set" button is triggered on the training dataset selection interface, the test dataset associated with the target training dataset is determined; when the "Start Training" button is triggered, the benchmark model is trained using the target training dataset, and the trained benchmark model is tested using the test dataset.

[0076] As another optional implementation, after determining the target training dataset, when the "Next" button is triggered on the training dataset selection interface, a test dataset with the same language type as the target benchmark model is determined, and a test dataset selection interface is output; based on the selection operation received on the test dataset selection interface, the target test dataset is determined.

[0077] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0078] By employing a method that determines the target benchmark model based on a first selection operation received on the model selection interface and obtains the training dataset corresponding to the target benchmark model; determines the target training dataset corresponding to the target benchmark model based on a second selection operation received on the training dataset selection interface; determines the test dataset based on the target training dataset; trains the target benchmark model based on the target training dataset; and tests the trained target benchmark model based on the test dataset, the method effectively solves the technical problem of inaccurate test results caused by the lack of objectivity of test data determined by conventional methods. This approach achieves the acquisition of high-quality test data and accurate evaluation of the technical performance of the translation model.

[0079] Example 2

[0080] Based on Embodiment 1, Embodiment 2 of this application discloses a training data management method, referring to... Figure 2 Before step S40, the following are also included:

[0081] Step S210: Upon receiving the association processing instruction, output the selected interface of the test dataset;

[0082] Step S220: Based on the third selection operation received from the selection interface of the test dataset, determine the preset quantity and the name of the test dataset;

[0083] Optionally, step S40 includes:

[0084] Step S230: Randomly select the preset number of training data from the target training dataset to form the test dataset, and name the test dataset according to the name to determine the test dataset;

[0085] Optionally, after step S40, the following steps may also be included:

[0086] Step S240: Associate and store the test dataset with the target training dataset.

[0087] In this embodiment, the associated processing instruction is the display interface of the training dataset, which is triggered by the "preprocessing" button.

[0088] As an optional implementation, when an association processing instruction is received on the display interface of the training dataset, the target training dataset corresponding to the association processing instruction is determined, and a selection interface for the test dataset is output. The selection interface can determine the name, filtering method, and preset quantity of the test dataset according to the received third selection operation; the preset quantity and name of the test dataset are determined according to the third selection operation; when the "Start Preprocessing" button is detected to be triggered, a preset number of training data are randomly selected from the target training dataset, the training data are combined to form the test dataset, and the test dataset is named using the name corresponding to the third selection operation; after the test dataset is determined, the test dataset is associated and stored with the target training dataset.

[0089] Optionally, before step S40, the following may also be included:

[0090] Step S250: When it is detected that the target training dataset does not have an associated test dataset, the association processing instruction is received, and the step of outputting the selected interface of the test dataset when the association processing instruction is received is executed.

[0091] Step S260, when the target training dataset is detected to have an associated test dataset, the step of determining the test dataset based on the target training dataset includes:

[0092] Obtain the test dataset associated with the target training dataset.

[0093] As an optional implementation, after the target training dataset is selected in the training dataset selection interface, the selection interface pops up a "Use the test set associated with the training set" button; after the button is detected to be triggered, it is queried whether the target training set has an associated test dataset.

[0094] For example, when there is no associated test dataset, the system jumps to the display interface of the target training dataset. When the "Preprocessing" button is detected to be triggered, the system receives the association processing instruction and executes the step of outputting the selected interface of the test dataset when the association processing instruction is received.

[0095] For example, when it is detected that the target training dataset has an associated test dataset, the associated test dataset is obtained. The target training dataset may have undergone multiple preprocessing steps, and the associated test dataset is the test dataset obtained from the most recent preprocessing.

[0096] Optionally, step S230 includes:

[0097] Step S231: Determine the domain type and sentence pair length corresponding to the association processing instruction;

[0098] Step S232: Randomly extract training data from the target training dataset that matches the domain type and the sentence pair length, according to the preset quantity.

[0099] As an optional implementation, the target training dataset corresponding to the association processing instruction is determined, and the domain type and sentence pair length of the target training dataset are determined; training data is randomly extracted from the target training dataset according to the preset number corresponding to the third selected operation, wherein the extracted training data matches the domain type and the sentence pair length.

[0100] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0101] By employing a third selection operation received from the selection interface based on the test dataset, a preset quantity and name of the test dataset are determined. From the target training dataset, the preset quantity of training data is randomly selected to form the test dataset, which is then named according to the specified name. The test dataset is then associated and stored with the target training dataset. Therefore, this method effectively solves the technical problem of inaccurate test results due to the lack of objectivity in test data determined using conventional methods, achieving the acquisition of high-quality test data and accurately evaluating the technical performance of the translation model.

[0102] Example 3

[0103] Based on Embodiment 1, this application discloses a training data management method, referring to... Figure 3 After step S50, the following steps are also included:

[0104] Step S310: Output the target benchmark model after testing, and store the target benchmark model in the memory bank after naming it;

[0105] As an optional implementation, if the test result is higher than the expected value after the target benchmark model test is completed, it is determined that the training result of the target benchmark model meets the preset requirements, and the target benchmark model is named and stored in the memory bank.

[0106] Step S320: Upon receiving a query instruction for translation data, output the translation data selection interface corresponding to the query instruction;

[0107] Step S330: Based on the fourth selection operation received by the translation data selection interface, determine the target translation data and output the details interface of the target translation data;

[0108] In this embodiment, the memory stores the tested target benchmark model, and the translation data selection interface displays the target benchmark model. It can output target benchmark models that meet the filtering criteria based on received filtering instructions. The target translation data is the selected target benchmark model, and the details interface lists the translation data corresponding to the target benchmark model, such as the source text and the translated text.

[0109] As an optional implementation, when the "memory bank" button is triggered, the translation data selection interface is output. This interface displays the tested target benchmark model and its attribute information, such as name, language type, domain type, creator, and creation time. Based on the fourth selection operation received by the translation data selection interface, the filtering conditions corresponding to the fourth selection operation are determined, and target benchmark models that meet the filtering conditions are selected. After detecting that the information bar of the target benchmark model is clicked, the target benchmark model is identified as the target translation data, and the user is redirected to the details interface of the target translation data, which displays the translation data of the target benchmark model.

[0110] Step S340: Based on the operation instructions received from the details interface, perform corresponding data processing operations on the target translation data.

[0111] Optionally, step S340 includes at least one of the following:

[0112] Step S341: Upon receiving a data insertion instruction, determine the base sentence pair corresponding to the data insertion instruction and the content to be inserted, and insert the content to be inserted after the base sentence pair;

[0113] As an optional implementation, when the "Insert" button is detected to be triggered on the details screen, a base sentence pair to be inserted is determined, and the content to be inserted is inserted after the base sentence pair according to the content to be inserted corresponding to the data insertion instruction.

[0114] For example, the list number of the baseline sentence pair is 7. After the content to be inserted is added to the details interface, the list number of the content to be inserted is 8, and the list numbers of subsequent sentence pairs are sequentially extended.

[0115] Step S342: Upon receiving a data merging instruction, determine the target sentence pair corresponding to the data merging instruction, and merge the target sentence pair into a single sentence pair;

[0116] As an optional implementation, when the "Merge" button is triggered in the details interface, the target sentence pair to be merged is determined, and the target sentence pair is merged into one sentence pair. Specifically, the original text corresponding to the target sentence pair is integrated into paragraph one, and the translation of the target sentence pair is also integrated into paragraph two. Paragraph one is added to the original text of the merged sentence pair, and paragraph two is added to the translation of the merged sentence pair.

[0117] Step S343: Upon receiving a data import instruction, determine the data file to be imported corresponding to the data import instruction, and update the target translation data according to the data file to be imported.

[0118] As an optional implementation, when the "Import" button is detected to be triggered on the details screen, the bilingual file to be inserted is determined, and the target translation data is updated according to the original text and translation of the bilingual file.

[0119] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages:

[0120] By storing the tested target benchmark model in a memory bank, the target benchmark model can be managed according to the memory bank page. The page allows for adding, deleting, modifying, and querying the target benchmark model, and also supports receiving bilingual files to be imported and updating the target benchmark model. This effectively solves the technical problem in related technologies where optimizing and iterating the tested translation model requires retraining, thus achieving efficient optimization and iteration of the translation model and greatly saving the maintenance time and cost of the translation model.

[0121] This application also proposes a training data management device, referring to... Figure 4 , Figure 4 This is a schematic diagram of the training data management device structure of the hardware operating environment involved in the embodiments of this application.

[0122] like Figure 4As shown, the training data management device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0123] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the training data management device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0124] Optionally, the memory 1005 is electrically connected to the processor 1001. The processor 1001 can be used to control the operation of the memory 1005 and can also read the data in the memory 1005 to implement training data management.

[0125] Optionally, such as Figure 4 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a training data management program.

[0126] Optionally, in Figure 4 In the training data management device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with users; the processor 1001 and memory 1005 in the training data management device of this application can be set in the training data management device.

[0127] like Figure 4 As shown, the training data management device calls the training data management program stored in the memory 1005 through the processor 1001 and executes the relevant steps of the training data management method provided in this application embodiment:

[0128] Upon receiving a training task creation instruction, the system outputs the model selection interface corresponding to the training task.

[0129] Based on the first selection operation received from the model selection interface, the target benchmark model is determined, and the training dataset corresponding to the target benchmark model is obtained.

[0130] Output the selection interface for the training dataset, and determine the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface;

[0131] Determine the test dataset based on the target training dataset;

[0132] The target benchmark model is trained based on the target training dataset, and the trained target benchmark model is tested based on the test dataset.

[0133] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0134] Upon receiving the association processing instruction, the selected interface of the test dataset is output;

[0135] Based on the third selection operation received from the selection interface of the test dataset, determine the preset quantity and the name of the test dataset;

[0136] The step of determining the test dataset based on the target training dataset includes:

[0137] In the target training dataset, a preset number of training data are randomly selected to form the test dataset, and the test dataset is named according to the name to determine the test dataset;

[0138] After the step of determining the test dataset based on the target training dataset, the method further includes:

[0139] The test dataset is stored in association with the target training dataset.

[0140] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0141] When it is detected that the target training dataset does not have an associated test dataset, the association processing instruction is received, and the step of outputting the selected interface of the test dataset when the association processing instruction is received is executed.

[0142] When the target training dataset is detected to have an associated test dataset, the step of determining the test dataset based on the target training dataset includes:

[0143] Obtain the test dataset associated with the target training dataset.

[0144] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0145] Determine the domain type and sentence pair length corresponding to the association processing instruction;

[0146] According to the preset quantity, training data that matches the domain type and the sentence pair length are randomly selected from the target training dataset.

[0147] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0148] Output the selection interface for the training dataset, wherein the training dataset is in the same language as the target benchmark model;

[0149] Upon receiving the second selection operation, determine the domain type and training set name corresponding to the second selection operation;

[0150] The target training dataset is determined based on the domain type and the training set name.

[0151] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0152] Determine the model name, language type, and domain type corresponding to the first selected operation;

[0153] Identify the target benchmark model in the model selection interface that matches the model name, language type, and domain type.

[0154] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0155] Output the target benchmark model after testing, and store the target benchmark model in the memory bank after naming it;

[0156] Upon receiving a query command for translation data, the corresponding translation data selection interface is output.

[0157] Based on the fourth selection operation received from the translation data selection interface, the target translation data is determined, and the details interface of the target translation data is output.

[0158] Based on the operation instructions received from the details interface, corresponding data processing operations are performed on the target translation data.

[0159] Optionally, the processor 1001 may call the training data management program stored in the memory 1005 and further perform the following operations:

[0160] Upon receiving a data insertion instruction, determine the base sentence pair corresponding to the data insertion instruction and the content to be inserted, and insert the content to be inserted after the base sentence pair;

[0161] Upon receiving a data merging instruction, the target sentence pair corresponding to the data merging instruction is determined, and the target sentence pair is merged into a single sentence pair.

[0162] Upon receiving a data import instruction, the system determines the data file to be imported corresponding to the data import instruction and updates the target translation data based on the data file to be imported.

[0163] Furthermore, embodiments of this application also propose a computer-readable storage medium storing a training data management program, which, when executed by a processor, implements the relevant steps of any embodiment of the training data management method described above.

[0164] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. This application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0169] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0170] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A training data management method, characterized in that, The training data management method includes: Upon receiving a training task creation instruction, the system outputs the model selection interface corresponding to the training task. Based on the first selection operation received from the model selection interface, the target benchmark model is determined, and the training dataset corresponding to the target benchmark model is obtained. Output the selection interface for the training dataset, and determine the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface; Upon receiving an association processing instruction, the selected interface of the test dataset is output; Based on the third selection operation received from the selection interface of the test dataset, determine the preset quantity and the name of the test dataset; In the target training dataset, a preset number of training data are randomly selected to form the test dataset, and the test dataset is named according to the name to determine the test dataset; The test dataset is associated and stored with the target training dataset; The step of randomly selecting the preset number of training data includes: Determine the domain type and sentence pair length corresponding to the association processing instruction; According to the preset quantity, randomly select training data from the target training dataset that matches the domain type and the sentence pair length; The target benchmark model is trained based on the target training dataset, and the trained target benchmark model is tested based on the test dataset.

2. The training data management method as described in claim 1, characterized in that, Before the step of randomly selecting the preset number of training data from the target training dataset to form the test dataset, and naming the test dataset according to the name, to determine the test dataset, the method further includes: When it is detected that the target training dataset does not have an associated test dataset, the association processing instruction is received, and the step of outputting the selected interface of the test dataset when the association processing instruction is received is executed. When the target training dataset is detected and an associated test dataset is found, the step of determining the test dataset based on the target training dataset includes: Obtain the test dataset associated with the target training dataset.

3. The training data management method as described in claim 1, characterized in that, The steps of outputting the selection interface for the training dataset and determining the target training dataset corresponding to the target benchmark model based on the second selection operation received from the selection interface include: Output the selection interface for the training dataset, wherein the training dataset is in the same language as the target benchmark model; Upon receiving the second selection operation, determine the domain type and training set name corresponding to the second selection operation; The target training dataset is determined based on the domain type and the training set name.

4. The training data management method as described in claim 1, characterized in that, The steps of determining the target benchmark model based on the first selection operation received from the model selection interface and obtaining the training dataset corresponding to the target benchmark model include: Determine the model name, language type, and domain type corresponding to the first selected operation; Identify the target benchmark model in the model selection interface that matches the model name, language type, and domain type.

5. The training data management method as described in claim 1, characterized in that, After the steps of training the target benchmark model based on the target training dataset and testing the trained target benchmark model based on the test dataset, the method further includes: Output the target benchmark model after testing, and store the target benchmark model in the memory bank after naming it; Upon receiving a query command for translation data, the corresponding translation data selection interface is output. Based on the fourth selection operation received from the translation data selection interface, the target translation data is determined, and the details interface of the target translation data is output. Based on the operation instructions received from the details interface, corresponding data processing operations are performed on the target translation data.

6. The training data management method as described in claim 5, characterized in that, The step of performing corresponding data processing operations on the target translation data based on the operation instructions received from the details interface includes at least one of the following: Upon receiving a data insertion instruction, determine the base sentence pair corresponding to the data insertion instruction and the content to be inserted, and insert the content to be inserted after the base sentence pair; Upon receiving a data merging instruction, the target sentence pair corresponding to the data merging instruction is determined, and the target sentence pair is merged into a single sentence pair. Upon receiving a data import instruction, the system determines the data file to be imported corresponding to the data import instruction and updates the target translation data based on the data file to be imported.

7. A training data management device, characterized in that, The system includes a memory, a processor, and a training data management program stored in the memory and executable on the processor. When the processor executes the training data management program, it implements the steps of the training data management method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a training data management program, which, when executed by a processor, implements the steps of the training data management method as described in any one of claims 1 to 6.

Images