Training data length adjustment method and apparatus, device, medium and program product

Abstract

The present disclosure relates to a training data length adjustment method and apparatus, a device, a medium and a program product. The training data length adjustment method comprises: acquiring training data to be adjusted; on the basis of first data lengths of the training data, sorting the training data to obtain sorted data; splitting the sorted data to obtain ordered data buckets; and determining grouped data on the basis of the ordered data buckets, the grouped data being data obtained by length adjustment on the training data during model training. In the present disclosure, length adjustment is performed on training data by means of grouping processing on the basis of lengths, so as to minimize additional computing power consumption caused by data padding processing, thus improving overall computing efficiency and performance.

Description

Methods, apparatus, equipment, media, and procedures for adjusting training data length

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411472679.4, filed on October 21, 2024, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to a method, apparatus, device, medium, and program product for adjusting the length of training data. Background Technology

[0004] With the rapid development of artificial intelligence, large language models have achieved remarkable results in natural language processing tasks such as dialogue question answering, text generation, and text translation. Currently, training large models typically requires a large amount of computing resources and time. Furthermore, in practical applications, different tasks have their own unique requirements, and different scenarios have diverse needs. Therefore, it is often necessary to perform fine-tuning on pre-trained large models to better adapt them to specific task and scenario requirements.

[0005] Current methods for training and fine-tuning large models often use domain-specific datasets, such as question-answering task datasets. The data in these datasets are usually of inconsistent lengths, so padding is required to make the data the same length. However, this operation will affect the actual training efficiency.

[0006] Therefore, the related techniques suffer from low training efficiency in large model training and fine-tuning due to byte padding of the dataset. Summary of the Invention

[0007] In view of this, the present disclosure provides a method, apparatus, device, medium and program product for adjusting training data length.

[0008] In a first aspect, this disclosure provides a method for adjusting the length of training data, the method comprising:

[0009] Obtain the training data to be adjusted;

[0010] The training data is sorted according to the first data length to obtain sorted data;

[0011] The sorted data is split into ordered data buckets;

[0012] Grouped data is determined based on ordered data buckets, where grouped data is the data after length adjustment of the training data during model training.

[0013] In one optional implementation, the sorted data is split to obtain ordered data buckets, including:

[0014] Get the second data length of the sorted data;

[0015] Compare the length differences between the second data lengths;

[0016] Sorted data with length differences less than a threshold are grouped into the same bucket to obtain ordered data buckets.

[0017] In one alternative implementation, determining grouped data based on ordered data buckets includes:

[0018] Distribute the data within ordered data buckets to computing devices;

[0019] The data within the computing device is grouped to obtain grouped data.

[0020] In one optional implementation, data within the computing device is grouped to obtain grouped data, including:

[0021] Get the smallest unit value in the group;

[0022] The data within the computing device is grouped according to the smallest unit value of the group, resulting in grouped data.

[0023] In one alternative implementation, after determining the grouped data based on the ordered data buckets, the method further includes:

[0024] Randomize the grouped data;

[0025] The randomly arranged data is used as sample data for iteratively updating the model parameters during the model training process.

[0026] In one optional implementation, the training data is sorted according to a first data length to obtain sorted data, including:

[0027] The training data is concatenated according to a preset template format to obtain concatenated data;

[0028] Determine the third data length for the concatenated data;

[0029] The concatenated data is sorted according to the third data length to obtain sorted data.

[0030] Secondly, this disclosure provides a model training apparatus, which includes a dataset processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule.

[0031] The dataset processing submodule is used to perform the training data length adjustment method of the first aspect, obtain grouped data, and send the grouped data to the encoding submodule;

[0032] The encoding sub-model is used to segment the grouped data, obtain segmentation information, label the segmentation information, obtain label information, convert the label information into digital identifiers, and send the digital identifiers to the model training process calculation sub-module.

[0033] The model training process calculation submodule is used to identify digit identifiers, use the text information corresponding to the digit identifiers as training data for model training, and adopt a distributed strategy to encapsulate the model during the model training process.

[0034] The checkpoint submodule is used to save key information updated by the computation submodule during model training.

[0035] Thirdly, this disclosure provides an apparatus for adjusting the length of training data, the apparatus comprising:

[0036] The acquisition module is used to acquire the training data to be adjusted.

[0037] The first sorting module is used to sort the training data according to the first data length of the training data to obtain sorted data;

[0038] The splitting module is used to split the sorted data into ordered data buckets;

[0039] The module determines the grouped data based on the ordered data buckets. The grouped data is the data after the length of the training data has been adjusted during model training.

[0040] Fourthly, this disclosure provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the training data length adjustment method of the first aspect or any corresponding embodiment described above.

[0041] Fifthly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the training data length adjustment method of the first aspect or any corresponding embodiment described above.

[0042] In a sixth aspect, this disclosure provides a computer program product, including computer instructions for causing a computer to execute the training data length adjustment method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure, the accompanying drawings used in the description of the specific embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 is a flowchart illustrating a method for adjusting training data length according to an embodiment of the present disclosure;

[0045] Figure 2 is a schematic diagram of obtaining ordered data buckets according to an embodiment of the present disclosure;

[0046] Figure 3 is a schematic diagram of grouping data within a computing device according to an embodiment of the present disclosure;

[0047] Figure 4 is a schematic diagram of arranging grouped data in random order according to an embodiment of the present disclosure;

[0048] Figure 5 is a schematic diagram of the overall process of the method for adjusting the length of training data according to an embodiment of the present disclosure;

[0049] Figure 6 is an overall block diagram of a model training apparatus according to an embodiment of the present disclosure;

[0050] Figure 7 is a structural block diagram of a device for adjusting the training data length according to an embodiment of the present disclosure; and

[0051] Figure 8 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0053] In some large-scale application scenarios of dialogue question answering tasks, due to the diverse needs, it is necessary to carry out fine-tuning of the pre-trained large model so that it can better adapt to the specific task and scenario requirements.

[0054] Currently, in fine-tuning methods for large model training data, inconsistent data lengths are common. This necessitates padding the data to the same length, but the sheer volume of training data in the training dataset severely impacts training efficiency. To address this issue, this disclosure provides a method for adjusting training data length. It should be noted that the steps shown in the flowchart can be executed in a computer system, such as a set of computer-executable instructions. Furthermore, while a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order.

[0055] This embodiment provides a method for adjusting the length of training data. Figure 1 is a flowchart illustrating the method for adjusting the length of training data according to an embodiment of this disclosure. As shown in Figure 1, this method can be applied to the server side, and the method flow includes the following steps:

[0056] Step S101: Obtain the training data to be adjusted.

[0057] Optionally, the server obtains training data corresponding to the training model. This training data can be data of the question-and-answer task type, such as "How is the weather today? - Sunny" or "What is the weather like today? - Clear skies." As can be seen from the examples of question-and-answer task type data above, these training data vary in length. In this embodiment of the disclosure, it is necessary to adjust the length of these training data.

[0058] It should be noted that the current question-answering tasks should be of the same domain, which will help improve the generalization and robustness of the model during training.

[0059] Step S102: Sort the training data according to the first data length to obtain sorted data.

[0060] Optionally, the length of the obtained training data is referred to here as the first data length. The training data needs to be sorted according to the length of the first data length to obtain sorted data. It should be noted that the sorting can be done from longest to shortest or from shortest to longest.

[0061] Step S103: Split the sorted data to obtain ordered data buckets.

[0062] Optionally, in this embodiment of the disclosure, after obtaining the sorted data, it is necessary to split the sorted data and group certain sorted data together according to certain rules, storing them in the same data bucket. For example, sorted data with similar lengths can be stored in the same data bucket. When there are multiple data buckets, each data bucket will contain training data with similar lengths that are sorted by length. Since the training data in each data bucket is sorted, these data buckets can also be called ordered data buckets.

[0063] Step S104: Determine the grouped data according to the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data during model training.

[0064] Optionally, after obtaining the ordered data buckets, the server will sample data within the ordered data buckets and then group the sampled data to obtain grouped data. It is understood that grouping the sampled data is usually a further splitting process, which results in multiple micro-batch grouped data. These grouped data are the training data that will be adjusted in length and input into the model during model training.

[0065] In this embodiment, the training data to be adjusted is obtained; the training data is sorted according to its first data length to obtain sorted data; the sorted data is split to obtain ordered data buckets; and grouped data is determined based on the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data during model training. This length-grouping approach for adjusting the training data minimizes the additional computational cost associated with data padding, thereby improving overall computational efficiency and performance.

[0066] In some alternative implementations, the sorted data is split to obtain ordered data buckets, including:

[0067] Get the second data length of the sorted data;

[0068] Compare the length differences between the second data lengths;

[0069] Sorted data with length differences less than a threshold are grouped into the same bucket to obtain ordered data buckets.

[0070] Optionally, in this embodiment of the disclosure, after obtaining the sorted data, it is necessary to obtain the data length of each sorted data, which is referred to here as the second data length. Then, the length differences between these second data lengths are compared. A threshold is set, for example, 0.8, which represents the minimum value between any two second data lengths that are close in length. When the length difference is less than this threshold, the sorted data are considered to be close in length, and the server will divide these sorted data into the same bucket. Since the sorted data has already been sorted from shortest to longest, the sorted data obtained in each bucket are all close in length and sorted. These buckets can then be arranged and stored as ordered data buckets. The bucketing effect can be seen in Figure 2.

[0071] In this embodiment of the disclosure, sorted data with length differences less than a threshold are grouped into the same bucket, which reduces the length of the filler and improves training efficiency.

[0072] In some alternative implementations, grouping data is determined based on ordered data buckets, including:

[0073] Distribute the data within ordered data buckets to computing devices;

[0074] The data within the computing device is grouped to obtain grouped data.

[0075] Optionally, some or all of the data within an ordered data bucket may be collected into different computing devices, where there can be at least one computing device. For example, all data in ordered data bucket 1 may be distributed to computing device 0, computing device 1, and computing device n. Similarly, all data in ordered data bucket 2 may be distributed to computing device 0, computing device 1, and computing device n.

[0076] The data within each computing device is then grouped to obtain grouped data. This grouping essentially involves further micro-batch partitioning of the data within the computing devices, thereby further shortening the length of the training data input into the training model.

[0077] In some alternative implementations, data within the computing device is grouped to obtain grouped data, including:

[0078] Get the smallest unit value in the group;

[0079] The data within the computing device is grouped according to the smallest unit value of the group, resulting in grouped data.

[0080] Optionally, embodiments of this disclosure may define a minimum grouping unit value, such as 2. This minimum grouping unit value can be a batch processing value set during micro-batch processing. As shown in Figure 3, the data within the computing device is grouped using the minimum grouping unit value, resulting in multiple grouped data sets. In Figure 3, each computing device contains multiple grouped data sets.

[0081] In this embodiment of the disclosure, by grouping the data on each computing device according to the smallest unit value of the group, the amount of training data input to the training model is reduced, thereby improving the overall efficiency and reliability of model fine-tuning.

[0082] In some alternative implementations, after determining the grouped data based on the ordered data buckets, the method further includes:

[0083] Randomize the grouped data;

[0084] The randomly arranged data is used as sample data for iteratively updating the model parameters during the model training process.

[0085] Optionally, in this embodiment of the disclosure, after obtaining the grouped data from the micro-batch processing, these grouped data can be randomly arranged, and then the randomly arranged training data can be used as the sample data to be sampled each time the model or model parameters are iterated and updated during model training. As shown in Figure 4, the grouped data is shuffled, and each computing device stores unordered grouped data.

[0086] In this embodiment of the disclosure, the sample data for model training is made random by randomly shuffling the grouped data, so as to obtain a better training effect.

[0087] In some optional implementations, the training data is sorted according to a first data length to obtain sorted data, including:

[0088] The training data is concatenated according to a preset template format to obtain concatenated data;

[0089] Determine the third data length for the concatenated data;

[0090] The concatenated data is sorted according to the third data length to obtain sorted data.

[0091] Optionally, in this embodiment of the disclosure, a preset template format is set, such as "###Instruction:\n{instruction}\n\n###Input:\n{input}\n\n###Response:\n". The obtained training data is concatenated using the preset template format to obtain concatenated data, and then this concatenated data is used as the dataset for subsequent model training.

[0092] The server obtains the length of these concatenated data, which is referred to as the third data length. Then, the data is sorted according to the third data length to obtain data sorted from shortest to longest or longest to shortest.

[0093] As shown in Figure 5, Figure 5 is a schematic diagram of the overall process of the training data length adjustment method according to an embodiment of the present disclosure. The specific process is as follows:

[0094] Obtain the raw data;

[0095] Sort the original data by length;

[0096] Get the sorted dataset;

[0097] The sorted dataset is split into multiple ordered data buckets;

[0098] Data is collected from each ordered data bucket and distributed to different computing devices;

[0099] Group the data within each computing device into micro-batches;

[0100] The grouped data is shuffled to obtain sample data for input training the model.

[0101] This disclosure also provides a model training apparatus, which includes a dataset processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule.

[0102] The dataset processing submodule is used to execute the training data length adjustment method mentioned in the above embodiments, obtain grouped data, and send the grouped data to the encoding submodule.

[0103] The encoding sub-model is used to segment the grouped data, obtain segmentation information, label the segmentation information, obtain label information, convert the label information into digital identifiers, and send the digital identifiers to the model training process calculation sub-module.

[0104] The model training process calculation submodule is used to identify digit identifiers, use the text information corresponding to the digit identifiers as training data for model training, and adopt a distributed strategy to encapsulate the model during the model training process.

[0105] The checkpoint submodule is used to save key information updated by the computation submodule during model training.

[0106] Optionally, as shown in Figure 6, the model training device includes a dataset processing submodule 1, an encoding submodule 2, a model training process calculation submodule 3, and a checkpoint submodule 4. The dataset processing submodule 1 is connected to the encoding submodule 2, the encoding submodule 2 is connected to the model training process calculation submodule 3, and the model training process calculation submodule 3 is connected to the checkpoint module 4.

[0107] It should be noted that the dataset processing submodule 1 here is used to execute the training data length adjustment method disclosed in the above embodiments to obtain grouped data. The specific implementation process is the same as in the above embodiments and will not be repeated here. Then the obtained grouped data is sent to the encoding submodule 2.

[0108] The main function of the encoding submodule 2 is to segment the text in the grouped data, that is, to divide the input text into individual "tags" to obtain tag information, which can be words, characters or sub-words, etc.

[0109] The tagging information is then converted into a numeric identifier, such as a data ID, using an encoding method. For example, the numeric ID corresponding to "Galso" is 1101; the numeric ID corresponding to "GIt" is 1102; and the numeric ID corresponding to "_n" is 1107. Each identifier has a unique numeric ID.

[0110] In this embodiment of the disclosure, these numeric IDs are used as inputs to the training model. As can be seen from Figure 6, the numeric IDs are sent to the model training process calculation submodule 3.

[0111] The model training process computation submodule 3 primarily encapsulates the model, supporting the direct use of existing models from open-source libraries or custom optimized models. The capabilities provided by model training process computation submodule 3 mainly include:

[0112] Complete the local model initialization. For example, build the model based on the model configuration file.

[0113] The model is quantized to 8 bits. The weights and activation values ​​of the linear layers in the model are represented as 8-bit integers, which reduces the memory usage during model training while maintaining the model's performance to some extent.

[0114] In each layer of the Transformer architecture, a low-rank bypass matrix is ​​added. The pre-trained model parameters are fixed, and only the bypass model parameters are trained and updated.

[0115] Meanwhile, in the model encapsulation of the computation submodule 3 during the model training process, a distributed strategy is adopted to provide distributed training capabilities within the model encapsulation.

[0116] During the model training process, the calculation submodule performs model training, while the checkpoint submodule 4 saves and loads key information during the training process. For example, it periodically saves key information such as model parameters and weights so that it can be restored to the previous state if training is interrupted or problems occur.

[0117] In this embodiment of the disclosure, when the checkpoint submodule 4 saves the key information updated by the model training process calculation submodule 3 during model training, a distributed saving method can be adopted. Specifically, each computing device saves its own weights, there is no parameter transfer between computing devices, and each computing device only needs to process the data it contains, greatly speeding up the saving process. Distributed saving can also be done asynchronously. For example, after the weights are transferred to the CPU, training can continue, and the CPU process writes the data to the disk simultaneously.

[0118] In addition, the model training device also has the following functions:

[0119] Model training: Responsible for executing the model training process, including model computation and iterative updates of model parameters.

[0120] Optimizer Management: Manage the optimizer, such as setting the learning rate and updating strategies.

[0121] Training log recording: Records various metrics and information during the training process, such as loss value and accuracy.

[0122] Training control: Controls the start, stop, and pause of training.

[0123] In this embodiment, the dataset processing submodule, encoding submodule, model training process calculation submodule, and checkpoint submodule are organically combined to form a unique framework structure design. Because this framework structure provides easy-to-use distributed training capabilities, it can perform model pre-training and fine-tuning with a large number of parameters. At the same time, the asynchronous distributed checkpoint saving capability effectively shortens the training time, improves the hardware utilization during the training process, and improves the overall efficiency and reliability of model fine-tuning.

[0124] This embodiment also provides a training data length adjustment device for implementing the embodiments described above; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0125] This embodiment provides a device for adjusting the length of training data, as shown in Figure 7, including:

[0126] The acquisition module 701 is used to acquire the training data to be adjusted;

[0127] The first sorting module 702 is used to sort the training data according to the first data length of the training data to obtain sorted data;

[0128] The splitting module 703 is used to split the sorted data to obtain ordered data buckets;

[0129] The module 704 determines the grouped data based on the ordered data buckets, where the grouped data is the data after the length of the training data has been adjusted during model training.

[0130] In this embodiment of the disclosure, the length of the training data is adjusted by length grouping, which can minimize the additional computing power consumption caused by data padding and improve the overall computing efficiency and performance.

[0131] In some alternative implementations, the splitting module 703 includes:

[0132] The `get` submodule is used to obtain the second data length of the sorted data.

[0133] The comparison submodule is used to compare the length difference between the second data lengths;

[0134] The partitioning submodule is used to divide sorted data with a length difference less than a threshold into the same bucket, thus obtaining ordered data buckets.

[0135] In some alternative implementations, the determining module 704 includes:

[0136] The distribution submodule is used to distribute data within ordered data buckets to computing devices;

[0137] The grouping submodule is used to group data within the computing device to obtain grouped data.

[0138] In some alternative implementations, the grouping submodule includes:

[0139] Get unit, used to retrieve the smallest unit value in a group;

[0140] A grouping unit is used to group data within a computing device based on the smallest grouping unit value, resulting in grouped data.

[0141] In some alternative embodiments, the device further includes:

[0142] The second sorting module is used to randomly arrange the grouped data after determining the grouped data based on the ordered data buckets;

[0143] The configuration module is used to use randomly arranged data as sample data for iteratively updating model parameters during model training.

[0144] In some alternative implementations, the first sorting module 702 includes:

[0145] The splicing submodule is used to splice the training data according to a preset template format to obtain spliced ​​data;

[0146] The determination submodule is used to determine the third data length of the concatenated data;

[0147] The sorting submodule is used to sort the concatenated data according to the length of the third data to obtain sorted data.

[0148] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0149] In this embodiment, the model training device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0150] This disclosure also provides a computer device having the model training apparatus shown in FIG7 above.

[0151] Please refer to Figure 8, which is a schematic diagram of the structure of a computer device provided in an optional embodiment of this disclosure. As shown in Figure 8, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 uses one processor 10 as an example.

[0152] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0153] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0154] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0155] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0156] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0157] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0158] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0159] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for adjusting the length of training data, comprising: Obtain the training data to be adjusted; The training data is sorted according to the first data length to obtain sorted data; The sorted data is split to obtain ordered data buckets; Grouped data is determined based on the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data during model training.

2. The method according to claim 1, wherein, The step of splitting the sorted data to obtain ordered data buckets includes: Obtain the second data length of the sorted data; Compare the length differences between the second data lengths; The sorted data with length differences less than a threshold are divided into the same bucket to obtain the ordered data bucket.

3. The method according to claim 1 or 2, wherein, The step of determining grouped data based on the ordered data buckets includes: Distribute the data within the ordered data buckets to computing devices; The data within the computing device is grouped to obtain the grouped data.

4. The method according to claim 3, wherein, The step of grouping the data within the computing device to obtain the grouped data includes: Get the smallest unit value in the group; The data within the computing device is grouped according to the minimum unit value of the grouping to obtain the grouped data.

5. The method according to claim 1 or 4, wherein, After determining the grouped data based on the ordered data buckets, the method further includes: The grouped data is randomly arranged; The randomly arranged data is used as sample data for iteratively updating the model parameters during the model training process.

6. The method according to any one of claims 1-5, wherein, The step of sorting the training data according to a first data length to obtain sorted data includes: The training data is concatenated according to a preset template format to obtain concatenated data; Determine the third data length of the spliced ​​data; The concatenated data is sorted according to the third data length to obtain the sorted data.

7. A model training apparatus, comprising a dataset processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule; wherein, The dataset processing submodule is configured to execute the training data length adjustment method as described in claim 1, obtain grouped data, and send the grouped data to the encoding submodule; The encoding sub-model is configured to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation sub-module. The model training process calculation submodule is configured to identify the digital identifier, use the text information corresponding to the digital identifier as training data for model training, and the model training process calculation submodule uses a distributed strategy to encapsulate the model during the model training process. The checkpoint submodule is configured to save key information updated by the model training process calculation submodule during model training.

8. A device for adjusting the length of training data, comprising: The acquisition module is configured to acquire the training data to be adjusted. The first sorting module is configured to sort the training data according to the first data length of the training data to obtain sorted data; The splitting module is configured to split the sorted data to obtain ordered data buckets; The determination module is configured to determine grouped data based on the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data during model training.

9. A computer device, comprising: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the training data length adjustment method according to any one of claims 1 to 6.

10. A computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method for adjusting the length of training data as described in any one of claims 1 to 6.

11. A computer program product comprising computer instructions, wherein, The computer instructions are used to cause the computer to perform the method for adjusting the length of training data as described in any one of claims 1 to 6.

Images