Training data length adjusting method and device, equipment, medium and program product

Abstract

The invention relates to the field of artificial intelligence, in particular to a training data length adjusting method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring training data to be adjusted; sorting the training data according to the first data length of the training data to obtain sorted data; splitting the sorted data to obtain ordered data buckets; grouping data are determined according to the ordered data buckets, and the grouping data are data obtained after length adjustment is completed on the training data when model training is executed. According to the method, the length of the training data is adjusted by a length grouping processing means, so that extra computing power consumption caused by data filling processing can be avoided as much as possible, and the overall computing efficiency and performance are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a method, device, equipment, medium and program product for adjusting the length of training data. Background Art

[0002] With the rapid development of artificial intelligence, large language models have achieved remarkable results in natural language processing tasks such as dialogue question answering, text generation, and text translation. At present, the training of large models usually requires a lot of computing resources and time. In addition, in actual application scenarios, since different tasks have their own unique requirements and different scenarios also have diverse needs, it is often necessary to carry out fine-tuning operations on the pre-trained large models so that they can better adapt to specific tasks and scenario requirements.

[0003] Current large model training and fine-tuning methods usually use domain-specific datasets, such as question-answering task-type datasets. The data in these datasets are usually of inconsistent lengths, so padding is required to make the data the same length, but this operation will affect the actual training efficiency.

[0004] Therefore, the related technology will cause low training efficiency due to byte padding of the data set during large model training and fine-tuning. Summary of the invention

[0005] In view of this, the present disclosure provides a method, apparatus, device, medium and program product for adjusting the length of training data to solve the problem of low training efficiency due to byte padding of the data set in the related art during fine-tuning of large model training.

[0006] In a first aspect, the present disclosure provides a method for adjusting the length of training data, the method comprising:

[0007] Obtaining training data to be adjusted;

[0008] Sorting the training data according to the first data length of the training data to obtain sorted data;

[0009] Split the sorted data to get ordered data buckets;

[0010] The grouped data is determined according to the ordered data buckets, wherein the grouped data is the data after the length of the training data is adjusted when executing the model training.

[0011] In the disclosed embodiment, the training data to be adjusted is obtained; the training data is sorted according to the first data length of the training data to obtain sorted data; the sorted data is split to obtain ordered data buckets; and grouped data is determined according to the ordered data buckets, wherein the grouped data is the data after the length adjustment of the training data when executing the model training. In this way, the length of the training data is adjusted by the length grouping processing method, which can avoid the extra computing power consumption caused by the data padding processing as much as possible, and improve the overall computing efficiency and performance.

[0012] In an optional implementation, the sorted data is split to obtain ordered data buckets, including:

[0013] Obtain the second data length of the sorted data;

[0014] comparing the length difference between the second data lengths;

[0015] The sorted data with length differences less than the threshold are divided into the same bucket to obtain ordered data buckets.

[0016] In the disclosed embodiment, by dividing the sorted data whose length difference between the data lengths is less than a threshold into the same bucket, the padding length is reduced and the training efficiency is improved.

[0017] In an optional implementation, determining the grouped data according to the ordered data buckets includes:

[0018] Distribute the data in the ordered buckets to computing devices;

[0019] The data in the computing device is grouped to obtain grouped data.

[0020] In an optional implementation, grouping data in a computing device to obtain grouped data includes:

[0021] Get the minimum unit value of the group;

[0022] The data in the computing device is grouped according to the minimum unit value of the grouping to obtain grouped data.

[0023] In the embodiment of the present disclosure, by grouping the data on each computing device according to the minimum unit value of the grouping, the amount of training data input into the training model is reduced, thereby improving the overall efficiency and reliability of model fine-tuning.

[0024] In an optional implementation, after determining the grouped data according to the ordered data buckets, the method further includes:

[0025] Randomly arrange the grouped data;

[0026] The randomly arranged data is used as sample data for iteratively updating model parameters during model training.

[0027] In the disclosed embodiment, the grouped data is randomly shuffled so that the sample data for model training has randomness, thereby obtaining a better training effect.

[0028] In an optional implementation, sorting the training data according to the first data length of the training data to obtain sorted data includes:

[0029] The training data is spliced ​​according to a preset template format to obtain spliced ​​data;

[0030] Determine a third data length of the spliced ​​data;

[0031] The concatenated data is sorted according to the third data length to obtain sorted data.

[0032] In a second aspect, the present disclosure provides a model training device, which includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule;

[0033] A data set processing submodule, used to execute the method for adjusting the length of training data of the first aspect, obtain grouped data, and send the grouped data to the encoding submodule;

[0034] The encoding sub-model is used to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation sub-module;

[0035] The model training process calculation submodule is used to identify the digital identifier and use the text information corresponding to the digital identifier as training data for model training. The model training process calculation submodule adopts a distributed strategy to encapsulate the model during the model training process.

[0036] The checkpoint submodule is used to save the key information updated by the calculation submodule during the model training process.

[0037] In the disclosed embodiment, the data set processing submodule, the encoding submodule, the model training process calculation submodule, and the checkpoint submodule are organically combined to form a unique framework structure design. Since the framework structure provides simple and easy-to-use distributed training capabilities, it can perform model pre-training and fine-tuning of a large number of parameters. At the same time, the asynchronous distributed checkpoint saving capability effectively shortens the training time and improves the hardware utilization during the training process.

[0038] In a third aspect, the present disclosure provides a device for adjusting the length of training data, the device comprising:

[0039] An acquisition module, used for acquiring training data to be adjusted;

[0040] A first sorting module, used for sorting the training data according to a first data length of the training data to obtain sorted data;

[0041] The splitting module is used to split the sorted data to obtain ordered data buckets;

[0042] A determination module determines grouped data according to the ordered data buckets, wherein the grouped data is data after the length of the training data is adjusted when executing the model training.

[0043] In the disclosed embodiment, the length of the training data is adjusted by length grouping processing, which can minimize the extra computing power consumption caused by data padding processing, improve the overall computing efficiency and performance, and improve the overall efficiency and reliability of model fine-tuning.

[0044] In a fourth aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the method for adjusting the length of training data of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0045] In a fifth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for adjusting the length of training data of the first aspect or any corresponding embodiment thereof.

[0046] In a sixth aspect, the present disclosure provides a computer program product, including computer instructions, which are used to enable a computer to execute the method for adjusting the length of training data of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 is a flowchart of a method for adjusting the length of training data according to an embodiment of the present disclosure;

[0049] Figure 2 is a schematic diagram of obtaining ordered data buckets according to an embodiment of the present disclosure;

[0050] Figure 3 is a schematic diagram of grouping data within a computing device according to an embodiment of the present disclosure;

[0051] Figure 4 is a schematic diagram of arranging packet data in a random order according to an embodiment of the present disclosure;

[0052] Figure 5 is a schematic diagram of the overall flow of the method for adjusting the length of training data according to an embodiment of the present disclosure;

[0053] Figure 6 is an overall block diagram of a model training device according to an embodiment of the present disclosure;

[0054] Figure 7 is a structural block diagram of a device for adjusting the length of training data according to an embodiment of the present disclosure;

[0055] Figure 8 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0057] In some large-model application scenarios for dialogue question-answering tasks, due to diverse needs, it is necessary to perform detailed fine-tuning operations on the pre-trained large model so that it can better adapt to specific tasks and scenario requirements.

[0058] Currently, in the fine-tuning method of large model training data, there is usually a situation where the data length is inconsistent. At this time, the data needs to be supplemented to the same length by padding. However, due to the huge volume of training data in the training data set, the efficiency of actual training will be seriously affected during the padding process. In order to solve the above problems, according to an embodiment of the present disclosure, a method embodiment for adjusting the length of training data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0059] In this embodiment, a method for adjusting the length of training data is provided. Figure 1is a flow chart of a method for adjusting the length of training data according to an embodiment of the present disclosure, such as Figure 1 As shown, the method can be applied to the server side, and the method flow includes the following steps:

[0060] Step S101, obtaining training data to be adjusted.

[0061] Optionally, the server obtains training data corresponding to the training model, and the training data may be data of a question-answering task type, such as "how is the weather today - sunny", "what is the weather like today - clear sky", etc. As can be seen from the above examples of data of question-answering task types, these training data are of different lengths. In the embodiments of the present disclosure, the length of these training data needs to be adjusted.

[0062] It should be noted that the current question-answering task type should be in the same field, which is conducive to improving the generalization and robustness of the model when training the model.

[0063] Step S102: sorting the training data according to the first data length of the training data to obtain sorted data.

[0064] Optionally, the data length of the training data is obtained, which is referred to as the first data length. The training data needs to be sorted according to the length of the first data length to obtain sorted data. It should be noted that the sorting here can be sorted from long to short, or from short to long.

[0065] Step S103, split the sorted data to obtain ordered data buckets.

[0066] Optionally, in the disclosed embodiment, after obtaining the above-mentioned sorted data, it is necessary to split the sorted data, combine some sorted data together according to certain rules, and store them in the same data bucket, for example, store sorted data with similar lengths in the same data bucket. In the case of multiple data buckets, each data bucket will contain training data with similar lengths and sorted by length. Since the training data in each data bucket is sorted, these data buckets can also be called ordered data buckets.

[0067] Step S104, determining grouped data according to the ordered data buckets, wherein the grouped data is data after the length of the training data is adjusted when executing the model training.

[0068] Optionally, after obtaining the ordered data buckets, the server samples the data in the ordered data buckets, and then groups the sampled data to obtain grouped data. It can be understood that when grouping the sampled data, it is usually a further splitting process, so that multiple micro-batches of grouped data can be obtained. These grouped data are the training data that are finally adjusted in length and need to be input into the model when executing model training.

[0069] In the disclosed embodiment, the training data to be adjusted is obtained; the training data is sorted according to the first data length of the training data to obtain sorted data; the sorted data is split to obtain ordered data buckets; and grouped data is determined according to the ordered data buckets, wherein the grouped data is the data after the length adjustment of the training data when executing the model training. In this way, the length of the training data is adjusted by the length grouping processing method, which can avoid the extra computing power consumption caused by the data padding processing as much as possible, and improve the overall computing efficiency and performance.

[0070] In some optional implementations, the sorted data is split to obtain ordered data buckets, including:

[0071] Obtain the second data length of the sorted data;

[0072] comparing the length difference between the second data lengths;

[0073] The sorted data with length differences less than the threshold are divided into the same bucket to obtain ordered data buckets.

[0074] Optionally, in the disclosed embodiment, after obtaining the sorted data, it is necessary to obtain the data length of each sorted data, where the data length of the sorted data is referred to as the second data length. Then compare the length differences between these second data lengths. At the same time, set a threshold, such as 0.8, which is used to represent the minimum value of the similar lengths between any two second data lengths. When the length difference is less than the threshold, the sorted data are considered to be similar in length. At this time, the server will divide the sorted data into the same bucket. Since the sorted data has been sorted from short to long before, the sorted data obtained in each bucket are all of similar length and well-sorted data. At this time, these buckets can be arranged and stored as ordered data buckets. The bucketing effect can be seen in Figure 2 .

[0075] In the disclosed embodiment, by dividing the sorted data whose length difference between the data lengths is less than a threshold into the same bucket, the padding length is reduced and the training efficiency is improved.

[0076] In some optional implementations, determining the grouped data according to the ordered data buckets includes:

[0077] Distribute the data in the ordered buckets to computing devices;

[0078] The data in the computing device is grouped to obtain grouped data.

[0079] Optionally, part or all of the data in the ordered data bucket will be collected into different computing devices, and the number of computing devices here can be at least one. For example, all the data in the ordered data bucket 1 will be distributed to computing device 0, computing device 1, and computing device n. Similarly, all the data in the ordered data bucket 2 will be distributed to computing device 0, computing device 1, and computing device n.

[0080] Then the data in each computing device is grouped to obtain grouped data. It can be understood that the grouping here is actually to further divide the data in the computing device into micro batches, so that the length of the training data input into the training model can be further shortened.

[0081] In some optional implementations, grouping data in a computing device to obtain grouped data includes:

[0082] Get the minimum unit value of the group;

[0083] The data in the computing device is grouped according to the minimum unit value of the grouping to obtain grouped data.

[0084] Optionally, the embodiment of the present disclosure may define a minimum unit value for grouping, such as a minimum unit value for grouping of 2. The minimum unit value for grouping may be a batch processing value set during micro-batch processing, such as Figure 3 As shown, the data in the computing device is grouped using the minimum unit value of the grouping, and then a plurality of grouped data are obtained. Figure 3 In the example, each computing device contains a plurality of grouped data.

[0085] In the embodiment of the present disclosure, by grouping the data on each computing device according to the minimum unit value of the grouping, the amount of training data input into the training model is reduced, thereby improving the overall efficiency and reliability of model fine-tuning.

[0086] In some optional implementations, after determining the grouped data according to the ordered data buckets, the method further includes:

[0087] Randomly arrange the grouped data;

[0088] The randomly arranged data is used as sample data for iteratively updating model parameters during model training.

[0089] Optionally, in the disclosed embodiment, after obtaining the grouped data for micro-batch processing, the grouped data may be randomly arranged, and then the randomly arranged training data is used as the sample data to be sampled each time the model or model parameter is iteratively updated during the model training process. Figure 4 As shown, the grouped data are shuffled, and disordered grouped data are stored in each computing device.

[0090] In the disclosed embodiment, the grouped data is randomly shuffled so that the sample data for model training has randomness, thereby obtaining a better training effect.

[0091] In some optional implementations, sorting the training data according to the first data length of the training data to obtain sorted data includes:

[0092] The training data is spliced ​​according to a preset template format to obtain spliced ​​data;

[0093] Determine a third data length of the spliced ​​data;

[0094] The concatenated data is sorted according to the third data length to obtain sorted data.

[0095] Optionally, in the disclosed embodiment, a preset template format is set, such as "###Instruction:\n{instruction}\n\n###Input:\n{input}\n\n###Response:\n", and the acquired training data is spliced ​​using the preset template format to obtain spliced ​​data, which is then used as a data set for subsequent model training.

[0096] The server obtains the data length of the spliced ​​data, which is referred to as the third data length here, and then sorts the data according to the third data length to obtain sorted data from short to long or from long to short.

[0097] like Figure 5 As shown, Figure 5 : is a schematic diagram of the overall process of the method for adjusting the length of training data according to an embodiment of the present disclosure, and the specific process is as follows:

[0098] Get the original data;

[0099] Sort the original data by length;

[0100] Get the sorted data set;

[0101] Split the sorted data set to obtain multiple ordered data buckets;

[0102] Collect data from each ordered data bucket onto different computing devices;

[0103] Group the data into micro-batches within each computing device;

[0104] The grouped data is shuffled to obtain sample data for input into the training model.

[0105] In the embodiment of the present disclosure, a model training device is also provided, which includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule;

[0106] The data set processing submodule is used to execute the method for adjusting the length of training data mentioned in the above embodiments, obtain grouped data, and send the grouped data to the encoding submodule;

[0107] The encoding sub-model is used to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation sub-module;

[0108] The model training process calculation submodule is used to identify the digital identifier and use the text information corresponding to the digital identifier as training data for model training. The model training process calculation submodule adopts a distributed strategy to encapsulate the model during the model training process.

[0109] The checkpoint submodule is used to save the key information updated by the calculation submodule during the model training process.

[0110] Alternatively, if Figure 6 As shown, the model training device includes a data set processing submodule 1, an encoding submodule 2, a model training process calculation submodule 3, and a checkpoint submodule 4. The data set processing submodule 1 is connected to the encoding submodule 2, the encoding submodule 2 is connected to the model training process calculation submodule 3, and the model training process calculation submodule 3 is connected to the checkpoint submodule 4.

[0111] It should be noted that the data set processing submodule 1 here is used to execute the training data length adjustment method disclosed in the above embodiments to obtain grouped data. The specific implementation process refers to the above embodiments and will not be repeated. The obtained grouped data is then sent to the encoding submodule 2.

[0112] The main function of the encoding submodule 2 is to segment the text in the grouped data, that is, to divide the input text into independent "tags" to obtain tag information, which can be words, characters or subwords.

[0113] The tag information is then converted into a digital identifier, such as a data ID, through an encoding method. For example, the digital ID corresponding to "Galso" is: 1101; the digital ID corresponding to "GIt" is: 1102; the digital ID corresponding to "_n" is: 1107. Each identification information has a unique digital ID corresponding to it.

[0114] In the disclosed embodiment, these digital IDs are used as inputs for training the model. Figure 6 It can be concluded that the digital ID is sent to the model training process calculation submodule 3.

[0115] The model training process calculation submodule 3 mainly completes the encapsulation of the model, and supports the direct use of existing models on the open source library or customized optimized models. The capabilities provided by the model training process calculation submodule 3 mainly include:

[0116] Complete local model initialization. For example, build a model based on the model configuration file.

[0117] The model is 8-bit quantized. The linear layer weights and activation values ​​in the model are represented as 8-bit integers, which reduces the memory usage during model training while maintaining the model performance to a certain extent.

[0118] A bypass low-rank matrix is ​​added to each layer of the Transformer architecture, the pre-trained model parameters are fixed, and only the bypass model parameters are trained and updated.

[0119] At the same time, in the model encapsulation of the model training process calculation submodule 3, a distributed strategy is adopted so that the model encapsulation provides it with distributed training capabilities.

[0120] During the model training process, the calculation submodule performs model training, and the checkpoint submodule 4 saves and loads key information in the training process. For example, key information such as model parameters and weights are saved regularly so that the model can be restored to the previous state when training is interrupted or problems occur.

[0121] In the disclosed embodiment, when the checkpoint submodule 4 is used to save the key information updated by the model training process calculation submodule 3 during the model training process, a distributed saving method can be used, specifically: each computing device saves the weights it holds, there is no parameter transfer between computing devices, and a single computing device only needs to process the part of the data it contains, which greatly speeds up the saving speed. Distributed saving can also be done asynchronously, for example, training can continue after the weights are transferred to the CPU, and the CPU process writes the data to the disk.

[0122] In addition, the model training device also has the following functions:

[0123] Model training: responsible for executing the model training process, including model calculation and iterative updating of model parameters.

[0124] Optimizer management: Manage optimizers, such as setting learning rate, update strategy, etc.

[0125] Training log recording: records various indicators and information during the training process, such as loss value, accuracy, etc.

[0126] Training control: control the start, stop, pause and other operations of training.

[0127] In the disclosed embodiment, the data set processing submodule, the encoding submodule, the model training process calculation submodule, and the checkpoint submodule are organically combined to form a unique framework structure design. Since the framework structure provides simple and easy-to-use distributed training capabilities, model pre-training and fine-tuning of a large number of parameters can be performed. At the same time, the asynchronous distributed checkpoint saving capability effectively shortens the training time, improves the hardware utilization rate during the training process, and improves the overall efficiency and reliability of model fine-tuning.

[0128] In the present embodiment, a device for adjusting the length of training data is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made are not repeated here. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0129] This embodiment provides a device for adjusting the length of training data. Figure 7 As shown, including:

[0130] An acquisition module 701 is used to acquire training data to be adjusted;

[0131] A first sorting module 702, used to sort the training data according to a first data length of the training data to obtain sorted data;

[0132] A splitting module 703 is used to split the sorted data to obtain ordered data buckets;

[0133] The determination module 704 determines the grouped data according to the ordered data buckets, wherein the grouped data is the data after the length of the training data is adjusted when executing the model training.

[0134] In the disclosed embodiment, the length of the training data is adjusted by length grouping processing, which can avoid the extra computing power consumption caused by data padding processing as much as possible and improve the overall computing efficiency and performance.

[0135] In some optional implementations, the splitting module 703 includes:

[0136] An acquisition submodule, used for acquiring a second data length of the sorted data;

[0137] A comparison submodule, used for comparing the length difference between the second data lengths;

[0138] The partitioning submodule is used to partition the sorted data whose length difference is less than a threshold into the same bucket to obtain ordered data buckets.

[0139] In some optional implementations, the determination module 704 includes:

[0140] A distribution submodule, used to distribute the data in the ordered data buckets to the computing devices;

[0141] The grouping submodule is used to group the data in the computing device to obtain grouped data.

[0142] In some optional implementations, the grouping submodule includes:

[0143] Get unit, used to get the minimum unit value of the group;

[0144] The grouping unit is used to group the data in the computing device according to the grouping minimum unit value to obtain grouped data.

[0145] In some optional embodiments, the device further comprises:

[0146] A second sorting module is used to randomly arrange the grouped data after determining the grouped data according to the ordered data buckets;

[0147] The setting module is used to use the randomly arranged data as sample data for iteratively updating model parameters during model training.

[0148] In some optional implementations, the first sorting module 702 includes:

[0149] A splicing submodule is used to splice the training data according to a preset template format to obtain spliced ​​data;

[0150] A determination submodule, used to determine a third data length of the spliced ​​data;

[0151] The sorting submodule is used to sort the concatenated data according to the third data length to obtain sorted data.

[0152] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0153] The model training device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0154] The present disclosure also provides a computer device having the above Figure 7 The model training apparatus shown.

[0155] See also Figure 8 , Figure 8 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0156] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0157] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0158] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0160] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0161] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0162] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0163] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for adjusting the length of training data, characterized in that: The method comprises: Obtaining training data to be adjusted; Sorting the training data according to the first data length of the training data to obtain sorted data; Splitting the sorted data to obtain ordered data buckets; The grouped data is determined according to the ordered data buckets, wherein the grouped data is data obtained after length adjustment of the training data when executing model training.

2. The method according to claim 1, characterized in that The step of splitting the sorted data to obtain ordered data buckets includes: Obtaining a second data length of the sorted data; comparing the length difference between the second data lengths; The sorted data whose length difference is less than a threshold value are divided into the same bucket to obtain the ordered data bucket.

3. The method according to claim 1, characterized in that The determining of the grouped data according to the ordered data buckets comprises: Distributing the data in the ordered data buckets to computing devices; The data in the computing device is grouped to obtain the grouped data.

4. The method according to claim 3, characterized in that The step of grouping the data in the computing device to obtain the grouped data includes: Get the minimum unit value of the group; The data in the computing device is grouped according to the grouping minimum unit value to obtain the grouped data.

5. The method according to claim 1 or 4, characterized in that: After determining the grouped data according to the ordered data buckets, the method further includes: Randomly arranging the grouped data; The randomly arranged data is used as sample data for iteratively updating model parameters during model training.

6. The method according to claim 1, characterized in that The step of sorting the training data according to the first data length of the training data to obtain sorted data includes: splicing the training data according to a preset template format to obtain spliced ​​data; Determining a third data length of the spliced ​​data; The concatenated data is sorted according to the third data length to obtain the sorted data.

7. A model training device, characterized in that: The model training device includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule; The data set processing submodule is used to execute the method for adjusting the length of training data according to claim 1 to obtain grouped data, and send the grouped data to the encoding submodule; The encoding sub-model is used to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation sub-module; The model training process calculation submodule is used to identify the digital identifier, use the text information corresponding to the digital identifier as training data for model training, and the model training process calculation submodule adopts a distributed strategy to encapsulate the model during the model training process; The checkpoint submodule is used to save the key information updated by the model training process calculation submodule during the model training process.

8. A device for adjusting the length of training data, characterized in that: The device comprises: An acquisition module, used for acquiring training data to be adjusted; A first sorting module, used for sorting the training data according to a first data length of the training data to obtain sorted data; A splitting module, used for splitting the sorted data to obtain ordered data buckets; A determination module determines grouped data according to the ordered data buckets, wherein the grouped data is data after the length of the training data is adjusted when executing model training.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for adjusting the length of training data according to any one of claims 1 to 6 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for adjusting the length of training data according to any one of claims 1 to 6.

11. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for adjusting the length of training data according to any one of claims 1 to 6.

Images