Method, device, equipment, medium and program product for training data length adjustment

Abstract

The present disclosure relates to the field of artificial intelligence, and particularly relates to a method and device for adjusting the length of training data, equipment, medium and program product. The method comprises: obtaining training data to be adjusted; sorting the training data according to the first data length of the training data to obtain sorted data; splitting the sorted data to obtain ordered data buckets; and determining grouped data according to the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data during model training. The present disclosure adjusts the length of the training data by length grouping processing means, which can avoid the additional power consumption caused by the padding processing of the data as much as possible, and improve the overall computing efficiency and performance.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a method and device for adjusting the length of training data, equipment, medium and program product. BACKGROUND

[0002] With the rapid development of artificial intelligence, large language models have made significant achievements in natural language processing tasks such as dialogue question answering, text generation, and text translation. Currently, the training of large models usually requires a large amount of computing resources and time, and in actual application scenarios, due to the unique requirements of different tasks and the diverse needs of different scenarios, fine-tuning operations are often needed for pre-trained large models to better adapt to specific task and scenario requirements.

[0003] Current large model training fine-tuning methods often use specific domain datasets, such as question and answer task type datasets. The data in these datasets is usually inconsistent in length, so padding is needed to supplement the data to the same length, but this operation will affect the actual training efficiency.

[0004] Therefore, in the related art, the training efficiency is low due to byte padding of the dataset in large model training fine-tuning. SUMMARY

[0005] Therefore, the present disclosure provides a method and device for adjusting the length of training data to solve the problem of low training efficiency due to byte padding of the dataset in large model training fine-tuning in the related art.

[0006] In a first aspect, the present disclosure provides a method for adjusting the length of training data, comprising:

[0007] obtaining training data to be adjusted;

[0008] sorting the training data according to the first data length of the training data to obtain sorted data;

[0009] splitting the sorted data to obtain ordered data buckets;

[0010] determining grouped data from the ordered data buckets, wherein the grouped data is the data after length adjustment of the training data for model training.

[0011] In the embodiments of the present disclosure, the training data to be adjusted is obtained; the training data is sorted according to a first data length of the training data to obtain sorted data; the sorted data is split to obtain ordered data buckets; and grouped data is determined according to the ordered data buckets, wherein the grouped data is data after length adjustment of the training data during model training. In this way, the length adjustment of the training data by length grouping processing means can avoid additional computing power consumption caused by data padding processing as much as possible, and improve overall computing efficiency and performance.

[0012] In an optional implementation, splitting the sorted data to obtain the ordered data buckets comprises:

[0013] Obtaining a second data length of the sorted data;

[0014] Comparing length differences between the second data lengths;

[0015] Dividing the sorted data with a length difference less than a threshold to the same bucket to obtain the ordered data buckets.

[0016] In the embodiments of the present disclosure, by dividing the sorted data with a length difference less than a threshold into the same bucket, the length of padding is reduced, and the training efficiency is improved.

[0017] In an optional implementation, determining the grouped data according to the ordered data buckets comprises:

[0018] Distributing data in the ordered data buckets to computing devices;

[0019] Grouping the data in the computing devices to obtain the grouped data.

[0020] In an optional implementation, grouping the data in the computing devices to obtain the grouped data comprises:

[0021] Obtaining a grouping minimum unit value;

[0022] Grouping the data in the computing devices according to the grouping minimum unit value to obtain the grouped data.

[0023] In the embodiments of the present disclosure, by grouping the data on each computing device according to the grouping minimum unit value, the number of training data input into the training model is reduced, and the overall efficiency and reliability of model fine-tuning are improved.

[0024] In an optional implementation, after determining the grouped data according to the ordered data buckets, the method further comprises:

[0025] Randomly arranging the grouped data;

[0026] The randomly arranged data is used as sample data for iterative updating of model parameters in a model training process.

[0027] In the embodiments of the present disclosure, the grouping data is randomly shuffled to make the sample data for model training random, so as to obtain a better training effect.

[0028] In an optional implementation, the training data is sorted according to a first data length of the training data to obtain sorted data, including:

[0029] The training data is spliced according to a preset template format to obtain spliced data.

[0030] A third data length of the spliced data is determined.

[0031] The spliced data is sorted according to the third data length to obtain the sorted data.

[0032] In a second aspect, the present disclosure provides a model training device, which includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule.

[0033] The data set processing submodule is configured to execute the method for adjusting the training data length in the first aspect to obtain the grouping data and send the grouping data to the encoding submodule.

[0034] The encoding submodule is configured to divide the grouping data to obtain division information, mark the division information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation submodule.

[0035] The model training process calculation submodule is configured to identify the digital identifier, use the text information corresponding to the digital identifier as the training data for model training, and encapsulate the model using a distributed strategy in the model training process.

[0036] The checkpoint submodule is configured to save the key information updated by the model training process calculation submodule in the model training process.

[0037] In the embodiments of the present disclosure, the data set processing submodule, the encoding submodule, the model training process calculation submodule, and the checkpoint submodule are organically combined to form a unique framework structure design. The framework structure provides simple and easy-to-use distributed training capabilities, can perform model pre-training and fine-tuning of large parameter quantities, and has asynchronous distributed checkpoint saving capabilities, which effectively shortens the training time and improves the hardware utilization rate in the training process.

[0038] In a third aspect, the present disclosure provides a device for adjusting the length of training data, which includes:

[0039] an acquisition module configured to acquire training data to be adjusted;

[0040] a first sorting module configured to sort the training data according to a first data length of the training data to obtain sorted data;

[0041] a splitting module configured to split the sorted data to obtain ordered data buckets;

[0042] a determination module configured to determine grouped data according to the ordered data buckets, wherein the grouped data is data after length adjustment of the training data during model training.

[0043] In the embodiments of the present disclosure, the length adjustment of the training data by the length grouping processing means can avoid the additional computing power consumption caused by the padding processing as much as possible, improve the overall computing efficiency and performance, and improve the overall efficiency and reliability of the model fine-tuning.

[0044] In a fourth aspect, the present disclosure provides a computer device, comprising a memory and a processor, the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the training data length adjustment method of the first aspect or any of the corresponding embodiments thereof.

[0045] In a fifth aspect, the present disclosure provides a computer readable storage medium, the computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the training data length adjustment method of the first aspect or any of the corresponding embodiments thereof.

[0046] In a sixth aspect, the present disclosure provides a computer program product, comprising computer instructions, and the computer instructions are used to make a computer execute the training data length adjustment method of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0048] Figure 1 is a flowchart of the training data length adjustment method according to the embodiments of the present disclosure;

[0049] Figure 2 is a schematic diagram of obtaining ordered data buckets according to the embodiments of the present disclosure;

[0050] Figure 3 is a schematic diagram of grouping data in a computing device according to an embodiment of the present disclosure;

[0051] Figure 4 is a schematic diagram of reordering grouped data according to an embodiment of the present disclosure;

[0052] Figure 5 is a schematic diagram of the overall flow of the method for adjusting the length of training data according to an embodiment of the present disclosure;

[0053] Figure 6 is a schematic diagram of the overall block diagram of the model training device according to an embodiment of the present disclosure;

[0054] Figure 7 is a schematic diagram of the structure of the device for adjusting the length of training data according to an embodiment of the present disclosure;

[0055] Figure 8 is a schematic diagram of the hardware structure of the computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] To make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0057] In some large model application scenarios of dialogue question and answer tasks, due to diversified needs, fine tuning operations need to be carried out on pre-trained large models, so that they can better adapt to specific task and scene requirements.

[0058] Currently, in the fine tuning method of large model training data, there is usually a case of inconsistent data length, which needs to be supplemented to the same length by padding. However, due to the large amount of training data in the training data set, the actual training efficiency will be seriously affected in the padding process. In order to solve the above problems, according to the embodiments of the present disclosure, a method for adjusting the length of training data is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0059] In the present embodiment, a method for adjusting the length of training data is provided, Figure 1is a flowchart of a method for training data length adjustment according to an embodiment of the present disclosure, as shown in the figure, the method can be applied to the server side, and the method flow includes the following steps: Figure 1

[0060] In step S101, training data to be adjusted is obtained.

[0061] Optionally, the server obtains training data corresponding to the training model, which can be data of a question and answer task type, such as "today's weather is sunny", "today's weather is clear" and the like. As can be seen from the above examples of data of the question and answer task type, the lengths of these training data are different, and in the present embodiment, the lengths of these training data need to be adjusted.

[0062] It should be noted that the current question and answer task type should be in the same field, which is beneficial to improve the generalization and robustness of the model when training the model.

[0063] In step S102, the training data is sorted according to a first data length of the training data, to obtain sorted data.

[0064] Optionally, the data length of the training data, referred to as the first data length here, is obtained. The training data needs to be sorted according to the length of the first data length to obtain sorted data. It should be noted that the sorting here can be from long to short or from short to long.

[0065] In step S103, the sorted data is split to obtain ordered data buckets.

[0066] Optionally, in the present embodiment, after obtaining the above sorted data, the sorted data needs to be split, and some sorted data is combined together according to certain rules and stored in the same data bucket, such as storing sorted data with similar lengths in the same data bucket. In the case where the number of data buckets is multiple, each data bucket will store training data with similar lengths and sorted by length. Since the training data in each data bucket is sorted, these data buckets can also be referred to as ordered data buckets.

[0067] In step S104, the ordered data buckets are used to determine grouped data, wherein the grouped data is data after length adjustment of the training data when the model is trained.

[0068] ​Optionally, after obtaining the ordered data buckets, the server samples data in the ordered data buckets, and then groups the sampled data to obtain grouped data. It can be understood that grouping the sampled data is usually a further splitting process, so that multiple micro-batch grouped data can be obtained, which is the training data that needs to be input into the model after length adjustment during model training.

[0069] In the embodiments of the present disclosure, the training data to be adjusted is obtained; the training data is sorted according to a first data length of the training data to obtain sorted data; the sorted data is split to obtain ordered data buckets; and grouped data is determined according to the ordered data buckets, wherein the grouped data is data after length adjustment of the training data during model training. In this way, length grouping processing means is used to adjust the length of the training data, which can avoid the additional computing power consumption caused by padding processing and improve the overall computing efficiency and performance.

[0070] In some optional embodiments, the sorted data is split to obtain ordered data buckets, including:

[0071] A second data length of the sorted data is obtained.

[0072] The length difference between the second data lengths is compared.

[0073] The sorted data with a length difference less than a threshold value is divided into the same bucket to obtain the ordered data buckets.

[0074] Optionally, in the embodiments of the present disclosure, after obtaining the sorted data, the data length of each sorted data needs to be obtained, which is referred to as a second data length of the sorted data. Then the length difference between these second data lengths is compared. At the same time, a threshold value is set, such as 0.8, which is used to represent the minimum value of the length proximity between any two second data lengths. When the length difference is less than the threshold value, it is considered that the lengths of these sorted data are similar, and the server will divide these sorted data into the same bucket. Since the sorted data has been sorted in order from short to long, the sorted data obtained in each bucket is length-proximate and sorted. At this time, these buckets can be arranged and stored as ordered data buckets, and the bucket effect can be referred to as Figure 2 .

[0075] In the embodiments of the present disclosure, by dividing the sorted data with a length difference between data lengths less than a threshold value into the same bucket, the padding length is reduced, and the training efficiency is improved.

[0076] In some optional embodiments, the grouped data is determined according to the ordered data buckets, including:

[0077] distributing data in the ordered data bucket to the computing devices;

[0078] grouping the data in the computing devices to obtain grouped data.

[0079] Optionally, part or all of the data in the ordered data bucket is collected into different computing devices, and the number of the computing devices can be at least one. For example, all data in the ordered data bucket 1 is distributed to the computing device 0, the computing device 1, and the computing device n. Similarly, all data in the ordered data bucket 2 is distributed to the computing device 0, the computing device 1, and the computing device n.

[0080] Then, the data in each computing device is grouped to obtain grouped data. It can be understood that the grouping is essentially further micro-batch division of the data in the computing device, so as to further shorten the length of the training data in the input training model.

[0081] In some optional embodiments, grouping the data in the computing devices to obtain grouped data comprises:

[0082] obtaining a grouping minimum unit value;

[0083] grouping the data in the computing devices according to the grouping minimum unit value to obtain grouped data.

[0084] Optionally, the disclosure embodiment can define a grouping minimum unit value, for example, the grouping minimum unit value is 2. The grouping minimum unit value can be a batch processing value set in micro-batch processing, for example, the batch processing value is 2. Figure 3 As shown in FIG. 6, the data in the computing devices is grouped by using the grouping minimum unit value, and then a plurality of grouped data is obtained. In Figure 3 In the embodiment, each computing device contains a plurality of grouped data.

[0085] In the disclosure embodiment, by grouping the data on each computing device according to the grouping minimum unit value, the number of training data input into the training model is reduced, and the overall efficiency and reliability of model fine-tuning are improved.

[0086] In some optional embodiments, after the grouped data is determined according to the ordered data bucket, the method further comprises:

[0087] randomly arranging the grouped data;

[0088] using the randomly arranged data as sample data for iterative updating of model parameters in the model training process.

[0089] Optionally, in the embodiments of the present disclosure, after obtaining the micro-batch processed grouped data, the grouped data can be randomly arranged, and then the randomly arranged training data is used as the sample data required for each model or model parameter iteration update in the model training process. As shown in FIG. 8, the grouped data is shuffled, and the in-memory data of each computing device is disordered grouped data. Figure 4

[0090] In the embodiments of the present disclosure, by randomly shuffling the grouped data, the sample data for model training is made random to obtain better training effect.

[0091] In some optional embodiments, the training data is sorted according to a first data length of the training data to obtain sorted data, including:

[0092] The training data is spliced according to a preset template format to obtain spliced data;

[0093] A third data length of the spliced data is determined;

[0094] The spliced data is sorted according to the third data length to obtain sorted data.

[0095] Optionally, in the embodiments of the present disclosure, a preset template format is set, such as "###Instruction:\n{instruction}\n\n###Input:\n{input}\n\n###Response:\n", and the obtained training data is spliced using the preset template format to obtain spliced data, and then the spliced data is used as the data set for subsequent model training.

[0096] The server obtains the data length of the spliced data, which is referred to as a third data length of the spliced data, and then sorts the data according to the third data length to obtain sorted data in ascending or descending order.

[0097] As shown in FIG. 8, Figure 5 Figure 5 is a schematic diagram of the overall process of the method for adjusting the training data length according to the embodiments of the present disclosure, and the specific process is as follows:

[0098] Obtain the original data;

[0099] Sort the original data by length;

[0100] Obtain the sorted data set;

[0101] Split the sorted data set to obtain multiple ordered data buckets;

[0102] ​​Collect data from each sorted bucket onto different computing devices;

[0103] Group the data into micro-batches within each computing device;

[0104] The grouped data is shuffled to obtain sample data for input into the training model.

[0105] In an embodiment of the present disclosure, a model training device is also provided, which includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule;

[0106] The data set processing submodule is used to execute the method for adjusting the length of training data mentioned in the above embodiments to obtain packet data, and send the packet data to the encoding submodule;

[0107] The encoding sub-model is used to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marked information, convert the marked information into a digital identifier, and send the digital identifier to the model training process calculation sub-module;

[0108] The model training process calculation submodule is used to identify the digital identifier and use the text information corresponding to the digital identifier as training data for model training. The model training process calculation submodule adopts a distributed strategy to encapsulate the model during the model training process;

[0109] The checkpoint submodule is used to save the key information updated by the calculation submodule during the model training process.

[0110] Alternatively, as Figure 6 As shown, the model training device includes a data set processing submodule 1, an encoding submodule 2, a model training process calculation submodule 3, and a checkpoint submodule 4. The data set processing submodule 1 is connected to the encoding submodule 2, the encoding submodule 2 is connected to the model training process calculation submodule 3, and the model training process calculation submodule 3 is connected to the checkpoint submodule 4.

[0111] It should be noted that the data set processing submodule 1 here is used to execute the method for adjusting the length of training data disclosed in the above embodiments to obtain grouped data. The specific implementation process is referred to the above embodiments and will not be repeated here. The obtained grouped data is then sent to the encoding submodule 2.

[0112] The main function of the encoding submodule 2 is to segment the text in the grouped data, that is, to divide the input text into independent "tags" to obtain tag information, which can be words, characters or subwords.

[0113] Then the mark information is converted into a digital mark by an encoding method, such as a data ID. For example, the data ID corresponding to "Galso" is 1101, the data ID corresponding to "Git" is 1102, and the data ID corresponding to "_n" is 1107. Each mark information has a unique digital ID corresponding to it.

[0114] In the embodiments of the present disclosure, these digital IDs are taken as inputs of the training model, and the model training process calculation submodule 3 is used to calculate the model. Figure 6 As can be seen from the above, the digital ID is sent to the model training process calculation submodule 3.

[0115] The model training process calculation submodule 3 mainly completes the encapsulation of the model, supports the direct use of the existing model on the open source library or the self-defined optimized model. The capabilities provided by the model training process calculation submodule 3 mainly include:

[0116] Complete the initialization of the local model. For example, the model is constructed according to the model configuration file.

[0117] 8-bit quantization processing of the model. The linear layer weight and the activation value in the model are expressed as 8-bit integers, which reduces the memory occupation in the model training process and maintains the performance of the model to a certain extent.

[0118] A bypass low-rank matrix is added in each layer of the Transformer architecture, the pre-training model parameters are fixed, and only the bypass model parameters are trained and updated.

[0119] Meanwhile, in the model encapsulation of the model training process calculation submodule 3, a distributed strategy is adopted, so that the model encapsulation provides the capability of distributed training for the model.

[0120] During the model training process of the model training process calculation submodule, the checkpoint submodule 4 saves and loads the key information in the training process, such as regularly saving the parameters, weights and other key information of the model, so as to restore the previous state when the training is interrupted or problems occur.

[0121] In the embodiments of the present disclosure, when the checkpoint submodule 4 saves the key information updated by the model training process calculation submodule 3 in the model training process, a distributed saving method can be used, specifically: each computing device saves the weights it holds, there is no parameter transmission between computing devices, and a single computing device only needs to process the data it contains, which greatly speeds up the saving speed. Distributed saving can also be in an asynchronous manner, such as transferring the weights to the CPU and then continuing the training, while the CPU process writes the data to the disk.

[0122] In addition, the model training device also has the following functions:

[0123] Model training: responsible for executing the training process of the model, including model calculation and iterative updating of model parameters.

[0124] Optimizer management: manage the optimizer, such as setting the learning rate, update strategy, etc.

[0125] Training log recording: record various indicators and information during the training process, such as loss value, accuracy, etc.

[0126] Training control: control the start, stop, pause, etc. of the training operation.

[0127] In the embodiments of the present disclosure, the data set processing submodule, the encoding submodule, the model training process calculation submodule, and the checkpoint submodule are organically combined to form a unique framework structure design. Since the framework structure provides simple and easy-to-use distributed training capabilities, it can perform model pre-training and fine-tuning with large parameter quantities, and at the same time, the asynchronous distributed checkpoint saving capability effectively shortens the training time, improves the hardware utilization rate in the training process, and improves the overall efficiency and reliability of model fine-tuning.

[0128] In the embodiments, a training data length adjustment apparatus is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware implementation is also possible and contemplated.

[0129] The present embodiment provides a training data length adjustment apparatus, as shown in Figure 7 , comprising:

[0130] The acquisition module 701 is configured to acquire training data to be adjusted.

[0131] The first sorting module 702 is configured to sort the training data according to the first data length of the training data to obtain sorted data.

[0132] The splitting module 703 is configured to split the sorted data to obtain ordered data buckets.

[0133] The determination module 704 is configured to determine grouped data according to the ordered data buckets, wherein the grouped data is data after length adjustment of the training data when performing model training.

[0134] In the embodiments of the present disclosure, the length grouping processing means is used to adjust the length of the training data, which can avoid the additional computing power consumption caused by the padding processing of the data as much as possible, and improve the overall computing efficiency and performance.

[0135] In some optional embodiments, the splitting module 703 comprises:

[0136] an acquisition sub-module, configured to acquire second data lengths of the sorted data;

[0137] a comparison sub-module, configured to compare length differences between the second data lengths;

[0138] a division sub-module, configured to divide the sorted data with a length difference less than a threshold into a same bucket, to obtain ordered data buckets.

[0139] In some optional embodiments, the determining module 704 comprises:

[0140] a distribution sub-module, configured to distribute data in the ordered data buckets to computing devices;

[0141] a grouping sub-module, configured to group the data in the computing devices, to obtain grouped data.

[0142] In some optional embodiments, the grouping sub-module comprises:

[0143] an acquisition unit, configured to acquire a grouping minimum unit value;

[0144] a grouping unit, configured to group the data in the computing devices according to the grouping minimum unit value, to obtain the grouped data.

[0145] In some optional embodiments, the apparatus further comprises:

[0146] a second sorting module, configured to randomly arrange the grouped data after determining the grouped data according to the ordered data buckets;

[0147] a setting module, configured to set the randomly arranged data as sample data for iterative updating of model parameters in a model training process.

[0148] In some optional embodiments, the first sorting module 702 comprises:

[0149] a splicing sub-module, configured to splice the training data according to a preset template format, to obtain spliced data;

[0150] a determining sub-module, configured to determine third data lengths of the spliced data;

[0151] a sorting sub-module, configured to sort the spliced data according to the third data lengths, to obtain sorted data.

[0152] Further function descriptions of the above various modules and units are the same as those of the corresponding embodiments, and will not be described here.

[0153] The model training apparatus in the embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0154] The embodiment of the present disclosure also provides a computer device having the above Figure 7 model training apparatus.

[0155] Please refer to Figure 8 , Figure 8 is a structural schematic diagram of a computer device provided by an optional embodiment of the present disclosure, as Figure 8 shown, the computer device includes one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common mainboard or in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device, such as a display device coupled to the interface. In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memories, if necessary. Also, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 In the embodiment, the processor 10 is taken as an example.

[0156] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a generic array logic, or any combination thereof.

[0157] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0158] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for at least one function, etc. The data storage area can store data created by the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory such as at least one disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include memory that is remotely located with respect to the processor 10, and which can be connected to the computer device through a network. Examples of such networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communications network, and combinations thereof.

[0159] The memory 20 can include a volatile memory, such as a random access memory, and / or can include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid state memory device. The memory 20 can also include an array of memories of the same kind or different kinds.

[0160] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0161] The embodiments of the present disclosure also provide a computer readable storage medium, and the method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or recorded in a storage medium, or implemented as computer code to be originally stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned kinds of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0162] Part of the present disclosure can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present disclosure can be invoked or provided. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file and the like, and accordingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0163] Although the embodiments of the present disclosure are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for adjusting the length of training data, characterized in that: The method comprises: Obtain the training data to be adjusted; sorting the training data according to the first data length of the training data to obtain sorted data; Splitting the sorted data to obtain ordered data buckets; Determining grouped data according to the ordered data buckets, wherein the grouped data is data after length adjustment of the training data when executing model training; The determining of grouped data according to the ordered data buckets includes: Distributing the data in the ordered data buckets to computing devices; The data in the computing device is grouped to obtain the grouped data; the grouping characterization of the data in the computing device further divides the data in the computing device into micro-batches.

2. The method according to claim 1, characterized in that The step of splitting the sorted data to obtain ordered data buckets includes: Acquire a second data length of the sorted data; comparing the length difference between the second data lengths; The sorted data with length differences less than a threshold are divided into the same bucket to obtain the ordered data bucket.

3. The method according to claim 1, characterized in that The grouping of the data in the computing device to obtain the grouped data includes: Get the minimum unit value of the group; The data in the computing device is grouped according to the grouping minimum unit value to obtain the grouped data.

4. The method according to claim 1 or 3, characterized in that After determining the grouped data according to the ordered data buckets, the method further includes: Randomly arranging the grouped data; The randomly arranged data is used as sample data for iteratively updating the model parameters during the model training process.

5. The method according to claim 1, wherein Sorting the training data according to the first data length of the training data to obtain sorted data includes: Splicing the training data according to a preset template format to obtain spliced ​​data; Determining a third data length of the spliced ​​data; The spliced ​​data is sorted according to the third data length to obtain the sorted data.

6. A model training device, characterized in that: The model training device includes a data set processing submodule, an encoding submodule, a model training process calculation submodule, and a checkpoint submodule; The data set processing submodule is configured to execute the method for adjusting the length of training data according to claim 1 to obtain packet data, and send the packet data to the encoding submodule; The encoding sub-model is used to segment the grouped data to obtain segmentation information, mark the segmentation information to obtain marking information, convert the marking information into a digital identifier, and send the digital identifier to the model training process calculation sub-module; The model training process calculation submodule is used to identify the digital identifier and use the text information corresponding to the digital identifier as training data to perform model training, and the model training process calculation submodule adopts a distributed strategy to encapsulate the model during the model training process; The checkpoint submodule is used to save key information updated by the model training process calculation submodule during the model training process.

7. A device for adjusting the length of training data, characterized in that: The device comprises: An acquisition module, used to obtain training data to be adjusted; A first sorting module, configured to sort the training data according to a first data length of the training data to obtain sorted data; A splitting module, used for splitting the sorted data to obtain ordered data buckets; A determination module, which determines grouped data according to the ordered data buckets, wherein the grouped data is data after the length of the training data is adjusted when executing model training; The determining of grouped data according to the ordered data buckets includes: Distributing the data in the ordered data buckets to computing devices; The data in the computing device is grouped to obtain the grouped data; the grouping characterization of the data in the computing device further divides the data in the computing device into micro-batches.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for adjusting the length of training data according to any one of claims 1 to 5 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for adjusting the length of training data according to any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for adjusting the length of training data according to any one of claims 1 to 5.

Images