Deep learning model training method and device, equipment and medium

Abstract

The invention relates to a deep learning model training method, device, equipment and medium, a deep learning model to be trained comprises a third-level model, a first-level model and a second-level model, the third-level model and the first-level model are connected in series, and the first-level model and the second-level model are connected in series. The deep learning model training method comprises the steps of obtaining a first sample data set, a second sample data set and a third sample data set; inputting the first sample data set into the three-level model to obtain a first result; obtaining a fourth sample data set based on the first result, and inputting the fourth sample data set into the primary model to obtain a second result; and obtaining a fifth sample data set according to the first result, the second result, the second sample data set and the third sample data set, and training the secondary model based on the fifth sample data set to obtain a trained deep learning model, thereby improving the calculation efficiency of the obtained trained deep learning model and reducing the calculation duration.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a deep learning model training method, device, equipment and medium. Background Art

[0002] Deep learning is a new research direction in the field of machine learning. It refers to a machine learning technology that learns and infers specific tasks based on artificial neural network models. The main advantage of this technology is that it can automatically learn and extract key features from raw data, thereby reducing dependence on professional knowledge in related fields, and its application scenarios cover multiple fields such as image, speech, and natural language processing. A deep learning model usually refers to a learning model composed of a series of interconnected neural network models such as recurrent neural network models (RNN) and convolutional neural networks (CNN). Deep learning models play a vital role in executing deep learning tasks.

[0003] In the existing deep learning model training process, each neural network model in a series of interconnected neural network models is usually trained separately to obtain each trained neural network model, and then each trained neural network model is connected to each other to obtain a trained deep learning model.

[0004] Therefore, the series of interconnected neural network models in the trained deep learning model cannot directly interact with each other. The neural network model of each endpoint needs to interact with the neural network model of the main control node to achieve further interaction between data, resulting in low computational efficiency and long computational time of the deep learning model. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a deep learning model training method, device, equipment and medium, wherein the deep learning model to be trained includes a three-level model, a first-level model and a second-level model, the three-level model and the first-level model are connected in series, and the first-level model and the second-level model are connected in series.

[0006] A first aspect of an embodiment of the present disclosure provides a deep learning model training method, the method comprising:

[0007] Obtain a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set contains sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set contains sample data in the field to which the third sample data set belongs;

[0008] Inputting the first sample data set into the three-level model so that the three-level model outputs a first result;

[0009] Preprocessing the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set, and inputting the fourth sample data set into the primary model so that the primary model outputs the second result;

[0010] The first result, the second result, the second sample data set and the third sample data set are preprocessed to obtain a fifth sample data set, and the secondary model is trained based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

[0011] A second aspect of an embodiment of the present disclosure provides a deep learning model training device, the device comprising:

[0012] A data acquisition module, used to acquire a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set contains sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set contains sample data in the field to which the third sample data set belongs;

[0013] A first input module, used for inputting a first sample data set into the three-level model so that the three-level model outputs a first result;

[0014] A second input module is used to preprocess the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set, and input the fourth sample data set to the primary model so that the primary model outputs a second result;

[0015] The third input module is used to preprocess the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set, and train the secondary model based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

[0016] A third aspect of an embodiment of the present disclosure provides an electronic device, the device comprising:

[0017] Memory;

[0018] Processor; and

[0019] A computer program, wherein the computer program is stored in a memory and is configured to be executed by a processor to implement the deep learning model training method as described in the first aspect above.

[0020] The fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the deep learning model training method described in the first aspect above is implemented.

[0021] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0022] The deep learning model training method, apparatus, device and medium provided by the embodiments of the present disclosure can obtain a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model and a third sample data set corresponding to the second-level model, wherein the first sample data set contains sample data in the field to which the second sample data set and the third sample data set belong; the second sample data set contains sample data in the field to which the third sample data set belongs, input the first sample data set into the three-level model so that the three-level model outputs a first result, pre-process the first result, the first sample data set and the second sample data set to obtain a fourth sample data set, and input the fourth sample data set into the first-level model so that the first-level model outputs a second result. The first result, the second result, the second sample data set and the third sample data set are preprocessed to obtain a fifth sample data set, and the secondary model is trained based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained, thereby, the three-level model and the first-level model are connected in series, the first-level model and the second-level model are connected in series, and the sample data sets input into the three-level model, the first-level model and the second-level model are partially connected in parallel, thereby ensuring data interaction between the three-level model, the first-level model and the second-level model, thereby improving the computational efficiency of the trained deep learning model and reducing the computational time. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0025] Figure 1 is a flowchart of a deep learning model training method provided by an embodiment of the present disclosure;

[0026] Figure 2It is a structural diagram of a deep learning model training process provided by an embodiment of the present disclosure;

[0027] Figure 3 It is a structural schematic diagram of a standard unit in a deep learning model training process provided by an embodiment of the present disclosure;

[0028] Figure 4 It is a structural schematic diagram of a deep learning model training device provided by an embodiment of the present disclosure;

[0029] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0034] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0035] Typically, in the existing deep learning model training process, each neural network model in a series of interconnected neural network models is trained separately to obtain each trained neural network model, and then each trained neural network model is connected to each other to obtain a trained deep learning model. Therefore, data cannot be directly interacted between the series of interconnected neural network models in the trained deep learning model, and manual operation is required to interact with the data, resulting in low computational efficiency and long computational time for the deep learning model. To address this problem, the present disclosure provides a deep learning model training method, which is described below in conjunction with a specific embodiment.

[0036] Figure 1 It is a flowchart of a deep learning model training method provided in an embodiment of the present disclosure. The method can be executed by a deep learning model training device. The deep learning model training device can be implemented in software and / or hardware. The deep learning model training device can be configured in an electronic device, such as a server or a terminal or a server cluster.

[0037] like Figure 1 As shown, the deep learning model to be trained in the deep learning model training method provided by the embodiment of the present disclosure includes a three-level model, a first-level model and a second-level model. The three-level model and the first-level model are connected in series, and the first-level model and the second-level model are connected in series. The deep learning model training method includes the following steps.

[0038] S110. Obtain a first sample data set corresponding to the third-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set includes sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set includes sample data in the field to which the third sample data set belongs.

[0039] In an embodiment of the present disclosure, when the electronic device needs to train a deep learning model to be trained, or receives an instruction to train a deep learning model to be trained, it obtains a first sample data set corresponding to the third-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model.

[0040] In the embodiments of the present disclosure, the three-level model, the first-level model and the second-level model may be the same machine learning model or different machine learning models. For example, the three-level model, the first-level model and the second-level model may adopt the same basic structure or different basic structures.

[0041] Exemplarily, the three-level model can use the T5-Base model as the basic structure, including 12 layers and 200M parameters; the first-level model and the second-level model can choose the T5-Small model as the basic structure, including 6 layers and 60M parameters.

[0042] Among them, the three-level model, the first-level model and the second-level model are models that meet different levels of complexity of the same type of tasks, such as text tasks, image processing tasks, feature extraction tasks, audio processing tasks, etc., but the corresponding tasks have different levels of complexity.

[0043] Furthermore, the first sample data set includes sample data related to the second sample data set and the third sample data set, and the second sample data set includes sample data related to the third sample data set.

[0044] Specifically, the first sample data set includes sample data in the same field as the second sample data set and the third sample data set, and the second sample data set includes sample data in the same field as the third sample data set.

[0045] For example, the first sample data set may be a data set in a general massive knowledge base, the second sample data set may be a data set in a knowledge base in a specific field, such as a data set in the vehicle field, and the third sample data set may be a data set in a knowledge base in a specific aspect of a specific field, such as a data set in the vehicle field question and answer aspect. The first sample data set includes a data set in the vehicle field, and further includes a data set in the vehicle field question and answer aspect.

[0046] Optionally, specialized fields may include medicine, aerospace, teaching, etc., which are not limited here.

[0047] Figure 2 is a structural diagram of a deep learning model training process provided by an embodiment of the present disclosure, such as Figure 2 As shown, the three-level model and the one-level model are connected in series, and the one-level model and the two-level model are connected in series, which include a first sample data set corresponding to the three-level model, a second sample data set corresponding to the one-level model, and a third sample data set corresponding to the two-level model.

[0048] In some embodiments of the present disclosure, the deep learning model may further include multiple N-level models, wherein the sample data set of the N-level model includes sample data related to the first sample data set. It can also be understood that the sample set of the N-level model is more than that of the three-level model.

[0049] S120: Input the first sample data set into the three-level model so that the three-level model outputs a first result.

[0050] In the embodiment of the present disclosure, after acquiring the first sample data set, the second sample data set and the third sample data set, the electronic device inputs the first sample data set into the three-level model so that the three-level model outputs the first result.

[0051] The first result includes the first sample data and the labeled data corresponding to the first sample data.

[0052] Specifically, after acquiring the first sample data set, the electronic device inputs the first sample data set into the three-level model, and the three-level model processes the first sample data set to obtain a processing result corresponding to the first sample data set, thereby causing the three-level model to output a first result.

[0053] S130, preprocessing the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set, and inputting the fourth sample data set into the primary model so that the primary model outputs the second result.

[0054] In the disclosed embodiment, after obtaining the first result, the electronic device preprocesses the first result, the first sample data set and the second sample data set to obtain a fourth sample data set, and inputs the fourth sample data set into the primary model so that the primary model outputs the second result.

[0055] The second result includes the fourth sample data and the labeled data corresponding to the fourth sample data.

[0056] In the embodiments of the present disclosure, preprocessing may include data screening processing, data standardization processing, data alignment processing, etc.

[0057] Specifically, after obtaining the first result, the electronic device preprocesses the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set for training the first-level model, inputs the fourth sample data set into the first-level model, and the first-level model processes the fourth sample data set to obtain a processing result corresponding to the fourth sample data set, thereby enabling the first-level model to output the second result.

[0058] S140. Preprocess the first result, the second result, the second sample data set, and the third sample data set to obtain a fifth sample data set, and train the secondary model based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

[0059] In an embodiment of the present disclosure, after obtaining the second result, the electronic device preprocesses the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set, and trains the secondary model based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

[0060] In the embodiments of the present disclosure, preprocessing may include data screening processing, data standardization processing, data alignment processing, etc.

[0061] Specifically, after obtaining the second result, the electronic device preprocesses the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set for training the secondary model, inputs the fifth sample data set into the secondary model, and the secondary model processes the fifth sample data set to obtain a processing result corresponding to the fifth sample data set, and then trains the deep learning model to be trained using a self-supervised training method to obtain a trained deep learning model.

[0062] In an embodiment of the present disclosure, a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model can be obtained, wherein the first sample data set includes sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set includes sample data in the field to which the third sample data set belongs. The first sample data set is input into the three-level model so that the three-level model outputs a first result, the first result, the first sample data set, and the second sample data set are preprocessed to obtain a fourth sample data set, the fourth sample data set is input into the first-level model so that the first-level model outputs a second result, the first result, the second result, the second sample data set, and the third sample data set are preprocessed to obtain a fifth sample data set, and the second-level model is trained based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained, thereby, by connecting the three-level model and the first-level model in series, connecting the first-level model and the second-level model in series, and connecting the sample data sets input into the three-level model, the first-level model, and the second-level model in partial parallel, data interaction between the three-level model, the first-level model, and the second-level model is ensured, thereby improving the computational efficiency of the obtained trained deep learning model while reducing the computational time.

[0063] Based on the above-mentioned embodiment of the present disclosure, preprocessing the first result, the first sample data set, and the second sample data set in S130 to obtain a fourth sample data set may specifically include: screening out a first target result that meets a first preset condition from the first result; screening out a first target sample data set that meets the first preset condition from the first sample data set; and preprocessing the first target result, the first target sample data set, and the second sample data set to obtain a fourth sample data set.

[0064] In the embodiment of the present disclosure, the first preset condition may be a condition that meets the training requirements of the first-level model, such as the sample data belonging to the field corresponding to the first-level model.

[0065] Optionally, data preprocessing may include standardization, alignment, format conversion, etc. of the data.

[0066] The first target result is a first target sample data set with labeled data.

[0067] Specifically, after obtaining the first result, the electronic device filters the first result through a preset first filtering function to screen out a first target result that meets the first preset condition, and at the same time filters out a first target sample data set that meets the first preset condition from the first sample data set through the preset first filtering function, performs data preprocessing on the first target result, the first target sample data set, and the second sample data set, so as to obtain a fourth sample data set.

[0068] like Figure 2 As shown, the electronic device can filter the first result output by the three-level model through the first filtering function F1(x) to obtain the first target result, and at the same time filter the first sample data set through F1(x) to screen out the first target sample data set that meets the first preset condition in the first sample data set, and obtain the fourth sample data set after performing data format conversion, data standardization processing and data pairing processing on the first target result, the first target sample data set and the second sample data set through the preprocessing function F2(x), and input it into the first-level model.

[0069] In the embodiment of the present disclosure, the first result and the first sample data set can be screened respectively to obtain the first target result and the first target sample data set, and then the first target result and the first target sample data set are used together with the second sample data set as the input of the first-level model, and the first-level model is trained so that there is a logical dependence between the third-level model and the first-level model, and data interaction can be automatically performed, thereby improving the computational efficiency of the deep learning model.

[0070] Furthermore, in S140, the first result, the second result, the second sample data set and the third sample data set are preprocessed to obtain a fifth sample data set, which may specifically include: screening out a second target result that meets the second preset condition from the first result; screening out a third target result that meets the second preset condition from the second result; screening out a second target sample data set that meets the second preset condition from the second sample data set; and performing data preprocessing on the second target result, the third target result, the second target sample data set and the third sample data set to obtain a fifth sample data set.

[0071] In the disclosed embodiment, the second preset condition may be a condition that satisfies the training requirements of the secondary model, such as the sample data being data of a specific aspect in the field corresponding to the secondary model.

[0072] Specifically, after obtaining the second result, the electronic device filters the first result through a preset second filtering function to screen out the second target result that meets the second preset condition, and at the same time filters the second result through a preset third filtering function to screen out the third target result that meets the second preset condition, and filters out the second target sample data set that meets the second preset condition from the first sample data set through the preset third filtering function, and performs data preprocessing on the second target result, the third target result, the first target sample data set and the second sample data set to obtain the fifth sample data set.

[0073] like Figure 2 As shown, the electronic device can filter the first result output by the three-level model through the second filtering function F1-1(x) to obtain the second target result, filter the second result output by the first-level model through the third filtering function F1-2(x) to obtain the third target result, and at the same time filter the second sample data set through the third filtering function F1-2(x) to filter out the second target sample data set that meets the second preset condition, and then preprocess the second target result, the third target result, the second target sample data set and the third sample data set through the preprocessing function F3(x) to obtain the fifth sample data set.

[0074] In the embodiments of the present disclosure, the first result, the second result and the second sample data set can be screened and filtered, and then the second target result, the third target result, the second target sample data set and the third sample data set obtained by screening and filtering can be preprocessed to obtain a fifth sample data set. The secondary model can be trained with the fifth sample data set so that there is a logical dependency between the tertiary model, the primary model and the secondary model, and data interaction can be performed, thereby further improving the computational efficiency of the deep learning model.

[0075] Based on the above embodiments of the present disclosure, the three-level model, the first-level model and the second-level model can be abstractly processed to obtain standard units. The deep learning model training method can also include: extracting model parameters corresponding to the three-level model, the first-level model and the second-level model respectively; determining the target model parameters from the model parameters; generating standard units based on the target model parameters; performing target processing on the standard units to obtain a deep learning model.

[0076] Among them, the target model parameters can be model parameters shared by the three-level model, the first-level model and the second-level model.

[0077] The standard unit can also be understood as the unified model structure information in the three-level model, the first-level model and the second-level model except for the number of layers and the parameter information of each layer.

[0078] Specifically, the electronic device can respectively extract the model parameters in the three-level model, the first-level model and the second-level model, such as the number of layers of the three-level model, the first-level model and the second-level model, the parameters corresponding to each layer, the position information of each layer in the model, etc., and then determine the same model parameters in the three-level model, the first-level model and the second-level model from the extracted model parameters, determine them as the target model parameters, and generate standard units according to the target model parameters.

[0079] Furthermore, target processing is performed on the standard units to obtain a deep learning model, which may specifically include: performing value assignment processing on the standard units to obtain a target model of a preset structure; and freely combining the target models to obtain a deep learning model.

[0080] The assignment process can be understood as instantiation processing based on the standard unit according to design parameters and the like.

[0081] Exemplarily, the standard unit can be a relatively standard transformer structure, such as the outermost layer is the Encoder-Decoder structure, or the layer parameters of the Decoder layer can be assigned to 0, and there is only an Encoder layer; the internal structure can be a layernorm+attention+dropout+layernorm structure, etc., and the transformer structure is assigned on this basis to obtain the target model, so that the standard unit can be reused, reducing the implementation complexity of the deep learning model during combined training.

[0082] Figure 3 is a schematic diagram of a structure of a standard unit in a deep learning model training process provided by an embodiment of the present disclosure, such as Figure 3As shown, the standard unit includes standard input interface and output interface, through which data interaction with models at other levels in the deep learning model is realized, wherein the input interface includes a data input interface and a result input interface of the output result of the previous level model.

[0083] In the disclosed embodiment, the electronic device can generate standard units according to the three-level model, the first-level model and the second-level model, and freely combine the levels based on the standard units, so as to optimize the deep learning model in terms of the number of layers, improve the rate of the deep learning model training process, and reduce the complexity of the deep learning model training.

[0084] Figure 4 Schematic diagram of the structure of a deep learning model training device provided in an embodiment of the present disclosure. The deep learning model training device in the embodiment of the present disclosure can be set in an electronic device, and the electronic device can be a server or a terminal or a server cluster, which is not limited here.

[0085] like Figure 4 As shown, the deep learning model training device 400 may include a data acquisition module 410, a first input module 420, a second input module 430 and a third input module 440.

[0086] The data acquisition module 410 may be used to acquire a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set includes sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set includes sample data in the field to which the third sample data set belongs;

[0087] The first input module 420 may be used to input a first sample data set into the three-level model so that the three-level model outputs a first result;

[0088] The second input module 430 may be used to preprocess the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set, and input the fourth sample data set to the primary model so that the primary model outputs the second result;

[0089] The third input module 440 can be used to preprocess the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set, and train the secondary model based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

[0090] In an embodiment of the present disclosure, a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model can be obtained, wherein the first sample data set includes sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set includes sample data in the field to which the third sample data set belongs. The first sample data set is input into the three-level model so that the three-level model outputs a first result, the first result, the first sample data set, and the second sample data set are preprocessed to obtain a fourth sample data set, the fourth sample data set is input into the first-level model so that the first-level model outputs a second result, the first result, the second result, the second sample data set, and the third sample data set are preprocessed to obtain a fifth sample data set, and the second-level model is trained based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained, thereby, by connecting the three-level model and the first-level model in series, connecting the first-level model and the second-level model in series, and connecting the sample data sets input into the three-level model, the first-level model, and the second-level model in partial parallel, data interaction between the three-level model, the first-level model, and the second-level model is ensured, thereby improving the computational efficiency of the obtained trained deep learning model while reducing the computational time.

[0091] In some embodiments of the present disclosure, the second input module 430 may include a first screening unit 4301 , a second screening unit 4302 , and a first preprocessing unit 4303 .

[0092] The first screening unit 4301 may be configured to screen out a first target result that satisfies a first preset condition from the first result.

[0093] The second screening unit 4302 may be configured to screen out a first target sample data set that meets a first preset condition from the first sample data set.

[0094] The first preprocessing unit 4303 may be configured to perform data preprocessing on the first target result, the first target sample data set, and the second sample data set to obtain a fourth sample data set.

[0095] In some embodiments of the present disclosure, the third input module 440 may include a third screening unit 4401 , a fourth screening unit 4402 , a fifth screening unit 4403 and a second preprocessing unit 4404 .

[0096] The third screening unit 4401 may be configured to screen out second target results that meet a second preset condition from the first results.

[0097] The fourth screening unit 4402 may be configured to screen out third target results satisfying a second preset condition from the second results.

[0098] The fifth screening unit 4403 may be configured to screen out a second target sample data set that meets a second preset condition from the second sample data set.

[0099] The second preprocessing unit 4404 may be configured to perform data preprocessing on the second target result, the third target result, the second target sample data set, and the third sample data set to obtain a fifth sample data set.

[0100] In some embodiments of the present disclosure, the deep learning model training device 400 may also include a parameter extraction module 450, a parameter determination module 460, a standard unit generation module 470 and a target processing module 480.

[0101] The parameter extraction module 450 can be used to extract model parameters corresponding to the third-level model, the first-level model and the second-level model respectively.

[0102] The parameter determination module 460 may be configured to determine target model parameters from the model parameters.

[0103] The standard cell generation module 470 may be used to generate a standard cell based on the target model parameters.

[0104] The target processing module 480 can be used to perform target processing on the standard unit to obtain a deep learning model.

[0105] In some embodiments of the present disclosure, the target processing module 480 may include an assignment processing unit 4801 and a free combination unit 4802 .

[0106] The assignment processing unit 4801 can be used to perform assignment processing on the standard unit to obtain a target model of a preset structure.

[0107] The free combination unit 4802 can be used to freely combine the target models to obtain a deep learning model.

[0108] It should be noted that Figure 4 The deep learning model training device 400 shown can execute each step in the above-mentioned deep learning model training method embodiment, and realize each process and effect in the above-mentioned deep learning model training method embodiment, which will not be elaborated here.

[0109] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0110] In the disclosed embodiment, Figure 5 The electronic device shown can be a server or a terminal or a server cluster, wherein the terminal can specifically include a mobile phone, a computer or a tablet computer, a vehicle-mounted terminal, or any device that can be used for the deep learning model training method, etc., without limitation here.

[0111] like Figure 5 As shown, the electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0112] Specifically, the processor 510 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0113] The memory 520 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 520 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 520 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 520 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 520 is a non-volatile solid-state memory. In a particular embodiment, the memory 520 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0114] The processor 510 reads and executes the computer program instructions stored in the memory 520 to perform the steps of the deep learning model training method provided in the embodiment of the present disclosure.

[0115] In one example, the electronic device may further include a transceiver 530 and a bus 540. Figure 5 As shown, the processor 510, the memory 520 and the transceiver 530 are connected via a bus 540 and communicate with each other.

[0116] The bus 540 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 540 may include one or more buses.

[0117] The embodiments of the present disclosure also provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the deep learning model training method provided by the embodiments of the present disclosure.

[0118] The above-mentioned storage medium may, for example, include a memory 520 of computer program instructions, and the above-mentioned instructions may be executed by the processor 510 of the electronic device to complete the deep learning model training method provided in the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0119] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0120] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A deep learning model training method, characterized in that: The method comprises: Obtaining a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set contains sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set contains sample data in the field to which the third sample data set belongs; Inputting the first sample data set into the three-level model so that the three-level model outputs a first result; preprocessing the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set, and inputting the fourth sample data set into the primary model so that the primary model outputs a second result; The first result, the second result, the second sample data set and the third sample data set are preprocessed to obtain a fifth sample data set, and the secondary model is trained based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

2. The method according to claim 1, characterized in that The preprocessing of the first result, the first sample data set, and the second sample data set to obtain a fourth sample data set includes: Filtering out a first target result that meets a first preset condition from the first results; Filtering out a first target sample data set that meets the first preset condition from the first sample data set; Data preprocessing is performed on the first target result, the first target sample data set, and the second sample data set to obtain the fourth sample data set.

3. The method according to claim 1, characterized in that The preprocessing the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set includes: Filtering out a second target result that meets a second preset condition from the first result; Filtering out a third target result that meets the second preset condition from the second results; Filtering out a second target sample data set that meets the second preset condition from the second sample data set; Data preprocessing is performed on the second target result, the third target result, the second target sample data set, and the third sample data set to obtain the fifth sample data set.

4. The method according to claim 1, characterized in that: The method further comprises: Extracting model parameters corresponding to the third-level model, the first-level model and the second-level model respectively; determining target model parameters from the model parameters; generating a standard cell based on the target model parameters; Target processing is performed on the standard unit to obtain the deep learning model.

5. The method according to claim 4, characterized in that The performing target processing on the standard unit to obtain the deep learning model includes: Performing value assignment processing on the standard unit to obtain a target model of a preset structure; The target models are freely combined to obtain the deep learning model.

6. A deep learning model training device, characterized in that: The device comprises: A data acquisition module, used to acquire a first sample data set corresponding to the three-level model, a second sample data set corresponding to the first-level model, and a third sample data set corresponding to the second-level model, wherein the first sample data set includes sample data in the field to which the second sample data set and the third sample data set belong; and the second sample data set includes sample data in the field to which the third sample data set belongs; A first input module, used for inputting the first sample data set into the three-level model, so that the three-level model outputs a first result; A second input module is used to preprocess the first result, the first sample data set and the second sample data set to obtain a fourth sample data set, and input the fourth sample data set to the primary model so that the primary model outputs a second result; The third input module is used to perform the preprocessing on the first result, the second result, the second sample data set and the third sample data set to obtain a fifth sample data set, and train the secondary model based on the fifth sample data set to obtain a trained deep learning model, wherein a self-supervised training method is adopted when training the deep learning model to be trained.

7. The device according to claim 6, characterized in that The second input module includes a first screening unit, a second screening unit and a first pre-processing unit; The first screening unit is used to screen out a first target result that meets a first preset condition from the first result; The second screening unit is used to screen out a first target sample data set that meets the first preset condition from the first sample data set; The first preprocessing unit is used to perform data preprocessing on the first target result, the first target sample data set, and the second sample data set to obtain the fourth sample data set.

8. The device according to claim 6, characterized in that The third input module includes a third screening unit, a fourth screening unit, a fifth screening unit and a second pre-processing unit; The third screening unit is used to screen out the second target result that meets the second preset condition from the first result; The fourth screening unit is used to screen out a third target result that meets the second preset condition from the second result; The fifth screening unit is used to screen out a second target sample data set that meets the second preset condition from the second sample data set; The second preprocessing unit is used to perform data preprocessing on the second target result, the third target result, the second target sample data set and the third sample data set to obtain the fifth sample data set.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images