Model training method and device
By combining training feature extraction networks and processing networks, the problem of time-consuming calibration of sample data before training by machine learning models is solved, the accuracy of the model is improved, and model optimization under limited sample size is realized.
Patent Information
- Application Number
- CN202111434781.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-11-29
AI Technical Summary
In the prior art, the calibration of sample data before training of machine learning models requires a lot of manpower and time, resulting in limited sample size during training and low model accuracy.
By obtaining multiple sample input data and benchmark output data, combining training feature extraction networks and processing networks, the trained networks are used for alternating optimization to improve the accuracy of the model.
In the case of limited sample size, the accuracy of the machine learning model is improved through alternating training, and the optimization of the model training process is achieved.
Smart Images

Figure CN114065952B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for model training. Background Art
[0002] With the research and development of machine learning technology, machine learning models have been applied in many fields. Before application, machine learning models need to be trained to a certain extent.
[0003] For a machine learning model, model training usually involves: calibrating sample data to obtain the corresponding benchmark output, inputting the sample data into the model to obtain the actual output of the model, and adjusting the parameters in the model based on the actual output of the model and the benchmark output corresponding to the sample data, thereby completing the model training.
[0004] Before training the model, the sample data needs to be calibrated, which takes a lot of manpower and time. This means that the amount of samples that can be used when training the model is generally limited, resulting in a low accuracy rate of the trained model. Summary of the invention
[0005] The embodiment of the present application provides a method and device for model training, which can solve the problem of low accuracy of the trained model in the prior art. The technical solution is as follows:
[0006] In a first aspect, a model training method is provided, the method comprising:
[0007] Acquire first sample input data and corresponding first reference output data, second sample input data and corresponding second reference output data, wherein the first sample input data and the second sample input data have the same data type;
[0008] Based on the first sample input data and the first benchmark output data, performing combined training on the feature extraction network and the first processing network;
[0009] Based on the second sample input data and the second benchmark output data, the trained feature extraction network and the second processing network are combined trained.
[0010] In a possible implementation, based on the first sample input data and the first reference output data, the feature extraction network and the first processing network are combined for training, including:
[0011] Inputting the first sample input data into a feature extraction network to obtain first feature information corresponding to the first sample input data;
[0012] Inputting the first feature information into the first processing network to obtain first actual output data corresponding to the first sample input data;
[0013] Based on the first benchmark output data and the first actual output data, the feature extraction network and the first processing network are adjusted.
[0014] Based on the second sample input data and the second benchmark output data, the trained feature extraction network and the second processing network are combined and trained, including:
[0015] Inputting the second sample input data into the trained feature extraction network to obtain second feature information corresponding to the second sample input data;
[0016] Inputting the second feature information into the second processing network to obtain second actual output data corresponding to the second sample input data;
[0017] Based on the second benchmark output data and the second actual output data, the trained feature extraction network and the second processing network are adjusted.
[0018] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the first processing network includes a semantic correlation network, and the second processing network includes a semantic correlation network. Figure 1 or
[0019] The first sample input data includes two sample sentences, the second sample input data includes one sample sentence, the first processing network includes a semantic correlation network, and the second processing network includes a word category recognition network.
[0020] In a possible implementation, the method further includes:
[0021] Acquire third sample input data and corresponding third reference output data, wherein the third sample input data and the first sample input data have the same data type;
[0022] Based on the third sample input data and the third benchmark output data, the feature extraction network and the third processing network after the second training are combined and trained.
[0023] In a possible implementation, based on the third sample input data and the third benchmark output data, the feature extraction network and the third processing network after the second training are combined and trained, including:
[0024] Inputting the third sample input data into the feature extraction network after the second training to determine third feature information corresponding to the third sample input data;
[0025] Inputting the third feature information into a third processing network to determine third actual output data corresponding to the third sample input data;
[0026] Based on the third benchmark output data and the third actual output data, the parameters of the feature extraction network and the third processing network after the second training are adjusted.
[0027] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the third sample input data includes one sample sentence, the first processing network includes a semantic correlation network, and the second processing network includes a semantic correlation network. Figure 1 The consistency network, the third processing network includes a word category recognition network.
[0028] In a second aspect, a model training device is provided, the device comprising:
[0029] An acquisition module, used for acquiring first sample input data and corresponding first reference output data, second sample input data and corresponding second reference output data, wherein the first sample input data and the second sample input data have the same data type;
[0030] The training module is used to perform combined training on the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data; and to perform combined training on the trained feature extraction network and the second processing network based on the second sample input data and the second benchmark output data.
[0031] In a possible implementation, the training module is used to:
[0032] Inputting the first sample input data into a feature extraction network to obtain first feature information corresponding to the first sample input data;
[0033] Inputting the first feature information into the first processing network to obtain first actual output data corresponding to the first sample input data;
[0034] Adjusting parameters of a feature extraction network and a first processing network based on the first benchmark output data and the first actual output data;
[0035] Inputting the second sample input data into the trained feature extraction network to obtain second feature information corresponding to the second sample input data;
[0036] Inputting the second feature information into the second processing network to obtain second actual output data corresponding to the second sample input data;
[0037] Based on the second benchmark output data and the second actual output data, the trained feature extraction network and the second processing network are adjusted.
[0038] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the first processing network includes a semantic correlation network, and the second processing network includes a semantic correlation network. Figure 1 or
[0039] The first sample input data includes two sample sentences, the second sample input data includes one sample sentence, the first processing network includes a semantic correlation network, and the second processing network includes a word category recognition network.
[0040] In one possible implementation,
[0041] The acquisition module is further used to acquire third sample input data and corresponding third reference output data, wherein the third sample input data and the first sample input data have the same data type;
[0042] The training module is also used to perform combined training on the feature extraction network and the third processing network after the second training based on the third sample input data and the third benchmark output data.
[0043] In a possible implementation, the training module is used to:
[0044] Inputting the third sample input data into the feature extraction network after the second training to determine third feature information corresponding to the third sample input data;
[0045] Inputting the third feature information into a third processing network to determine third actual output data corresponding to the third sample input data;
[0046] Based on the third benchmark output data and the third actual output data, the parameters of the feature extraction network and the third processing network after the second training are adjusted.
[0047] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the third sample input data includes one sample sentence, the first processing network includes a semantic correlation network, and the second processing network includes a semantic correlation network. Figure 1 The consistency network, the third processing network includes a word category recognition network.
[0048] In a third aspect, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the operations performed by the model training method.
[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the operations performed by the model training method.
[0050] In a fifth aspect, a computer program product is provided, the computer program product comprising a computer program code, and when the computer program code is executed by a computer device, the computer device executes the method of the above-mentioned first aspect and its possible implementation methods.
[0051] The beneficial effects of the technical solution provided by the embodiment of the present application are:
[0052] The scheme mentioned in the embodiment of the present application, first, can complete the training of the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data. Then, the training of the trained feature extraction network and the second processing network can be completed based on the second sample input data and the corresponding second benchmark output data. Using this method, a feature extraction network can be shared for multiple machine learning models with a certain degree of processing relevance, that is, the feature extraction network and the first processing network constitute the first machine learning model, and the feature extraction network and the second processing network constitute the second machine learning model. In this way, when the number of samples calibrated for the first machine learning model and the number of samples calibrated for the second machine learning model are relatively limited, through the above method, the training process of the first machine learning model and the training process of the second machine learning model can both train and optimize the feature extraction network, that is, the training process of the first machine learning model will also play a training role for the second machine learning model, and the training process of the second machine learning model will also play a training role for the first machine learning model. Thus, in the case of a limited number of samples, the accuracy of the machine learning model can be better improved through training. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0054] Figure 1 It is a flow chart of a model training method provided in an embodiment of the present application;
[0055] Figure 2 is a schematic diagram of an intent classification system provided in an embodiment of the present application;
[0056] Figure 3 It is a flow chart of a model training method provided in an embodiment of the present application;
[0057] Figure 4 It is a flowchart of a model training provided in an embodiment of the present application;
[0058] Figure 5 is a schematic diagram of a word category system provided in an embodiment of the present application;
[0059] Figure 6 It is a flow chart of a model training method provided in an embodiment of the present application;
[0060] Figure 7 It is a flowchart of a model training provided in an embodiment of the present application;
[0061] Figure 8 It is a flowchart of a model training provided in an embodiment of the present application;
[0062] Fig. 9 It is a flowchart of a model training provided in an embodiment of the present application;
[0063] Fig.10 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;
[0064] Fig.11 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0066] The method of model training provided in the embodiment of the present application can be applied to the training of machine learning models. The so-called machine learning (ML) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.
[0067] The server can train multiple machine learning models synchronously. Multiple machine learning models can be machine learning models of the same type, that is, the data processed by multiple machine learning models can have the same data type. For example, multiple machine learning models are models for processing image data, or multiple machine learning models are models for processing text data, or multiple machine learning models are models for processing audio data, and so on. Multiple machine learning models can be machine learning models with a certain degree of processing relevance, that is, multiple machine learning models require the same characteristics of input data in the process of determining output data, for example, semantic relevance models and semantic relevance models. Figure 1 All consistency models require the semantic characteristics of text sentences.
[0068] Multiple machine learning models can have the same feature extraction network or share the same feature extraction network. For example, using the BERT (Bidirectional Encoder Representations from Transformer) network as the semantic relevance model and semantic Figure 1 Consistent models, the BERT networks in these two models have the same structure and parameters, or, the same BERT network is used as the semantic relevance model and word category recognition model of the feature extraction network, and so on.
[0069] The model training method provided in the embodiment of the present application can be implemented with the help of blockchain technology. The so-called blockchain is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer.
[0070] The underlying blockchain platform can include processing modules such as user management, basic services, smart contracts, and operation monitoring. Among them, the user management module is responsible for the identity information management of all blockchain participants, including maintaining public and private key generation (account management), key management, and the maintenance of the correspondence between the user's real identity and the blockchain address (authority management), etc., and, under authorization, supervises and audits the transactions of certain real identities and provides risk control rule configuration (risk control audit); the basic service module is deployed on all blockchain node devices to verify the validity of business requests, and records valid requests to storage after consensus is reached. For a new business request, the basic service first adapts the interface for parsing and authentication (interface adaptation), and then encrypts the business information through the consensus algorithm (consensus management). The smart contract module is responsible for the registration and issuance of contracts, as well as contract triggering and contract execution. Developers can define the contract logic in a programming language and publish it to the blockchain (contract registration). According to the logic of the contract terms, the key or other events are called to trigger the execution and complete the contract logic. It also provides the function of contract upgrade and cancellation. The operation monitoring module is mainly responsible for the deployment, configuration modification, contract setting, cloud adaptation and real-time status visualization output of the product during the product release process, such as alarm, network monitoring, node equipment health monitoring, etc.
[0071] The platform product service layer provides the basic capabilities and implementation framework for typical applications. Developers can build on these basic capabilities and overlay business features to complete the blockchain implementation of business logic.
[0072] The application service layer provides application services based on blockchain solutions for business participants to use.
[0073] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, smart medical care, etc. The embodiments of the present application provide a method for model training, and the execution subject of the method can be a server. The server may include a processor, a memory, and a communication component, etc.
[0074] The processor may be a CPU (Central Processing Unit) or a SoC (System on Chip), etc. The processor may be used to obtain first sample input data and corresponding first reference output data, to determine first feature information corresponding to the first sample input data, to determine first actual output data corresponding to the first sample input data, to perform combined training on a feature extraction network and a first processing network, and so on.
[0075] The memory can be various volatile memories or non-volatile memories, such as SSD (Solid State Disk), DRAM (Dynamic Random Access Memory), etc. The memory can be used to store pre-stored data, intermediate data, and result data in the model training process. For example, the first sample input data, the first reference output data corresponding to the first sample input data, the first feature information corresponding to the first sample input data, the first actual output data corresponding to the first sample input data, and so on.
[0076] The communication component may be a wired network connector, a WiFi (Wireless Fidelity) module, a Bluetooth module, a cellular network communication module, etc. The communication component may be used to transmit data with other devices, which may be other servers or user terminals, etc., wherein the user terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The communication component may be used to receive first sample input data and corresponding first reference input data sent by a terminal, and may be used to send the relevant parameters of the trained feature extraction network and the first processing network to other servers, etc.
[0077] The embodiment of the present application provides a method for training multiple machine learning models. The first machine learning model may include a feature extraction network and a first processing network, the second machine learning model may include a feature extraction network and a second processing network, and the third machine learning model may include a feature extraction network and a third processing network. The feature extraction network in the above three machine learning models may be a shared feature extraction network. The three machine learning models corresponding to the above three processing networks require the same characteristics of the input data in the process of determining the output data, for example, the semantic relevance model and the semantic relevance model. Figure 1 All consistency models require the semantic characteristics of text sentences.
[0078] The present application embodiment provides Figure 1 The processing flow of the model training method shown includes the following processing steps:
[0079] S101 . Acquire first sample input data and corresponding first reference output data, second sample input data and corresponding second reference output data.
[0080] The first sample input data and the second sample input data may have the same data type, for example, the first sample input data and the second sample input data are text data, the first sample input data and the second sample input data are image data, the first sample input data and the second sample input data are audio data, and so on.
[0081] The server may pre-store a sample library corresponding to the model to be trained, and the sample library may include sample input data and corresponding benchmark output data. The sample input data in the sample libraries corresponding to different models may be completely identical, partially identical, or completely different. For example, the first sample library corresponding to the first machine learning model includes 30 sample data items A1-A30, and the second sample library corresponding to the second machine learning model also includes the 30 sample data items A1-A30, or the first sample library includes 30 sample data items A1-A30, and the second sample library includes 30 sample data items A15-A45, or the first sample library includes 30 sample data items A1-A30, and the second sample library includes 30 sample data items B1-B30, and so on.
[0082] When the server starts training the model, it can obtain sample input data and benchmark output data corresponding to the sample input data from the sample library corresponding to the model to be trained. For example, when the server trains the first machine learning model and the second machine learning model, it can obtain the first sample input data and the corresponding first benchmark output data from the first sample library corresponding to the first machine learning model, obtain the second sample input data and the corresponding second benchmark output data from the second sample library corresponding to the second machine learning model, and so on.
[0083] S102: Based on the first sample input data and the first benchmark output data, perform combined training on the feature extraction network and the first processing network.
[0084] The server may input the acquired first sample input data into a feature extraction network to obtain corresponding first feature information. The first feature information may be a vector or a matrix.
[0085] When the first feature information is a vector, the dimension of the vector can be fixed. For example, the server inputs a sample text sentence including any number of words into the feature extraction network to obtain a 128-dimensional vector, or the server inputs image data of any pixel value into the feature extraction network to obtain a 256-dimensional vector, etc. The dimension of the vector can be preset according to the needs of the training process, for example, 128 dimensions, 400 dimensions, 768 dimensions, etc.
[0086] When the first feature information is a vector, the dimension of the vector can be determined according to the first sample input data. For example, when the first sample input data is a text sentence, each character corresponds to 2 dimensions in the vector. If the first sample input data includes 10 characters, the first feature information is a 20-dimensional vector. If the first sample input data includes 30 characters, the first feature information is a 60-dimensional vector, and so on.
[0087] When the first feature information is a matrix, the number of rows and columns of the matrix can be fixed. For example, the server inputs a sample text sentence including any number of words into the feature extraction network to obtain a 100*200 matrix, or inputs a sample image including any pixel value into the feature extraction network to obtain a 512*512 matrix, and so on.
[0088] The server may input the first feature information into the first processing network to obtain the first actual output data corresponding to the first sample input data. The server may input the first reference output data and the first actual output data into the loss function, calculate the corresponding loss value, adjust the parameters of the feature extraction network and the first processing network according to the loss value, and complete the first training of the first machine learning model.
[0089] S103: Based on the second sample input data and the second benchmark output data, perform combined training on the trained feature extraction network and the second processing network.
[0090] The server can input the acquired second sample input data into the trained feature extraction network (after completing the first training of the first machine learning model) to obtain the second feature information corresponding to the second sample input data. The second feature information can be a vector or a matrix, and the specific form can be the same as the first feature information, which will not be repeated here.
[0091] The server may input the second feature information into the second processing network to obtain the second actual output data corresponding to the second sample input data. The server may input the second reference output data and the second actual output data into the loss function, calculate the corresponding loss value, and adjust the parameters of the trained feature extraction network (after one training) and the second processing network according to the loss value to complete the first training of the second machine learning model.
[0092] After the server completes the first training of the first machine learning model and the first training of the second machine learning model, it can obtain the fourth sample input data and the corresponding fourth benchmark output data, the fifth sample input data and the corresponding fifth benchmark output data, and the fourth sample input data and the fifth sample input data can have the same data type as the first sample input data.
[0093] The server may input the fourth sample input data into the feature extraction network after the second training (after completing the first training of the second machine learning model) to obtain the corresponding fourth feature information. The server may input the fourth feature information into the trained first processing network to obtain the fourth actual output data corresponding to the fourth sample input data. The server may input the fourth reference output data and the fourth actual output data into the loss function, calculate the corresponding loss value, and adjust the parameters of the trained feature extraction network (after two trainings) and the trained first processing network (after one training) according to the loss value to complete the second training of the first machine learning model.
[0094] After the server completes the second training of the first machine learning model, the server can input the fifth sample input data into the feature extraction network after three trainings (after completing the second training of the first machine learning model) to obtain the corresponding fifth feature information. The server can input the fifth feature information into the second processing network to obtain the fifth actual output data corresponding to the fifth sample input data. The server can input the fifth benchmark output data and the fifth actual output data into the loss function, calculate the corresponding loss value, and adjust the parameters of the trained feature extraction network (after three trainings) and the second processing network (after one training) according to the loss value to complete the second training of the second machine learning model.
[0095] By analogy, based on a large amount of sample input data and corresponding benchmark output data, the server can repeatedly perform alternating training of the first machine learning model and the second machine learning model. The server can use the number of training times as a condition for the end of training, or it can use the model accuracy as a condition for the end of training. For example, after the first machine learning model and the second machine learning model are trained 1,000 times respectively, the training can be stopped, or, in the 100 trainings of the first machine learning model and the second machine learning model, the accuracy of the first machine learning model and the second machine learning model both reach 95%, the training can be stopped, and so on.
[0096] The above content introduces the case of training two models (a first machine learning model and a second machine learning model). This method can also be applied to the training of more than two models, that is, it can also include a third machine learning model, a fourth machine learning model, and so on. Next, the method of training the model can also be used to train the third machine learning model as an example. Other cases are similar and will not be described in detail.
[0097] First, in step S101, the server may also obtain third sample input data and corresponding third reference output data, and the third sample input data may have the same data type as the first sample input data and the second sample input data.
[0098] Secondly, after the server completes the first training of the second machine learning model, the third sample input data can be input into the feature extraction network after the second training (the first training of the second machine learning model is completed) to obtain the corresponding third feature information. The form of the third feature information can be the same as the first feature information, and will not be repeated.
[0099] Then, the server may input the third feature information into the third processing network to obtain third actual output data corresponding to the third sample input data.
[0100] Finally, the server can input the third benchmark output data and the third actual output data into the loss function, calculate the corresponding loss value, and adjust the parameters of the feature extraction network (after two trainings) and the third processing network after the second training according to the loss value to complete the first training of the third machine learning model.
[0101] After completing the first training of the third machine learning model, the server may repeat the training of the first machine learning model, the second machine learning model, and the third machine learning model multiple times until a preset training stop condition is met. The training stop condition is similar to that mentioned in step S103 and will not be repeated here.
[0102] The above is an overall introduction to the model training method provided by this application. Next, the embodiments of this application will be based on the semantic relevance model, semantic Figure 1 The consistency model and word category recognition model are taken as examples to explain in detail. Other cases are similar and will not be described in detail.
[0103] The semantic relevance model may include a feature extraction network and a semantic relevance network (ie, a first processing network), and the corresponding sample input data may include a first sample text sentence and a second sample text sentence. Figure 1 The consistency model can include feature extraction network and semantic Figure 1 The consistency network (i.e., the second processing network), and the corresponding sample input data may include a third sample text sentence and a fourth sample text sentence. The word category recognition model may include a feature extraction network and a word category recognition network (i.e., the third processing network), and the corresponding sample input data may include a fifth sample text sentence.
[0104] Developers can pre-establish an intent classification system. The intent classification system can include multiple levels, and each level can include multiple intents. For example, the intent classification system is as follows: Figure 2As shown, the intent classification system is divided into two levels. The first level includes 9 intents, including drugs / equipment, treatment / surgery / plan, examination and testing, symptoms, diseases, hospitals / departments, doctors, fees / medical insurance / social security and others. The second level includes definition explanation, content description, judgment basis, how to deal with and others, totaling 5 intents. In the intent classification system, the number of levels and the categories of intents included in each level can be determined according to actual conditions. In the server, the identification information corresponding to each intent in the intent classification system can be pre-stored. For example, the identification information corresponding to the first-level intent "disease" is "JB_1", and the identification information corresponding to the second-level intent "content description" is "NRMS_2", and so on. The server determines the intent Figure 1 When comparing the benchmark output data and actual output data corresponding to the consistency model, the intent classification system can be used as a basis.
[0105] The present application embodiment provides a semantic correlation model and a semantic Figure 1 The model training method for training the consistent model is as follows: Figure 3 As shown, the following processing steps are included:
[0106] S201, obtaining first sample input data and corresponding semantic relevance benchmark values (i.e., first benchmark output data), second sample input data and corresponding semantic relevance benchmark values. Figure 1 The consistency reference value (ie, the second reference output data) is set.
[0107] The first sample input data and the second sample input data have the same data type. The data type may be text data, and the second sample input data and the first sample input data may have the same or different contents. For example, "Which acidic foods give birth to a girl" and "What acidic foods should be eaten to give birth to a daughter" are both the first sample input data and the second sample input data, or "Which acidic foods give birth to a girl" and "What acidic foods should be eaten to give birth to a daughter" are the first sample input data, and "What is diabetes" and "The definition of diabetes" are the second sample input data.
[0108] When the server starts training the model, it needs to determine the sample input data and corresponding benchmark output data of the model to be trained.
[0109] The first sample input data may include a first sample text sentence and a second sample text sentence, and the corresponding semantic relevance benchmark value may be the relevance between the two sample text sentences. There are many ways to express relevance, for example, expressed as a percentage, 0 (i.e., 0%) represents completely irrelevant, 100% represents very relevant, any percentage between 0 and 100% represents the degree of relevance between the two sample text sentences, for another example, expressed as a number, 0 represents irrelevant, 3 represents relatively irrelevant, 6 represents generally relevant, 8 represents relatively relevant, 10 represents very relevant, for another example, expressed as a character string, XG-1 represents irrelevant, XG-2 represents generally relevant, and XG-3 represents relevant. Other expression methods are similar to the above and will not be repeated.
[0110] The second sample input data may include a third sample text sentence and a fourth sample text sentence, and the corresponding meanings are Figure 1 The consistency benchmark value can be the meaning between these two sample text sentences. Figure 1 Consistency. Figure 1 Consistency can be expressed in two ways: Figure 1 Consistent intent is marked as "1", inconsistent intent is marked as "0". Figure 1 Consistency can also be expressed as a string, for example, Figure 1 The mark for the intention is "YZ", and the mark for the intention is "BYZ". Other representation methods are similar to the above and will not be repeated here.
[0111] For different models to be trained, the server can select sample input data and benchmark output data corresponding to the models to be trained in the sample library. For example, the first sample input data obtained by the server is "What acidic foods give birth to girls" and "What acidic foods should be eaten to give birth to daughters", and the semantic relevance benchmark value is "8". For another example, the second sample input data obtained by the server is "What is diabetes" and "The definition of diabetes", and the semantic relevance benchmark value is "8". Figure 1 The consistency benchmark value is "1", and so on.
[0112] S202: Based on the first sample input data and the semantic relevance benchmark value, perform combined training on the feature extraction network and the semantic relevance network.
[0113] The server may input the first sample text sentence and the second sample text sentence into a feature extraction network (such as a BERT network, etc.) to obtain corresponding semantic feature information. The semantic feature information may be in the form of a vector or a matrix. The form of the semantic feature information is the same as that in step S102 and will not be described in detail. The process of determining the semantic feature information may be as follows:
[0114] First, the server can determine the word vector corresponding to each word in the first sample text sentence and the second sample text sentence. The dimension of each word vector can be different. For example, the first sample text sentence is "What acidic food should I eat to give birth to a daughter", where "birth" corresponds to a 20-dimensional word vector, "acid" corresponds to a 30-dimensional word vector, and so on. The dimension of each word vector can be pre-set according to the needs of the training process, and the dimension of each word vector can be the same, for example, 64 dimensions, 128 dimensions, 400 dimensions, 768 dimensions, and so on.
[0115] Secondly, the server can splice the word vectors corresponding to each word belonging to the same sample text sentence. For example, the first sample text sentence is "What acidic food should I eat if I give birth to a daughter?" The word vector corresponding to each word is determined through the feature extraction network (each word corresponds to a 10-dimensional vector), and the 10 word vectors corresponding to the sample text sentence are spliced to obtain a 100-dimensional sentence vector, and so on.
[0116] Finally, the server can process the sentence vector corresponding to the first sample text sentence and the sentence vector corresponding to the second sample text sentence to obtain semantic feature information corresponding to the first sample input data. The processing method can be to concatenate the two sentence vectors into one sentence vector, or to determine a sentence matrix based on the two sentence vectors, etc.
[0117] Optionally, when the semantic feature information is a vector, the dimension of the vector may be fixed. For example, the first sample text sentence and the second sample text sentence each include 10 characters, and the corresponding semantic feature information is a 64-dimensional vector. The first sample text sentence and the second sample text sentence each include 30 characters, and the corresponding semantic feature information is also a 64-dimensional vector, etc. In this case, if the dimension of the vector corresponding to the first sample input data is less than the preset dimension threshold, it can be supplemented with "0". For example, if the dimension of the vector corresponding to the first sample input data is 60 dimensions, and the dimension of the preset semantic feature information is 64 dimensions, it is necessary to add 4-dimensional "0"s at the end of the above vector to obtain a 64-dimensional vector, and so on.
[0118] Optionally, the server may obtain a sentence matrix corresponding to the sample text sentence based on each word vector belonging to the same sample text sentence. For example, the first sample text sentence is "What acidic foods should I eat to give birth to a daughter", each word corresponds to a 768-dimensional word vector, and each word vector is used as a row according to the order of each word in the first sample text sentence, that is, a sentence matrix of 10 (rows) * 768 (columns) is determined, and so on. The server may obtain a total sentence matrix as semantic feature information based on the sentence matrix corresponding to the first sample text sentence and the sentence matrix corresponding to the second sample text sentence.
[0119] Optionally, the server may determine the word matrix corresponding to each word in the first sample text sentence and the second sample text sentence, and the number of rows and columns of each word matrix may be fixed, for example, the word matrix corresponding to each word in the sample text sentence is a 3*3 matrix, or the word matrix corresponding to each word in the sample text sentence is a 64*128 matrix, or the word matrix corresponding to each word in the sample text sentence is a 256*128 matrix, and so on. The processor may perform corresponding operations (such as concatenation, addition, etc.) on each word matrix belonging to the same sample text sentence to obtain a sentence matrix corresponding to the sample text sentence. The server may obtain a total sentence matrix as semantic feature information based on the sentence matrix corresponding to the first sample text sentence and the sentence matrix corresponding to the second sample text sentence.
[0120] The server can input the semantic feature information into the semantic relevance network to obtain the semantic relevance prediction value corresponding to the first sample input data. For example, the first sample input data is "Which acidic foods give birth to a girl" and "What acidic foods should be eaten to give birth to a daughter", and the semantic feature information is a 200-dimensional feature vector. The server inputs this 200-dimensional feature vector into the semantic relevance network and obtains the semantic relevance prediction value "8".
[0121] The server can input the semantic relevance baseline value and the semantic relevance prediction value into the loss function, calculate the corresponding loss value, adjust the parameters of the feature extraction network and the semantic relevance network according to the loss value, and complete the first training of the semantic relevance model. For example, the first sample input data is "Which acidic foods give birth to a girl" and "What acidic foods should be eaten to give birth to a daughter", the semantic relevance baseline value is "9", and the semantic relevance prediction value is "8". The server inputs "9" and "8" into the loss function, determines the corresponding loss value, and further adjusts the semantic relevance model (i.e., the feature extraction network and the semantic relevance network) through the loss value. Once the parameter adjustment is completed, the first training of the semantic relevance model is completed.
[0122] S203, based on the second sample input data and the Figure 1 The consistency benchmark value is used to compare the trained feature extraction network and the Figure 1 The consistent network is trained in combination.
[0123] The server can input the third sample text sentence and the fourth sample text sentence into the trained feature extraction network (after completing the first training of the semantic relevance model) to obtain the corresponding intention feature information. The intention feature information can be in the form of a vector or a matrix. The form of the intention feature information is the same as that in step S102 and will not be repeated here. The process of determining the intention feature information can be similar to the process of determining the semantic feature information and will not be repeated here.
[0124] The server can input the intent feature information corresponding to each sample sentence into the intent Figure 1 The consistency network is used to obtain the meaning corresponding to the second sample input data. Figure 1 Prediction value of consistency.
[0125] meaning Figure 1 The process of determining the consistency prediction value can be as follows:
[0126] First, the server can input the intent feature information into the intent Figure 1 In the consistency network, the identification information of all levels of intent corresponding to the third sample text sentence and the identification information of all levels of intent to which the fourth sample text sentence belongs can be determined. For example, the third sample text sentence is "What is diabetes?" Figure 2 In the intent classification system shown, the intent identification information corresponding to the sample text sentence is determined to be "JB_1" and "DYJS_2", and so on.
[0127] If the intent classification system also has three-level intent and four-level intent, then the intents of each level corresponding to the sample text sentence are determined in turn, which will not be repeated here.
[0128] Then, the server can determine the intent of each sample text sentence based on all the levels of intent. Figure 1 If the intention identification information corresponding to the third sample text is exactly the same as the intention identification information corresponding to the fourth sample text sentence, then the intention of the third sample text sentence and the fourth sample text sentence is determined to be Figure 1 To Figure 1 The consistency prediction value can be represented by "1". If the intention identification information corresponding to the third sample text is not completely the same as or completely different from the intention identification information corresponding to the fourth sample text sentence, it is determined that the intentions of the third sample text sentence and the fourth sample text sentence are inconsistent, and the intentions are not consistent. Figure 1 The predicted consistency value can be represented by "0".
[0129] The server can Figure 1 Consistency benchmarks and Figure 1 The consistent prediction value is input into the loss function, the corresponding loss value is calculated, and the trained feature extraction network (after one training) and the intention Figure 1 The consistent network is used to adjust parameters and complete the Figure 1 First training of the consistency model.
[0130] The server completes the first training and semantic relevance model Figure 1 After the first training of the consistency model, you can continue to train the semantic relevance model and the semantic Figure 1 The consistent model is trained multiple times, such as Figure 4As shown. When the server detects that the two models meet the preset training end conditions, the training ends. The process of multiple trainings and the conditions for the end of training have been described above and will not be repeated here.
[0131] The model training method provided in the embodiment of the present application can be used to train semantic relevance models and semantic Figure 1 The semantic relevance model is trained together with the semantic consistency model. Figure 1 The semantic relevance models assist each other. When determining the semantic relevance between two input text sentences, the trained semantic relevance model can not only consider the literal meaning of the two text sentences, but also add the judgment of the deep intention to be expressed by the text sentences, thereby improving the accuracy of the trained semantic relevance model.
[0132] Developers can predetermine a word category system, which can include several word categories, each of which can include several words. The word categories can be determined according to the specific needs of model training. For example, when training a model in the medical field, the word categories predetermined by the developer can be as follows: Figure 5 As shown, it includes 18 categories including symptoms, population, disease, part, intention word, medicine, action, qualifier, food, indicator, surgical plan, department, location, result, equipment, hospital, name, treatment plan, etc. In the server, identification information corresponding to each word category can be pre-stored, and the identification information can be a number or a string, etc. For example, the identification information corresponding to "population" is "RQ", and the identification information corresponding to "disease" is "JB", or the identification information corresponding to "population" is "1", and the identification information corresponding to "disease" is "3", and so on. When determining the actual output data and benchmark output data corresponding to the word category recognition model, the server can use the above-mentioned word categories as a basis.
[0133] The present application embodiment provides a model training method for training a semantic correlation model and a word category recognition model, and the processing flow is as follows: Figure 6 As shown, the following processing steps are included:
[0134] S301, obtaining first sample input data and corresponding semantic relevance benchmark values (ie, first benchmark output data), second sample input data and corresponding word category recognition benchmark values (ie, second benchmark output data).
[0135] The first sample input data and the second sample input data have the same data type. The data type may be text data, for example, the first sample input data is "What acidic foods are good for giving birth to a girl" and "What acidic foods should be eaten for giving birth to a daughter", and the second sample input data is "How to treat a baby's stomachache".
[0136] When the server starts training the model, it needs to determine the sample input data and corresponding benchmark output data of the model to be trained.
[0137] The first sample input data may include a first sample text sentence and a second sample text sentence, and the corresponding semantic relevance reference value may be the relevance between the two sample text sentences. The relevance representation has been described above and will not be repeated here.
[0138] The first sample input data may include a fifth sample text sentence, and the corresponding word category identification reference value may be identification information corresponding to the word category to which the word included in the fifth sample text sentence belongs. The word category identification information is as described above and will not be repeated here.
[0139] S302: Based on the first sample input data and the semantic relevance benchmark value, perform combined training on the feature extraction network and the semantic relevance network.
[0140] First, the server can input the first sample input data into the feature extraction network to obtain semantic feature information corresponding to the first sample input data. Secondly, the server can input the semantic feature information into the semantic processing network to obtain a semantic relevance prediction value corresponding to the first sample input data. Finally, the server can adjust the feature extraction network and the semantic relevance network based on the semantic relevance baseline value and the semantic relevance prediction value. The specific processing process is the same as step S202 and will not be repeated here.
[0141] S303: Based on the second sample input data and the word category recognition benchmark value, the trained feature extraction network and the word category recognition network are combined for training.
[0142] The server can input the fifth sample text sentence into the trained (first training of the semantic relevance model) feature extraction network to obtain word feature information corresponding to the fifth sample text sentence. The word feature information can be in the form of a vector or a matrix. The form and determination process of the word feature information can be similar to the form and determination process of the semantic feature information, and will not be repeated here.
[0143] The server may input the word feature information into the word category recognition network to determine the word category recognition prediction value corresponding to the fifth sample text sentence.
[0144] The process of determining the predicted value of word category recognition can be as follows:
[0145] First, the server can determine all the words included in the fifth sample text sentence based on the word feature information. For example, the fifth sample text sentence is "How to treat baby's stomachache", and the words included in this sample sentence are "baby", "stomachache", "how" and "treatment".
[0146] Then, the server can determine the identification information corresponding to the word category to which each word belongs, that is, determine the word category to which each word belongs. For example, the string corresponding to "baby" is "RQ", the string corresponding to "stomachache" is "ZZ", the string corresponding to "how" is "Na (does not belong to any word category)", the string corresponding to "treatment" is "YTC", and so on.
[0147] Finally, the server may output all identification information in sequence as a word category recognition prediction value corresponding to the fifth sample text sentence. For example, the actual output data corresponding to “How to treat baby's stomach ache” is “RQ ZZ Na YTC”, and so on.
[0148] The server can input the word category recognition benchmark value and the word category recognition prediction value into the loss function, calculate the corresponding loss value, and adjust the parameters of the trained feature extraction network (after one training) and the word category recognition network according to the loss value to complete the first training of the word category recognition model.
[0149] After the server completes the first training of the semantic relevance model and the first training of the word category recognition model, it can continue to train the semantic relevance model and the word category recognition model multiple times, such as Figure 7 As shown. When the server detects that the two models meet the preset training end conditions, the training ends. The process of multiple trainings and the conditions for the end of training have been described above and will not be repeated here.
[0150] Using the model training method provided in the embodiment of the present application, the semantic relevance model and the word category recognition model can be trained together. During training, the semantic relevance model and the word category recognition model assist each other. When determining the semantic relevance between two input text sentences, the trained semantic relevance model can not only consider the literal meaning of the two text sentences, but also strengthen the importance of the keywords included in the text sentences in determining the semantic relevance, thereby improving the accuracy of the trained semantic relevance model.
[0151] The present application embodiment provides a semantic correlation model, a semantic Figure 1 The model training method for training the consistency model and the word category recognition model is as follows: Figure 8 As shown, the following processing steps are included:
[0152] S401, obtaining first sample input data and corresponding semantic relevance benchmark values (i.e., first benchmark output data), second sample input data and corresponding semantic relevance benchmark values. Figure 1 The consistency benchmark value (ie, the second benchmark output data), the third sample input data and the corresponding word category recognition benchmark value (ie, the third benchmark output data).
[0153] The first sample input data, the second sample input data and the third sample input data may have the same data type, which may be text data, which is the same as steps S201 and S301 and will not be described in detail here.
[0154] S402: Based on the first sample input data and the semantic relevance benchmark value, perform combined training on the feature extraction network and the semantic relevance network.
[0155] First, the server can input the first sample input data into the feature extraction network to obtain semantic feature information corresponding to the first sample input data. Secondly, the server can input the semantic feature information into the semantic processing network to obtain a semantic relevance prediction value corresponding to the first sample input data. Finally, the server can adjust the feature extraction network and the semantic relevance network based on the semantic relevance baseline value and the semantic relevance prediction value. The specific processing process is the same as step S202 and will not be repeated here.
[0156] S403, based on the second sample input data and the intention Figure 1 The consistency benchmark value is used to compare the trained feature extraction network and the Figure 1 The consistent network is trained in combination.
[0157] First, the server can input the second sample input data into the trained feature extraction network (after completing the first training of the semantic relevance model) to obtain the intent feature information corresponding to the second sample input data. Figure 1 The consistency network is used to obtain the meaning corresponding to the second sample input data. Figure 1 Finally, the server can Figure 1 Consistency benchmarks and Figure 1 Consistency prediction value, the trained feature extraction network (after one training) and the intention Figure 1 The specific processing process is the same as step S203 and will not be repeated here.
[0158] S404: Based on the third sample input data and the word category recognition benchmark value, the feature extraction network and the word category recognition network after the second training are combined for training.
[0159] First, the server can input the third sample into the data, and after inputting the second training (complete the meaning Figure 1 The server may input the word feature information into the word category recognition network to obtain the word category recognition prediction value corresponding to the third sample input data. Finally, the server may adjust the parameters of the feature extraction network (after two trainings) and the word category recognition network after the second training based on the word category recognition benchmark value and the word category recognition prediction value. The specific processing process is the same as step S303 and will not be repeated here.
[0160] The server completes the semantic relevance model, Figure 1 After the first training of the consistency model and word category recognition model, you can continue to train the semantic relevance model, semantic Figure 1 The consistency model and word category recognition model are trained multiple times, such as Fig. 9 As shown. When the server detects that the two models meet the preset training end conditions, the training ends. The process of multiple trainings and the conditions for the end of training have been described above and will not be repeated here.
[0161] The model training method provided in the embodiment of the present application can be used to train semantic relevance models, semantic Figure 1 The semantic consistency model and word category recognition model are trained together. During training, the semantic relevance model, semantic Figure 1 The consistency model and the word category recognition model assist each other. When determining the semantic relevance between two input text sentences, the trained semantic relevance model can not only consider the literal meaning of the two text sentences, but also add the judgment of the true intention to be expressed by the text sentences, and can also strengthen the importance of the keywords included in the text sentences in determining the semantic relevance, thereby improving the accuracy of the trained semantic relevance model.
[0162] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0163] The scheme mentioned in the embodiment of the present application, first, can complete the training of the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data. Then, the training of the trained feature extraction network and the second processing network can be completed based on the second sample input data and the corresponding second benchmark output data. Using this method, a feature extraction network can be shared for multiple machine learning models with a certain degree of processing relevance, that is, the feature extraction network and the first processing network constitute the first machine learning model, and the feature extraction network and the second processing network constitute the second machine learning model. In this way, when the number of samples calibrated for the first machine learning model and the number of samples calibrated for the second machine learning model are relatively limited, through the above method, the training process of the first machine learning model and the training process of the second machine learning model can both train and optimize the feature extraction network, that is, the training process of the first machine learning model will also play a training role for the second machine learning model, and the training process of the second machine learning model will also play a training role for the first machine learning model. Thus, in the case of a limited number of samples, the accuracy of the machine learning model can be better improved through training.
[0164] The present application embodiment provides a device for training a model, which may be the server in the above embodiment, such as Fig.10 As shown, the device comprises:
[0165] An acquisition module 1010 is used to acquire first sample input data and corresponding first reference output data, second sample input data and corresponding second reference output data, wherein the first sample input data and the second sample input data have the same data type;
[0166] The training module 1020 is used to perform combined training on the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data; and to perform combined training on the trained feature extraction network and the second processing network based on the second sample input data and the second benchmark output data.
[0167] In a possible implementation, the training module 1020 is used to:
[0168] Inputting the first sample input data into a feature extraction network to obtain first feature information corresponding to the first sample input data;
[0169] Inputting the first feature information into the first processing network to obtain first actual output data corresponding to the first sample input data;
[0170] Adjusting parameters of a feature extraction network and a first processing network based on the first benchmark output data and the first actual output data;
[0171] Inputting the second sample input data into the trained feature extraction network to obtain second feature information corresponding to the second sample input data;
[0172] Inputting the second feature information into the second processing network to obtain second actual output data corresponding to the second sample input data;
[0173] Based on the second benchmark output data and the second actual output data, the trained feature extraction network and the second processing network are adjusted.
[0174] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the first processing network includes a semantic correlation network, and the second processing network includes a semantic correlation network. Figure 1 or
[0175] The first sample input data includes two sample sentences, the third sample input data includes one sample sentence, the first processing network includes a semantic correlation network, and the third processing network includes a word category recognition network.
[0176] In one possible implementation,
[0177] The acquisition module 1010 is further configured to acquire third sample input data and corresponding third reference output data, wherein the third sample input data and the first sample input data have the same data type;
[0178] The training module 1020 is further used to perform combined training on the feature extraction network and the third processing network after the second training based on the third sample input data and the third benchmark output data.
[0179] In a possible implementation, the training module 1020 is used to:
[0180] Inputting the third sample input data into the feature extraction network after the second training to determine third feature information corresponding to the third sample input data;
[0181] Inputting the third feature information into a third processing network to determine third actual output data corresponding to the third sample input data;
[0182] Based on the third benchmark output data and the third actual output data, the parameters of the feature extraction network and the third processing network after the second training are adjusted.
[0183] In a possible implementation, the first sample input data includes two sample sentences, the second sample input data includes two sample sentences, the third sample input data includes one sample sentence, the first processing network includes a semantic correlation network, the second processing network includes a semantic correlation network, Figure 1The consistency network, the third processing network includes a word category recognition network.
[0184] The scheme mentioned in the embodiment of the present application, first, can complete the training of the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data. Then, the training of the trained feature extraction network and the second processing network can be completed based on the second sample input data and the corresponding second benchmark output data. Using this method, a feature extraction network can be shared for multiple machine learning models with a certain degree of processing relevance, that is, the feature extraction network and the first processing network constitute the first machine learning model, and the feature extraction network and the second processing network constitute the second machine learning model. In this way, when the number of samples calibrated for the first machine learning model and the number of samples calibrated for the second machine learning model are relatively limited, through the above method, the training process of the first machine learning model and the training process of the second machine learning model can both train and optimize the feature extraction network, that is, the training process of the first machine learning model will also play a training role for the second machine learning model, and the training process of the second machine learning model will also play a training role for the first machine learning model. Thus, in the case of a limited number of samples, the accuracy of the machine learning model can be better improved through training.
[0185] It should be noted that: the model training device provided in the above embodiment only uses the division of the above functional modules as an example during model training. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the model training device provided in the above embodiment and the model training method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0186] An embodiment of the present application provides a computer device, which may be the server in the above embodiment. Fig.11 1 is a schematic diagram of the structure of the computer device. The computer device 1100 may have relatively large differences due to different configurations or performances, and may include one or more CPUs (Central Processing Units, processors) 1110 and one or more memories 1120, wherein the memory 1120 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 1110 to implement the methods provided by the above-mentioned various method embodiments. Of course, the computer device may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The computer device may also include other components for implementing device functions, which will not be described in detail here.
[0187] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the method of model training in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.
[0188] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0189] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A model training method, It is characterized in that The model is applied in the medical field, and the method comprises: Obtaining first sample input data and corresponding first benchmark output data, second sample input data and corresponding second benchmark output data, wherein the first sample input data and the second sample input data have the same data type, and the second benchmark output data is determined based on an intent classification system, wherein the intent classification system includes multiple levels, and each level includes multiple intents; Based on the first sample input data and the first benchmark output data, performing combined training on a feature extraction network and a first processing network, the first processing network comprising a semantic relevance network; Input the second sample input data into a feature extraction network that has been trained once to obtain corresponding intent feature information, wherein the second sample input data includes a third sample text sentence and a fourth sample text sentence; input the intent feature information into a second processing network, and based on the identification information corresponding to each intent in the pre-stored intent classification system, determine the identification information of all levels of intent corresponding to the third sample text sentence and the identification information of all levels of intent to which the fourth sample text sentence belongs, wherein the second processing network includes an intent consistency network; determine second actual output data according to all levels of intent corresponding to each sample text sentence, wherein the second actual output data is an intent consistency prediction value; Based on the second sample input data and the second actual output data, the parameters of the feature extraction network and the intention consistency network after one training are adjusted.
2. The method according to claim 1, It is characterized in that The combining training of the feature extraction network and the first processing network based on the first sample input data and the first benchmark output data includes: Input the first sample input data into a feature extraction network to obtain first feature information corresponding to the first sample input data; input the first feature information into a first processing network to obtain first actual output data corresponding to the first sample input data; and adjust the parameters of the feature extraction network and the first processing network based on the first benchmark output data and the first actual output data.
3. The method according to claim 1, It is characterized in that The first sample input data includes two sample sentences.
4. The method according to claim 1, It is characterized in that The method further comprises: Acquire third sample input data and corresponding third reference output data, wherein the third sample input data and the first sample input data have the same data type; Based on the third sample input data and the third benchmark output data, the feature extraction network and the third processing network after the second training are combined for training.
5. The method according to claim 4, It is characterized in that The combining training of the secondarily trained feature extraction network and the third processing network based on the third sample input data and the third benchmark output data comprises: Inputting the third sample input data into the feature extraction network after secondary training to determine third feature information corresponding to the third sample input data; Inputting the third feature information into the third processing network to determine third actual output data corresponding to the third sample input data; Based on the third benchmark output data and the third actual output data, the parameters of the feature extraction network and the third processing network after the second training are adjusted.
6. The method according to claim 4, It is characterized in that The first sample input data includes two sample sentences, the third sample input data includes one sample sentence, and the third processing network includes a word category recognition network.
7. A device for model training, It is characterized in that The model is applied in the medical field, and the device comprises: an acquisition module, configured to acquire first sample input data and corresponding first benchmark output data, second sample input data and corresponding second benchmark output data, wherein the first sample input data and the second sample input data have the same data type, and the second benchmark output data is determined based on an intent classification system, wherein the intent classification system includes multiple levels, and each level includes multiple intents; A training module is used to perform combined training on a feature extraction network and a first processing network based on the first sample input data and the first benchmark output data, wherein the first processing network includes a semantic relevance network; the second sample input data is input into the feature extraction network after one training to obtain corresponding intent feature information, wherein the second sample input data includes a third sample text sentence and a fourth sample text sentence; the intent feature information is input into a second processing network, and based on the identification information corresponding to each intent in the pre-stored intent classification system, the identification information of all levels of intent corresponding to the third sample text sentence and the identification information of all levels of intent to which the fourth sample text sentence belongs are determined, wherein the second processing network includes an intent consistency network; according to all levels of intent corresponding to each sample text sentence, second actual output data is determined, wherein the second actual output data is an intent consistency prediction value; based on the second sample input data and the second actual output data, the parameters of the feature extraction network and the intent consistency network after one training are adjusted.
8. The device according to claim 7, It is characterized in that The training module is used to: Inputting the first sample input data into a feature extraction network to obtain first feature information corresponding to the first sample input data; inputting the first feature information into a first processing network to obtain first actual output data corresponding to the first sample input data; Based on the first benchmark output data and the first actual output data, the feature extraction network and the first processing network are adjusted.
9. The device according to claim 7, It is characterized in that The first sample input data includes two sample sentences.
10. The device according to claim 7, It is characterized in that The acquisition module is further used to acquire third sample input data and corresponding third reference output data, wherein the third sample input data and the first sample input data have the same data type; The training module is further used to perform combined training on the feature extraction network and the third processing network after secondary training based on the third sample input data and the third benchmark output data.
11. The device according to claim 10, It is characterized in that The training module is used to: Inputting the third sample input data into the feature extraction network after secondary training to determine third feature information corresponding to the third sample input data; Inputting the third feature information into the third processing network to determine third actual output data corresponding to the third sample input data; Based on the third benchmark output data and the third actual output data, the parameters of the feature extraction network and the third processing network after the second training are adjusted.
12. The device according to claim 10, It is characterized in that The first sample input data includes two sample sentences, the third sample input data includes one sample sentence, and the third processing network includes a word category recognition network.
13. A computer device, It is characterized in that The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the model training method according to any one of claims 1 to 6.
14. A computer-readable storage medium, It is characterized in that The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the model training method as described in any one of claims 1 to 6.
15. A computer program product, It is characterized in that The computer program product includes a computer program code. When the computer program code is executed by a computer device, the computer device executes the model training method described in any one of claims 1 to 6.
