Neural network training and deployment methods, text translation methods and related products

Abstract

The present application provides a neural network training and deployment method, a text translation method and related products, wherein the present application first obtains the target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information used to represent the carrying capacity of the target device; then based on the deployment requirement information, the neural network to be deployed on the target device is obtained from the trained neural network; finally, the neural network to be deployed is deployed on the target device. The present application can select suitable neural networks for deployment from the target neural network and the various sub-neural networks included in the target neural network according to the carrying capacity of the target device, avoiding the defect of training the neural network multiple times according to the carrying capacity of different devices, improving the training and deployment efficiency of the neural network, and saving training resources.

Description

Technical Field

[0001] The present application relates to the field of computer and deep learning technology, and more specifically, to a neural network training and deployment method, a text translation method, and related products. Background Art

[0002] With the development of deep learning technology, various neural network structures have played a key role in tasks such as image processing and natural language processing, and have demonstrated superior performance. However, due to the different computing power or carrying capacity of the devices deploying neural networks in different usage scenarios, for neural networks with the same function, it is necessary to train neural networks of different sizes according to the computing power of different devices. After the training is completed, the trained neural networks are deployed on matching devices.

[0003] In the above training method, the parameters of each neural network cannot be shared, the training efficiency is low, and the training resources are wasted. Summary of the invention

[0004] The embodiments of the present application at least provide a neural network training and deployment method to improve the training efficiency of neural networks of different scales.

[0005] In a first aspect, an embodiment of the present application provides a neural network deployment method, comprising:

[0006] Acquire a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information for indicating the carrying capacity of the target device;

[0007] Based on the deployment requirement information, obtaining a neural network to be deployed on the target device from the trained neural network;

[0008] Deploy the neural network to be deployed on the target device.

[0009] In this regard, it is possible to select a suitable neural network for deployment from the target neural network and the various sub-neural networks included in the target neural network according to the carrying capacity of the target device, which means that there is no need to train the neural network multiple times according to the carrying capacity of different devices to meet the deployment requirements of different devices for various neural networks. Instead, the trained target neural network and the multiple sub-neural networks included in the target neural network itself are used to determine the neural network that meets the deployment requirements (which may be the target neural network itself, and / or one or more sub-neural networks included in the target neural network) for deployment and application, which saves the resources consumed in training different neural networks and improves the deployment efficiency of neural networks.

[0010] In a possible implementation, the deployment requirement information includes at least one of the following:

[0011] The computing capability information of the target device; and the network speed requirement information.

[0012] In this implementation, the neural network to be deployed suitable for the target device can be screened out according to the computing power information of the target device, ensuring that the neural network to be deployed can be used normally after being deployed on the target device. The neural network to be deployed that meets the computing speed requirement can be screened out according to the network speed requirement information.

[0013] In a possible implementation, before acquiring the target neural network, the method further includes:

[0014] Extracting multiple sub-neural networks from the neural network to be trained;

[0015] Using a first real training set to train the neural network to be trained to obtain a first neural network, and using the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network;

[0016] Inputting a plurality of real samples into the first neural network for processing respectively, obtaining a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;

[0017] The first neural network is trained using the second real training set, and each first sub-neural network in a plurality of first sub-neural networks is trained respectively using the second real training set and the first predicted training set to obtain the target neural network.

[0018] This implementation uses the first predicted training set formed by the output of the first neural network in the middle state of neural network training to train the neural network to be trained and each sub-neural network extracted from the neural network to be trained, so that each sub-neural network and the neural network to be trained can promote each other and realize parameter sharing between the neural network to be trained and each sub-neural network. The trained target neural network is trimmed, and the obtained sub-neural network can be deployed on a device with a corresponding carrying capacity, that is, a neural network suitable for different carrying capacities can be obtained through one training, which improves the training efficiency of the neural network and saves training resources.

[0019] In a possible implementation, the using the second real training set to train the first neural network, and using the second real training set and the first predicted training set to train each first sub-neural network in a plurality of first sub-neural networks respectively to obtain the target neural network includes:

[0020] Using a second real training set to train the first neural network to obtain a second neural network, and using the second real training set and the first predicted training set to train each first sub-neural network in a plurality of first sub-neural networks respectively to obtain a second sub-neural network corresponding to each first sub-neural network;

[0021] Acquire a third real training set, and generate a second predicted training set based on associated samples associated with the training samples in the third real training set;

[0022] The second neural network is trained using the second predicted training set, and each second sub-neural network in a plurality of second sub-neural networks is trained respectively using the third real training set and the second predicted training set to obtain the target neural network.

[0023] This implementation method, in the process of training the neural network to be trained and each sub-neural network, further combines the new real training set, i.e., the third real training set mentioned above, and the second predicted training set associated with the real training set for training, so as to better realize parameter sharing between the neural network to be trained and each sub-neural network.

[0024] In a possible implementation, the using the second real training set and the first predicted training set to train each first sub-neural network in the plurality of first sub-neural networks respectively includes:

[0025] Using the second real training set to train each first sub-neural network in the plurality of first sub-neural networks, respectively, to generate a first objective function corresponding to each first sub-neural network;

[0026] Using the first prediction training set to train each first sub-neural network in the plurality of first sub-neural networks, respectively, to generate a second objective function corresponding to each first sub-neural network;

[0027] For each first sub-neural network among the multiple first sub-neural networks, the first sub-neural network is trained based on a first objective function and a second objective function corresponding to the first sub-neural network.

[0028] In this implementation, the sub-neural network is trained together using the first objective function constructed from the first predicted training set and the second objective function constructed from the second real training set, which can facilitate parameter sharing between the neural network to be trained and the sub-neural network.

[0029] In a possible implementation, the training of the first sub-neural network based on the first objective function and the second objective function corresponding to the first sub-neural network includes:

[0030] Obtaining the number of iterations of respectively training each first sub-neural network in the plurality of first sub-neural networks using the second real training set to generate a first objective function corresponding to each first sub-neural network, and / or the number of iterations of respectively training each first sub-neural network in the plurality of first sub-neural networks using the first predicted training set to generate a second objective function corresponding to each first sub-neural network;

[0031] Based on the obtained number of iterations, determine the function adjustment ratio;

[0032] Generate a target training function based on the first target function, the second target function and the function adjustment ratio;

[0033] The first sub-neural network is trained using the target training function.

[0034] This implementation uses a function adjustment ratio determined based on the number of iterations to integrate the first objective function and the second objective function, and uses the integrated target training function to train the sub-neural network. This can reduce the impact of noise in the output of the first neural network on the neural network training and ensure the accuracy of the trained neural network. At the same time, using the target training function obtained by integrating the first objective function and the second objective function to train the sub-neural network can facilitate parameter sharing between the neural network to be trained and the sub-neural network.

[0035] In a possible implementation manner, the training samples in the third real training set include words to be translated and standard translation words;

[0036] The step of generating a second predicted training set based on associated samples associated with training samples in the third real training set comprises:

[0037] Determining a predicted word that appears after the word to be translated based on the word to be translated in the training sample and a plurality of texts including the word to be translated;

[0038] Combining the predicted vocabulary with the standard translation vocabulary to form an associated sample of the training sample;

[0039] The second predicted training set is generated by using the associated samples respectively corresponding to each training sample in the third real training set.

[0040] In this implementation, a prediction training set is constructed using predicted words that may appear after the words to be translated, and the prediction training set is used to train the first neural network and each first sub-neural network, which can promote parameter sharing between the neural network to be trained and the sub-neural networks.

[0041] In a second aspect, the present application provides a neural network training method, comprising:

[0042] Extracting multiple sub-neural networks from the neural network to be trained;

[0043] Using a first real training set to train the neural network to be trained to obtain a first neural network, and using the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network;

[0044] Inputting a plurality of real samples into the first neural network for processing respectively, obtaining a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results;

[0045] The first neural network is trained using a second real training set, and each first sub-neural network in a plurality of first sub-neural networks is trained using the second real training set and the first predicted training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are all trained neural networks.

[0046] In a third aspect, the present application provides a text translation method, the method comprising:

[0047] Get the text to be translated;

[0048] Translating the text to be translated by a translation neural network deployed on the target device to obtain a translation result;

[0049] Wherein, the translation neural network is the above-mentioned neural network to be deployed, and / or, the translation neural network includes the target neural network obtained by the above-mentioned neural network training method and / or at least one sub-neural network in the target neural network.

[0050] By using the translation neural network deployed on the target device to translate text, the translated text can be obtained more accurately.

[0051] In a fourth aspect, the present application provides a neural network deployment device, comprising:

[0052] An acquisition module, used to acquire a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information used to represent the carrying capacity of the target device;

[0053] A network screening module, configured to obtain a neural network to be deployed on the target device from the trained neural network based on the deployment requirement information;

[0054] The network deployment module is used to deploy the neural network to be deployed on the target device.

[0055] In a fifth aspect, the present application provides a neural network training device, comprising:

[0056] A sub-network extraction module, used to extract multiple sub-neural networks from the neural network to be trained;

[0057] A first training module is used to train the neural network to be trained using a first real training set to obtain a first neural network, and to train each of the multiple sub-neural networks using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network;

[0058] A sample prediction module, used to input a plurality of real samples into the first neural network for processing respectively, obtain a sample processing result corresponding to each real sample, and generate a first prediction training set based on the plurality of real samples and the plurality of obtained sample processing results;

[0059] The second training module is used to train the first neural network using a second real training set, and to train each first sub-neural network in a plurality of first sub-neural networks using the second real training set and the first predicted training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are all trained neural networks.

[0060] In a sixth aspect, the present application provides a text translation device, comprising:

[0061] A text acquisition module is used to acquire the text to be translated;

[0062] A translation module, used to translate the text to be translated through a translation neural network deployed on a target device to obtain a translation result;

[0063] Wherein, the translation neural network is the above-mentioned neural network to be deployed, and / or, the translation neural network includes the target neural network obtained by the above-mentioned neural network training method and / or at least one sub-neural network in the target neural network.

[0064] In the seventh aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, steps of the above-mentioned neural network deployment method, neural network training method, or text translation method are performed.

[0065] In an eighth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the above-mentioned neural network deployment method, or neural network training method, or text translation method.

[0066] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solutions of the present application. It should be understood that the following drawings only illustrate certain embodiments of the present application and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0068] Figure 1 A flowchart of a neural network deployment method provided in an embodiment of the present application is shown;

[0069] Figure 2 A flowchart of a neural network training method provided in an embodiment of the present application is shown;

[0070] Figure 3 A flowchart of a text translation method provided in an embodiment of the present application is shown;

[0071] Figure 4 A schematic diagram of the structure of a neural network deployment device provided in an embodiment of the present application is shown;

[0072] Figure 5 A schematic diagram of the structure of a neural network training device provided in an embodiment of the present application is shown;

[0073] Figure 6 A schematic diagram showing the structure of a text translation device provided in an embodiment of the present application is shown;

[0074] Figure 7 A schematic diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0076] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0077] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0078] In view of the defects of low efficiency and high resource consumption caused by the need to train neural networks for devices with different loads separately in the prior art, the present application provides a neural network deployment and training method, wherein the present application uses a predicted training set formed by the output of a real training set and a neural network in an intermediate state of training to jointly train the neural network to be trained and each sub-neural network extracted from the neural network to be trained, so that each sub-neural network and the neural network to be trained can promote each other and realize parameter sharing between the neural network to be trained and each sub-neural network. The trained target neural network is trimmed, and the obtained sub-neural network can be deployed on a device with a corresponding load capacity, that is, a device suitable for different load capacities can be obtained through one training, which improves the training and deployment efficiency of neural networks of different scales and saves training resources.

[0079] The neural network deployment and training method and related products provided by this application are described in detail below through specific embodiments.

[0080] The neural network deployment method provided in the present application is applied to select a neural network to be deployed from a trained target neural network based on information such as the carrying capacity of the target device, and deploy the selected neural network to be deployed on the target device. Specifically, Figure 1 As shown, the neural network deployment method provided by the present application may include the following steps:

[0081] S110, obtaining a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information for indicating the carrying capacity of the target device.

[0082] Here, the target neural network includes multiple sub-neural networks, and the sub-neural networks include some neural network layers and / or some neurons in the neural network layers in the target neural network. The target neural network has the largest network scale and requires equipment with a larger carrying capacity to carry it. The sub-neural networks have relatively small network scales and can be carried by equipment with a smaller carrying capacity. Parameters are shared between the target neural network and each sub-neural network. The above network scale corresponds to the structural complexity of the neural network. The larger the network scale, the more complex the structure of the neural network, the more neural network layers or neurons in the neural network layers are included, and the smaller the network scale, the simpler the structure of the neural network, and the fewer neural network layers or neurons in the neural network layers are included.

[0083] The above-mentioned information indicating the carrying capacity of the target device may specifically be the computing power information of the target device. In addition, the deployment requirement information may also include network speed requirement information, which indicates the information on the requirement for the computing speed of the neural network to be deployed. Different neural networks have different computing speeds due to different network scales. The larger the network scale of the neural network, that is, the more complex the structure of the neural network, the slower its computing speed. On the contrary, the smaller the network scale of the neural network, that is, the simpler the structure of the neural network, the faster its computing speed. When screening the neural network to be deployed, not only the carrying capacity of the target device can be considered, but also the requirement for the computing speed of the neural network to be deployed. When the computing speed requirement is not high and the equipment carrying capacity is strong, a neural network with a larger network scale can be selected. When a higher computing speed is required and the equipment carrying capacity is weak, a neural network with a smaller network scale can be selected.

[0084] S120. Based on the deployment requirement information, obtain a neural network to be deployed on the target device from the trained neural network.

[0085] According to the computing power information of the target device, the neural network to be deployed is selected to be suitable for the target device to ensure that the neural network to be deployed can be used normally after being deployed on the target device. According to the network speed requirement information, the neural network to be deployed that meets the computing speed requirement is selected to meet the requirements of the corresponding application scenario.

[0086] S130, deploying the neural network to be deployed on the target device.

[0087] The above-mentioned embodiment can select a suitable neural network for deployment from the target neural network and the various sub-neural networks included in the target neural network according to the carrying capacity of the target device, thereby avoiding the defect of having to train the neural network multiple times according to the carrying capacity of different devices, improving the training and deployment efficiency of the neural network, and saving training resources.

[0088] In some embodiments, before executing step S110, the following steps S1001-S1004 for training the target neural network may also be executed:

[0089] S1001. Extract multiple sub-neural networks from the neural network to be trained.

[0090] Here, the neural network to be trained is a neural network with a maximum network scale to be trained, and the maximum network scale can be manually set in advance according to actual application requirements, corresponding to the maximum characteristic dimension of the neural network. The maximum network scale can be specifically set according to the carrying capacity of the device on which the neural network is to be deployed. For example, for a neural network that needs to be deployed on a device with a smaller carrying capacity such as a mobile phone, the above-mentioned maximum network scale is set according to the maximum carrying capacity of the device with a smaller carrying capacity, and is set smaller; for a neural network to be deployed on a device with a larger carrying capacity such as a server, the above-mentioned maximum network scale is set according to the maximum carrying capacity of the device with a larger carrying capacity, and is set larger.

[0091] In addition to setting the maximum network size, the minimum network size also needs to be set when extracting sub-neural networks. The neural network corresponding to the minimum network size has the simplest structure compared to other sub-neural networks and neural networks to be trained, and includes the least neural network layers or neurons. The minimum network size is also manually set in advance according to actual application requirements, corresponding to the minimum feature dimension of the neural network. The minimum network size can be set specifically according to the carrying capacity of the device on which the neural network is to be deployed. For example, for neural networks that need to be deployed on devices with smaller carrying capacity, such as mobile phones, the minimum network size is set based on the minimum carrying capacity of the above-mentioned devices with smaller carrying capacity.

[0092] In this step, when extracting sub-neural networks, they are randomly extracted according to the minimum network size, and the network size of the extracted sub-neural networks needs to be greater than or equal to the above minimum network size. In addition, if the extracted sub-neural networks do not include the sub-neural network with the minimum network size, it is necessary to extract another neural network with the minimum network size as one of the sub-neural networks.

[0093] According to the above records, the minimum network size is determined according to the minimum carrying capacity of the device to be deployed. If the minimum network size is not set, when extracting sub-neural networks, sub-neural networks with smaller network sizes may be extracted, and then these sub-neural networks with smaller network sizes are trained. However, these sub-neural networks with smaller network sizes will not need to be deployed on the device. It can be seen that training such unused sub-neural networks will not only reduce the training efficiency of the neural network, but also waste training resources.

[0094] The neural network to be trained may be a neural network for image processing or a neural network for text processing. For example, the neural network to be trained may be a Transformer model.

[0095] In each iteration, multiple sub-neural networks are randomly extracted from the neural network to be trained, so that the parameters of the neural network to be trained and each sub-neural network can be shared through multiple trainings.

[0096] S1002. Use a first real training set to train the neural network to be trained to obtain a first neural network, and use the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network.

[0097] Here, the first real training set includes multiple real training samples, each training sample includes a real sample and a standard value corresponding to the real sample. For example, for a neural network performing translation, the training sample may include a word to be translated and a standard translation word after translation. Here, the word to be translated is a real word that needs to be translated, and is not a word predicted based on the word to be translated.

[0098] The first real training set is obtained in advance before executing this step. In this step, the neural network to be trained and the multiple randomly selected sub-neural networks are trained separately using the first real training set, with the aim of making the order of magnitude of the parameter gradients of the neural network to be trained and each sub-neural network as consistent as possible, and reaching a certain compromise for the shared parameters.

[0099] In the above, the neural network to be trained is trained using the first real training set to obtain the first neural network, and the sub-neural network is trained using the first real training set to obtain the first sub-neural network. The first neural network and the first sub-neural network are not the neural networks to be obtained by training, and the first neural network and each first sub-neural network have similar parameters of the same attributes / or the same definition, but parameter sharing cannot be achieved. The first neural network and the first sub-neural network are neural networks in an intermediate state obtained during training.

[0100] S1003. Input a plurality of real samples into the first neural network for processing respectively, obtain a sample processing result corresponding to each real sample, and generate a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results.

[0101] For the neural network that performs translation, the real sample can be the vocabulary to be translated; the sample processing result can be the vocabulary translated by the first neural network. In this step, the output of the first neural network in the intermediate state of neural network training is used to form a training set for training each sub-neural network, that is, the first predicted training set mentioned above. Using the first predicted training set to train each sub-neural network eliminates the mutual influence between each sub-neural network, which is conducive to parameter sharing between the neural network to be trained and each sub-neural network.

[0102] The above-mentioned multiple real samples may all or partly belong to the above-mentioned first real training set, or may be samples completely different from the samples in the above-mentioned first real training set, and this application does not limit this.

[0103] S1004: Use the second real training set to train the first neural network, and use the second real training set and the first predicted training set to train each first sub-neural network in the plurality of first sub-neural networks respectively to obtain the target neural network.

[0104] Here, the second real training set includes multiple real training samples, each training sample includes a real sample and a standard value corresponding to the real sample. For example, for a neural network performing translation, the training sample may include a word to be translated and a standard translation word after translation. Here, the word to be translated is a real word that needs to be translated, and is not a word predicted based on the word to be translated.

[0105] The second real training set may be completely different from or partially the same as the first real training set, and this application does not limit this. The multiple real samples in step 1003 may all or partly belong to the second real training set, or may be completely different from the samples in the first real training set, and this application does not limit this.

[0106] The second real training set is obtained in advance before executing this step. This step uses the second real training set to train the first neural network, and uses the second real training set and the first predicted training set to train the first sub-neural network, which can achieve parameter sharing between each sub-neural network and the neural network to be trained, thereby improving the efficiency of neural network training and reducing resource consumption.

[0107] This implementation uses the first predicted training set formed by the output of the first neural network in the middle state of neural network training to train the neural network to be trained and each sub-neural network extracted from the neural network to be trained, so that each sub-neural network and the neural network to be trained can promote each other and realize parameter sharing between the neural network to be trained and each sub-neural network. The trained target neural network is trimmed, and the obtained sub-neural network can be deployed on a device with a corresponding carrying capacity, that is, a neural network suitable for different carrying capacities can be obtained through one training, which improves the training efficiency of the neural network and saves training resources.

[0108] In some embodiments, in the above step 1004, each first sub-neural network in a plurality of first sub-neural networks is trained respectively using the second real training set and the first predicted training set, which can be specifically implemented using the following steps S10041-S10043:

[0109] S10041. Use the second real training set to train each first sub-neural network in the multiple first sub-neural networks respectively, and generate a first objective function corresponding to each first sub-neural network.

[0110] The first objective function is obtained by supervised learning using real training samples.

[0111] S10042. Use the first prediction training set to train each first sub-neural network in the multiple first sub-neural networks respectively, and generate a second target function corresponding to each first sub-neural network.

[0112] The second objective function is obtained by supervised learning using training samples formed by the output of the first neural network in an intermediate state of neural network training.

[0113] This application does not limit the execution order of the above S10041 and S10042.

[0114] S10043. For each first sub-neural network among the multiple first sub-neural networks, train the first sub-neural network based on a first objective function and a second objective function corresponding to the first sub-neural network.

[0115] Here, the first objective function and the second objective function may be used to form a target training function for training the first sub-neural network, and then the target training function is used to train the first sub-neural network. This may be implemented using the following sub-steps S100431-S100434:

[0116] S100431. Obtain the number of iterations of the first objective function corresponding to each first sub-neural network generated by training each first sub-neural network in the plurality of first sub-neural networks respectively using the second real training set, and / or the number of iterations of the second objective function corresponding to each first sub-neural network generated by training each first sub-neural network in the plurality of first sub-neural networks respectively using the first predicted training set.

[0117] Here, the number of iterations for generating the first objective function is equal to the number of iterations for generating the second objective function, and the number of iterations can be obtained by selecting one of them.

[0118] S100432. Determine the function adjustment ratio based on the obtained number of iterations.

[0119] Here, when determining the function adjustment ratio, it can be determined by using the number of iterations and a preset linear function. The linear function can be a monotonically increasing function. As the number of iterations increases, the value of the function adjustment ratio increases. When the number of iterations increases to a certain value, the function adjustment ratio no longer changes. In specific implementation, the simulated annealing algorithm can be used to determine the value of the function adjustment ratio.

[0120] S100433. Generate a target training function based on the first objective function, the second objective function and the function adjustment ratio.

[0121] Here, the target training function can be formed using the following formula:

[0122] Target training function = first target function + function adjustment ratio × second target function

[0123] According to the description in step S100432, when the number of iterations is small, the value of the function adjustment ratio is small, the second objective function has a small effect on the target training function, and when the target training function is used to train the sub-neural network, the second objective function has a small effect on the sub-neural network training. As the number of iterations increases, the second objective function has a larger effect on the target training function. When the target training function is used to train the sub-neural network, the second objective function has an increasingly larger effect on the sub-neural network training. The above design concept of the function adjustment ratio is: when the number of iterations is small, the output noise of the first neural network is large. At this time, in order to ensure the accuracy of the training, the second objective function generated by the first prediction training set needs to be set to have a smaller effect on the target training function. As the number of iterations increases, the output noise of the first neural network becomes smaller. At this time, the second objective function generated by the first prediction training set can be set to have a larger effect on the target training function, which will not affect the training accuracy of the sub-neural network.

[0124] The above-mentioned function adjustment ratio and the above formula are used to integrate the first objective function and the second objective function, and the sub-neural network is trained using the integrated target training function, which can reduce the impact of noise in the output of the first neural network on the neural network training and ensure the accuracy of the trained neural network. At the same time, the sub-neural network is trained using the target training function after integrating the first objective function corresponding to the second real training set and the second objective function corresponding to the predicted training set, and the first neural network is trained using the second real training set, which can achieve parameter sharing between each sub-neural network and the neural network to be trained.

[0125] S100434. Train the first sub-neural network using the target training function.

[0126] The target neural network trained by the method in the above embodiment can complete the processing task of non-machine reorganized text. For scenarios with higher requirements, such as the translation of reorganized text, after the first neural network is trained using the second real training set in step S1004, and the first sub-neural network is trained respectively using the second real training set and the first predicted training set, and before the trained target neural network is obtained, the following steps 1 to 2 may also be included:

[0127] Step 1: Obtain a third real training set, and generate a second predicted training set based on associated samples associated with training samples in the third real training set.

[0128] Here, the third real training set includes multiple real training samples, each training sample includes a real sample and a standard value corresponding to the real sample. For example, for a neural network performing translation, the training sample may include a word to be translated and a standard translation word after translation. Here, the word to be translated is a real word that needs to be translated, and is not a word predicted based on the word to be translated.

[0129] Any two of the second real training set, the first real training set, and the third real training set may be completely different or partially the same, and this application does not limit this. The multiple real samples in step 1003 may all or partly belong to the second real training set, or may be completely different from the samples in the first real training set, or may be completely different from the samples in the third real training set, and this application does not limit this.

[0130] For the neural network performing translation, the second prediction training set may be generated by using the following sub-steps 11 to 13:

[0131] Sub-step 11: determining a predicted word that appears after the word to be translated based on the word to be translated in the training sample and a plurality of texts including the word to be translated.

[0132] The predicted words are not real words, but words that may appear after the words to be translated and are inferred based on multiple texts containing the words to be translated. The predicted words may be adjacent to the words to be translated or not.

[0133] Sub-step 12: combining the predicted vocabulary and the standard translation vocabulary to form an associated sample of the training sample.

[0134] Sub-step 13: Generate the second predicted training set by using the associated samples corresponding to each training sample in the third real training set.

[0135] A prediction training set is constructed using the predicted words that may appear after the words to be translated, and the first neural network and each first sub-neural network are trained using the prediction training set, so that the neural network to be trained and the sub-neural networks can achieve parameter sharing.

[0136] Step 2: Use the second predicted training set to train the second neural network, and use the third real training set and the second predicted training set to train each of the multiple second sub-neural networks to obtain the target neural network.

[0137] The second neural network is obtained by training the first neural network using the second real training set. The second sub-neural network is obtained by training the first sub-neural network using the second real training set and the first predicted training set respectively.

[0138] The above-mentioned using the third real training set and the second predicted training set to train the second sub-neural network respectively can be specifically implemented by using the following sub-steps 21 to 23:

[0139] Sub-step 21: Use the third real training set to train the second sub-neural network to generate a third objective function.

[0140] The third objective function is obtained by supervised learning using real training samples.

[0141] Sub-step 22: training the second sub-neural network using the second prediction training set to generate a fourth objective function.

[0142] The fourth objective function is obtained by supervised learning using training samples formed by the predicted vocabulary.

[0143] This application does not limit the execution order of the above sub-steps 21 and 22.

[0144] Sub-step 23: training the second sub-neural network based on the third objective function and the fourth objective function.

[0145] Here, the third objective function and the fourth objective function can be used to form a comprehensive training function for training the second sub-neural network, and then the second sub-neural network is trained using the comprehensive training function. The comprehensive training function can be formed using the following sub-formula:

[0146] Target training function = third target function + adjustment ratio parameter value × fourth target function

[0147] The above-mentioned adjustment ratio parameter value can be a preset constant, or it can be determined by using a simulated annealing method. The determination method can be the same as the determination method of the above-mentioned function adjustment ratio. Specifically, it can be determined by using a preset linear function based on the number of iterations of generating the third objective function or the fourth objective function. The adjustment ratio parameter value; the greater the number of iterations of generating the third objective function or the fourth objective function, the greater the adjustment ratio parameter value; when the number of iterations of generating the third objective function or the fourth objective function increases to a certain value, the adjustment ratio parameter value no longer changes. The linear function that determines the adjustment ratio parameter value can be different from or the same as the linear function that determines the function adjustment ratio, and the adjustment ratio parameter value can be the same as or different from the value of the function adjustment ratio, which is not limited in this application.

[0148] The fourth objective function constructed using the second predicted training set and the third objective function constructed using the third real training set are used together to train the sub-neural network, which can enable the largest-scale neural network and the sub-neural network to achieve parameter sharing.

[0149] In the above embodiment, during the process of training the neural network to be trained and each sub-neural network, a new real training set, i.e., the third real training set, and a second predicted training set associated with the real training set are further combined for training. More training sets are used for more training times, which can better achieve parameter sharing between the neural network to be trained and each sub-neural network.

[0150] The target neural network obtained by training in the above embodiment of the present application shares parameters with each sub-neural network, and the desired sub-neural network can be obtained by directly cutting from the target neural network, without the need to retrain the applicable neural network. Under the premise of ensuring the accuracy of the neural network, the training time of the neural network is greatly reduced, the training efficiency of the neural network is improved, and resource consumption is reduced. The target neural network trained in the above embodiment is suitable for various application scenarios. For example, for text translation scenarios, the trained target neural network can be applied to devices with strong carrying capacity such as servers. After the trained target neural network is installed on the server, text translation can be performed more accurately. At the same time, the sub-neural network in the target neural network can be applied to devices with weak carrying capacity such as mobile terminals. After the trained sub-neural network is installed on the mobile terminal, text translation can be performed more accurately, avoiding multiple training of neural networks of different network sizes, improving the training efficiency of the neural network, and saving training resources. For another example, for image recognition scenarios, the trained target neural network can be applied to devices with strong carrying capacity such as servers. After the trained target neural network is installed on the server, image recognition can be performed more accurately. At the same time, the sub-neural network in the target neural network can be applied to devices with weak carrying capacity such as mobile terminals. After the trained sub-neural network is installed on the mobile terminal, image recognition can be performed more accurately, avoiding multiple training of neural networks with different network sizes, improving the training efficiency of the neural network, and saving training resources. For another example, for reading comprehension scenarios, the trained target neural network can be applied to devices with strong carrying capacity such as servers. After the trained target neural network is installed on the server, the answer to the reading comprehension can be determined more accurately. At the same time, the sub-neural network in the target neural network can be applied to devices with weak carrying capacity such as mobile terminals. After the trained sub-neural network is installed on the mobile terminal, the answer to the reading comprehension can be determined more accurately, avoiding multiple training of neural networks with different network sizes, improving the training efficiency of the neural network, and saving training resources.

[0151] The present application also provides a neural network training method, which can be applied on a client or server to train neural networks of different network sizes with shared parameters. Specifically, Figure 2 As shown, the neural network training method provided in this application includes the following steps:

[0152] S210, extracting multiple sub-neural networks from the neural network to be trained.

[0153] S220, using a first real training set to train the neural network to be trained to obtain a first neural network, and using the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network.

[0154] S230: Input a plurality of real samples into the first neural network for processing respectively, obtain a sample processing result corresponding to each real sample, and generate a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results.

[0155] S240. Use the second real training set to train the first neural network, and use the second real training set and the first predicted training set to train each first sub-neural network in the multiple first sub-neural networks to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are all trained neural networks.

[0156] The training process of the neural network to be trained in this embodiment is the same as the training process of the neural network to be trained in the above embodiment, and will not be repeated here.

[0157] The present application also provides a text translation method, which is applied to a target device, such as Figure 3 As shown, including:

[0158] S310: Obtain the text to be translated.

[0159] S320: Translate the text to be translated by means of a translation neural network deployed on the target device to obtain a translation result.

[0160] Wherein, the translation neural network is the neural network to be deployed in the above embodiment, and / or, the translation neural network includes the target neural network obtained by the above neural network training method and / or at least one sub-neural network in the target neural network.

[0161] By using the translation neural network deployed on the target device to translate text, the translated text can be obtained more accurately.

[0162] Corresponding to the above-mentioned neural network deployment method, the present application also discloses a neural network deployment device, each module in the device can implement each step in the neural network deployment method of each embodiment above, and can achieve the same beneficial effects, so the same parts will not be repeated here. Specifically, Figure 4 As shown, the neural network deployment device includes:

[0163] Acquisition module 410, for acquiring a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are trained neural networks, and the deployment requirement information includes information for indicating the carrying capacity of the target device.

[0164] The network screening module 420 is used to obtain the neural network to be deployed on the target device from the trained neural network based on the deployment requirement information.

[0165] The network deployment module 430 is used to deploy the neural network to be deployed on the target device.

[0166] Corresponding to the above-mentioned neural network training method, the present application also discloses a neural network training device, each module in the device can implement each step in the neural network training method of each embodiment above, and can achieve the same beneficial effect, therefore, the same parts will not be described here in detail. Specifically, Figure 5 As shown, the neural network training device disclosed in the present application includes:

[0167] The sub-network extraction module 510 is used to extract multiple sub-neural networks from the neural network to be trained.

[0168] The first training module 520 is used to train the neural network to be trained using the first real training set to obtain a first neural network, and to train each of the multiple sub-neural networks using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network.

[0169] The sample prediction module 530 is used to input multiple real samples into the first neural network for processing, obtain sample processing results corresponding to each real sample, and generate a first prediction training set based on the multiple real samples and the obtained multiple sample processing results.

[0170] The second training module 540 is used to train the first neural network using a second real training set, and to train each first sub-neural network in a plurality of first sub-neural networks using the second real training set and the first predicted training set to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are all trained neural networks.

[0171] Corresponding to the above-mentioned text translation method, the present application also discloses a text translation device, each module in which can implement each step in the text translation method of each embodiment above, and can achieve the same beneficial effect, therefore, the same parts will not be described in detail here. Specifically, Figure 6 As shown, a text translation device disclosed in the present application includes:

[0172] The text acquisition module 610 is used to acquire the text to be translated.

[0173] The translation module 620 is used to translate the text to be translated through the translation neural network deployed on the target device to obtain a translation result. The translation neural network is the neural network to be deployed in the above embodiment, and / or the translation neural network includes the target neural network obtained by the neural network training method in the above embodiment and / or at least one sub-neural network in the target neural network.

[0174] Corresponding to the neural network deployment method, neural network training method or text translation method in the above embodiments, the present application embodiment further provides an electronic device 700, such as Figure 7 FIG. 7 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present application, including:

[0175] Processor 71, memory 72, and bus 73; memory 72 is used to store execution instructions, including internal memory 721 and external memory 722; the internal memory 721 here is also called internal memory, which is used to temporarily store the calculation data in the processor 71 and the data exchanged with the external memory 722 such as a hard disk. The processor 71 exchanges data with the external memory 722 through the internal memory 721. When the electronic device 700 is running, the processor 71 communicates with the memory 72 through the bus 73, so that the processor 71 executes the following instructions corresponding to the neural network deployment method: obtain the target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information used to represent the carrying capacity of the target device; based on the deployment requirement information, from the trained neural network, obtain the neural network to be deployed on the target device; deploy the neural network to be deployed on the target device. The trained target neural network includes a trained first neural network and a trained sub-neural network. Or the processor 71 executes the following instructions corresponding to the neural network training method: extract multiple sub-neural networks from the neural network to be trained; use the first real training set to train the neural network to be trained to obtain a first neural network, and use the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network; input multiple real samples into the first neural network for processing to obtain a sample processing result corresponding to each real sample, and generate a first predicted training set based on the multiple real samples and the obtained multiple sample processing results; use the second real training set to train the first neural network, and use the second real training set and the first predicted training set to train each of the multiple first sub-neural networks to obtain a target neural network, wherein the target neural network and the sub-neural networks included in the target neural network are all trained neural networks. Or enable the processor 71 to execute the following instructions corresponding to the text translation method: obtain the text to be translated; translate the text to be translated through the translation neural network deployed on the target device to obtain a translation result; wherein the translation neural network is the neural network to be deployed in the above embodiment, and / or the translation neural network includes the target neural network obtained by the neural network training method in the above embodiment and / or at least one sub-neural network in the target neural network.

[0176] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the neural network deployment method, the neural network training method, or the text translation method in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0177] The computer program product of the neural network deployment method, or the neural network training method, or the text translation method provided in the embodiments of the present application includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the neural network deployment method, or the neural network training method, or the text translation method in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0178] The present application also provides a computer program, which implements any one of the methods of the aforementioned embodiments when executed by a processor. The computer program product can be implemented in hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0179] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0182] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0183] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A neural network deployment method, characterized in that: include: Acquire a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information for indicating the carrying capacity of the target device; Based on the deployment requirement information, obtaining a neural network to be deployed on the target device from the trained neural network; Deploying the neural network to be deployed on the target device; Before obtaining the target neural network, the method further includes: Extracting multiple sub-neural networks from the neural network to be trained; The neural network to be trained is trained using a first real training set to obtain a first neural network, and each of the plurality of sub-neural networks is trained using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network; the training samples in the first real training set include words to be translated and standard translation words after translation; Inputting a plurality of real samples into the first neural network for processing respectively, obtaining a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results; The first neural network is trained using a second real training set to obtain a second neural network, and each of the first sub-neural networks in a plurality of first sub-neural networks is trained using the second real training set and the first predicted training set to obtain a second sub-neural network corresponding to each first sub-neural network; the training samples in the second real training set include words to be translated and standard translation words after translation; Acquire a third real training set, and generate a second predicted training set based on associated samples associated with the training samples in the third real training set; Using the second predicted training set to train the second neural network, and using the third real training set and the second predicted training set to train each second sub-neural network in a plurality of second sub-neural networks, respectively, to obtain the target neural network; The training samples in the third real training set include words to be translated and standard translation words; The step of generating a second predicted training set based on associated samples associated with training samples in the third real training set comprises: Determining a predicted word that appears after the word to be translated based on the word to be translated in the training sample and a plurality of texts including the word to be translated; Combining the predicted vocabulary with the standard translation vocabulary to form an associated sample of the training sample; The second predicted training set is generated by using the associated samples respectively corresponding to each training sample in the third real training set.

2. The neural network deployment method according to claim 1, characterized in that: The deployment requirement information includes at least one of the following: The computing capability information of the target device; and the network speed requirement information.

3. The neural network deployment method according to claim 1, characterized in that: The using the second real training set and the first predicted training set to train each first sub-neural network in the plurality of first sub-neural networks respectively comprises: Using the second real training set to train each first sub-neural network in the plurality of first sub-neural networks, respectively, to generate a first objective function corresponding to each first sub-neural network; Using the first prediction training set to train each first sub-neural network in the plurality of first sub-neural networks, respectively, to generate a second objective function corresponding to each first sub-neural network; For each first sub-neural network among the multiple first sub-neural networks, the first sub-neural network is trained based on a first objective function and a second objective function corresponding to the first sub-neural network.

4. The neural network deployment method according to claim 3, characterized in that: The training of the first sub-neural network based on the first objective function and the second objective function corresponding to the first sub-neural network includes: Obtaining the number of iterations of respectively training each first sub-neural network in the plurality of first sub-neural networks using the second real training set to generate a first objective function corresponding to each first sub-neural network, and / or the number of iterations of respectively training each first sub-neural network in the plurality of first sub-neural networks using the first predicted training set to generate a second objective function corresponding to each first sub-neural network; Based on the obtained number of iterations, determine the function adjustment ratio; Generate a target training function based on the first target function, the second target function and the function adjustment ratio; The first sub-neural network is trained using the target training function.

5. A neural network training method, characterized in that: include: Extracting multiple sub-neural networks from the neural network to be trained; Using a first real training set to train the neural network to be trained to obtain a first neural network, and using the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network; The training samples in the first real training set include words to be translated and standard translation words after translation; Inputting a plurality of real samples into the first neural network for processing respectively, obtaining a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results; Using a second real training set to train the first neural network to obtain a second neural network, and using the second real training set and the first predicted training set to train each first sub-neural network in a plurality of first sub-neural networks respectively to obtain a second sub-neural network corresponding to each first sub-neural network; The training samples in the second real training set include words to be translated and standard translation words after translation; Acquire a third real training set, and generate a second predicted training set based on associated samples associated with the training samples in the third real training set; The second neural network is trained using the second predicted training set, and each of the second sub-neural networks in the plurality of second sub-neural networks is trained using the third real training set and the second predicted training set to obtain a target neural network; the target neural network is used to translate the text to be translated to obtain a translation result; The training samples in the third real training set include words to be translated and standard translation words; The step of generating a second predicted training set based on associated samples associated with training samples in the third real training set comprises: Determining a predicted word that appears after the word to be translated based on the word to be translated in the training sample and a plurality of texts including the word to be translated; Combining the predicted vocabulary with the standard translation vocabulary to form an associated sample of the training sample; The second predicted training set is generated by using the associated samples respectively corresponding to each training sample in the third real training set.

6. A text translation method, characterized in that: The method comprises: Get the text to be translated; Translating the text to be translated by a translation neural network deployed on the target device to obtain a translation result; Wherein, the translation neural network is the neural network to be deployed as described in any one of claims 1 to 4, and / or the translation neural network includes the target neural network obtained by the neural network training method described in claim 5 and / or at least one sub-neural network in the target neural network.

7. A neural network deployment device, characterized in that: include: An acquisition module, used to acquire a target neural network and deployment requirement information; the target neural network and the sub-neural networks included in the target neural network are all trained neural networks, and the deployment requirement information includes information used to represent the carrying capacity of the target device; A network screening module, configured to obtain a neural network to be deployed on the target device from the trained neural network based on the deployment requirement information; A network deployment module, used to deploy the neural network to be deployed on the target device; The device also includes a training module for extracting a plurality of sub-neural networks from the neural network to be trained; Using a first real training set to train the neural network to be trained to obtain a first neural network, and using the first real training set to train each of the multiple sub-neural networks to obtain a first sub-neural network corresponding to each sub-neural network; The training samples in the first real training set include words to be translated and standard translation words after translation; Inputting a plurality of real samples into the first neural network for processing respectively, obtaining a sample processing result corresponding to each real sample, and generating a first prediction training set based on the plurality of real samples and the obtained plurality of sample processing results; Using a second real training set to train the first neural network to obtain a second neural network, and using the second real training set and the first predicted training set to train each first sub-neural network in a plurality of first sub-neural networks respectively to obtain a second sub-neural network corresponding to each first sub-neural network; The training samples in the second real training set include words to be translated and standard translation words after translation; Acquire a third real training set, and generate a second predicted training set based on associated samples associated with the training samples in the third real training set; train the second neural network using the second predicted training set, and train each of the plurality of second sub-neural networks using the third real training set and the second predicted training set, respectively, to obtain the target neural network; The target neural network is used to translate the text to be translated to obtain a translation result; The training samples in the third real training set include words to be translated and standard translation words; the training module is specifically used to: determine the predicted words that appear after the words to be translated based on the words to be translated in the training samples and a plurality of texts including the words to be translated; Combining the predicted vocabulary with the standard translation vocabulary to form an associated sample of the training sample; The second predicted training set is generated by using the associated samples respectively corresponding to each training sample in the third real training set.

8. A neural network training device, characterized in that: include: A sub-network extraction module, used to extract multiple sub-neural networks from the neural network to be trained; A first training module is used to train the neural network to be trained using a first real training set to obtain a first neural network, and to train each of the multiple sub-neural networks using the first real training set to obtain a first sub-neural network corresponding to each sub-neural network; The training samples in the first real training set include words to be translated and standard translation words after translation; A sample prediction module, used to input a plurality of real samples into the first neural network for processing respectively, obtain a sample processing result corresponding to each real sample, and generate a first prediction training set based on the plurality of real samples and the plurality of obtained sample processing results; A second training module is used to train the first neural network using a second real training set to obtain a second neural network, and to train each first sub-neural network in a plurality of first sub-neural networks using the second real training set and the first predicted training set to obtain a second sub-neural network corresponding to each first sub-neural network; The training samples in the second real training set include words to be translated and standard translation words after translation; Acquire a third real training set, and generate a second predicted training set based on associated samples associated with the training samples in the third real training set; train the second neural network using the second predicted training set, and train each of the plurality of second sub-neural networks using the third real training set and the second predicted training set, respectively, to obtain a target neural network; The training samples in the third real training set include words to be translated and standard translation words; the second training module is specifically used to: determine the predicted words that appear after the words to be translated based on the words to be translated in the training samples and a plurality of texts including the words to be translated; Combining the predicted vocabulary with the standard translation vocabulary to form an associated sample of the training sample; The second predicted training set is generated by using the associated samples respectively corresponding to each training sample in the third real training set.

9. A text translation device, characterized in that: include: A text acquisition module is used to acquire the text to be translated; A translation module, used to translate the text to be translated through a translation neural network deployed on a target device to obtain a translation result; Wherein, the translation neural network is the neural network to be deployed as described in any one of claims 1 to 4, and / or the translation neural network includes the target neural network obtained by the neural network training method described in claim 5 and / or at least one sub-neural network in the target neural network.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the neural network deployment method as described in any one of claims 1 to 4, the neural network training method as described in claim 5, or the text translation method as described in claim 6 is executed.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the neural network deployment method as described in any one of claims 1 to 4, or the neural network training method as described in claim 5, or the text translation method as described in claim 6.

Images