Method and device for training multilingual model, computer device and storage medium

Abstract

This application belongs to the field of artificial intelligence and relates to a training method for a multilingual model, including: acquiring a preset monolingual pre-trained model; constructing a first initial multilingual model based on the monolingual pre-trained model; adjusting the first initial multilingual model based on the first parameters of the first encoder within the monolingual pre-trained model to obtain a second initial multilingual model; acquiring pre-collected training data; and finally training the second initial multilingual model using the training data to obtain a target multilingual model. This application also provides a training device, computer equipment, and storage medium for a multilingual model. Furthermore, this application relates to blockchain technology, allowing the target multilingual model to be stored in a blockchain. This application, based on the use of a monolingual pre-trained model, effectively reduces the training cost of the target multilingual model and improves the training and generation efficiency of the target multilingual model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to training methods, apparatus, computer equipment and storage media for multilingual models. Background Technology

[0002] Existing speech recognition models typically only recognize one language. To recognize multiple languages, multiple models must be deployed simultaneously. This significantly increases the difficulty of deployment, consumes substantial resources, and is also cumbersome during training. With technological advancements, it has become possible to use a single model to recognize multiple languages. Multilingual models greatly simplify deployment, launch, and maintenance, and model training is much easier. Current multilingual model training often directly replicates single-language model training methods, using a mixture of data from multiple languages ​​for training. This approach requires a long training time, resulting in high training costs and low model generation efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a training method, apparatus, computer device, and storage medium for a multilingual model, in order to solve the technical problem that existing multilingual model training methods typically directly copy single-language model training methods, using a mixture of data from multiple languages ​​for training. This training method requires a long training time, resulting in high model training costs and low model generation efficiency.

[0004] To address the aforementioned technical problems, this application provides a method for training a multilingual model, employing the following technical solution:

[0005] Obtain a pre-trained monolingual model;

[0006] A first initial multilingual model is constructed based on the aforementioned monolingual pre-trained model;

[0007] Based on the first parameters of the first encoder in the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain the second initial multilingual model.

[0008] Acquire pre-collected training data;

[0009] The second initial multilingual model is trained using the training data to obtain the target multilingual model.

[0010] Furthermore, the step of constructing the first initial multilingual model based on the monolingual pre-trained model specifically includes:

[0011] Obtain the size of the first encoder of the monolingual pre-trained model; and,

[0012] Obtain the layer number of the first encoder;

[0013] Construct an initial model that has the same size and number of layers as the first encoder;

[0014] The initial model is subjected to a preset random initialization process to obtain the first initial multilingual model.

[0015] Furthermore, the step of adjusting the first initial multilingual model based on the first parameters of the encoder within the monolingual pre-trained model to obtain the second initial multilingual model specifically includes:

[0016] Obtain the first model parameters of the monolingual pre-trained model, and obtain the first parameters of the first encoder from the first model parameters; and,

[0017] Obtain the second model parameters of the first initial multilingual model, and obtain the second parameters of the second encoder within the first initial multilingual model from the second model parameters;

[0018] Replace the second parameter in the first initial multilingual model with the first parameter;

[0019] Keeping the third parameter of the first decoder in the first initial multilingual model unchanged, the second initial multilingual model is obtained.

[0020] Furthermore, the step of training the second initial multilingual model using the training data to obtain the target multilingual model specifically includes:

[0021] Freeze the fourth parameter of the third encoder within the second initial multilingual model;

[0022] The training data is used to train the fifth parameter of the second decoder within the second initial multilingual model;

[0023] When it is determined that the first loss function corresponding to the first training task that uses the training data to train the fifth parameter has converged, the freeze on the fourth parameter in the second initial multilingual model is lifted;

[0024] The training data is used to jointly train the fourth parameter in the third encoder and the fifth parameter in the second decoder;

[0025] When the second loss function corresponding to the second training task, which uses the training data to train the fourth parameter and the fifth parameter, converges, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

[0026] Furthermore, after the step of jointly training the fourth parameter in the third encoder and the fifth parameter in the second decoder using the training data, the method further includes:

[0027] The feature data within the third encoder is obtained during the process of training the fourth parameter of the third encoder using the training data.

[0028] A language recognition task is constructed based on a preset fully connected layer and the feature data;

[0029] The fourth parameter is optimized based on the language identification task.

[0030] The step of completing the training of the second initial multilingual model and using the trained second initial multilingual model as the target multilingual model when the second loss function corresponding to the second training task trained using the training data for the fourth and fifth parameters converges includes:

[0031] After completing the optimization processing of the fourth parameter based on the language recognition task, and determining that the second loss function corresponding to the second training task of training the fourth parameter and the fifth parameter using the training data has converged, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

[0032] Furthermore, the step of acquiring the pre-collected training data specifically includes:

[0033] Acquire pre-collected initial voice data;

[0034] The initial speech data is processed based on a preset language sampling algorithm to ensure that the number of speech items in each language contained in the initial speech data is equal to a preset value, thereby obtaining the processed initial speech data.

[0035] The processed initial speech data is used as the training data.

[0036] Furthermore, after the step of training the second initial multilingual model using the training data to obtain the target multilingual model, the method further includes:

[0037] Receive the voice to be recognized;

[0038] The speech to be recognized is input into the target multilingual model, and the target multilingual model outputs the speech recognition result corresponding to the speech to be recognized.

[0039] Receive the speech recognition result returned by the target multilingual model;

[0040] The speech recognition results are displayed.

[0041] To address the aforementioned technical problems, this application also provides a training device for a multilingual model, employing the following technical solution:

[0042] The first acquisition module is used to acquire a preset monolingual pre-trained model;

[0043] The building module is used to build a first initial multilingual model based on the monolingual pre-trained model;

[0044] The processing module is used to adjust the first initial multilingual model based on the first parameters of the first encoder in the monolingual pre-trained model to obtain a second initial multilingual model.

[0045] The second acquisition module is used to acquire pre-collected training data;

[0046] The training module is used to train the second initial multilingual model using the training data to obtain the target multilingual model.

[0047] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0048] Obtain a pre-trained monolingual model;

[0049] A first initial multilingual model is constructed based on the aforementioned monolingual pre-trained model;

[0050] Based on the first parameters of the first encoder in the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain the second initial multilingual model.

[0051] Acquire pre-collected training data;

[0052] The second initial multilingual model is trained using the training data to obtain the target multilingual model.

[0053] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0054] Obtain a pre-trained monolingual model;

[0055] A first initial multilingual model is constructed based on the aforementioned monolingual pre-trained model;

[0056] Based on the first parameters of the first encoder in the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain the second initial multilingual model.

[0057] Acquire pre-collected training data;

[0058] The second initial multilingual model is trained using the training data to obtain the target multilingual model.

[0059] Compared with the prior art, the embodiments of this application have the following main advantages:

[0060] This embodiment of the application obtains a preset monolingual pre-trained model, then constructs a first initial multilingual model based on the monolingual pre-trained model, and then adjusts the first initial multilingual model based on the first parameters of the first encoder within the monolingual pre-trained model to obtain a second initial multilingual model. Subsequently, pre-collected training data is acquired, and finally, the second initial multilingual model is trained using the training data to obtain a target multilingual model. This embodiment of the application uses a monolingual pre-trained model and training data to train and generate a target multilingual model. Since the monolingual pre-trained model already has initial values, it can accelerate the training speed of the multilingual model, reduce the training cost of the multilingual model, and effectively improve the training and generation rate of the multilingual model. Attached Figure Description

[0061] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0063] Figure 2 A flowchart of an embodiment of the training method for the multilingual model according to this application;

[0064] Figure 3 This is a schematic diagram of the structure of an embodiment of a training device for a multilingual model according to this application;

[0065] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0070] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0071] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0072] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0073] It should be noted that the training method for the multilingual model provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the training device for the multilingual model is generally set in the server / terminal device.

[0074] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0075] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of a training method for a multilingual model according to this application. The training method for the multilingual model includes the following steps:

[0076] Step S201: Obtain the preset monolingual pre-trained model.

[0077] In this embodiment, the training method for the multilingual model runs on an electronic device (e.g., Figure 1 The server / terminal device shown can acquire the monolingual pre-trained model via wired or wireless connection. It should be noted that the aforementioned wireless connection methods include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods. Specifically, the existing monolingual model (i.e., the teacher model) can be retrained and fine-tuned using pre-collected initial speech data, and the teacher model can then guide the student model's training. During the training process, the student model is trained to approximate the posterior probability distribution of the teacher model as closely as possible, thereby generating the pre-trained monolingual model, i.e., the aforementioned monolingual pre-trained model.

[0078] Step S202: Construct a first initial multilingual model based on the monolingual pre-trained model.

[0079] In this embodiment, a model with the same encoder size and number of layers as the aforementioned monolingual pre-trained model, and randomly initialized, can be constructed as the first initial multilingual model. The first initial multilingual model also includes a decoder. The encoder can also be represented as an encoder, and the decoder can also be represented as a decoder.

[0080] Step S203: Based on the first parameters of the first encoder in the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain the second initial multilingual model.

[0081] In this embodiment, the second parameters of the second encoder in the first initial multilingual model can be replaced based on the first parameters of the first encoder in the monolingual pre-trained model, thereby obtaining the aforementioned second initial multilingual model.

[0082] Step S204: Obtain the pre-collected training data.

[0083] In this embodiment, pre-collected initial speech data can be acquired first, and then the initial speech data can be processed based on a preset language sampling algorithm to ensure that the number of speech items in each language contained in the initial speech data is equal to the preset value, thereby obtaining the above-mentioned training data.

[0084] Step S205: Use the training data to train the second initial multilingual model to obtain the target multilingual model.

[0085] In this embodiment, the second initial multilingual model is generated based on a monolingual pre-trained model. Using a monolingual pre-trained model can effectively accelerate the training speed of the multilingual model and improve its recognition performance. Compared with directly initializing the model for training, the pre-trained model already has relatively good initial values, thus accelerating model training and reducing training costs. Furthermore, the pre-trained model can also improve model performance to some extent. The specific implementation process of training the second initial multilingual model using training data to obtain the target multilingual model will be further described in detail in subsequent embodiments of this application, and will not be elaborated upon here.

[0086] This application obtains a pre-set monolingual pre-trained model, then constructs a first initial multilingual model based on the monolingual pre-trained model. Next, based on the first parameters of the first encoder within the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain a second initial multilingual model. Subsequently, pre-collected training data is acquired, and finally, the second initial multilingual model is trained using the training data to obtain a target multilingual model. This application uses a monolingual pre-trained model and training data to train and generate a target multilingual model. Since the monolingual pre-trained model already contains initial values, it can accelerate the training speed of the multilingual model, reduce the training cost, and effectively improve the training and generation rate of the multilingual model.

[0087] In some alternative implementations, step S202 includes the following steps:

[0088] Obtain the size of the first encoder of the monolingual pre-trained model. And,

[0089] In this embodiment, the size of the first encoder of the monolingual pre-trained model can be obtained from the model information by querying the model information of the monolingual pre-trained model.

[0090] Obtain the layer number of the first encoder.

[0091] In this embodiment, the number of layers of the first encoder of the monolingual pre-trained model can be obtained from the model information by querying the model information of the monolingual pre-trained model.

[0092] Construct an initial model that has the same size and number of layers as the first encoder.

[0093] In this embodiment, the initial model also includes a decoder whose output can be determined based on multilingual text in the preset training data.

[0094] The initial model is subjected to a preset random initialization process to obtain the first initial multilingual model.

[0095] In this embodiment, the aforementioned random initialization process refers to randomly initializing the parameters of the initial model. Random initialization can be understood as follows: Assuming a multilayer perceptron has only one element in the output layer (simplification), if the parameters of each hidden unit are initialized to the same value, then during forward propagation, each hidden unit will calculate the same value based on the same input and pass it to the output layer. During backward propagation, the gradient values ​​of the parameters of each hidden unit are equal. Therefore, these parameters will still have equal gradient values ​​after using gradient optimization algorithms. Subsequent iterations will follow the same pattern. In this case, regardless of the number of hidden layers / units, the essence of a hidden layer is that only one hidden unit is functioning. Therefore, we typically perform random initialization on the model parameters of neural networks, especially the weight parameters. Random initialization is crucial for breaking symmetry before optimization.

[0096] This application constructs a first initial multilingual model by directly using the encoder information in a monolingual pre-trained model, which enables the subsequent construction of the final target multilingual model based on the obtained first initial multilingual model, thereby improving the construction efficiency of the multilingual model.

[0097] In some optional implementations of this embodiment, step S203 includes the following steps:

[0098] Obtain the first model parameters of the monolingual pre-trained model, and obtain the first parameters of the first encoder from the first model parameters. Also,

[0099] In this embodiment, the first model parameters of the monolingual pre-trained model may include at least the parameters of the encoder and the parameters of the decoder within the monolingual pre-trained model.

[0100] Obtain the second model parameters of the first initial multilingual model, and obtain the second parameters of the second encoder within the first initial multilingual model from the second model parameters.

[0101] In this embodiment, the second model parameters of the first initial multilingual model may include at least the parameters of the encoder and the parameters of the decoder within the first initial multilingual model.

[0102] Replace the second parameter in the first initial multilingual model with the first parameter.

[0103] Keeping the third parameter of the first decoder in the first initial multilingual model unchanged, the second initial multilingual model is obtained.

[0104] After obtaining the first initial multilingual model, this application adjusts the parameters of the encoder within the first initial multilingual model using the first model parameters based on the monolingual pre-trained model to construct a second initial multilingual model. This allows the final target multilingual model to be quickly constructed based on the obtained second initial multilingual model, thereby improving the efficiency of multilingual model construction.

[0105] In some alternative implementations, step S205 includes the following steps:

[0106] Freeze the fourth parameter of the third encoder within the second initial multilingual model.

[0107] In this embodiment, freezing the fourth parameter of the third encoder in the second initial multilingual model means not training the fourth parameter of the third encoder in the second initial multilingual model first.

[0108] The training data is used to train the fifth parameter of the second decoder within the second initial multilingual model.

[0109] In this embodiment, the process of training the second initial multilingual model using the training data includes two steps. The first step is to freeze the fourth parameter of the third encoder in the second initial multilingual model and then use the training data to train the fifth parameter of the second decoder in the second initial multilingual model. This ensures that the parameters of the third encoder and the decoder in the second initial multilingual model are matched.

[0110] When it is determined that the first loss function corresponding to the first training task that uses the training data to train the fifth parameter has converged, the freeze on the fourth parameter in the second initial multilingual model is lifted.

[0111] In this embodiment, the first loss function mentioned above can specifically be the cross-entropy loss function.

[0112] The training data is used to jointly train the fourth parameter in the third encoder and the fifth parameter in the second decoder.

[0113] In this embodiment, the second step of training the second initial multilingual model using training data, which involves jointly training the fourth parameter in the third encoder and the fifth parameter in the second decoder using the training data, is to obtain a multilingual model that better matches the data.

[0114] When the second loss function corresponding to the second training task, which uses the training data to train the fourth parameter and the fifth parameter, converges, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

[0115] In this embodiment, the second loss function can specifically be the cross-entropy loss function. After the training data is input into the second initial multilingual model, the encoder within the second initial multilingual model will input the audio feature sequence x = (x1, ..., x...) from the training data. T Mapping to the intermediate feature sequence h = (h1, ..., h) K Then, the decoder within the second initial multilingual model maps the intermediate feature sequence h to the output sequence y = (y1, ..., y2). L That is, h t =encoder(x 1: ), y u =decoder(h t ,y 1: ), after obtaining y u Then with its labeled y ′ u After calculating the cross-entropy, backpropagation is used to update the network parameters, thereby training the second initial multilingual model. The above y ′ u It can be obtained from the training data.

[0116] After constructing a second initial multilingual model based on a monolingual pre-trained model, this application can train the second initial multilingual model based on pre-collected training data, thereby quickly obtaining the desired target multilingual model and effectively improving the construction efficiency of the multilingual model.

[0117] In some alternative implementations, step S205 further includes the following steps:

[0118] The feature data within the third encoder is obtained during the process of training the fourth parameter of the third encoder using the training data.

[0119] In this embodiment, a typical end-to-end speech recognition model consists of multiple encoders and multiple decoders. The encoders process the acoustic features, and the decoders then obtain the recognition result. In existing technologies, there is interference between multiple languages, and some words in different languages ​​have very similar pronunciations, making model learning difficult.

[0120] A language recognition task is constructed based on a preset fully connected layer and the feature data.

[0121] In this embodiment, the language recognition task refers to the task of mapping the feature data in the third encoder to the corresponding language type through the fully connected layer after obtaining the feature data in the process of training the fourth parameter of the third encoder using training data.

[0122] The fourth parameter is optimized based on the language recognition task.

[0123] In this embodiment, by adding a language recognition task during the training process of the second initial multilingual model, the fourth parameter of the third encoder in the second initial multilingual model can be optimized, thereby solving the problem of mutual interference between languages.

[0124] The step of completing the training of the second initial multilingual model and using the trained second initial multilingual model as the target multilingual model when the second loss function corresponding to the second training task trained using the training data for the fourth and fifth parameters converges includes:

[0125] After completing the optimization processing of the fourth parameter based on the language recognition task, and determining that the second loss function corresponding to the second training task of training the fourth parameter and the fifth parameter using the training data has converged, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

[0126] This application adds a language identification task during the training process of the second initial multilingual model using training data. Since language identification can amplify the differences between languages ​​to a certain extent and reduce confusion, it can improve the model's ability to distinguish languages ​​and thus improve the recognition effect of the multilingual model.

[0127] In some optional implementations of this embodiment, step S204 includes the following steps:

[0128] Acquire the pre-collected initial voice data.

[0129] In this embodiment, the initial speech data may include speech data from different languages. Due to differences in the number of users and the scope of use, the difficulty of data collection and annotation varies for different languages. Languages ​​with wide usage and large populations, such as Chinese and English, can easily have large amounts of data collected, and annotation is relatively easy; however, languages ​​with smaller usage, such as regional dialects, face greater challenges in data collection and annotation. Therefore, when training a multilingual model, the significant differences in the amount of data collected from multiple languages ​​can cause the model's recognition results to favor the language with the larger amount of data.

[0130] The initial speech data is processed based on a preset language sampling algorithm to ensure that the number of speech items in each language contained in the initial speech data is equal to a preset value, thus obtaining the processed initial speech data.

[0131] In this embodiment, the values ​​of the preset values ​​are not specifically limited and can be set according to actual usage requirements. The language sampling algorithm includes: the number of speech samples for each language is l1…l k , where l max ={l1…l k}, for any speech i, if l i <l max Then l i Upsampled to l max Ensure that the number of entries for each language is 1. max During training, each language is sampled on average, so the proportion of each language in a single batch is 1 / k.

[0132] The processed initial speech data is used as the training data.

[0133] This application adds data sampling during the training of the multilingual model. Specifically, it processes the pre-collected initial speech data based on a preset language sampling algorithm to obtain training data. Data sampling ensures a relative balance of data volume for different languages, thus guaranteeing the recognition performance of the multilingual model in less commonly spoken languages.

[0134] In some optional implementations of this embodiment, after step S205, the electronic device may further perform the following steps:

[0135] Receive the voice to be recognized.

[0136] In this embodiment, the speech to be recognized can be speech data input by the relevant user that needs to be recognized and processed.

[0137] The speech to be recognized is input into the target multilingual model, and the target multilingual model outputs the speech recognition result corresponding to the speech to be recognized.

[0138] Receive the speech recognition result returned by the target multilingual model.

[0139] The speech recognition results are displayed.

[0140] In this embodiment, the display method of the above-mentioned speech recognition results is not limited and can be determined according to the actual usage needs. For example, it can be displayed in the current interface in the form of text, or played in the form of speech, etc.

[0141] After generating a target multilingual model based on a monolingual pre-trained model, this application enables accurate and rapid speech recognition processing of user-input speech in different languages ​​based on this target multilingual model, thereby improving the accuracy and efficiency of speech recognition processing and enhancing the user experience.

[0142] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned target multilingual model, the target multilingual model can also be stored in a node of a blockchain.

[0143] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0144] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0145] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0147] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0148] Further reference Figure 3 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of a training device for a multilingual model, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0149] like Figure 3 As shown, the multilingual model training device 300 described in this embodiment includes: a first acquisition module 301, a construction module 302, a processing module 303, a second acquisition module 304, and a training module 305. Wherein:

[0150] The first acquisition module 301 is used to acquire a preset monolingual pre-trained model;

[0151] Module 302 is used to construct a first initial multilingual model based on the monolingual pre-trained model;

[0152] Processing module 303 is used to adjust the first initial multilingual model based on the first parameters of the first encoder in the monolingual pre-trained model to obtain a second initial multilingual model.

[0153] The second acquisition module 304 is used to acquire pre-collected training data;

[0154] The training module 305 is used to train the second initial multilingual model using the training data to obtain the target multilingual model.

[0155] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0156] In some optional implementations of this embodiment, the construction module 302 includes:

[0157] The first acquisition submodule is used to acquire the size of the first encoder of the monolingual pre-trained model; and...

[0158] The second acquisition submodule is used to acquire the number of layers of the first encoder;

[0159] The first construction submodule is used to construct an initial model that has the same size and number of layers as the first encoder.

[0160] The first generation submodule is used to perform a preset random initialization process on the initial model to obtain the first initial multilingual model.

[0161] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0162] In some optional implementations of this embodiment, the processing module 303 includes:

[0163] The third acquisition submodule is used to acquire the first model parameters of the monolingual pre-trained model, and to acquire the first parameters of the first encoder from the first model parameters; and...

[0164] The fourth acquisition submodule is used to acquire the second model parameters of the first initial multilingual model, and to acquire the second parameters of the second encoder in the first initial multilingual model from the second model parameters;

[0165] A replacement submodule is used to replace the second parameter in the first initial multilingual model with the first parameter.

[0166] The second generation submodule is used to keep the third parameter of the first decoder in the first initial multilingual model unchanged, and obtain the second initial multilingual model.

[0167] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned implementation method, and will not be repeated here.

[0168] In some optional implementations of this embodiment, the training module 305 includes:

[0169] The first processing submodule is used to freeze the fourth parameter of the third encoder within the second initial multilingual model;

[0170] The first training submodule is used to train the fifth parameter of the second decoder within the second initial multilingual model using the training data;

[0171] The second processing submodule is used to unfreeze the fourth parameter in the second initial multilingual model when it is determined that the first loss function corresponding to the first training task that uses the training data to train the fifth parameter has converged.

[0172] The second training submodule is used to train the fourth parameter in the third encoder and the fifth parameter in the second decoder together using the training data;

[0173] The first determining submodule is used to complete the training of the second initial multilingual model when it is determined that the second loss function corresponding to the second training task trained using the training data to train the fourth parameter and the fifth parameter has converged, and to use the trained second initial multilingual model as the target multilingual model.

[0174] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0175] In some optional implementations of this embodiment, the training module 305 includes:

[0176] The fifth acquisition submodule is used to acquire feature data within the third encoder obtained during the training of the fourth parameter of the third encoder using the training data;

[0177] The second construction submodule is used to construct a language recognition task based on a preset fully connected layer and the feature data;

[0178] The third processing submodule is used to optimize the fourth parameter based on the language recognition task;

[0179] The third generation submodule includes:

[0180] After completing the optimization processing of the fourth parameter based on the language recognition task, and determining that the second loss function corresponding to the second training task of training the fourth parameter and the fifth parameter using the training data has converged, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

[0181] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0182] In some optional implementations of this embodiment, the second acquisition module 304 includes:

[0183] The sixth acquisition submodule is used to acquire the pre-collected initial voice data;

[0184] The fourth processing submodule is used to process the initial speech data based on a preset language sampling algorithm to ensure that the number of speech segments in each language contained in the initial speech data is equal to a preset value, so as to obtain the processed initial speech data.

[0185] The second determining submodule is used to use the processed initial speech data as the training data.

[0186] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0187] In some optional implementations of this embodiment, the training device for the multilingual model further includes:

[0188] The first receiving module is used to receive the voice to be recognized;

[0189] The output module is used to input the speech to be recognized into the target multilingual model, and output the speech recognition result corresponding to the speech to be recognized through the target multilingual model;

[0190] The second receiving module is used to receive the speech recognition result returned by the target multilingual model;

[0191] The display module is used to display the speech recognition results.

[0192] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the training method of the multilingual model in the aforementioned embodiments, and will not be repeated here.

[0193] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0194] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0195] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0196] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for training methods of multilingual models. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0197] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, such as computer-readable instructions for executing the training method of the multilingual model.

[0198] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0199] Compared with the prior art, the embodiments of this application have the following main advantages:

[0200] In this embodiment, a preset monolingual pre-trained model is obtained, and then a first initial multilingual model is constructed based on the monolingual pre-trained model. Next, based on the first parameters of the first encoder within the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain a second initial multilingual model. Subsequently, pre-collected training data is acquired, and finally, the second initial multilingual model is trained using the training data to obtain a target multilingual model. This embodiment uses a monolingual pre-trained model and training data to train and generate a target multilingual model. Since the monolingual pre-trained model already contains initial values, it can accelerate the training speed of the multilingual model, reduce the training cost of the multilingual model, and effectively improve the training and generation rate of the multilingual model.

[0201] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the multilingual model training method described above.

[0202] Compared with the prior art, the embodiments of this application have the following main advantages:

[0203] In this embodiment, a preset monolingual pre-trained model is obtained, and then a first initial multilingual model is constructed based on the monolingual pre-trained model. Next, based on the first parameters of the first encoder within the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain a second initial multilingual model. Subsequently, pre-collected training data is acquired, and finally, the second initial multilingual model is trained using the training data to obtain a target multilingual model. This embodiment uses a monolingual pre-trained model and training data to train and generate a target multilingual model. Since the monolingual pre-trained model already contains initial values, it can accelerate the training speed of the multilingual model, reduce the training cost of the multilingual model, and effectively improve the training and generation rate of the multilingual model.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0205] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for training a multi-lingual model, the method comprising: Includes the following steps: Obtain a pre-trained monolingual model; A first initial multilingual model is constructed based on the aforementioned monolingual pre-trained model; Based on the first parameters of the first encoder in the monolingual pre-trained model, the first initial multilingual model is adjusted to obtain the second initial multilingual model. Acquire pre-collected training data; The second initial multilingual model is trained using the training data to obtain the target multilingual model; The step of adjusting the first initial multilingual model based on the first parameters of the encoder in the monolingual pre-trained model to obtain the second initial multilingual model specifically includes: Obtain the first model parameters of the monolingual pre-trained model, and obtain the first parameters of the first encoder from the first model parameters; and, Obtain the second model parameters of the first initial multilingual model, and obtain the second parameters of the second encoder within the first initial multilingual model from the second model parameters; Replace the second parameter in the first initial multilingual model with the first parameter; Keeping the third parameter of the first decoder in the first initial multilingual model unchanged, the second initial multilingual model is obtained; The step of training the second initial multilingual model using the training data to obtain the target multilingual model specifically includes: Freeze the fourth parameter of the third encoder within the second initial multilingual model; The training data is used to train the fifth parameter of the second decoder within the second initial multilingual model; When it is determined that the first loss function corresponding to the first training task that uses the training data to train the fifth parameter has converged, the freeze on the fourth parameter in the second initial multilingual model is lifted; The training data is used to jointly train the fourth parameter in the third encoder and the fifth parameter in the second decoder; When the second loss function corresponding to the second training task, which uses the training data to train the fourth parameter and the fifth parameter, converges, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

2. The training method for the multilingual model according to claim 1, characterized in that, The step of constructing the first initial multilingual model based on the monolingual pre-trained model specifically includes: Obtain the size of the first encoder of the monolingual pre-trained model; and, Obtain the layer number of the first encoder; Construct an initial model that has the same size and number of layers as the first encoder; The initial model is subjected to a preset random initialization process to obtain the first initial multilingual model.

3. The training method for the multilingual model according to claim 1, characterized in that, After the step of jointly training the fourth parameter in the third encoder and the fifth parameter in the second decoder using the training data, the method further includes: The feature data within the third encoder is obtained during the process of training the fourth parameter of the third encoder using the training data. A language recognition task is constructed based on a preset fully connected layer and the feature data; The fourth parameter is optimized based on the language identification task. The step of completing the training of the second initial multilingual model and using the trained second initial multilingual model as the target multilingual model when the second loss function corresponding to the second training task trained using the training data for the fourth and fifth parameters converges includes: After completing the optimization processing of the fourth parameter based on the language recognition task, and determining that the second loss function corresponding to the second training task of training the fourth parameter and the fifth parameter using the training data has converged, the training of the second initial multilingual model is completed, and the trained second initial multilingual model is used as the target multilingual model.

4. The training method for the multilingual model according to claim 1, characterized in that, The step of acquiring the pre-collected training data specifically includes: Acquire pre-collected initial voice data; The initial speech data is processed based on a preset language sampling algorithm to ensure that the number of speech items in each language contained in the initial speech data is equal to a preset value, thereby obtaining the processed initial speech data. The processed initial speech data is used as the training data.

5. The training method for the multilingual model according to claim 1, characterized in that, After the step of training the second initial multilingual model using the training data to obtain the target multilingual model, the method further includes: Receive the voice to be recognized; The speech to be recognized is input into the target multilingual model, and the target multilingual model outputs the speech recognition result corresponding to the speech to be recognized. Receive the speech recognition result returned by the target multilingual model; The speech recognition results are displayed.

6. A training device for a multilingual model, characterized in that, include: The first acquisition module is used to acquire a preset monolingual pre-trained model; The building module is used to build a first initial multilingual model based on the monolingual pre-trained model; The processing module is used to adjust the first initial multilingual model based on the first parameters of the first encoder in the monolingual pre-trained model to obtain a second initial multilingual model. The second acquisition module is used to acquire pre-collected training data; The training module is used to train the second initial multilingual model using the training data to obtain the target multilingual model; The processing module includes: The third acquisition submodule is used to acquire the first model parameters of the monolingual pre-trained model, and to acquire the first parameters of the first encoder from the first model parameters; and The fourth acquisition submodule is used to acquire the second model parameters of the first initial multilingual model, and to acquire the second parameters of the second encoder in the first initial multilingual model from the second model parameters; A replacement submodule is used to replace the second parameter in the first initial multilingual model with the first parameter. The second generation submodule is used to keep the third parameter of the first decoder in the first initial multilingual model unchanged, and obtain the second initial multilingual model. The training module includes: The first processing submodule is used to freeze the fourth parameter of the third encoder within the second initial multilingual model; The first training submodule is used to train the fifth parameter of the second decoder within the second initial multilingual model using the training data; The second processing submodule is used to unfreeze the fourth parameter in the second initial multilingual model when it is determined that the first loss function corresponding to the first training task that uses the training data to train the fifth parameter has converged. The second training submodule is used to train the fourth parameter in the third encoder and the fifth parameter in the second decoder together using the training data; The first determining submodule is used to complete the training of the second initial multilingual model when it is determined that the second loss function corresponding to the second training task trained using the training data to train the fourth parameter and the fifth parameter has converged, and to use the trained second initial multilingual model as the target multilingual model.

7. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the training method for a multilingual model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the training method for the multilingual model as described in any one of claims 1 to 5.

Images