Model training method, translation method, device, electronic device and storage medium

Abstract

The present invention relates to artificial intelligence technology in the financial field and discloses a model training method and a translation method. The model training method comprises: obtaining training data; concatenating a target sentence and multiple context sentences included in the training data to obtain a first sequence; determining a target loss function corresponding to a context-aware model based on the first sequence; and iteratively training the context-aware model based on the target loss function to obtain a target model. The present invention also relates to a translation method. The present invention also provides a model training device, a translation device, an electronic device, and a readable storage medium. The present invention can fully utilize context information to train a context-aware model.

Description

Technical Field

[0001] The present invention relates to the fields of finance and artificial intelligence technology, and in particular to a model training method, a translation method, a device, an electronic device, and a storage medium. Background Art

[0002] In the financial field, especially in business scenarios with foreign customers, in order to ensure smooth communication, it is usually necessary to translate the corpus output by foreign customers.

[0003] Currently, to ensure translation accuracy, corpus is typically fed into a context-aware model, which then translates the target sentence based on its context. However, existing context-aware models fail to fully utilize contextual information, resulting in unrelated sentences and significant semantic errors, which reduces the accuracy of the translation results. Summary of the Invention

[0004] The present invention provides a model training method, a translation method, an apparatus, an electronic device and a storage medium, the main purpose of which is to improve the accuracy of translation results.

[0005] To achieve the above object, the present invention provides a model training method, which includes:

[0006] Acquire training data; the training data includes a target sentence and multiple context sentences, wherein the context sentences are sentences in the training data that have a contextual relationship with the target sentence;

[0007] Concatenating the target sentence and multiple context sentences included in the training data to obtain a first sequence;

[0008] Determining a target loss function corresponding to a context-aware model based on the first sequence;

[0009] The context-aware model is iteratively trained based on the target loss function to obtain a target model.

[0010] Optionally, the context-aware model includes an encoder and a decoder, and the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder;

[0011] The determining, based on the first sequence, a target loss function corresponding to the context-aware model includes:

[0012] Determining a first loss function corresponding to an encoder of the context-aware model based on the first sequence;

[0013] Determine a second sequence based on the output result of the encoder; the second sequence includes a plurality of second subsequences, each second subsequence is associated with a sentence;

[0014] Determine a second loss function corresponding to the encoder of the context-aware model based on the sentences associated with each second subsequence and the sentences included in the training data.

[0015] Optionally, the first sequence includes multiple first subsequences, each first subsequence is associated with a sentence, and each first subsequence associated with a context sentence corresponds to a separation identifier;

[0016] The determining, based on the first sequence, a first loss function corresponding to the encoder of the context-aware model includes:

[0017] Obtaining a first sub-loss function corresponding to a context sentence associated with each first sub-sequence in the encoder, and a second sub-loss function corresponding to a target sentence associated with the first sequence in the encoder;

[0018] Determine a third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first subsequence associated with each context sentence;

[0019] The sum of the second sub-loss function corresponding to the target sentence in the encoder and the third sub-loss function corresponding to each context sentence in the encoder is determined as the first loss function corresponding to the encoder of the context-aware model.

[0020] Optionally, determining a third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and a separation identifier corresponding to the first subsequence associated with each context sentence includes:

[0021] Inputting the separation sequence represented by each separation marker into the fully connected layer of the context-aware model to obtain a dimensional sequence;

[0022] Inputting the dimension sequence into a normalized exponential function to obtain a weight value corresponding to each context sentence;

[0023] The product of the weight value corresponding to each context sentence and the first sub-loss function corresponding to each context sentence in the encoder is determined as the third sub-loss function corresponding to each context sentence in the encoder.

[0024] Optionally, determining the second sequence based on the output result of the encoder includes:

[0025] Obtaining a target sentence and multiple context sentences output by the encoder; wherein the target sentence and multiple context sentences output by the encoder correspond to a language different from the language corresponding to the target sentence and multiple context sentences included in the training data;

[0026] The target sentence output by the encoder and multiple context sentences are concatenated to obtain a second sequence.

[0027] In order to solve the above problem, the present invention further provides a translation method, which comprises:

[0028] Obtaining a text to be translated; the text to be translated includes multiple sentences;

[0029] Inputting the text to be translated into the target model described above to obtain a translation result corresponding to each sentence in the text to be translated;

[0030] Based on the translation result corresponding to each sentence, a translation result corresponding to the text to be translated is determined.

[0031] In order to solve the above problems, the present invention further provides a model training device, which includes:

[0032] An acquisition module is configured to acquire training data; the training data includes a target sentence and a plurality of context sentences, wherein the context sentences are sentences in the training data that have a contextual relationship with the target sentence;

[0033] a processing module, configured to concatenate the target sentence and the plurality of context sentences included in the training data to obtain a first sequence;

[0034] A determination module, configured to determine a target loss function corresponding to a context-aware model based on the first sequence;

[0035] A training module is used to iteratively train the context-aware model based on the target loss function to obtain a target model.

[0036] In order to solve the above problem, the present invention further provides a translation device, comprising:

[0037] An acquisition module, configured to acquire a text to be translated; the text to be translated includes a plurality of sentences;

[0038] A processing module, configured to input the text to be translated into the target model described above, and obtain a translation result corresponding to each sentence in the text to be translated;

[0039] The determination module is configured to determine a translation result corresponding to the text to be translated based on the translation result corresponding to each sentence.

[0040] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0041] a memory storing at least one computer program; and

[0042] The processor executes the computer program stored in the memory to implement the above-mentioned model training method, or implements the above-mentioned translation method.

[0043] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one computer program. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned model training method or the above-mentioned translation method.

[0044] The present invention obtains training data; the training data includes a target sentence and multiple context sentences, and the context sentence is a sentence in the training data that has a contextual relationship with the target sentence; the target sentence and multiple context sentences included in the training data are spliced ​​to obtain a first sequence; based on the first sequence, the target loss function corresponding to the context-aware model is determined; based on the target loss function, the context-aware model is iteratively trained to obtain a target model. In an embodiment of the present invention, since the above-mentioned first sequence is obtained by splicing the target sentence and multiple context sentences, the above-mentioned first sequence can reflect the context information of the target sentence, and then in the training process of the context-aware model, the target loss function is determined based on the first sequence, and the context information is fully utilized to train the context-aware model to obtain the target model. In other words, the above-mentioned target model is obtained by training using the context information of each sentence, so as to ensure that there is a contextual relationship between the sentences output by the target model, thereby improving the accuracy of the translation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of a model training method provided in one embodiment of the present invention;

[0046] Figure 2 A flowchart of a translation method provided by an embodiment of the present invention;

[0047] Figure 3 A functional module diagram of a model training device provided by one embodiment of the present invention;

[0048] Figure 4 A functional module diagram of a translation device provided by one embodiment of the present invention;

[0049] Figure 5 A schematic structural diagram of an electronic device for implementing the model training method and the translation method provided in one embodiment of the present invention.

[0050] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] An embodiment of the present invention provides a model training method. The execution subject of the model training method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the model training method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0053] Reference Figure 1 , which is a flow chart of a model training method provided by one embodiment of the present invention.

[0054] In this embodiment, the model training method includes the following steps S1-S5:

[0055] S1. Obtain training data.

[0056] Optionally, the model training method provided in this embodiment can be applied to a model training device, which is communicatively connected to an external electronic device. In one optional implementation, the electronic device sends training data to the model training device, so that the model training device obtains the training data. In another optional implementation, the developer performs corresponding operations on the model training device to generate training time, so that the model training device obtains the training data.

[0057] The training data includes a target sentence and multiple context sentences. The target sentence can be understood as a sentence among the multiple sentences included in the training data, and the context sentence is a sentence in the training data that has a contextual relationship with the target sentence.

[0058] S2. Concatenate the target sentence and multiple context sentences included in the training data to obtain a first sequence.

[0059] In this step, after obtaining the training data, the target sentence and multiple context sentences included in the training data can be concatenated to obtain a first sequence. It should be understood that the first sequence can reflect the context information of the target sentence.

[0060] The first sequence includes multiple first subsequences, and each first subsequence is associated with a sentence in the training data.

[0061] S3. Based on the first sequence, determine the target loss function corresponding to the context-aware model.

[0062] In this step, after obtaining the first sequence, a target loss function corresponding to the context-aware model can be determined based on the first sequence. The target loss function is a loss function for iteratively training the context-aware model.

[0063] For details on how to determine the implementation of the target loss function corresponding to the context-aware model, please refer to the subsequent embodiments.

[0064] S4. Iteratively train the context-aware model based on the target loss function to obtain a target model.

[0065] In this step, after determining the target loss function, the context-aware model is iteratively trained using the target loss function to obtain the target model.

[0066] An optional implementation method is that during the iterative training process, if the function value of the target loss function is lower than a preset threshold value under consecutive preset number of iterations, it is determined that the context-aware model training is completed and the target model is obtained.

[0067] Another optional implementation is that during the iterative training process, if the number of iterations reaches a preset number, it is determined that the context-aware model training is completed and the target model is obtained.

[0068] In this embodiment of the present invention, since the first sequence is obtained by concatenating the target sentence and multiple context sentences, it can reflect the contextual information of the target sentence. Consequently, during the training of the context-aware model, a target loss function is determined based on the first sequence, fully utilizing the contextual information to train the context-aware model and obtain a target model. In other words, the target model is trained using the contextual information of each sentence, thereby ensuring that contextual relationships exist between the sentences output by the target model, thereby improving the accuracy of the translation results.

[0069] In addition, in business scenarios targeting foreign customers, since the translation method provided by the embodiment of the present invention can provide better translation effects for the text to be translated, the translation results obtained can more accurately describe the user portrait.

[0070] In an embodiment of the present invention, the context-aware model includes an encoder and a decoder, and the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder;

[0071] The determining, based on the first sequence, a target loss function corresponding to the context-aware model includes:

[0072] Determining a first loss function corresponding to an encoder of the context-aware model based on the first sequence;

[0073] Determine a second sequence based on the output result of the encoder; the second sequence includes a plurality of second subsequences, each second subsequence is associated with a sentence;

[0074] Determine a second loss function corresponding to the encoder of the context-aware model based on the sentences associated with each second subsequence and the sentences included in the training data.

[0075] It should be understood that the context-aware model includes an encoder and a decoder, and the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder.

[0076] In this embodiment, after determining the first sequence, a first loss function corresponding to the encoder of the context-aware model can be determined based on the first sequence. For specific implementation methods of determining the first loss function, please refer to the subsequent embodiments.

[0077] It should be understood that the output result of the encoder is a target sentence and multiple context sentences, wherein the language corresponding to the target sentence and multiple context sentences output by the encoder is different from the language corresponding to the target sentence and multiple context sentences included in the training data.

[0078] In this embodiment, the second sequence can be determined based on the output of the encoder, wherein the second sequence includes multiple second subsequences, each second subsequence being associated with a sentence. For specific implementations of how to determine the second sequence, please refer to the subsequent embodiments.

[0079] In this embodiment, the second loss function corresponding to the decoder of the context-aware model can be determined based on the following formula:

[0080]

[0081] Among them, L represents the second loss function, Indicates the sentences included in the training data, x i represents the i-th sentence included in the training data, N is the number of sentences included in the training data, The statement representing the second subsequence association, y i represents the i-th statement associated with the second subsequence, and M is the number of statements associated with the second subsequence.

[0082] In this embodiment, the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder, wherein the above-mentioned first loss function is determined based on the first sequence representing the context information, so the above-mentioned loss function can reflect the context information.

[0083] In the embodiment of the present invention, the first sequence includes multiple first subsequences, each first subsequence is associated with a sentence, and each first subsequence associated with a context sentence corresponds to a separation identifier;

[0084] The determining, based on the first sequence, a first loss function corresponding to the encoder of the context-aware model includes:

[0085] Obtaining a first sub-loss function corresponding to a context sentence associated with each first sub-sequence in the encoder, and a second sub-loss function corresponding to a target sentence associated with the first sequence in the encoder;

[0086] Determine a third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first subsequence associated with each context sentence;

[0087] The sum of the second sub-loss function corresponding to the target sentence in the encoder and the third sub-loss function corresponding to each context sentence in the encoder is determined as the first loss function corresponding to the encoder of the context-aware model.

[0088] It should be understood that each sentence corresponds to a loss function in the encoder, where the loss function corresponding to the context sentence in the encoder can be called the first sub-loss function, and the loss function corresponding to the target sentence in the encoder can be called the second sub-loss function.

[0089] In this embodiment, a first sub-loss function corresponding to the context sentence associated with each first sub-sequence in the encoder and a second sub-loss function corresponding to the target sentence associated with the first sequence in the encoder are obtained. Optionally, the first sub-loss function and the second sub-loss function are preset loss functions.

[0090] As described above, each first subsequence associated with a context statement corresponds to a separator identifier, optionally a separator. In this embodiment, a third sub-loss function corresponding to each context statement in the encoder can be determined based on the first sub-loss function corresponding to each context statement in the encoder and the separator identifier corresponding to the first subsequence associated with each context statement. For a specific implementation method, please refer to the subsequent embodiments.

[0091] Furthermore, the first loss function corresponding to the encoder of the context-aware model can be determined by the following formula:

[0092]

[0093] in, Represents the first loss function corresponding to the encoder, α j-k L(x j-k ,y j-k ) and α j-k+1 L(x j -k+1 ,y j-k+1 ) represent the first sub-loss function, L(x j ,y j ) represents the second sub-loss function, α j-k and α j-k+1 All represent weight values, (x j-k ,y j-k ) represents the jKth context statement, (x j-k+1 ,y j-k+1 ) represents the K+1th context sentence, (x j ,y j ) represents the target sentence, and K is a positive integer greater than 1.

[0094] In an embodiment of the present invention, determining the third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first subsequence associated with each context sentence includes:

[0095] Inputting the separation sequence represented by each separation marker into the fully connected layer of the context-aware model to obtain a dimensional sequence;

[0096] Inputting the dimension sequence into a normalized exponential function to obtain a weight value corresponding to each context sentence;

[0097] The product of the weight value corresponding to each context sentence and the first sub-loss function corresponding to each context sentence in the encoder is determined as the third sub-loss function corresponding to each context sentence in the encoder.

[0098] As described above, each first subsequence corresponds to a separation identifier.

[0099] In this embodiment, the separation sequence represented by each separation mark can be input into the fully connected layer of the context-aware model to obtain a dimension sequence. The dimension of the fully connected layer is d*1, where d represents the preset hidden state dimension; the separation sequence can be expressed as {s j-K ,s j-K+1 ,…,s j-1}.

[0100] Furthermore, the dimension sequence is input into the normalized exponential (softmax) function to obtain the weight value corresponding to each context sentence. Optionally, the weight value corresponding to each context sentence can be expressed as α = (α j-K ,α j-K+1 ,…,α j-1 ).

[0101] The product of the weight value corresponding to each context sentence and the first sub-loss function corresponding to each context sentence in the encoder is determined as the third sub-loss function corresponding to each context sentence in the encoder.

[0102] In this embodiment, different weight values ​​are set for different context sentences, so as to highlight some context sentences that provide effective information for the translation of the target sentence among multiple context sentences, so as to make the translation result of the target sentence more accurate.

[0103] In addition, the above weight values ​​are generated during the model training process, that is, the specific values ​​of the above weight values ​​are not subject to human interference, thereby ensuring the accuracy of the weight values.

[0104] In the embodiment of the present invention, determining the second sequence based on the output result of the encoder includes:

[0105] Obtaining a target sentence and multiple context sentences output by the encoder; wherein the target sentence and multiple context sentences output by the encoder correspond to a language different from the language corresponding to the target sentence and multiple context sentences included in the training data;

[0106] The target sentence output by the encoder and multiple context sentences are concatenated to obtain a second sequence.

[0107] The above second sequence can be understood as the input of the decoder.

[0108] In this embodiment, the target sentence and multiple context sentences output by the encoder are obtained. It should be understood that the language corresponding to the target sentence and multiple context sentences output by the encoder is different from the language corresponding to the target sentence and multiple context sentences included in the training data.

[0109] For example, in the scenario of applying the target model for English translation, the language corresponding to the sentences included in the training data is English, and the sentences output by the encoder are in Chinese.

[0110] Furthermore, the target sentence output by the encoder and the multiple context sentences are concatenated to obtain a second sequence. Optionally, the second sequence includes multiple subsequences, each subsequence is associated with a sentence, and each subsequence associated with a context sentence corresponds to a separation marker.

[0111] An embodiment of the present invention provides a translation method. The execution subject of the translation method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the translation method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0112] Reference Figure 2 FIG. 1 is a flow chart of a translation method according to an embodiment of the present invention. In this embodiment, the translation method includes the following steps S5-S7:

[0113] S5. Obtain the text to be translated.

[0114] In this step, the translation method can be applied to a translation device that can be communicatively connected to an external electronic device. In one alternative embodiment, the external electronic device sends the text to be translated to the translation device, so that the translation device obtains the text to be translated. In another alternative embodiment, a relevant person performs corresponding operations on the translation device to generate the text to be translated, so that the translation device obtains the text to be translated.

[0115] The text to be translated includes multiple sentences.

[0116] S6. Input the text to be translated into a target model to obtain a translation result corresponding to each sentence in the text to be translated.

[0117] In this step, the text to be translated is input into the trained target model, and the target model processes the text to be translated and outputs the translation result corresponding to each sentence in the text to be translated.

[0118] S7. Determine the translation result corresponding to the text to be translated based on the translation result corresponding to each sentence.

[0119] In this step, the translation results corresponding to each sentence can be spliced ​​to obtain the translation result corresponding to the text to be translated.

[0120] In this embodiment, the trained target model is applied to perform text processing on the text to be translated. Since the target model makes full use of context information during the training process, the translation result corresponding to the text to be translated obtained by applying the target model can accurately represent the context of each sentence, thereby improving the accuracy of the translation result.

[0121] like Figure 3 , which is a functional module diagram of a model training device provided by one embodiment of the present invention.

[0122] The model training device 100 of the present invention can be installed in an electronic device. Depending on the functions to be implemented, the model training device 100 may include an acquisition module 101, a processing module 102, a determination module 103, and a training module 104. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and can perform a fixed function, and is stored in the memory of the electronic device.

[0123] In this embodiment, the functions of each module / unit are as follows:

[0124] The acquisition module 101 is used to acquire training data; the training data includes a target sentence and multiple context sentences, and the context sentences are sentences in the training data that have a contextual relationship with the target sentence;

[0125] The processing module 102 is configured to concatenate the target sentence and multiple context sentences included in the training data to obtain a first sequence;

[0126] The determining module 103 is configured to determine a target loss function corresponding to a context-aware model based on the first sequence;

[0127] The training module 104 is configured to iteratively train the context-aware model based on the target loss function to obtain a target model.

[0128] In detail, the context-aware model includes an encoder and a decoder, and the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder;

[0129] The determining module 103 is specifically configured to:

[0130] Determining a first loss function corresponding to an encoder of the context-aware model based on the first sequence;

[0131] Determine a second sequence based on the output result of the encoder; the second sequence includes a plurality of second subsequences, each second subsequence is associated with a sentence;

[0132] Determine a second loss function corresponding to the encoder of the context-aware model based on the sentences associated with each second subsequence and the sentences included in the training data.

[0133] In detail, the first sequence includes a plurality of first subsequences, each first subsequence is associated with a sentence, and each first subsequence associated with a context sentence corresponds to a separation identifier;

[0134] The determining module 103 is further specifically configured to:

[0135] Obtaining a first sub-loss function corresponding to a context sentence associated with each first sub-sequence in the encoder, and a second sub-loss function corresponding to a target sentence associated with the first sequence in the encoder;

[0136] Determine a third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first subsequence associated with each context sentence;

[0137] The sum of the second sub-loss function corresponding to the target sentence in the encoder and the third sub-loss function corresponding to each context sentence in the encoder is determined as the first loss function corresponding to the encoder of the context-aware model.

[0138] In detail, the determining module 103 is further configured to:

[0139] Inputting the separation sequence represented by each separation marker into the fully connected layer of the context-aware model to obtain a dimensional sequence;

[0140] Inputting the dimension sequence into a normalized exponential function to obtain a weight value corresponding to each context sentence;

[0141] The product of the weight value corresponding to each context sentence and the first sub-loss function corresponding to each context sentence in the encoder is determined as the third sub-loss function corresponding to each context sentence in the encoder.

[0142] In detail, the determining module 103 is further configured to:

[0143] Obtaining a target sentence and multiple context sentences output by the encoder; wherein the target sentence and multiple context sentences output by the encoder correspond to a language different from the language corresponding to the target sentence and multiple context sentences included in the training data;

[0144] The target sentence output by the encoder and multiple context sentences are concatenated to obtain a second sequence.

[0145] In this embodiment of the present invention, since the first sequence is obtained by concatenating the target sentence and multiple context sentences, it can reflect the contextual information of the target sentence. Consequently, during the training of the context-aware model, a target loss function is determined based on the first sequence, fully utilizing the contextual information to train the context-aware model and obtain a target model. In other words, the target model is trained using the contextual information of each sentence, thereby ensuring that contextual relationships exist between the sentences output by the target model, thereby improving the accuracy of the translation results.

[0146] like Figure 4 FIG. 1 is a functional module diagram of a translation device provided by an embodiment of the present invention.

[0147] The translation device 200 of the present invention can be installed in an electronic device. Depending on the functionality to be implemented, the translation device 200 may include an acquisition module 201, a processing module 202, and a determination module 203. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the electronic device's memory.

[0148] In this embodiment, the functions of each module / unit are as follows:

[0149] The acquisition module 201 is used to acquire a text to be translated; the text to be translated includes multiple sentences;

[0150] The processing module 202 is used to input the text to be translated into the target model described above to obtain a translation result corresponding to each sentence in the text to be translated;

[0151] The determination module 203 is configured to determine a translation result corresponding to the text to be translated based on the translation result corresponding to each sentence.

[0152] In this embodiment, the trained target model is applied to perform text processing on the text to be translated. Since the target model makes full use of context information during the training process, the translation result corresponding to the text to be translated obtained by applying the target model can accurately represent the context of each sentence, thereby improving the accuracy of the translation result.

[0153] like Figure 5 , which is a schematic diagram of the structure of an electronic device for implementing the model training method and the translation method provided by one embodiment of the present invention.

[0154] The electronic device may include a processor 10, a memory 11, a communication interface 12 and a bus 13, and may also include computer programs stored in the memory 11 and executable on the processor 10, such as a model training program and a translation program.

[0155] Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example: SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the target object recognition program, etc., but can also be used to temporarily store data that has been output or is to be output.

[0156] In some embodiments, the processor 10 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or executes programs or modules (such as object recognition programs) stored in the memory 11 and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0157] The communication interface 12 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0158] The bus 13 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 13 may be divided into an address bus, a data bus, a control bus, etc. The bus 13 is configured to enable communication between the memory 11 and at least one processor 10.

[0159] Figure 5 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 5 The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0160] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0161] Furthermore, the electronic device may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.

[0162] Optionally, the electronic device may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0163] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0164] The object recognition program stored in the memory 11 of the electronic device is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0165] Get training data;

[0166] Concatenating the target sentence and multiple context sentences included in the training data to obtain a first sequence;

[0167] Determining a target loss function corresponding to a context-aware model based on the first sequence;

[0168] The context-aware model is iteratively trained based on the target loss function to obtain a target model.

[0169] Alternatively, you can implement:

[0170] Get the text to be translated;

[0171] Inputting the text to be translated into the target model to obtain a translation result corresponding to each sentence in the text to be translated;

[0172] Based on the translation result corresponding to each sentence, a translation result corresponding to the text to be translated is determined.

[0173] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0174] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0175] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0176] Get training data;

[0177] Concatenating the target sentence and multiple context sentences included in the training data to obtain a first sequence;

[0178] Determining a target loss function corresponding to a context-aware model based on the first sequence;

[0179] The context-aware model is iteratively trained based on the target loss function to obtain a target model.

[0180] Alternatively, you can implement:

[0181] Get the text to be translated;

[0182] Inputting the text to be translated into the target model to obtain a translation result corresponding to each sentence in the text to be translated;

[0183] Based on the translation result corresponding to each sentence, a translation result corresponding to the text to be translated is determined.

[0184] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0185] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0186] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0187] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0188] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0189] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0190] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0191] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0192] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A model training method, characterized in that: include: Get training data; The training data includes a target sentence and a plurality of context sentences, wherein the context sentences are sentences in the training data that have a contextual relationship with the target sentence; Concatenating the target sentence and multiple context sentences included in the training data to obtain a first sequence; Determining a target loss function corresponding to a context-aware model based on the first sequence; Iteratively training the context-aware model based on the target loss function to obtain a target model; The context-aware model includes an encoder and a decoder, and the target loss function includes a first loss function corresponding to the encoder and a second loss function corresponding to the decoder; The determining, based on the first sequence, a target loss function corresponding to the context-aware model includes: determining, based on the first sequence, a first loss function corresponding to an encoder of the context-aware model; determining a second sequence based on an output result of the encoder; the second sequence includes a plurality of second subsequences, each second subsequence being associated with a sentence; and determining, based on the sentence associated with each second subsequence and the sentence included in the training data, a second loss function corresponding to the encoder of the context-aware model; The first sequence includes a plurality of first subsequences, each first subsequence is associated with a sentence, and each first subsequence associated with a context sentence corresponds to a separation identifier; The method of determining the first loss function corresponding to the encoder of the context-aware model based on the first sequence includes: obtaining the first sub-loss function corresponding to the context sentence associated with each first sub-sequence in the encoder, and the second sub-loss function corresponding to the target sentence associated with the first sequence in the encoder; determining the third sub-loss function corresponding to each context sentence in the encoder based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first sub-sequence associated with each context sentence; and determining the sum of the second sub-loss function corresponding to the target sentence in the encoder and the third sub-loss function corresponding to each context sentence in the encoder as the first loss function corresponding to the encoder of the context-aware model.

2. The method according to claim 1, wherein The determining, based on the first sub-loss function corresponding to each context sentence in the encoder and the separation identifier corresponding to the first subsequence associated with each context sentence, a third sub-loss function corresponding to each context sentence in the encoder includes: Inputting the separation sequence represented by each separation marker into the fully connected layer of the context-aware model to obtain a dimensional sequence; Inputting the dimension sequence into a normalized exponential function to obtain a weight value corresponding to each context sentence; The product of the weight value corresponding to each context sentence and the first sub-loss function corresponding to each context sentence in the encoder is determined as the third sub-loss function corresponding to each context sentence in the encoder.

3. The method according to claim 1, wherein The determining the second sequence based on the output result of the encoder comprises: Obtaining a target sentence and multiple context sentences output by the encoder; wherein the target sentence and multiple context sentences output by the encoder correspond to a language different from the language corresponding to the target sentence and multiple context sentences included in the training data; The target sentence output by the encoder and multiple context sentences are concatenated to obtain a second sequence.

4. A translation method, applicable to a target model obtained based on the model training method according to any one of claims 1 to 3, characterized in that: include: Acquire a text to be translated, where the text to be translated includes multiple sentences; Inputting the text to be translated into the target model to obtain a translation result corresponding to each sentence in the text to be translated; Based on the translation result corresponding to each sentence, a translation result corresponding to the text to be translated is determined.

5. A model training device for implementing the model training method according to any one of claims 1 to 3, characterized in that: The device comprises: An acquisition module is configured to acquire training data; the training data includes a target sentence and a plurality of context sentences, wherein the context sentences are sentences in the training data that have a contextual relationship with the target sentence; a processing module, configured to concatenate the target sentence and the plurality of context sentences included in the training data to obtain a first sequence; A determination module, configured to determine a target loss function corresponding to a context-aware model based on the first sequence; A training module is used to iteratively train the context-aware model based on the target loss function to obtain a target model.

6. A translation device, characterized in that: include: The acquisition module is used to obtain the text to be translated; The text to be translated includes multiple sentences; A processing module, configured to input the text to be translated into a target model obtained by the model training method according to any one of claims 1 to 3, and obtain a translation result corresponding to each sentence in the text to be translated; The determination module is configured to determine a translation result corresponding to the text to be translated based on the translation result corresponding to each sentence.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model training method as described in any one of claims 1 to 3, or execute the translation method as described in claim 4.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the model training method according to any one of claims 1 to 3, or implements the translation method according to claim 4.

Images