Oral-written conversion method and device based on reinforcement learning, equipment and medium

Abstract

The application provides a spoken-to-written conversion method and device based on reinforcement learning, equipment and medium, wherein the method comprises: obtaining a spoken text; inputting the spoken text into a conversion model to obtain a written text output by the conversion model; the conversion model is obtained by reinforcement learning, taking the editing operation of each word in the sample spoken text as an action, and taking the semantic consistency between the sample written text obtained by performing the editing operation and the sample spoken text and / or the written degree of the sample written text as a reward. The method, device, equipment and medium provided by the application break the limitation of insufficient labeled data in the process of reinforcement learning, and the semantic consistency and written degree give high-level and interpretable rewards. The conversion model obtained by application of the conversion model ensures the reliability and interpretability of the conversion from spoken text to written text.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, device, and medium for converting spoken to written language based on reinforcement learning. Background Technology

[0002] Because people use language differently when speaking and writing, and because there may be syntactic and grammatical errors, lack of fluency, and noise during speech recording, the accessibility and readability of spoken text obtained from speech recognition can all be affected.

[0003] Therefore, converting spoken text into written text is crucial for reducing the difficulty of understanding the text content. However, insufficient labeled data and poor interpretability are currently significant research challenges in spoken-to-written text conversion studies. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for spoken-to-written language conversion based on reinforcement learning, in order to address the shortcomings of insufficient text conversion annotation data and poor interpretability in existing technologies.

[0005] This invention provides a method for converting spoken to written language based on reinforcement learning, comprising:

[0006] Obtain spoken text;

[0007] The spoken text is input into the conversion model to obtain the written text output by the conversion model;

[0008] The transformation model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text obtained by the editing operations and the sample spoken text and / or the degree of formalization of the sample written text.

[0009] According to the present invention, a method for converting spoken to written language based on reinforcement learning is provided, wherein the steps for obtaining the conversion model include:

[0010] The editing operation of the previous word in the sample spoken text and the semantic features of the current word in the sample spoken text are input into the strategy model to obtain the editing operation of the current word output by the strategy model. The next word of the current word is returned as the current word to obtain the editing operation, until the editing operation of each word in the sample spoken text is obtained.

[0011] Based on the editing operations of each word in the sample spoken text, the sample written text is determined;

[0012] The reward is determined based on the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text;

[0013] Based on the reward, the policy model is subjected to reinforcement learning to obtain the transformation model.

[0014] According to a reinforcement learning-based spoken-to-written language conversion method provided by the present invention, the step of determining the reward based on the semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text includes:

[0015] The first reward is determined based on the editing operations of each word in the sample spoken text and the editing tags of the sample spoken text;

[0016] The reward is determined based on the first reward, the degree of semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text.

[0017] According to the present invention, a method for converting spoken language to written language based on reinforcement learning includes the following steps for obtaining the semantic features of each word segment in the sample spoken text:

[0018] The sample spoken text is input into the language model to obtain the semantic features of each word segment in the sample spoken text output by the language sound model;

[0019] The language model is obtained by supervised classification learning and fine-tuning based on edit tags of spoken text segmentation; or, the language model is an unsupervised pre-trained language model.

[0020] According to a reinforcement learning-based spoken-to-written language conversion method provided by the present invention, the step of obtaining the semantic consistency degree includes:

[0021] The sample spoken text and the sample written text are input into the consistency scoring model to obtain the semantic consistency degree output by the consistency scoring model;

[0022] The consistency scoring model is trained based on positive and negative sample pairs. The positive sample pairs include a preset spoken text and a preset written text with semantic consistency. The negative sample pairs include the preset spoken text and a perturbed text obtained by perturbing the preset spoken text.

[0023] According to a reinforcement learning-based spoken-to-written language conversion method provided by the present invention, the step of obtaining the degree of writtenization includes:

[0024] The sample written text is input into the writtenness scoring model to obtain the writtenness level output by the writtenness scoring model;

[0025] The written proficiency scoring model is a classification model trained on preset spoken texts and preset written texts.

[0026] The present invention also provides a speech-to-written language conversion device based on reinforcement learning, comprising:

[0027] The acquisition unit is used to acquire spoken text;

[0028] A conversion unit is used to input the spoken text into a conversion model to obtain the written text output by the conversion model;

[0029] The transformation model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text obtained by the editing operations and the sample spoken text and / or the degree of formalization of the sample written text.

[0030] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the speech-to-written language conversion method based on reinforcement learning as described above.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the speech-to-written language conversion method based on reinforcement learning as described above.

[0032] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the speech-to-written language conversion method based on reinforcement learning as described above.

[0033] The present invention provides a method, apparatus, device, and medium for converting spoken to written language based on reinforcement learning. It uses the editing operations of each word segment in a sample spoken text as actions, and the degree of semantic consistency and / or the degree of formalization of the sample written text obtained from the editing operations as rewards. Reinforcement learning is then performed to obtain a conversion model. This reinforcement learning process overcomes the limitations of insufficient labeled data, and the degree of semantic consistency and formalization provides high-level, interpretable rewards. Applying the resulting conversion model to text conversion ensures the reliability and interpretability of the spoken-to-written text conversion. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the spoken-to-written language conversion method based on reinforcement learning provided by the present invention.

[0036] Figure 2 This is a flowchart illustrating the method for obtaining the conversion model provided by the present invention;

[0037] Figure 3 This is a schematic diagram of reinforcement learning provided by the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of the speech-to-written language conversion device based on reinforcement learning provided by the present invention;

[0039] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] In research on spoken-to-written text conversion, insufficient labeled data and poor interpretability are significant challenges. To address this issue, this invention provides a spoken-to-written language conversion method based on reinforcement learning. Figure 1 This is a flowchart illustrating the spoken-to-written language conversion method based on reinforcement learning provided by the present invention, as shown below. Figure 1 As shown, the method includes:

[0042] Step 110: Obtain the spoken text.

[0043] Specifically, spoken text refers to spoken text that requires text conversion. Spoken text can be transcribed text generated by an automatic speech recognition system. For example, it can be text recorded and transcribed at a meeting, text transcribed in real time during clinical diagnosis, or text obtained by transcribing customer inquiries through the intelligent transcription function of a call center in a smart customer service scenario.

[0044] Step 120: Input the spoken text into the conversion model to obtain the written text output by the conversion model;

[0045] The transformation model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text obtained by the editing operations and the sample spoken text and / or the degree of formalization of the sample written text.

[0046] Considering the current lack of labeled data for text conversion from spoken to written language, this invention provides a modeling method that achieves autonomous evolution from spoken to written language through reinforcement learning. This modeling method does not rely solely on labeled data, thereby mitigating the impact of insufficient labeled data on the modeling effect. Furthermore, this modeling method is interpretable from the perspective of language mechanisms, thus making the subsequent text conversion more reliable.

[0047] In this process, the modeling problem of the translation from spoken to written language can be defined as a reward-based reinforcement learning problem. Under the reinforcement learning problem, the editing operations on each token in the sample spoken text can be used as reinforcement learning actions. These editing operations can be keeping (KEEP) or deleting (DELETE) a single token. During reinforcement learning, a policy model can be constructed first—that is, a model whose parameters are initialized. This policy model is then used to predict the editing operations for each token in the sample spoken text, thus obtaining the sample written text obtained by editing the sample spoken text based on the predicted editing operations.

[0048] To evaluate the quality of the estimated editing operations and calculate rewards, this embodiment of the invention uses semantic consistency and / or formalization as high-level signals for reward evaluation. Semantic consistency assesses the semantic consistency between the sample spoken and written texts. It is understood that the conversion from spoken to written language should not alter the semantics of the text itself; a higher degree of semantic consistency results in a higher reward. Formalization assesses the probability that the sample written text is considered formal written language; a higher degree of formalization results in a higher reward.

[0049] The acquisition of semantic consistency and formalization levels occurs after the reinforcement learning process involves editing each word segment in the sample spoken text and executing the complete action trajectory for the sample spoken text to obtain the sample written text. Therefore, the reward obtained here is sparse, specific to the sample written text with the complete action trajectory. Furthermore, the acquisition of semantic consistency and formalization levels does not require prior annotation of the sample spoken text, and is therefore not limited by insufficient labeled data.

[0050] After receiving a reward based on semantic consistency and / or written form, the reward can be applied to update and iterate the parameters in the policy model, thereby obtaining the transformation model.

[0051] Specifically, in step 120, the conversion from spoken text to written text can be achieved based on the conversion model obtained from reinforcement learning.

[0052] The method provided in this invention uses the editing operation of each word segment in the sample spoken text as the action, and the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text as the reward, to perform reinforcement learning to obtain a conversion model. The reinforcement learning process is free from the limitation of insufficient labeled data, and the degree of semantic consistency and formalization provides high-level, interpretable rewards. Applying the conversion model obtained in this way to perform text conversion ensures the reliability and interpretability of the conversion from spoken text to written text.

[0053] Based on the above embodiments, Figure 2 This is a flowchart illustrating the method for obtaining the conversion model provided by the present invention, as shown below. Figure 2 As shown, the steps for obtaining the transformation model include:

[0054] Step 210: Input the editing operation of the previous word in the sample spoken text and the semantic features of the current word in the sample spoken text into the strategy model to obtain the editing operation of the current word output by the strategy model, and return the next word of the current word as the current word to obtain the editing operation, until the editing operation of each word in the sample spoken text is obtained.

[0055] Specifically, Figure 3 This is a schematic diagram of reinforcement learning provided by the present invention, for reference. Figure 3 , Figure 3 In reinforcement learning for transformation models, an observation represents the current state of a problem or system. Specifically, this current state can be divided into two parts: one part represents natural language, specifically the semantic features of the current segment in the sample spoken text; the other part represents the decision action, specifically the editing operation of the previous segment preceding the current segment in the sample spoken text. It can be understood that the current segment referred to here is the segment that requires editing during the process of judging the editing operations of each segment in the sample spoken text.

[0056] By inputting the semantic features of the current word segmentation and the editing operation of the previous word segmentation into the policy model, the policy model can then determine the editing operation for the current word segmentation based on this information, and output the editing operation for the current word segmentation. Here, the editing operation for the current word segmentation corresponds to... Figure 3 The action in the current word segmentation can be either deletion or retention.

[0057] After obtaining the editing operation for the current word segment, it is considered that the judgment of the editing operation for the current word segment is complete. The next word segment after the current word segment can be taken as the current word segment. For the new current word segment, the semantic features of the new current word segment and the editing operation of the previous word segment can be input into the policy model to obtain the new editing operation for the current word segment. Repeat the above operation until the editing operations of all words in the sample spoken text are obtained.

[0058] Step 220: Based on the editing operations of each word in the sample spoken text, determine the sample written text.

[0059] Specifically, the sample spoken text can be edited based on the editing operations of each word segment. In this process, the conversion from spoken to written language is treated as a text editing problem. By editing the sample spoken text, the converted sample written text can be obtained. Figure 3 The environment is considered. For example, the word segmentation sequence of the sample spoken text is {Oh,I,love,that,show}, where the editing operation corresponding to "Oh" is deletion, and the editing operation corresponding to the other word segments is retention. After editing the sample spoken text based on their respective editing operations, the sample written text can be obtained, and the word segmentation sequence of the sample written text is {I,love,that,show}.

[0060] Step 230: Determine the reward based on the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text.

[0061] Specifically, after obtaining the sample written text, the semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text, can be calculated. Based on this, the reward can be determined directly by applying the semantic consistency degree, or by applying the degree of formalization, or by weighted summing the semantic consistency degree and the degree of formalization to obtain the reward. This embodiment of the invention does not specifically limit the specific methods used.

[0062] Step 240: Based on the reward, perform reinforcement learning on the policy model to obtain the transformation model.

[0063] Specifically, after receiving a reward, the reward can be used as guidance to apply a reinforcement learning strategy, iterating the parameters of the policy model to obtain a transformed model. Here, the reinforcement learning strategy corresponds to... Figure 3 The policy in this context could be, for example, a reinforcement learning algorithm based on policy gradients.

[0064] Based on any of the above embodiments, step 230 includes:

[0065] The first reward is determined based on the editing operations of each word in the sample spoken text and the editing tags of the sample spoken text;

[0066] The reward is determined based on the first reward, the degree of semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text.

[0067] Specifically, when calculating rewards, in addition to considering semantic consistency and formality, the first reward can also be automatically calculated based on whether the editing operation is correct.

[0068] For the sample spoken text, the editing operations required to convert it into written text can be collected in advance, i.e., editing tags can be set for the sample spoken text. Therefore, by comparing the editing operations for each word segment in the sample spoken text with the editing tags, it can be determined whether the editing operation for each word segment was selected correctly, thus obtaining the first reward.

[0069] Furthermore, the accuracy, recall, and F1 score of all word segmentation editing operations in the sample spoken text can be statistically analyzed, and the F1 score improvement (dense) brought by each editing operation or the F1 score (sparse) of the complete trajectory editing operation sequence can be used as the first reward.

[0070] Based on this, the first reward can be combined with a reward determined based on semantic consistency and written form, for example, by adding or weighting the two together, to obtain a reward that guides the reinforcement learning algorithm.

[0071] Based on any of the above embodiments, the step of obtaining the semantic features of each word segment in the sample spoken text applied in step 210 includes:

[0072] The sample spoken text is input into the language model to obtain the semantic features of each word segment in the sample spoken text output by the speech model;

[0073] The language model is obtained by supervised classification learning and fine-tuning based on the edit tags of each word segment in a preset spoken text; or, the language model is an unsupervised pre-trained language model.

[0074] Specifically, conventional language models, such as BERT (Bidirectional Encoder Representation from Transformers), are pre-trained on unsupervised text data. Therefore, the language model used in this embodiment of the invention can be an unsupervised pre-trained language model. Furthermore, considering that the semantic features directly output by an unsupervised pre-trained language model have a weak representational quality in text conversion tasks, directly using them as input to the conversion model may not yield optimal conversion results.

[0075] To facilitate subsequent reinforcement learning training, in this embodiment of the invention, based on the unsupervised pre-trained language model, a pre-set spoken text can be used as a sample, and the editing tags carried by each word segment in the pre-set spoken text can be used as labels to perform supervised fine-tuning of the unsupervised pre-trained language model, thereby enhancing the representation of the semantic features output by the language model in the text conversion task.

[0076] Therefore, in the reinforcement learning process of the transformation model, the semantic features of word segmentation in the sample spoken text, which are part of the input of the policy model, are obtained by the language model output through supervised fine-tuning.

[0077] The method provided in this invention, by supervising the fine-tuning of the language model to adapt to the text conversion task, enhances the representation of the semantic features of each word in the sample spoken text under the text conversion task, thereby improving the reinforcement learning effect of the conversion model.

[0078] Based on any of the above embodiments, the steps for obtaining the semantic consistency degree as a reward in the transformation model reinforcement learning include:

[0079] The sample spoken text and the sample written text are input into the consistency scoring model to obtain the semantic consistency degree output by the consistency scoring model;

[0080] The consistency scoring model is trained based on positive and negative sample pairs. The positive sample pairs include a preset spoken text and a preset written text with semantic consistency. The negative sample pairs include the preset spoken text and a perturbed text obtained by perturbing the preset spoken text.

[0081] Specifically, the consistency scoring model is used to score whether the semantics of two input texts are consistent and output the semantic consistency level. It can be understood that the closer the semantics of the two texts are, the higher the semantic consistency score.

[0082] For training the consistency scoring model, positive and negative sample pairs can be used. The positive sample pairs include a preset spoken text and a preset written text. It can be understood that the preset written text is the written language label of the preset spoken text, and the preset spoken text and the preset written text are semantically consistent. The negative sample pairs include a preset spoken text and a perturbation text. The preset spoken text and the perturbation text are semantically different. In order to facilitate the construction of negative sample pairs, the preset spoken text can be perturbed. Specifically, based on the preset spoken text, key semantic components can be added or deleted. The perturbation text obtained after adding or deleting key semantic components will necessarily have a different semantic meaning than the original preset spoken text. That is, it can be considered that the perturbation text is semantically different from the preset spoken text.

[0083] After obtaining the positive and negative sample pairs, a consistency scoring model can be trained based on these pairs. Specifically, during training, the consistency scoring model can learn the features of both positive and negative sample pairs separately, thereby achieving the ability to distinguish between them. That is, the trained consistency scoring model possesses the ability to distinguish between positive and negative sample pairs. After inputting both spoken and written sample texts into the consistency scoring model, it can output the probability that these two are a positive sample pair, i.e., the probability that the spoken and written sample texts are semantically consistent. This probability can be used as the degree of semantic consistency.

[0084] Based on any of the above embodiments, the steps for obtaining the degree of formalization as a reward in the transformation model reinforcement learning include:

[0085] The sample written text is input into the writtenness scoring model to obtain the writtenness level output by the writtenness scoring model;

[0086] The written proficiency scoring model is a classification model trained on preset spoken texts and preset written texts.

[0087] Specifically, the written quality scoring model is used to score the written quality of input text and output the written quality score. Training the written quality scoring model can be achieved using pre-set spoken texts and pre-set written texts.

[0088] Here, the preset spoken text is a pre-determined spoken text, and the preset written text is a pre-defined written text. The difference between the preset spoken text and the preset written text is that they belong to spoken text and written text, respectively. The written text scoring model can be understood as a classification model used to distinguish between spoken text and written text. During training based on the preset spoken text and preset written text, the written text scoring model can learn the differences between the preset spoken text and preset written text; that is, the trained written text scoring model can have the ability to distinguish between preset spoken text and preset written text. After inputting sample written text into the written text scoring model, the model can output the probability that the sample written text belongs to written text, and this probability can be used as the written text level.

[0089] Based on any of the above embodiments, a conversion model for reinforcement learning-based spoken-to-written language conversion can be determined based on the following steps:

[0090] Obtain the sample spoken text and perform textual annotation on the sample spoken text to obtain the editing tags of the sample spoken text.

[0091] Inputting sample spoken text into a supervised fine-tuned language model yields the semantic features of each word segment in the sample spoken text output by the language model; alternatively, inputting sample spoken text into a pre-trained unsupervised language model yields the semantic features of each word segment in the sample spoken text output by the pre-trained unsupervised language model.

[0092] A strategy model is established, and the semantic features of the first segment in the sample spoken text are input into the strategy model to obtain the editing operation of the first segment output by the strategy model. Then, the editing operation of the first segment and the semantic features of the second segment are input into the strategy model to obtain the editing operation of the second segment output by the strategy model. Next, the editing operation of the second segment and the semantic features of the third segment are input into the strategy model to obtain the editing operation of the third segment output by the strategy model, and so on, until the editing operations of all segments in the sample spoken text are obtained.

[0093] It should be noted that, in this process, editing operations and semantic features, when used as input to the policy model, can be represented in the following form:

[0094] S t =Concat(S laNguage S action )

[0095] In the formula, S t This indicates the current state of word segmentation. Concat is the concatenation operation. language It is the semantic feature of the current word segmentation, Saction One-hot representation of the editing operation of the previous word segment.

[0096] After obtaining all the word segmentation editing operations in the sample spoken text, the editing operations of all word segmentation can be compared with the editing tags of the sample spoken text to determine the first reward. In addition, the sample spoken text is edited by applying all the word segmentation editing operations to obtain the sample written text. The second reward is determined based on the degree of semantic consistency between the sample written text and the sample spoken text and the degree of formalization of the sample written text.

[0097] Based on the first and second rewards, the total reward is determined, and a reinforcement learning algorithm is applied to iterate the parameters of the policy model under the guidance of the total reward, thereby obtaining the transformation model.

[0098] The reinforcement learning algorithm used here can be the Proximal Policy Optimization (PPO) algorithm, which combines the ideas of A2C (Advantage Actor-Critic) and TRPO (Trust Region Policy Optimization). The main idea of ​​PPO is that after each policy update, the new policy should not differ significantly from the old policy. PPO uses a pruning method to avoid excessively large updates, ensuring that the new and old policies perform similarly for the same state. Specifically, the policy model consists of a shared two-layer fully connected network (100 dimensions) for both the policy and value networks, and separate output layers for each policy and value.

[0099] The method provided in this invention uses the editing operation of each word segment in the sample spoken text as the action, and the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text as the reward, to perform reinforcement learning to obtain a conversion model. The reinforcement learning process is free from the limitation of insufficient labeled data, and the degree of semantic consistency and formalization provides high-level, interpretable rewards. Applying the conversion model obtained in this way to perform text conversion ensures the reliability and interpretability of the conversion from spoken text to written text.

[0100] The following describes the speech-to-written language conversion device based on reinforcement learning provided by the present invention. The speech-to-written language conversion device based on reinforcement learning described below can be referred to in correspondence with the speech-to-written language conversion method based on reinforcement learning described above.

[0101] Figure 4 This is a schematic diagram of the structure of the speech-to-written language conversion device based on reinforcement learning provided by the present invention, as shown below. Figure 4 As shown, the device includes:

[0102] Acquisition unit 410 is used to acquire spoken text;

[0103] The conversion unit 420 is used to input the spoken text into the conversion model to obtain the written text output by the conversion model;

[0104] The transformation model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text obtained by the editing operations and the sample spoken text and / or the degree of formalization of the sample written text.

[0105] The apparatus provided in this invention uses the editing operation of each word segment in the sample spoken text as the action, and the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text as the reward, to perform reinforcement learning to obtain a conversion model. The reinforcement learning process is free from the limitation of insufficient labeled data, and the degree of semantic consistency and formalization provides high-level, interpretable rewards. Applying the conversion model obtained thereby to perform text conversion ensures the reliability and interpretability of the conversion from spoken text to written text.

[0106] Based on any of the above embodiments, the device further includes:

[0107] The prediction unit is used to input the editing operation of the previous word in the sample spoken text and the semantic features of the current word in the sample spoken text into the strategy model, obtain the editing operation of the current word output by the strategy model, and return the next word of the current word as the current word to obtain the editing operation, until the editing operation of each word in the sample spoken text is obtained.

[0108] An editing unit is used to determine the sample written text based on the editing operation of each word in the sample spoken text;

[0109] The reward unit is used to determine the reward based on the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text;

[0110] A reinforcement learning unit is used to perform reinforcement learning on the policy model based on the reward to obtain the transformation model.

[0111] Based on any of the above embodiments, the reward unit is specifically used for:

[0112] The first reward is determined based on the editing operations of each word in the sample spoken text and the editing tags of the sample spoken text;

[0113] The reward is determined based on the first reward, the degree of semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text.

[0114] Based on any of the above embodiments, the prediction unit is further configured to:

[0115] The sample spoken text is input into the language model to obtain the semantic features of each word segment in the sample spoken text output by the speech model;

[0116] The language model is obtained through supervised fine-tuning based on a preset spoken text with edit tags.

[0117] Based on any of the above embodiments, the reward unit is further configured to:

[0118] The sample spoken text and the sample written text are input into the consistency scoring model to obtain the semantic consistency degree output by the consistency scoring model;

[0119] The consistency scoring model is trained based on positive and negative sample pairs. The positive sample pairs include a preset spoken text and a preset written text with semantic consistency. The negative sample pairs include the preset spoken text and a perturbed text obtained by perturbing the preset spoken text.

[0120] Based on any of the above embodiments, the reward unit is further configured to:

[0121] The sample written text is input into the writtenness scoring model to obtain the writtenness level output by the writtenness scoring model;

[0122] The written proficiency scoring model is a classification model trained on preset spoken texts and preset written texts.

[0123] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a reinforcement learning-based spoken-to-written language conversion method. This method includes: acquiring spoken text; inputting the spoken text into a conversion model to obtain written text output by the conversion model; the conversion model is obtained through reinforcement learning by using the editing operations of each word segment in the sample spoken text as actions, and using the semantic consistency between the sample written text obtained from the editing operations and the sample spoken text and / or the degree of formalization of the sample written text as rewards.

[0124] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the reinforcement learning-based spoken-to-written language conversion method provided by the above methods. The method includes: acquiring spoken text; inputting the spoken text into a conversion model to obtain written text output by the conversion model; the conversion model is obtained through reinforcement learning by taking the editing operation of each word segment in the sample spoken text as the action and the semantic consistency between the sample written text obtained by the editing operation and the sample spoken text and / or the degree of formalization of the sample written text as the reward.

[0126] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the reinforcement learning-based spoken-to-written language conversion method provided by the above methods. The method includes: acquiring spoken text; inputting the spoken text into a conversion model to obtain written text output by the conversion model; wherein the conversion model is obtained through reinforcement learning by taking the editing operation of each word segment in the sample spoken text as the action, and taking the semantic consistency between the sample written text obtained by the editing operation and the sample spoken text and / or the degree of formalization of the sample written text as the reward.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for converting spoken to written language based on reinforcement learning, characterized in that, include: Obtain spoken text; The spoken text is input into the conversion model to obtain the written text output by the conversion model; The conversion model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text as rewards. The written sample text was obtained by editing the spoken sample text based on the editing operation. The steps for obtaining the conversion model include: The editing operation of the previous word in the sample spoken text and the semantic features of the current word in the sample spoken text are input into the strategy model to obtain the editing operation of the current word output by the strategy model. The next word of the current word is returned as the current word to obtain the editing operation, until the editing operation of each word in the sample spoken text is obtained. Based on the editing operations of each word in the sample spoken text, the sample written text is determined; The first reward is determined based on the editing operations of each word in the sample spoken text and the editing tags of the sample spoken text; The reward is determined based on the first reward, the degree of semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text; Based on the reward, the strategy model is subjected to reinforcement learning to obtain the conversion model; The steps for obtaining the semantic features of each word segment in the sample spoken text include: The sample spoken text is input into a language model to obtain the semantic features of each word segment in the sample spoken text output by the language model. The language model is obtained by supervised classification learning and fine-tuning based on the edit tags of each word segment in a preset spoken text; or, the language model is an unsupervised pre-trained language model.

2. The method for converting spoken to written language based on reinforcement learning according to claim 1, characterized in that, The steps for obtaining the semantic consistency level include: The sample spoken text and the sample written text are input into the consistency scoring model to obtain the semantic consistency degree output by the consistency scoring model; The consistency scoring model is trained based on positive and negative sample pairs. The positive sample pairs include a preset spoken text and a preset written text with semantic consistency. The negative sample pairs include the preset spoken text and a perturbed text obtained by perturbing the preset spoken text.

3. The method for converting spoken to written language based on reinforcement learning according to claim 1, characterized in that, The steps for obtaining the degree of written content include: The sample written text is input into the writtenness scoring model to obtain the writtenness level output by the writtenness scoring model; The written proficiency scoring model is a classification model trained on preset spoken texts and preset written texts.

4. A spoken-to-written language conversion device based on reinforcement learning, characterized in that, include: The acquisition unit is used to acquire spoken text; A conversion unit is used to input the spoken text into a conversion model to obtain the written text output by the conversion model; The conversion model is obtained through reinforcement learning by taking the editing operations of each word segment in the sample spoken text as actions and the degree of semantic consistency between the sample written text and the sample spoken text and / or the degree of formalization of the sample written text as rewards. The written sample text was obtained by editing the spoken sample text based on the editing operation. The reinforcement learning-based spoken-to-written language conversion device also includes: The prediction unit is used to input the editing operation of the previous word in the sample spoken text and the semantic features of the current word in the sample spoken text into the strategy model, obtain the editing operation of the current word output by the strategy model, and return the next word of the current word as the current word to obtain the editing operation, until the editing operation of each word in the sample spoken text is obtained. An editing unit is used to determine the sample written text based on the editing operation of each word in the sample spoken text; The reward unit is used to determine a first reward based on the editing operation of each word in the sample spoken text and the editing tags of the sample spoken text; and to determine the reward based on the first reward, the degree of semantic consistency between the sample written text and the sample spoken text, and / or the degree of formalization of the sample written text. A reinforcement learning unit is used to perform reinforcement learning on the policy model based on the reward to obtain the conversion model; The prediction unit is also used for: The sample spoken text is input into a language model to obtain the semantic features of each word segment in the sample spoken text output by the language model. The language model is obtained by supervised classification learning and fine-tuning based on the edit tags of each word segment in a preset spoken text; or, the language model is an unsupervised pre-trained language model.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the speech-to-written language conversion method based on reinforcement learning as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the speech-to-written language conversion method based on reinforcement learning as described in any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech-to-written language conversion method based on reinforcement learning as described in any one of claims 1 to 3.

Images