Text enhancement method and device and model training method and device

Abstract

The invention discloses a text enhancement method and device and a model training method and device.According to the text enhancement method and device, word segmentation is conducted on a first text to obtain a plurality of first vocabularies, then second vocabularies having an association relation with each first vocabulary in the aspect of pronunciation are determined, and finally text enhancement processing is conducted on the first text and the second vocabularies, so that the text enhancement efficiency is improved. And replacing at least one first vocabulary in the first text with the corresponding second vocabulary to obtain at least one second text. Therefore, by replacing the first vocabulary in the first text with the second vocabulary with the pronunciation association relationship and performing text enhancement on the first text, the quality of the training text can be improved while the quantity and diversity of the training text data are increased.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a text enhancement method and device, and a model training method and device. Background Art

[0002] Natural Language Processing (NLP) is an important branch of artificial intelligence, which aims to enable computers to understand, process and generate natural language text data. Among them, text enhancement is a common method of natural language processing technology. For example, in the training process of artificial intelligence models based on text processing technology, text enhancement can be used to modify, generate or synthesize text data to improve the quality of training text, enrich text content or enhance text features, thereby increasing the quantity and diversity of training data. Summary of the invention

[0003] In response to the above technical problems, the present application provides a text enhancement method and device, and a model training method and device.

[0004] In a first aspect, the present application provides a text enhancement method, comprising:

[0005] Segmenting the first text to obtain a plurality of first words;

[0006] Determining a second word corresponding to each first word, wherein the pronunciation of the first word is associated with the pronunciation of the second word;

[0007] Text enhancement processing is performed based on the first text and the plurality of second words to obtain at least one second text, where the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

[0008] In a second aspect, the present application provides a model training method, comprising:

[0009] Acquire a training sample set, the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary;

[0010] Inputting the first sample text in each sample text pair into the model to be trained for text enhancement prediction to obtain a predicted enhanced text;

[0011] A loss function value is calculated based on the predicted enhanced text and the second sample text, and the parameters of the model to be trained are adjusted based on the loss function value.

[0012] In a third aspect, the present application provides a text enhancement device, comprising:

[0013] A word segmentation module, used for performing word segmentation on the first text to obtain a plurality of first words;

[0014] An association module, used to determine a second word corresponding to each first word, wherein the pronunciation of the first word is associated with the pronunciation of the second word;

[0015] The enhancement module is used to perform text enhancement processing based on the first text and multiple second words to obtain at least one second text, where the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

[0016] In a fourth aspect, the present application provides a model training device, comprising:

[0017] An acquisition module is used to acquire a training sample set, the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary;

[0018] A prediction module, used for inputting the first sample text in each sample text pair into the model to be trained to perform text enhancement prediction to obtain a predicted enhanced text;

[0019] The parameter adjustment module is used to calculate the loss function value according to the predicted enhanced text and the second sample text, and adjust the parameters of the model to be trained according to the loss function value.

[0020] In a fifth aspect, the present application also provides a computer device, including a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of any text enhancement method or model training method provided in the embodiments of the present application.

[0021] In a sixth aspect, the present application also provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps of any text enhancement method or model training method provided in the embodiments of the present application.

[0022] In the seventh aspect, the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of any text enhancement method or model training method provided in the embodiments of the present application.

[0023] In summary, the present application performs word segmentation on the first text to obtain multiple first words, then determines the second words that are associated with each first word in pronunciation, and finally performs text enhancement processing on the first text and the multiple second words, and replaces at least one first word in the first text with the corresponding second word to obtain at least one second text. In this way, by replacing the first word in the first text with the second word that is associated with pronunciation, the first text is enhanced, which can increase the quantity and diversity of training data while also improving the quality of the training text. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a schematic diagram of an application environment of a text enhancement method provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of an application scenario of a text enhancement method provided in an embodiment of the present application;

[0027] Figure 3 It is a flowchart of a text enhancement method provided in an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of a Mandarin vowel tongue position map provided in an embodiment of the present application;

[0029] FIG. 5( a ) and FIG. 5( b ) are schematic diagrams of a five-degree marking method for Chinese tones provided in an embodiment of the present application;

[0030] Figure 6 It is a flowchart of a model training method provided in an embodiment of the present application;

[0031] Figure 7 It is a flowchart of a method for determining a predictive enhanced text provided in an embodiment of the present application;

[0032] Figure 8 It is a flowchart of a text enhancement method provided in a specific embodiment of the present application;

[0033] Fig. 9 is a structural schematic diagram of a text enhancement device provided in an embodiment of the present application;

[0034] Fig.10It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0035] Fig.11 It is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to be clearer, the following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0037] In one embodiment of the present application, a text enhancement method and a model training method can be run on a local terminal device or a server. When the method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0038] In order to better understand the text enhancement method and device, and the model training method and device provided in the embodiments of the present application, the application environment applicable to the embodiments of the present application is first described below.

[0039] See also Figure 1 , Figure 1 A schematic diagram of an application environment of a text enhancement method provided in an embodiment of the present application is shown. As an implementation mode, the text enhancement method and the model training method provided in an embodiment of the present application can be applied to the same electronic device. The electronic device can be as follows: Figure 1 The server 110 shown in the figure can be connected to the terminal device 120 via a network. The network is used to provide a medium for a communication link between the server 110 and the terminal device 120. The network can include various connection types, such as wired communication links, wireless communication links, etc., which are not limited in the embodiments of the present application. Optionally, in other embodiments, the text enhancement method and the model training method can also be run in other types of electronic devices such as smart phones and laptops.

[0040] It should be understood that Figure 1The server 110, network and terminal device 120 are merely illustrative. Any number of servers, networks and terminal devices may be configured as required. For example, the server 110 may be a physical server or a server cluster composed of multiple servers, and the terminal device 120 may be a mobile phone, a tablet, a desktop computer, a laptop computer, and the like. It is understood that the embodiments of the present application may also allow multiple terminal devices 120 to access the server 110 at the same time.

[0041] In some embodiments, the terminal device 120 can record the user's voice to obtain the user's audio data. Further, the terminal device 120 sends the user's audio data to the server 110 through the network. After the server 110 receives the user's audio data, the audio data can be processed by a text enhancement method described in an embodiment of the present application.

[0042] As another implementation, the text enhancement method and model training method provided in the embodiment of the present application can be applied to different electronic devices. For example, the text enhancement method is applied to electronic device A, the model training method is applied to electronic device B, and so on. The embodiment of the present application does not limit the electronic device to which the above method is applied.

[0043] The following is a detailed description in conjunction with the accompanying drawings. In this embodiment, the execution subject is taken as an example of the server 110. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments. Although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the accompanying drawings.

[0044] The text enhancement method and model training method provided by the embodiment of the present application can be applied to any scene that needs to be enhanced in text, and the method can be applied to the products of these scenes. The description of the same sentence by the user in different emotional states will produce different description methods, so that the pronunciation change situation for the vocabulary in the sentence will occur when describing. For example, the pronunciation change can include two situations of strong reading and weak reading. Strong reading refers to clearer and louder for some syllables, and weak reading refers to compressing or fuzzifying some syllables. The pronunciation change of each language will follow certain rules, therefore, server 110 can judge the voice data input by the user based on the pronunciation change rule of the language, so as to more accurately obtain the needs of the user, to give corresponding feedback. Figure 2 Schematic diagram of an application scenario of a text enhancement method provided in an embodiment of the present application. Figure 2As shown, taking the intelligent customer service system as an example, the intelligent customer service system is a system that uses artificial intelligence technology to provide customer service and support. It is usually used to handle user consultations, problem feedback, provide suggestions, etc., to improve user experience and improve service efficiency. In an embodiment of the present application, the intelligent customer service system may include modules such as a user terminal 201, a speech recognition module 202, and an intention recognition module 203. The intention recognition module 203 is to determine the consciousness expressed by the text content, such as different emotional categories such as positive, negative or neutral emotions, so as to provide corresponding replies for different consciousnesses. The user terminal 201 collects user voice data and sends it to the intelligent customer service system. The speech recognition module 202 extracts the voice data to obtain the text to be recognized. Then, the intention recognition module 203 recognizes the text to be recognized, thereby judging the meaning or emotional information expressed by the text, giving a corresponding intelligent reply, and improving the quality of intelligent customer service.

[0045] The accuracy of the above-mentioned scenarios needs to be obtained by training based on a large amount of sample data. Therefore, when training data, a large amount of sample data is required, and data enhancement is an indispensable part of the process of training data. In the current natural language processing, the text enhancement method is usually to randomly replace synonyms, reorganize sentences, synthesize data, or randomly insert, delete, exchange words, etc. to the text data, and the enhanced text formed is random and limited to the same language. Therefore, how to enhance the text of natural language based on the language rules of multiple languages ​​is a technical problem to be solved in the current field of natural language processing. In various languages, there are pronunciation change rules. Taking the pronunciation change rules including strong reading and weak reading as an example, strong reading and weak reading are mostly used in function words, such as prepositions in Chinese and articles in English. In view of this, the present application intends to respect the particularity of the language itself, based on the pronunciation change theory of the language, and propose guidance from the phonetic level to the vocabulary and grammar level to complete text enhancement, so as to efficiently and concisely generate a large amount of training corpus for the training of tasks such as classification and sequence annotation in the later stage.

[0046] Figure 3 is a flowchart of a text enhancement method provided in an embodiment of the present application. Figure 3 As shown, in the embodiment of the present application, the specific process of the text enhancement method can be performed by the processor in the above-mentioned server 110 to execute the following steps 301-303, which are described in detail below.

[0047] Step 301: segment a first text to obtain a plurality of first words.

[0048] In an embodiment of the present application, the first text refers to the text to be enhanced, and the text to be enhanced refers to the text that needs to be data enhanced in the target language. In one example, the first text can be directly obtained from existing text data, such as from a text database including pre-formed sample texts. In another example, the first text can also be obtained by converting the acquired voice data into text through voice recognition. Among them, the text enhancement of the first text is based on the pronunciation change law of the vocabulary. Therefore, after obtaining the first text, it is necessary to split the first text into smaller units, and obtain multiple first words by segmenting the first text. Taking the first text as ["put on the table"] as an example, the first text can be segmented into ["put", "in", "table", "on"]. In this way, "put", "in", "table", "on" are multiple first words of the first text "put on the table".

[0049] Step 302: Determine a second word corresponding to each of the first words, wherein the pronunciation of the first words is associated with the pronunciation of the second words.

[0050] In the embodiment of the present application, the above-mentioned association relationship refers to that the relationship between the pronunciation of the second vocabulary and the pronunciation of the first vocabulary is a relationship that follows the pronunciation change law of the language. Taking the pronunciation change law of the language as strong reading change and weak reading change as an example, the pronunciation of the first vocabulary can be subjected to strong and weak reading feature extraction, that is, the pronunciation of the first vocabulary is strengthened or weakened, and the pronunciation corresponding to the pronunciation of the first vocabulary is obtained after the strengthening and / or weakening, and then the corresponding second vocabulary is obtained according to the pronunciation after the strengthening and / or weakening. The second vocabulary after the pronunciation is strengthened is the strong reading vocabulary of the first vocabulary, and the second vocabulary after the pronunciation is weakened is the weak reading vocabulary of the first vocabulary. Taking the second vocabulary as the weak reading vocabulary of the first vocabulary as an example, in the first text's participle ["rest", "in", "table", "on"], the weak reading vocabulary of the first vocabulary "in the four tones zai" may correspond to several situations, such as the light tone in zai, or be omitted, or all become "light tone de", therefore, the weak reading pinyin of "in" may be zai, de or not read. Therefore, the second vocabulary corresponding to the first vocabulary may be zai, de or not read.

[0051] Step 303: Perform text enhancement processing based on the first text and multiple second words to obtain at least one second text, where the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

[0052] In the embodiment of the present application, for each first text, multiple vocabulary replacements can be performed, and each replacement can generate a second text. After the first text is segmented, the second vocabulary is obtained after each first vocabulary is pronounced differently, and then the first vocabulary at the position can be replaced with the corresponding second vocabulary by randomly selecting different positions of the first text, thereby completing the whole sentence enhancement of the first text. The position randomly selected by the first text can be one or more, and the second vocabulary replaced by each position can be any one of all second vocabulary with pronunciation association with the first vocabulary. Still taking the first vocabulary after the first text segmentation as ["rest", "in", "table", "on"] as an example, the second vocabulary obtained after the pronunciation of "in" is weakened can be weakly read as "of", the second vocabulary obtained after the pronunciation of "table" is weakened can be "table of", and the pronunciation of "in" and "table" is weakened, then the second text after the first text ["rest on the table"] is replaced can be ["rest on the table of"]. In addition to the above-mentioned enhancement method of weakening the pronunciation of the first vocabulary in the first text, the first text can also be enhanced by strengthening the pronunciation of the first vocabulary, and the first text can be enhanced by strengthening and weakening the pronunciation of the first vocabulary at the same time. By random replacement, the number and diversity of enhanced texts are increased while following the language rules.

[0053] The embodiment of the present application performs word segmentation on the first text to obtain multiple first words, then determines the second words that have a pronunciation association relationship with each first word, and finally performs text enhancement processing on the first text and the multiple second words, and replaces at least one first word in the first text with a corresponding second word to obtain at least one second text. In this way, based on the pronunciation change law of the language, by replacing the first word in the first text with a second word with a pronunciation association relationship, a large number of second texts with language change laws can be generated efficiently and concisely, which can increase the quantity and diversity of training data while improving the quality of the training text. In addition, since the embodiment of the present application follows the pronunciation change law of the language, it may not be limited to a single language, so the embodiment of the present application also provides guidance on mixed language fusion, and provides a powerful text enhancement implementation plan for the subsequent expansion of monolingual texts to multilingual texts and the expansion of minority languages ​​to multilingual texts.

[0054] In the embodiment of the present application, for each first word, in step 302, first, multiple candidate words associated with the pronunciation of the first word may be determined. Then, the vector distance between the first word and each candidate word is determined. Finally, at least one of the multiple candidate words is selected as the second word based on the vector record.

[0055] In the embodiments of the present application, a candidate word refers to all words that have a pronunciation association with the first word. However, during the text enhancement process, candidate words with relatively weak pronunciation associations may not be used as the second word to replace the first word in text enhancement. Therefore, in order to reduce the computational burden on the server, candidate words with relatively weak pronunciation associations among the multiple candidate words corresponding to the first word can be excluded, and only candidate words with relatively strong pronunciation associations are selected as the second word for word replacement in text enhancement. Among them, the strength of the pronunciation association between the candidate word and the first word can be determined by the vector distance between the words. For example, in one example, an association threshold can be set. If the vector distance between the candidate word and the first word is greater than the association threshold, it means that the pronunciation association between the candidate word and the first word is relatively weak. If the vector distance between the candidate word and the first word is less than or equal to the association threshold, it means that the pronunciation association between the candidate word and the first word is relatively strong. Therefore, candidate words less than or equal to the association threshold can be used as the second word. In another example, the candidate word with the smallest vector distance from the first word among the candidate words can be selected as the second word. Still taking the word segmentation of the above-mentioned first text as ["put", "on", "table", "top"] as an example, the pronunciation of the second word corresponding to the pronunciation "zài (in the fourth tone)" of the first word may correspond to several situations, such as the light tone "zài", or it may be omitted, or it may become "de (in the light tone)", so the pronunciation of the second word corresponding to the pronunciation of the first word may be "zài", "de", or not pronounced. In one example, assuming that the pronunciations of the second words less than or equal to the association threshold are "zài" and "de", then the light tone "zài" and the light tone "de" can be used as the second words for "zài".

[0056] In the embodiments of the present application, the pronunciation information of the first word can be determined first. Then, the pronunciation information of the first word is subjected to pronunciation association mapping to obtain multiple replacement pronunciation information. Finally, the words matching each replacement pronunciation information are determined to obtain multiple candidate words.

[0057] In the embodiment of the present application, the pronunciation information refers to the pronunciation corresponding to the first vocabulary, such as the pinyin of Chinese, the phonetic symbol of English, etc. The candidate vocabulary is obtained by changing the pronunciation information corresponding to the first vocabulary. Therefore, the pronunciation information of the first vocabulary can be determined first, and then the pronunciation information of each first vocabulary is subjected to pronunciation association mapping to obtain multiple replacement pronunciation information. Among them, the pronunciation association mapping refers to associating the pronunciation information of the first vocabulary with the pronunciation information that conforms to the pronunciation change law of the language. The replacement pronunciation information refers to the pronunciation information that has a pronunciation association mapping relationship with the pronunciation information of the first vocabulary. In an example, each vocabulary, the pronunciation information corresponding to each vocabulary, and the replacement pronunciation information corresponding to each pronunciation information can be stored in a setting mapping library. The setting mapping library can be a database constructed in advance, which corresponds vocabulary, pronunciation information of vocabulary, and replacement pronunciation information of the pronunciation information one by one. Considering that there may be multiple pronunciation information for the same vocabulary, and multiple vocabulary for the same pronunciation information, in the setting mapping library, the vocabulary corresponding to the replacement pronunciation information may be multiple, and these vocabulary can all be candidate vocabulary. The candidate vocabulary can be further selected according to demand later.

[0058] In an embodiment of the present application, the pronunciation information of the first vocabulary is subjected to pronunciation association mapping to obtain multiple replacement pronunciation information. The step can be performed by vector encoding the pronunciation information, determining the pronunciation change characteristics based on the vector encoding, and then obtaining the replacement pronunciation information based on the pronunciation change characteristics.

[0059] Specifically, the vector encoding of the pronunciation information can be determined by the pronunciation position and pronunciation method of the pronunciation information, and the tone of the pronunciation information. First, a first vector encoding is generated according to the phoneme pronunciation position and pronunciation method of the pronunciation information of the first vocabulary, and the first vector encoding represents the phoneme pronunciation position and pronunciation method of the pronunciation information. Then, a second vector encoding is generated according to the tone of the pronunciation information of the first vocabulary, and the second vector encoding represents the tone of the pronunciation information. Finally, the pronunciation information vector encoding of the first vocabulary is generated according to the first vector encoding and the second vector encoding.

[0060] In an embodiment of the present application, encoding the pronunciation information of the first vocabulary can be determined by a coding model. The coding model is used to convert the pronunciation information into the corresponding pronunciation information vector coding, and the coding model is a model constructed based on the pronunciation position, pronunciation method and tone of the phoneme of the pronunciation information. Phoneme refers to the smallest unit with a distinguishing function in speech. Each language has its own specific phoneme system, and the vocabulary and phonetic structure of the language are formed by a combination of different phonemes. Different phonemes can be distinguished by different pronunciation positions. For example, the pronunciation position can include but is not limited to labial sounds, dental sounds, palatal sounds, alveolar sounds, velar sounds and glottal sounds.

[0061] The articulation method refers to how the speech organs work together to produce a specific sound when pronouncing a voice. The articulation method may include, but is not limited to: voiced sounds, voiceless sounds, plosives, fricatives, approximants, open sounds, trills, consonants, vowels, and nasal sounds, etc. The place of articulation and the articulation method of each phoneme can be assigned a number respectively, so as to form a first vector encoding.

[0062] On this basis, in order to better distinguish the pronunciation data, it is also necessary to distinguish the tones of the pronunciation information. Therefore, the embodiment of the present application can also generate a second vector encoding according to the tones of the pronunciation information. Finally, the first vector encoding and the second vector encoding are combined to obtain a string of vector encoding representations. In this way, the place of articulation and the articulation method of the phonemes of the pronunciation information, as well as the tones, all correspond to a vector encoding representation. This vector encoding can accurately represent the pronunciation data corresponding to the first vocabulary. The encoding model constructed in the above embodiment can vectorize the text vocabulary based on the characteristics of the pronunciation information, making the processing of data based on the pronunciation change rules of the language more regular, not limited to a single language, and improving the application breadth of the model.

[0063] Next, taking Mandarin Chinese as an example, the construction of the encoding model will be illustrated. The encoding model is used to perform pinyin vector encoding on the vocabulary of Mandarin Chinese. Table 1 is a table of the place of articulation and the articulation method of the initials in Mandarin Chinese. Figure 4 It is a schematic diagram of a vowel tongue position map of Mandarin Chinese provided in the embodiment of the present application.

[0064] Table 1

[0065]

[0066] Combined with Table 1 and Figure 4 As shown, the encoding model of Mandarin Chinese needs to be constructed based on the table of the place of articulation and the articulation method of the initials and the vowel tongue position map. Among them, different weights need to be set for different articulation methods and different places of articulation. Each place of articulation is assigned a code, and each articulation method is assigned a code. In this way, each vowel, consonant, and tone can be represented by a string of codes.

[0067] On this basis, Mandarin Chinese also includes multiple tones, so it is also necessary to encode the tones of Mandarin Chinese. As shown in Figures 5(a) and 5(b), it is a schematic diagram of a five-degree marking method for Chinese tones provided in the embodiment of the present application. The tone of the light tone in Mandarin Chinese mainly depends on the tone of the previous syllable. Specifically, it includes the following situations.

[0068] (1) The light tone after the high level tone is a short low falling tone, with a tone value of 31. For example, "先生 (xiān sheng)", "衣服 (yī fu)", "东西 (dōng xi)", and "耽误 (dān wu)", etc.

[0069] (2) The neutral tone after the rising tone is a mid tone with a tone value of 3. For example, "student", "others", "understand", "cheap", etc.

[0070] (3) The neutral tone after the falling tone is a low tone with a tone value of 1. For example, "joke", "polite", "nurse", "madam", etc.

[0071] (4) The neutral tone after the rising-falling tone is a semi-high flat tone with a tone value of 44. For example, "inside" and "wood", etc.

[0072] It should be noted that the timbre and loudness of the neutral tone syllable may change to a certain extent. For example, the tongue position of the vowel may tend to the center, and the initial consonant may also change, etc. In this way, by vector encoding the tones of Mandarin, different words in Mandarin can be more accurately distinguished.

[0073] After determining the vector encoding of the pronunciation information of the first vocabulary, the pronunciation change characteristics of each first vocabulary can be determined first according to the vector encoding of the pronunciation information of each first vocabulary. Then, multiple replacement pronunciation information corresponding to the pronunciation information of each first vocabulary can be determined according to the pronunciation change characteristics.

[0074] In the embodiments of the present application, the pronunciation change characteristics change according to the phoneme change and tone change of the pronunciation information. Therefore, the pronunciation change characteristics of the pronunciation information of each first vocabulary can be determined according to the phoneme change and tone change of the pronunciation information, so as to determine the replacement pronunciation information of each first vocabulary. As mentioned above, there may be multiple replacement pronunciation information corresponding to the pronunciation information of each first vocabulary. Therefore, the replacement pronunciation information corresponding to the pronunciation information of each first vocabulary may be more than one. Therefore, after performing pronunciation association mapping on the pronunciation information of the first vocabulary, multiple replacement pronunciation information can be obtained.

[0075] Taking the strengthening pronunciation information and weakening pronunciation information as an example, Table 2 is the change characteristic table of strong and weak readings in Mandarin. As shown in Table 2, in Mandarin, the change characteristics of the strong and weak reading text pairs are mainly reflected in three aspects: initial consonant change, final vowel change, and tone change.

[0076] Table 2

[0077]

[0078] (1) Initial consonant change. For example, the initial consonant z tends to change to d. Both z and d are voiceless consonants. To play a role in voice prompt by retaining the characteristic that voiceless consonants become voiced consonants after voicelessness. When pronouncing / z / , the tongue does not touch the upper gum, leaving a tiny space for friction, the air flow is strong, and the vocal cords vibrate. While when pronouncing / d / , a brief closure is achieved and then immediately separated, the vocal cords vibrate but no air is expelled. Therefore, it is obvious that the difficulty of pronouncing / d / is lower than that of / z / , which conforms to the principle of labor saving in communication.

[0079] (2) Final changes, such as ai. Whether it changes from front to middle or back, from middle to back, or vice versa; whether it shows a trend of changing from a diphthong to a single vowel; these are all calculated from the encoding vector of the encoded pronunciation position and method.

[0080] (3) Changes in tone, for example, from first, second, third, and fourth tones to a light tone.

[0081] The above is only an example of setting the pronunciation change rules of strong and weak pronunciations in Mandarin. For each language, there will be corresponding pronunciation change rules. Therefore, it can be set based on each language to obtain replacement pronunciation information.

[0082] In the embodiment of the present application, the replacement pronunciation information may correspond to multiple words, but if the difference between the word corresponding to the replacement pronunciation information and the first word is too small, it is not necessary to generate another candidate word to replace the first word. Therefore, the candidate words corresponding to the replacement pronunciation information can also be screened, so as to select the words that meet the conditions as the candidate words.

[0083] Specifically, candidate words can be screened by pronunciation feature vector distance and length variation. Pronunciation feature vector distance is used to measure the pronunciation difficulty difference between two words. For example, the pronunciation difficulty of different words can be encoded by a machine model to obtain the pronunciation difficulty characteristics of each word, and the corresponding pronunciation vector distance can be obtained by calculating the vector distance between the pronunciation difficulty characteristics of different words. The pronunciation feature vector distance can also be calculated based on the pronunciation information vector encoding of different words. The length variation refers to the result obtained by comparing the lengths of two words. For example, if the length of the candidate word is 5 (which can represent 5 characters) and the length of the first word is 3, the length of the candidate word and the length variation of the first word are increased by 2, that is, the candidate word is longer than the first word. If the length of the candidate word is 3 and the length of the first word is 5, the length of the candidate word and the length variation of the first word are reduced by 2, that is, the candidate word is shorter than the first word. Wherein, the "length" can be a quantitative value used to quantify the length of the word, for example, it can be represented by the number of characters contained in the word.

[0084] In an embodiment of the present application, the pronunciation feature vector distance and length change between the pronunciation information of the first vocabulary and each replacement pronunciation information can be determined respectively. When the pronunciation feature vector distance between the replacement pronunciation information and the pronunciation information of the first vocabulary is greater than a first threshold value and the length change between the replacement pronunciation information and the pronunciation information of the first vocabulary satisfies a set change range, the vocabulary corresponding to the replacement pronunciation information is determined as a candidate vocabulary. Among them, the first threshold value is a threshold value that makes the distance between the pronunciation feature vector of the candidate vocabulary and the first vocabulary more obvious, and the set change amount is a set value that makes the length change between the candidate vocabulary and the first vocabulary meet the length change rule between the candidate vocabulary and the first vocabulary.

[0085] The pronunciation information of the strengthened vocabulary and the pronunciation information of the weakened vocabulary are taken as examples below. The replacement pronunciation information may include a strong reading replacement pronunciation and a weak reading replacement pronunciation, and the candidate vocabulary may include a strong reading candidate vocabulary and a weak reading candidate vocabulary. The above-mentioned determination of multiple candidate vocabulary may include determining multiple strong reading candidate vocabulary and determining multiple weak reading candidate vocabulary.

[0086] Taking the determination of strong reading candidate vocabulary as an example, firstly, it is necessary to obtain the strong reading candidate vocabulary corresponding to the strong reading replacement pronunciation of the first vocabulary, the pronunciation feature vector distance and length variation between the strong reading candidate vocabulary and the first vocabulary. When the pronunciation feature vector distance between the strong reading vocabulary and the first vocabulary is greater than the first threshold value and the length of the strong reading vocabulary is greater than or equal to the first vocabulary, the strong reading vocabulary is determined as a candidate strong reading vocabulary. Taking the determination of weak reading candidate vocabulary as an example again, it is necessary to first obtain the weak reading candidate vocabulary corresponding to the weak reading replacement pronunciation of the first vocabulary, the pronunciation feature vector distance and length variation between the weak reading candidate vocabulary and the first vocabulary. When the pronunciation feature vector distance between the weak reading vocabulary and the first vocabulary is greater than the first threshold value and the length of the weak reading vocabulary is less than or equal to the first vocabulary, the weak reading vocabulary is determined as a candidate weak reading vocabulary. The embodiment of the present application selects candidate vocabulary from the vocabulary corresponding to the replacement pronunciation information through the pronunciation feature vector and the length variation, which can reduce unnecessary calculation amount and improve the efficiency of feature extraction of pronunciation changes.

[0087] In the embodiment of the present application, in step 303, the enhancement mode parameter, enhancement ratio and enhancement position index of the first text may be obtained first, and then text enhancement processing may be performed based on the enhancement mode parameter, enhancement ratio, enhancement position index, the first text and a plurality of second words to obtain at least one second text.

[0088] Specifically, after obtaining the second vocabulary corresponding to the first vocabulary after the first text is segmented, the parameters such as the enhancement method parameter, enhancement ratio and enhancement position index of the first text can be obtained by means of a random function. Then, according to the enhancement method parameter and enhancement ratio of the first text, the first vocabulary corresponding to the enhancement position index is replaced with the corresponding second vocabulary to obtain at least one second text.

[0089] Among them, the enhancement mode parameter refers to the enhancement mode of the first text. Taking the pronunciation change rule as the strengthening vocabulary and weakening vocabulary, the enhancement mode parameter may include strong reading enhancement, weak reading enhancement and strong and weak reading bidirectional enhancement. Strong reading enhancement is to replace the strong reading vocabulary of the first text, weak reading enhancement refers to the replacement of weak reading vocabulary of the first text, and strong and weak reading bidirectional enhancement refers to replacing part of the strong reading vocabulary of the first text and replacing part of the weak reading vocabulary of the first text. The enhancement ratio refers to the ratio of strong reading vocabulary replacement and the ratio of weak reading vocabulary replacement, which may include strong reading ratio and weak reading ratio. For example, assuming that the number of the first vocabulary is 10, the strong reading ratio is 0.3, and the weak reading ratio is 0.4, then 3 of the 10 first vocabulary need to be replaced with strong reading vocabulary, 4 with weak reading vocabulary, and the remaining 3 vocabulary remain unchanged. The enhancement position index refers to the position of the first vocabulary that needs to be replaced. Taking the replacement of strong reading vocabulary and the replacement of weak reading vocabulary as an example, the enhancement position index may include strong reading position index and weak reading position index. For example, if the strong reading position index is [1,5,8] and the weak reading index is [2.6.9.12], it means that in the first text, the positions where the strong reading words need to be replaced are the 1st, 5th and 8th words to be enhanced, and the positions where the weak reading words need to be replaced are the 2nd, 6th, 9th and 12th words to be enhanced. The above replacement method is just one of the examples of random replacement. The random function can also generate any parameters such as enhancement method parameters, enhancement ratio and enhancement position index, so as to achieve random text enhancement of the first text, so that the number of text enhancements is larger and the variation method is diverse, thereby improving the robustness and generalization of subsequent models.

[0090] The above-mentioned text enhancement method can be executed by a model for text enhancement, and the model for text enhancement can segment the first text to obtain multiple first words, and then extract the features of pronunciation changes based on the first words to obtain a second word that has a pronunciation association relationship with the first word. Therefore, the embodiment of the present application also provides a model training method, and in practical applications, the parameters of the model can also be adjusted in real time to improve the accuracy and generalization ability of text enhancement.

[0091] Figure 6 is a flow chart of a model training method provided in an embodiment of the present application. Figure 6 As shown, in an embodiment of the present application, the specific process of the model training method can be performed by the processor in the above-mentioned server 110 to execute the following steps 601 to 603, which are described in detail below.

[0092] Step 601, obtaining a training sample set, the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary.

[0093] In an embodiment of the present application, a training sample set is composed of a plurality of sample text pairs, and the sample text pairs include a first sample vocabulary and a second sample vocabulary corresponding to each first sample vocabulary and having a pronunciation association relationship. Among them, the first sample vocabulary can be obtained through a text corpus of a language. A text corpus is a data set of text samples, which may include data, articles, conversations, social media posts, web page content, and the like. A plurality of first sample vocabularies may be obtained after word segmentation of the first sample text. The second sample vocabulary may be obtained by word segmentation of the second sample text. In one example, the text corpus may be text information obtained through multiple channels. In another example, the text corpus may also be speech data obtained through multiple channels, and then the language data is converted into text. In addition, during model training and actual application, the training sample set may be added at any time to expand the amount of data and improve the performance of the model.

[0094] Step 602: Input the first sample text in each sample text pair into the model to be trained to perform text enhancement prediction to obtain predicted enhanced text.

[0095] In the embodiment of the present application, the model to be trained refers to an initial model that has not been trained for text enhancement. The first sample text in each sample text pair is input into the model to be trained, and the corresponding predicted enhanced text can be obtained. The predicted enhanced text is the output corresponding to the first sample text output by the model.

[0096] Step 603: Calculate a loss function value based on the predicted enhanced text and the second sample text, and adjust the parameters of the model to be trained based on the loss function value.

[0097] After obtaining the predicted enhanced text output by the model to be trained, the predicted enhanced text can be compared and analyzed with the second sample text in the sample text, and the loss function value can be obtained according to the comparison and analysis results. For example, the loss function value can be determined by calculating the mean square error, cross entropy loss, log likelihood loss or cosine similarity between the predicted enhanced text and the second sample text. Among them, the loss function value of the predicted enhanced text and the second sample text can be calculated by first segmenting the predicted enhanced text to obtain a predicted sample vocabulary, and then comparing and analyzing the predicted sample vocabulary with the second sample vocabulary to obtain the loss function value of the predicted enhanced text and the second sample text. Then, the training model can be iteratively updated by minimizing the loss function through gradient descent or its variants until the model reaches the convergence condition to obtain a trained model. Among them, the convergence condition can include that the loss function value is less than a certain set threshold or remains unchanged, or the number of iterations reaches a set number of times, etc.

[0098] The embodiment of the present application is based on the sample text composed of the first sample text and the second sample text to be trained. The model parameters of the model to be trained are adjusted by the loss function of the second sample text and the predicted enhanced text output by the model to be trained, so that the model to be trained reaches the convergence condition, thereby obtaining the trained model. In this way, the first text that needs to be enhanced can be enhanced by the trained model to obtain the second text with a pronunciation association relationship with the first text. Thereby increasing the quantity and diversity of the training data, and improving the quality of the training text. In addition, since the model is based on the sample pair with a pronunciation association relationship, the pronunciation change law of the language is followed, and it is not limited to a single language. Therefore, the embodiment of the present application also provides guidance on mixed language fusion, and provides a powerful text enhancement implementation scheme for the subsequent expansion of monolingual texts to multilingual texts and the expansion of few languages ​​to multilingual texts.

[0099] Figure 7 is a flow chart of a method for determining a predictive enhanced text provided in an embodiment of the present application. Figure 7 As shown, in the embodiment of the present application, in step 602, the method for determining the predicted enhanced text may include the following steps 701 to 703.

[0100] Step 701: Segment a first sample text to obtain a plurality of first sample words.

[0101] In the embodiment of the present application, the first sample text refers to the sample text to be enhanced during the model training process. The text enhancement of the first sample text is based on the pronunciation change law of the vocabulary. Therefore, after obtaining the first sample text, it is necessary to split the first sample text into smaller units, that is, to segment the first word, so as to obtain multiple first sample words.

[0102] Step 702: Determine a predicted sample word corresponding to each first sample word, wherein the pronunciation of the predicted sample word is associated with the pronunciation of the first sample word.

[0103] In the embodiment of the present application, the association relationship means that the relationship between the pronunciation of the predicted sample vocabulary and the pronunciation of the first sample vocabulary follows the law of pronunciation change of the language. Taking the law of pronunciation change of the language as strong reading change and weak reading change as an example, the pronunciation of the first sample vocabulary can be subjected to strong and weak reading feature extraction, that is, the pronunciation of the first sample vocabulary is strengthened or weakened, and the strengthened pronunciation and / or weakened pronunciation corresponding to the pronunciation of the first sample vocabulary is obtained, and then the corresponding predicted sample vocabulary is obtained according to the strengthened pronunciation and / or weakened pronunciation. The predicted sample vocabulary after pronunciation strengthening is the predicted sample vocabulary of the first sample vocabulary for strong reading, and the predicted sample vocabulary after pronunciation weakening is the predicted sample vocabulary of the first sample vocabulary for weak reading.

[0104] Step 703: Perform text enhancement processing based on the first sample text and multiple predicted sample words to obtain at least one predicted enhanced text, wherein the predicted enhanced text is obtained by replacing at least one first sample word in the first sample text with a corresponding predicted sample word.

[0105] In an embodiment of the present application, for each first sample text, multiple sample vocabulary replacements can be performed, and each replacement can generate a predicted enhanced text. For example, after the first sample text is segmented, each first sample vocabulary is pronounced differently to obtain a predicted sample vocabulary, and then different positions of the first sample text can be randomly selected, and the first sample vocabulary at the position is replaced with the corresponding predicted sample vocabulary, thereby completing the whole sentence enhancement of the first sample text. The randomly selected position of the first sample text can be one or more, and the predicted sample vocabulary replaced by each position can be any one of all predicted sample vocabulary associated with the first sample vocabulary in pronunciation. By means of random replacement, while following the laws of language, the number and diversity of predicted enhanced texts are also increased.

[0106] The embodiment of the present application performs word segmentation on the first sample text to obtain multiple first sample words, then determines the predicted sample words that have a pronunciation association relationship with each first sample word, and finally performs text enhancement processing on the first sample text and the multiple predicted sample words, and replaces at least one first sample word in the first sample text with the corresponding predicted sample text to obtain at least one predicted enhanced text. In this way, based on the pronunciation change law of the language, by replacing the first sample words in the first sample text with the predicted sample words with pronunciation association relationship, a large number of predicted enhanced texts with language change laws can be generated efficiently and concisely, which can increase the quantity and diversity of the data for model training while also improving the quality of the model prediction data. In addition, since the embodiment of the present application follows the pronunciation change law of the language, it may not be limited to a single language, so the embodiment of the present application also provides guidance on mixed language fusion, and provides a powerful text enhancement implementation plan for the subsequent expansion of monolingual texts to multilingual texts and the expansion of minority languages ​​to multilingual texts.

[0107] In the embodiment of the present application, for each first sample word, in step 702, first, a plurality of candidate sample words associated with the pronunciation of the first sample word may be determined. Then, a vector distance between the first sample word and each candidate sample word is determined. Finally, at least one of the plurality of candidate sample words is selected as a predicted sample word based on the vector distance.

[0108] In an embodiment of the present application, the candidate sample vocabulary refers to all the vocabulary that has pronunciation association with the first sample vocabulary. However, in the text enhancement process, the candidate sample vocabulary with weak pronunciation association may not be used as the predicted sample vocabulary in the text enhancement to replace the first sample vocabulary. Therefore, in order to reduce the computing burden of the server, the candidate sample vocabulary with weak pronunciation association among the multiple candidate sample vocabulary corresponding to the first sample vocabulary may be eliminated, and only the candidate sample vocabulary with strong pronunciation association may be selected as the predicted sample vocabulary for vocabulary replacement in the text enhancement. Among them, the strength of the pronunciation association between the candidate sample vocabulary and the first sample vocabulary can be determined by the vector distance between the vocabulary. For example, in one example, an association threshold can be set. If the vector distance between the candidate sample vocabulary and the first sample vocabulary is greater than the association threshold, it means that the pronunciation association between the candidate sample vocabulary and the first sample vocabulary is weak. If the vector distance between the candidate sample vocabulary and the first sample vocabulary is less than or equal to the association threshold, it means that the pronunciation association between the candidate sample vocabulary and the first sample vocabulary is strong. Therefore, the candidate sample vocabulary less than or equal to the association threshold can be used as the predicted sample vocabulary. In another example, the candidate sample vocabulary with the smallest vector distance from the first sample vocabulary in the candidate sample vocabulary can be selected as the predicted sample vocabulary.

[0109] In an embodiment of the present application, the pronunciation information of the first sample vocabulary may be determined first, then the pronunciation information of the first sample vocabulary is subjected to pronunciation association mapping to obtain a plurality of replacement pronunciation information, and finally, the vocabulary that matches each replacement pronunciation information is determined to obtain a plurality of candidate sample vocabulary.

[0110] In the embodiment of the present application, the pronunciation information refers to the pronunciation corresponding to the first sample vocabulary, such as the pinyin of Chinese, the phonetic symbol of English, etc. The candidate sample vocabulary is obtained by changing the pronunciation information corresponding to the first sample vocabulary. Therefore, the pronunciation information of the first sample vocabulary can be determined first, and then the pronunciation information of each first sample vocabulary is mapped by pronunciation association to obtain multiple replacement pronunciation information. Among them, the pronunciation association mapping refers to associating the pronunciation information of the first sample vocabulary with the pronunciation information that conforms to the pronunciation change law of the language. The replacement pronunciation information refers to the pronunciation information that has a pronunciation association mapping relationship with the pronunciation information of the first sample vocabulary. In an example, each vocabulary, the pronunciation information corresponding to each vocabulary, and the replacement pronunciation information corresponding to each pronunciation information can be stored in the setting mapping library. The setting mapping library can be a database constructed in advance, which corresponds vocabulary, pronunciation information of vocabulary, and replacement pronunciation information of the pronunciation information one by one. Considering that there may be multiple pronunciation information for the same vocabulary, and multiple vocabulary for the same pronunciation information, in the setting mapping library, the vocabulary corresponding to the replacement pronunciation information may be multiple, and these vocabulary can all be candidate sample vocabulary. The candidate sample vocabulary can be further selected according to demand later.

[0111] In an embodiment of the present application, the pronunciation information of the first sample vocabulary is subjected to pronunciation association mapping, and the step of obtaining multiple replacement pronunciation information can be achieved by vector encoding the pronunciation information, determining the pronunciation change characteristics based on the vector encoding, and then obtaining the replacement pronunciation information based on the pronunciation change characteristics.

[0112] Specifically, the vector encoding of the pronunciation information can be determined by the pronunciation position and pronunciation method of the pronunciation information, as well as the tone of the pronunciation information. First, a first vector encoding is generated according to the phoneme pronunciation position and pronunciation method of the pronunciation information of the first sample vocabulary. Then, a second vector encoding is generated according to the tone of the pronunciation information of the first sample vocabulary. Then, the pronunciation information vector encoding of the first sample vocabulary is generated according to the first vector encoding and the second vector encoding. In an embodiment of the present application, the pronunciation change feature changes according to the phoneme change and tone change of the pronunciation information. Therefore, the pronunciation change feature of each first sample vocabulary can be determined according to the pronunciation information vector encoding of each first sample vocabulary. Multiple replacement pronunciation information corresponding to the pronunciation information of each first sample vocabulary is determined according to the pronunciation change feature.

[0113] In the embodiment of the present application, the replacement pronunciation information may correspond to multiple words, but if the difference between the word corresponding to the replacement pronunciation information and the first sample word is too small, there is no need to generate another candidate sample word to replace the first sample word. Therefore, the candidate sample words corresponding to the replacement pronunciation information can also be screened, so as to select the words that meet the conditions as the candidate sample words.

[0114] Specifically, the candidate sample vocabulary can be screened by the pronunciation feature vector distance and the length change. The pronunciation feature vector distance is used to measure the pronunciation difficulty difference between two vocabularies. In an embodiment of the present application, the pronunciation feature vector distance and the length change between the pronunciation information of the first sample vocabulary and each replacement pronunciation information can be determined respectively. When the pronunciation feature vector distance between the replacement pronunciation information and the pronunciation information of the first sample vocabulary is greater than the first threshold value and the length change between the replacement pronunciation information and the pronunciation information of the first sample vocabulary meets the set change range, the vocabulary corresponding to the replacement pronunciation information is determined as a candidate sample vocabulary. Among them, the first threshold value is a threshold value that makes the pronunciation feature vector distance between the candidate sample vocabulary and the first sample vocabulary more obvious, and the set change amount is a set value that makes the length change amount of the candidate sample vocabulary and the first sample vocabulary meet the length change law between the candidate vocabulary and the first sample vocabulary.

[0115] In the embodiment of the present application, in step 703, the enhancement mode parameter, enhancement ratio and enhancement position index of the first sample text may be obtained first. Then, text enhancement processing is performed based on the enhancement mode parameter, enhancement ratio, enhancement position index, the first sample text and multiple predicted sample words to obtain at least one predicted enhanced text.

[0116] Specifically, after obtaining the predicted sample vocabulary of the first sample vocabulary after the first sample text is segmented, the parameters such as the enhancement method parameter, enhancement ratio and enhancement position index of the first sample text can be obtained by means of a random function. Then, according to the enhancement method parameter and enhancement ratio of the first sample text, the first sample vocabulary corresponding to the enhancement position index is replaced with the corresponding predicted sample vocabulary to obtain at least one predicted enhanced text.

[0117] Among them, the enhancement mode parameter refers to the enhancement mode of the first sample text. The pronunciation change law is used to strengthen the vocabulary and weaken the vocabulary. Strong reading enhancement refers to the replacement of strong reading vocabulary for the first sample text, weak reading enhancement refers to the replacement of weak reading vocabulary for the first sample text, and strong and weak reading bidirectional enhancement refers to the replacement of some strong reading vocabulary for the first sample text and the replacement of some weak reading vocabulary for the first sample text. The enhancement ratio refers to the ratio of strong reading vocabulary replacement and the ratio of weak reading vocabulary replacement, which can include strong reading ratio and weak reading ratio. For example, assuming that the number of first sample vocabulary is 10, the strong reading ratio is 0.3, and the weak reading ratio is 0.4, then 3 of the 10 first sample vocabulary need to be replaced with strong reading vocabulary, 4 are replaced with weak reading vocabulary, and the remaining 3 words remain unchanged. The enhancement position index refers to the position of the first sample vocabulary that needs to be replaced. Taking the replacement of strong reading vocabulary and the replacement of weak reading vocabulary as an example, the enhancement position index can include strong reading position index and weak reading position index. The above replacement method is just one of the examples of random replacement. The random function can also generate arbitrary enhancement method parameters, enhancement ratios, enhancement position indexes and other parameters, so as to achieve random text enhancement of the first sample text, so that the number of text enhancements is larger and the variation methods are diverse, thereby improving the robustness and generalization of subsequent models.

[0118] In the embodiment of the present application, the above-mentioned trained model can be used to implement Figure 3 As an example, for example Figure 8 The above is a flowchart of a method for text enhancement from model training to execution based on the trained model provided in a specific embodiment of the present application. Taking the text enhancement method of Mandarin Chinese as an example, in a specific embodiment of the present application, the specific process of the text enhancement method can be implemented by the following steps 801-809, which are described in detail below.

[0119] Step 801 : Obtain a training sample set of Mandarin, where the training sample set includes a plurality of sample text pairs, and each sample text pair includes a first sample text and a second sample text.

[0120] Step 802: segment the first sample text to obtain a plurality of first sample words.

[0121] Step 803, determining the pinyin of each first sample, where the pinyin is the pronunciation information of a word in Mandarin;

[0122] Step 804, performing pronunciation association mapping on the pinyin of the first sample vocabulary to obtain multiple replacement pinyins;

[0123] Among them, the pronunciation association mapping includes: first, respectively obtain the pinyin of multiple first sample words from the set mapping library. The set mapping library may include polyphones (same shape but different pronunciation) and homophones (same pronunciation but different shape). Then, the pinyin is input into the encoding model to obtain the pinyin vector encoding. Among them, the encoding model is constructed by the pronunciation position, pronunciation method and intonation of the pinyin, so that the vector distance can reflect the pronunciation position and method distance. Then, according to the set strong and weak reading rules of Mandarin, multiple replacement pinyins of the pinyin of each first sample word are determined respectively, wherein the strong and weak reading rules are the pronunciation change characteristics of strengthening and weakening vocabulary in Mandarin.

[0124] Step 805, determine the vocabulary that matches each replacement pinyin, and obtain multiple candidate sample vocabulary.

[0125] Specifically, the pronunciation feature vector distance and length change between the pinyin of the first sample vocabulary and each replacement pinyin are determined respectively. When the pronunciation feature vector distance between the replacement pinyin and the pinyin of the first sample vocabulary is greater than a first threshold and the length change between the replacement pinyin and the pinyin of the first sample vocabulary satisfies a set change range, the vocabulary corresponding to the replacement pinyin is determined as a candidate sample vocabulary.

[0126] Step 806: Select at least one candidate sample word as a predicted sample word based on the vector distance between the first sample word and each candidate sample word, and use the predicted sample word to replace the first sample word in the first sample text to obtain a predicted sample text.

[0127] Step 807, calculate the loss function value according to the predicted enhanced text and the second sample text, and iteratively update the model to be trained according to the loss function value until the model convergence condition is met to obtain a trained model.

[0128] Step 808: input the first text into the trained model to obtain at least one second text, where the second text is an enhanced text of the first text.

[0129] The model first performs word segmentation on the first text to obtain first words. Then, a second word corresponding to each first word and having a pronunciation association relationship with the first word is determined. Finally, text enhancement processing is performed based on the first text and multiple second words to obtain at least one second text, and the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

[0130] Step 809: Use the enhanced second text to supplement training data, for example, to train a user intent recognition model.

[0131] The present application embodiment takes Mandarin as an example, and the pronunciation rules (for example, strong and weak reading rules) of the sample vocabulary of the target language are constructed for the text enhancement model, so that the model follows the strong and weak reading theory of the language, and the text content vector of the target language is characterized, so that the text enhancement model can process data based on the characteristics of the language, and the key hidden layer basis is provided for the subsequent language specificity extraction model. In this way, the quality of the training text can be improved while increasing the quantity and diversity of the training data. In addition, the present application embodiment is not limited to a single language, so the present application embodiment also provides the guidance of mixed language fusion, and the interactive influence of the pronunciation change characteristics between different languages ​​can be explored in actual operation, forming a multilingual mixed text enhancement data, for the subsequent expansion of a single language to a multilingual text, a powerful text enhancement implementation scheme is provided for the expansion of a few languages ​​to a multilingual text.

[0132] It should be understood that, although each step in the flowcharts involved in the above-mentioned embodiments is displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0133] Based on the same inventive concept, the embodiment of the present application also provides a text enhancement device for implementing the above-mentioned text enhancement method, and a model training device for implementing the above-mentioned model training method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more text enhancement devices and model training devices provided below can refer to the above-mentioned limitations on the text enhancement method and model training method, and the specific limitations will not be repeated here.

[0134] This embodiment also provides a text enhancement device, which can be integrated into Figure 1 In the server 110 shown. Fig. 9 FIG. 1 is a schematic diagram of the structure of a text enhancement device provided in an embodiment of the present application. Fig. 9As shown, the text enhancement device may include a segmentation module 901, an association module 902 and an enhancement module 903. The segmentation module 901 is used to segment the first text to obtain a plurality of first words. The association module 902 is used to determine the second word corresponding to each first word, and the pronunciation of the first word has an association relationship with the pronunciation of the second word. The enhancement module 903 is used to perform text enhancement processing based on the first text and the plurality of second words to obtain at least one second text, and the second text is obtained by replacing at least one first word in the first text with the corresponding second word.

[0135] The above-mentioned word segmentation module 901, association module 902 and enhancement module 903 can respectively execute steps 301-303 in the above-mentioned method embodiment. For the detailed introduction of these modules, please refer to the introduction of the corresponding steps of the above-mentioned method, which will not be repeated here.

[0136] This embodiment also provides a model training device, which can be integrated into Figure 1 In the server 110 shown. Fig.10 Schematic diagram of a model training device provided in an embodiment of the present application. Fig.10 As shown, the model training device may include an acquisition module 1001, a prediction module 1002 and a parameter adjustment module 1003. The acquisition module 1001 is used to acquire a training sample set, the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary. The prediction module 1002 is used to input the first sample text in each sample text pair into the model to be trained for text enhancement prediction, and obtain the predicted enhanced text. The parameter adjustment module 1003 is used to calculate the loss function value based on the predicted enhanced text and the second sample text, and adjust the parameters of the model to be trained according to the loss function value.

[0137] The acquisition module 1001, prediction module 1002 and parameter adjustment module 1003 can respectively execute steps 601-603 in the above method embodiment. For a detailed introduction of these modules, please refer to the introduction of the corresponding steps of the above method, which will not be repeated here.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a computer device, which can be a server or a terminal device. The computer device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the above-mentioned text enhancement method are implemented. Various functions are thereby implemented, for example: segmenting the first text to obtain multiple first words; determining the second word corresponding to each first word, and the pronunciation of the first word has an associated relationship with the pronunciation of the second word; performing text enhancement processing based on the first text and multiple second words to obtain at least one second text, and the second text is obtained by replacing at least one first word in the first text with the corresponding second word. When the processor executes the computer program, the steps of the above-mentioned model training method can also be implemented. Thereby, various functions are realized, for example: obtaining a training sample set, the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary; respectively inputting the first sample text in each sample text pair into the model to be trained for text enhancement prediction to obtain a predicted enhanced text; calculating a loss function value based on the predicted enhanced text and the second sample text, and iteratively updating the model to be trained based on the loss function value until the model convergence condition is reached to obtain a trained model.

[0139] By adopting the scheme of the application embodiment, based on the pronunciation change law of the language, by replacing the first vocabulary in the first text with the second vocabulary with a pronunciation association relationship, a large number of second texts with language change laws can be efficiently and concisely generated, which can increase the quantity and diversity of training data while improving the quality of the training text. In addition, since the embodiment of the application follows the pronunciation change law of the language, it is not limited to a single language. Therefore, the embodiment of the application also provides guidance for mixed language fusion, providing a powerful text enhancement implementation plan for the subsequent expansion of monolingual texts to multilingual texts and the expansion of minority languages ​​to multilingual texts.

[0140] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0141] In one embodiment, the computer device is a terminal device, for example, its internal structure diagram can be as follows: Fig.11As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a text enhancement method and a model training method are implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0142] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0144] Since the computer program stored in the computer-readable storage medium can execute any one of the text enhancement methods and model training methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the text enhancement methods and model training methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0145] Based on the same inventive concept, the embodiment of the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations in the above embodiments.

[0146] It should be noted that the object data (including but not limited to user device information, user personal information, etc.) and conversation data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0147] Any reference to the memory, database or other media used in the embodiments provided in the present application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (Re RAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0148] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0149] In the above-mentioned text enhancement device, model training device, computer-readable storage medium, computer equipment, and computer program product embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described text enhancement device, computer-readable storage medium, computer program product, computer equipment and its corresponding units can refer to the description of the text enhancement method and model training method in the above embodiments, and will not be repeated here.

[0150] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The above is a detailed introduction to a text enhancement method, a model training method, a text enhancement device, a model training device, a computer device, a computer-readable storage medium and a computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text enhancement method, It is characterized in that include: Segmenting the first text to obtain a plurality of first words; Determining a second word corresponding to each of the first words, wherein the pronunciation of the first words is associated with the pronunciation of the second words; Text enhancement processing is performed based on the first text and multiple second words to obtain at least one second text, where the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

2. The text enhancement method according to claim 1, It is characterized in that For each of the first words, determining a second word corresponding to the first word includes: determining a plurality of candidate words associated with the first word pronunciation; Determining a vector distance between the first word and each of the candidate words; At least one is selected from the plurality of candidate words as the second word based on the vector distance.

3. The text enhancement method according to claim 2, It is characterized in that The determining of a plurality of candidate words associated with the first word pronunciation comprises: Determining pronunciation information of the first vocabulary; Performing pronunciation association mapping on the pronunciation information of the first vocabulary to obtain a plurality of replacement pronunciation information; The vocabulary that matches each of the replacement pronunciation information is determined to obtain the plurality of candidate vocabulary.

4. The text enhancement method according to claim 3, It is characterized in that The pronunciation information of the first vocabulary is subjected to pronunciation association mapping to obtain a plurality of replacement pronunciation information, including: Generate a first vector code according to the phoneme pronunciation position and pronunciation method of the pronunciation information of the first vocabulary; generating a second vector code according to the tone of the pronunciation information of the first vocabulary; Generate a pronunciation information vector code of the first vocabulary according to the first vector code and the second vector code; Determining a pronunciation change feature of each of the first words according to the pronunciation information vector encoding of each of the first words; A plurality of replacement pronunciation information corresponding to each of the pronunciation information of the first vocabulary words is determined according to the pronunciation change characteristics.

5. The text enhancement method according to claim 3, It is characterized in that The step of determining each word that matches the replacement pronunciation information to obtain the plurality of candidate words comprises: respectively determining the pronunciation feature vector distance and length change between the pronunciation information of the first vocabulary and each of the replacement pronunciation information; When the pronunciation feature vector distance between the replacement pronunciation information and the pronunciation information of the first vocabulary is greater than a first threshold and the length change between the replacement pronunciation information and the pronunciation information of the first vocabulary satisfies a set change range, the vocabulary corresponding to the replacement pronunciation information is determined as the candidate vocabulary.

6. The text enhancement method according to claim 1, It is characterized in that The performing text enhancement processing based on the first text and the plurality of second words to obtain at least one second text comprises: Obtaining enhancement mode parameters, enhancement ratio, and enhancement position index of the first text; Text enhancement processing is performed based on the enhancement mode parameter, the enhancement ratio, the enhancement position index, the first text and a plurality of second words to obtain at least one second text.

7. A model training method, It is characterized in that include: Acquire a training sample set, the training sample set comprising a plurality of sample text pairs, each sample text pair comprising a first sample text and a second sample text, the first sample text comprising a first sample vocabulary, and the second sample text comprising a second sample vocabulary associated with the pronunciation of the first sample vocabulary; Inputting the first sample text in each sample text pair into the model to be trained to perform text enhancement prediction to obtain predicted enhanced text; A loss function value is calculated according to the predicted enhanced text and the second sample text, and parameters of the model to be trained are adjusted according to the loss function value.

8. The model training method according to claim 7, It is characterized in that The text enhancement prediction includes: Segmenting the first sample text to obtain a plurality of first sample words; Determine a predicted sample word corresponding to each of the first sample words, wherein the pronunciation of the predicted sample word is associated with the pronunciation of the first sample word; Text enhancement processing is performed based on the first sample text and multiple predicted sample words to obtain at least one predicted enhanced text, where the predicted enhanced text is obtained by replacing at least one first sample word in the first sample text with a corresponding predicted sample word.

9. The model training method according to claim 8, It is characterized in that For each of the first sample words, determining a predicted sample word corresponding to the first sample word includes: determining a plurality of candidate sample words associated with the first sample word pronunciation; Determining a vector distance between the first sample vocabulary and each of the candidate sample vocabulary; At least one is selected from the plurality of candidate sample words as the predicted sample word based on the vector distance.

10. A text enhancement device, It is characterized in that include: A word segmentation module, used for performing word segmentation on the first text to obtain a plurality of first words; an associating module, configured to determine a second word corresponding to each of the first words, wherein the pronunciation of the first words is associated with the pronunciation of the second words; The enhancement module is used to perform text enhancement processing based on the first text and multiple second words to obtain at least one second text, where the second text is obtained by replacing at least one first word in the first text with a corresponding second word.

11. A model training device, It is characterized in that include: An acquisition module, configured to acquire a training sample set, wherein the training sample set includes a plurality of sample text pairs, each sample text pair includes a first sample text and a second sample text, the first sample text includes a first sample vocabulary, and the second sample text includes a second sample vocabulary associated with the pronunciation of the first sample vocabulary; A prediction module, used for inputting the first sample text in each sample text pair into the model to be trained to perform text enhancement prediction to obtain a predicted enhanced text; A parameter adjustment module is used to calculate a loss function value based on the predicted enhanced text and the second sample text, and adjust the parameters of the model to be trained according to the loss function value.

12. A computer device, It is characterized in that It includes a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the text enhancement method as described in any one of claims 1 to 6 or the model training method as described in any one of claims 7 to 9.

13. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute the steps of the text enhancement method as described in any one of claims 1 to 6 or the model training method as described in any one of claims 7 to 9.

14. A computer program product, It is characterized in that The computer program product comprises a computer program, and the computer program is used by a processor to execute the steps of the text enhancement method as described in any one of claims 1 to 6 or the model training method as described in any one of claims 7 to 9.

Images