Real-time voice translation method for slang semantic association

Abstract

The invention relates to the technical field of language processing, in particular to a slang semantic association real-time speech translation method, which comprises the following steps of: creating a slang semantic association lexicon, storing words or phrases in the slang semantic association lexicon, marking corresponding slang for each word or phrase in the slang semantic association lexicon, and translating the slang semantic association lexicon into a slang semantic association lexicon; synchronously marking contents based on each word or phrase stored in the slang semantic association lexicon, and carrying out differentiated management on the words or phrases; by constructing a slang semantic association lexicon and a slang lexicon, user voice information is subjected to noise reduction processing, text information is output, synchronous comprehensive retrieval is performed, slang in the text information is captured, matched words or phrases are further searched in the lexicon for replacement, and the user experience is improved. According to the method, the slang-containing text information is converted into statements which are easier to understand by the robot, so that a man-machine interaction scene is served, it is guaranteed that conversation is smoother in the man-machine interaction process, and better man-machine interaction experience is brought to a user.

Description

Technical Field

[0001] The present invention relates to the field of language processing technology, and in particular to a real-time speech translation method for slang semantic association. Background Art

[0002] Slang is a popular, colloquial form of expression widely used among the public. It often has regional characteristics, reflecting the habits, cultural traditions, and emotional attitudes of the people of a particular region. Slang is particularly prominent in spoken English conversations, contributing to the increasing succinctness of spoken English.

[0003] The invention patent with application number 202211456897.X discloses a method for generating a semantic association analysis model, which includes: obtaining multiple historical query information and the query frequency corresponding to each historical query information; classifying the multiple historical query information according to the query frequency to obtain query information sets corresponding to multiple query types; based on the query type corresponding to any query information set, performing association analysis on any query information set to obtain at least one historical query information pair corresponding to any query information set and association label information corresponding to any historical query information pair; the association label information represents the degree of association between the two historical query information in any historical query information pair; based on the at least one historical query information pair and the association label information, training a preset semantic association analysis model to obtain a target semantic association analysis model.

[0004] This application aims to solve the problem that "existing technologies often use language representation models to encode query information combinations, and then train semantic association analysis models based on pre-labeled query information combination samples. However, in the existing model training process, there are problems such as insufficient manually labeled data and relatively single training data categories, which leads to insufficient semantic association analysis accuracy of the semantic association analysis model."

[0005] However, in human-computer interaction scenarios, since slang may be present in the user's spoken language, the robot may not be able to correctly understand the user's narration, resulting in a dull response from the robot or failure to successfully output responses within the robot's capabilities.

[0006] To this end, a real-time speech translation method based on slang semantic association is proposed. Summary of the Invention

[0007] In view of the above shortcomings of the prior art, the present invention provides a real-time speech translation method based on slang semantic association, which solves the technical problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A real-time speech translation method based on slang semantic association, comprising:

[0010] A slang semantic association vocabulary is created, words or phrases are stored in the slang semantic association vocabulary, corresponding slang is marked for each word or phrase in the slang semantic association vocabulary, and words or phrases are differentiated and managed synchronously based on the marked content of each word or phrase stored in the slang semantic association vocabulary; user voice information is collected in real time, the user voice information is converted into text information, and a slang vocabulary is generated according to the slang corresponding to each differentiated storage interval in the slang semantic association vocabulary; each slang in the slang vocabulary is used as a query target, and the same item is searched in the text information converted from the user voice information; if the same item is not found, the query is terminated; if the same item is found, the query target corresponding to the same item is obtained, and the word or phrase in the corresponding differentiated storage interval in the slang semantic association vocabulary is retrieved based on the obtained query target; a word or phrase is selected from the retrieved word or phrase, and the text information obtained from the user voice information conversion that has the same marked content as the selected word or phrase is replaced with the selected word or phrase; after the replacement is completed, the text information obtained from the user voice information conversion is output.

[0011] Furthermore, when performing differentiated management of the words or phrases stored in the slang semantic association word library, a plurality of differentiated storage intervals are created in the slang semantic association word library, and the words or phrases are stored in the differentiated storage intervals so that the tag content of the words or phrases stored in the same differentiated storage interval is the same;

[0012] The words or phrases stored in the slang semantic association word library are manually uploaded by the user terminal. Each time the user terminal uploads a word or phrase, the uploaded word or phrase is marked with the corresponding slang before the upload operation is performed;

[0013] Among them, when the user uploads a word or phrase to the slang semantic association vocabulary, the slang semantic association vocabulary synchronously uses the word or phrase uploaded by the user as a query target, and searches for the same item in the words or phrases stored in the slang semantic association vocabulary. When the same item is found, the query target is discarded. If the same item is not found, it is placed in the corresponding differentiated storage interval based on the tag content of the uploaded word or phrase.

[0014] Furthermore, the operation of searching for the same item in the slang semantic association vocabulary for the query target is:

[0015]

[0016] Where: n is the total number of words and phrases in the slang semantic association vocabulary; m is the total number of tagged slang corresponding to each storage interval in the slang semantic association vocabulary; I(c i ,t)、I(c j ,f) is the indicator function; c i is the i-th word or phrase in the slang semantic association vocabulary; c j is the jth tagged slang in the slang semantic association vocabulary; t and f are the query target and the slang tagged by the query target;

[0017] Among them, t and f both satisfy When the query target is found, the discard operation is performed on the query target, otherwise, the storage operation is performed.

[0018] Furthermore, the user voice information is audio data in any data format. Before the user voice information is converted into text information, noise reduction processing is synchronously performed on the user voice information. After the noise reduction processing is completed, the operation of converting the user voice information into text information is executed;

[0019] In the stage of converting the user's voice information into text information, any application software having the function of converting voice information into text information is used to perform the conversion operation;

[0020] In the slang word library generation stage, slang words marked by each word or phrase are captured in each storage interval of the slang semantic association word library, and the slang word library is constructed with all the captured slang words.

[0021] Furthermore, the noise reduction processing logic of the user voice information is expressed as:

[0022]

[0023] Where: Y m [k] is the spectrum after noise reduction; L is the length of the speech signal; x m [q] is the discrete time sequence corresponding to the m-th frame signal; r is the imaginary unit; N is the number of points selected when performing fast Fourier transform; k is the frequency discrete point number; q is the discrete time point number; M m [k] is the spectrum mask;

[0024] The noise reduction process is completed by obtaining the spectrum of each frame of the user's voice information after noise reduction based on the above formula, and then performing an inverse short-time Fourier transform on the spectrum of each frame after noise reduction. The signals of each frame are spliced ​​together in sequence to reconstruct the noise reduction speech audio time series signal.

[0025] Furthermore, the spectrum mask M m [k] is calculated using the following formula:

[0026]

[0027] Where: T[k] is the frequency band energy threshold;

[0028] described

[0029] Where: a and b are weight coefficients; P is the number of previous and next frames considered when calculating the frequency band energy threshold; E s [k] is the average energy of the s-th frame signal at frequency point k; std represents the standard deviation operation.

[0030] Furthermore, when no identical item is found in the search, the process jumps synchronously to the output stage of the text information converted from the user's voice information, and performs an output operation on the text information converted from the user's voice information.

[0031] Furthermore, the query targets corresponding to the same items in the query are all slangs;

[0032] When retrieving words or phrases from the slang semantic association vocabulary, slang is used as the recognition target, the tag content of each word or phrase in the slang semantic association vocabulary is recognized, and the word or phrase with the same tag content as the recognition target is used as the retrieving target, and the retrieving operation is performed.

[0033] Furthermore, after retrieving words or phrases from the slang semantic association lexicon, all retrieved words or phrases are simultaneously distinguished based on their marked slang to obtain a plurality of word or phrase sets, and a word or phrase is selected from each word or phrase set for replacement operation;

[0034] Wherein, if there is at least one word or phrase in the set of words or phrases, and there is only one word or phrase in the set of words or phrases, the words or phrases in the set are directly selected for the replacement operation;

[0035] When the number of words or phrases included in the word or phrase set is not one, the context distance between each word or phrase and the text information converted from the user's voice information is calculated, and the word or phrase with the smallest context distance is selected for the replacement operation.

[0036] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0037] The present invention provides a real-time speech translation method for slang semantic association. During execution, the method constructs a slang semantic association vocabulary and a slang vocabulary to perform noise reduction processing on user voice information and then output text information. A simultaneous comprehensive search is performed to capture slang in the text information, and matching words or phrases are further searched in the vocabulary for replacement. The text information containing slang is converted into sentences that are easier for robots to understand, thereby serving human-computer interaction scenarios, ensuring smoother dialogue during the human-computer interaction process, and providing users with a better human-computer interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0039] Figure 1 A flowchart of a real-time speech translation method based on slang semantic association is shown in FIG. DETAILED DESCRIPTION

[0040] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0041] The present invention will be further described below with reference to the embodiments.

[0042] Example:

[0043] A real-time speech translation method of slang semantic association in this embodiment, such as Figure 1 Shown, including:

[0044] Creating a slang semantic association lexicon, storing words or phrases in the slang semantic association lexicon, marking corresponding slang for each word or phrase in the slang semantic association lexicon, and simultaneously distinguishing and managing the words or phrases based on the marked content of each word or phrase stored in the slang semantic association lexicon;

[0045] When performing differentiated management on the words or phrases stored in the slang semantic association word library, a plurality of differentiated storage intervals are created in the slang semantic association word library, and the words or phrases are stored in the differentiated storage intervals so that the tag contents of the words or phrases stored in the same differentiated storage interval are all the same;

[0046] The words or phrases stored in the slang semantic association word library are manually uploaded by the user end. Each time the user end uploads a word or phrase, the uploaded word or phrase is marked with the corresponding slang before the upload operation is performed;

[0047] When a user uploads a word or phrase to the slang semantic association lexicon, the slang semantic association lexicon uses the word or phrase uploaded by the user as a query target, searches for identical terms among the words or phrases stored in the slang semantic association lexicon, and discards the query target when an identical term is found. If no identical term is found, the query target is placed in a corresponding differentiated storage interval based on the tag content of the uploaded word or phrase.

[0048] The query target searches for the same item in the slang semantic association vocabulary as follows:

[0049]

[0050] Where: n is the total number of words and phrases in the slang semantic association vocabulary; m is the total number of tagged slang corresponding to each storage interval in the slang semantic association vocabulary; I(c i ,t)、I(c j ,f) is the indicator function; c i is the i-th word or phrase in the slang semantic association vocabulary; c j is the jth tagged slang in the slang semantic association vocabulary; t and f are the query target and the slang tagged by the query target;

[0051] Among them, t and f both satisfy When , the query target is discarded, otherwise, the query target is stored;

[0052] Through the above logic formula, designated storage or discard decision conditions are provided for words or phrases newly loaded into the slang semantic association vocabulary.

[0053] Collect user voice information in real time, convert the user voice information into text information, and generate a slang word library based on the slang words corresponding to each storage interval in the slang semantic association word library;

[0054] The user voice information is audio data in any data format. Before converting the user voice information into text information, the user voice information is subjected to noise reduction processing. After the noise reduction processing is completed, the user voice information is converted into text information.

[0055] In the stage of converting the user's voice information into text information, any application software having the function of converting voice information into text information is used to perform the conversion operation;

[0056] In the slang word library generation stage, slang words marked by each word or phrase are captured in each storage interval of the slang semantic association word library, and a slang word library is constructed with all the captured slang words;

[0057] The noise reduction processing logic of user voice information is expressed as follows:

[0058]

[0059] Where: Y m [k] is the spectrum after noise reduction; L is the length of the speech signal; x m [q] is the discrete time sequence corresponding to the m-th frame signal; r is the imaginary unit; N is the number of points selected when performing fast Fourier transform; k is the frequency discrete point number; q is the discrete time point number; M m [k] is the spectrum mask;

[0060] The noise reduction process is completed by obtaining the spectrum of each frame of the user's voice information after noise reduction based on the above formula, and then performing an inverse short-time Fourier transform on the spectrum of each frame after noise reduction. The signals of each frame are sequentially spliced ​​together to reconstruct the noise reduction time series signal of the voice audio.

[0061] Spectrum Mask M m [k] is calculated using the following formula:

[0062]

[0063] Where: T[k] is the frequency band energy threshold;

[0064]

[0065] Where: a and b are weight coefficients; P is the number of previous and next frames considered when calculating the frequency band energy threshold; E s [k] is the average energy of the s-th frame signal at frequency point k; std represents the standard deviation operation;

[0066] Through the above logic formula, the user's voice information is subjected to noise reduction processing to ensure that the text information converted from the user's voice information is accurately and stably output.

[0067] Each slang word in the slang word library is used as a query target, and the same item is searched in the text information converted from the user's voice information;

[0068] If no identical item is found, the process ends; if the identical item is found, a query target corresponding to the identical item is obtained, and a word or phrase in a corresponding distinguished storage interval is retrieved from the slang semantic association word library based on the obtained query target;

[0069] Select a word or phrase from the retrieved words or phrases, and replace the text in the text information converted from the user's voice information with the selected word or phrase, which has the same marking content as the selected word or phrase;

[0070] Output of text information converted from user voice information after replacement is completed;

[0071] The query targets corresponding to the same items are all slang;

[0072] When retrieving words or phrases from the slang semantic association vocabulary, the slang is used as a recognition target, the tag content of each word or phrase in the slang semantic association vocabulary is recognized, and the word or phrase with the same tag content as the recognition target is used as a retrieving target, and the retrieving operation is performed;

[0073] After retrieving words or phrases from the slang semantic association vocabulary, all retrieved words or phrases are simultaneously distinguished based on their marked slang to obtain a plurality of word or phrase sets, and a word or phrase is selected from each word or phrase set for replacement operation;

[0074] Wherein, if there is at least one word or phrase in the set of words or phrases, and there is only one word or phrase in the set of words or phrases, the words or phrases in the set are directly selected for the replacement operation;

[0075] When the number of words or phrases included in the word or phrase set is not one, the context distance between each word or phrase and the text information converted from the user's voice information is calculated, and the word or phrase with the smallest context distance is selected for the replacement operation.

[0076] In this embodiment, in a human-computer interaction scenario, the user's narration is converted into sentences that are easier for the robot to understand, further improving the continuous interaction capability of the robot and the user.

[0077] It should be noted that during the above replacement operation phase, the context distance reference is applied, and the following logic is used to calculate the context distance:

[0078] Words (phrases) and slang are recorded as A and B respectively;

[0079]

[0080] Where: D KL (P A ||P B ) and D KL (P B ||P A ) has the same calculation logic;

[0081] Where: D context is the contextual distance between word (phrase) A and slang B; DKL (P A ||P B ) is the KL divergence of word (phrase) A relative to slang B; D KL (P B ||P A ) is the KL divergence of slang B relative to word (phrase) A;

[0082]

[0083] Where: is the context probability distribution vector of word (phrase) A; is the context probability distribution vector of slang B;

[0084] like Figure 1 As shown, when the same item is not found in the query, the process jumps to the output stage of the text information converted from the user voice information, and performs an output operation on the text information converted from the user voice information.

[0085] Through the above settings, the execution logic of the method in the above embodiment is further improved, ensuring that when the same item is not queried, the above method is further executed based on the specified logic, forming an execution logic closed loop.

[0086] In summary, during the execution of the method in the above embodiment, by constructing a slang semantic association vocabulary and a slang vocabulary, the user voice information is subjected to noise reduction processing and then output as text information, and a simultaneous comprehensive search is performed to capture the slang in the text information. The vocabulary is further searched for matching words or phrases for replacement, and the text information containing slang is converted into sentences that are easier for the robot to understand, thereby serving the human-computer interaction scenario, ensuring a smoother conversation during the human-computer interaction process, and bringing a better human-computer interaction experience to the user.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A real-time speech translation method for slang semantic association, characterized in that: include: Creating a slang semantic association lexicon, storing words or phrases in the slang semantic association lexicon, marking corresponding slang for each word or phrase in the slang semantic association lexicon, and simultaneously distinguishing and managing the words or phrases based on the marked content of each word or phrase stored in the slang semantic association lexicon; Collect user voice information in real time, convert the user voice information into text information, and generate a slang word library based on the slang words corresponding to each storage interval in the slang semantic association word library; Each slang word in the slang word library is used as a query target, and the same item is searched in the text information converted from the user's voice information; When no identical items are found, the query ends. When the same item is found in the search, a query target corresponding to the same item is obtained, and a word or phrase in a corresponding distinguished storage interval is retrieved from the slang semantic association word library based on the obtained query target; Select a word or phrase from the retrieved words or phrases, and replace the text in the text information converted from the user's voice information with the selected word or phrase, which has the same marking content as the selected word or phrase; After the replacement is completed, the text information obtained by converting the user's voice information is output.

2. The method for real-time speech translation of slang semantic association according to claim 1, wherein: When performing differentiated management on the words or phrases stored in the slang semantic association word library, a plurality of differentiated storage intervals are created in the slang semantic association word library, and the words or phrases are stored in the differentiated storage intervals so that the tag contents of the words or phrases stored in the same differentiated storage interval are all the same; The words or phrases stored in the slang semantic association word library are manually uploaded by the user terminal. Each time the user terminal uploads a word or phrase, the uploaded word or phrase is marked with the corresponding slang before the upload operation is performed; Among them, when the user uploads a word or phrase to the slang semantic association vocabulary, the slang semantic association vocabulary synchronously uses the word or phrase uploaded by the user as a query target, and searches for the same item in the words or phrases stored in the slang semantic association vocabulary. When the same item is found, the query target is discarded. If the same item is not found, it is placed in the corresponding differentiated storage interval based on the tag content of the uploaded word or phrase.

3. The method for real-time speech translation of slang semantic association according to claim 2, wherein: The operation of searching for the same item in the slang semantic association vocabulary for the query target is: Where: n is the total number of words and phrases in the slang semantic association vocabulary; m is the total amount of tagged slang words corresponding to each storage interval in the slang semantic association vocabulary; I(c i ,t)、I(c j ,f) is the indicator function; c i is the i-th word or phrase in the slang semantic association vocabulary; cj is the j-th marked slang in the slang semantic association vocabulary; t and f are the query target and the slang marked by the query target; Among them, t and f both satisfy When the query target is found, the discard operation is performed on the query target, otherwise, the storage operation is performed.

4. The method for real-time speech translation of slang semantic association according to claim 1, wherein: The user voice information is audio data in any data format. Before converting the user voice information into text information, the user voice information is subjected to noise reduction processing. After the noise reduction processing is completed, the user voice information is converted into text information. In the stage of converting the user's voice information into text information, any application software having the function of converting voice information into text information is used to perform the conversion operation; In the slang word library generation stage, slang words marked by each word or phrase are captured in each storage interval of the slang semantic association word library, and the slang word library is constructed with all the captured slang words.

5. The method for real-time speech translation of slang semantic association according to claim 4, wherein: The noise reduction processing logic of the user voice information is expressed as follows: Where: Y m [k] is the spectrum after noise reduction; L is the length of the speech signal; x m [q] is the discrete time sequence corresponding to the m-th frame signal; r is the imaginary unit; N is the number of points selected when performing fast Fourier transform; k is the frequency discrete point number; q is the discrete time point number; M m [k] is the spectrum mask; The noise reduction process is completed by obtaining the spectrum of each frame of the user's voice information after noise reduction based on the above formula, and then performing an inverse short-time Fourier transform on the spectrum of each frame after noise reduction. The signals of each frame are spliced ​​together in sequence to reconstruct the noise reduction speech audio time series signal.

6. The method for real-time speech translation of slang semantic association according to claim 5, characterized in that: The spectrum mask M m [k] is calculated using the following formula: Where: T[k] is the frequency band energy threshold; described Where: a and b are weight coefficients; P is the number of previous and next frames considered when calculating the frequency band energy threshold; E s [k] is the average energy of the s-th frame signal at frequency point k; std represents the standard deviation operation.

7. The method for real-time speech translation of slang semantic association according to claim 1, wherein: When no identical item is found in the search, the process jumps synchronously to the output stage of the text information converted from the user's voice information, and performs an output operation on the text information converted from the user's voice information.

8. The method for real-time speech translation of slang semantic association according to claim 1, wherein: The query targets corresponding to the same items in the query are all slang; When retrieving words or phrases from the slang semantic association vocabulary, slang is used as the recognition target, the tag content of each word or phrase in the slang semantic association vocabulary is recognized, and the word or phrase with the same tag content as the recognition target is used as the retrieving target, and the retrieving operation is performed.

9. The method for real-time speech translation of slang semantic association according to claim 1, wherein: After retrieving words or phrases from the slang semantic association vocabulary, all retrieved words or phrases are simultaneously distinguished based on their marked slang to obtain a plurality of word or phrase sets, and a word or phrase is selected from each word or phrase set for replacement operation; Wherein, if there is at least one word or phrase in the set of words or phrases, and there is only one word or phrase in the set of words or phrases, the words or phrases in the set are directly selected for the replacement operation; When the number of words or phrases included in the word or phrase set is not one, the context distance between each word or phrase and the text information converted from the user's voice information is calculated, and the word or phrase with the smallest context distance is selected for the replacement operation.

Images