Real-time speech translation method, system and visual terminal

By using pause points to identify and generate abstract sequences in the speech translation technology, the problem of long and high cost of speech translation in the prior art is solved, real-time and key content are highlighted, and communication efficiency is improved.

CN114781407BActive Publication Date: 2024-11-22IOL WUHAN INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210420352.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-21
Publication Date
2024-11-22
Estimated Expiration
2042-04-21

AI Technical Summary

Technical Problem

When existing real-time voice translation technology processes a large number of phonological sequences, the translation time is long and costly, and it is unable to effectively highlight the key content in voice interactions, affecting communication efficiency, especially in situations where professional vocabulary is included.

Method used

By monitoring pause points in the speech sequence, converting them into a text sequence and generating a summary sequence, only the abstract sequence is translated and displayed, combining speech string recognition and text and picture display modules to highlight key content.

Benefits of technology

The amount of data in speech translation is reduced, the real-time translation is ensured, the integrity and readability of key content is improved, and communication efficiency is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114781407B_ABST
    Figure CN114781407B_ABST
Patent Text Reader

Abstract

The present invention proposes a real-time speech translation method, system and visualization terminal, which belong to the field of real-time speech translation and display technology. The method includes S1: starting the target speech sequence recognition process when the speech sequence input is monitored; S2: identifying the first pause point when the input speech sequence is identified to have the target speech string; S3: converting the speech sequence before the first pause point into a text sequence; S4: generating a summary sequence based on the text sequence; S5: translating the summary sequence and displaying it. The system includes a speech input module, a speech string recognition module, a speech-to-text module, a summary generation module, a text translation module and a text-image display module. The present invention also proposes a visualization terminal for implementing the method. The present invention generates a summary sequence based on the target speech string detected by the original speech sequence before executing the translation display, which reduces the data volume of the speech translation and can maintain the integrity and readability of the key content while ensuring the real-time performance of the speech translation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of real-time speech translation and display, and in particular relates to a real-time speech translation method, a system and a visualization terminal, and a storage medium for realizing the method. Background Art

[0002] Real-time voice translation, also known as real-time simultaneous translation, can recognize an audio stream of unlimited length into text in real time through real-time voice transcription. In the existing technology, there are many mainstream translation software that can realize real-time voice translation, and even claim to support translation between Chinese and English, Japanese, Russian, Korean, French, Spanish, Vietnamese, and other languages, covering commonly used international languages, and are widely used in scenarios such as live broadcasts of international events and cross-language conferences.

[0003] Chinese invention patent publication CN114239613A proposes a real-time speech translation method, which outputs the translation corresponding to the original language text at the current moment according to the original language text at the current moment and the prefix of a certain length of the translation at the historical moment, so as to ensure that the prefix of a certain length of the translation at the current moment is consistent with the prefix of a certain length of the translation at the historical moment, thereby effectively alleviating the problem of instability of the output translation in real-time speech translation.

[0004] However, in actual applications, both the above mainstream translation software and existing speech translation methods are based on algorithms or models to directly translate the original speech sequence word by word. When the original speech sequence data volume is large, the translation time is long and the translation cost is high (many translation software are charged by data volume or translation time).

[0005] However, the inventors have found that in actual situations involving voice interaction and communication, not all voice interactions are effectively recorded. Since each person's expression habits are different, there are inevitably some meaningless voice sequences in the voice recording that are meaningless to the communication between the two parties. If these sequences are omitted, it will not affect the communication efficiency. Therefore, it is not necessary to translate the original text word for word.

[0006] In some special occasions, especially those involving specific words and words in specific formats, such as voice communication in the field of chemistry and molecule, the voice sequence usually contains a large number of professional chemical words. The existing technology of translating the original voice into text word by word cannot highlight the key subsequences in these voice sequences, making it impossible for both parties in the voice interaction to quickly and accurately absorb the key content, affecting the efficiency of voice communication.

[0007] Correspondingly, the two parties in the interaction may pay more attention to the key parts of the speech sequence, and hope to identify and highlight this key target sequence to achieve better communication results. However, the existing technology does not take these issues into consideration. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes a real-time speech translation method, system and visualization terminal, and a storage medium for implementing the method.

[0009] In a first aspect of the present invention, a method for real-time speech translation is provided, the method comprising the following steps:

[0010] S1: When a speech sequence input is detected, the target speech sequence recognition process is started;

[0011] S2: when the target speech sequence recognition process recognizes that the input speech sequence has a target speech string, continue to recognize the first pause point of the input speech sequence after the target speech string;

[0012] S3: converting the speech sequence before the first pause point into a text sequence;

[0013] S4: generating a summary sequence based on the text sequence;

[0014] S5: Translate the summary sequence and display it.

[0015] In the technical solution of the present invention, the input speech sequence is a source speech sequence in a first language;

[0016] The target speech string is a preset target noun speech string, and the target noun includes chemical substances and / or chemical reaction formulas.

[0017] The step S3 converts the speech sequence before the first pause point into a text sequence, specifically comprising:

[0018] The source speech sequence in the first language before the first pause point is converted into a source text sequence in the first language.

[0019] The step S4 generates a summary sequence based on the text sequence, specifically comprising:

[0020] The text sequence includes a target text sequence and a non-target text sequence;

[0021] The target text sequence is a text sequence corresponding to the target voice string;

[0022] The non-target text sequence is other text sequences in the text sequence converted in step S3 except the target text sequence;

[0023] Generate a summary sequence to be combined based on the non-target text sequence;

[0024] The summary sequence to be combined is semantically combined with the target text sequence to obtain the summary sequence.

[0025] The source text sequence in the first language includes a target text sequence corresponding to the target speech string;

[0026] The step S4 of generating a summary sequence based on the text sequence specifically includes:

[0027] generating a source text summary sequence in the first language based on the source text sequence in the first language, wherein the source text summary sequence retains the target text sequence;

[0028] The step S5 translates the summary sequence and displays it, specifically comprising:

[0029] translating a source text summary sequence in the first language into an output text sequence in a second language, wherein the output text sequence includes a target output text in the second language corresponding to the target text sequence;

[0030] The target output text in the second language corresponding to the target text sequence is displayed in a predetermined format.

[0031] Displaying the target output text in the second language corresponding to the target text sequence in a predetermined format, specifically including:

[0032] At least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances are displayed in a picture.

[0033] In a second aspect of the present invention, a real-time speech translation system is provided, the system comprising a speech input module, a speech string recognition module, a speech-to-text module, a summary generation module, a text translation module and a text-image display module.

[0034] The speech input module is used to input a speech sequence, and the input speech sequence is a speech sequence of a first source language;

[0035] The speech string recognition module is used to recognize whether the speech sequence of the first source language contains a target speech string of the first source language;

[0036] The speech-to-text module is used to convert the speech sequence in the first source language into a source text sequence in the first source language;

[0037] The summary generation module is used to generate a summary text in the first source language based on the source text sequence in the first source language;

[0038] The text translation module is used to translate the summary text in the first source language into a target text in a second target language;

[0039] The text-image display module is used to display the target text in the second target language in a predetermined text-image format;

[0040] Wherein, the target speech string is a preset target noun speech string, and the target noun includes chemical substances and / or chemical reaction formulas;

[0041] The displaying of the target text in the second target language in a predetermined graphic format specifically includes: displaying at least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances in the form of a picture.

[0042] The system also includes a voice broadcast module;

[0043] The voice broadcast module is used for voice broadcasting the target text in the second target language.

[0044] The system also includes a stall point detection module;

[0045] The pause point detection module is used to detect pause points in a speech sequence of a first source language.

[0046] The voice string recognition module is connected to the pause point monitoring module;

[0047] When the pause point detection module detects a pause point in the speech sequence of the first source language, the speech string recognition module is activated.

[0048] In a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0049] In a fourth aspect of the present invention, a visualization terminal is provided, which includes a human-machine display interface, the human-machine display interface is connected to a memory and a processor, the processor stores computer program instructions, and the computer program instructions are executed by the processor to implement the steps of the method described in the first aspect.

[0050] The present invention generates a summary sequence based on the target voice string detected by the original voice sequence before performing translation display, which reduces the data volume of the voice translation and can maintain the integrity and readability of the key content while ensuring the real-time performance of the voice translation.

[0051] Different from the prior art approach of translating the original text word by word and sentence by sentence, the present invention generates a source text summary sequence in the first language based on the source text sequence in the first language before performing the formal translation, and then only performs translation on the source text summary sequence in the first language, thereby reducing the data volume of speech translation and ensuring the real-time performance of speech translation.

[0052] At the same time, when generating a summary, the embodiment of the present invention does not generate the summary based on the entire text sequence, but instead separates the target text sequence, generates a to-be-combined summary sequence based on the non-target text sequence, and then semantically combines the to-be-combined summary sequence with the target text sequence to obtain the summary sequence, thereby ensuring that the summary sequence completely retains the target text sequence.

[0053] Further advantages of the present invention will be further reflected in detail in the specific embodiments section in conjunction with the drawings of the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 It is a main flow chart of a method for real-time speech translation according to an embodiment of the present invention;

[0056] Figure 2 yes Figure 1 A data processing flow chart of the real-time speech translation method;

[0057] Figure 3-Figure 5 is a schematic diagram of speech translation effects under different preset target speech strings;

[0058] Figure 6 A system architecture diagram of a real-time speech translation system according to an embodiment of the present invention;

[0059] Figure 7 It is a system architecture diagram of a real-time speech translation system according to another preferred embodiment of the present invention;

[0060] Figure 8 is realized Figure 1 or Figure 2 A schematic diagram of the structure of a visualization terminal device of the method. DETAILED DESCRIPTION

[0061] The invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0062] First of all, it should be noted that the present invention relates to translation between different languages, so the embodiment part will reflect expressions in different languages, such as English sequence input, which is an indispensable part of explaining the technical solution of the present invention and will not cause the technical solution of the present invention to be unclear, and complies with the relevant provisions of the Patent Law.

[0063] Figure 1 The present invention is a flow chart of a method for real-time speech translation according to an embodiment of the present invention.

[0064] Figure 1 The method includes steps S1-S5, and each step is specifically implemented as follows:

[0065] S1: When a speech sequence input is detected, the target speech sequence recognition process is started;

[0066] S2: when the target speech sequence recognition process recognizes that the input speech sequence has a target speech string, continue to recognize the first pause point of the input speech sequence after the target speech string;

[0067] Obviously, the “first pause point after the target speech string” here means “the first pause point after the target speech string”;

[0068] S3: converting the speech sequence before the first (first) pause point into a text sequence;

[0069] S4: generating a summary sequence based on the text sequence;

[0070] S5: Translate the summary sequence and display it.

[0071] For more details, see Figure 2 .

[0072] The input speech sequence is a source speech sequence in a first language;

[0073] Converting a source speech sequence in the first language before the first pause point into a source text sequence in the first language;

[0074] It can be seen that this embodiment does not directly convert all or directly the source speech sequence of the first language into the source text sequence of the first language, but when the target speech sequence recognition process recognizes that the input speech sequence has a target speech string, it continues to recognize the first pause point of the input speech sequence after the target speech string, and then converts the source speech sequence of the first language before the first pause point into the source text sequence of the first language, thereby reducing the amount of data for subsequent transcription and translation processes and reducing the amount of data for the real-time performance of speech transcription and translation.

[0075] Next, in Figure 2 As a further improvement, a source text summary sequence in the first language is generated based on the source text sequence in the first language, and the source text summary sequence in the first language is translated into an output text sequence in the second language.

[0076] It can be seen that, unlike the prior art method of translating the original text word by word and sentence by sentence, this embodiment generates a source text summary sequence in the first language based on the source text sequence in the first language before performing the formal translation, and then only performs translation on the source text summary sequence in the first language, which reduces the data volume of speech translation and can ensure the real-time performance of speech translation.

[0077] Finally, the target output text in the second language corresponding to the target text sequence is displayed in a predetermined format.

[0078] Specifically, in various embodiments of the present invention, the target speech string is a preset target noun speech string, and the target noun includes a chemical substance and / or a chemical reaction formula;

[0079] At least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances are displayed in a picture.

[0080] More specifically, the input speech sequence is a source speech sequence in a first language;

[0081] The target speech string is a preset target noun speech string, and the target noun includes a chemical substance and / or a chemical reaction formula;

[0082] The step S3 converts the speech sequence before the first pause point into a text sequence, specifically comprising:

[0083] The source speech sequence in the first language before the first pause point is converted into a source text sequence in the first language.

[0084] The step S4 generates a summary sequence based on the text sequence, specifically comprising:

[0085] The text sequence includes a target text sequence and a non-target text sequence;

[0086] The target text sequence is a text sequence corresponding to the target voice string;

[0087] The non-target text sequence is other text sequences in the text sequence converted in step S3 except the target text sequence;

[0088] Generate a summary sequence to be combined based on the non-target text sequence;

[0089] The summary sequence to be combined is semantically combined with the target text sequence to obtain the summary sequence.

[0090] Meanwhile, the source text sequence in the first language includes a target text sequence corresponding to the target speech string;

[0091] The step S4 of generating a summary sequence based on the text sequence specifically includes:

[0092] generating a source text summary sequence in the first language based on the source text sequence in the first language, wherein the source text summary sequence retains the target text sequence;

[0093] The step S5 translates the summary sequence and displays it, specifically comprising:

[0094] translating a source text summary sequence in the first language into an output text sequence in a second language, wherein the output text sequence includes a target output text in the second language corresponding to the target text sequence;

[0095] The target output text in the second language corresponding to the target text sequence is displayed in a predetermined format.

[0096] It can be seen that when generating a summary, the embodiment of the present invention does not generate the summary based on the entire text sequence, but instead separates the target text sequence, generates a to-be-combined summary sequence based on the non-target text sequence, and then semantically combines the to-be-combined summary sequence with the target text sequence to obtain the summary sequence, thereby ensuring that the summary sequence completely retains the target text sequence.

[0097] Therefore, in summary, it can be seen that the technical solution of the present invention reduces the data volume of speech translation compared to the prior art, and can maintain the integrity and readability of key content while ensuring the real-time performance of speech translation.

[0098] See next Figure 3-Figure 5 , further illustrating the improved technical effects of the present invention.

[0099] exist Figure 3-Figure 5 In the embodiment of , the speech sequence input is a spoken English sequence:

[0100] “There are two situations for the reaction of sodium hydroxide withcarbon dioxide: First: a small amount of carbon dioxide reacts with sodiumhydroxide to generate sodium carbonate and water; The reaction of sodiumhydroxide and carbon dioxide to generate sodium carbonate is carried outfirst; if the carbon dioxide is excessive, the reaction of carbon dioxide andsodium carbonate is carried out. complete removal of carbondioxide.When the sodium hydroxide is completely consumed,carbon dioxide reacts with sodium carbonate to form sodium bicarbonate.That is to say,whensodium hydroxide is still present,sodium bicarbonate cannot be formed.”

[0101] For the above spoken English speech sequence, the existing real-time speech translation technology will directly translate it into:

[0102] "There are two situations in the reaction of sodium hydroxide and carbon dioxide:

[0103] First: a small amount of carbon dioxide reacts with sodium hydroxide to produce sodium carbonate and water;

[0104] First, the sodium hydroxide and carbon dioxide react to form sodium carbonate; if the carbon dioxide is excessive, the carbon dioxide and sodium carbonate react again.

[0105] Second: Excess carbon dioxide reacts with sodium hydroxide to produce sodium bicarbonate;

[0106] To remove carbon dioxide, an excess of sodium hydroxide must be used to ensure that the carbon dioxide is completely removed. When the sodium hydroxide is completely consumed, the carbon dioxide reacts with the sodium carbonate to form sodium bicarbonate. In other words, sodium bicarbonate cannot be generated when sodium hydroxide is still present. "

[0107] As a simple data comparison, here, the real-time voice translation data volume is about (800) characters, the translated result is about (200) characters, and the translation result is a text description in plain text format, so that the communicating parties cannot see where the key point of the passage is.

[0108] When the data volume of the original speech sequence is large, the translation time is long and the translation cost is high (many translation software are charged by data volume or translation time).

[0109] For this purpose, see Figure 3 . Figure 3 In the embodiment, it is assumed that the target speech string is a preset target noun speech string, and the target noun includes chemical substances, that is, the preset target noun speech string is identified in units of chemical substances;

[0110] At this time, when the speech sequence input is detected, the target speech sequence recognition process is started;

[0111] For the above-mentioned English oral input speech sequence, the timing when the target speech sequence recognition process recognizes that the input speech sequence contains a target speech string is:

[0112] Identify the presence of chemical substances (sodium hydroxide, carbon dioxide) in “There are two situations for the reaction of sodium hydroxide with carbon dioxide:”

[0113] Then, continue to identify the first pause point (i.e., colon) of the input speech sequence after the target speech string;

[0114] At this time, the speech sequence before the first pause point (There are two situations for the action of sodium hydroxide with carbon dioxide) is converted into a text sequence (There are two situations for the reaction of sodium hydroxide with carbon dioxide);

[0115] Then, a summary sequence (the reaction of sodium hydroxide with carbon dioxide, two situations) is generated based on the text sequence (There are two situations for the reaction of sodium hydroxide with carbon dioxide).

[0116] It can be understood that here, since the text sequence (There are two situations for the reaction of sodium hydroxide with carbon dioxide) itself is short, the actually generated summary sequence (thereaction of sodium hydroxide with carbon dioxide, two situations) is similar to the text sequence itself. However, if the text sequence is longer (as will be seen later), the actually generated summary sequence will be more concise.

[0117] Then, the actually generated summary sequence (the reaction of sodium hydroxide with carbon dioxide, two situations) is translated, that is, the source text summary sequence in the first language is translated into an output text sequence in the second language (the reaction of sodium hydroxide with carbon dioxide has two situations), wherein the output text sequence includes a target output text (sodium hydroxide and carbon dioxide) in the second language corresponding to the target text sequence (sodium hydroxide with carbon dioxide).

[0118] Finally, the target output text in the second language corresponding to the target text sequence is displayed in a predetermined format, that is, Figure 3 “NaOH+CO2=?” in the text.

[0119] That is, the target output text in the second language corresponding to the target text sequence is displayed in a predetermined format, specifically including: displaying at least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances in the form of a picture.

[0120] Obviously, Figure 3 In this way, the integrity and readability of key content can be maintained while ensuring the real-time performance of voice translation.

[0121] Figure 3 The example is relatively simple and is not enough to reflect the advantage of the present application in reducing the amount of data in real time during large-scale long-term voice input.

[0122] Therefore, continue to see Figure 4 and Figure 5 .

[0123] exist Figure 4 and Figure 5 In the embodiment, it is assumed that the target speech string is a preset target noun speech string, and the target noun includes chemical substances and chemical reaction formulas, that is, the preset target noun speech string is identified in units of chemical substances and chemical reaction formulas, that is, processing is started only when chemical combination reaction formulas of different compounds actually appear in the spoken speech sequence.

[0124] At this time, when the speech sequence input is detected, the target speech sequence recognition process is started;

[0125] For the above-mentioned English oral input speech sequence, the timing when the target speech sequence recognition process recognizes that the input speech sequence contains a target speech string is:

[0126] Identify the existence of chemical substances (sodium hydroxide, carbon dioxide) in "There are two situations for the reaction of sodium hydroxide with carbon dioxide:" and also identify the existence of a chemical reaction formula for the actual combination of chemical substances (a small amount of carbon dioxide reacts with sodium hydroxide to generates sodium carbonate and water) in "First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water"

[0127] Then, continue to identify the first pause point (i.e., semicolon) of the input speech sequence after the target speech string;

[0128] At this time, the speech sequence before the first pause point (There are two situations for the action of sodium hydroxide with carbon dioxide: First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water;) is converted into a text sequence (There are two situations for the reaction of sodium hydroxide with carbon dioxide: First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water;);

[0129] Then, a summary sequence (a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water) is generated based on the text sequence (There are two situations for the reaction ofsodium hydroxide with carbon dioxide: First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water).

[0130] Then, the actually generated summary sequence (a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water) is translated, that is, the source text summary sequence in the first language is translated into an output text sequence (a small amount of carbon dioxide reacts with sodium hydroxide to generate water and sodium carbonate) in the second language, and the output text sequence includes a target output text (a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water) in the second language corresponding to the target text sequence (a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water).

[0131] Finally, the target output text in the second language corresponding to the target text sequence is displayed in a predetermined format, that is, Figure 4 “2NaOH+CO2=Na2CO3+H2O”.

[0132] Simple data comparison: At this time, the actual amount of data translated is (a small amount of carbondioxide reacts with sodium hydroxide to generate sodium carbonate and water), about 100 characters, which is only 1 / 4 of the existing technology.

[0133] The advantages of reducing data volume can be further seen in Figure 5 .

[0134] Figure 5 In the method, the method further comprises:

[0135] When the target speech sequence recognition process recognizes that the input speech sequence has a target speech string, continue to recognize whether the input speech sequence still has the same target speech string, and if so, continue to recognize until the same target speech string cannot be recognized;

[0136] At this time, it is equivalent to merging all original sequences containing the same target speech string.

[0137] Then, identify the pause point.

[0138] Specifically, Figure 5 In the process, all the original English speech sequences are taken as "speech sequences before the first pause point", and then "speech sequences before the first pause point are converted into text sequences".

[0139] It should be noted that although all processing is done here, it does not involve translation. It is only a speech-to-text process, and the system processing burden is relatively low.

[0140] Then, a summary sequence is generated based on the text sequence (The reaction of sodium hydroxide with carbon dioxide: First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water; Second: excess carbon dioxide reacts with sodium hydroxide to generate sodium bicarbonate)

[0141] Translating and displaying the summary sequence;

[0142] The translation result is "There are two situations in which sodium hydroxide reacts with carbon dioxide: 1. A small amount of carbon dioxide reacts with sodium hydroxide to produce sodium carbonate and water; 2. Excess carbon dioxide reacts with sodium hydroxide to produce sodium bicarbonate."

[0143] Simple data comparison: At this time, the amount of data actually translated is (The reaction of sodium hydroxide with carbon dioxide: First: a small amount of carbon dioxide reacts with sodium hydroxide to generate sodium carbonate and water; Second: excess carbon dioxide reacts with sodium hydroxide to generate sodium bicarbonate), about 250 characters, which is only 250 / 800 of the existing technology.

[0144] The results are shown as Figure 5 The two reaction formulas shown in the picture, namely the chemical formula corresponding to the chemical substance and the chemical reaction formula corresponding to the combination of chemical substances, are shown in the picture.

[0145] To implement the above method, see Figure 6-Figure 7 .

[0146] Figure 6In the figure, a real-time speech translation system is shown, the system includes a speech input module, a speech string recognition module, a speech-to-text module, a summary generation module, a text translation module and a text-image display module.

[0147] The speech input module is used to input a speech sequence, and the input speech sequence is a speech sequence of a first source language;

[0148] The speech string recognition module is used to recognize whether the speech sequence of the first source language contains a target speech string of the first source language;

[0149] The speech-to-text module is used to convert the speech sequence in the first source language into a source text sequence in the first source language;

[0150] The summary generation module is used to generate a summary text in the first source language based on the source text sequence in the first source language;

[0151] The text translation module is used to translate the summary text in the first source language into a target text in a second target language;

[0152] The text-image display module is used to display the target text in the second target language in a predetermined text-image format;

[0153] Wherein, the target speech string is a preset target noun speech string, and the target noun includes chemical substances and / or chemical reaction formulas;

[0154] The displaying of the target text in the second target language in a predetermined graphic format specifically includes: displaying at least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances in the form of a picture.

[0155] Figure 7 In the embodiment, the system further comprises a voice broadcast module and a pause point detection module, and the voice string recognition module is connected to the pause point monitoring module.

[0156] The speech broadcast module is used to voice broadcast the target text of the second target language; the pause point detection module is used to detect the pause points in the speech sequence of the first source language;

[0157] When the pause point detection module detects a pause point in the speech sequence of the first source language, the speech string recognition module is activated.

[0158] It should be pointed out that Figure 1-Figure 5 The steps or methods and processes can be automatically implemented by computer program instructions. Figure 8, provides a terminal device, which can be a data interaction device, including a bus, a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium.

[0159] Specifically, the terminal device may be a visual terminal, the visual terminal comprising a human-machine display interface, the human-machine display interface connecting a memory and a processor, the processor storing computer program instructions, and the processor executing the computer program instructions to implement Figure 1-Figure 5 The steps or the method or process.

[0160] It can be seen that the present invention generates a summary sequence based on the target voice string detected from the original voice sequence before performing translation display, which reduces the data volume of the voice translation and can maintain the integrity and readability of the key content while ensuring the real-time performance of the voice translation.

[0161] Specifically, unlike the prior art approach of translating the original text word by word and sentence by sentence, the present invention generates a source text summary sequence in the first language based on the source text sequence in the first language before performing the formal translation, and then only performs translation on the source text summary sequence in the first language, thereby reducing the data volume of speech translation and ensuring the real-time performance of speech translation.

[0162] At the same time, when generating a summary, the embodiment of the present invention does not generate the summary based on the entire text sequence, but instead separates the target text sequence, generates a to-be-combined summary sequence based on the non-target text sequence, and then semantically combines the to-be-combined summary sequence with the target text sequence to obtain the summary sequence, thereby ensuring that the summary sequence completely retains the target text sequence.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0164] The present invention does not particularly specify the structure of some modules, which shall be subject to the contents recorded in the prior art. The prior art mentioned in the above background technology section of the present invention can be used as part of the present invention to understand the meaning of some technical features or parameters. The scope of protection of the present invention shall be subject to the contents actually recorded in the claims.

Claims

1. A real-time speech translation method, characterized in that: The method comprises the following steps: S1: When a speech sequence input is detected, the target speech sequence recognition process is started; S2: when the target speech sequence recognition process recognizes that the input speech sequence has a target speech string, continue to recognize the first pause point of the input speech sequence after the target speech string; the target speech string is a preset target noun speech string, and the target noun includes a chemical substance and / or a chemical reaction formula; S3: converting the speech sequence before the first pause point into a text sequence; S4: generating a summary sequence based on the text sequence; S5: translating and displaying the summary sequence; The step S4 generates a summary sequence based on the text sequence, specifically comprising: The text sequence includes a target text sequence and a non-target text sequence; The target text sequence is a text sequence corresponding to the target voice string; The non-target text sequence is other text sequences in the text sequence converted in step S3 except the target text sequence; Generate a summary sequence to be combined based on the non-target text sequence; semantically combining the summary sequence to be combined with the target text sequence to obtain the summary sequence; The step S5 translates the summary sequence and displays it, specifically comprising: Translating a source text summary sequence in a first language into an output text sequence in a second language, wherein the output text sequence includes a target output text in the second language corresponding to the target text sequence; and displaying the target output text in the second language corresponding to the target text sequence in a predetermined format; Displaying the target output text in the second language corresponding to the target text sequence in a predetermined format, specifically including: At least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances are displayed in a picture.

2. A method for real-time speech translation as claimed in claim 1, characterized in that: The input speech sequence is a source speech sequence in a first language; The step S3 converts the speech sequence before the first pause point into a text sequence, specifically comprising: The source speech sequence in the first language before the first pause point is converted into a source text sequence in the first language.

3. A real-time speech translation system for implementing the real-time speech translation method according to claim 1 or 2, the system comprising a speech input module, a speech string recognition module, a speech-to-text module, a summary generation module, a text translation module and a text-image display module, characterized in that: The speech input module is used to input a speech sequence, and the input speech sequence is a speech sequence of a first source language; The speech string recognition module is used to recognize whether the speech sequence of the first source language contains a target speech string of the first source language; The speech-to-text module is used to convert the speech sequence in the first source language into a source text sequence in the first source language; The summary generation module is used to generate a summary text in the first source language based on the source text sequence in the first source language; The text translation module is used to translate the summary text in the first source language into a target text in a second target language; The text-image display module is used to display the target text in the second target language in a predetermined text-image format; Wherein, the target speech string is a preset target noun speech string, and the target noun includes chemical substances and / or chemical reaction formulas; The displaying of the target text in the second target language in a predetermined graphic format specifically includes: displaying at least one chemical formula corresponding to at least one chemical substance and / or a chemical reaction formula corresponding to a combination of chemical substances in the form of a picture.

4. A real-time speech translation system as claimed in claim 3, characterized in that: The system also includes a voice broadcast module; The voice broadcast module is used for voice broadcasting the target text in the second target language.

5. A real-time speech translation system as claimed in claim 3, characterized in that: The system also includes a stall point detection module; The pause point detection module is used to detect pause points in a speech sequence of a first source language.

6. A real-time speech translation system as claimed in claim 5, characterized in that: The voice string recognition module is connected to the pause point monitoring module; When the pause point detection module detects a pause point in the speech sequence of the first source language, the speech string recognition module is activated.

7. A visualization terminal, comprising a human-machine display interface, wherein the human-machine display interface is connected to a memory and a processor, wherein the processor stores computer program instructions, and the computer program instructions are executed by the processor to implement a real-time speech translation method as described in claim 1 or 2.