Speech translation method, electronic device, storage medium and computer program product

Abstract

The invention discloses a speech translation method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of signal processing, and the method comprises the steps: carrying out the real-time segmentation of a speech feature sequence obtained in real time according to a semantic boundary in the speech feature sequence, and obtaining a current semantic block; translating each grammar unit in the current semantic block in parallel to obtain semantic representation of the current semantic block; a preliminary translation result of the current semantic block is generated in combination with preceding text information of the current semantic block and the semantic representation, and the preceding text information is determined by historical semantic blocks segmented before the current semantic block; the part, with semantics changed, in the preliminary translation result is corrected in combination with following information of the current semantic block, a target translation result is obtained, and the following information is determined by future semantic blocks segmented after the current semantic block. The technical problem that a streaming translation scheme based on autoregression is insufficient in real-time performance is solved.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, and in particular to a speech translation method, electronic device, storage medium, and computer program product. Background Technology

[0002] With increasingly frequent global exchanges, real-time speech translation technology has become a key tool for breaking down language barriers, especially in full-duplex dialogue scenarios that require natural and fluent interaction. Users expect to receive synchronous translation while speaking to maintain the continuity of the conversation. Current mainstream solutions are mainly based on autoregressive models for translation. These models generate translation results word by word. While this ensures translation quality, it has an inherent latency problem. Translation can only begin after the preceding output is completed or a complete sentence is input. Therefore, current autoregressive streaming translation solutions suffer from insufficient real-time performance.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a speech translation method, electronic device, storage medium, and computer program product, which aims to solve the technical problem of insufficient real-time performance of current autoregressive streaming translation schemes.

[0005] To achieve the above objectives, this application proposes a speech translation method, which includes: The real-time acquired speech feature sequence is segmented in real time according to the semantic boundaries in the speech feature sequence to obtain the current semantic block; Parallel translation of each syntactic unit in the current semantic block yields the semantic representation of the current semantic block; A preliminary translation result of the current semantic block is generated by combining the contextual information of the current semantic block and the semantic representation, wherein the contextual information is determined by the historical semantic blocks segmented before the current semantic block; By combining the context information of the current semantic block, the semantically changed parts in the preliminary translation result are corrected to obtain the target translation result, wherein the context information is determined by the future semantic blocks segmented after the current semantic block.

[0006] Optionally, the semantic boundary includes natural semantic pause points, and the step of segmenting the real-time acquired speech feature sequence according to the semantic boundary in the speech feature sequence to obtain the current semantic block includes: Prosodic feature analysis is performed on the real-time acquired speech feature sequence, wherein the prosodic features include at least one of fundamental frequency, speech rate, and pause duration; Based on the prosodic features, the natural semantic pause points are extracted from the speech feature sequence; If the natural semantic pause point is extracted, the current semantic block is divided from the speech feature sequence based on the natural semantic pause point.

[0007] Optionally, the semantic boundary includes text semantic demarcation points, and the step of segmenting the real-time acquired speech feature sequence according to the semantic boundaries in the speech feature sequence to obtain the current semantic block further includes: Convert the real-time acquired speech feature sequence into a text sequence; Detect the part-of-speech information of each word in the text sequence, and detect the preset sentence structure satisfied by each word based on the part-of-speech information; The fields in the text sequence that satisfy the preset sentence structure are identified, and the start and end points of the fields are used as the text semantic demarcation points. The current semantic block is divided from the speech feature sequence based on the text semantic demarcation points.

[0008] Optionally, the step of generating a preliminary translation result of the current semantic block by combining the contextual information of the current semantic block and the semantic representation further includes: The context information of the current semantic block is extracted based on a preset context cache pool, wherein the context information includes the translation results of historical semantic blocks, and the historical semantic blocks are semantic blocks segmented before the current semantic block; The aforementioned information is incorporated into the encoding and decoding process of the semantic representation to obtain a preliminary translation result.

[0009] Optionally, the step of correcting the semantically changed parts of the preliminary translation result by combining the contextual information of the current semantic block to obtain the target translation result includes: Obtain the context information of the current semantic block, wherein the context information is the text meaning of the future semantic block, and the future semantic block is a semantic block segmented after the current semantic block; Determine whether the following information changes the contextual meaning of the preliminary translation result; If the contextual information changes the meaning of the preliminary translation result, the semantically altered parts of the preliminary translation result are corrected based on the contextual information and the preliminary translation result to obtain the target translation result; Without altering the contextual meaning of the preliminary translation result, the preliminary translation result shall be taken as the target translation result.

[0010] Optionally, the step of determining whether the following information changes the contextual meaning of the preliminary translation result includes: Determine whether there is semantically adjusted content in the following information, wherein the semantically adjusted content includes at least one of negation words and transition words; If the semantically adjusted content exists in the following context, it is determined that the following context changes the contextual meaning of the preliminary translation result; If the semantic adjustment content is not present in the following context, it is determined that the following context has not changed the contextual meaning of the preliminary translation result.

[0011] Optionally, before the step of real-time segmentation of the acquired speech feature sequence according to the semantic boundaries in the speech feature sequence, the method further includes: Real-time monitoring of voice signals during voice conversations; The speech signal is preprocessed to obtain a speech feature sequence, wherein the preprocessing includes at least one of noise reduction, speech activity detection, and feature extraction.

[0012] Furthermore, to achieve the above objectives, this application also proposes a voice translation device, which includes: The semantic segmentation module is used to segment the real-time acquired speech feature sequence according to the semantic boundaries in the speech feature sequence to obtain the current semantic block; The first translation module is used to translate each syntactic unit in the current semantic block in parallel to obtain the semantic representation of the current semantic block; The second translation module is used to generate a preliminary translation result of the current semantic block by combining the contextual information of the current semantic block and the semantic representation, wherein the contextual information is determined by the historical semantic blocks segmented before the current semantic block; The third translation module is used to correct the semantically changed parts in the preliminary translation result by combining the context information of the current semantic block, so as to obtain the target translation result. The context information is determined by the future semantic block segmented after the current semantic block.

[0013] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the speech translation method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the speech translation method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the speech translation method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: By performing real-time semantic block segmentation based on semantic boundaries on the real-time acquired speech feature sequences, the problem of semantic incompleteness caused by the lack of clear sentence boundaries in streaming input is solved, ensuring the integrity of the meaning of each semantic block to be translated and avoiding the semantic incompleteness caused by simple segmentation according to a fixed time window. By translating the grammatical units within the current semantic block in parallel, the latency caused by word-by-word translation in autoregressive models is overcome, achieving low latency characteristics in the translation process. By combining the preceding context information to generate preliminary translation results, the context of historical semantic blocks is effectively utilized, alleviating the problem of poor semantic coherence caused by the lack of historical dependence in non-autoregressive models. At the same time, a correction mechanism based on the following context information is introduced to perform semantic calibration of the preliminary translation results through future semantic blocks, solving the defect of insufficient translation accuracy of non-autoregressive models when there is a lack of complete context. Compared to existing methods that translate word by word based on autoregressive models, this application translates each grammatical unit in the current semantic block in parallel, retaining the low latency advantage of parallel processing. At the same time, through semantic boundary segmentation, context combination, and result correction, it solves the problems of semantic incoherence and poor accuracy caused by the inability of parallel translation to consider contextual information. It can output high-quality and highly coherent translation results with low latency, achieving the best balance between latency and quality in full-duplex dialogue scenarios. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the speech translation method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the speech translation method of this application; Figure 3 This is a flowchart illustrating the third embodiment of the speech translation method of this application; Figure 4This is a schematic diagram of the module structure of the speech translation device of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the speech translation method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] Current mainstream solutions are mainly based on autoregressive models for translation. These models generate translation results word by word, which can guarantee translation quality, but they have an inherent delay problem. Translation can only start after the preceding output is completed or the complete sentence is input.

[0024] This application provides a solution that addresses the semantic incompleteness caused by the lack of clear sentence boundaries in streaming input by performing real-time semantic block segmentation based on semantic boundaries on the real-time acquired speech feature sequence. This ensures the integrity of the meaning of each semantic block to be translated and avoids the semantic incompleteness caused by simple segmentation according to a fixed time window. By translating the grammatical units within the current semantic block in parallel, it overcomes the latency caused by word-by-word translation in autoregressive models and achieves low latency in the translation process. By combining the preceding context information to generate preliminary translation results, it effectively utilizes the context of historical semantic blocks, alleviating the problem of poor semantic coherence caused by the lack of historical dependence in non-autoregressive models. At the same time, it introduces a correction mechanism based on the following context information to perform semantic calibration on the preliminary translation results through future semantic blocks, solving the defect of insufficient translation accuracy of non-autoregressive models when there is a lack of complete context. Compared to existing methods that translate word by word based on autoregressive models, this application translates each grammatical unit in the current semantic block in parallel, retaining the low latency advantage of parallel processing. At the same time, through semantic boundary segmentation, context combination, and result correction, it solves the problems of semantic incoherence and poor accuracy caused by the inability of parallel translation to consider contextual information. It can output high-quality and highly coherent translation results with low latency, achieving the best balance between latency and quality in full-duplex dialogue scenarios.

[0025] It should be noted that the execution subject of each embodiment of the speech translation method of this application can be an electronic device with data processing, network communication and program running functions, such as a tablet computer, personal computer, mobile phone, etc., and the electronic device has a built-in or external microphone array.

[0026] Reference Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the speech translation method of this application.

[0027] In this embodiment, the speech translation method includes steps S10 to S40: Step S10: The real-time acquired speech feature sequence is segmented in real time according to the semantic boundaries in the speech feature sequence to obtain the current semantic block; A speech feature sequence is an ordered set of feature vectors obtained by extracting features from the speech signals of two parties during a speech dialogue. Feature extraction of speech signals can be achieved through feature extraction algorithms, such as Mel-frequency cepstral coefficients (MFCC) and linear predictive coding (LPC), which can extract speech feature parameters from the speech signal and arrange these speech feature parameters in chronological order to form a speech feature sequence.

[0028] Semantic boundaries refer to the boundaries that distinguish different semantic units in a speech feature sequence, representing transitions, pauses, or logical divisions in semantic content. A semantic unit is the smallest unit in a speech feature sequence that carries independent semantic meaning; it can be a word, a phrase, or a sentence component. The determination of semantic boundaries is based on the semantic content expressed by the speech feature sequence, rather than simply on physical speech features (such as pitch and intensity). Since the voice dialogue between communicating parties is continuous, there are no obvious physical markers separating different semantic units in actual speech streams; therefore, semantic analysis is needed to identify these boundaries.

[0029] Optionally, semantic boundaries can include natural semantic pauses and textual semantic demarcation points. Natural semantic pauses refer to the natural pauses in a speech feature sequence caused by speech features (such as energy decay, fundamental frequency drop, silence duration, sentence ending, etc.), reflecting the pauses in the speaker's semantic expression. Textual semantic demarcation points refer to the semantic text boundaries defined in a speech feature sequence through linguistic features (such as punctuation, conjunctions, lexical cohesion) or semantic content changes (such as topic shifts, logical transitions), reflecting semantic shifts in logical structure.

[0030] The current semantic block is the smallest complete semantic unit segmented from the real-time acquired speech feature sequence based on semantic boundaries. It contains the complete semantic information expressed by the part of speech currently being processed, and can be a phrase, clause, or simple sentence. Because it is segmented in real time, the current semantic block is continuously updated with the continuous input of speech, always representing the semantic content being processed at the current moment.

[0031] For example, semantic boundaries can be extracted from speech feature sequences using a multimodal boundary detection model. The multimodal boundary detection model is a pre-trained large model used for semantic block segmentation. Through processing by the multimodal boundary detection model, the semantic boundaries in the speech feature sequence can be clearly defined, and the speech feature sequence can be divided into semantic blocks based on these semantic boundaries. Each identified semantic block is then extracted from the speech feature sequence.

[0032] Understandably, by identifying semantic boundaries in the speech feature sequence and dividing it into independent semantic blocks, the subsequent translation process can be carried out based on the semantic blocks as a whole, without the need for word-by-word translation. Furthermore, since the semantic block division is performed in real time, one block is divided for each semantic block identified, and semantic division can be performed without receiving the complete sentence, which significantly improves real-time performance. At the same time, the logical order of the original speech is preserved through temporal processing, ensuring the semantic coherence of the generated results.

[0033] Understandably, by acquiring speech feature sequences in real time and identifying semantic boundaries within these sequences, and dividing them into independent semantic blocks, the subsequent translation process can be based on these semantic blocks as a whole, eliminating the need for word-by-word translation. Furthermore, since the semantic block division is performed in real time, each semantic block is divided as soon as it is identified, semantic segmentation can be carried out without receiving the complete sentence, significantly improving real-time performance. At the same time, the logical order of the original speech is preserved through temporal processing, ensuring the semantic coherence of the generated results.

[0034] In one possible implementation, the step S10 is preceded by: Step S101: Monitor the voice signal during the voice dialogue in real time; Voice dialogue is a dynamic process in which two parties communicate by transmitting information and completing communication through voice signals, such as human-computer dialogue. Voice dialogue can refer to full-duplex dialogue. Because full-duplex dialogue allows both parties to engage in two-way voice interaction simultaneously, it is necessary to monitor the voice signals during the dialogue in real time to ensure the continuity of two-way voice communication. This allows for real-time processing of the monitored voice signals, reducing the latency of subsequent translation.

[0035] Speech signals are the raw signals generated during speech dialogues. They can be in the form of analog waveforms or digital samples, and include information such as speech content, background noise, and environmental reverberation. They can be acquired from speech dialogues using audio acquisition hardware such as microphone arrays and audio acquisition cards. These audio acquisition hardware devices support multi-channel speech input and can adapt to different dialogue scenarios.

[0036] Step S102: Preprocess the speech signal to obtain a speech feature sequence, wherein the preprocessing includes at least one of noise reduction, speech activity detection and feature extraction.

[0037] Preprocessing is the initial processing of speech signals to eliminate interference introduced during acquisition (such as noise, distortion, DC offset, etc.) and to unify the signal format.

[0038] Speech feature sequences are numerical sequences that represent speech characteristics after preprocessing speech signals, such as time-domain features, frequency-domain features, and cepstral features. By extracting speech feature sequences, the data dimension of the original speech signal is compressed while retaining key acoustic information, resulting in a more concise data processing format, thereby improving the efficiency of subsequent data processing.

[0039] Optionally, preprocessing may include at least one of noise reduction, voice activity detection (VAD), and feature extraction. Noise reduction refers to the process of separating clean speech from noisy speech and removing environmental noise (such as fan noise, keyboard noise, traffic noise, etc.). Voice activity detection is the process of determining whether there are valid speech segments in a speech signal, used to distinguish between speech and silence segments. By removing silence segments from the speech signal, valid speech segments containing only speech content can be obtained. Feature extraction refers to the process of extracting key features from the speech signal. Key features can be time-domain features, frequency-domain features, and cepstral features, etc., used to convert the speech signal into a more compact and robust representation. For example, the speech signal can be converted into Mel-Frequency Cepstral Coefficients (MFCC) or a log-Mel spectrum. MFCC converts the speech signal into a low-dimensional feature representation by simulating the nonlinear perception of frequency by the human ear. The log-Mel spectrum combines the nonlinear frequency perception of the Mel filter bank and the dynamic range adjustment of logarithmic compression to convert the speech or audio signal into a time-frequency representation feature map.

[0040] It should be noted that the speech feature sequence is acquired in real time during the speech dialogue. Since the speech dialogue is conducted in real time, the speech signal acquisition is also in real time. However, real-time acquisition is actually a multi-round high-frequency signal acquisition process. When one round of speech signal acquisition is completed, the speaker may not have finished speaking a sentence during the speech dialogue. Therefore, the speech feature sequence obtained from the speech signal does not correspond to a complete sentence. In other words, when dividing the speech feature sequence into semantic blocks, it may not be possible to divide it into a complete semantic block. It is necessary to combine the content of subsequent rounds of speech signal acquisition to determine the semantic block.

[0041] Understandably, by monitoring the voice signals during the voice dialogue in real time, the system ensures that it can start recording the signal the instant the voice dialogue begins, so that the capture of the voice signal is synchronized with the dialogue, eliminating the problem of insufficient real-time performance caused by signal capture delay; at the same time, by preprocessing the voice signal into a more concise voice feature sequence, the data processing efficiency is improved, thereby further enhancing the real-time performance of the translation.

[0042] Step S20: Translate each syntactic unit in the current semantic block in parallel to obtain the semantic representation of the current semantic block; A grammatical unit, or token, refers to the smallest semantic or grammatical unit within the current semantic block. A token can be a word, subword, punctuation mark, number, etc. It can be extracted from the current semantic block according to predefined grammatical rules (such as word-by-word segmentation). Different languages ​​may use different predefined grammatical rules for segmentation, but all adhere to the grammatical system of that language.

[0043] Semantic representation is an abstract representation of the semantics expressed by the current semantic block. It is a representation form that computers can understand and process, such as data structures like vectors and matrices. It contains the key semantic information of the current semantic block and removes redundant information at the phonetic and syntactic levels.

[0044] Optionally, the syntactic units in the current semantic block can be translated in parallel using a non-autoregressive (NAT) model. The NAT model is a pre-trained model for translation processing, which adopts a Transformer-based encoder-decoder architecture. In this architecture, the encoder encodes the current semantic block, and the decoder translates the syntactic units (i.e., tokens) of the current semantic block in parallel through a parallel attention mechanism. The output of the NAT model is the semantic representation of the current semantic block.

[0045] Alternatively, the NAT model can be pre-trained on a large-scale bilingual corpus (including pre-collected streaming speech-text alignment data) to ensure the model's ability to understand fragmented semantics.

[0046] Understandably, by translating each syntactic unit in the current semantic block in parallel, the sequential dependency that requires generation per token in autoregressive streaming translation schemes is broken. Each syntactic unit is processed simultaneously without waiting for previous results, which greatly shortens the overall processing time. Originally, the time taken to process each unit serially was the sum of the processing times of each unit. After parallel processing, the time taken depends on the processing time of the longest unit, which significantly improves the translation speed and effectively solves the problem of insufficient real-time performance caused by sequential generation in autoregressive schemes.

[0047] Step S30: Combine the contextual information of the current semantic block with the semantic representation to generate a preliminary translation result of the current semantic block, wherein the contextual information is determined by the historical semantic blocks segmented before the current semantic block; The preceding context information represents the translation results of historical semantic blocks (semantic blocks that have already undergone translation processing) segmented before the current semantic block. This can include output hidden states or semantic unit representations. The output hidden state is the state of the historical semantic block after decoding by the NAT model's decoder, while the semantic unit representation is the semantic representation of all syntactic units generated after the historical semantic block is processed by the NAT model. The preceding context information represents the translation information corresponding to the historical semantic blocks. By incorporating this information when translating the current semantic block, a preliminary translation result with semantic coherence can be obtained.

[0048] Understandably, by extracting contextual information from historical semantic blocks as translation context when processing the current semantic block, the initial translation result is ensured to maintain semantic coherence with the preceding content. This allows each grammatical unit to be translated in parallel while simultaneously referencing the preceding text in parallel processing mode, avoiding translation deviations caused by a lack of context. This effectively improves the real-time performance of the overall translation process while ensuring translation quality.

[0049] Step S40: Combine the context information of the current semantic block to correct the parts of the preliminary translation result that have changed semantically, and obtain the target translation result. The context information is determined by the future semantic block segmented after the current semantic block.

[0050] The following information refers to the semantic block information that will be translated further after obtaining the preliminary translation result. It can be the textual meaning of the future semantic block (i.e., the semantic block segmented after the current semantic block). The following information represents the information represented by the future voice dialogue content. Since the voice dialogue is conducted in real time, voice signals are continuously received, thereby updating the voice feature sequence. If the updated voice feature sequence changes the meaning of the currently translated content (i.e., the preliminary translation result), in order to ensure the accuracy of the translation, it is necessary to modify the currently translated content in conjunction with the following information, thereby obtaining the target translation result that integrates the context information of the current semantic block. The target translation result is the target language representation (e.g., English, Japanese, etc.) of the voice content of the communicating parties during the voice dialogue.

[0051] Optionally, the final target translation result can be converted into target language speech using Text-to-Speech (TTS) technology and simultaneously output to the user interface (such as a display screen, speaker, etc.), achieving real-time synchronization of "source speech input - target language output". TTS is a technology that converts written text into natural and fluent speech output through computers or dedicated equipment. Its core lies in achieving seamless conversion between text and speech through acoustic models and vocoders, enabling machines to imitate human speech pronunciation.

[0052] Understandably, by introducing a correction mechanism based on contextual information, the semantic incompleteness problem caused by the lack of future context in parallel translation is solved. After generating the initial translation results, the future semantic blocks are used for context verification and dynamic correction. While maintaining the low latency advantage of parallel computing, the translation accuracy is improved to a level close to that of an autoregressive model through a one-time semantic integrity compensation, ultimately achieving the optimal balance between latency and quality.

[0053] In this embodiment, by acquiring the speech feature sequence during the voice dialogue in real time, the system can continuously receive and process the user's voice input, achieving low-latency capture of the speech signal. By dividing the speech feature sequence into semantic blocks according to the semantic boundaries in the speech feature sequence, the integrity of the meaning of each semantic block to be translated is ensured, avoiding the semantic incompleteness caused by simply dividing it according to a fixed time window. In the translation stage, the parallel processing of each grammatical unit is used to overcome the serialization generation bottleneck of the autoregressive model, significantly reducing the computational latency. This allows the translation process to proceed without waiting for the entire sentence to end, greatly reducing the output latency and meeting the high real-time requirements of full-duplex dialogue. When generating the preliminary result, the system integrates the preceding context information to ensure coherence, so that the translation result of each block is not completely isolated, ensuring logical connection with the preceding dialogue and improving the accuracy and naturalness of the translation. Furthermore, by utilizing the following context information, ambiguities, errors, or inappropriate parts in the preliminary translation result can be corrected, so that the final output result achieves a quality level close to that of a complete sentence translation while maintaining low latency. Compared to methods that rely solely on parallel translation, this application addresses the semantic incoherence and poor accuracy caused by parallel translation's inability to consider contextual information through context-aware and result correction mechanisms. This enables the output of high-quality, highly coherent translation results with low latency.

[0054] Based on the first embodiment described above, a second embodiment of the speech translation method of this application is proposed. In this embodiment, content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. (Refer to...) Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the audio recording method of this application. In this embodiment, semantic boundaries include natural semantic pause points, and step S10 includes: Step S201: Perform prosodic feature analysis on the real-time acquired speech feature sequence, wherein the prosodic features include at least one of fundamental frequency, speech rate, and pause duration. Prosodic features refer to suprasegmental features in speech sequences that go beyond a single phoneme or word and reflect rhythm, stress, and intonation.

[0055] Optionally, prosodic features may include at least one of fundamental frequency, speech rate, and pause duration. Fundamental frequency refers to the lowest frequency produced by vocal cord vibration and is used to determine pitch features in speech sequence features; speech rate refers to the number of syllables or words output per unit time (e.g., 1 second) and can be measured in syllables / second or words / minute; pause duration is the duration of silent intervals in speech, such as pauses between sentences, between phrases, or caused by the speaker's breathing or thinking.

[0056] Step S202: Extract natural semantic pause points from the speech feature sequence based on prosodic features; Step S203: If natural semantic pause points are extracted, the current semantic block is divided from the speech feature sequence based on the natural semantic pause points.

[0057] For example, natural semantic pauses can be extracted from a speech feature sequence by using a prosodic feature analysis unit (PSU). The PSU is a program module used to analyze the fundamental frequency, speech rate, and pause duration in speech sequence features. By parsing the input speech feature sequence through the PSU, natural semantic pauses reflecting the speaker's semantic expression intervals can be extracted. Based on these natural semantic pauses, the speech sequence features can be divided into semantic blocks. For example, in the sentence "The weather is nice today∥we plan to go to the park," "∥" represents the speaker's pause point, i.e., the natural semantic pause point. Therefore, "∥" can be used as the dividing criterion to divide the long sentence "The weather is nice today∥we plan to go to the park" into two clauses (semantic blocks): "The weather is nice today" and "We plan to go to the park."

[0058] Understandably, by locating natural semantic pauses based on acoustic parameters such as fundamental frequency, speech rate, and pause duration, and clarifying semantic boundaries, speech streams can be divided into complete semantic units. This overcomes the mechanical limitations of fixed time window segmentation, avoids semantic ambiguity and loss of coherence caused by improper segmentation, provides structurally complete input units for subsequent parallel translation, and improves translation accuracy.

[0059] In one feasible implementation, the semantic boundary also includes text semantic demarcation points, and step S10 further includes: Step S211: Convert the real-time acquired speech feature sequence into a text sequence; A text sequence is a sequence of natural language symbols (such as Chinese characters, words, and punctuation marks) arranged in a linear order. It can be obtained in real time by converting speech sequence features through an Automatic Speech Recognition (ASR) model. The ASR model can map continuous speech sequence features into discrete text sequences, which are in a computer-readable text format.

[0060] Step S212: Detect the part-of-speech information of each word in the text sequence, and detect the preset sentence structure that each word satisfies based on the part-of-speech information; Part-of-speech (POS) information is used to label the grammatical category (e.g., noun, verb, adjective) of each word in a text sequence, representing the function of each word in a sentence. The POS information of each word in a text sequence can be identified through preset POS tagging rules. These rules record common words and their corresponding POS. For example, after POS tagging, "I want a glass of water" can be identified as: "I" as a noun / subject, "want" as a verb / predicate, "a glass" as a quantifier / adverbial, and "water" as a noun / object.

[0061] Predefined sentence structures are sentence composition patterns pre-defined according to language rules, including the arrangement rules of syntactic components (such as subject, predicate, and object), such as subject-predicate structures and verb-object structures. For example, "I want a glass of water" includes the subject "I" and the predicate "want," satisfying the subject-predicate structure: "I want."

[0062] Optionally, for words not recorded in the preset part-of-speech tagging rules, the part-of-speech of the unrecorded words can be determined by combining the analysis capabilities of large language models (such as GPT, deepseek, etc.), thereby obtaining the corresponding part-of-speech information and improving the accuracy and efficiency of detection.

[0063] Similarly, when comparing a text sequence with a preset sentence structure, the analytical capabilities of a large language model can be used to directly determine the preset sentence structure satisfied in the text sequence, thereby improving the accuracy and efficiency of the comparison.

[0064] Step S213: Determine the fields in the text sequence that satisfy the preset sentence structure, and use the start and end points of the fields as text semantic demarcation points, and divide the current semantic block from the speech feature sequence according to the text semantic demarcation points.

[0065] It should be noted that the speech sequence features are acquired in real time during the speech dialogue, and may not correspond to a complete sentence. Furthermore, the text sequence converted from the speech sequence features may also be incomplete. Therefore, when extracting semantic blocks from these incomplete text sequences, semantic block division is only performed when a complete semantic unit (i.e., one that satisfies the preset sentence structure) is detected. The start and end points of the detected semantic unit are the text semantic boundary points, and the current semantic block is separated from the text sequence based on these boundary points. When no complete semantic unit is detected, the speech feature sequence during the speech dialogue is continuously recorded, and semantic blocks are detected and extracted from the text sequence converted from the speech feature sequence until no new speech feature sequence input is available.

[0066] For example, an ASR model is used to convert a speech sequence into a text sequence. The part-of-speech (POS) information of each word in the text sequence is labeled based on preset POS tagging rules, which record common words and their corresponding POS. Simultaneously, the POS combinations of each word in the speech sequence are matched with a preset sentence structure to determine the preset sentence structure satisfied by the speech sequence. Based on the matched preset sentence structure, the current semantic block is segmented from the text sequence.

[0067] Understandably, by converting speech sequence features into text sequences in real time, a data foundation is provided for text-based semantic analysis. By detecting the part-of-speech information of each word in the text sequence and matching it with a preset sentence structure, potential semantic unit boundaries are accurately identified at the grammatical level. This divides the text sequence into current semantic blocks bounded by text semantic demarcation points, overcoming the shortcomings of fixed time window segmentation in ensuring semantic integrity. This provides grammatically complete and semantically self-consistent processing units for subsequent translation processes.

[0068] In this embodiment, a dual boundary detection mechanism combining acoustic and textual methods is employed to address the translation quality degradation caused by inaccurate semantic unit segmentation in streaming translation. By accurately locating natural semantic pauses based on prosodic features, speech stream segmentation is achieved. Simultaneously, by combining part-of-speech analysis and sentence structure parsing, semantic boundaries are verified and determined at the text level. This ensures that each semantic block possesses a complete grammatical structure and semantic independence, providing high-quality input units for subsequent parallel translation. Consequently, translation accuracy is significantly improved while maintaining low latency.

[0069] Based on the first and / or second embodiments described above, a third embodiment of the speech translation method of this application is proposed. In this embodiment, content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. (Refer to...) Figure 3 , Figure 3This is a flowchart illustrating the third embodiment of the audio recording method of this application. In this embodiment, step S30, which combines the contextual information of the current semantic block with its semantic representation to generate a preliminary translation result for the current semantic block, includes: Step S301: Extract the context information of the current semantic block based on the preset context cache pool. The context information includes the translation results of the historical semantic blocks, which are semantic blocks segmented before the current semantic block. The pre-defined context cache pool is a data structure used to store the translation results (such as output hidden states or semantic unit representations) corresponding to historical semantic blocks. It can be in the form of an array, queue, stack, etc. Historical semantic blocks are semantic blocks segmented before the current semantic block. The preceding information is the translation result of the historical semantic block, serving as a preliminary reference for translating the current semantic block. This information can include two types of data: output hidden states or semantic unit representations.

[0070] Step S302: In the process of encoding and decoding the semantic representation, the information from the preceding text is incorporated to obtain a preliminary translation result.

[0071] It should be noted that semantic representation is an abstract representation of the semantics expressed by the current semantic block. To convert it into natural language form, it needs to be encoded and decoded. During the encoding and decoding process, the preceding information in the preset context cache pool can be used as additional input and incorporated into the encoding and decoding process of the current semantic block to simulate the historical information dependency of the autoregressive model. This allows us to obtain a preliminary translation result containing the preceding information of the current semantic block.

[0072] Optionally, during the training phase of the NAT model, pre-built cross-block training samples can be used to enable the model to learn and utilize the contextual information of historical fragments during training, thereby improving the NAT model's ability to understand cross-block semantic associations. Each training sample in the pre-built cross-block training samples contains the end part of the current semantic block and the previous semantic block (such as the first 5-10 tokens). These cross-block training samples can represent the contextual information of the historical fragments corresponding to the current semantic block.

[0073] Understandably, by storing the hidden state or semantic representation of historical semantic blocks in a pre-defined context cache pool, the computational redundancy caused by the need for repeated computation of long-distance dependencies in autoregressive models is solved, the efficiency of context information retrieval is improved, and streaming translation can maintain context consistency even with low latency.

[0074] In one feasible implementation, step S40, which involves correcting the semantically changed parts of the preliminary translation result by incorporating contextual information from the current semantic block to obtain the target translation result, includes: Step S401: Obtain the context information of the current semantic block, wherein the context information is the text meaning of the future semantic block, and the future semantic block is the semantic block segmented after the current semantic block; It should be noted that when extracting semantic blocks from a text sequence, a semantic unit is only separated from the text sequence as a semantic block when a complete semantic unit is detected. When no complete semantic unit is detected, the speech feature sequence during the speech dialogue is continuously recorded, and semantic blocks are detected and extracted from the text sequence converted from the speech feature sequence. These extracted semantic blocks are the future semantic blocks segmented after the current semantic block. They contain the speaker's future speech communication content, i.e., context information, and can serve as the subsequent consideration information for the current semantic block.

[0075] Step S402: Determine whether the following information changes the contextual meaning of the preliminary translation result; Contextual meaning refers to the semantic plausibility of the preliminary translation within a complete sentence formed by combining the preliminary translation with the information in the following text. For example, if the following text contains negative words, transition words, or other content that affects the semantics of the preliminary translation, then the contextual meaning of the preliminary translation can be considered to have changed.

[0076] Step S403: If the contextual information changes the meaning of the preliminary translation result, correct the semantically changed parts of the preliminary translation result based on the contextual information and the preliminary translation result to obtain the target translation result; Step S404: If the following information does not change the contextual meaning of the preliminary translation result, the preliminary translation result is taken as the target translation result.

[0077] For example, for a future semantic block received after the current semantic block, it is first determined whether the contextual information corresponding to the future semantic block will change the contextual meaning of the preliminary translation result. This can be determined by detecting whether the contextual information contains content that affects the semantics of the preliminary translation result, or by using a large language model for semantic analysis. If the contextual information changes the contextual meaning of the preliminary translation result, the contextual information and the preliminary translation result are re-input into the NAT model for translation. By combining the preceding information (represented by the preliminary translation result) and the following information (represented by the contextual information) of the current semantic block, the parts of the preliminary translation result that have changed semantics can be effectively corrected, resulting in a target translation result that takes into account the context of the current semantic block. If the contextual information does not change the contextual meaning of the preliminary translation result, the preliminary translation result is directly output as the target translation result.

[0078] Understandably, by establishing a dynamic correction mechanism driven by contextual information, after obtaining the textual meaning of future semantic blocks, it is determined whether they affect the contextual meaning of the preliminary translation result. When semantic conflicts or missing information are detected, the preliminary translation result is corrected based on the contextual information, so that the final translation result maintains contextual coherence. This solves the problem of semantic incompleteness caused by the lack of future context in non-autoregressive parallel translation. While maintaining the low latency advantage of parallel translation, semantic compensation improves the accuracy of translation, achieving the optimal balance between latency and quality.

[0079] In one feasible implementation, step S402 includes: Step A01: Determine whether there is semantic adjustment content in the following information, wherein the semantic adjustment content includes at least one of negation words and transition words; Semantic adjustment content refers to words that can directly change the semantic direction of a sentence, including negative words (such as "no", "without") and transition words (such as "but", "however"). These words will cause semantic changes to the existing preliminary translation results. Therefore, when semantic adjustment content appears, it is necessary to correct the original semantics, that is, to adjust the preliminary translation results to a form that does not conflict with the information in the following text.

[0080] Optionally, the identification of semantic adjustment content can also be combined with the determination of large language models. The analytical capabilities of large language models can be used to identify the semantic adjustment content implied in the following information, such as antonyms, rhetorical questions, passive sentences, etc.

[0081] Step A02: If there is semantic adjustment content in the following information, determine whether the following information changes the contextual meaning of the preliminary translation result; Step A03: If there is no semantic adjustment content in the following information, determine that the following information has not changed the contextual meaning of the preliminary translation result.

[0082] For example, the system scans for semantic adjustments such as negation words and transition words in the following context. For instance, when translating the sentence "The plan was good but poorly executed," if the current semantic block is "The plan was good" and the transition word "but" is detected in the following context, it is determined that semantic adjustment exists. When semantic adjustment is detected in the following context, it is determined that the following context will change the contextual meaning of the initial translation result. For example, the initial translation of "The plan was good but poorly executed" as "This plan was good" needs to be corrected to "The plan itself was feasible but poorly executed." When no semantic adjustment is detected in the following context (e.g., the following context is "and successfully implemented"), it is determined that the following context will not change the contextual meaning of the initial translation result.

[0083] Understandably, by detecting semantically adjusted content, key logical identifiers such as negation words and transition words in the following information are identified to determine their semantic impact on the initial translation results. When such semantically adjusted content is detected, a correction process is triggered to ensure that the final output can correctly reflect the speaker's complete intent. This solves the problem of semantic coherence judgment caused by the lack of foresight in parallel translation, realizes the precise triggering of correction operations, avoids unnecessary computational overhead, and effectively improves the logical accuracy and contextual consistency of the translation results while maintaining low latency.

[0084] In this embodiment, by constructing a context-aware parallel translation and correction mechanism, the semantic incoherence problem caused by the lack of bidirectional context in non-autoregressive models in streaming scenarios is solved. First, a preset cache pool is used to extract historical hidden states as contextual information to ensure that the initial translation results are consistent with the preceding content. Then, by detecting key logical identifiers such as negation words and transition words in subsequent semantic blocks, the semantic changes caused by the following context information are dynamically determined and corrected. While maintaining the low latency advantage of parallel computing, the translation accuracy is improved by accurately utilizing bidirectional contextual information, achieving the optimal balance between latency and quality in full-duplex dialogue scenarios.

[0085] This application also provides a voice translation device; please refer to... Figure 4 The voice translation device includes: The semantic segmentation module 10 is used to segment the real-time acquired speech feature sequence according to the semantic boundaries in the speech feature sequence to obtain the current semantic block; The first translation module 20 is used to translate each syntactic unit in the current semantic block in parallel to obtain the semantic representation of the current semantic block; The second translation module 30 is used to generate a preliminary translation result of the current semantic block by combining the contextual information of the current semantic block and the semantic representation, wherein the contextual information is determined by the historical semantic blocks segmented before the current semantic block; The third translation module 40 is used to correct the semantically changed parts in the preliminary translation result by combining the context information of the current semantic block, so as to obtain the target translation result. The context information is determined by the future semantic block segmented after the current semantic block.

[0086] The speech translation device provided in this application adopts the speech translation method in the above embodiments. Compared with the prior art, the beneficial effects of the speech translation device provided in this application are the same as those of the speech translation method provided in the above embodiments. Furthermore, the other technical features in the speech translation device are the same as those disclosed in the methods of the above embodiments, and will not be described in detail here.

[0087] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the speech translation method in the first embodiment described above.

[0088] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0089] like Figure 5As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0090] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0091] Compared with the prior art, the beneficial effects of the electronic device provided in this application embodiment are the same as those of the speech translation method provided in the above embodiment, and other technical features of the electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0093] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the speech translation method in the above embodiments.

[0094] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0095] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0096] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the functions defined in the methods of the embodiments disclosed in this application.

[0097] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0099] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0100] The readable storage medium provided in this application embodiment is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-described speech translation method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application embodiment are the same as the beneficial effects of the speech translation method provided in the above-described embodiments, and will not be repeated here.

[0101] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the speech translation method described above.

[0102] Compared with the prior art, the beneficial effects of the computer program product provided in this application embodiment are the same as the beneficial effects of the speech translation method provided in the above embodiments, and will not be repeated here.

[0103] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A speech translation method, characterized by, The speech translation method includes: The real-time acquired speech feature sequence is segmented in real time according to the semantic boundaries in the speech feature sequence to obtain the current semantic block; Parallel translation of each syntactic unit in the current semantic block yields the semantic representation of the current semantic block; A preliminary translation result of the current semantic block is generated by combining the contextual information of the current semantic block and the semantic representation, wherein the contextual information is determined by the historical semantic blocks segmented before the current semantic block; By combining the context information of the current semantic block, the semantically changed parts in the preliminary translation result are corrected to obtain the target translation result, wherein the context information is determined by the future semantic blocks segmented after the current semantic block.

2. The speech translation method of claim 1, wherein, The semantic boundary includes natural semantic pause points, and the step of segmenting the real-time acquired speech feature sequence according to the semantic boundary in the speech feature sequence to obtain the current semantic block includes: Prosodic feature analysis is performed on the real-time acquired speech feature sequence, wherein the prosodic features include at least one of fundamental frequency, speech rate, and pause duration; Based on the prosodic features, the natural semantic pause points are extracted from the speech feature sequence; If the natural semantic pause point is extracted, the current semantic block is divided from the speech feature sequence based on the natural semantic pause point.

3. The speech translation method of claim 1, wherein, The semantic boundary includes text semantic demarcation points. The step of segmenting the real-time acquired speech feature sequence according to the semantic boundary in the speech feature sequence to obtain the current semantic block further includes: Convert the real-time acquired speech feature sequence into a text sequence; Detect the part-of-speech information of each word in the text sequence, and detect the preset sentence structure satisfied by each word based on the part-of-speech information; The fields in the text sequence that satisfy the preset sentence structure are identified, and the start and end points of the fields are used as the text semantic demarcation points. The current semantic block is divided from the speech feature sequence based on the text semantic demarcation points.

4. The speech translation method of claim 1, wherein, The step of generating a preliminary translation result for the current semantic block by combining the contextual information of the current semantic block and the semantic representation further includes: The context information of the current semantic block is extracted based on a preset context cache pool, wherein the context information includes the translation results of historical semantic blocks, and the historical semantic blocks are semantic blocks segmented before the current semantic block; The aforementioned information is incorporated into the encoding and decoding process of the semantic representation to obtain a preliminary translation result.

5. The speech translation method of claim 1, wherein, The step of correcting the semantically changed parts of the preliminary translation result by combining the context information of the current semantic block to obtain the target translation result includes: Obtain the context information of the current semantic block, wherein the context information is the text meaning of the future semantic block, and the future semantic block is a semantic block segmented after the current semantic block; Determine whether the following information changes the contextual meaning of the preliminary translation result; If the contextual information changes the meaning of the preliminary translation result, the semantically altered parts of the preliminary translation result are corrected based on the contextual information and the preliminary translation result to obtain the target translation result; Without altering the contextual meaning of the preliminary translation result, the preliminary translation result shall be taken as the target translation result.

6. The speech translation method as described in claim 5, characterized in that, The step of determining whether the following information changes the contextual meaning of the preliminary translation result includes: Determine whether there is semantically adjusted content in the following information, wherein the semantically adjusted content includes at least one of negation words and transition words; If the semantically adjusted content exists in the following context, it is determined that the following context changes the contextual meaning of the preliminary translation result; If the semantic adjustment content is not present in the following context, it is determined that the following context has not changed the contextual meaning of the preliminary translation result.

7. The speech translation method as described in claim 1, characterized in that, Before the step of real-time segmentation of the acquired speech feature sequence according to the semantic boundaries in the speech feature sequence, the method further includes: Real-time monitoring of voice signals during voice conversations; The speech signal is preprocessed to obtain a speech feature sequence, wherein the preprocessing includes at least one of noise reduction, speech activity detection, and feature extraction.

8. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the speech translation method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the speech translation method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the speech translation method as described in any one of claims 1 to 7.

Images