A method and apparatus for modeling an end-to-end speech translation model based on cross-language CTC

Abstract

The application relates to an end-to-end speech translation model modeling method and device based on cross-language CTC, and belongs to the technical field of natural language processing; the method solves the problem that the speech translation method in the prior art ignores the guidance of target language text to an encoder and the monotone hypothesis and conditional independent hypothesis problems of CTC; the modeling method comprises the following steps: constructing an initial speech translation model; the initial speech translation model comprises an acoustic encoder, a text encoder and a decoder; obtaining a speech data set; the speech data set comprises source language speech data, source language labeled text corresponding to the speech data and target language labeled text; the initial speech translation model is trained by using the speech data set, is iteratively updated by using a loss function, and the speech translation model is obtained.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an end-to-end speech translation modeling method and device based on cross-language CTC. Background Technology

[0002] Early speech translation models were designed by first transcribing speech into text using a speech recognition model, and then translating the transcribed text using a text translation model—a process known as cascaded speech translation. While cascaded speech translation achieved good performance, this approach suffers from problems such as error propagation, low translation efficiency, and loss of paralinguistic information in the speech.

[0003] Recently, researchers have proposed end-to-end speech translation, which uses a single model to generate text in a target language from speech in a specified source language. This method can effectively alleviate the aforementioned problems, but it faces two new challenges:

[0004] Task modeling is complex. Speech translation models need to learn how to generate text from source language speech, making convergence more difficult due to the cross-modal and cross-lingual nature of the model. Furthermore, since end-to-end models lack intermediate output information, it's challenging to target specific problems that arise during translation. Especially in real-world scenarios, audio may contain significant noise, and colloquial sentence structures are often irregular. Addressing these issues is a major challenge and pain point for end-to-end systems in practical applications.

[0005] The lack of data accumulation is a major obstacle to the development of speech translation. End-to-end speech translation is an emerging field, and most datasets have been labeled in recent years. The most commonly used dataset, MUST-C, only contains a few hundred hours of audio. The lack of data is the biggest obstacle to the development of speech translation.

[0006] To address these two issues, existing solutions use Conditional Tracing (CTC) as an auxiliary loss to predict the annotation problem corresponding to the input audio, thereby guiding the encoder's learning. However, existing methods neglect the guidance from the target language text for the encoder. Furthermore, CTC relies on two important assumptions: the monotonicity assumption and the conditional independence assumption, which to some extent hinder its application. Summary of the Invention

[0007] Based on the above analysis, the present invention aims to provide an end-to-end speech translation modeling method and device based on cross-language CTC; it addresses the problems of existing speech translation methods that only predict the source language text, ignoring the guidance from the target language text for the encoder, and the monotonicity and conditional independence assumptions of CTC, which to some extent hinder the use of CTC.

[0008] The object of the present invention is mainly achieved by the following technical solutions:

[0009] On the one hand, the present invention provides an end-to-end speech translation model modeling method based on cross-language CTC, including the following steps:

[0010] Obtain a speech data set; the speech data set includes source language speech data, source language annotation text corresponding to the speech data, and target language annotation text;

[0011] Construct an initial speech translation model; the initial speech translation model includes an acoustic encoder, a text encoder, and a decoder; the acoustic encoder is used to extract features from the input speech and perform source language semantic embedding to obtain an acoustic encoder hidden vector; the text encoder is used to extract features from the acoustic encoder hidden vector and perform target language semantic embedding to obtain a text encoder hidden vector; the decoder is used to decode based on the text encoder hidden vector to obtain a translation text corresponding to the input speech;

[0012] Use the speech data set to train the initial speech translation model, and through iterative update of the loss function, obtain the speech translation model.

[0013] Further, the acoustic encoder includes n layers of feature extraction layers connected in sequence and a first semantic embedding module provided in the l-th layer of feature extraction layers, where n is an integer greater than 1, and 2 / n < l < n; where

[0014] The n layers of feature extraction layers are used to sequentially extract features from the input source language speech data;

[0015] The semantic embedding module is used to perform semantic embedding on the output of the l-th layer of feature extraction layers of the acoustic encoder to obtain a tensor representation SIM(h l );

[0016] The tensor representation SIM(h l ) is the input of the (l + 1)-th layer of feature extraction layers of the encoder.

[0017] Further, the semantic embedding module performs semantic embedding through the following method:

[0018] Calculate the CTC distribution for the predicted source language text on the output of the l-th layer of feature extraction layers of the acoustic encoder;

[0019] Weight the source language word embedding matrix W through the CTC distribution to obtain a soft word embedding matrix with text representation; the source language word embedding matrix W is a model parameter;

[0020] The soft word embedding matrix is ​​added to the output of the l-th feature extraction layer to obtain a tensor representation SIM(h) with preliminary predicted content. l ).

[0021] Furthermore, the semantic embedding module obtains the tensor representation SIM(h) using the following formula. l ):

[0022] SIM(h l ) = h l +P SIM_CTC (π|h l )·W;

[0023] Among them, SIM(h l ) is the tensor representation with preliminary predicted content output by the semantic embedding module of the l-th feature extraction layer of the speech encoder; h l The output of the l-th feature extraction layer is π, where π is the predicted source language text; W is the source language word embedding matrix, and P is the source language word embedding matrix. SIM_CTC () represents the CTC distribution output by the l-th feature extraction layer of the encoder.

[0024] Furthermore, an adapter module is included between the acoustic encoder and the text encoder. This adapter module processes the latent vectors of the acoustic encoder to obtain an acoustic representation with source language semantic information. The acoustic representation Used to decode the input speech to obtain the translated text;

[0025] The processing of the acoustic encoder latent vector includes:

[0026] The latent vectors of the acoustic encoder are used to calculate the CTC distribution P for the predicted source language text. SRC_CTC ;

[0027] The CTC distribution P SRC_CTC The source language word embedding matrix is ​​weighted, and the weighted acoustic representation is added to the latent vector of the acoustic encoder to obtain an acoustic representation containing source language semantic information.

[0028] Furthermore, the text encoder includes n sequentially connected feature extraction layers and a second semantic embedding module disposed in the Kth feature extraction layer; the second semantic embedding module is used to perform target language semantic embedding, including:

[0029] The CTC distribution P for the predicted target language text is calculated using the output representation of the Kth layer of the text encoder. SIM_TGT_CTC (h K K is an integer greater than 1;

[0030] Through the CTC distribution P SIM_TGT_CTC (h K The word embedding matrix of the target language text is weighted, and the weighted representation is added to the output representation of the Kth layer to obtain the representation SIM(h) containing the semantic information of the target language text. K The representation of the target language text semantic information is the input of the (K+1)th layer of the text encoder.

[0031] Furthermore, the loss for the source language annotated text is calculated for the speech encoder and adapter module respectively, and the loss for the target language annotated text is calculated for the text encoder and decoder respectively, and different weights are set. After iterative updates, the speech translation model is obtained.

[0032] Furthermore, the loss weight of the acoustic encoder is set to 0.2, the loss of the adapter is set to 0.3, the loss weight of the middle layer of the text encoder is set to 0.2, the loss weight of the top layer of the text encoder is set to 0.3, and the loss weight of the decoder is set to 1.

[0033] Furthermore, the acoustic encoder includes 12 feature extraction layers, and a first semantic embedding module is set for the 6th and 9th layers respectively to perform source language semantic embedding.

[0034] On the other hand, a computer device is also provided, including at least one processor and at least one memory communicatively connected to said processor;

[0035] The memory stores instructions that can be executed by the processor to implement the aforementioned end-to-end speech translation modeling method based on cross-language CTC.

[0036] The beneficial effects of this technical solution are:

[0037] To address the problems of existing technologies that only use source language CTC and that CTC contains monotonic and conditional independence assumptions, this invention incorporates prediction information into the encoding process by calculating CTC loss in the intermediate layer of the encoder; and sets up an additional text encoder to predict target language text through CTC, thereby strengthening the supervision information while reducing the loss caused by the built-in assumptions of CTC, and significantly improving the performance of the speech translation model.

[0038] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0039] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0040] Figure 1 This is a flowchart illustrating the end-to-end speech translation modeling method based on cross-language CTC according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the speech translation model structure according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram illustrating the integration of prediction results according to an embodiment of the present invention; Detailed Implementation

[0043] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0044] This invention discloses an end-to-end speech translation modeling method based on cross-language CTC. The input is speech to be processed, and the acoustic encoder incorporates prediction information into the encoding process. The text encoder uses CTC to predict the target language text, which strengthens the supervision information and reduces the loss caused by the built-in assumptions of CTC, thus realizing high-quality end-to-end speech translation from speech data to target language text.

[0045] This embodiment presents an end-to-end speech translation modeling method based on cross-language CTC, such as... Figure 1 As shown, it includes the following steps:

[0046] Step S1: Obtain the speech dataset.

[0047] Specifically, this embodiment uses the MuST-C English-German dataset, which is commonly used in the field of speech translation and has annotations. The dataset includes source language speech data, source language annotation text corresponding to the speech data, and target language annotation text.

[0048] Step S2: Construct the initial speech translation model.

[0049] Specifically, such as Figure 2 As shown, the initial speech translation model in this embodiment includes an acoustic encoder, a text encoder, and a decoder;

[0050] Among them, the acoustic encoder is used to extract features from the input speech and embed the source language semantics to obtain the acoustic encoder latent vector;

[0051] Preferably, the acoustic encoder of this embodiment includes n layers of feature extraction layers connected in sequence and a first semantic embedding module disposed in the l-th layer of feature extraction layers, where n is an integer greater than 1, and 2 / n < l < n; wherein,

[0052] The n layers of feature extraction layers are used to sequentially extract features from the input source language speech data; the n layers of feature extraction layers can adopt n layers of the same Transformer layers or Conformer layers. In this embodiment, 12 layers of Transformer layers are adopted, and each layer respectively includes a self-attention module and a feed-forward neural network layer. If Conformer layers are adopted, a convolutional layer is further included in each layer to capture local information.

[0053] The first semantic embedding module is used to perform semantic embedding on the output of the l-th layer of feature extraction layers to obtain a tensor representation SIM(h l );

[0054] Specifically, the first semantic embedding module performs semantic embedding through the following method, as Figure 3 shown,

[0055] Calculate the CTC distribution for the predicted source language text on the output of the l-th layer of feature extraction layers of the speech encoder. In the CTC distribution, the predicted probability of each token in the corresponding vocabulary is included. The vocabulary refers to the set of words constituting the training set text data;

[0056] Weight the source language word embedding matrix W through the CTC distribution to obtain a soft word embedding matrix with text representation, which contains the CTC prediction information at the current layer; the source language word embedding matrix W is a model parameter, randomly initialized before training, and the parameters are continuously updated during training.

[0057] Add the soft word embedding matrix to the output of the l-th layer of feature extraction layers to obtain a tensor representation SIM(h l ), which is used as the input of the (l + 1)-th layer of feature extraction layers.

[0058] Through the above operations, preliminary prediction content is injected into the subsequent encoding of the model, so that continuous revision can be performed in subsequent predictions, alleviating the conditional independence assumption problem in CTC.

[0059] Preferably, the semantic embedding module obtains the tensor representation SIM(h[[ID=3']] l ) through the following formula:

[0060] SIM(h l ) = h l + P SIM_CTC (π|h l ) · W;

[0061] Among them, SIM(h l ) represents the tensor representation with preliminary predicted content output by the semantic embedding module of the l-th feature extraction layer of the speech encoder; h l P is the output of the l-th feature extraction layer, π is the vector representation of the predicted source language text; W is the source language word embedding matrix, and P... SIM_CTC () represents the CTC distribution of the output of the l-th feature extraction layer of the encoder.

[0062] In this embodiment, computation is performed at the upper layer of the encoder to ensure that CTC achieves good prediction results through sufficient encoding. For the 12-layer acoustic encoder in this embodiment, the first semantic embedding module can be inserted into the 6th and 9th feature extraction layers, respectively. Due to the conditional independence assumption of CTC itself, CTC does not consider the context of the prediction results during the prediction process, which can easily lead to problems such as repeated generation. Therefore, by integrating intermediate prediction results, the model can perceive the intermediate prediction results before the top-level prediction, thereby mitigating the conditional independence assumption.

[0063] As a specific embodiment, an adapter module can also be used between the acoustic encoder and the text encoder. The adapter module is used to process the latent vectors of the acoustic encoder to obtain an acoustic representation with source language semantic information. Acoustic representation Used to decode the input speech to obtain the translated text;

[0064] Specifically, the latent vector of the acoustic encoder is input into the adapter. First, the CTC distribution P of the latent vector of the acoustic encoder for the predicted source language text is calculated. SRC_CTC ;

[0065] Then, through the CTC distribution P SRC_CTC The source language word embedding matrix is ​​weighted, and the weighted acoustic representation is added to the latent vector of the acoustic encoder to obtain an acoustic representation containing source language semantic information. As shown in the following formula:

[0066]

[0067] Among them, h a This is the latent vector of the acoustic encoder.

[0068] Unlike the semantic embedding operation of the speech encoder, which is intended to alleviate the conditional independence assumption, the adapter module embeds the source language semantics into the encoder's latent vectors and then inputs them into the text encoder. This incorporates text information into the acoustic encoder's latent vectors, further enriching the information represented by the acoustic encoder's output vectors.

[0069] Furthermore, the text encoder is used to extract features from the acoustic encoder latent vector and embed the target language semantics to obtain the text encoder latent vector; the decoder is used to decode based on the text encoder latent vector to obtain the translated text corresponding to the input speech.

[0070] Preferably, the text encoder adopts a structure similar to that of the acoustic encoder, including n sequentially connected feature extraction layers and a second semantic embedding module disposed in the Kth feature extraction layer; the second semantic embedding module is used to perform target language semantic embedding.

[0071] Specifically, due to the monotonicity assumption inherent in CTC, it naturally holds true for cross-modal speech recognition; however, cross-language learning is difficult to achieve directly. Therefore, this embodiment uses a pre-trained text translation model to re-translate the source language text in the training sample set. The re-translated target language text is then used to replace the target language text in the training sample set. This results in a cleaner target language text with more reasonable alignment compared to the original target language text data in the training sample set, which is beneficial for training the text encoder.

[0072] Specifically, the text encoder extracts features from the acoustic encoder's latent vectors and embeds target language semantics using the following method to obtain the text encoder's latent vectors:

[0073] First, the CTC distribution P for the predicted target language text is calculated using the output representation of the Kth layer of the text encoder. SIM_TGT_CTC (h K K is an integer greater than 1;

[0074] The word embedding matrix of the target language text is weighted using the CTC distribution. The weighted representation is then added to the output representation of the Kth layer to obtain a tensor representation SIM(h) containing the semantic information of the target language text. K ); Tensor representation SIM(h) containing semantic information of the target language text K ) is the input of the K+1 layer of the text encoder.

[0075] Notably, similar to the acoustic encoder, the text encoder also performs calculations in the upper feature extraction layer, ensuring better performance of CTC. Furthermore, since this embodiment uses regenerated target language text as the training target for CTC, it effectively alleviates the monotonicity assumption problem in CTC.

[0076] Furthermore, the latent vectors of the text encoder are input into the decoder to generate the target language text corresponding to the speech to be translated.

[0077] The decoder of the speech translation model consists of multiple identical Transformer layers, each containing a self-attention module, an encoder-decoder attention module, and a feedforward neural network module. Since the text translation model already possesses a certain ability to predict the target language text within the encoder, the learning burden on the decoder is significantly reduced. The output of the text encoder is fed into the decoder of the end-to-end speech translation model to generate the final target language text.

[0078] Step S3: Train the initial speech translation model using the speech dataset, and obtain the speech translation model through iterative updates of the loss function.

[0079] Specifically, after acquiring the speech dataset, the first step is to extract frame-level feature sequences from the audio files using signal processing methods, focusing on the acoustic task. In this embodiment, pre-emphasis, framing, and windowing operations in signal processing are used to obtain frame-level feature sequences of the speech data in the dataset through Discrete Fourier Transform. The frame-level feature sequences can be 80-dimensional MFCC features or FBank features.

[0080] The frame-level feature sequences corresponding to the obtained speech data, as well as the source and target language texts corresponding to the speech data, are input into the speech encoder, adapter module, text encoder, and decoder of the speech translation model. The loss is calculated using a loss function, and different weights are set. After iterative updates, the speech translation model is obtained.

[0081] For the speech encoder, the loss is calculated using the CTC prediction distribution and the source language annotation text corresponding to the input audio, as shown in the following formula:

[0082]

[0083] The loss is minimized during training. To ensure the accuracy of CTC predictions, the CTC prediction results are supervised and weighted with a small weight, such as 0.2 in this embodiment, so that it does not excessively affect the learning of the loss of other modules.

[0084] For the adapter, the CTC loss is calculated using the representation output by the acoustic encoder, and this loss is minimized as shown in the following equation:

[0085]

[0086] Similarly, in order to improve the prediction accuracy of CTC, the CTC loss is minimized during training, and a larger weight is assigned to it than that of the speech encoder CTC. In this embodiment, it is set to 0.3 for better optimization.

[0087] For the text encoder, the loss is calculated using the intermediate CTC prediction distribution and the target language labeled text corresponding to the input audio. During training, this loss is minimized with a weight of 0.2.

[0088] The CTC loss is calculated using the output representation of the top layer of the text encoder and the target language labeled text. During training, the loss is continuously reduced by optimizing it, with a weight set to 0.3.

[0089] The cross-entropy loss is calculated using the decoder's output representation and the target language labeled text. Alignment optimization is used to continuously reduce this loss during training, with the weight set to 1.

[0090] The proposed method was validated on a speech translation task. Utilizing the commonly used MuST-C English-German dataset, this embodiment is based on a Transformer architecture, employing a 12-layer acoustic encoder, a 6-layer text encoder, and a 6-layer decoder, with a hidden layer dimension of 256. The base model achieved a BLEU score of 24.6, and adding cross-language CTC resulted in a BLEU score of 25.1. By mitigating the conditional independence assumption, a BLEU score of 26.0 was achieved. Furthermore, by using text generated by the text translation model as the learning target for the target language CTC, the monotonicity assumption was mitigated, ultimately achieving a BLEU score of 26.7, representing a 2.1 BLEU improvement over the baseline. The proposed method is highly beneficial for end-to-end speech translation models, effectively alleviating modeling pressure and achieving significant performance improvements.

[0091] In practical applications, the trained speech translation model is loaded, the speech to be recognized is received, the corresponding FBank features are extracted by signal processing tools, and the features are input into the speech translation model. The speech encoder, adapter, text encoder and decoder perform feature extraction and prediction to obtain the target language translation text corresponding to the speech to be recognized.

[0092] Another embodiment of the present invention also provides a computer device, including at least one processor and at least one memory communicatively connected to the processor;

[0093] The memory stores instructions that can be executed by the processor to implement the aforementioned end-to-end speech translation modeling method based on cross-language CTC.

[0094] In summary, the embodiments of the present invention provide an end-to-end speech translation modeling method based on cross-language CTC. By calculating the CTC loss in the intermediate layer of the encoder, the prediction information is incorporated into the encoding process. Furthermore, an additional text encoder is set up to predict the target language text through CTC. This not only strengthens the supervision information but also reduces the loss caused by the built-in monotonic and conditional independence assumptions of CTC, thus significantly improving the performance of the speech translation model.

[0095] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0096] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for modeling an end-to-end speech translation model based on cross-language CTC, characterized in that, Includes the following steps: Obtain a speech dataset; the speech dataset includes source language speech data, source language annotated text corresponding to the speech data, and target language annotated text; An initial speech translation model is constructed; the initial speech translation model includes an acoustic encoder, a text encoder, and a decoder; the acoustic encoder is used to extract features from the input speech and embed the source language semantics, obtaining an acoustic encoder latent vector; the text encoder is used to extract features from the acoustic encoder latent vector and embed the target language semantics, obtaining a text encoder latent vector; the decoder is used to decode based on the text encoder latent vector to obtain the translated text corresponding to the input speech; the acoustic encoder includes n sequentially connected feature extraction layers and a layer set at the nth layer. l The first semantic embedding module of the layer feature extraction layer, where n is an integer greater than 1. The nth feature extraction layer is used to sequentially extract features from the input source language speech data; the first semantic embedding module is used to perform feature extraction on the nth layer. l The output of the feature extraction layer is semantically embedded to obtain a tensor representation with preliminary predicted content. The tensor representation For the encoder number l+1 Input to the layer feature extraction layer; The semantic embedding module performs semantic embedding through the following method: the acoustic encoder's first... l The output of the feature extraction layer calculates the CTC distribution for the predicted source language text; the CTC distribution contains the predicted probability of each token in the vocabulary, which is the set of words constituting the training set text data; the source language word embedding matrix W is weighted by the CTC distribution to obtain a soft word embedding matrix with text representation; the source language word embedding matrix W is the model parameter; the soft word embedding matrix is ​​then compared with the first... l The outputs of the feature extraction layers are summed to obtain a tensor representation with preliminary prediction content. ; The text encoder includes n sequentially connected feature extraction layers and a second semantic embedding module disposed in the Kth feature extraction layer; the second semantic embedding module is used to perform target language semantic embedding, including: The CTC distribution for the predicted target language text is calculated using the output representation of the Kth layer of the text encoder. K is an integer greater than 1; Through the CTC distribution The word embedding matrix of the target language text is weighted, and the weighted representation is added to the output representation of the Kth layer to obtain a representation containing the semantic information of the target language text. The representation containing the target language text semantic information is the input of the (K+1)th layer of the text encoder. The initial speech translation model is trained using the speech dataset, and then iteratively updated using a loss function to obtain the final speech translation model.

2. The end-to-end speech translation modeling method based on cross-language CTC as described in claim 1, characterized in that, The semantic embedding module obtains the tensor representation using the following formula. : ； in, For the voice encoder l The semantic embedding module of the layer feature extraction layer outputs a tensor representation with preliminary predicted content; For the first l The output of the layer feature extraction layer, where π is the predicted source language text; W is the source language word embedding matrix. For the encoder number l The CTC distribution output by the layer feature extraction layer.

3. The end-to-end speech translation modeling method based on cross-language CTC as described in claim 1, characterized in that, An adapter module is also included between the acoustic encoder and the text encoder. This adapter module processes the latent vectors of the acoustic encoder to obtain an acoustic representation with source language semantic information. The acoustic representation Used to decode the input speech to obtain the translated text; The processing of the acoustic encoder latent vector includes: The latent vectors of the acoustic encoder are used to calculate the CTC distribution for the predicted source language text. ; The CTC distribution The source language word embedding matrix is ​​weighted, and the weighted acoustic representation is added to the latent vector of the acoustic encoder to obtain an acoustic representation containing source language semantic information. .

4. The end-to-end speech translation modeling method based on cross-language CTC according to claim 3, characterized in that, The loss for the source language annotated text is calculated for the speech encoder and adapter module respectively, and the loss for the target language annotated text is calculated for the text encoder and decoder respectively, with different weights set. After iterative updates, the speech translation model is obtained.

5. The end-to-end speech translation modeling method based on cross-language CTC according to claim 4, characterized in that, The loss weight of the acoustic encoder is set to 0.2, the loss of the adapter is set to 0.3, the loss weight of the middle layer of the text encoder is set to 0.2, the loss weight of the top layer of the text encoder is set to 0.3, and the loss weight of the decoder is set to 1.

6. The end-to-end speech translation modeling method based on cross-language CTC according to claim 5, characterized in that, The acoustic encoder includes 12 feature extraction layers, and the 6th and 9th layers are respectively equipped with a first semantic embedding module to perform source language semantic embedding.

7. A computer device, characterized in that, It includes at least one processor and at least one memory communicatively connected to the processor; The memory stores instructions that can be executed by the processor to implement the end-to-end speech translation modeling method based on cross-language CTC as described in any one of claims 1-6.

Images