Japanese speech translation method, terminal and storage medium based on machine translation

Through a machine translation-based method, combined with MFCC feature and voiceprint recognition, the problem of low accuracy of Japanese pronunciation translation is solved, and efficient Japanese pronunciation translation is achieved, especially in dealing with Japanese syllables and pronunciation complexity, tone similarity and spoken language. In terms of deformation.

CN117995167BActive Publication Date: 2024-11-01SHIJIAZHUANG ZHANGSI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410219555.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-28
Publication Date
2024-11-01
Estimated Expiration
2044-02-28

AI Technical Summary

Technical Problem

The general speech recognition model has low accuracy in Japanese pronunciation translation, mainly due to the interference caused by the complexity of syllables and pronunciation, similarity between pronunciation and intonation, and oral deformation and fuzzy expression of Japanese.

Method used

Using machine translation-based method, MFCC feature extraction and voiceprint recognition are performed by obtaining the Japanese voice to be translated, combining the features and translation results in the historical period, input them into the decoder, using the Mel frequency cepspectral coefficient and the speaker's Features reflect differences in rereading, continuous reading, and intonation, achieving efficient translation.

Benefits of technology

The accuracy of Japanese pronunciation translation is improved, and the accuracy and efficiency of translation are enhanced by taking into account the speaker's pronunciation preferences and differences in pronunciation characteristics.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117995167B_ABST
    Figure CN117995167B_ABST
Patent Text Reader

Abstract

The present invention provides a Japanese speech translation method, terminal and storage medium based on machine translation, which obtains Japanese speech to be translated; extracts features of the Japanese speech to be translated to obtain MFCC features; performs voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated; inputs the MFCC features and the first features corresponding to the identity information into a decoder to obtain a first translation result; wherein the first feature is determined according to the MFCC features identified in a historical period and the standard MFCC features corresponding to the first translation result in the historical period; the identity information corresponds to the first feature one by one. By adding the first feature of the speaker in speech recognition to reflect the differences in the speaker's stress, connected reading and voice intonation, and using the Mel frequency cepstral coefficient to extract effective information in the speech, namely, the MFCC features, the two features are input into the decoder to achieve efficient translation of Japanese speech.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of natural language processing and conversion technology, and in particular, relates to a Japanese speech translation method, terminal and storage medium based on machine translation. Background Art

[0002] With the continuous development of science and technology, language recognition technology has become more and more mature. Japanese voice translation refers to the conversion of Japanese speech into non-Japanese text through speech recognition technology. In recent years, with the continuous improvement of artificial intelligence technology, Japanese voice translation technology has also developed rapidly. Now, more and more scenarios require the use of Japanese voice translation technology.

[0003] Compared with other languages, Japanese speech translation has some difficulties. First, there are a large number of syllables and pronunciations in Japanese, which makes Japanese speech translation more difficult. In addition, there are many similarities between pronunciation and intonation in Japanese, which can easily lead to confusion. In addition, there are a large number of oral deformations and ambiguous expressions in Japanese, which can also cause certain interference in Japanese speech translation. Therefore, general speech recognition models usually have a low translation accuracy for Japanese speech. Summary of the invention

[0004] In view of this, the present invention provides a Japanese speech translation method, terminal and storage medium based on machine translation, aiming to solve the problem that the general speech recognition model usually has a low translation accuracy for Japanese speech.

[0005] A first aspect of an embodiment of the present invention provides a Japanese speech translation method based on machine translation, comprising:

[0006] Get the Japanese voice to be translated;

[0007] Extract features of the Japanese speech to be translated and obtain MFCC features;

[0008] Perform voiceprint recognition on the Japanese voice to be translated to determine the identity information of the Japanese voice to be translated;

[0009] Inputting the MFCC feature and the first feature corresponding to the identity information into a decoder to obtain a first translation result;

[0010] The first feature is determined according to the MFCC features identified in the historical period and the standard MFCC features corresponding to the first translation result in the historical period; and the identity information corresponds to the first feature one by one.

[0011] A second aspect of an embodiment of the present invention provides a Japanese speech translation device based on machine translation, comprising:

[0012] An acquisition module is used to acquire Japanese speech to be translated;

[0013] An extraction module is used to extract features of the Japanese speech to be translated and obtain MFCC features;

[0014] A determination module, used to perform voiceprint recognition on the Japanese speech to be translated, and determine the identity information of the Japanese speech to be translated;

[0015] A translation module, used for inputting the MFCC feature and the first feature corresponding to the identity information into a decoder to obtain a first translation result;

[0016] The first feature is determined according to the MFCC features identified in the historical period and the standard MFCC features corresponding to the first translation result in the historical period; and the identity information corresponds to the first feature one by one.

[0017] A third aspect of an embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the Japanese speech translation method based on machine translation of the first aspect are implemented.

[0018] A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the Japanese speech translation method based on machine translation in the first aspect are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The Japanese speech translation method, terminal and storage medium based on machine translation provided by the embodiment of the present invention obtain Japanese speech to be translated; extract features of the Japanese speech to be translated to obtain MFCC features; perform voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated; input the MFCC features and the first features corresponding to the identity information into the decoder to obtain the first translation result; wherein the first feature is determined according to the MFCC features identified in the historical period and the standard MFCC features corresponding to the first translation result in the historical period; the identity information corresponds to the first feature one by one. By adding the first feature of the speaker in speech recognition to reflect the differences in the speaker's stress, connected reading and voice intonation, and using the Mel frequency cepstral coefficient to extract effective information in the speech, namely, the MFCC features, these two features are input into the decoder to achieve efficient translation of Japanese speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 is a flow chart of a Japanese speech translation method based on machine translation provided by an embodiment of the present invention;

[0023] Figure 2 It is a structural schematic diagram of a Japanese speech translation device based on machine translation provided by an embodiment of the present invention;

[0024] Figure 3 It is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0026] Figure 1 FIG. 1 is a flowchart of a Japanese speech translation method based on machine translation provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0027] S110, obtaining Japanese speech to be translated.

[0028] In the embodiment of the present invention, the Japanese speech to be translated can be speech in any scenario, such as speech when a user uses a speech translator, voice instructions when a user uses Japanese to interact intelligently with a non-Japanese device, etc., and is not limited here.

[0029] In some embodiments, after obtaining the Japanese speech to be translated, the method further includes: preprocessing the Japanese speech to be translated to remove noise in the Japanese speech to be translated.

[0030] In the embodiment of the present invention, it is usually necessary to input the Japanese speech signal into a first-order filter for pre-emphasis, and then perform framing and windowing to capture spectrum details, thereby completing the pre-processing of the Japanese speech to be translated.

[0031] S120, extracting features of the Japanese speech to be translated to obtain MFCC (Mel Frequency Cepstrum Coefficient) features.

[0032] In the embodiments of the present invention, MFCC is widely used for feature extraction, which can effectively improve the recognition rate and accuracy.

[0033] In some embodiments, S120 includes: performing fast Fourier transform on the Japanese speech to be translated to obtain a spectrogram; inputting the spectrogram into a Mel filter, performing logarithmic transform on the obtained Mel acoustic signal to convert it into a cepstrum; performing discrete cosine transform on the cepstrum to obtain MFCC features.

[0034] In the embodiment of the present invention, after performing a fast Fourier transform on each frame of the input Japanese speech to be translated, the energy contained in the speech signal of each frame at different frequencies can be obtained, and the speech signal spectrum of each frame is merged in the order of time expansion. The essence of the Mel filter is a triangular filter, which is composed of two orthogonal sine waves, whose frequencies are 0 and 1, respectively called the fundamental wave and the harmonic wave.

[0035] S130, performing voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated.

[0036] In some embodiments, S130 includes: inputting the Japanese speech to be translated into a pre-established voiceprint recognition model to determine the identity information of the Japanese speech to be translated.

[0037] In the embodiment of the present invention, the pre-established voiceprint recognition model can be a neural network model or a deep learning model. In the present invention, there is no need to process the voiceprint information too much, and it is only used to determine the identity of the speaker and distinguish different speakers, so the model should be set as small as possible.

[0038] S140, inputting the MFCC feature and the first feature corresponding to the identity information into the decoder to obtain a first translation result. The first feature is determined based on the MFCC feature identified in the historical period and the standard MFCC feature corresponding to the first translation result in the historical period; and the identity information corresponds to the first feature one by one.

[0039] In the embodiment of the present invention, the first translation result may be non-Japanese text, such as Chinese.

[0040] In some embodiments, the method further includes: obtaining standard speech data, text data, dictionary data, and at least one group of non-standard speech data corresponding to the standard speech data; training a bidirectional long short-term memory network model based on time series data classification according to the standard speech data to obtain a target acoustic model; training a second neural network according to the text data to obtain a target language model; training a first neural network according to at least one group of non-standard speech data; and constructing a decoder according to the target acoustic model, the first neural network model, the dictionary data, and the target language model.

[0041] In an embodiment of the present invention, the standard voice data is a pre-made text read by a professional announcer, and an expert performs simple word-by-word recognition on the standard voice data. The standard voice data and the simple recognition result are used as input and output to train the acoustic model, and the acoustic model can parse the input voice signal into the initial Japanese text. The text data is shuffled, the disordered text is used as input and the normal text is used as output, and the target language model can be obtained by training the second neural network. After the training is completed, the above Japanese text is translated through the stored dictionary data to obtain a disordered Chinese text, and then re-arranged according to the semantics and grammar according to the target language model to obtain a correctly expressed Chinese text.

[0042] In the embodiment of the present invention, the bidirectional long short-term memory network model includes an input layer, a forward layer, a backward layer and an output layer. The bidirectional long short-term memory network is an improved structure of the long short-term memory network, which not only considers the impact of historical information of speech data on the current frame, but also considers the effect of future information on the current frame. Therefore, its network structure has information transmission processes in both positive and negative directions in the time series.

[0043] In order to improve translation efficiency, the present invention further improves the bidirectional long short-term memory network model through the first neural network model, that is, one or more hidden layers are added between the backward layer and the output layer of the bidirectional long short-term memory network model, and the output result of the first neural network model is the parameter of the hidden layer.

[0044] The output of the first neural network can reflect the user's pronunciation preference to a certain extent, thereby detecting differences in stress, connected speech, voice intonation, and deviation and unclearness of specific pronunciations / syllables, thereby effectively improving the accuracy of the speech model and making the recognition rate of the constructed decoder higher.

[0045] In an embodiment of the present invention, for a piece of standard speech read out by a professional announcer, people with different oral deformations and habits are found to read it out repeatedly, and multiple non-standard speech data corresponding to the standard speech data are obtained. Experts manually label the stress, connected reading, wrong pauses, unclear specific pronunciation / syllables, specific pronunciation / syllable deviations, intonation deviations and the like in these non-standard speech data, and give a first feature corresponding to the label, and formulate a training set and a test set to train the first neural network.

[0046] The above-mentioned first neural network is very dependent on the information given by historical data, that is, the user has used this device for voice recognition before. In some embodiments, a plurality of first neural network models are arranged in the decoder; the above-mentioned plurality of first neural network models are trained by different training sets. Cluster analysis is performed on the above-mentioned non-standard voice data to divide all participants into groups with different habits and spoken language deformations. Although there are still certain differences in the pronunciation habits of the people in each group, they also have certain similarities. The decoder records the deviation curve of the characteristic values ​​of the standard voice data input in the training set of each first neural network and each non-standard voice data, and takes the average value of the obtained multiple curves for fitting to obtain the first deviation curve corresponding to the first neural network.

[0047] If the user is a new user, that is, there is no MFCC feature in the historical period, then the MFCC feature is input into the decoder, and the initial first translation result is obtained through the input layer, forward layer, backward layer and output layer. The standard speech data corresponding to the initial first translation result is searched, and then the current MFCC feature is used as the historical data, and the deviation between it and the standard MFCC feature corresponding to the standard speech data is calculated as the second deviation curve. The similarity of each first deviation curve and the second deviation curve is calculated, and the first neural network corresponding to the first deviation curve with the highest similarity is selected, which means that the current user belongs to the group corresponding to the first neural network and has similar pronunciation preferences. The first feature is calculated using the first neural network, the current MFCC feature, and the standard MFCC feature, and then the first feature and the current MFCC feature are input into the decoder to obtain the final first translation result.

[0048] In some embodiments, the method further includes: inputting the MFCC features and the standard MFCC features into the first neural network to obtain a first difference feature; inputting the MFCC features and the standard MFCC features into the first neural network to which a local self-attention mechanism is added to obtain a second difference feature; and determining the first feature corresponding to the identity information based on the first difference feature and the second difference feature.

[0049] In the embodiment of the present invention, the first difference feature is used to represent the overall difference between the user speech segment and the standard speech, and the first neural network with the local self-attention mechanism can effectively consider the local features, and its output is used to represent the local difference between the user speech segment and the standard speech. The specific local self-attention mechanism is set by experts according to the language scenario to be applied and is not limited here.

[0050] In some embodiments, determining the first feature corresponding to the identity information based on the first difference feature and the second difference feature includes: adding the first difference feature and the second difference feature to obtain the first feature corresponding to the identity information.

[0051] In the embodiment of the present invention, the first difference feature and the second difference feature are both matrices composed of eigenvalues. By adding the two matrices, the first feature, i.e., the parameter of the hidden layer of the above-mentioned improved bidirectional long short-term memory network model, can be obtained.

[0052] In some embodiments, the first feature includes a habit feature, a tone feature, and an expression feature; determining the first feature corresponding to the identity information based on the first difference feature and the second difference feature includes: determining the tone feature based on the first difference feature; determining the habit feature and the expression feature based on the second difference feature.

[0053] In addition to the above-mentioned direct addition method, in an embodiment of the present invention, the pronunciation characteristics of the user can also be divided in detail, for example, into habitual features, intonation features, and expression features. Among them, the habitual features are used to indicate that the user has stress, connected reading, wrong pauses, unclear specific pronunciation / syllables, etc.; the intonation features are used to indicate that the user has high intonation, low intonation, etc.; the expression features are used to indicate that the user has specific pronunciation / syllable deviations, oral deformation, and fuzzy expressions. Usually, the intonation of the same user will not fluctuate, so only the overall feature difference with the standard voice, that is, the above-mentioned first difference feature, needs to be considered to determine it as the intonation feature. The user's stress, connected reading or pronunciation deviation, etc., usually only exist in a part of a speech, so more attention is paid to local information, that is, the above-mentioned second difference feature. In an embodiment of the present invention, two hidden layers are set in the above-mentioned improved bidirectional long short-term memory network model, the first hidden layer corresponds to the influence of intonation features on speech recognition, and the second hidden layer corresponds to the influence of habitual features and expression features on speech recognition.

[0054] In summary, the beneficial effects of the present invention are:

[0055] By adding the speaker's first feature to speech recognition to reflect the differences in speaker stress, connected speech, and voice intonation, the Mel-frequency cepstral coefficients, namely MFCC features, are used to extract effective information from the speech. These two features are input into the decoder to achieve efficient translation of Japanese speech.

[0056] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0057] Figure 2 FIG. 1 is a schematic diagram of the structure of a Japanese speech translation device based on machine translation provided by an embodiment of the present invention. Figure 2 As shown, the Japanese speech translation device 2 based on machine translation includes:

[0058] An acquisition module 210 is used to acquire Japanese speech to be translated;

[0059] An extraction module 220 is used to extract features of the Japanese speech to be translated to obtain MFCC features;

[0060] A determination module 230 is used to perform voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated;

[0061] A translation module 240, configured to input the MFCC feature and the first feature corresponding to the identity information into a decoder to obtain a first translation result;

[0062] The first feature is determined according to the MFCC features identified in the historical period and the standard MFCC features corresponding to the first translation result in the historical period; and the identity information corresponds to the first feature one by one.

[0063] Optionally, the Japanese speech translation device 2 based on machine translation also includes: a first feature extraction module, used to input MFCC features and standard MFCC features into the first neural network to obtain a first difference feature; input the MFCC features and the standard MFCC features into the first neural network to which a local self-attention mechanism is added to obtain a second difference feature; and determine the first feature corresponding to the identity information based on the first difference feature and the second difference feature.

[0064] Optionally, the first feature extraction module is specifically used to add the first difference feature and the second difference feature to obtain a first feature corresponding to the identity information.

[0065] Optionally, the first feature includes a habit feature, a tone feature, and an expression feature; the first feature extraction module is specifically used to: determine the tone feature according to the first difference feature; and determine the habit feature and the expression feature according to the second difference feature.

[0066] Optionally, the Japanese speech translation device 2 based on machine translation also includes: a training module, used to obtain standard speech data, text data, dictionary data and at least one group of non-standard speech data corresponding to the standard speech data; according to the standard speech data, a bidirectional long short-term memory network model based on time series data classification is trained to obtain a target acoustic model; according to the text data, the second neural network is trained to obtain a target language model; according to the at least one group of non-standard speech data, the first neural network is trained; according to the target acoustic model, the first neural network model, the dictionary data, and the target language model, a decoder is constructed.

[0067] Optionally, the extraction module 220 is used to: perform fast Fourier transform on the Japanese speech to be translated to obtain a spectrogram; input the spectrogram into a Mel filter, perform logarithmic transform on the obtained Mel acoustic signal to convert it into a cepstrum; perform discrete cosine transform on the cepstrum to obtain MFCC features.

[0068] Optionally, the determination module 230 is used to: input the Japanese speech to be translated into a pre-established voiceprint recognition model to determine the identity information of the Japanese speech to be translated.

[0069] Optionally, the Japanese speech translation device 2 based on machine translation further includes a preprocessing module, which is used to preprocess the Japanese speech to be translated to remove noise in the Japanese speech to be translated.

[0070] The Japanese speech translation device based on machine translation provided in this embodiment can be used to execute the above method embodiment. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.

[0071] Figure 3 FIG. 1 is a schematic diagram of a terminal provided by an embodiment of the present invention. Figure 3 As shown, a terminal 3 provided in one embodiment of the present invention includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, the steps in the above-mentioned Japanese speech translation method embodiments based on machine translation are implemented, for example Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 2 The functions of modules 210 to 240 are shown.

[0072] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more modules / units are stored in the memory 31, and are executed by the processor 30 to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the terminal 3.

[0073] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0074] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0075] The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal 3. The memory 31 is used to store computer programs and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0076] An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned Japanese speech translation method embodiment based on machine translation are implemented.

[0077] The computer-readable storage medium stores a computer program 32, which includes program instructions. When the program instructions are executed by the processor 30, all or part of the processes in the above-mentioned embodiment method are implemented. The computer program 32 can also be used to instruct the relevant hardware to complete. The computer program 32 can be stored in a computer-readable storage medium. When the computer program 32 is executed by the processor 30, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program 32 includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0078] The computer-readable storage medium may be an internal storage unit of the terminal of any of the foregoing embodiments, such as a hard disk or memory of the terminal. The computer-readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0079] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0080] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0081] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0082] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0083] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0086] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A Japanese speech translation method based on machine translation, characterized in that: include: Get the Japanese voice to be translated; Extracting features of the Japanese speech to be translated to obtain MFCC features; Performing voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated; Inputting the MFCC feature and a first feature corresponding to the identity information into a decoder to obtain a first translation result; Wherein, the first feature is determined according to the MFCC feature identified in the historical period and the standard MFCC feature corresponding to the first translation result in the historical period; the identity information corresponds to the first feature one by one; The method further comprises: Inputting the MFCC feature and the standard MFCC feature into a first neural network to obtain a first difference feature; Inputting the MFCC feature and the standard MFCC feature into a first neural network with a local self-attention mechanism added to obtain a second difference feature; Determining a first feature corresponding to the identity information according to the first difference feature and the second difference feature; The method further comprises: Acquire standard voice data, text data, dictionary data, and at least one group of non-standard voice data corresponding to the standard voice data; According to the standard speech data, a bidirectional long short-term memory network model based on time series data classification is trained to obtain a target acoustic model; Training a second neural network according to the text data to obtain a target language model; Training the first neural network according to the at least one set of non-standard speech data; The decoder is constructed according to the target acoustic model, the first neural network model, the dictionary data, and the target language model.

2. The Japanese speech translation method based on machine translation according to claim 1, characterized in that: Determining a first feature corresponding to the identity information according to the first difference feature and the second difference feature includes: The first difference feature and the second difference feature are added to obtain a first feature corresponding to the identity information.

3. The Japanese speech translation method based on machine translation according to claim 1, characterized in that: The first features include habit features, intonation features, and expression features; Determining a first feature corresponding to the identity information according to the first difference feature and the second difference feature includes: determining the intonation feature according to the first difference feature; The habit feature and the expression feature are determined according to the second difference feature.

4. The Japanese speech translation method based on machine translation according to claim 1, characterized in that: Feature extraction is performed on the Japanese speech to be translated to obtain MFCC features, including: Performing a fast Fourier transform on the Japanese speech to be translated to obtain a spectrogram; The spectrogram is input into a Mel filter, and the obtained Mel acoustic signal is logarithmically transformed to be converted into a cepstrum; The cepstrum is subjected to discrete cosine transform to obtain MFCC features.

5. The Japanese speech translation method based on machine translation according to claim 1, characterized in that: Performing voiceprint recognition on the Japanese speech to be translated to determine the identity information of the Japanese speech to be translated includes: The Japanese speech to be translated is input into a pre-established voiceprint recognition model to determine the identity information of the Japanese speech to be translated.

6. The Japanese speech translation method based on machine translation according to any one of claims 1 to 5, characterized in that: After obtaining the Japanese speech to be translated, the method further includes: preprocessing the Japanese speech to be translated to remove noise in the Japanese speech to be translated.

7. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that:When the processor executes the computer program, the steps of the Japanese speech translation method based on machine translation as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the Japanese speech translation method based on machine translation as described in any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Method and apparatus for speech translation, device and computer readable storage medium

    CN108766414A

  • Speech translation method and device and storage medium

    CN114765024A

  • Translation method and device, electronic equipment and storage medium

    CN117351929A

  • Information processor, information processing method and program

    JP2021124530A