Deep learning-based multi-scene japanese interpretation intelligent interactive training method and system

Abstract

This invention discloses a multi-scenario intelligent interactive training method and system for Japanese interpretation based on deep learning. It acquires real-time voice data from users, converts this data into text data using an ASR speech recognition model, preprocesses the text data to obtain preprocessed text data, establishes a multi-granularity Japanese language feature extraction model, identifies honorific sequence features based on the model, extracts particle sequence features using an encoder, and compares word order scene differences using a syntactic analysis tree. The preprocessed text data is input into the multi-granularity Japanese language feature extraction model to obtain Japanese language features. These features are then input into an RNN neural network for judgment to determine Japanese error types. Based on these error types, corrected Japanese speech data is generated for multi-scenario intelligent interactive training of users. This makes Japanese interpretation training more accurate and efficient.

Description

Technical Field

[0001] This invention relates to the field of language education technology, and in particular to a multi-scenario intelligent interactive training method and system for Japanese interpretation based on deep learning. Background Technology

[0002] With increasingly frequent global exchanges, the demand for Japanese interpretation continues to grow in business, academic, and everyday scenarios, constantly raising the bar for interpreters' language accuracy and scenario adaptability. Traditional Japanese interpretation training relies heavily on manual guidance, which suffers from low efficiency, lack of personalization, and limited scenario coverage, making it difficult to meet large-scale and diverse training needs. Existing intelligent interpretation training systems mostly focus on correcting errors in a single language dimension, lacking deep adaptation to the characteristics of Japanese grammar. Core issues such as misuse of honorifics, confusion of particles, and disordered word order are difficult to accurately identify, resulting in low efficiency for users' multi-scenario Japanese training. Summary of the Invention

[0003] The purpose of this invention is to solve the above-mentioned problems by designing a multi-scenario Japanese interpretation intelligent interactive training method and system based on deep learning.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned multi-scenario Japanese interpretation intelligent interactive training method based on deep learning:

[0005] The user's real-time voice data is acquired, and the real-time voice data is converted into text data using an ASR speech recognition model. The text data is then preprocessed to obtain preprocessed text data.

[0006] A multi-granular Japanese language feature extraction model was established. The honorific sequence features were identified based on the LSTM-Attention model, the particle sequence features were extracted using the Transformer encoder, and the word order scene difference features were compared through syntactic analysis tree.

[0007] The preprocessed text data is input into the multi-granularity Japanese language feature extraction model to obtain Japanese language features;

[0008] The Japanese language features are input into an RNN neural network for judgment to obtain the Japanese error type. Based on the Japanese error type, Japanese corrected speech data is generated to train users for multi-scenario intelligent interaction.

[0009] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the step of acquiring the user's real-time voice data, converting the real-time voice data into text data using an ASR speech recognition model, and preprocessing the text data to obtain preprocessed text data includes:

[0010] The system acquires real-time voice data from users, trains and optimizes Japanese based on the Conformer model to obtain an ASR speech recognition model, and uses the ASR speech recognition model to recognize the real-time voice data to obtain text data.

[0011] Remove meaningless interjections, duplicate characters, and garbled characters with recognition errors from the text data; correct the mixed use of Japanese kanji and kana.

[0012] By using Japanese word segmentation tools in conjunction with interpretation scenarios, we can optimize word segmentation rules, break down compound words and honorific words, and mark their parts of speech.

[0013] The process involves converting colloquial expressions into standard written language and transforming the segmented text into a fixed-length vector sequence to obtain preprocessed text data.

[0014] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the establishment of a multi-granularity Japanese language feature extraction model, the identification of honorific sequence features based on the LSTM-Attention model, the extraction of particle sequence features using the Transformer encoder, and the comparison of word order scene differences through syntactic analysis trees include:

[0015] A Bi-LSTM bidirectional long short-term memory network is used. The input layer is the preprocessed word embedding vector, the hidden layer dimension is set to 256, the number of layers is 2, and a dropout layer is added to prevent overfitting.

[0016] An Attention mechanism is introduced into the LSTM output layer to calculate the weight of each token for honorific recognition, focusing on honorific prefixes, honorific verbs, and honorific sentence structures to generate a weighted honorific feature vector.

[0017] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the establishment of a multi-granularity Japanese language feature extraction model, the identification of honorific sequence features based on the LSTM-Attention model, the extraction of particle sequence features using the Transformer encoder, and the comparison of word order scene differences through syntactic analysis trees include:

[0018] The Transformer encoder has 6 coding blocks, each containing a multi-head self-attention mechanism and a feedforward neural network. It uses LayerNorm normalization and residual connections to alleviate the gradient vanishing problem.

[0019] Given that Japanese particles have no real meaning but determine the logic of a sentence, a self-attention mechanism is used to calculate the association weights between particles and the subject, object, and predicate, filter out irrelevant words, and extract dimensional particle feature vectors.

[0020] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the establishment of a multi-granularity Japanese language feature extraction model, the identification of honorific sequence features based on the LSTM-Attention model, the extraction of particle sequence features using the Transformer encoder, and the comparison of word order scene differences through syntactic analysis trees include:

[0021] The Japanese constant syntactic analysis model is used to parse the syntactic structure of the preprocessed text, generate a syntactic analysis tree containing phrase types and grammatical relations, and mark the syntactic position of each word;

[0022] Construct a multi-scenario standard Japanese syntax tree library, compare the user syntax tree with the standard syntax tree of the same scenario, calculate the tree edit distance, extract word order misalignment features, and generate word order scenario difference feature vectors;

[0023] We use a weighted concatenation method to fuse honorific feature vectors, particle feature vectors, and word order scene difference feature vectors into Japanese language features.

[0024] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the step of inputting the Japanese language features into an RNN neural network for judgment to obtain the Japanese error type, and generating Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interactive training for the user, includes:

[0025] A bidirectional GRU network is used, with a 1024-dimensional multi-granularity feature vector as the input layer, a 512-dimensional hidden layer, and an error type label as the output layer.

[0026] Japanese language features are input into a bidirectional GRU network. The correlation logic between features is captured through temporal modeling. The probability of each error type is output through the softmax function, and the type with the highest probability is used as the judgment result.

[0027] Furthermore, in the aforementioned deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, the step of inputting the Japanese language features into an RNN neural network for judgment to obtain the Japanese error type, and generating Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interactive training for the user, includes:

[0028] The system calls upon a multi-scenario Japanese language correction rule library, combines it with the user's original text to generate correction text, and then uses a Japanese TTS model to convert the correction text into Japanese correction speech data.

[0029] The Japanese corrected speech data is broadcast to the user in real time through an audio player, and a multi-scenario reinforcement training mechanism is triggered for high-frequency error types.

[0030] Furthermore, in the deep learning-based multi-scenario Japanese interpreting intelligent interactive training system, the system includes the following modules:

[0031] The Japanese data acquisition module is used to acquire the user's real-time voice data, convert the real-time voice data into text data using an ASR speech recognition model, and preprocess the text data to obtain preprocessed text data.

[0032] The recognition model building module is used to build a multi-granular Japanese language feature extraction model. It identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using the Transformer encoder, and compares word order scene differences features through syntactic analysis tree.

[0033] The language feature extraction module is used to input the preprocessed text data into the multi-granularity Japanese language feature extraction model to obtain Japanese language features.

[0034] The intelligent interactive training module is used to input the Japanese language features into the RNN neural network for judgment, obtain the Japanese error type, and generate Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interactive training for users.

[0035] Furthermore, in the deep learning-based multi-scenario Japanese interpretation intelligent interactive training system, the language feature extraction module includes the following sub-modules:

[0036] A submodule is established to use a bidirectional GRU network, with an input layer of 1024-dimensional multi-granularity feature vectors, a hidden layer dimension of 512, and an output layer of error type labels.

[0037] The judgment submodule is used to input Japanese language features into a bidirectional GRU network, capture the correlation logic between features through temporal modeling, output the probability of each error type through the softmax function, and take the type with the highest probability as the judgment result.

[0038] Furthermore, in the deep learning-based multi-scenario Japanese interpretation intelligent interactive training system, the language feature extraction module includes the following sub-modules:

[0039] The correction submodule is used to call the multi-scenario Japanese correction rule library, combine it with the user's original text to generate correction text, and use a Japanese TTS model to convert the correction text into Japanese correction speech data;

[0040] The training submodule is used to broadcast the Japanese corrected speech data to the user in real time through an audio player, and to trigger a multi-scenario reinforcement training mechanism for high-frequency error types.

[0041] Its beneficial effects lie in its adaptability to various training scenarios, including quiet indoor environments and noisy outdoor environments, through dynamic parameter adjustment and noise reduction. The ASR model, optimized for Japanese pronunciation, achieves an accuracy rate exceeding 95%, laying a high-quality data foundation for subsequent processing. It boasts high feature extraction accuracy, with multi-granularity models focusing on core dimensions such as honorifics, particles, and word order. Through the synergistic effect of LSTM-Attention, Transformer encoders, and syntactic parsing trees, it comprehensively captures and integrates multi-dimensional features, significantly improving the accuracy of error recognition. The error judgment model based on a bidirectional GRU network can accurately identify 12 specific error types, generating targeted corrective speech and text prompts. Combined with reinforcement training mechanisms and personalized reports, it forms a complete training loop. This reduces training costs, improves training efficiency, and caters to the personalized needs of users at different levels, making Japanese interpretation training more accurate and efficient. Attached Figure Description

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0043] Figure 1 This is a schematic diagram of the first embodiment of the deep learning-based multi-scenario Japanese interpretation intelligent interactive training method in this invention.

[0044] Figure 2 This is a schematic diagram of the second embodiment of the deep learning-based multi-scenario Japanese interpretation intelligent interactive training method in this invention.

[0045] Figure 3 This is a schematic diagram of the first embodiment of the deep learning-based multi-scenario Japanese interpretation intelligent interactive training system in this invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0047] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0048] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, a multi-scenario Japanese interpreting intelligent interactive training method based on deep learning is presented. This multi-scenario Japanese interpreting intelligent interactive training method includes the following steps:

[0049] Step 101: Obtain the user's real-time voice data, use the ASR speech recognition model to convert the real-time voice data into text data, preprocess the text data to obtain preprocessed text data.

[0050] Specifically, in this embodiment, real-time voice data of the user is acquired, and Japanese is trained and optimized based on the Conformer model to obtain an ASR speech recognition model. The ASR speech recognition model is then used to recognize the real-time voice data to obtain text data. Meaningless interjections, repeated characters, and garbled characters with recognition errors are removed from the text data, and the mixing of Japanese kanji and kana is corrected. Japanese word segmentation tools are used in combination with the interpretation scenario to optimize the word segmentation rules, split compound words and honorific words, and mark the parts of speech. Colloquial expressions are converted into standard written language, and the segmented text is converted into a fixed-length vector sequence to obtain preprocessed text data.

[0051] Voice data acquisition and processing are fundamental components of the Japanese intelligent interactive training system, directly determining the accuracy and reliability of subsequent model calculations. This requires end-to-end optimization from raw speech to standardized text, adapting to diverse application scenarios. The core objective of the real-time voice data acquisition module is to obtain high-quality, highly adaptable voice materials. To achieve this, a multi-device compatible and multi-scenario adaptive acquisition architecture must be constructed, incorporating precise noise reduction technology. At the device access level, the module must support various common audio devices such as microphones, headphone microphones, and desktop microphones, achieving plug-and-play functionality through multi-protocol adaptation via USB, Bluetooth, and other protocols, thus resolving device compatibility issues for different users. To address the differences in acoustic characteristics across various application scenarios, a dynamically adjustable sampling parameter system is pre-set. The sampling rate is uniformly set to 16kHz, a parameter verified through extensive Japanese speech testing, which fully covers the frequency range of Japanese pronunciation. This avoids data redundancy caused by excessively high sampling rates while preventing the loss of pronunciation details due to insufficient sampling rates. The bit depth is fixed at 16bit to ensure the quantization accuracy of the speech data and reproduce the volume fluctuations and timbre characteristics of the pronunciation. The channel mode can be flexibly switched according to the scenario. Mono acquisition is used in single-person speech training scenarios to reduce irrelevant data interference and improve acquisition efficiency, while multi-person interactive training or business meeting scenarios switch to stereo to retain the positional information and speech layers of different speakers. To address noise interference in complex environments, the module incorporates a dual noise reduction mechanism. It precisely removes both steady-state and non-steady-state noise from the environment, such as outdoor wind noise, indoor air conditioning noise, and background human voices, through spectral subtraction. This algorithm analyzes the difference between the speech spectrum and the noise spectrum to attenuate the noise frequency band while preserving the original speech signal to the maximum extent possible. Furthermore, it combines Wiener filtering technology to correct distortion issues inherent in the device itself, including uneven microphone frequency response and signal attenuation during audio transmission. An adaptive filtering algorithm dynamically adjusts the filtering parameters, ultimately ensuring that the signal-to-noise ratio of the acquired speech data is ≥30dB, meeting the input standard for subsequent ASR recognition.

[0052] ASR speech recognition and text data preprocessing are crucial steps connecting speech acquisition and feature extraction. Accurate conversion from speech signals to standardized text is essential to provide high-quality input for subsequent model computation. The ASR recognition module employs an end-to-end Conformer fine-tuning model optimized for Japanese. This model integrates the self-attention mechanism of the Transformer with the local feature extraction capabilities of CNNs, effectively capturing the temporal features and local pronunciation details of Japanese speech. Compared to traditional ASR models, it achieves higher accuracy in spoken language and multi-scenario speech recognition.

[0053] In the model input stage, the mel-spectrum features of the denoised speech data need to be extracted first, extracting 24-dimensional mel coefficients, and adding first-order and second-order difference features at the same time. The difference features can strengthen the temporal change information of the speech, compensate for the deficiency of single mel coefficients in capturing dynamic pronunciation, and enable the model to more accurately identify pronunciation phenomena such as liaison and促音in Japanese. To optimize the recognition accuracy of Japanese-specific pronunciations, the model is fine-tuned based on a large-scale Japanese speech corpus, which covers spoken language data in multiple scenarios such as daily conversations, business negotiations, and academic exchanges. At the same time, it focuses on supplementing labeled data for easily confused pronunciations such as促音,拗音, and长音. By repeatedly iterating training to correct the model parameters, reduce the recognition errors caused by the complexity of Japanese pronunciation, and the target recognition accuracy is ≥95%. After recognition, the original text with punctuation marks is output, strictly retaining the word order, pauses, and tone features of the user's spoken language, providing a complete original basis for subsequent text preprocessing and error determination.

[0054] In the text data preprocessing stage, multiple rounds of standardization operations need to be performed to form text data that meets the model input requirements. In the text cleaning stage, meaningless modal particles are removed through regular expressions and semantic analysis algorithms, such as common modal particles in spoken language likeええ,あのう,まあ, etc. At the same time, duplicate characters, misrecognized garbled characters, and invalid symbols are filtered. For the problem of mixed use of Japanese Chinese characters and kana, a unified format specification is established. According to the application scenario, kana expressions such asかんじare unified into the Chinese character form of漢字or kept in the pure kana format to avoid format chaos affecting subsequent processing. The word segmentation process uses the MeCab word segmentation tool combined with the IPAdic dictionary, and at the same time optimizes the word segmentation rules for the interpretation scenario. Through manual annotation, the splitting logic of compound words and honorific vocabulary is adjusted. For example,お待たせしましたis accurately split intoお / 待たせ / しました, and the词性is also marked synchronously, including nouns, verbs, particles, honorific prefixes, auxiliary verbs, etc., laying a foundation for subsequent grammar feature extraction. In the normalization process, colloquial expressions are converted into standard written language. For example, colloquial suffixes such as~だよ,~ねare changed to the standard forms of~です,~ます, and the formats of numbers and dates are unified. The Arabic numeral format or the Japanese era format such as January, Heisei 48 can be selected according to the needs. At the same time, redundant components of repeated emphasis words and colloquialisms are removed to improve the normativity of the text. In the sequence formatting stage, the segmented text is converted into a vector sequence with a fixed length of 512 tokens. The insufficient length is supplemented through Padding operations, and the overly long text is truncated through Truncation operations to ensure the unity of the input sequence format. Then, the pre-trained FastText-Japanese Japanese word vector model is used to initialize the word embedding vector, converting the text information into numerical features recognizable by the model, preparing for multi-granularity language feature extraction.

[0055] Step 102: Establish a multi-granular Japanese language feature extraction model, identify honorific sequence features based on the LSTM-Attention model, extract particle sequence features using the Transformer encoder, and compare word order scene differences features through syntactic analysis tree.

[0056] Specifically, this embodiment employs a Bi-LSTM bidirectional long short-term memory network. The input layer is the preprocessed word embedding vector, the hidden layer dimension is set to 256, and the number of layers is 2. A dropout layer is added to prevent overfitting. An attention mechanism is introduced into the LSTM output layer to calculate the weight of each token for honorific recognition, focusing on honorific prefixes, honorific verbs, and honorific sentence structures to generate a weighted honorific feature vector. The Transformer encoder consists of 6 encoding blocks, each containing a multi-head self-attention mechanism and a feedforward neural network. LayerNorm normalization and residual connections are used to alleviate the gradient vanishing problem. Taking advantage of the fact that Japanese particles have no real meaning but determine the sentence logic, the self-attention mechanism is used to calculate the association weights between particles and the subject, object, and predicate, filtering out irrelevant words and extracting dimensional particle feature vectors. A multi-scenario standard Japanese syntax tree library is constructed. The user syntax tree is compared with the standard syntax tree of the same scenario, the tree edit distance is calculated, word order misalignment features are extracted, and word order scenario difference feature vectors are generated. The honorific feature vectors, particle feature vectors and word order scenario difference feature vectors are merged into Japanese language features using a weighted concatenation method.

[0057] The construction and execution of a multi-granularity Japanese language feature extraction model is the core computational component of the system. Through a layered architecture design, it captures core Japanese grammatical features, achieving accurate language feature extraction and verification. The model as a whole adopts a three-layer feature extraction architecture, focusing on three core Japanese grammatical dimensions: honorifics, particles, and word order. Through parallel computation of sub-models and feature fusion, it achieves comprehensive capture of multi-granular features, adapting to the complex grammatical structure, rigorous honorific system, and crucial function of particles in Japanese. The LSTM-Attention honorific sequence feature extraction sub-model is specifically designed for the complexity of the Japanese honorific system. Japanese honorifics are divided into three categories: respectful, humble, and formal. The usage of honorifics varies significantly depending on the context and the recipient, representing a key challenge and a high-risk area for errors in Japanese learning and application. This sub-model uses a bidirectional LSTM (Bi-LSTM) network as its basic architecture. The input layer receives preprocessed word embedding vectors, and the hidden layer dimension is set to 256 with 2 layers. It captures the contextual information of honorific sequences through bidirectional propagation. A dropout layer with a dropout rate of 0.3 is added to randomly discard some neuron connections, effectively preventing overfitting and improving generalization ability. To enhance the accurate capture of honorific features, a Bahdanau attention mechanism is introduced into the LSTM output layer. This mechanism can calculate the weight of each token for honorific recognition and automatically focus on honorific prefixes. This model extracts honorifics by including お / ご, honorific verbs, いたすなさるくださる, and honorific sentence structures, weakening the interference of irrelevant words, and generating a weighted honorific feature vector with a dimension of 256 to accurately distinguish the usage differences of different types of honorifics. The model is trained using a specialized annotated corpus of Japanese honorifics, covering various scenarios including daily life, business, and academic use. The annotations include honorific type, applicable scenarios, and collocation rules. The loss function is cross-entropy loss, and the optimizer is the Adam optimizer with a learning rate set to 1e-4. Through repeated iterations of training, the accuracy on the validation set is stabilized above 92%, ensuring the accuracy of honorific feature extraction.

[0058] The Transformer encoder particle sequence feature extraction sub-model focuses on capturing the dependency relationships of Japanese particles. Although Japanese particles have no actual semantic meaning, they play a core role in connecting words and expressing logical relationships. Misuse or omission of particles such as が / を / に / で can directly lead to logical inconsistencies in sentences and are a high-frequency type of error in Japanese expression. This sub-model uses a 6-layer Transformer encoder as its core architecture, with each layer containing an 8-head multi-head self-attention mechanism and a feedforward neural network with a hidden layer dimension of 512. The multi-head self-attention mechanism simultaneously captures the dependency relationships between particles and preceding and following words of different dimensions. The feedforward neural network performs non-linear transformation on the features output by the attention mechanism. At the same time, LayerNorm normalization and residual connections are introduced to effectively alleviate the gradient vanishing problem in deep network training and improve the model training stability. The model calculates the association weights between particles and subjects, objects, and predicates through the self-attention mechanism, filters out irrelevant word interference, and accurately extracts particle feature vectors of dimension 512, which can effectively identify problems such as particle misuse, omission, and incorrect use. To enhance adaptability to interpreting scenarios, the model is first pre-trained on general Japanese corpora such as JCoLA to master basic Japanese grammatical structures and particle usage. Then, it is fine-tuned using multi-scenario interpreting particle error data to specifically optimize particle feature recognition capabilities in interpreting scenarios, adapting to the real-time and conversational expression characteristics of interpreting. The syntactic analysis tree word order difference feature extraction sub-model is designed based on the core word order characteristics of Japanese subject-object-verb. Through syntactic structure parsing and comparison, it captures the word order differences between user expressions and standard Japanese.

[0059] The sub-model employs a BERT-based Japanese syntax analyzer to construct a consistency syntactic parse tree. This tree performs deep syntactic structure parsing on the preprocessed text, generating syntactic parse trees that include phrase types (noun phrases, verb phrases, prepositional phrases, etc.) and grammatical relations (subject-verb, verb-object, modifier, etc.), accurately annotating the syntactic position and function of each word. Simultaneously, a multi-scenario standard Japanese syntax tree library is constructed, covering various scenarios such as daily conversation, business negotiations, academic exchanges, and public speeches. Each scenario includes a large number of standard Japanese expressions and their syntactic tree structures, serving as a benchmark for word order comparison. By calculating the tree edit distance (TreeEditDistance) between the user's syntax tree and the standard syntax tree in the same scenario, the degree of word order difference is quantified, and word order misalignment features are accurately extracted, such as object fronting, predicate position error, modifier misalignment, and adverbial word order disorder, generating a word order difference feature vector with a dimension of 256. In the feature fusion stage, a weighted concatenation method is used to merge the honorific feature vector, particle feature vector, and word order difference feature vector into a multi-granular Japanese language feature vector with a dimension of 1024. The weight of each feature is determined through cross-validation, with a particle feature weight of 0.4, an honorific feature weight of 0.3, and a word order feature weight of 0.3, highlighting the core role of particles in Japanese logical expression while taking into account the importance of honorifics and word order, ensuring that the fused features can comprehensively reflect the grammatical level of Japanese expression.

[0060] Step 103: Input the preprocessed text data into the multi-granularity Japanese language feature extraction model to obtain Japanese language features;

[0061] Specifically, this embodiment establishes a closed-loop process for multi-granularity language feature extraction, error judgment, and intelligent interactive training generation, achieving full-process intelligence from feature analysis to error correction and model iteration. The feature extraction execution stage must ensure input data adaptability, computational parallelism, and feature validity. First, input adaptation processing is performed, synchronously inputting the preprocessed 512-token word embedding sequence into the three sub-models. A temporal synchronization mechanism ensures the temporal consistency of the input data, and a format verification module verifies the uniformity of the data format to avoid computational errors caused by data inconsistency. Subsequently, the parallel feature extraction process is initiated, with the three sub-models operating independently and simultaneously, outputting honorific, auxiliary word, and word order feature vectors respectively. These are then weighted and concatenated using the model's built-in feature fusion module to generate the final 1024-dimensional multi-granularity language feature vector. This parallel computing architecture significantly improves feature extraction efficiency, meeting the timeliness requirements of real-time interactive training. The feature verification stage is crucial for ensuring the accuracy of subsequent error detection. By calculating the norms value of the feature vectors, abnormal feature vectors exceeding the preset range [0.5, 5.0] are eliminated. These outliers typically originate from speech acquisition distortion, text preprocessing errors, or model computational deviations. If verification fails, a retry mechanism is triggered, returning to the speech acquisition stage to re-acquire data and re-execute the entire process to ensure that the feature vectors input to the error detection model meet the required quality. The error detection and intelligent interactive training generation stage, based on the extracted multi-granular features, achieves accurate error detection, targeted correction, and closed-loop optimization, constructing an intelligent interactive system of recognition-correction-training-iteration.

[0062] Step 104: Input the Japanese language features into the RNN neural network for judgment to obtain the Japanese error type, and generate Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interaction training for users.

[0063] Specifically, this embodiment employs a bidirectional GRU network with a 1024-dimensional multi-granularity feature vector as the input layer, a 512-dimensional hidden layer, and error type labels as the output layer. Japanese language features are input into the bidirectional GRU network, and temporal modeling captures the correlation logic between features. The probability of each error type is output via a softmax function, and the type with the highest probability is used as the judgment result. A multi-scenario Japanese correction rule library is invoked, combined with the user's original text, to generate correction text. A Japanese TTS model is then used to convert the correction text into Japanese corrected speech data. This Japanese corrected speech data is then played back to the user in real-time via an audio player. For high-frequency error types, a multi-scenario reinforcement training mechanism is triggered.

[0064] The RNN neural network error type determination module is constructed using a bidirectional GRU (Gated Recurrent Unit) network. GRU, as an optimized version of RNN, effectively alleviates the vanishing gradient problem in long sequences through the design of reset and update gates. Compared to traditional LSTM networks, it is simpler and more computationally efficient, making it suitable for real-time error determination scenarios. The model input layer receives a 1024-dimensional multi-granularity feature vector, the hidden layer dimension is set to 512, and the output layer corresponds to an error type label system, covering 4 major categories and 12 subcategories: misuse of honorifics, incorrect particle usage, disordered word order, and mixed errors. Specifically, it includes subcategories such as confusion between honorific and humble language, misuse of honorific prefixes, misuse of が and を, omission of particles, postposition of predicate, preposition of object, and misplacement of modifiers, achieving accurate error classification. In the error determination process, after the feature vector is input into the bidirectional GRU network, temporal modeling captures the correlation logic between features. The softmax function outputs the probability of each error type, selecting the type with the highest probability as the final determination result. A probability threshold of 0.6 is set; errors below this threshold are considered unresolved errors, and the user is prompted to re-check via voice. To avoid misjudgments, the system re-collects speech data for secondary recognition. Model training utilizes a pre-labeled Japanese interpretation corpus, covering multiple scenarios and levels of Japanese expression errors. Cross-entropy loss is used as the loss function, and the AdamW optimizer is employed. Overfitting is suppressed by incorporating a weight decay mechanism. Training is completed until the error type identification accuracy on the validation set is ≥90%, ensuring the reliability of error identification. The corrected speech data generation module generates precise and scenario-based correction content based on the identified error type and converts it into speech for real-time interactive guidance. The correction text generation stage calls upon a multi-scenario Japanese correction rule library. This rule library stores standard expressions categorized by error type and scenario. Combined with the semantic core of the user's original text, targeted correction text is generated: honorific errors are replaced with standard honorific phrases for the corresponding scenario, while also annotating the applicable scenarios and collocation rules; particle errors are corrected by adjusting the particle type and word collocation to ensure logical coherence; disordered word order is reconstructed to conform to subject-object-verb logic, preserving the user's original semantics and expressive intent; mixed errors are corrected sequentially according to priority, with core errors affecting semantic understanding being corrected first.

[0065] The speech synthesis uses a Japanese TTS model based on WaveNet. This model can generate natural and fluent Japanese speech with a tone close to that of a real person. The sampling rate is kept at 16kHz, consistent with the acquisition process, to ensure clear sound quality and strong adaptability. At the same time, the speech parameters are adjusted according to the scenario. In daily conversation scenarios, the speech speed is moderate and the intonation is natural, which fits the spoken expression habits; in business scenarios, the speech speed is slightly slower and the intonation is formal to highlight professionalism; in academic scenarios, the speech speed is stable and the emphasis is clear, which is easy for users to understand and imitate.

[0066] I. Specialized Training Scenarios for Business Negotiation Interpretation

[0067] Target users: Professionals who need to frequently conduct Japanese business communication, such as foreign trade practitioners, Japanese business commissioners, cross-border e-commerce operators, etc.

[0068] Scenario application process: Users select the business negotiation scenario on the platform, which can be further subdivided into sub-scenarios such as contract negotiation, price negotiation, cooperation proposal, and complaint handling. The platform loads the standard dialogue corpus corresponding to the scenario (including Japanese business honorific norms, industry term libraries, and common negotiation sentence patterns).

[0069] Users simulate business communication through real-time voice input: For example, simulating introducing products to Japanese customers (This product has excellent energy-saving performance and competitive prices), negotiating delivery dates (Could you extend the delivery date by one week), etc. with interpretation expressions.

[0070] The system converts the voice into text through the trained ASR voice recognition model (adapted to the formal pronunciation of business scenarios and noise reduction processing), and inputs it into the multi-granularity feature extraction model after preprocessing:

[0071] Identify whether the use of honorifics conforms to the business scenario norms, whether the applicable scenarios of お願いします and ご考慮ください are confused, and whether there are misuses of honorifics and humble words. For example, mis-speaking 貴社の提案を検討させていただきます as 貴社の提案を検討します;

[0072] Extract the usage characteristics of auxiliary words, misuses of に / で / を. For example, mis-speaking 契約書に署名する as 契約書を署名する;

[0073] Compare with the standard syntactic tree of business scenarios to judge whether the word order conforms to the Japanese business expression logic, whether mis-speaking 弊社は来月に大阪支社を設立する予定です as 弊社は大阪支社を来月に設立する予定です.

[0074] The two-way GRU error judgment model outputs the specific error types, improper use of business honorifics and humble words, misuses of auxiliary words に / を. The system calls the business scenario correction rule library to generate corrected text, corrects the wrong expression to 弊社は来月、大阪支社を設立する予定です, generates natural and fluent business tone corrected speech through the Japanese TTS model, and broadcasts it to the user in real time.

[0075] For high-frequency errors, if there are multiple misuses of business honorifics, trigger the intensive training mechanism: Push 3-5 sets of similar business scenario dialogue simulation questions, require users to repeat the training until the error rate is lower than 5%; At the same time, generate a "Business Interpretation Ability Report", mark the short boards in dimensions such as honorific norms, auxiliary word usage, and word order logic, and recommend targeted learning materials, such as business honorific special manuals and negotiation sentence pattern templates.

[0076] Scene value: It helps professionals quickly master the core norms of business scene interpretation, avoid communication misunderstandings caused by language errors, disrespect caused by improper use of honorifics, and semantic deviations caused by incorrect use of auxiliary words, and improve the efficiency and professionalism of business communication.

[0077] II. Simulated training scenario for simultaneous interpretation at academic conferences

[0078] Target users: Japanese majors, candidates preparing for the CATTI Japanese interpretation certificate, academic researchers (who need to participate in simultaneous / interactive interpretation at international academic conferences).

[0079] Scene application process: Users select the academic conference scene and subdivide sub-scenes such as paper presentation, Q&A, keynote speech, and academic discussion. The platform loads the academic term libraries corresponding to the scenes, including Japanese terms in professional fields such as computer science, medicine, and engineering, and academic Japanese syntax norms, as well as the correct use of passive sentences and indirect citation sentences.

[0080] User simulates academic simultaneous interpretation: The platform plays original audio clips of academic conferences and recordings of research reports by Japanese scholars. Users conduct real-time interpretation retelling, such as interpreting the sentence "This research aims to improve the accuracy of correcting Japanese grammar errors by AI", and the system collects users' voices in real time.

[0081] Focused processing by the trained model: Precise recognition of academic honorifics, the applicable boundaries of "発表させていただきます" and "紹介します" in academic scenarios; the collocation of auxiliary words corresponding to professional terms, such as the incorrect expression "研究データに基づいて分析する" instead of "研究データを基づいて分析する"; the word order logic of academic sentences and the position of modifying components in long sentences, to avoid unclear conveyance of academic viewpoints due to chaotic word order.

[0082] After error determination, the system generates an academic interpretation correction plan: It not only provides corrected voices and texts but also marks the language norms unique to academic scenarios, such as the priority use principle of passive sentences in academic papers; aiming at the real-time requirement of simultaneous interpretation, it optimizes the correction feedback speed (delay ≤ 1 second) to adapt to the rhythm of simultaneous interpretation training.

[0083] The platform supports multiple rounds of simultaneous interpretation simulation. Users can repeatedly practice academic conference clips on different topics. The system records the changes in error types for each training and generates a simultaneous interpretation ability improvement curve to assist in exam preparation and strengthening of academic practical abilities.

[0084] Scene value: It fills the gap in special training tools for academic interpretation, helps users adapt to the language norms and real-time requirements of academic scenarios, and improves the accuracy and fluency of interpretation in professional fields.

[0085] Its beneficial effects lie in its adaptability to various training scenarios, including quiet indoor environments and noisy outdoor environments, through dynamic parameter adjustment and noise reduction. The ASR model, optimized for Japanese pronunciation, achieves an accuracy rate exceeding 95%, laying a high-quality data foundation for subsequent processing. It boasts high feature extraction accuracy, with multi-granularity models focusing on core dimensions such as honorifics, particles, and word order. Through the synergistic effect of LSTM-Attention, Transformer encoders, and syntactic parsing trees, it comprehensively captures and integrates multi-dimensional features, significantly improving the accuracy of error recognition. The error judgment model based on a bidirectional GRU network can accurately identify 12 specific error types, generating targeted corrective speech and text prompts. Combined with reinforcement training mechanisms and personalized reports, it forms a complete training loop. This reduces training costs, improves training efficiency, and caters to the personalized needs of users at different levels, making Japanese interpretation training more accurate and efficient.

[0086] Please see Figure 2 In the deep learning-based multi-scenario Japanese interpretation intelligent interactive training method, real-time voice data of users is acquired, and the real-time voice data is converted into text data using an ASR speech recognition model. The text data is then preprocessed to obtain preprocessed text data, which includes the following steps.

[0087] Step 201: Obtain the user's real-time voice data, train and optimize the Japanese based on the Conformer model to obtain the ASR speech recognition model, and use the ASR speech recognition model to recognize the real-time voice data to obtain text data.

[0088] Step 202: Remove meaningless interjections, duplicate characters, and garbled characters with recognition errors from the text data, and correct the mixing of Japanese kanji and kana;

[0089] Step 203: Optimize word segmentation rules using Japanese word segmentation tools in the context of interpretation, split compound words and honorific words, and mark their parts of speech;

[0090] Step 204: Convert colloquial expressions into standard written language, and convert the segmented text into a fixed-length vector sequence to obtain preprocessed text data.

[0091] The above describes embodiments of the deep learning-based multi-scenario Japanese interpretation intelligent interactive training method of the present invention. Please refer to [link / reference]. Figure 3 In the deep learning-based multi-scenario Japanese interpreting intelligent interactive training system, the system includes the following modules:

[0092] The Japanese data acquisition module is used to acquire the user's real-time voice data, convert the real-time voice data into text data using the ASR speech recognition model, and preprocess the text data to obtain preprocessed text data.

[0093] The recognition model building module is used to build a multi-granular Japanese language feature extraction model. It identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using the Transformer encoder, and compares word order scene differences features through syntactic analysis tree.

[0094] The language feature extraction module is used to input preprocessed text data into the multi-granularity Japanese language feature extraction model to obtain Japanese language features;

[0095] The intelligent interactive training module is used to input Japanese language features into the RNN neural network for judgment, obtain the Japanese error type, and generate Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interactive training for users.

[0096] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and equivalents.

Claims

1. A multi-scenario Japanese interpreting intelligent interactive training method based on deep learning, characterized in that, The multi-scenario Japanese interpretation intelligent interactive training method includes the following steps: The user's real-time voice data is acquired, and the real-time voice data is converted into text data using an ASR speech recognition model. The text data is then preprocessed to obtain preprocessed text data. A multi-granular Japanese language feature extraction model was established. The honorific sequence features were identified based on the LSTM-Attention model, the particle sequence features were extracted using the Transformer encoder, and the word order scene difference features were compared through syntactic analysis tree. The preprocessed text data is input into the multi-granularity Japanese language feature extraction model to obtain Japanese language features; The Japanese language features are input into an RNN neural network for judgment to obtain the Japanese error type. Based on the Japanese error type, Japanese corrected speech data is generated to train users for multi-scenario intelligent interaction.

2. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The process of acquiring the user's real-time voice data, converting the real-time voice data into text data using an ASR speech recognition model, and preprocessing the text data to obtain preprocessed text data includes: The system acquires real-time voice data from users, trains and optimizes Japanese based on the Conformer model to obtain an ASR speech recognition model, and uses the ASR speech recognition model to recognize the real-time voice data to obtain text data. Remove meaningless interjections, duplicate characters, and garbled characters with recognition errors from the text data; correct the mixed use of Japanese kanji and kana. By using Japanese word segmentation tools in conjunction with interpretation scenarios, we can optimize word segmentation rules, break down compound words and honorific words, and mark their parts of speech. The process involves converting colloquial expressions into standard written language and transforming the segmented text into a fixed-length vector sequence to obtain preprocessed text data.

3. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The aforementioned multi-granularity Japanese language feature extraction model identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using a Transformer encoder, and compares word order scene differences using a syntactic analysis tree, including: A Bi-LSTM bidirectional long short-term memory network is used. The input layer is the preprocessed word embedding vector, the hidden layer dimension is set to 256, the number of layers is 2, and a dropout layer is added to prevent overfitting. An Attention mechanism is introduced into the LSTM output layer to calculate the weight of each token for honorific recognition, focusing on honorific prefixes, honorific verbs, and honorific sentence structures to generate a weighted honorific feature vector.

4. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The aforementioned multi-granularity Japanese language feature extraction model identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using a Transformer encoder, and compares word order scene differences using a syntactic analysis tree, including: The Transformer encoder has 6 coding blocks, each containing a multi-head self-attention mechanism and a feedforward neural network. It uses LayerNorm normalization and residual connections to alleviate the gradient vanishing problem. Given that Japanese particles have no real meaning but determine the logic of a sentence, a self-attention mechanism is used to calculate the association weights between particles and the subject, object, and predicate, filter out irrelevant words, and extract dimensional particle feature vectors.

5. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The aforementioned multi-granularity Japanese language feature extraction model identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using a Transformer encoder, and compares word order scene differences using a syntactic analysis tree, including: The Japanese constant syntactic analysis model is used to parse the syntactic structure of the preprocessed text, generate a syntactic analysis tree containing phrase types and grammatical relations, and mark the syntactic position of each word; Construct a multi-scenario standard Japanese syntax tree library, compare the user syntax tree with the standard syntax tree of the same scenario, calculate the tree edit distance, extract word order misalignment features, and generate word order scenario difference feature vectors; We use a weighted concatenation method to fuse honorific feature vectors, particle feature vectors, and word order scene difference feature vectors into Japanese language features.

6. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The process of inputting the Japanese language features into an RNN neural network for judgment to obtain the Japanese error type, and generating Japanese corrected speech data based on the Japanese error type for multi-scenario intelligent interaction training for users includes: A bidirectional GRU network is used, with a 1024-dimensional multi-granularity feature vector as the input layer, a 512-dimensional hidden layer, and an error type label as the output layer. Japanese language features are input into a bidirectional GRU network. The correlation logic between features is captured through temporal modeling. The probability of each error type is output through the softmax function, and the type with the highest probability is used as the judgment result.

7. The multi-scenario Japanese interpretation intelligent interactive training method based on deep learning as described in claim 1, characterized in that, The process of inputting the Japanese language features into an RNN neural network for judgment to obtain the Japanese error type, and generating Japanese corrected speech data based on the Japanese error type for multi-scenario intelligent interaction training for users includes: The system calls upon a multi-scenario Japanese language correction rule library, combines it with the user's original text to generate correction text, and then uses a Japanese TTS model to convert the correction text into Japanese correction speech data. The Japanese corrected speech data is broadcast to the user in real time through an audio player, and a multi-scenario reinforcement training mechanism is triggered for high-frequency error types.

8. A multi-scenario Japanese interpreting intelligent interactive training system based on deep learning, characterized in that: The multi-scenario Japanese interpretation intelligent interactive training system includes the following modules: The Japanese data acquisition module is used to acquire the user's real-time voice data, convert the real-time voice data into text data using an ASR speech recognition model, and preprocess the text data to obtain preprocessed text data. The recognition model building module is used to build a multi-granular Japanese language feature extraction model. It identifies honorific sequence features based on the LSTM-Attention model, extracts particle sequence features using the Transformer encoder, and compares word order scene differences features through syntactic analysis tree. The language feature extraction module is used to input the preprocessed text data into the multi-granularity Japanese language feature extraction model to obtain Japanese language features. The intelligent interactive training module is used to input the Japanese language features into the RNN neural network for judgment, obtain the Japanese error type, and generate Japanese corrected speech data based on the Japanese error type to conduct multi-scenario intelligent interactive training for users.

9. The deep learning-based multi-scenario Japanese interpretation intelligent interactive training system as described in claim 8, characterized in that, The language feature extraction module includes the following sub-modules: A submodule is established to use a bidirectional GRU network, with an input layer of 1024-dimensional multi-granularity feature vectors, a hidden layer dimension of 512, and an output layer of error type labels. The judgment submodule is used to input Japanese language features into a bidirectional GRU network, capture the correlation logic between features through temporal modeling, output the probability of each error type through the softmax function, and take the type with the highest probability as the judgment result.

10. The deep learning-based multi-scenario Japanese interpretation intelligent interactive training system as described in claim 8, characterized in that, The language feature extraction module includes the following sub-modules: The correction submodule is used to call the multi-scenario Japanese correction rule library, combine it with the user's original text to generate correction text, and use a Japanese TTS model to convert the correction text into Japanese correction speech data; The training submodule is used to broadcast the Japanese corrected speech data to the user in real time through an audio player, and to trigger a multi-scenario reinforcement training mechanism for high-frequency error types.

Images