Data interaction method, device and equipment of portable translation equipment and storage medium
By constructing a language database and using word segmentation and classification algorithms to process recording data, and combining with the Transformer model for translation, the problem of inaccurate translation of large language models is solved, and a more accurate and smooth translation effect is achieved.
Patent Information
- Application Number
- CN202510145573.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-10
- Publication Date
- 2025-03-25
AI Technical Summary
In the prior art, large language models are inaccurate in translation, and their adaptability and translation depth are limited.
By constructing a language database, including standard pronunciation of official common languages in all countries, the recording data is processed in combination with word segmentation and classification algorithms, the language is determined, and the Transformer model is used for translation.
It achieves more accurate and smoother translation results, smooth processing flow, and response time is between milliseconds and seconds.
Smart Images
Figure CN119692367A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of big data resource service technology, and in particular to a data interaction method, apparatus, device and storage medium for a portable translation device. Background Art
[0002] As international communication continues, cross-language communication is becoming more and more important in various fields such as industry, commerce, technology, and culture. For multinational companies and international cooperation projects, face-to-face language translation technology can greatly improve communication efficiency, work efficiency, etc., saving a lot of time and labor costs.
[0003] At present, in order to improve real-time communication between multiple languages, portable devices with real-time translation are usually used. Most portable devices with real-time translation usually use a machine translation large language model to activate the translation ability of the model by providing examples to the basic large language model. This large language model provides a translation method that does not require training and can be quickly deployed according to specific translation needs (such as translation in a specific field). However, this method is constrained by fixed parameters, which limits its adaptability and translation depth. Summary of the invention
[0004] The main purpose of the present application is to provide a data interaction method, device, equipment and storage medium for a portable translation device to solve the problem of inaccurate translation of large language models in the prior art.
[0005] In order to achieve the above objectives, this application provides the following technical solutions:
[0006] A data interaction method for a portable translation device, the data interaction method being applied to a person wearing the portable translation device, the data interaction method comprising:
[0007] Step S1, constructing a language database according to a preset strategy, wherein the language database includes standard pronunciations of all official common languages of the country;
[0008] Step S2, in response to a voice trigger signal of the portable translation device, continuously recording the pronunciation of a person based on the voice trigger signal to form recording data;
[0009] Step S3, performing word segmentation processing on the recorded data by using a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data;
[0010] Step S4, classifying each word segmentation sub-data in the to-be-processed data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm;
[0011] Step S5, based on the same data set to be processed, obtaining the official common language corresponding to all the word segmentation sub-data, and defining the official common language with the largest number of occurrences as the language used in the recording data;
[0012] Step S6, constructing a Transformer model and translating the recorded data based on the language used to obtain a standard pronunciation based on the target language;
[0013] Step S7: outputting the standard pronunciation based on the target language to the portable translation device for playback.
[0014] As a further improvement of the present application, step S2, in response to the voice trigger signal of the portable translation device, continuously recording the pronunciation of the person based on the voice trigger signal to form recording data, includes:
[0015] Step S21, obtaining several historical recording data of the same user;
[0016] Step S22, performing word segmentation processing on each historical recording data by using a word segmentation algorithm to obtain a to-be-processed historical data set having a plurality of historical word segmentation sub-data;
[0017] Step S23, dividing the historical data set to be processed into a training set and a sample set according to a preset ratio;
[0018] Step S24, inputting the training set into a preset neural network model and training it to obtain a language habit prediction model;
[0019] Step S25, inputting the sample set into the language habit prediction model to obtain a language habit prediction value;
[0020] Step S26, defining the language habit prediction value as the recording data of the same user.
[0021] As a further improvement of the present application, in step S24, the training set is input into a preset neural network model and trained to obtain a language habit prediction model, including:
[0022] Step S241, defining a topological relationship of the preset neural network model, wherein the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected;
[0023] Step S242, outputting the training set to the input layer, training the training set by the least squares estimation method, and obtaining a root mean square error between the sample set and the current training result based on each training;
[0024] Step S243, obtaining the minimum value of the root mean square error between the sample set and all training results;
[0025] Step S244, obtaining a training result corresponding to the minimum value as the language habit prediction model.
[0026] As a further improvement of the present application, in step S3, the recorded data is segmented by a segmentation algorithm to obtain a data set to be processed having a plurality of segmentation sub-data, including:
[0027] Step S31, performing word segmentation processing on the recorded data by using SentencePiece software to find the word segmentation points of the recorded data;
[0028] Step S32, dividing the recorded data into a plurality of segmented words based on all segmentation points;
[0029] Step S33, marking each word segmentation by decoding the time tag, and obtaining a word segmentation sub-data based on one tag.
[0030] As a further improvement of the present application, step S4, classifying each word segmentation sub-data in the to-be-processed data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm, includes:
[0031] Step S41, define the data set to be processed X = {x1, x2, ..., x j ,…,x m}, x j is the jth word segmentation sub-data, and m is the number of all word segmentation sub-data;
[0032] Step S42, defining a category set C = {y1, y2, ..., y k ,…,y n}, where y k is the kth official common language in the category set C, and n is the number of official common languages of all countries;
[0033] Step S43, calculate the conditional probability of the j-th word segmentation sub-data in each official common language according to formula (1):
[0034]
[0035] Among them, P(X|y k ) is the conditional probability of the kth official common language based on the dataset X to be processed; P(y k ) is the marginal probability of the kth official common language; P(x j |y k ) is the conditional probability of the jth word segmentation sub-data under the kth official common language;
[0036] Step S44: classify each word segmentation sub-data into the official common language with the highest conditional probability.
[0037] As a further improvement of the present application, step S6, constructing a Transformer model and translating the recording data based on the language used to obtain a standard pronunciation based on the target language, includes:
[0038] Step S61, respectively obtaining a representation vector of each word segmentation sub-data;
[0039] Step S62, constructing a word representation vector matrix including all identification vectors based on the same data set to be processed;
[0040] Step S63, inputting the word representation vector matrix into the Encoder block of the Transformer model, and obtaining the encoding information matrix of the same data set to be processed after passing through at least six Encoder blocks;
[0041] Step S64, inputting the encoding information matrix into the Decoder block of the Transformer model;
[0042] Step S65, searching for the target language word segment corresponding to each coded information in the coded information matrix in sequence through at least six Decoder blocks;
[0043] Step S66, sequentially combining all target language segmentations to obtain a standard pronunciation based on the target language.
[0044] As a further improvement of the present application, in step S7, the standard pronunciation based on the target language is output to the portable translation device for playback, and then the following steps are included:
[0045] Step S8, storing the recording data and the standard pronunciation based on the target language in the portable translation device and the cloud.
[0046] In order to achieve the above objectives, this application also provides the following technical solutions:
[0047] A data interaction device for a portable translation device, the data interaction device is applied to the data interaction method for the portable translation device as described above, the data interaction device comprises:
[0048] A language database construction module, used to construct a language database according to a preset strategy, wherein the language database includes standard pronunciations of all official common languages of the country;
[0049] A recording data acquisition module, configured to respond to a voice trigger signal of the portable translation device and continuously record a person's pronunciation based on the voice trigger signal to form recording data;
[0050] The module for acquiring the data set to be processed is used to perform word segmentation processing on the recording data through a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data;
[0051] A classification module for the data set to be processed, used to classify each word segmentation sub-data in the data set to be processed into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm;
[0052] A recording data language definition module, used to obtain the official common language corresponding to all word segmentation sub-data based on the same data set to be processed, and define the official common language with the most occurrences as the language used in the recording data;
[0053] A target language standard pronunciation acquisition module is used to construct a Transformer model and translate the recording data based on the language used to obtain a standard pronunciation based on the target language;
[0054] The target language standard pronunciation sending module is used to output the standard pronunciation based on the target language to the portable translation device for playback.
[0055] In order to achieve the above objectives, this application also provides the following technical solutions:
[0056] An electronic device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, a data interaction method for a portable translation device as described above is implemented.
[0057] In order to achieve the above objectives, this application also provides the following technical solutions:
[0058] A storage medium stores program instructions, and when the program instructions are executed by a processor, the data interaction method of a portable translation device as described above can be implemented.
[0059] The present application constructs a language database according to a preset strategy, and the language database includes the standard pronunciations of all official common languages of all countries; in response to a voice trigger signal of a portable translation device, the pronunciation of a person is continuously recorded based on the voice trigger signal to form recording data; the recording data is segmented by a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data; each word segmentation sub-data in the data set to be processed is classified into the standard pronunciation with the highest probability of overlap of the respective voiceprints by a classification algorithm; based on the same data set to be processed, the official common languages corresponding to all word segmentation sub-data are obtained, and the official common language with the most occurrences is defined as the language used in the recording data; a Transformer model is constructed and the recording data is translated based on the language used to obtain the standard pronunciation based on the target language; the standard pronunciation based on the target language is output to the portable translation device for playback. This application uses a classification algorithm and a word segmentation algorithm to collaboratively perform real-time recognition and real-time processing on the original recording data obtained by sound pickup, and then translates the processed audio data through a Transformer model. Compared with traditional large language models, the translation content of this application is more accurate and the context is smoother. In addition, the processing flow of this application is relatively smooth and not cumbersome, and can usually respond in milliseconds to seconds. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 A schematic diagram of a step flow chart of an embodiment of a data interaction method for a portable translation device of the present application;
[0061] Figure 2 This is a structural schematic diagram of an embodiment of a data interaction device of a portable translation device of the present application;
[0062] Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application;
[0063] Figure 4 This is a schematic diagram of the structure of an embodiment of the storage medium of the present application. DETAILED DESCRIPTION
[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0065] The terms "first", "second" and "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of the present application (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.
[0066] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0067] like Figure 1 As shown, this embodiment provides an embodiment of a data interaction method for a portable translation device. In this embodiment, the data interaction method is applied to a person wearing a portable translation device.
[0068] Preferably, the portable translation device can be worn by the speaker as well as the listener.
[0069] Specifically, the data interaction method includes the following steps:
[0070] Step S1, constructing a language database according to a preset strategy, the language database includes the standard pronunciations of all official common languages of the country.
[0071] Preferably, the preset strategy is to download the general data set and the language data set from the WMT, CCMT and UM-Corpus websites as the language database of this embodiment, and the aforementioned data sets can be directly obtained without difficulty.
[0072] Step S2, in response to the voice trigger signal of the portable translation device, continuously recording the pronunciation of the person based on the voice trigger signal to form recording data.
[0073] Step S3, performing word segmentation processing on the recorded data by using a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data.
[0074] Preferably, the word segmentation algorithm is the basic technology in natural language processing. Its task is to segment a continuous text sequence into meaningful word units, that is, vocabulary units. Common word segmentation algorithms are as follows:
[0075] ① Rule-based word segmentation: word segmentation is performed through a predefined rule library, such as regular expressions, dictionary matching, etc., which is suitable for some highly structured languages.
[0076] ② Statistical word segmentation: Utilize the learning of large-scale corpus, such as Hidden Markov Model (HMM) and Conditional Random Field (CRF), to segment words according to contextual probability.
[0077] ③ Word segmentation based on word frequency statistics: According to the word frequency table, words with higher frequency are given priority for segmentation, which is suitable for simple texts.
[0078] ④ Deep learning word segmentation: Use neural network models, such as recurrent neural network (RNN), long short-term memory network (LSTM) or Transformer, to perform word segmentation through end-to-end learning.
[0079] ⑤ Adaptive Dynamic Vocabulary (ADV): It combines dictionary and statistical information to handle both common words and new words.
[0080] Step S4, classifying each word segmentation sub-data in the data set to be processed into the standard pronunciation with the highest probability of overlap with the respective voiceprints through a classification algorithm.
[0081] Preferably, the classification algorithm of this embodiment is preferably Bayesian classification.
[0082] Specifically, Bayesian classification is a general term for a class of classification algorithms. These algorithms are all based on the Bayesian theorem, so they are collectively called Bayesian classification. The Bayesian theorem is that let the sample space of the experiment E be S, A be the event of E, B1, B2,…, B n is a partition of S, and P(A)>0,P(B i )>0,i=1,2,…,n, then Bayes’ theorem is expressed as follows:
[0083]
[0084] It should be noted that Bayes' theorem is a well-known theorem, and its symbolic meaning is not repeated here, and the symbolic meaning of the theorem is not interchangeable with the symbolic meanings of other formulas in this embodiment.
[0085] Step S5, based on the same data set to be processed, the official common language corresponding to all the word segmentation sub-data is obtained, and the official common language with the largest number of occurrences is defined as the language used in the recording data.
[0086] Step S6, constructing a Transformer model and translating the recorded data based on the language used to obtain a standard pronunciation based on the target language.
[0087] Preferably, the transformer model is proposed in the AttentionIsAllYouNeed paper, and has achieved very good results in multiple NLP tasks. It can be said that the current development of NLP cannot be separated from the transformer. The biggest feature is that it abandons the traditional CNN and RNN, and the entire network structure is completely composed of the Attention mechanism. Due to its excellent performance and friendliness to downstream tasks, or the downstream tasks can only be fine-tuned to get good results, people in the field of computer vision have been trying to introduce transformers. Recently, there have been some attempts with good results, such as detr and deformable detr in the field of object detection, vision transformer in the field of classification, etc. This article starts from the transformer structure and analyzes it in combination with the transformer achievements in vision.
[0088] Preferably, the Transformer is an Encoder-Decoder architecture, so the middle Transformer can be divided into two parts: the encoding component and the decoding component. The encoding component consists of multiple layers of encoders (Encoders), usually 6 layers of encoders, and other layers can be tried in actual use. The decoding component is also composed of decoders (Decoders) with the same number of layers.
[0089] Step S7: outputting the standard pronunciation based on the target language to the portable translation device for playback.
[0090] Further, step S2, in response to the voice trigger signal of the portable translation device, continuously recording the pronunciation of the person based on the voice trigger signal to form recording data, includes:
[0091] Step S21, obtaining several historical recording data of the same user.
[0092] Step S22, performing word segmentation processing on each historical recording data by using a word segmentation algorithm to obtain a to-be-processed historical data set having a plurality of historical word segmentation sub-data.
[0093] Step S23, dividing the historical data set to be processed into a training set and a sample set according to a preset ratio.
[0094] Preferably, the preset ratio can be set to 8:2, so as to divide the image data into a training set and a sample set in a ratio of 8:2.
[0095] Preferably, in actual application, a certain proportion of validation sets are required to verify the accuracy of the model, that is, verification is required after training is completed, and the sample set prediction is started only after the verification is successful. Usually, the image data is divided into training set, validation set, and sample set in a ratio of 70%:15%:15%, that is, 70% of the data is the training set, 15% of the data is the validation set, and 15% of the data is the sample set.
[0096] Step S24, inputting the training set into a preset neural network model and performing training to obtain a language habit prediction model.
[0097] Step S25, inputting the sample set into the language habit prediction model to obtain a language habit prediction value.
[0098] Step S26, defining the language habit prediction value as the recording data of the same user.
[0099] Furthermore, in step S24, the training set is input into a preset neural network model and trained to obtain a language habit prediction model, including:
[0100] Step S241, defining the topological relationship of the preset neural network model, the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected.
[0101] Step S242, output the training set to the input layer, train the training set by the least squares estimation method, and obtain the root mean square error between the sample set and the current training result based on each training.
[0102] Preferably, the least squares estimation method is a nonlinear least squares method, which is used to solve nonlinear optimization problems, such as the neural network model in this embodiment. The method uses the gradient information of the objective function to adjust the parameters in each iteration to find the best model parameter estimate. The method uses a so-called adjustment factor, which balances the gradient descent and Gauss-Newton method according to the current iteration step. In each iteration, the method calculates a parameter increment and then decides whether to accept the parameter increment by comparing the size of the current model residual and the model residual estimated using the new parameter.
[0103] Preferably, the iterative step size of the least squares estimation method is given by Representation.
[0104] Among them, h L-M is the iterative step length of the least squares estimation method, r is the residual of the current model, Jr is the Jacobian matrix of the current model residual, is the transposed matrix of the Jacobian matrix of the current model residual, μ is the above adjustment factor, I is the unit matrix, is the covariance matrix.
[0105] Preferably, the order of the identity matrix is the same as the order of the Jacobian matrix.
[0106] Preferably, when μ>0, is a positive number, tends to the Gauss-Newton method; when μ>>0, Tends to balance gradient descent.
[0107] Step S243, obtaining the minimum value of the root mean square error between the sample set and all training results.
[0108] Step S244, obtaining the training result corresponding to the minimum value as a language habit prediction model.
[0109] Furthermore, the neural network model is represented by the following formula:
[0110]
[0111] Among them, y is the neural network model; x n is the nth input node of the input layer, each input node corresponds to a set of training data in the training set, is the preset weight from the mth input node of the input layer to the nth input node of the hidden layer; is the threshold of the nth input node connected to the hidden layer; tansig(x) is the transfer function, and
[0112] Preferably, the numbers in the brackets of the symbols in the above formula are the number of layers, for example, the superscript (1) is the first layer, that is, the input layer, and the superscript (1, 2) is from the first layer to the second layer, that is, the input layer to the hidden layer.
[0113] Preferably, the training model training a neural network usually requires providing a large amount of data, namely a data set; the data set is generally divided into three categories, namely the above-mentioned training set (training set), validation set (validation set) and test set (test set).
[0114] Among them, one epoch is the process of training once with all the samples in the training set. The so-called training once refers to one forward pass and one back pass. When the number of samples in an epoch (i.e., training set) is too large, training once may consume too much time, and it is not necessary to use all the data in the training set for each training. In this case, the entire training set needs to be divided into multiple small blocks, that is, divided into multiple batches for training. An epoch consists of one or more batches, where a batch is a part of the training set. Each training process only uses a part of the data, i.e., a batch. The process of training a batch is an iteration.
[0115] Preferably, the neural network training specifically includes a perceptron, which is composed of two layers of neurons. The input layer receives external input signals and transmits them to the output layer. The output layer is MP neurons, and the step function is y j =f(∑ i w i ·x i -θ i ), x i is the nth input node.
[0116] Preferably, given a training data set, the weight w i (i=1,2,...,n) and training bias θ i It can be obtained through learning, θ i It can be understood as a weight w corresponding to a fixed value of -1,0. i+1 .
[0117] It should be noted that the step function here has no interchangeable symbolic meaning with other formulas in the embodiment. This step function is only used for principle explanation and does not participate in the calculation of other formulas.
[0118] Preferably, in this embodiment, the number of neural network training times can be set to 1000 times.
[0119] Preferably, the learning rate from the 1st to the 500th epoch can be set to 0.01, the learning rate from the 501st to the 750th epoch can be set to 0.001, and the learning rate from the 751st to the 1000th epoch can be set to 0.0001.
[0120] It can be understood that the neural network training of this embodiment mainly includes the following ideas:
[0121] ① Initialize the weights and bias items in the network.
[0122] Initializing parameter values (output unit weights, bias terms and hidden unit weights, bias terms are all model parameters) is to activate forward propagation, obtain the output value of each layer element, and then obtain the value of the loss function.
[0123] ②Activate forward propagation to obtain the output value of each layer and the expected value of the loss function of each layer.
[0124] ③According to the loss function, calculate the error term of the output unit and the error term of the hidden unit.
[0125] Calculate various errors, calculate the gradient of the parameters with respect to the loss function, or calculate partial derivatives according to the chain rule of calculus. For partial derivatives of vectors or matrices in a composite function, the partial derivative of the function inside the composite function is always multiplied on the left; for partial derivatives of scalars in a composite function, the partial derivative of the function inside the composite function can be multiplied on the left or on the right.
[0126] ④Update the weights and bias items in the neural network.
[0127] ⑤ Repeat ② to ④ until the loss function is less than the preset bias or the number of iterations is used up, and the parameters output at this time are the current optimal parameters.
[0128] Preferably, the root mean square error is characterized by the following formula:
[0129]
[0130] Among them, RMSE is the root mean square error, n is the number of training data groups, x i is the true value of the i-th group of training data, is the training result after the i-th group of training data is trained.
[0131] Furthermore, in step S3, the recorded data is segmented by a segmentation algorithm to obtain a data set to be processed having a plurality of segmentation sub-data, including:
[0132] Step S31, performing word segmentation processing on the recorded data by using SentencePiece software to find the word segmentation points of the recorded data.
[0133] Preferably, in addition to the SentencePiece software, Stanford NLP and moses tokenizer tools can also be used to perform word segmentation and byte pair processing operations on the data.
[0134] Step S32, dividing the recorded data into a plurality of segmented words based on all segmentation points.
[0135] Step S33, marking each word segmentation by decoding the time tag, and obtaining a word segmentation sub-data based on one tag.
[0136] Specifically, the decoding time tag is a DTS tag, and PAD, UNK, and EOS tags may also be used.
[0137] Furthermore, step S4, classifying each word segmentation sub-data in the to-be-processed data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm, includes:
[0138] Step S41, define the data set to be processed X = {x1, x2, ..., x j ,…,x m}, x j is the jth word segmentation sub-data, and m is the number of all word segmentation sub-data.
[0139] Step S42, define a category set C = {y1, y2, ..., y k ,…,y n}, where y k is the kth official common language in category set C, and n is the number of official common languages of all countries.
[0140] Step S43, calculate the conditional probability of the j-th word segmentation sub-data in each official common language according to formula (1):
[0141]
[0142] Among them, P(X|y k ) is the conditional probability of the kth official common language based on the dataset X to be processed; P(y k ) is the marginal probability of the kth official common language; P(x j |y k ) is the conditional probability of the jth word segmentation sub-data under the kth official common language.
[0143] Step S44: classify each word segmentation sub-data into the official common language with the highest conditional probability.
[0144] Preferably, the naive Bayes classification assumes that the existence of a specific feature in a class is independent of the existence of any other feature, that is, each feature is independent of each other. Therefore, there are some constraints on the actual situation. If there is a correlation between attributes, the classification accuracy will be reduced, but in actual application, the classification effect of naive Bayes is relatively accurate. Naive Bayes solves the probability of each category appearing under the condition that the item appears for a given item to be classified. The largest probability is considered to be the category to which the item to be classified belongs.
[0145] Specifically, Naive Bayes is defined as follows:
[0146] ① Let x = {a1, a2, a3, ..., a n} is an item to be classified, and each a is a feature of x.
[0147] ②There is a category set c = {y1, y2, y3, ..., y m}.
[0148] ③ Calculate P(y1|x), P(y2|x), ..., P(y m |x).
[0149] ④If P(y k |x)=max{P(y1|x),P(y2|x),...,P(y m |x)}, then x∈y k .
[0150] Then calculate the conditional probabilities in step ③ through the following steps:
[0151] Find a set of items to be classified with known classifications. This set is called a training sample set.
[0152] The conditional probability estimates of each feature attribute in each category are obtained by statistics. That is:
[0153] P(a1|y1),P(a2|y1),……,P(a n |y1)
[0154] P(a1|y2),P(a2|y2),……,P(a n |y2);
[0155] …
[0156] P(a1|y m ),P(a2|y m ),……,P(a n |y m );
[0157] Assuming that each feature attribute is conditionally independent, according to the Bayesian principle:
[0158] P(y i |x)=P(x|y i )P(y i ) / P(x).
[0159] Since the denominator is a constant for all categories, we only need to maximize the numerator. And because each feature attribute is conditionally independent, then:
[0160] P(x|y i )P(y i )=P(a1|yi )P(a2|y i )……P(a n |y i )P(y i ).
[0161] It should be noted that the above preferred contents are also explanations of the principles, and the meanings of their symbols are not interchangeable with the meanings of the symbols of other formulas in this embodiment.
[0162] Furthermore, in step S6, a Transformer model is constructed and the recording data is translated based on the language used to obtain a standard pronunciation based on the target language, including:
[0163] Step S61, respectively obtain the representation vector of each word segmentation sub-data.
[0164] Step S62, constructing a word representation vector matrix containing all identification vectors based on the same data set to be processed.
[0165] Step S63, input the word representation vector matrix into the Encoder block of the Transformer model, and obtain the encoding information matrix of the same data set to be processed after passing through at least six Encoder blocks.
[0166] Step S64, input the encoding information matrix into the Decoder block of the Transformer model.
[0167] Step S65, using at least six Decoder blocks to sequentially search for the target language word segment corresponding to each coded information in the coded information matrix.
[0168] Step S66, sequentially combining all target language segmentations to obtain a standard pronunciation based on the target language.
[0169] Preferably, since Transformer is essentially an Encoder-Decoder architecture, the middle Transformer can be divided into two parts: an encoding component and a decoding component.
[0170] Each encoder consists of two sublayers: the Self-Attention layer and the Position-wise Feed Forward Network (FFN). The structure of each encoder is the same, but they use different weight parameters. The input of the encoder will first flow into the Self-Attention layer. It allows the encoder to use the information of other words in the input sentence when encoding a specific word (it can be understood as: when we translate a word, we not only pay attention to the current word, but also pay attention to the information of other words). We will introduce the internal structure of Self-Attention in detail later. Then, the output of the Self-Attention layer will flow into the feedforward network. The decoder also has these two layers in the encoder, but there is an attention layer between them (i.e. Encoder-DecoderAttention), which is used to help the decoder focus on the relevant parts of the input sentence (similar to the attention in the seq2seq model).
[0171] Preferably, as with normal NLP tasks, each word is first converted to a word vector using a word embedding algorithm. In the Transformer model, the dimension of the word embedding vector is 512. Each word is embedded into a vector of size 512. Embedding only occurs in the bottom-level encoder, and all encoders receive a list of vectors of size 512. The bottom encoder receives the word embedding vector, and the other encoders receive the output of the previous encoder. This list size is a configurable hyperparameter - basically this parameter is the length of the longest sentence in the training dataset. After the embedding operation is completed on the input sequence, each word flows through two layers of the encoder.
[0172] Preferably, the encoder receives a vector as input. The encoder first passes these vectors to the Self-Attention layer, then to the feed-forward network, and finally passes the output to the next encoder.
[0173] Specifically, the workflow of the Transformer model is as follows:
[0174] Step 1: Get the representation vector X of each word in the input sentence. X is obtained by adding the embedding of the word (Embedding is the feature extracted from the original data) and the embedding of the word position.
[0175] Step 2: Pass the obtained word representation vector matrix (each row is the representation x of a word) into the Encoder. After 6 Encoder blocks, the encoding information matrix C of all words in the sentence can be obtained. The word vector matrix is represented by , n is the number of words in the sentence, and d is the dimension of the representation vector (d = 512). The dimension of the matrix output by each Encoder block is exactly the same as the input.
[0176] Step 3: Pass the encoded information matrix C output by the encoder to the decoder, and the decoder will translate the next word i+1 according to the currently translated words 1 to i. During use, when translating to word i+1, the word after i+1 needs to be covered by the Mask operation.
[0177] Further, in step S7, the standard pronunciation based on the target language is output to the portable translation device for playback, and then, the following steps are included:
[0178] Step S8, storing the recording data and the standard pronunciation based on the target language in the portable translation device and the cloud.
[0179] For example, this embodiment provides a practical application example with simple implementation difficulty, which is mainly as follows:
[0180] ①Convert speech into text through openAI's whisper-base package.
[0181] ② After whisper converts the speech into text, the text is input into the gemma2:2b model through the ONNX interface to complete the translation operation. The advantage of the gemma2:2b model is that the process can run stably on mobile devices.
[0182] ③ After the gemma2:2b model completes the translation operation, the translated text can be converted into standard speech for subsequent output through the TextToSpeech model (such as chatTTS), or the TTS API to speech tool on the mobile device can be used directly.
[0183] It is worth noting that the mobile device can be installed on a safety helmet to form an integrated protective gear with protection and language translation.
[0184] This embodiment constructs a language database according to a preset strategy, and the language database includes the standard pronunciations of all official common languages of the country; responds to the voice trigger signal of the portable translation device, continuously records the pronunciation of the person based on the voice trigger signal to form recording data; performs word segmentation processing on the recording data through a word segmentation algorithm to obtain a data set to be processed with a plurality of word segmentation sub-data; classifies each word segmentation sub-data in the data set to be processed into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm; obtains the official common language corresponding to all the word segmentation sub-data based on the same data set to be processed, and defines the official common language with the most occurrences as the language used in the recording data; constructs a Transformer model and translates the recording data based on the language used to obtain the standard pronunciation based on the target language; and outputs the standard pronunciation based on the target language to the portable translation device for playback. This embodiment uses a classification algorithm and a word segmentation algorithm to collaboratively perform real-time recognition and real-time processing on the original recording data obtained by sound pickup, and then translates the processed audio data through a Transformer model. Compared with the traditional large language model, the translation content of this embodiment is more accurate and the context is smoother. In addition, the processing flow of this embodiment is smoother and less cumbersome, and can usually respond in milliseconds to seconds.
[0185] like Figure 2 As shown, this embodiment provides an embodiment of a data interaction device for a portable translation device. In this embodiment, the data interaction device is applied to the data interaction method in the above embodiment. The data interaction device includes a language database construction module 1, a recording data acquisition module 2, a to-be-processed data set acquisition module 3, a to-be-processed data set classification module 4, a recording data language definition module 5, a target language standard pronunciation acquisition module 6, and a target language standard pronunciation sending module 7, which are electrically connected in sequence.
[0186] Among them, the language database construction module 1 is used to construct a language database according to a preset strategy, and the language database includes the standard pronunciations of all official common languages of the country; the recording data acquisition module 2 is used to respond to the voice trigger signal of the portable translation device, and continuously record the pronunciation of the person based on the voice trigger signal to form recording data; the processing data set acquisition module 3 is used to perform word segmentation processing on the recording data through a word segmentation algorithm to obtain a processing data set with a plurality of word segmentation sub-data; the processing data set classification module 4 is used to classify each word segmentation sub-data in the processing data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm; the recording data language definition module 5 is used to obtain the official common language corresponding to all word segmentation sub-data based on the same processing data set, and define the official common language with the most occurrences as the language used in the recording data; the target language standard pronunciation acquisition module 6 is used to build a Transformer model and translate the recording data based on the language used to obtain the standard pronunciation based on the target language; the target language standard pronunciation sending module 7 is used to output the standard pronunciation based on the target language to the portable translation device for playback.
[0187] Furthermore, the recording data acquisition module 2 specifically includes a first recording data acquisition submodule, a second recording data acquisition submodule, a third recording data acquisition submodule, a fourth recording data acquisition submodule, a fifth recording data acquisition submodule, and a sixth recording data acquisition submodule, which are electrically connected in sequence; the first recording data acquisition submodule is electrically connected to the language database construction module, and the sixth recording data acquisition submodule is electrically connected to the processing data set acquisition module.
[0188] Among them, the first recording data acquisition submodule is used to obtain several historical recording data of the same user; the second recording data acquisition submodule is used to perform word segmentation processing on each historical recording data through a word segmentation algorithm to obtain a historical data set to be processed with several historical word segmentation sub-data; the third recording data acquisition submodule is used to divide the historical data set to be processed into a training set and a sample set according to a preset ratio; the fourth recording data acquisition submodule is used to input the training set into a preset neural network model and train it to obtain a language habit prediction model; the fifth recording data acquisition submodule is used to input the sample set into the language habit prediction model to obtain a language habit prediction value; the sixth recording data acquisition submodule is used to define the language habit prediction value as the recording data of the same user.
[0189] Furthermore, the fourth recording data acquisition submodule specifically includes a first recording data acquisition unit, a second recording data acquisition unit, a third recording data acquisition unit, and a fourth recording data acquisition unit that are electrically connected in sequence; the first recording data acquisition unit is electrically connected to the third recording data acquisition submodule, and the fourth recording data acquisition unit is electrically connected to the fifth recording data acquisition submodule.
[0190] Among them, the first recording data acquisition unit is used to define the topological relationship of the preset neural network model, and the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected; the second recording data acquisition unit is used to output the training set to the input layer, train the training set by the least squares estimation method, and obtain the root mean square error between the sample set and the current training result based on each training; the third recording data acquisition unit is used to obtain the minimum value of the root mean square error between the sample set and all training results; the fourth recording data acquisition unit is used to obtain the training result corresponding to the minimum value as a language habit prediction model.
[0191] Furthermore, the module for acquiring the data set to be processed 3 specifically includes a first submodule for acquiring the data set to be processed, a second submodule for acquiring the data set to be processed, and a third submodule for acquiring the data set to be processed, which are electrically connected in sequence; the first submodule for acquiring the data set to be processed is electrically connected to the sixth submodule for acquiring the recorded data, and the third submodule for acquiring the data set to be processed is electrically connected to the classification module for the data set to be processed.
[0192] Among them, the first submodule for obtaining the data set to be processed is used to perform word segmentation processing on the recorded data through the SentencePiece software to find the word segmentation points of the recorded data; the second submodule for obtaining the data set to be processed is used to divide the recorded data into several word segments based on all the word segmentation points; the third submodule for obtaining the data set to be processed is used to mark each word segmentation by decoding the time tag, and obtain a word segmentation sub-data based on one tag.
[0193] Furthermore, the processing data set classification module 4 specifically includes a first processing data set classification submodule, a second processing data set classification submodule, a third processing data set classification submodule, and a fourth processing data set classification submodule, which are electrically connected in sequence; the first processing data set classification submodule is electrically connected to the third processing data set acquisition submodule, and the fourth processing data set classification submodule is electrically connected to the recording data language definition module.
[0194] The first processing data set classification submodule is used to define the processing data set X = {x1, x2, ..., x j ,…,x m}, x j is the jth word segmentation sub-data, and m is the number of all word segmentation sub-data.
[0195] The second to-be-processed dataset classification submodule is used to define a category set C = {y1, y2, ..., y k ,…,y n}, where y kis the kth official common language in category set C, and n is the number of official common languages of all countries.
[0196] The third dataset classification submodule is used to calculate the conditional probability of the jth word segmentation sub-data in each official common language according to formula (1):
[0197]
[0198] Among them, P(X|y k ) is the conditional probability of the kth official common language based on the dataset X to be processed; P(y k ) is the marginal probability of the kth official common language; P(x j |y k ) is the conditional probability of the jth word segmentation sub-data under the kth official common language.
[0199] The fourth to-be-processed data set classification submodule is used to classify each word segmentation sub-data into the official common language with the highest conditional probability.
[0200] Furthermore, the target language standard pronunciation acquisition module 6 specifically includes a first target language standard pronunciation acquisition submodule, a second target language standard pronunciation acquisition submodule, a third target language standard pronunciation acquisition submodule, a fourth target language standard pronunciation acquisition submodule, a fifth target language standard pronunciation acquisition submodule, and a sixth target language standard pronunciation acquisition submodule, which are electrically connected in sequence; the first target language standard pronunciation acquisition submodule is electrically connected to the recording data language definition module, and the sixth target language standard pronunciation acquisition submodule is electrically connected to the target language standard pronunciation sending module.
[0201] Among them, the first target language standard pronunciation acquisition submodule is used to obtain the representation vector of each word segmentation sub-data respectively; the second target language standard pronunciation acquisition submodule is used to construct a word representation vector matrix containing all identification vectors based on the same data set to be processed; the third target language standard pronunciation acquisition submodule is used to input the word representation vector matrix into the Encoder block of the Transformer model, and obtain the encoding information matrix of the same data set to be processed after passing through at least six Encoder blocks; the fourth target language standard pronunciation acquisition submodule is used to input the encoding information matrix into the Decoder block of the Transformer model; the fifth target language standard pronunciation acquisition submodule is used to sequentially search for the target language segmentation corresponding to each encoding information of the encoding information matrix through at least six Decoder blocks; the sixth target language standard pronunciation acquisition submodule is used to sequentially combine all target language segmentations to obtain the standard pronunciation based on the target language.
[0202] Furthermore, the data interaction device also includes an original data and translation data storage module electrically connected to the target language standard pronunciation sending module 7, and the original data and translation data storage module is used to store the recording data and the standard pronunciation based on the target language in the portable translation device and the cloud.
[0203] It should be noted that this embodiment is a functional module embodiment based on the above method embodiment. The preferred, expanded, limited and exemplary parts of this embodiment can refer to the above embodiment and will not be described in detail in this embodiment.
[0204] This embodiment constructs a language database according to a preset strategy, and the language database includes the standard pronunciations of all official common languages of the country; responds to the voice trigger signal of the portable translation device, continuously records the pronunciation of the person based on the voice trigger signal to form recording data; performs word segmentation processing on the recording data through a word segmentation algorithm to obtain a data set to be processed with a plurality of word segmentation sub-data; classifies each word segmentation sub-data in the data set to be processed into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm; obtains the official common language corresponding to all the word segmentation sub-data based on the same data set to be processed, and defines the official common language with the most occurrences as the language used in the recording data; constructs a Transformer model and translates the recording data based on the language used to obtain the standard pronunciation based on the target language; and outputs the standard pronunciation based on the target language to the portable translation device for playback. This embodiment uses a classification algorithm and a word segmentation algorithm to collaboratively perform real-time recognition and real-time processing on the original recording data obtained by sound pickup, and then translates the processed audio data through a Transformer model. Compared with the traditional large language model, the translation content of this embodiment is more accurate and the context is smoother. In addition, the processing flow of this embodiment is smoother and less cumbersome, and can usually respond in milliseconds to seconds.
[0205] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 As shown, the electronic device 8 includes a processor 81 and a memory 82 coupled to the processor 81 .
[0206] The memory 82 stores program instructions for implementing a data interaction method for a portable translation device according to any of the above embodiments.
[0207] The processor 81 is used to execute program instructions stored in the memory 82 to perform data interaction of the portable translation device.
[0208] The processor 81 may also be referred to as a CPU (Central Processing Unit). The processor 81 may be an integrated circuit chip having signal processing capabilities. The processor 81 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0209] Further, Figure 4 This is a schematic diagram of the structure of a storage medium according to an embodiment of the present application. Figure 4 The storage medium 9 of the embodiment of the present application stores program instructions 91 that can implement all the above methods, wherein the program instructions 91 can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.
[0210] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0211] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.
Claims
1. A data interaction method for a portable translation device, the data interaction method is applied to a person wearing the portable translation device, characterized in that: The data interaction method comprises: Step S1, constructing a language database according to a preset strategy, wherein the language database includes standard pronunciations of all official common languages of the country; Step S2, in response to a voice trigger signal of the portable translation device, continuously recording the pronunciation of a person based on the voice trigger signal to form recording data; Step S3, performing word segmentation processing on the recorded data by using a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data; Step S4, classifying each word segmentation sub-data in the to-be-processed data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm; Step S5, based on the same data set to be processed, obtaining the official common language corresponding to all the word segmentation sub-data, and defining the official common language with the largest number of occurrences as the language used in the recording data; Step S6, constructing a Transformer model and translating the recorded data based on the language used to obtain a standard pronunciation based on the target language; Step S7: outputting the standard pronunciation based on the target language to the portable translation device for playback.
2. The data interaction method according to claim 1, characterized in that: Step S2, in response to the voice trigger signal of the portable translation device, continuously recording the pronunciation of the person based on the voice trigger signal to form recording data, including: Step S21, obtaining several historical recording data of the same user; Step S22, performing word segmentation processing on each historical recording data by using a word segmentation algorithm to obtain a to-be-processed historical data set having a plurality of historical word segmentation sub-data; Step S23, dividing the historical data set to be processed into a training set and a sample set according to a preset ratio; Step S24, inputting the training set into a preset neural network model and training it to obtain a language habit prediction model; Step S25, inputting the sample set into the language habit prediction model to obtain a language habit prediction value; Step S26, defining the language habit prediction value as the recording data of the same user.
3. The data interaction method according to claim 2, characterized in that: Step S24, inputting the training set into a preset neural network model and training it to obtain a language habit prediction model, including: Step S241, defining a topological relationship of the preset neural network model, wherein the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected; Step S242, outputting the training set to the input layer, training the training set by the least squares estimation method, and obtaining the root mean square error between the sample set and the current training result once for each training; Step S243, obtaining the minimum value of the root mean square error between the sample set and all training results; Step S244, obtaining a training result corresponding to the minimum value as the language habit prediction model.
4. The data interaction method according to claim 1, characterized in that: Step S3, performing word segmentation processing on the recording data by using a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data, including: Step S31, performing word segmentation processing on the recorded data by using SentencePiece software to find the word segmentation points of the recorded data; Step S32, dividing the recorded data into a plurality of segmented words based on all segmentation points; Step S33, marking each word segmentation by decoding the time tag, and obtaining a word segmentation sub-data based on one tag.
5. The data interaction method according to claim 1, characterized in that: Step S4, classifying each word segmentation sub-data in the to-be-processed data set into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm, including: Step S41, define the data set to be processed X = {x1, x2, ..., x j ,…,x m }, x j is the jth word segmentation sub-data, and m is the number of all word segmentation sub-data; Step S42, defining a category set C = {y1, y2, ..., y k ,…,y n }, where y k is the kth official common language in the category set C, and n is the number of official common languages of all countries; Step S43, calculate the conditional probability of the j-th word segmentation sub-data in each official common language according to formula (1): Among them, P(Xy k ) is the conditional probability of the kth official common language based on the dataset X to be processed; P(y k ) is the marginal probability of the kth official common language; P(x j y k ) is the conditional probability of the jth word segmentation sub-data under the kth official common language; Step S44: classify each word segmentation sub-data into the official common language with the highest conditional probability.
6. The data interaction method according to claim 1, characterized in that: Step S6, constructing a Transformer model and translating the recorded data based on the language used to obtain a standard pronunciation based on the target language, including: Step S61, respectively obtaining a representation vector of each word segmentation sub-data; Step S62, constructing a word representation vector matrix including all identification vectors based on the same data set to be processed; Step S63, inputting the word representation vector matrix into the Encoder block of the Transformer model, and obtaining the encoding information matrix of the same data set to be processed after passing through at least six Encoder blocks; Step S64, inputting the encoding information matrix into the Decoder block of the Transformer model; Step S65, searching for the target language word segment corresponding to each coded information in the coded information matrix in sequence through at least six Decoder blocks; Step S66, sequentially combining all target language segmentations to obtain a standard pronunciation based on the target language.
7. The data interaction method according to claim 1, characterized in that: Step S7, outputting the standard pronunciation based on the target language to the portable translation device for playback, and then comprising: Step S8, storing the recording data and the standard pronunciation based on the target language in the portable translation device and the cloud.
8. A data interaction device for a portable translation device, the data interaction device being applied to the data interaction method for a portable translation device according to any one of claims 1 to 7, characterized in that: The data interaction device comprises: A language database construction module, used to construct a language database according to a preset strategy, wherein the language database includes standard pronunciations of all official common languages of the country; A recording data acquisition module, configured to respond to a voice trigger signal of the portable translation device and continuously record a person's pronunciation based on the voice trigger signal to form recording data; The module for acquiring the data set to be processed is used to perform word segmentation processing on the recording data through a word segmentation algorithm to obtain a data set to be processed having a plurality of word segmentation sub-data; A classification module for the data set to be processed, used to classify each word segmentation sub-data in the data set to be processed into the standard pronunciation with the highest probability of overlap of the respective voiceprints through a classification algorithm; A recording data language definition module, used to obtain the official common language corresponding to all word segmentation sub-data based on the same data set to be processed, and define the official common language with the most occurrences as the language used in the recording data; A target language standard pronunciation acquisition module is used to construct a Transformer model and translate the recording data based on the language used to obtain a standard pronunciation based on the target language; The target language standard pronunciation sending module is used to output the standard pronunciation based on the target language to the portable translation device for playback.
9. An electronic device, characterized in that: It includes a processor and a memory coupled to the processor, wherein the memory stores program instructions that can be executed by the processor; when the processor executes the program instructions stored in the memory, a data interaction method for a portable translation device as described in any one of claims 1 to 7 is implemented.
10. A storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed by the processor, the data interaction method of a portable translation device according to any one of claims 1 to 7 can be implemented.