A method, system and device for blind translation of Chinese combining rules

Abstract

The application relates to the field of computer word processing, in particular to a Chinese blind translation method, system and device combined with rules. The method comprises the following steps: obtaining Chinese character data to be translated; inputting the Chinese character data to be translated into a large language model to obtain Braille translation; the large language model comprises a word embedding layer, a learnable rule layer, an encoding layer and a decoding layer; the Chinese character data to be translated is input into the word embedding layer to generate a word vector, and is input into the learnable rule layer to generate a rule vector; the word vector and the rule vector are fused to obtain a constraint enhanced feature vector; the constraint enhanced feature vector is input into the encoding layer to be encoded, and then is decoded through the decoding layer to obtain the Braille translation. The application combines Braille conversion rules, can solve the problem of converting Braille for multi-sound Chinese characters, and has good application value.

Description

Technical Field

[0001] This application relates to the field of computer word processing, specifically to a method, system, and device for rule-based Chinese-to-Blind translation. Background Technology

[0002] Driven by the growing needs of the visually impaired community, high-precision Chinese-Braille translation technology has become a crucial area requiring breakthroughs. Among existing technologies, patent CN111581985B provides a Transformer-based Chinese-Braille translation method. This method achieves sentence-level Chinese-Braille digital resource conversion through a multi-head self-attention mechanism, significantly reducing the cost of corpus construction. However, this method has limitations in extracting contextual information, especially when handling complex sentences, where translation accuracy and semantic coherence still need improvement. Another patent, CN116681037A, proposes an automatic Chinese-Braille conversion method based on a pre-trained model. This method aims to reduce reliance on parallel data through pre-training and fine-tuning, achieving good translation results even with limited data. Nevertheless, this method may have limitations in handling broader language structures and contexts, particularly when dealing with sentences highly dependent on contextual understanding, where accuracy and naturalness may be insufficient. Summary of the Invention

[0003] To address the aforementioned issues, this application aims to develop a rule-based method for Chinese-Braille translation. This method not only effectively improves the accuracy of Chinese-Braille conversion but also provides more natural and accurate translation results when handling complex contexts and long sentences, thereby better meeting the reading needs and knowledge acquisition abilities of the visually impaired community. Specifically, it includes: Obtain the Chinese characters to be translated; The Chinese characters to be translated are input into a large language model for translation to obtain Braille translation; The large language model includes a word embedding layer, a learnable rule layer, an encoding layer, and a decoding layer. The Chinese characters to be translated are input to the word embedding layer to generate word vectors, and to the learnable rule layer to generate rule vectors. The word vectors and rule vectors are fused to obtain a constraint-enhanced feature vector. The constraint-enhanced feature vector is input to the encoding layer for encoding, and then to the decoding layer for decoding to obtain the Braille translation.

[0004] Optionally, the learnable rule layer maps and matches the converted Chinese character data with the corresponding rule ID through a mapping table to generate a rule vector; the rules include any one or more of the following: no applicable rules, rules for initial consonants forming syllables, rules for final vowels forming syllables, special spelling rules, rules for omitting tones, punctuation matching rules, rules for linking proper nouns, rules for abbreviating vocabulary, rules for linking and separating words, and custom extended rules.

[0005] Optionally, the learnable rule layer also includes pinyin and tone embedding. After converting Chinese character data into pinyin, it is converted into pinyin vectors and tone vectors. The rule IDs corresponding to pinyin and tone are obtained through a mapping table. The rule IDs of Chinese characters, pinyin and tone are used to generate rule vectors. The pinyin vectors, tone vectors, word vectors and rule vectors are fused to obtain a second constraint enhancement feature vector. The second constraint enhancement feature vector is input to the encoding layer for encoding, and then decoded through the decoding layer to obtain Braille translation.

[0006] Optionally, the construction of the mapping table includes: obtaining a Chinese character dataset, segmenting the Chinese character data into words, associating the segmented words with any one or more of the following rules: no applicable rules, proper noun linking rules, vocabulary abbreviation rules, linking and splitting rules, punctuation matching rules, and custom extension rules, and / or converting the segmented words into pinyin, associating the pinyin with any one or more of the following rules: initial consonant self-syllable rules, final vowel self-syllable rules, special spelling rules, and tone omission rules, assigning a globally unique ID to each independent rule, and obtaining the mapping table.

[0007] Optionally, the training process of the large language model is as follows: acquiring Chinese character data and Braille labels; segmenting the Chinese character data into word vectors; passing the segmented Chinese characters through a learnable rule layer to obtain rule vectors; fusing the word vectors and rule vectors and inputting them into the encoder and decoder for translation to obtain translated Braille; calculating the loss function until the loss function remains unchanged to obtain the large language model; the loss function includes translation cross-entropy loss, rule compliance loss, and embedding alignment loss; the embedding alignment loss is obtained by calculating the distance between the word vectors and the rule vectors; the rule compliance loss is obtained by detecting the correspondence between the translated Braille and the rule vectors; and the translation cross-entropy loss is obtained by the correspondence between the translated Braille and the Braille labels.

[0008] Optionally, the fusion is obtained by dynamically assigning weights to vectors of different dimensions through a gating mechanism and then summing them by weight. Rule embedding weights are added to the rule constraint vectors, and pinyin and tone embedding weights are added to the polyphonic character vectors.

[0009] Optionally, the encoding layer includes K encoders, and the decoding layer includes K decoders. Each encoder or decoder sequentially includes a self-attention layer, a first-batch normalization layer, a forward propagation layer, and a second-batch normalization layer. In the first-batch normalization layer, the output of the previous layer and the initial input are fused through residual connections. In the second-batch normalization layer, the output of the previous layer and the output of the first-batch normalization layer are fused through residual connections. K is a natural number greater than 1.

[0010] The purpose of this invention is to provide a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-described method for Chinese-Blind translation based on the combination rules.

[0011] The purpose of this invention is to provide a computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, wherein the computer program or instructions are executed by the processor to implement the above-described method for Chinese-Blind Translation based on the combination rules.

[0012] The purpose of this invention is to provide a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the above-described method for Chinese-Blind translation based on the combination rules.

[0013] Advantages of this invention: 1. Addressing the current challenges in Braille translation for visually impaired individuals, particularly the difficulty in integrating linguistic structure and polyphonic characters, leading to inaccurate translations, this invention proposes a pre-trained model for Chinese-Braille translation. This model utilizes learnable rule embedding to transform mandatory grammatical rules of standard Braille, such as abbreviations, tone markings, linking, and omissions, into learnable feature vectors. These vectors are then integrated with Chinese semantic, pinyin, and tone features at the encoder input. This allows the pre-trained model to retain strong semantic understanding capabilities while intrinsically learning Braille grammatical constraints. The goal is to transform the large model from simply generating data based on corpora to understanding Braille rules and generating data according to national standards. This completely resolves the three major pain points in Chinese-Braille translation: rule violations, mistranslation of polyphonic characters, and non-standard generation. It achieves accurate Chinese-Braille translation and compliant generation, effectively addressing the technical problems of traditional methods such as rule-semantic separation, low compliance rates, and error propagation.

[0014] 2. To address the semantic disconnect between Chinese characters and Braille, this invention adds alignment loss during model training, bringing Chinese character embeddings and corresponding Braille embeddings closer together in the vector space, or bringing Pinyin embeddings and Braille embeddings closer together in the vector construction, thereby improving the accuracy and reliability of translation. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart of a method for combining rules in Chinese-English translation provided in an embodiment of the present invention; Figure 2 A schematic diagram of a rule-based Chinese-to-Blind Translation system provided in an embodiment of the present invention; Figure 3 A schematic diagram of a computer device provided in an embodiment of the present invention; Figure 4 The Chinese word segmentation model provided in this embodiment of the invention: Bert-CRF architecture diagram; Figure 5 This invention provides a correction example in the process of constructing a Chinese-Braille parallel gold standard corpus. Figure 6 These are the 62 basic Braille characters defined in the "National General Braille Scheme" provided in this embodiment of the invention; Figure 7 Examples of input and output for the mT5-large model provided in this embodiment of the invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0018] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as S101, S102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0019] Figure 1 The schematic diagram of the rule-based Chinese-to-English translation method provided in this embodiment of the invention specifically includes: S1: Obtain the Chinese characters to be translated; In one embodiment, the Chinese character data to be translated is segmented and / or converted to Pinyin to obtain segmented data and Pinyin data. The segmented data and Pinyin data are then input into a large language model for translation to obtain Braille translation.

[0020] In one embodiment, the tool is implemented using Python, relying on the jieba word segmentation and pypinyin libraries. It supports unified conversion of Chinese characters, numbers, punctuation, and English characters, ensuring accurate conversion of all types of characters.

[0021] In another embodiment, the Chinese character data to be translated is segmented using a word segmentation model, specifically a BERT+CRF model, with a structure including BERT-base-chinese → dropout → linear → CRF → layernorm. A pre-trained BERT-base-chinese model is used as the encoder to obtain the contextual representation of each Chinese character through deep semantic modeling. Subsequently, a Conditional Random Field (CRF) layer is used to globally optimize the sequence, achieving refined recognition of Chinese word boundaries. In the conversion process, the Chinese text first generates a contextual vector representation through the BERT encoder, and then the CRF layer constrains the sequence labels to ensure the consistency and rationality of the segmentation results. Compared with existing methods, BERT-CRF can more accurately handle polyphonic characters, long sentences, and rare words, thus significantly improving the accuracy and stability of Braille conversion.

[0022] S2: Input the Chinese character data to be translated into a large language model to obtain Braille translation; The large language model includes a word embedding layer, a learnable rule layer, an encoding layer, and a decoding layer. The Chinese characters to be translated are input to the word embedding layer to generate word vectors, and to the learnable rule layer to generate rule vectors. The word vectors and rule vectors are fused to obtain a constraint-enhanced feature vector. The constraint-enhanced feature vector is input to the encoding layer for encoding, and then to the decoding layer for decoding to obtain the Braille translation.

[0023] In one embodiment, the learnable rule layer maps and matches the converted Chinese character data with the corresponding rule ID through a mapping table to generate a rule vector; the rules include any one or more of the following: no applicable rules, rules for initial consonants forming syllables, rules for final vowels forming syllables, special spelling rules, rules for omitting tones, punctuation matching rules, rules for linking proper nouns, rules for abbreviating vocabulary, rules for linking and separating words, and custom extended rules.

[0024] Optionally, the learnable rule layer also includes pinyin and tone embedding. After converting Chinese character data into pinyin, it is converted into pinyin vectors and tone vectors. The rule IDs corresponding to pinyin and tone are obtained through a mapping table. The rule IDs of Chinese characters, pinyin and tone are used to generate rule vectors. The pinyin vectors, tone vectors, word vectors and rule vectors are fused to obtain a second constraint enhancement feature vector. The second constraint enhancement feature vector is input to the encoding layer for encoding, and then decoded through the decoding layer to obtain Braille translation.

[0025] In one embodiment, the construction of the mapping table includes: obtaining a Chinese character dataset, segmenting the Chinese character data into words, associating the segmented words with any one or more of the following rules: no applicable rules, proper noun linking rules, vocabulary abbreviation rules, linking and splitting rules, punctuation matching rules, and custom extension rules, and / or converting the segmented words into pinyin, associating the pinyin with any one or more of the following rules: initial consonant self-syllable rules, final vowel self-syllable rules, special spelling rules, and tone omission rules, assigning a globally unique ID to each independent rule, and obtaining the mapping table.

[0026] In one embodiment, the training process of the large language model is as follows: acquiring Chinese character data and Braille labels; segmenting the Chinese character data into word vectors; passing the segmented Chinese characters through a learnable rule layer to obtain rule vectors; fusing the word vectors and rule vectors and inputting them into the encoder and decoder for translation to obtain translated Braille; calculating the loss function until the loss function remains unchanged to obtain the large language model; the loss function includes translation cross-entropy loss, rule compliance loss, and embedding alignment loss; the embedding alignment loss is obtained by calculating the distance between the word vectors and the rule vectors; the rule compliance loss is obtained by detecting the correspondence between the translated Braille and the rule vectors; and the translation cross-entropy loss is obtained by the correspondence between the translated Braille and the Braille labels.

[0027] Optionally, when the fused feature is the second constraint enhancement feature vector, the model's loss function also includes syllable alignment loss, which is obtained by calculating the cosine distance between the pinyin vector and the Braille label vector, and is used to constrain the matching between the pinyin vector and the Braille vector.

[0028] In another embodiment, the model further includes a Braille structure perception layer. The constraint enhancement feature vector or the second constraint enhancement feature vector is fused with the Braille prior vector through the Braille structure perception layer before being input to the encoder for encoding. The fusion is obtained by adding each bit sequentially. The Braille prior vector is a 6-dot fixed structure prior vector, generated using sine and cosine encoding based on the 6-dot physical structure of the "National General Braille Scheme." It is a globally fixed prior and does not participate in training. This prior vector is added position-by-position to the fusion features of Chinese character semantics, rules, pinyin, and tone, enabling the model to learn the prior knowledge of the Braille dot matrix structure during the encoding stage. This constrains the output Braille to conform to the 6-dot physical standard from the source, effectively solving the problems of garbled characters, illegal dots, and misaligned formats caused by the lack of Braille structure cognition in traditional models.

[0029] Specifically, each Braille character is a unique 6-bit binary vector (with fixed dot numbers), and a sine-cosine fixed encoding (the standard method of Transformer native positional encoding) is used to map the 6-dot structure into a 1024-dimensional fixed vector: 1) Use a 6-bit basic point vector [1,1,1,1,1,1] (representing a complete 6-point structure); 2) By using sine and cosine functions, the 6-dimensional vector is expanded and mapped to a 1024-dimensional fixed vector; 3) The vector values ​​are fixed and remain unchanged throughout the training / inference process.

[0030] The Braille structure perception layer adds a "Braille structure cue word" to the model. Previously, the model treated Braille as ordinary characters. Now, it directly injects 6-dot priors to know the dot matrix rules of Braille, so that the decoder will not exceed the legal range of 6 dots when generating, and there will be no cross-dot, missing dot, or multi-dot errors.

[0031] In one embodiment, the fusion is obtained by dynamically assigning weights to vectors of different dimensions through a gating mechanism and then summing them by weight. Rule embedding weights are added to the rule constraint vectors, and pinyin and tone embedding weights are added to the polyphonic character vectors.

[0032] Alternatively, the fusion can also be performed by splicing or weighted fusion.

[0033] In one embodiment, the encoding layer includes K encoders, and the decoding layer includes K decoders. Each encoder or decoder sequentially includes a self-attention layer, a first-batch normalization layer, a forward propagation layer, and a second-batch normalization layer. In the first-batch normalization layer, the output of the previous layer and the initial input are fused through residual connections. In the second-batch normalization layer, the output of the previous layer and the output of the first-batch normalization layer are fused through residual connections. K is a natural number greater than 1.

[0034] In one embodiment, the encoder and decoder of the large language model are based on the mT5-large model architecture. The Chinese characters to be translated are processed through a word embedding layer and a learnable rule layer to obtain constrained and enhanced feature vectors, which are then input into the mT5-large model for encoding and decoding to obtain the Braille translation. The encoder and decoder architecture of the large language model can also be other models suitable for text translation.

[0035] In one specific embodiment, the learnable embedding layer of the present invention transforms explicit Braille grammar rules into continuous vectors that the model can learn, and integrates them with text word embeddings, so that the model can perceive Braille normative constraints during the encoding stage, rather than simply learning them implicitly from the corpus.

[0036] In one specific embodiment, this invention extracts 1000 Chinese sentences from the People's Daily corpus and manually constructs a parallel gold standard corpus of Chinese and Braille. After performing word segmentation on the corpus and implementing an automatic Chinese-to-Braille translation tool based on jieba, we found certain limitations in the word segmentation stage. jieba word segmentation mainly relies on dictionary matching, making it difficult to handle out-of-vocabulary words (such as emerging internet slang or professional terms), and it lacks a deep understanding of contextual semantics, resulting in limited accuracy in complex contexts, thus affecting the overall conversion effect. To improve the accuracy and robustness of word segmentation, we further trained a Bert-CRF model specifically for Chinese word segmentation. This model is based on pre-trained bert-base-chinese and combines CRF (Conditional Random Field) for Chinese word segmentation. BERT is a deep pre-trained language model proposed by Google, possessing bidirectional contextual modeling capabilities and able to generate high-quality word and sentence vectors. Its Chinese variant, bert-base-chinese, is pre-trained on large-scale Chinese corpora (such as Chinese Wikipedia and news texts), supporting Chinese character-level and word-level modeling. CRF (Contextualized Randomization Rendering) is a probabilistic graphical model widely used in sequence labeling tasks. It can effectively model the transition relationships between labels and achieve globally optimal prediction. By combining BERT's strong semantic modeling capabilities with CRF's structured prediction mechanism, the BERT-CRF model can improve the consistency of sequence labeling while maintaining semantic understanding. Experimental results show that this model performs excellently in Chinese word segmentation tasks, especially in handling ambiguous segmentation and complex contexts. Figure 4 The overall structure of the model is shown. The first layer, BERT: a pre-trained language model, provides context-sensitive word vector representations, serving as a powerful feature extractor for sequence labeling tasks. The second layer, LayerNorm: normalizes BERT output features, stabilizing the training process and accelerating convergence. The third layer, Dropout: randomly deactivates some neurons to prevent the model from overfitting the training data. The fourth layer, Linear: linearly maps 768-dimensional BERT features to the label space, generating a label score for each token. The fifth layer, CRF: models the transition relationships between labels and decodes them using the Viterbi algorithm to obtain the globally optimal label sequence. The introduction of this model significantly improves the word segmentation accuracy of the Chinese-to-Blind Automatic Translation Tool in complex contexts.

[0037] Furthermore, the present invention collected Chinese corpora from the Leipzig Corpus and the Modern Chinese Grammar Information Dictionary. During the preprocessing, it was found that some corpora contained special symbols (such as %, / , etc.) not standardized in the National General Braille Scheme. For these symbols that cannot be represented by Braille norms, the present invention performed data cleaning. Subsequently, a Braille text was generated using a Chinese-to-Braille automatic conversion tool, and the alignment data between Chinese and Braille was initially constructed. After further fine processing through an alignment algorithm, a high-quality Chinese-Braille parallel corpus was formed, as Figure 5 shown. In the Chinese-to-Braille automatic conversion tool, the Chinese text is first converted into pinyin and then mapped to Braille. In this process, errors may occur in the pinyin conversion stage due to the existence of polyphonic characters, but the Braille mapping stage is based on a fixed correspondence between initials and Braille, so there will be no deviation itself. Therefore, only the errors in the pinyin conversion stage need to be corrected to complete the overall correction. Take "refugee" as an example. The correct pinyin of the character "难" should be the fourth tone "nàn", but the tool wrongly converts it to the second tone "nán". Since the Braille corresponding to "nan2" is " ", while the correct "nan4" should be " ", so by correcting " " to " ", the correction can be completed.

[0038] To verify the accuracy of the conversion tool and the quality of the corpus, we aligned the Braille results generated by the tool character by character with the gold standard corpus and calculated the character-level accuracy rate:

[0039] Among them, represents the number of correctly matched characters, represents the total number of characters in the gold standard. Character-level evaluation can accurately measure the technical accuracy of the tool at the Braille coding level.

[0040] Meanwhile, we performed word-by-word alignment and counted the word-level accuracy rate:

[0041] Among them, represents the number of correctly matched words, represents the total number of words in the gold standard. This index is used to measure the overall accuracy of information transmission.

[0042] In a specific embodiment, the National General Braille Scheme defines 62 basic Braille characters as the smallest constituent units of all Braille symbols ( Figure 6(As shown). However, some Braille characters were not included in the original vocabulary of the mT5-large / mBART-large-50 model. To ensure that the model can accurately recognize these characters, this invention first adds all 62 Braille characters to the tokenizer using the tokenizer.add_tokens() method; then, it uses the model.resize_token_embeddings() method to adjust the embedding layer dimensions to match the updated vocabulary. This processing effectively improves the model's ability to recognize and represent Braille input.

[0043] Furthermore, this invention also discovered that the bert-base-chinese vocabulary contains missing commonly used Chinese punctuation marks, such as full-width single quotes (''), double quotes (“”), dashes (——), and ellipses (…). To improve the model's ability to process real text, this invention compared and supplemented the missing parts with the People's Republic of China National Standard "Usage of Punctuation Marks," and utilized this information during the training phase, thereby enhancing the model's versatility and accuracy.

[0044] For word segmentation networks, this invention compares word segmentation models with different structures and obtains... Figure 4 The network structure shown below performs best, and the comparison results are shown in Table 1: 1: bert-base-chinese→dropout→linear→CRF→layernorm; 2: bert-base-chinese→dropout→bilstm→linear→CRF→layernorm; 3: bert-base-chinese→dropout→linear→CRF; Table 1 Word segmentation models with different network structures

[0045] In the data preprocessing stage, this invention converts the original word segmentation and annotation data into a (B, I, E, S) tag system. Specifically, B (Begin) indicates the start of a word, I (Inside) indicates the middle of a word, E (End) indicates the end of a word, and S (Single) indicates a single character forming an independent word. This annotation system can more precisely characterize the boundaries of Chinese words, helping to improve the sequence annotation capabilities of the word segmentation model.

[0046] In one specific embodiment, the technical solution of the present invention is constructed as follows: Step 1: Systematized definition and discrete coding of Braille rules; The embedding layer only supports numerical index input. Therefore, it is necessary to standardize the Braille grammar rules first and map them to unique integer IDs. Focusing on the core constraints of the "National Common Braille Plan", in terms of spelling and tone rules, it includes the rules for initials forming syllables on their own, finals forming syllables on their own, special spelling rules, and tone abbreviation rules, etc. For example, for syllables such as "zhi", "chi", "shi", "zi", "ci", "si", the final "i" is omitted, and the initial forms the syllable on its own. Finals such as "ai", "en" can form syllables on their own. "他" and "它" are specially spelled as " ", " ". In terms of tone abbreviation, for syllables with the initial "f", the symbol for the first tone is omitted; for syllables with initials such as "p, m, t, n, h", the symbol for the second tone is omitted; for syllables with initials such as "b, d, l, g, k, j", the symbol for the fourth tone is omitted, and the symbol for the fourth tone is also omitted when the final forms a syllable on its own. In addition, there are also some specified syllables that directly omit the tone. Based on the above rules, the present invention sets corresponding limiting conditions for the output content to ensure that the generated Braille results strictly conform to the specifications of the plan in terms of form and content. In terms of abbreviation rules, the present invention has formulated a set of controllable and extensible abbreviation logics for words such as "的", "么", "你", "他", "她", "它" according to the plan, as shown in Table 2 below, covering the entire translation scenario; Table 2

[0047] Assign a globally unique integer ID to each independent rule, construct a fixed mapping table, without repetition and omission.

[0048] Corpus preprocessing: Annotate the Chinese-Braille parallel corpus, assign rule IDs to each Token, and generate a rule_ids sequence of the same length as input_ids. For example: Input text: 银行 的 利率; Tokenized Tokens: 银行 的 利率; Rule ID sequence: [6, 3, 0] (abbreviation for proper nouns, abbreviation of "的", no rule); Padding and alignment: Keep the same length as the model input sequence, and fill the blank positions with 0.

[0049] Step 2: Construction of the structure of the learnable rule embedding layer; rule_ids rule index (e.g., 0 = no rule, 1 = abbreviation of the light tone), consistent with the dimension of the model hidden layer (1024 dimensions), injecting explicit grammar constraints.

[0050] The learnable rule embedding layer is a neural network embedding layer with trainable parameters. The hidden layers have the same dimension as the large language model and are not fixed encoding mappings. A two-dimensional parameter matrix is ​​randomly initialized, and the parameters are continuously updated through backpropagation during training. It is also equipped with a gating fusion module to dynamically balance the weights of text features and rule features.

[0051] Step 3: Word embedding layer (native to the model, bound to a Braille vocabulary); The `input_ids` is a vocabulary index (Braille characters and Chinese words), and the model's word segmenter vocabulary (including expanded Braille characters) is consistent with the hidden layer dimension of the model (1024 dimensions), encoding the semantic information of the text. The mT5 model's vocabulary does not contain Braille characters, so before inputting Braille text, the vocabulary must be expanded by adding standard Braille characters and special symbols to the word segmenter; the word embedding matrix is ​​expanded to adapt to the new vocabulary length, and the embedding vectors of newly added Braille tokens are initialized and optimized (rather than purely randomized).

[0052] Step 4: Fusion; By combining structural fusion and training collaboration, semantic features (from the vocabulary / word embedding) and rule features (from the rule embedding layer) work together and are optimized synchronously, ultimately outputting translation results that are more in line with Braille standards.

[0053] 1. Structural-level union (feature fusion, forward propagation logic); 1) Dual-channel input generation; For the same Chinese text, preprocessing generates two sets of sequences of equal length: input_ids: obtained by querying the vocabulary through the word segmenter, input into the word embedding layer, and generate text_embeds (semantic features); rule_ids: Obtained through the rule mapping table, input into the rule embedding layer to generate rule_embeds (rule features).

[0054] 2) Gated dynamic fusion (core joint node); Using a standard gating mechanism, weights are automatically learned, and two sets of features of the same dimension are weighted and fused together: fused = gate × text_embeds + (1 gate)×rule_embeds The gating coefficient is learned autonomously by the model. In strong rule scenarios (such as abbreviations), the weight of rule features is amplified, while in ordinary scenarios, they are balanced and integrated.

[0055] 3) Send to the backbone network; The fused features replace the original word embeddings and are input into the self-attention layer of the mT5 encoder. The two sets of features jointly participate in global semantic encoding, achieving deep structural binding.

[0056] 2. Joint training layer (parameter co-optimization); Structural fusion is just the foundation. True fusion means that word embedding parameters, rule embedding parameters, and model backbone parameters are updated synchronously and supervised by each other during training, rather than being optimized independently.

[0057] The training data must simultaneously include: Chinese text (for generating input_ids) + rule annotations (for generating rule_ids) + Braille labels, ensuring that the two sets of embedding layers can learn aligned features based on the same batch of corpus.

[0058] Step 5: Input the fused features to the standard position of the large language model (mT5) encoder. The large language model inputs the self-attention and feedforward networks, outputs the encoded features, and then passes them to the decoder to generate Braille translation results.

[0059] Conventional translation only uses cross-entropy loss (NLLLoss). This invention adds two constraint losses to force the model to learn consistent rules, which is crucial for improving translation accuracy. The loss function is expressed as: Ltotal=α Lnll+β Lrule+γ Lembed Here, Lnll represents the basic translation loss, the standard sequence generation loss, which measures the difference between the Chinese generated by the model and the tags, ensuring the semantic accuracy of the translation; Lrule represents the rule compliance loss, which penalizes outputs that violate Braille grammar rules, detects whether the model outputs Braille conforms to grammar rules, and amplifies the loss of violating samples with weights, forcing the model to comply with rule constraints during training; Lembed represents the embedding loss, which constrains the semantic distance between the rule embedding and the corresponding Braille token embedding, ensuring that the rules are aligned with lexical features and avoiding the separation of the two types of embeddings.

[0060] The training strategy consists of three stages. The first stage trains the word embedding layer and the learnable rule embedding layer with a learning rate of 1e-4, using a small-scale dataset from the Chinese-Blind parallel corpus to quickly converge the rule vectors and connect them, fixing the backbone parameters. The second stage trains the word embedding layer, the learnable rule embedding layer, and the encoder layer with a learning rate of 5e-5, using the Chinese-Blind parallel corpus to fuse rule features with text features. The third stage trains all model parameters with a learning rate of 1e-5, using the Chinese-Blind parallel corpus and the augmented corpus for global collaborative optimization, achieving deep binding between rules and semantics.

[0061] In a specific embodiment, the gating mechanism dynamically generates weight coefficients in the range of [0, 1], automatically allocates the fusion ratio of text semantic features and braille rule features, and enables the model to adaptively determine whether to trust semantic information or grammar rules more according to the input tokens.

[0062] The gating module only contains a single linear layer (weight matrix W, bias b), which is part of the model's trainable parameters and is updated synchronously with the backbone network and the rule embedding layer.

[0063] Training phase control logic: The model automatically adjusts the parameters of the linear layer according to the loss function (translation cross-entropy loss + rule compliance loss + embedding alignment loss (optional) + pinyin alignment loss (optional)); For strongly rule-constrained tokens (such as the braille abbreviation "de", proper nouns), the gating value will approach 1, and rule features are preferred; For general semantic tokens (such as common nouns, verbs), the gating value will approach 0.5, evenly fusing the two types of features; For tokens without rules, the gating value approaches 0, and text semantic features are fully used.

[0064] In a specific embodiment, for vectors embedding pinyin and tones, during the training process, for each Chinese-braille parallel corpus, 4 groups of equally long sequence IDs (input_ids (Chinese character IDs), rule_ids (rule IDs), pinyin_ids (correct pinyin IDs), tone_ids (tone IDs)) are generated in parallel, with the length aligned to the length of the Chinese character sequence. The problem of unresolved polyphonic characters is completed through manual annotation and pinyin correction tools (such as the pypinyin context mode) for pinyin conversion. The use of single-character default pinyin is prohibited, and the correct context pinyin must be used. The four groups of equally long sequences are dynamically weighted and fused through the gating mechanism, and the fused result is fed to the encoding layer.

[0065] In a specific embodiment, the mT5-large model is selected. mT5 is a multilingual text generation model. Its pre-training covers 101 languages and adopts a unified "text-to-text" framework, which is particularly suitable for converting Chinese into structured braille output. Compared with the original T5, the hidden layer dimension of mT5-large is 1024, the feed-forward layer dimension is 2816, and both the encoder and the decoder are 24 layers. It has a powerful context modeling ability and can handle complex semantic relationships and rules involved in braille conversion (such as tone abbreviation, abbreviation rules, etc.), achieving end-to-end accurate conversion. The input and output of the model are as Figure 7 shown.

[0066] The Chinese-Braille translation task used one type of training dataset (News) and three types of external validation datasets (Web, Wikipedia, and Gold). A data overview is shown in Table 3. This invention collected 1 million Chinese sentences from the "zho_news_2007-2009_1M-sentences" file in the Leipzig Corpus and combined them with approximately 80,000 words from the *Modern Chinese Grammar Information Dictionary* provided by the Institute of Computational Linguistics, Peking University. After data cleaning and processing using an automatic Chinese-Braille translation tool, a parallel Chinese-Braille corpus (i.e., the News dataset) covering both word-level and sentence-level granularity was constructed. This dataset contains a total of 1,080,276 samples, each consisting of a Chinese character string and its corresponding Braille string.

[0067] The Web and Wikipedia datasets were derived from the "zho-simp-tw_web_2014_100K-sentences" and "zho_wikipedia_2014_100K-sentences" files in the Leipzig Corpus, respectively. Each contains 100,000 Chinese sentences and has undergone standard data cleaning processes. Because the "zho_wikipedia_2014_100K-sentences" file contains a large number of traditional Chinese characters, it was simplified to simplified Chinese characters to reduce interference with the model's recognition capabilities. Ultimately, both the Web and Wikipedia datasets contain 100,000 samples, each consisting of a Chinese character string and its corresponding Braille string.

[0068] The Gold dataset is a manually constructed gold standard corpus of Chinese-Braille. 1000 sentences were selected from the *People's Daily*, and their corresponding Braille texts were generated using an automatic Chinese-to-Braille translation tool. Three annotators then proofread each sentence to ensure accuracy. The final Gold dataset contains 1000 high-quality samples, each consisting of a Chinese character string and its corresponding Braille string.

[0069] To prevent overfitting, we divided the News dataset into training, validation, and test sets in an 8:1:1 ratio. Table 4 shows examples of each dataset.

[0070] Table 3 Dataset Overview

[0071] Table 4 Examples of datasets used in the Chinese-Blind Translation Task

[0072] In one specific embodiment, in the Chinese-Blind Translation task, this invention uses the News dataset to fine-tune the mT5-large model and evaluates its generalization ability on three external validation datasets: Web, Wikipedia, and Gold. The main hyperparameter settings of the model are shown in Table 5. During training, the model uses the AdamW optimizer with an initial learning rate of 5e-5. Validation and model checkpoint saving are performed every 5000 steps to prevent overfitting. The number of training epochs is set based on preliminary experimental results: mT5-large converges after 2 epochs, and further training has limited performance improvement.

[0073] The Chinese word segmentation task was based on the PKU dataset from SIGHAN 2005 and trained using the Bert-CRF model. The encoder used the pre-trained model bert-base-chinese, which has strong context modeling capabilities. The optimizer was AdamW, with an initial learning rate of 2e-5, an epsilon of 1e-8, and a maximum gradient norm of 1.0 to prevent gradient explosion. Linear decay was used for learning rate scheduling, with a warmup step of 1 epoch and a total of 25 training epochs. Training samples were dynamically loaded in batches to efficiently utilize GPU resources and reduce memory usage. The entire training process was completed on a single NVIDIA V100 GPU and took 5 hours.

[0074] Table 5 Model Hyperparameters

[0075] In terms of hardware configuration, mT5-large was trained on a single NVIDIA V100 GPU. During the model evaluation phase, we used BLEU (Bilingual Evaluation Understudy) as an automated evaluation metric to measure the degree of matching between the model's output translation and the reference translation at the n-gram level. A higher score indicates better translation quality, ranging from 0 to 100.

[0076] During model training, multiple metrics are used for periodic evaluation on the validation set, including Precision, Recall, F1-score, and Accuracy. The F1-score is the primary performance metric used to select the best model. At the end of each epoch, a complete evaluation is performed on the validation set, and the parameters of the best-performing model are saved.

[0077] In a specific embodiment, the mT5-C2B model (Chinese to Braille translation) performs excellently in all test sets, with good translation quality and generalization ability. Its BLEU score on the News test set is as high as 99.7809, almost achieving perfect conversion. On the Web and Wikipedia test sets, it reaches 97.8066 and 94.9131 respectively, indicating that it still maintains stable performance on heterogeneous data. On the more challenging Gold manually annotated test set, it obtains 93.0375, further verifying its robustness and practicality.

[0078] The BERT-CRF model (Chinese word segmentation) performs excellently on the SIGHAN 2005 dataset, with both precision and recall reaching 0.980, which are 0.020 and 0.033 higher than those of the previous best LTP-3.2.0 model (Precision = 0.960, Recall = 0.947) respectively. This shows that BERT-CRF is not only applicable to the Chinese word segmentation task but also takes into account prediction accuracy and coverage ability.

[0079] In a specific embodiment, to ensure the quality of the Chinese-Braille parallel gold standard corpus, we developed a strict manual correction process. Three annotators checked the Braille results generated by the automatic conversion tool based on the People's Daily corpus one by one, and mainly corrected the misjudgment of the polyphonic character in "refugee" (should be "nàn" instead of "nán") and other pinyin conversion errors. Table 6 shows the correction results after the comprehensive evaluation of the three annotators. Statistics show that the overall error rate of the automatic conversion tool is only 0.5%, indicating its high stability and reliability.

[0080] Table 6 Manual Correction Results of Chinese-Braille Parallel Gold Standard Corpus

[0081] In addition, the present invention also conducts a comparative analysis of the automatic conversion results and the manual correction results at the character level and the word level. As shown in Table 7, the character-level matching rate reaches 99.66%, and the word-level matching rate is 99.11%. Both indicators show that the Chinese to Braille automatic conversion tool performs very well in corpus generation, which also indirectly proves that the data quality of the Chinese-Braille parallel corpus and the Chinese-Braille parallel gold standard corpus is high, providing a solid data foundation for subsequent model training and evaluation.

[0082] Table 7 Comparative Analysis Accuracy of Chinese to Braille Automatic Conversion Results and Manual Correction Results

[0083] This invention expands the vocabulary before model fine-tuning to improve the recognition capability of Braille characters. After expansion, the model can cover all 62 basic Braille characters. Taking mT5-large as an example, the BLEU score improved by 0.5532 after vocabulary expansion, verifying the positive effect of vocabulary expansion on improving model performance.

[0084] The results of comparing the mT5-large model of this invention with other models in the Chinese-Blind Translation Conversion Task are shown in Table 8. It can be found that the model of this invention has the best results, achieving an accuracy of 99.78%.

[0085] Table 8

[0086] The present invention also discloses a computer program product or system, including a computer program that, when executed by a processor, implements the above-described method steps.

[0087] Figure 2 The schematic diagram of the rule-based Chinese-to-English translation system provided in this embodiment of the invention specifically includes: Acquisition Unit: Acquire the Chinese character data to be translated; Translation unit: Inputs the Chinese character data to be translated into a large language model for translation to obtain Braille translation; The large language model includes a word embedding layer, a learnable rule layer, an encoding layer, and a decoding layer. The Chinese characters to be translated are input to the word embedding layer to generate word vectors, and to the learnable rule layer to generate rule vectors. The word vectors and rule vectors are fused to obtain a constraint-enhanced feature vector. The constraint-enhanced feature vector is input to the encoding layer for encoding, and then to the decoding layer for decoding to obtain the Braille translation.

[0088] Figure 3 An embodiment of the present invention provides a schematic diagram of a computer device, specifically including: A memory and a processor; the memory is used to store program instructions; the processor is used to invoke the program instructions, when the program instructions are executed according to any of the above-described methods for Chinese-Blind Translation based on the combined rules.

[0089] The present invention also discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, a method for Chinese-Blind Translation based on any of the above-described combination rules is provided.

[0090] The verification results of this verification embodiment show that assigning inherent weights to indications can improve the performance of this method compared to the default settings. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated; the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0091] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0092] The computer device provided by the present invention has been described in detail above. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A rule-based method for Chinese-English translation for the illiterate, characterized in that, include: Obtain the Chinese characters to be translated; The Chinese characters to be translated are input into a large language model for translation to obtain Braille translation; The large language model includes a word embedding layer, a learnable rule layer, an encoding layer, and a decoding layer. The Chinese characters to be translated are input to the word embedding layer to generate word vectors, and to the learnable rule layer to generate rule vectors. The word vectors and rule vectors are fused to obtain a constraint-enhanced feature vector. The constraint-enhanced feature vector is input to the encoding layer for encoding, and then to the decoding layer for decoding to obtain the Braille translation.

2. The method for combining rules in Chinese-English translation for the blind according to claim 1, characterized in that, The learnable rule layer maps and matches the converted Chinese character data with the corresponding rule ID through a mapping table to generate a rule vector; the rules include any one or more of the following: no applicable rules, rules for initial consonants forming syllables, rules for final vowels forming syllables, special spelling rules, rules for omitting tones, punctuation matching rules, rules for linking proper nouns, rules for abbreviating vocabulary, rules for linking and separating words, and custom extended rules.

3. The method for combining rules in Chinese-English translation for the blind according to claim 1, characterized in that, The learnable rule layer also includes pinyin and tone embedding. After converting Chinese character data into pinyin, it is converted into pinyin vectors and tone vectors. The rule IDs corresponding to pinyin and tone are obtained through a mapping table. The rule IDs of Chinese characters, pinyin and tone are used to generate rule vectors. The pinyin vector, tone vector, word vector and rule vector are fused to obtain the second constraint enhancement feature vector. The second constraint enhancement feature vector is input to the encoding layer for encoding, and then decoded through the decoding layer to obtain Braille translation.

4. The method for combining rules in Chinese-English translation for the blind according to claim 2, characterized in that, The construction of the mapping table includes: obtaining a Chinese character dataset, segmenting the Chinese character data into words, associating the segmented words with any one or more of the following rules: no applicable rules, proper noun linking rules, vocabulary abbreviation rules, linking and splitting rules, punctuation matching rules, and custom extension rules, and / or converting the segmented words into pinyin, associating the pinyin with any one or more of the following rules: initial consonant self-syllable rules, final vowel self-syllable rules, special spelling rules, and tone omission rules, assigning a globally unique ID to each independent rule, and obtaining the mapping table.

5. The method for combining rules in Chinese-English translation for the blind according to claim 1, characterized in that, The training process of the large language model is as follows: acquire Chinese character data and Braille labels; segment the Chinese character data into word vectors; pass the segmented Chinese characters through a learnable rule layer to obtain rule vectors; fuse the word vectors and rule vectors and input them into the encoder and decoder for translation to obtain translated Braille; calculate the loss function until the loss function remains unchanged to obtain the large language model; the loss function includes translation cross-entropy loss, rule compliance loss, and embedding alignment loss; the embedding alignment loss is obtained by calculating the distance between word vectors and rule vectors; the rule compliance loss is obtained by detecting the correspondence between translated Braille and rule vectors; and the translation cross-entropy loss is obtained by the correspondence between translated Braille and Braille labels.

6. The method for combining rules in Chinese-English translation for the blind according to claim 1, characterized in that, The fusion is obtained by dynamically assigning weights to vectors of different dimensions through a gating mechanism and then summing them by weight. Rule embedding weights are added to the rule-constrained vectors, and pinyin and tone embedding weights are added to the polyphonic character vectors.

7. The method for combining rules in Chinese-English translation for the blind according to claim 1, characterized in that, The encoding layer includes K encoders, and the decoding layer includes K decoders. Each encoder or decoder sequentially includes a self-attention layer, a first-batch normalization layer, a forward propagation layer, and a second-batch normalization layer. In the first-batch normalization layer, the output of the previous layer and the initial input are fused through residual connections. In the second-batch normalization layer, the output of the previous layer and the output of the first-batch normalization layer are fused through residual connections. K is a natural number greater than 1.

8. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by a processor to implement the method for combining rules for Chinese-Blind translation as described in any one of claims 1-7.

9. A computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The computer program or instructions are executed by a processor to implement the method for combining rules for Chinese-Blind translation as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by a processor to implement the method for combining rules for Chinese-Blind translation as described in any one of claims 1-7.

Images