Chinese-blind conversion method based on consecutive character vector fusion Transform-GRU

Abstract

The invention relates to a Chinese-blind conversion method of Transform-GRU based on consecutive character vector fusion, and belongs to the field of natural language processing and generation. According to the method, firstly, a word embedding technology for optimizing word segmentation continuous writing rules is realized through a multi-layer attention linear network, it is guaranteed that a model learns word vectors containing rich word segmentation rules, and compared with a general embedding technology, the method has higher pertinence for Chinese-blind translation; a Markov word segmentation model based on Gaussian normalization is designed, accurate word segmentation is achieved, and continuous word vectors are fused; according to the encoder design, high-level semantic information extraction is realized by an improved encoder formed by stacking a plurality of unidirectional extension layers and local attention layers, so that the problem of high complexity of a Transform network is improved, and targeted optimization is realized on Chinese and braille alphabets with consistent word orders; and the decoder designs a mask grouping self-attention mechanism and a GRU layer fused by bidirectional mutual attention gating, so that the capability of capturing rich semantics of coding information and the reasoning capability of the network can be improved.

Description

Technical Field

[0001] The invention relates to a Chinese-Blind conversion method based on a Transformer-GRU of connected word vector fusion, belonging to the technical field of natural language processing. Background Art

[0002] As a large group of people with disabilities, the health of the blind is an urgent problem to be solved in promoting "Healthy China". Effective digitization of Braille will greatly facilitate the lives of the blind and enhance their sense of participation in society. Therefore, the digitization of Braille is urgent. To solve this problem, we first need to increase the promotion and social recognition of Braille and make it a powerful tool for blind people to learn. By digitizing Braille, we can provide the blind with more convenient and efficient learning methods and help them complete their education better. Digital technology can also help the blind better integrate into society and participate in various social and cultural activities through channels such as the Internet. Overall, committing to the digitization of Braille is an important step in promoting the full integration of the blind into society and improving their quality of life.

[0003] The most important task in the digitization of Braille is a high-performance Chinese-Blind conversion method. The current problems with Chinese-Blind conversion are: high error rate of word segmentation and continuous writing, and low accuracy of universal Braille conversion. my country has been vigorously promoting universal Braille in recent years. Compared with the current Braille, universal Braille completely solves the problem of tone ambiguity and can help the blind achieve a better reading experience. Practical Chinese-Blind conversion methods can greatly promote the promotion of universal Braille in my country. Summary of the invention

[0004] Based on the above-mentioned deficiencies, the present invention proposes a Chinese-Blind conversion method based on Transformer-GRU fusion of connected word vectors, which improves the accuracy of Chinese-Blind conversion.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions.

[0006] A Chinese-Blind conversion method based on Transformer-GRU with continuous writing vector fusion, comprising:

[0007] Step 1: Construct a sliding window-based masked attention Braille spelling rule word vector embedding network, including: a masked attention mechanism based on a symbol table, a judgment module based on grouped linear transformation, and a negative exponential two-dimensional inverse cross entropy loss module;

[0008] Step 1.1: Design a masked attention mechanism based on a comparison table, collect Chinese sentences after common Braille segmentation and concatenation from the Chinese-Braille segmentation comparison corpus, and the corresponding sentences that only use segmentation but not concatenation. Use a window of skip_window=2 to intercept five consecutive words in the sentence in turn, fill the missing part with padding and perform one-hot encoding. The input is X=[x i-2 ,x i-1 ,x i ,x i+1 ,x i+2 ], labelled as y i . Design a random mask self-attention mechanism module. Masking X here is to shield the influence of padding and punctuation on the attention score. Punctuation is written together with other parts in Braille. If its influence is not shielded, it is easy to cause the problem of sparse Braille words. Design the mask matrix attention score calculation as follows:

[0009]

[0010] Wherein, ○ is the Hadamard product, Q, K, V are the query matrix, key value matrix, and weight matrix of the input. Mask matrix M j as follows:

[0011] M j =[a ij ],i,j∈Z[0,5]

[0012]

[0013] Punlist is a punctuation table. This design enables the network to further model the relationship between words in the input Chinese corpus, and shield the interference of useless information through the mask matrix, so as to learn the braille connection rules.

[0014] Step 1.3: Design a grouped linear transformation to map each word vector to a low-dimensional space, and increase the nonlinear capability through the ReLU function to increase the information density of Braille in each dimension.

[0015]

[0016] Among them, split represents the split operation, W1 and W2 represent the fully connected network, that is, x j Split into x by dimension j1 With x j2 The two parts are connected through the fully connected network, which can greatly improve the computing efficiency. This layer of network can globally consider the input vector [u i-2 ,u i-1 ,u i ,ui+1 ,u i+2 ], through training iterations, the output of this layer is given Contains high-dimensional Braille information.

[0017] Step 1.4: Concatenate the results obtained in step 1.3, and use a fully connected layer to map the summation result to a one-dimensional space, and then use the sigmoid function to map the result to between [0,1].

[0018]

[0019] Where σ represents the sigmoid function. The final output probability indicates whether there is a word near the central character that needs to be connected.

[0020] Step 1.5: Design a negative exponential two-dimensional inverse cross entropy loss function. Assuming that the number of batches is n, the loss function is as follows.

[0021]

[0022] in, is the probability output obtained above, and P is the true value of the label.

[0023] Step 2: Freeze the network parameters trained in steps 1.2 to 1.3 as the continuous writing feature extractor, and take it out separately to connect to the subsequent Chinese-Blind conversion network. Design the Transformer-GRU model based on continuous writing vector fusion in the encoder-decoder framework to achieve Chinese-Blind translation. Among them, the encoder part is composed of two layers of masked local self-attention and one layer of global Gaussian self-attention mechanism stacked.

[0024] Step 2.1: First, design a Gaussian normalized Markov model for word segmentation. The hidden state space of the input character is described as:

[0025] SS={B,M,E,S}

[0026] Among them, B is the starting character, M is the middle character, E is the end character, and S is a separate character.

[0027] Design a word chunk scoring model:

[0028]

[0029] Among them, N represents the total number of statistical words in the corpus, and V represents the number of words belonging to x. i-1 The denominator σ is the variance of the number of words in the corpus vocabulary, and μ is the mean number of words in the vocabulary. i |x i-1 ,x i-2 ,...,xi-j ) is the co-occurrence probability, assuming that the input Chinese character vector sequence is X = {x1, x2, x3, ..., x n}, starting from the second character, calculate the co-occurrence probability of a word with the previous module. That is:

[0030]

[0031] If and only if: Scores(i)≥Scores(i-1),i=2,3,...

[0032] x i Take the word block as a whole, and weight the word vectors corresponding to the word block to get the word vector containing the continuous writing information, that is:

[0033]

[0034] The output is Among them, m≤n. The input Chinese characters will be segmented.

[0035] Step 2.2: Design a diagonal masked local attention mechanism model based on word order consistency to mask long-distance information, so that the network can focus on nearby input Chinese words during training. The attention score is calculated as follows.

[0036]

[0037] Among them, mask is the designed mask matrix, which is used to shield the Chinese characters whose distance from the central Chinese character is greater than 2-4. The mask matrix mask is as follows.

[0038]

[0039]

[0040] Among them, W1 and W2 are linear networks. Through grouped linear transformation, the network can be prevented from learning too many complex semantic relationships, and focus on the learning of tone omission and connected writing modeling.

[0041] Step 2.3: After the local attention layer, the grouped feedforward connection network is connected to encode and integrate the attention scores. The local attention and grouped feedforward networks together form an extension layer. The number of extension layers included in the shallow network can be customized by the user in the specific implementation. The kth row of the matrix is ​​named Indicates a new token, each line Perform the following transformation.

[0042]

[0043] Among them, χ k is the kth row of the output matrix of the grouped feedforward connected network. This layer can integrate the Braille rule encoding representation.

[0044] Step 2.4: In order to balance the network's learning weights for word segmentation rules and tone omission, and to prevent overfitting, a residual connection model with learnable weight parameters based on upper and lower bounds is designed as follows:

[0045]

[0046]

[0047] Among them, Φ(input) represents the generalized function represented by the first layer of expansion layer, Ψ(γ k ') represents the generalized function represented by the second extended layer. γ1 and γ2 are learnable parameters, u and s are user-defined hyperparameters, u is the upper bound, and s is the lower bound. By adjusting the upper and lower bound parameters to protect the pre-trained Braille splicing rule modeling, it is ensured that the rules contained in the splicing word embedding vector occupy a certain proportion.

[0048] Step 2.5: A Gaussian mask self-attention mechanism based on global sequence information is designed. The attention calculation scheme of the present invention is as follows.

[0049]

[0050] Among them, ○ is the Hadamard product, and Υ is the Gaussian distance mask matrix.

[0051]

[0052] This mask matrix incorporates sequence information into the attention mechanism. This is because Chinese characters and Braille have the same word order. Through the Gaussian distance mask matrix, the network can integrate the encoded representation with the decoded representation and learn the tone marking and connected writing rules.

[0053] Step 2.6: Each row of η i Take it out as a new token, pass it through the grouped linear transformation network and residual link.

[0054]

[0055] Further obtain the tokenγ i Make stitching.

[0056]

[0057] Step 3: The shallow network of the decoder needs to integrate the input Braille code and design a masked group self-attention mechanism to shield future information and information that is too far away to ensure that the network's attention is limited to the 5-gram range, so that the network allocates more attention weights to tokens from the same Braille unit. At the same time, a grouped linear transformation residual connection is designed to protect the positional encoding information in the shallow semantics.

[0058] Step 3.1: During the training phase, the external input of the decoder is the masked Braille text. Assume that the decoder external input embedding is Y = [y1…y m ], the attention score matrix is ​​calculated as follows:

[0059]

[0060] in, The matrix is ​​used to shield future information. The matrix is ​​used to assign attention weights, and the design rules are as follows:

[0061]

[0062]

[0063]

[0064] Will Each row of the matrix is ​​denoted as κ i The matrix is ​​transformed through a grouped linear transformation mechanism and a residual connection to extract depth information:

[0065]

[0066] Step 4: The present invention designs a bidirectional mutual attention gated fusion GRU unit to output γ at the encoding end i With the decoder input The mutual attention gate is used to model the deep correspondence information between the abstract semantics of Chinese and Braille. The GRU unit can give the network reasoning ability to avoid overfitting.

[0067] The memory output of the bidirectional mutual attention gated fusion GRU unit at the current time step is Assume that in step t, the output corresponding to the above encoding end is γ t =γ t ∈[γ1…γ n ], and the corresponding output of the decoder is The final output calculation formula is as follows:

[0068]

[0069] Among them, zt is the gate update at time t, r t Reset the gate for time t.

[0070] Then the updated gate model is as follows:

[0071]

[0072] The reset gate settings are as follows:

[0073]

[0074] Beneficial Effects

[0075] A masked attention word embedding network based on a variable sliding window is designed as a pre-training module. This module solves the problem that the previous word vectors contain almost no information about connected writing when used for Chinese-Blind conversion tasks, and drives the subsequent network to focus on how to connect writing during the training process, which specifically overcomes the biggest characteristic difficulty of the Chinese-Blind conversion task. The masked attention mechanism is different from the traditional word vector that directly connects the input to the fully connected layer. This design enables the output vector of the attention module to contain contextual information, thereby forcing the subsequent group connection layer to focus on the relationship between words and accelerate the learning of connected writing relationships. The two-dimensional negative exponential cross entropy used has a faster convergence speed than the original cross entropy loss.

[0076] The Gaussian normalized Markov word segmentation scheme designed by the present invention can more effectively judge whether new characters should be included in the word block as a segmentation whole by calculating the word block score. The encoder part is stacked with two layers of masked local self-attention and one layer of global Gaussian self-attention mechanism. The masked local self-attention module utilizes the prior information that Chinese and its corresponding Braille have the same word order. At the same time, the punctuation marks can be effectively identified through the punclist mask matrix, avoiding the sparsity problem caused by the continuous writing of Braille punctuation marks, forcing the network to focus on the modeling of the relationship between words in this part, so that it is easier to learn the rules of tone omission. For very few cases, such as two-part allegorical sayings, ancient poems, etc., the network needs the information of the whole sentence to judge whether syllables need to be omitted or continuous, but still follows the principle that the weight decreases as the distance from the center word increases. Therefore, compared with the general mask matrix, the Gaussian mask matrix network is more effective in distracting attention. Different from the traditional residual connection, the residual connection based on the learnable parameters of the upper and lower bounds can be customized by the user to avoid the infinite problem, and the residual weight can be dynamically balanced by the network to protect the shallow or deep information. The global Gaussian self-attention module is designed deeper in the network to make full use of deep features.

[0077] The decoder of the present invention designs a grouped linear transformation and bidirectional mutual attention gated fusion GRU under the self-attention mechanism. Compared with the traditional fully connected layer, the grouped linear transformation layer can effectively reduce the number of parameters. Combined with self-attention, it can alleviate the "dimensionality disaster" problem to a certain extent, play the role of multi-head attention, and greatly increase the training speed. The attention gating mechanism can deeply extract the weight correspondence between Chinese and Braille, and dynamically judge its application method through the weight gate and update gate, and finally participate in the GRU unit modeling output, improve the network reasoning ability, and avoid overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 : Overall flow chart;

[0079] Figure 2 : Pre-training module - linked word embedding network;

[0080] Figure 3 : Encoder part - masked self-attention and global Gaussian self-attention mechanism module;

[0081] Figure 4 : Decoder part - bidirectional gated fusion encoding network under self-attention mechanism;

[0082] Figure 5 : Encoder - Example sentence into positional encoded spelling embedding vector heat map;

[0083] Figure 6 : Encoder-input word vector dimension curve;

[0084] Figure 7 : Encoder-Word order consistency diagonal mask self-attention matrix heat map;

[0085] Figure 8 : Encoder-global Gaussian self-attention matrix heat map;

[0086] Fig. 9 : Decoder—GRU unit output heat map. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific examples in conjunction with the accompanying drawings.

[0088] like Figure 1 As shown, a Chinese-Blind conversion method based on Transformer-GRU with continuous writing vector fusion includes:

[0089] Step 1: In the pre-training phase, the word segmentation-concatenation comparison dataset is used to train the word embedding network. Figure 2As shown in the figure, the dimension of the input one-hot vector is set to the number of dictionary characters in the statistical data set, and the final output dimension of the attention layer and grouped linear transformation layer involved in the subsequent network calculation is set to 160. In order to avoid overfitting problems in the subsequent network training process, a .txt Chinese-Blind comparison dataset under different corpora is used for subsequent network calculations. The attention layer of this module does not add a multi-head mechanism because the training complexity of the continuous writing vector is much lower than that of the general word vector, and the multi-head mechanism will disperse the attention to different dimensions, making it difficult to extract global information. The number of pre-training rounds is EPOCH_pretrain=200, the initial learning rate is set to 1, and it is reduced by half every 20 rounds of iterations.

[0090] Step 2: The parameters for creating the model in the formal training phase are as follows: padding placeholder index is 0, unregistered word index is 1, batch size is 256, number of training rounds EPOCH is 2000, number of unidirectional expansion layers in encoder and decoder is 4, number of multi-head attention is 4, number of linear transformation groups of attention mechanism grouping is 4, number of feedforward network grouping is 2, input and output ligature vector dimensions are 180, dimension of feedforward network fully connected layer is 1024, dropout ratio is 0.03, maximum sentence length MAX_LENGTH is 40. The optimizer uses a custom NoamOpt, in which the learning rate reduction factor is 0.4 and the warmup factor is 3000. The training time is 11347sec, and the final loss is 0.00785. BPE with num_merge=500 is used to split the original unit to remove the word sparsity problem caused by braille ligature.

[0091] Step 3: In the test phase after the model is trained, the Chinese sentence "He can play both tennis and baseball." is input. First, the Chinese dictionary is used to convert the sentence into an ID number "[2,6,140,18,37,141,47,23,142,18,37,143,47,4,3]" by character. The first and last dimensions are the IDs of the start character "BOS" and the end character "EOS". The id number sentence is then embedded into the network through continuous writing to obtain the continuous writing vector, which is weightedly fused with the position encoding vector. The final vector heat map is shown in the attached figure. Figure 5 As shown, its dimension curve is as follows Figure 6 shown.

[0092] Step 4: In the Gaussian-normalized Markov word segmentation layer, the statistics of covariance probability are based on the open-source non-domain supervised HMM training set. The Gaussian normalization module assumes that in the actual corpus, the number of characters contained in the word block conforms to the Gaussian distribution with an expectation of 2, and the value of the variance σ depends on the corpus statistics, so that the probability density is placed in the denominator to offset the effect of the increase in the number of characters on the score weight. The Gaussian-normalized Markov model is used to segment "He can play both tennis and baseball." and the segmentation sentence ['BOS', 'he', 'both', 'play tennis'', ', 'and', 'play baseball', '. ', 'EOS'] is obtained. The vectors of the connected words belonging to the same word block are weighted and fused to obtain the vector encoding representation of the connected words.

[0093] Step 5: The word vector encoding representation is passed through the encoder, such as Figure 3 As shown in the figure, the encoder is composed of two expansion layers and a stack of global sequence information Gaussian mask self-attention + grouped linear transformation residual networks. Each expansion layer contains a diagonal mask attention mechanism model based on word order consistency, a grouped feedforward connection network, and a residual connection based on upper and lower bounds of learnable parameters. The word vector encoding layer first passes through a diagonal mask attention mechanism model based on word order consistency. This model assigns attention weights to word vectors within the 5-gram range, allowing the network to extract shallow layer connection relationships. The resulting heat map is shown in the figure below. Figure 7 shown.

[0094] Step 6: Next, the vector units are connected to the network through group feedforward. The group linear transformation can isolate the dependence of the fully connected layer on dimensional features and avoid the "curse of dimensionality".

[0095] Step 7: The vector units are connected through residual connections with learnable parameters based on upper and lower bounds. In the first expansion layer, The function is set before the input because the information extracted by Φ(input) is relatively shallow at this time. Increasing the weight of the input part will not destroy the information extracted by the network itself, but will help preserve the continuous information. In the second expansion layer, the information extracted by Ψ(Y1') is more abstract, so The function is set before Ψ(Y1') to protect the deep encoding information. In this task, the hyperparameter μ is set to 2, s is set to 1, and the residual coefficient learned by the network is It is 1.567.

[0096] Step 8: The network passes through the global sequence information Gaussian mask self-attention + grouped linear transformation residual layer. The deep network vector itself contains a lot of semantic relationship information, but it weakens the position relationship information. Through the global sequence information Gaussian mask self-attention, the network can encode the relative position between words and integrate the deep semantic information. Finally, the encoder outputs a heat map as shown below Figure 8shown.

[0097] Step 9: The decoding layer is composed of two expansion layers and bidirectional mutual attention gated fusion GRU units stacked together. Figure 4 As shown in Figure 1, to avoid word sparsity, all word chunk units in the training set are segmented by BPE after shielding the influence of punctuation marks, and then pass through two expansion layers after the embedding layer. The 2-gram mask grouping self-attention mechanism can integrate tokens that originally belong to the same continuous word chunk and shield future information. The final GRU unit output is as follows: Fig. 9 As shown in the figure. The final output Braille ASCII code of the example sentence "He can play both tennis and baseball." is "tg ihw2d9`7`k~"~2hw2d9`b 8k~"2". Since the Braille training set is segmented by BPE, the is the end of the segmentation unit. In order to avoid format problems during training, special characters need to be escaped. "'" is escaped to "`", and " / " is escaped to "~".

[0098] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU, characterized in that The steps of the method include: Step 1: Optimize the word embedding of the word segmentation and concatenation rules through a multi-layer attention linear network; Step 2: The Chinese sentence is passed through the multi-layer attention linear network trained in step 1 to obtain a continuous word vector embedding matrix, the continuous word vector embedding matrix is ​​passed through the Gaussian normalized Markov word segmentation model in the encoder to fuse the word vectors to obtain a word vector matrix, and the word vector matrix is ​​passed through multiple expansion layers to obtain the encoder output; In step 3, the predicted Braille is used as the decoder input, and the self-attention encoding matrix is ​​obtained through two expansion layers. The self-attention encoding matrix and the encoder output obtained in step 2 are fused with the GRU through bidirectional mutual attention gate to obtain the final Braille output.

2. According to claim 1, the Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU is characterized by: In the step 1, the multi-layer attention linear network is a masked attention Braille spelling rule word embedding vector network based on sliding window; The training set used for optimization is the original Chinese word segmentation text and the corresponding Chinese word segmentation connected text extracted from the Chinese-Blind comparative corpus; The training set is represented by X = (x1, x2, ..., x n ), n is the length of the sentence, and the part of the sentence that is less than n is padded with padding. In the training stage, a sliding window is used to intercept five consecutive words X = (x i-2 ,x i-1 ,x i ,x i+1 ,x i+2 ), the final output is the predicted probability Prediction probability Indicates the probability that the middle word and other words come from the same continuous writing module. The final pre-trained network is obtained through the user-defined number of iterations. In the application stage, the probability output layer is removed, and five consecutive words in the sentence are intercepted in turn based on the sliding window to maintain consistency, and finally the continuous writing word embedding matrix is ​​obtained.

3. The Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU according to claim 2 is characterized by: In the step 1, the design method of the sliding window-based masked attention braille regular word embedding vector network is: The first step is to encode the training set with one-hot word vectors. First, the masked attention calculation shown in formula (1) is performed on the five consecutive encoded vectors. The row vector of the attention matrix is ​​used as the deep semantic vector. The mask matrix M j As shown in formula (2): Among them, Attention(Q,K,V) is the attention score matrix, which can be decomposed row by row as is the five consecutive words cut off by the window; Q is the query matrix, K is the key matrix, and V is the value matrix; d k is the dimension of the word vector, Punlist is the punctuation table, is the Hadamard product; The second step is to map the token to a low-dimensional space through the grouped linear transformation expressed by formula (3) to increase the information density; Among them, concat represents the concatenation operation, split represents the splitting operation, and W1 and W2 represent u j Fully connected networks connected by different feature dimensions; The third step is to Generate the output probability through formula (4); Among them, σ is the sigmoid function; The fourth step is to calculate the loss using the negative exponential two-dimensional inverse cross entropy loss function shown in formula (5); Wherein, L is the loss function, is the probability output obtained in the third step, and P is the true value of the label.

4. The Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU according to claim 3 is characterized by: In the step 2, the method of embedding the connected word vector into the matrix through the Gaussian normalized Markov word segmentation model in the encoder to fuse the word vector to obtain the word vector feature is: The first step is to pass each character through the chunk scoring function shown in formula (6); Among them, N is the total number of words in the corpus, and V represents the words belonging to x. i-1 The number of state-type words, σ is the variance of the number of words in the corpus vocabulary, and μ is the mean number of words in the corpus; The second step is, if and formula (7) holds, set x i Include the current word block as a whole; when it is not true, i As the starting word of the next word block unit, and repeat formulas (6) and (7) to calculate the words contained in the next word block unit; Scores(i)≥Scores(i-1),i=2,3,... (7) The third step is to fuse the connected character vector units into connected word vectors using formula (8); Here, sum(·) represents the vector sum operation.

5. The Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU according to claim 4 is characterized by: In step 2, the extension layer in the encoder is designed as follows: In the first step, the diagonal mask local attention mechanism is used to mask out long-distance information. in, is the obtained attention score matrix, which can be decomposed row by row as is the encoding vector obtained after fusing the word vectors, is the query matrix, is the key matrix, is a value matrix, and the mask calculation method is shown in formulas (10) and (11); The second step is to integrate the attention scores through formula (12); Among them, H1 and H2 represent fully connected networks, and m represents the number of word blocks contained in the sentence; The third step is to protect the proportion of the continuous writing vector through formula (13); Among them, u and s are artificially set hyperparameters, indicating the The upper and lower bounds that can be achieved; The fourth step is to use formula (14) to calculate the continuous writing vector γ which contains more semantic information. k 'The mutual weights are integrated; in, is the attention score matrix, which can be decomposed row by row as The output of the third step, G calculation method is shown in formula (15), which can increase the weight of neighboring words in the attention matrix, conform to the principle of consistent word order of Chinese and Braille, and re-encode the relative distance information; Step 5: Based on formula (16), the residual network is used to avoid performance degradation. Where m is the number of word blocks contained in the sentence.

6. The Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU according to claim 5 is characterized by: In step 3, the extension layer in the decoder is designed as follows: The first step is to calculate the token attention scores within the BPE word and between adjacent words based on formula (17); in, To calculate the attention score matrix, it can be decomposed row by row as and The calculation method is shown in formulas (18) and (19); M i The size of is the number of BPE segmentation units contained in the three adjacent chunk units; The second step is to calculate the residual connection unit vector based on formula (20); Where m is the number of word blocks in the sentence.

7. The Chinese-Blind conversion method based on connected word vector fusion Transformer-GRU according to claim 6 is characterized by: In step 3, the method of obtaining the final Braille output by bidirectional mutual attention gated fusion GRU is: In the first step, the encoder output and the decoder token are fused through formula (21) to obtain the update gate parameters; Among them, the superscripts t and t-1 represent the time steps, σ is the sigmoid function, is the query value matrix, is the key value matrix, h t-1 The hidden output of the previous time step is z t is a number between [0,1], which determines how much content needs to be remembered. (z) is the fusion matrix; The second step is to fuse the encoder output and the decoder token through formula (22) to obtain the reset gate parameters; Among them, U (r) To reset the matrix, the reset gate determines how much of the content needs to be retained; The third step is to obtain the final output through formula (23); Among them, Ω is the transformation matrix, which is used to transform the hidden output h of the current time step t Convert to the current time step and output the Braille result

Images