A method for automatic conversion from Braille to Chinese

Abstract

The present invention relates to a method for automatic Braille-to-Chinese conversion, comprising: constructing a CBHG-SE model; training the CBHG-SE model based on a corpus consisting of Braille and corresponding Chinese text to obtain a trained CBHG-SE model; and converting an input Braille sentence based on the trained CBHG-SE model to obtain a Chinese translation of the input Braille. The proposed CBHG-SE model effectively solves the problem of single-step conversion from Braille to Chinese characters, letters, numbers, and punctuation, achieving a high level of conversion accuracy and achieving good results in converting Braille sentences of varying lengths into Chinese.

Description

Technical Field

[0001] The present invention relates to the technical field of computer word processing, in particular to a method for automatic conversion from Braille to Chinese. Background Art

[0002] Conversion from Chinese Braille to Chinese refers to the use of information technology to convert Braille into plain text forms such as Chinese characters, English, punctuation, and numbers. Braille consists of 64 squares, each of which is represented in a computer by its unique Braille ASCII code. A Braille string is input into the computer using the Braille ASCII code, and a Chinese string is generated through the conversion system. Since Braille only has Braille characters, while the target language, Chinese, contains Chinese characters, numbers, letters, and punctuation, the conversion process presents a one-to-many problem. Existing Braille-to-Chinese conversion generally uses statistical machine translation, which requires analyzing the characteristic patterns in parallel corpora and constructing a rule base. Existing Braille-to-Chinese conversion research faces the following challenges: First, existing models employ multi-step Braille-to-Chinese conversion, resulting in complex algorithms that affect translation efficiency and accuracy; second, it is difficult to simultaneously convert Braille into Chinese characters, English, punctuation, and numbers; and third, existing conversion models suffer from outdated algorithms, making it difficult to achieve effective conversion results. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for automatic conversion from Braille to Chinese to solve the defects in the above-mentioned prior art.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for automatic conversion from Braille to Chinese, comprising:

[0006] Constructing a CBHG-SE model, and training the CBHG-SE model based on a corpus consisting of Braille and corresponding Chinese to obtain a trained CBHG-SE model;

[0007] The input Braille sentence is converted based on the trained CBHG-SE model to obtain a Chinese translation result of the input Braille.

[0008] Preferably, constructing the CBHG-SE model includes: adding an SE module to the CBHG model to perform feature recalibration to obtain the CBHG-SE model, which is used to achieve mutual correlation between channels and enhance the feature extraction capability of the model.

[0009] Preferably, before training the CBHG-SE model, data preprocessing is included, and the data preprocessing process includes: generating a dictionary, adding tags, dictionary conversion, and dividing the data set; wherein the generating dictionary is used to generate a Chinese character dictionary and a Braille ASCII code dictionary by traversing the Chinese text and Braille text in the corpus, and obtain the maximum length of the Braille sentences and the Chinese sentences; the adding tags is used to add a start tag and an end tag to each of the Braille sentences and the Chinese sentences; the dictionary conversion is used to convert the Braille sentences and the Chinese sentences into coded representations through the Chinese character dictionary and the Braille ASCII code dictionary; and the dividing the data set is used to divide the converted sentences into a training set and a test set.

[0010] Preferably, each control corpus in the corpus contains a Chinese character string and a Braille ASCII code character string, and also contains Chinese, alphabetic, numerical, and punctuation character types; wherein, the Chinese sentences and their corresponding Braille ASCII code sequences are both submitted to Braille experts for proofreading to ensure the standardization and accuracy of the corpus.

[0011] Preferably, inputting the training set into the CBHG-SE model for training comprises: inputting the Braille sentences in the corpus into the CBHG-SE model one by one for training.

[0012] Preferably, the CBHG-SE model includes: a group of one-dimensional convolution filters, SE modules, residual connection operation modules, Highway networks and bidirectional GRU networks, wherein the one-dimensional convolution filters, the SE modules, the residual connection operation modules, the Highway networks and the bidirectional GRU networks are connected in sequence.

[0013] Preferably, predicting the input Braille sentence based on the trained CBHG-SE model includes:

[0014] The test set is input into the trained CBHG-SE model. The Braille sentences in the test set are encoded as the input sequence. The sequence is first convolved with K groups of one-dimensional convolution filters, and features are recalibrated through the SE module to obtain a one-dimensional convolution result. The one-dimensional convolution result is added to the original input sequence through a residual connection. The result obtained after the residual connection is input into the Highway network for extracting higher-level features. Finally, the feature vector obtained after the Highway network is input into the bidirectional GRU network to extract sequential features from forward to backward and reverse directions. The trained model ultimately achieves Braille-Chinese conversion.

[0015] Preferably, the Highway network is used to perform nonlinear transformation on the results obtained after the residual connection in proportion and then add them together.

[0016] Preferably, feature recalibration is performed by the SE module, including: generating a global descriptor by using an original feature map generated by encoding, convolution and pooling the Braille sentence through a squeezing operation, performing an excitation operation, learning channel attention based on a channel-dependent gating mechanism, and then reweighting the feature map to generate the output of the SE module to complete feature recalibration; and then combining the output result with a residual connection operation module; wherein the squeezing operation is a global average pooling method, the global descriptor aggregates information from all channels, and the original feature map is generated by encoding, convolution and pooling the Braille sentence.

[0017] Preferably, the process of performing the excitation operation includes a fully connected layer and an activation layer, wherein the fully connected layer is used to compress the global information obtained after global average pooling, and restore the compressed vector to obtain a weight vector; after the excitation operation is completed, the obtained weight vector is multiplied by the feature map, so that each feature map obtains global correlation features with other channels.

[0018] The beneficial effects of the present invention are:

[0019] (1) The CBHG-SE model proposed in this paper solves the problem of single-step conversion from Braille to Chinese, letters, numbers, and punctuation, achieves a high level of conversion accuracy, and also achieves good results in converting Braille sentences of different lengths into Chinese.

[0020] (2) The present invention can not only effectively promote communication between the blind and sighted people, but also has important significance for related work such as the promotion of Braille. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a structural diagram of the CBHG-SE model according to an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of the SE module structure of an embodiment of the present invention;

[0024] Figure 3The performance of each model in the embodiments of the present invention in large and small data sets;

[0025] Figure 4 Schematic diagram of BLEU values ​​of various weights for pure Chinese characters according to an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of BLEU values ​​of various weights under mixed sentences according to an embodiment of the present invention;

[0027] Figure 6 This is a schematic diagram of BLEU values ​​for different models on sentences of different lengths according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] A method for automatic conversion from Braille to Chinese, comprising:

[0031] Constructing a CBHG-SE model, and training the CBHG-SE model based on a corpus consisting of Braille and corresponding Chinese to obtain a trained CBHG-SE model;

[0032] The input Braille sentence is predicted based on the trained CBHG-SE model to obtain a Chinese translation result of the input Braille.

[0033] The corpus used in this embodiment contains 944,617 corpus pairs consisting of Braille and its corresponding Chinese text. The target Chinese text in the corpus includes 12,376,346 Chinese characters, 125,781 alphabetic characters, 421,581 numeric characters, and 1,508,464 punctuation characters.

[0034] Based on approximately 14.43 million words of Chinese news material from the People's Daily between 2007 and 2009, a large-scale national standard Braille Chinese-Braille corpus was constructed. Each entry in the corpus contains a Chinese character string and a Braille ASCII character string. The original news material contains a variety of character types, including Chinese characters, letters, numbers, and punctuation marks. Table 1 provides character count statistics for each character type. To achieve sentence-level comparison, this embodiment uses punctuation detection to segment the Chinese material into sentences based on the five punctuation marks: comma, period, semicolon, exclamation point, and question mark.

[0035] To construct the Braille comparison corpus, the segmented Chinese sentences were fed into the Java interface of the Chinese-Braille conversion system of the China Braille Digital Platform, which then returned the translated Braille ASCII code sequences for the national standard Braille. The standardization and accuracy of the corpus were further ensured by submitting the Chinese sentences and their corresponding Braille ASCII code sequences to Braille experts at the China Braille Publishing House for revision.

[0036] Table 1

[0037] Chinese characters letter number punctuation 12376346 125781 421581 1508464

[0038] In the data preprocessing part, there are three steps: dictionary generation, tag addition, dictionary conversion, and data set partitioning. In the dictionary generation part, the Chinese character dictionary and Braille ASCII code dictionary are generated by traversing the Chinese text and Braille text, and the maximum lengths of Braille sentences and Chinese sentences, maxlen and maxlen_ascii, are obtained. The three keys 0, 1, and 2 of the two dictionaries correspond to the zero-padding tags, respectively. <pad>', start tag' <start>', end tag' <end>'. In the tag adding stage, add a start tag at the beginning of each Braille sentence and Chinese sentence <start>', add an end tag at the end of the sentence' <end>', add zero padding mark when the length is less than its corresponding maximum length' <pad>', and convert it into a coded representation through the constructed Chinese character dictionary and Braille ASCII code dictionary, which serve as the encoder input and decoder input respectively. Finally, 3% of the data is randomly selected as the test set, and the remaining 97% of the data is used as the training set.

[0039] In the construction of CBHG-SE model, Figure 1 As shown in the figure, the model consists of five parts: a set of one-dimensional convolutional filters, a SE module, a residual connection operation, a Highway network, and a bidirectional GRU network. After the encoding operation, the Braille sentence is used as the input sequence and is first convolved with K sets of one-dimensional convolutional filters, where the kth set contains filters of width k (k = 1, 2, ..., K). These filters model the local and contextual information in the Braille sentence, thereby associating a character with 1, 2, ..., K characters in the context. The convolution outputs obtained after the convolution operation are stacked together and max-pooled along the time axis to increase the invariance of the current information, and the stride is set to 1 to preserve the original temporal resolution.

[0040] However, when performing one-dimensional convolution on Braille sentences, the convolutional neural network only captures local information. Since every character in a sentence is interconnected, using a local receptive field will ignore global correlation information. Therefore, increasing the correlation information between channels can effectively enhance the neural network's feature extraction capabilities.

[0041] This method adds a SE module before the residual connection operation to perform feature recalibration to achieve mutual correlation between channels. The construction process of the added SE module is as follows: Figure 2 As shown in Figure 1, a global descriptor is generated from the original feature map through a squeezing operation. This global descriptor aggregates information from all channels. An excitation operation is then performed to learn channel attention through a gating mechanism based on channel dependencies. The feature map X is then reweighted to generate the output of the SE block, which is then combined with the previous module to extract channel-related information.

[0042] The purpose of the squeeze operation is to extract global information. In order to make the network lightweight, information compression is also required without increasing the time complexity and space complexity. Therefore, the squeeze operation uses the global average pooling method. After global average pooling, the vector Z is obtained. Since the dimension of the input feature map X is L*C, where L represents the length and C represents the channel, the cth element of Z can be calculated by formula (1), F sq Represents a squeeze operation.

[0043]

[0044] This operation effectively performs a global average pooling on X. In convolutional neural networks, each channel's filter typically extracts features from a local receptive field. Therefore, each feature map in X cannot leverage its correlations with other feature maps. Furthermore, the corresponding receptive fields at lower network layers are smaller than those at higher layers, and the loss of channel correlation information is more severe. The squeezing operation effectively expands the receptive field globally, extracting richer features and improving the accuracy of the trained blind-to-Chinese translation.

[0045] After the squeezing operation is completed, the global information is obtained, and then the excitation operation is performed to fully capture the channel correlation. The two fully connected layers and two activation layers connected after global pooling constitute the excitation operation. The role of the first fully connected layer is to compress the global information obtained after global pooling. The role of the second fully connected layer is to restore the compressed vector to the same size as the vector after the global pooling layer, and obtain the vector S, as shown in formula (2), where the two activation functions δ and σ respectively select the Relu function and the Sigmoid function, W1 and W2 represent two fully connected operations, and F ex Represents the excitation operation, and Z is the feature vector obtained after the squeeze operation. After the excitation operation is completed, the obtained weight vector S is multiplied by the feature map X, as shown in formula (3), where F scale It represents the multiplication of the weight vector S and the feature map X, so that each feature map obtains the global correlation features with other channels.

[0046] S=F ex (Z,W)=σ(g(Z,W))=σ(W2δ(W1Z)) (2)

[0047]

[0048] The one-dimensional convolution obtained after the SE module is added to the original input sequence via a residual connection. All convolutional layers undergo batch normalization. The result of this residual connection is fed into a multi-layer Highway network to extract higher-level features. The Highway network performs a nonlinear transformation on the input at a specific scale and then adds the results together. Finally, the result after the Highway network is fed into a bidirectional GRU network to extract sequential features from both forward and backward directions. Through training, this ultimately achieves blind-to-Chinese conversion.

[0049] In order to test the performance of the CBHG-SE model, this embodiment conducted three experiments: the impact of large and small data sets on model performance, the accuracy of Braille-Chinese conversion under pure Chinese and mixed sentences, and the accuracy of Braille-Chinese conversion under Braille sentences of different lengths. The small data set is 450,000 Braille-Chinese comparison corpora selected from the large data set. The BLEU (Bilingual Evaluation Replacement) value evaluation method is used to evaluate the conversion accuracy, which can show the similarity of the two sentences. There are a total of four parameters representing weights, representing the focus of the evaluation. In addition, this embodiment selected the ConvS2S model and the Transformer model in the neural machine translation method and the CBHG model, the basic model of the CBHG-SE model proposed in the present invention, as the basic model and as a comparison model to test the performance of the CBHG-SE model.

[0050] In this embodiment, three basic models and the CBHG-SE model were trained on large and small data sets respectively, and the obtained model accuracy ACC values ​​were as follows: Figure 3 As shown in the figure, the increase in the amount of data used for training improves the performance of the four models, but the improvement for the CBHG-SE model is the smallest, with an improvement of only 0.37%. This shows that the CBHG-SE model can obtain good model performance through training in a relatively small dataset.

[0051] This example also uses four models to predict the conversion results on 10,000 Braille-pure Chinese character corpora and 10,000 Braille-mixed sentences, and calculates the BLEU value based on their corresponding Chinese labels. The experimental results are as follows: Figure 4 and Figure 5 As shown in the figure. (ω1,ω2,ω3,ω4) represents the weights of 1-gram, 2-gram, 3-gram, and 4-gram in BLEU calculation, satisfying ω1+ω2+ω3+ω4=1. The experimental results show that the CBHG-ECA model achieves the best conversion results when the target language is pure Chinese characters and mixed sentences. When the target language is pure Chinese characters, the BLEU value reaches 91.48, which is an excellent performance for the Blind-Chinese conversion task.

[0052] In the conversion test of the model on Braille sentences of different lengths, the present invention selected five length intervals, and each interval contained 10,000 test corpora. The basic model and the CBHG-SE model were tested on the corpora of each interval respectively. The results are as follows: Figure 6 Compared with other models, the CBHG-SE model achieves better conversion results in all length intervals. In the length interval (40, 60], the BLEU value can reach 91.59, which effectively solves the problem of converting Braille sentences of different lengths to Chinese.

[0053] The CBHG-SE model proposed in this paper effectively solves the problem of single-step conversion of Braille into Chinese characters, letters, numbers, and punctuation, achieving a high level of conversion accuracy and demonstrating good results in converting Braille sentences of varying lengths into Chinese. This invention not only effectively facilitates communication between the blind and sighted, but also has significant implications for Braille promotion and other related endeavors.

[0054] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.< / pad> < / end> < / start> < / end> < / start> < / pad>

Claims

1. A method for automatic conversion from Braille to Chinese, characterized in that: include: Constructing a CBHG-SE model, and training the CBHG-SE model based on a corpus consisting of Braille and corresponding Chinese to obtain a trained CBHG-SE model; Convert the input Braille sentence based on the trained CBHG-SE model to obtain a Chinese translation result of the input Braille; Constructing the CBHG-SE model includes: adding an SE module to the CBHG model to perform feature recalibration to obtain the CBHG-SE model, which is used to achieve mutual correlation between channels and enhance the feature extraction capability of the model; The CBHG-SE model includes: a set of one-dimensional convolution filters, a SE module, a residual connection operation module, a Highway network, and a bidirectional GRU network, wherein the one-dimensional convolution filters, the SE module, the residual connection operation module, the Highway network, and the bidirectional GRU network are connected in sequence; Converting the input Braille sentence based on the trained CBHG-SE model includes: The test set is input into the trained CBHG-SE model. The Braille sentences in the test set are encoded as the input sequence. The Braille sentences are first convolved with K groups of one-dimensional convolution filters. The features are recalibrated through the SE module to obtain a one-dimensional convolution result. The one-dimensional convolution result is added to the original input sequence through a residual connection. The result obtained after the residual connection is input into the Highway network for extracting higher-level features. Finally, the feature vector obtained after the Highway network is input into the bidirectional GRU network to extract sequential features from forward to backward and reverse directions. The trained model ultimately achieves Braille-Chinese conversion. The SE module performs feature recalibration, including: generating a global descriptor using an original feature map generated by encoding, convolution, and pooling the Braille sentence through a squeezing operation, performing an excitation operation, learning channel attention based on a channel-dependent gating mechanism, and then reweighting the feature map to generate the output of the SE module, thereby completing feature recalibration; and combining the output result with a residual connection operation module; wherein the squeezing operation is a global average pooling method, the global descriptor aggregates information from all channels, and the original feature map is generated by encoding, convolution, and pooling the Braille sentence.

2. The automatic conversion method from Braille to Chinese according to claim 1, characterized in that: Before training the CBHG-SE model, data preprocessing is included. The data preprocessing process includes: generating a dictionary, adding tags, dictionary conversion, and dividing the data set; wherein the dictionary generation is used to generate a Chinese character dictionary and a Braille ASCII code dictionary by traversing the Chinese text and Braille text in the corpus, and obtain the maximum length of the Braille sentences and the Chinese sentences; the tag addition is used to add a start tag and an end tag to each of the Braille sentences and the Chinese sentences; the dictionary conversion is used to convert the Braille sentences and the Chinese sentences into coded representations through the Chinese character dictionary and the Braille ASCII code dictionary; and the data set division is used to divide the converted sentences into a training set and a test set.

3. The automatic conversion method from Braille to Chinese according to claim 2, characterized in that: Each control corpus in the corpus contains a Chinese character string and a Braille ASCII code string, and also includes Chinese, alphabetic, numerical, and punctuation character types; among them, the Chinese sentences and their corresponding Braille ASCII code sequences are proofread by Braille experts to ensure the standardization and accuracy of the corpus.

4. The automatic conversion method from Braille to Chinese according to claim 3, characterized in that: Inputting the training set into the CBHG-SE model for training includes: inputting the Braille sentences in the corpus into the CBHG-SE model one by one for training.

5. The automatic conversion method from Braille to Chinese according to claim 1, characterized in that: The Highway network is used to perform nonlinear transformation on the results obtained after the residual connection in proportion and then add them together.

6. The automatic conversion method from Braille to Chinese according to claim 1, characterized in that: The process of performing the excitation operation includes a fully connected layer and an activation layer, wherein the fully connected layer is used to compress the global information obtained after global average pooling, and restore the compressed vector to obtain a weight vector; after the excitation operation is completed, the obtained weight vector is multiplied by the feature map, so that each feature map obtains a global correlation feature with other channels.

Images