A method and system for converting Chinese characters into universal Braille

Abstract

The present invention proposes a method and system for converting Chinese characters to universal Braille, comprising: constructing a deep learning model, and the deep learning model has two subtask branches that share feature encoding, wherein one subtask branch is used to segment Chinese text according to Braille rules, and the other subtask branch is used to mark pinyin and corresponding tones for each Chinese character; using Chinese character corpus marked with Braille segmentation and control corpus of Chinese character pinyin, alternately training the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and saving the current deep learning model as a recognition model; inputting Chinese character data to be converted to universal Braille into the recognition model, obtaining the segmentation information and toned pinyin information of the Chinese character data, and based on the segmentation information and the toned pinyin information, converting the Chinese character data into corresponding universal Braille text. The present invention can realize real-time and accurate conversion of Chinese characters to Braille.

Description

Technical Field

[0001] The present invention relates to the technical field of Braille processing, in particular to the field of automatic conversion from Chinese characters to general Braille. Background Art

[0002] Braille is an important way for the blind to read and obtain information. It is a tactile symbol system, printed on paper or displayed on a dot display, and read by touch. The basic unit of Braille is called a "cell", and one cell contains 6 dots. A total of 64 combinations can be formed by setting whether each dot has a dot (including the empty cell where none of the 6 dots have a dot), and these combinations constitute the most basic Braille symbols.

[0003] In order to generate Braille content, it is necessary to convert the text content used by ordinary people into Braille. For alphabetic languages, there is a direct mapping from letters to Braille symbols, and the conversion is relatively simple. Currently, computer systems are available for the automatic conversion of many languages such as English, Portuguese, Danish, Spanish, etc. to Braille. Due to the characteristics of Chinese Braille itself, the automatic conversion between Chinese and Braille is more difficult.

[0004] Since it is impossible to uniquely map Chinese characters to Braille symbols, Chinese Braille is defined as a phonetic script. In Chinese Braille, in most cases, 2 - 3 cells are used to represent a Chinese character, where one cell represents the initial consonant, one cell represents the final vowel, and in some cases, an additional cell is needed to represent the tone. The biggest difference between Chinese Braille and Chinese text lies in its "word segmentation and continuous writing" rule, that is, it is required to separate words with empty cells. Braille word segmentation is different from Chinese word segmentation, and many phrases in Chinese need to be written continuously in Braille. For word segmentation and continuous writing, more than 100 rules based on morphology, grammar, and semantics are given in the Chinese Braille standard, such as "‘不’ should be written continuously with verbs, modal verbs, adjectives, prepositions, and monosyllabic degree adverbs".

[0005] In order to further reduce ambiguity, Braille also has tone marking rules, that is, which syllables need to be explicitly marked with tones. There are currently two main Braille schemes in use, namely "Current Braille" and "Universal Braille". The word segmentation rules of the two Braille schemes are the same, and the main difference lies in the tone marking rules. In the current Braille, in order to save reading time and printing costs, it is stipulated that tones are only marked for easily confused words, unfamiliar words, ancient Chinese content words, and uncommon monosyllabic words. It is generally believed that the tone marking rate of current Braille is about 5%. However, a large number of unmarked syllables can easily cause confusion and affect reading and comprehension. Therefore, the National Language Commission issued the "Universal Braille" scheme in 2018. Although certain tone default settings are used in Universal Braille, after the default rule conversion, the tones of most syllables can be determined. Since its release in 2018, the country has begun to vigorously promote Universal Braille. However, due to the lack of tools and systems for automatic conversion from Chinese characters to Universal Braille and the low efficiency of manual conversion, there are currently few books and materials in Universal Braille, and the text corpus for Chinese character-Universal Braille comparison is still blank.

[0006] Currently, methods and systems for converting Chinese characters to current Braille have emerged, and some have been put into practical use in fields such as editing and publishing Braille books. However, methods and systems for converting Chinese characters to universal Braille are still rare.

[0007] There are three key issues in converting Chinese characters to Braille, namely Braille word segmentation, phonetic marking and tone marking.

[0008] Since the word segmentation rules of current Braille and universal Braille are the same, the tasks of Braille word segmentation are consistent. The so-called Braille word segmentation refers to the segmentation of connected Chinese character strings (or Braille symbol strings converted from Chinese characters) according to Braille rules. Early Braille word segmentation systems adopted rule-based or word library-based methods. The main idea is to first segment the Chinese text to be converted according to the Chinese word segmentation rules, and then make certain adjustments (mainly merging) to the Chinese word segmentation according to the Braille word segmentation rules or a pre-built Braille word segmentation library to obtain a Chinese character string segmented according to the Braille rules. However, Braille word segmentation involves highly subjective grammatical and semantic rules, which are difficult to define and process by computer. In addition, Braille connects many phrases in Chinese, and the combinations that can be connected are infinite and cannot be exhausted by a word segmentation library. Therefore, the performance of these methods is limited. In recent years, with the development of machine learning, especially deep learning, Braille word segmentation methods based on machine learning, especially deep learning, have emerged. These methods use machine learning models to implicitly characterize the Braille word segmentation and spelling specifications, avoiding the computer from directly processing complex grammatical and semantic rules, which can greatly improve the accuracy of Chinese-Braille conversion. The existing technology uses Braille corpus to train a Braille word segmentation model based on machine learning, which can automatically segment Braille strings. The model training of this method is based on Braille corpus. The advantage is that it is easy to obtain. The disadvantage is that Braille only represents the pronunciation of Chinese characters, which may cause ambiguity due to homophones, thus affecting the final word segmentation results. Subsequent existing technologies are based on Chinese character-current Braille comparison corpus, training deep learning models to directly segment Chinese character strings according to Braille rules, which has a higher accuracy rate than Braille string segmentation.

[0009] Pinyin marking refers to marking the pinyin of a Chinese character. Since there is a corresponding relationship between the syllables of pinyin and the syllables of Braille, after obtaining the pinyin, a mapping table between pinyin and Braille syllables can be used to obtain the corresponding Braille. The difficulty of pinyin marking lies in the processing of polyphones. The current method is generally based on the pinyin dictionary on the basis of Chinese word segmentation, combined with machine learning models (such as n-gram model, maximum entropy model, artificial neural network) and other methods to disambiguate polyphones.

[0010] Tone marking here refers to determining whether the syllable corresponding to each Chinese character needs to have a tone mark. In the current Braille, since it is stipulated that only easily confused words, unfamiliar words, ancient Chinese nouns, and uncommon monosyllabic words are marked with tones, it is necessary to automatically determine whether each word meets these conditions. These situations are very subjective and difficult to summarize through a vocabulary or some clear rules, so the existing technology uses a machine learning model to perform automatic judgment. In universal Braille, it is relatively simple, and there are clear rules that stipulate which tone can be omitted for each syllable (when reading, most of these default tones can be corresponded to certain tones based on the rules, so except for a few special cases, universal Braille actually realizes that each syllable is marked with tones), so it can be marked directly according to the rules.

[0011] The traditional method generally performs the above three steps in sequence, but this method has two problems: first, the time required for each step is superimposed, resulting in a longer overall time required; second, the latter step depends on the previous step. If the result of the previous step is wrong (such as word segmentation error), it will affect the subsequent steps (such as pinyin marking and tone marking), resulting in error superposition. If a unified machine learning model can be used to implement the above three steps at one time (for universal Braille, only the first two steps need to be implemented, because tone marking can be implemented using rules), the above two problems can be solved, and higher time efficiency and conversion accuracy can be obtained. However, this idea still has a huge difficulty for universal Braille: in order to use a model to simultaneously implement word segmentation and pinyin marking, a training corpus of Chinese characters and universal Braille is required, that is, a large amount of corpus is required, and these corpuses need to have both Braille word segmentation and pinyin marking information. At present, because universal Braille is in the promotion period, there are very few available corpuses, which is difficult to meet this requirement. Summary of the invention

[0012] The technical problem to be solved by the present invention is: in view of the current situation that there is a lack of or even no available large-scale Chinese character-universal Braille comparison corpus, two independent and currently available corpora - Chinese character corpora marked with Braille word segmentation and Chinese character-pinyin comparison corpora are used to train a model to simultaneously realize Braille word segmentation and pinyin marking, thereby achieving high-efficiency and high-accuracy conversion from Chinese characters to universal Braille.

[0013] Specifically, in view of the deficiencies of the prior art, the present invention proposes a method for converting Chinese characters to universal Braille, which includes:

[0014] Step 1: construct a deep learning model, and the deep learning model has two subtask branches that share feature encoding, one of which is used to segment Chinese text according to Braille rules, and the other is used to mark pinyin and corresponding tones for each Chinese character;

[0015] Step 2: Use the Chinese character corpus with marked Braille segmentation and the control corpus of Chinese character pinyin to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as the recognition model;

[0016] Step 3: input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and tone pinyin information of the Chinese character data, and convert the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0017] The present invention also proposes a method for converting Chinese characters to universal Braille, which comprises:

[0018] Step 1: construct a deep learning model, and the deep learning model has two subtask branches that share feature encoding, one of which is used to segment Chinese text according to Braille rules, and the other is used to mark pinyin and corresponding tones for each Chinese character;

[0019] Step 2: Use the Chinese character corpus with marked Braille segmentation and the Chinese character-pinyin comparison corpus to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as an intermediate model;

[0020] Step 3, obtain Chinese character-universal Braille comparison corpus, and divide it into multiple batches of data, each batch of data contains multiple data, each data contains Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin mark with tone corresponding to the Chinese character text, and use each batch of data to train the intermediate model in turn. During training, the Chinese character text of the batch data is used as input, and the output results of Braille word segmentation and Pinyin mark are generated through two subtask branches respectively. The loss function of each output result is calculated using the Braille word segmentation mark and the Pinyin mark with tone, and then the intermediate model is updated by the back propagation algorithm until the loss function of each output result converges or reaches a preset number of iterations, and the current intermediate model is saved as the recognition model;

[0021] Step 4: input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and tone pinyin information of the Chinese character data, and convert the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0022] The method for converting Chinese characters into universal Braille, wherein the deep learning model is a language representation model.

[0023] The present invention also proposes a Chinese character to universal Braille conversion system, which includes:

[0024] A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character;

[0025] A training module is used to use the Chinese character corpus with marked Braille segmentation and the control corpus of Chinese character pinyin to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as a recognition model;

[0026] The conversion module is used to input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and the tone pinyin information of the Chinese character data, and convert the Chinese character data into the corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0027] The present invention also proposes a Chinese character to universal Braille conversion system, which includes:

[0028] A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character;

[0029] The first training module is used to use the Chinese character corpus with marked Braille segmentation and the Chinese character-pinyin comparison corpus to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as an intermediate model;

[0030] The second training module is used to obtain Chinese character-universal Braille comparison corpus and divide it into multiple batches of data, each batch of data contains multiple data, each data contains Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin mark with tone corresponding to the Chinese character text, and train the intermediate model with each batch of data in turn. During training, the Chinese character text of the batch data is used as input, and the output results of Braille word segmentation and Pinyin mark are generated through two subtask branches respectively. The loss function of each output result is calculated using the Braille word segmentation mark and the Pinyin mark with tone, and then the intermediate model is updated by the back propagation algorithm until the loss function of each output result converges or reaches a preset number of iterations, and the current intermediate model is saved as the recognition model;

[0031] The conversion module is used to input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and the tone pinyin information of the Chinese character data, and convert the Chinese character data into the corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0032] The Chinese character to universal Braille conversion system, wherein the deep learning model is a language representation model.

[0033] The present invention also proposes a storage medium for storing a program of the method for converting any one of the Chinese characters into universal Braille.

[0034] The present invention also provides a client, which is used in any of the above-mentioned Chinese character to universal Braille conversion systems.

[0035] It can be seen from the above scheme that the advantages of the present invention are:

[0036] By adopting the method and system for converting Chinese characters to universal Braille proposed in the present invention, the problem of the lack of currently available Chinese character-universal Braille comparison corpus can be overcome. The Chinese character-universal Braille conversion model based on deep neural network can be trained by using relatively easy-to-obtain Braille word segmentation corpus and Chinese character-pinyin comparison corpus, so as to achieve high-accuracy Chinese character-universal Braille conversion. Applying this technology to Braille editing and publishing can improve the efficiency of Braille book publishing and enrich Braille book resources; applying this technology to e-book readers, browsers and other software on blind terminals can achieve real-time conversion of Chinese characters to Braille, and by outputting to a dot display, it can help the blind read Chinese documents in a Braille touch reading manner, eliminating reading difficulties. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is the overall flow chart of the present invention;

[0038] Figure 2 This is a structural diagram of the model of the present invention;

[0039] Figure 3 This is an example diagram of a Chinese character corpus with Braille word segmentation marked;

[0040] Figure 4 This is a sample diagram of the corpus comparing Chinese characters and current Braille word levels;

[0041] Figure 5 This is an example of the Chinese character-pinyin comparison corpus. DETAILED DESCRIPTION

[0042] The present invention proposes a method for converting Chinese characters into universal Braille, the main steps of which include:

[0043] 1. Build a deep learning model with two branches. The two branches are responsible for segmenting Chinese text according to Braille rules and annotating each Chinese character with its corresponding pinyin with tones. The two branches share a feature encoding module.

[0044] Preferably, the language representation model BERT can be used as a pre-training model, and two task branches can be set on top of BERT, corresponding to the two subtasks of converting Chinese characters to universal Braille: segmenting the Chinese text according to Braille rules (Braille segmentation subtask), and marking each Chinese character with its corresponding pinyin with tone (pinyin marking subtask).

[0045] 2. Use the Chinese character corpus marked with Braille segmentation and the Chinese character-pinyin comparison corpus to train the deep learning model constructed in step 1, and the training method is to alternately train the two sub-task branches.

[0046] The specific method of alternating training is: divide the Chinese character corpus marked with Braille segmentation and the Chinese character-pinyin comparison corpus into several batches, each batch contains several data. Then, each time, a batch of data is selected from the two corpora to train the model's task matching the corpus, and the training is stopped when the loss functions of the two tasks converge or the preset number of iterations is reached.

[0047] 3. Optionally, the model trained in step 2 is optimized using available Chinese character-universal Braille comparison corpus.

[0048] Firstly, the Chinese character corpus marked with Braille segmentation and the Chinese character-pinyin comparison corpus are obtained from the Chinese-general Braille corpus.

[0049] The optimization training method is as follows: the Chinese character-general Braille comparison corpus is divided into several batches, each batch contains several data, and each data contains: Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin with tones corresponding to the Chinese character text. Each batch of data is used for training in turn. During training, the Chinese character text of the batch of data is used as input, and the output results of Braille word segmentation and Pinyin marking are generated respectively through the two subtask branches of Braille word segmentation and Pinyin marking. Then, the loss function is calculated using the annotated data of Braille word segmentation and Pinyin marking in the training data, that is, the Braille word segmentation mark corresponding to the Chinese character text and the Pinyin with tones corresponding to the Chinese character text, and then the parameters of the entire model are updated through the back propagation algorithm.

[0050] 4. Use the model trained in step 2 or step 3 to automatically convert Chinese characters to universal Braille.

[0051] After the model is trained, a piece of input Chinese text is input into the model. After going through two subtask branches, the prediction results of the two subtasks can be obtained respectively. Based on the prediction results of the two subtasks, the corresponding universal Braille can be obtained.

[0052] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0053] The present invention proposes a method and system for converting Chinese to universal Braille based on deep learning. The deep learning method is used to analyze and extract features of the input Chinese sentence, and the pinyin information and word segmentation information of the Chinese sentence are obtained according to the extracted features, so that the corresponding universal Braille can be obtained. The process is as follows: Figure 1 shown.

[0054] The specific steps include:

[0055] 1. Build a deep learning model with two branches. The two branches are responsible for segmenting Chinese text according to Braille rules and annotating each Chinese character with its corresponding pinyin with tones. The two branches share a feature encoding module.

[0056] In one embodiment, BERT is used as a pre-trained model, and two task branches are set on top of BERT, corresponding to the two subtasks of converting Chinese characters to universal Braille: segmenting Chinese text according to Braille rules (Braille segmentation subtask), and marking each Chinese character with its corresponding pinyin with tone (pinyin marking subtask). The model structure is as follows Figure 2 shown.

[0057] like Figure 2 In the model shown, the bottom layer is BERT, and two fully connected layers are connected to the upper layer of BERT, one for predicting the word segmentation information of the input text, and the other for predicting the pinyin information of the input text. After obtaining these two pieces of information, the universal Braille corresponding to the input Chinese text can be obtained.

[0058] 2. Use the Chinese character corpus marked with Braille segmentation and the Chinese character-pinyin comparison corpus to train the deep learning model constructed in step 1, and the training method is to alternately train the two sub-task branches.

[0059] Since Universal Braille was released and implemented in 2018 and is currently in the early stages of promotion, there are very few corpora available for direct comparison between Chinese characters and Universal Braille. Therefore, two independent corpora are used to simultaneously train the deep learning model constructed in step 1.

[0060] The two types of corpora used are the Chinese character corpus with Braille segmentation marked and the Chinese character-pinyin comparison corpus.

[0061] (1) Chinese character corpus with Braille word segmentation marked. That is, the Chinese character corpus is marked with information about word segmentation according to Braille rules. The corpus can be the current Braille corpus with Chinese characters and Braille word-level comparisons, or other corpus obtained by manual or automatic annotation.

[0062] The Chinese character corpus marked with Braille word segmentation as the model input, where each Chinese sentence is marked in the following way: the word segmentation information of each character in the sentence is represented by "B", "M", "E", and "S". Among them, "S" represents single, indicating that it is a single character, meaning that this character can be used as a word alone; "B" represents begin, indicating that this character is the beginning of a word, that is, the first character of the word it belongs to; "M" represents middle, indicating that this character is the middle character of the word it belongs to, that is, neither the first character nor the last character of the word it belongs to; "E" represents end, indicating that this character is the last character of the word it belongs to.

[0063] Therefore, for the Chinese character corpus marked with Braille word segmentation as the model input, each piece of data is respectively a Chinese text and the word segmentation information corresponding to each of its characters. As Figure 3 shown is a Chinese text and its corresponding word segmentation information.

[0064] If other forms of Chinese character corpora marked with Braille word segmentation are adopted, they need to be converted into Figure 3 the annotation form shown.

[0065] In one embodiment, a corpus of Chinese characters and current Braille word-level correspondence is adopted. This corpus is as Figure 4 shown: the Chinese character corpus and the current Braille corpus are stored as two text files respectively, and Braille is stored in ASCII code form. The same lines of the two text files respectively represent the Chinese characters and the current Braille text of the same sentence, and spaces are used to represent word segmentation marks in both the Chinese character and the current Braille text. Therefore, the Chinese character text in this corpus is the Chinese character text segmented according to Braille rules. Since the word segmentation rules of current Braille and general Braille are the same, this corpus can be used to train the Braille word segmentation subtask in the Chinese character-general Braille conversion model. Before training, it is necessary to first convert the Figure 4 format shown on the left into Figure 3 the format shown. The conversion method is to mark each Chinese character as one of "B", "M", "E", "S" according to its position in the Braille word (a string of Chinese characters separated by spaces, such as "我", "的", "一贯", "做法" in the first line).

[0066] (2) Chinese character-pinyin correspondence corpus. The Chinese character-pinyin correspondence corpus refers to the corpus marked with the pinyin corresponding to each Chinese character. Since almost every character in general Braille can determine the tone, the pinyin marked in the corpus also needs to be pinyin with tones.

[0067] In one embodiment, the adopted Chinese character-pinyin correspondence corpus is as Figure 5As shown in the figure. In this corpus, each Chinese sentence will be annotated with its corresponding pinyin information. Pinyin includes syllables and tones. The tones are marked as follows: "1" for the first tone (yinping), "2" for the second tone (yangping), "3" for the third tone (shangsheng), "4" for the fourth tone (qusheng), and "5" for the light tone. At the same time, punctuation marks such as commas and periods, as well as Latin letters, Arabic numerals, etc., are represented by "P".

[0068] Due to the serious lack of corpora for Chinese characters and universal Braille, it is impossible to use one corpus to simultaneously train the two subtasks of the model constructed in step 1. If two models are trained separately with two corpora, the association information between the two tasks cannot be utilized (there is a correlation between pronunciation and word segmentation, and different pronunciations correspond to different word segmentations), and it is not conducive to the subsequent use of a small amount of Chinese and universal Braille comparison corpora to optimize the training of a unified model. Therefore, the present invention uses two independent corpora, a Chinese character corpus marked with Braille word segmentation and a Chinese character-pinyin comparison corpus, to simultaneously train the unified multi-branch model constructed in step 1. Using one model to implement two subtasks at the same time can better train the BERT module shared by the two subtask branches by utilizing the association information between the two subtasks, and is also more conducive to the subsequent training of the unified model with data that has both Braille word segmentation and pinyin annotations.

[0069] When training the model, an alternating training method is used: the Chinese character corpus marked with Braille segmentation and the Chinese character-pinyin comparison corpus are divided into several batches, and each batch contains 128 data in an optimized embodiment. Then, a batch of data is selected from the two corpora alternately each time to train the model's task matching the corpus, and the training is stopped when the loss functions of the two tasks converge or the preset number of iterations is reached.

[0070] 3. Optionally, a small amount of available Chinese character-universal Braille comparison corpus is used to optimize the model trained in step 2.

[0071] Universal Braille contains both Braille word segmentation information and the pronunciation information of each Chinese character corresponding to the Braille. This pronunciation information can be easily converted into Chinese pinyin with tones through a mapping table between Braille symbol strings and pinyin symbol strings. Therefore, the Chinese-Universal Braille corpus can be used to obtain the following formats: Figure 3 The Chinese character corpus and format marked with Braille segmentation are shown as follows Figure 5 The Chinese character-pinyin comparison corpus shown.

[0072] In one embodiment, the Chinese character-universal Braille corpus format used is the same as Figure 4 The Chinese text in this corpus is segmented by Braille and Figure 4 The format shown on the left is exactly the same, so from this text we can get Figure 3The Chinese character corpus with Braille segmentation labeled for input into the model is shown.

[0073] The Braille corpus in the Chinese-Universal Braille corpus and Figure 4 The format on the right is almost the same, except that some Braille symbols are added to the Braille words to indicate the tones of the corresponding phonetic characters. By parsing the symbol string of universal Braille, it can be converted into the corresponding Pinyin string according to the pronunciation (including initials, finals and tones). For example, a mapping table of universal Braille tones and Pinyin tones can be used to achieve the above conversion, thus obtaining Figure 5 The Chinese character-pinyin comparison corpus shown.

[0074] After obtaining the Chinese character corpus and the Chinese character-pinyin comparison corpus marked with braille segmentation from the Chinese character-general braille comparison corpus, these corpuses can be used to continue training the model trained in step 2. The method is: the Chinese character-general braille comparison corpus is divided into several batches, and each batch contains 128 data in an optimized embodiment, and each data also contains: Chinese character text, braille segmentation mark corresponding to the Chinese character text, and pinyin with tone corresponding to the Chinese character text. Use each batch of data for training in turn. During training, use the Chinese character text of the batch of data as input, and pass through the two subtask branches of braille segmentation and pinyin marking respectively to generate the output result of braille segmentation and the output result of pinyin marking, and then use the annotation data of braille segmentation and pinyin marking in the training data, that is, the braille segmentation mark corresponding to the Chinese character text and the pinyin with tone corresponding to the Chinese character text to calculate the loss function, and then update the parameters of the entire model through the back propagation algorithm.

[0075] 4. Use the model trained in step 2 or step 3 to automatically convert Chinese characters to universal Braille.

[0076] After the model is trained, for an input Chinese text, it is input into the model. After two subtask branches, the prediction results of the two subtasks can be obtained respectively, namely the Braille word segmentation mark and the pinyin with tone corresponding to the Chinese text. In one embodiment, the input Chinese text is "asteroid discovery.", and the word segmentation mark obtained after model prediction is "B ME SBME", and the pinyin with tone is "xiao3 xing2 xing1 de5 fa1 xian4 P".

[0077] Based on the corresponding Braille word segmentation marks and tone pinyin of Chinese text, the corresponding universal Braille can be obtained, and the method is as follows: first, the tone pinyin is segmented according to the word segmentation marks, and the tone pinyin syllables between "B" and "E" are connected into one word according to the position correspondence, and the tone pinyin syllables corresponding to the "S" mark are separated into separate words. As in the above embodiment, the pinyin string "xiao3xing2xing1 de5 fa1xian4 P" can be obtained. Then, the universal Braille text is obtained by using the mapping rules between the pinyin syllables and punctuation, symbols, numbers, etc. and the universal Braille symbol string, such as the Braille text (ASCII code encoding) obtained in the above embodiment is "H>'H*1H*AD F9H%"2".

[0078] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0079] The present invention also proposes a Chinese character to universal Braille conversion system, which includes:

[0080] A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character;

[0081] A training module is used to use the Chinese character corpus with marked Braille segmentation and the control corpus of Chinese character pinyin to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as a recognition model;

[0082] The conversion module is used to input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and the tone pinyin information of the Chinese character data, and convert the Chinese character data into the corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0083] The present invention also proposes a Chinese character to universal Braille conversion system, which includes:

[0084] A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character;

[0085] The first training module is used to use the Chinese character corpus with marked Braille segmentation and the Chinese character-pinyin comparison corpus to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as an intermediate model;

[0086] The second training module is used to obtain Chinese character-universal Braille comparison corpus and divide it into multiple batches of data, each batch of data contains multiple data, each data contains Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin mark with tone corresponding to the Chinese character text, and train the intermediate model with each batch of data in turn. During training, the Chinese character text of the batch data is used as input, and the output results of Braille word segmentation and Pinyin mark are generated through two subtask branches respectively. The loss function of each output result is calculated using the Braille word segmentation mark and the Pinyin mark with tone, and then the intermediate model is updated by the back propagation algorithm until the loss function of each output result converges or reaches a preset number of iterations, and the current intermediate model is saved as the recognition model;

[0087] The conversion module is used to input the Chinese character data to be converted into universal Braille into the recognition model, obtain the word segmentation information and the tone pinyin information of the Chinese character data, and convert the Chinese character data into the corresponding universal Braille text based on the word segmentation information and the tone pinyin information.

[0088] The Chinese character to universal Braille conversion system, wherein the deep learning model is a language representation model.

[0089] The present invention also proposes a storage medium for storing a program of any one of the methods for converting Chinese characters into universal Braille. The present invention also proposes a client for any one of the systems for converting Chinese characters into universal Braille.

Claims

1. A method for converting Chinese characters into universal Braille, characterized in that: include: Step 1: construct a deep learning model, and the deep learning model has two subtask branches that share feature encoding, one of which is used to segment Chinese text according to Braille rules, and the other is used to mark pinyin and corresponding tones for each Chinese character; Step 2: Use the Chinese character corpus with marked Braille segmentation and the control corpus of Chinese character pinyin to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as the recognition model; Step 3: inputting the Chinese character data to be converted into universal Braille into the recognition model, obtaining word segmentation information and tone pinyin information of the Chinese character data, and converting the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information; The bottom layer of this deep learning model is BERT, and two fully connected layers are connected to the upper layer of BERT, one for predicting the word segmentation information of the input text, and the other for predicting the pinyin information of the input text.

2. The method for converting Chinese characters into universal Braille as claimed in claim 1, characterized in that: This deep learning model is a language representation model.

3. A method for converting Chinese characters into universal Braille, characterized in that: include: Step 1: construct a deep learning model, and the deep learning model has two subtask branches that share feature encoding, one of which is used to segment Chinese text according to Braille rules, and the other is used to mark pinyin and corresponding tones for each Chinese character; Step 2: Use the Chinese character corpus with marked Braille segmentation and the Chinese character-pinyin comparison corpus to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as an intermediate model; Step 3, obtain Chinese character-universal Braille comparison corpus, and divide it into multiple batches of data, each batch of data contains multiple data, each data contains Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin mark with tone corresponding to the Chinese character text, and use each batch of data to train the intermediate model in turn. During training, the Chinese character text of the batch data is used as input, and the output results of Braille word segmentation and Pinyin mark are generated through two subtask branches respectively. The loss function of each output result is calculated using the Braille word segmentation mark and the Pinyin mark with tone, and then the intermediate model is updated by the back propagation algorithm until the loss function of each output result converges or reaches a preset number of iterations, and the current intermediate model is saved as the recognition model; Step 4: inputting the Chinese character data to be converted into universal Braille into the recognition model, obtaining word segmentation information and tone pinyin information of the Chinese character data, and converting the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information; The bottom layer of this deep learning model is BERT, and two fully connected layers are connected to the upper layer of BERT, one for predicting the word segmentation information of the input text, and the other for predicting the pinyin information of the input text.

4. The method for converting Chinese characters into universal Braille as claimed in claim 3, characterized in that: This deep learning model is a language representation model.

5. A system for converting Chinese characters into universal Braille, characterized in that: include: A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character; A training module is used to use the Chinese character corpus with marked Braille segmentation and the control corpus of Chinese character pinyin to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as a recognition model; A conversion module, used for inputting Chinese character data to be converted into universal Braille into the recognition model, obtaining word segmentation information and tone pinyin information of the Chinese character data, and converting the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information; The bottom layer of this deep learning model is BERT, and two fully connected layers are connected to the upper layer of BERT, one for predicting the word segmentation information of the input text, and the other for predicting the pinyin information of the input text.

6. The Chinese character to universal Braille conversion system as claimed in claim 5, characterized in that: This deep learning model is a language representation model.

7. A system for converting Chinese characters into universal Braille, characterized in that: include: A construction module for constructing a deep learning model, wherein the deep learning model has two subtask branches for sharing feature encoding, wherein one subtask branch is used for segmenting Chinese text according to Braille rules, and the other subtask branch is used for marking pinyin and corresponding tones for each Chinese character; The first training module is used to use the Chinese character corpus with marked Braille segmentation and the Chinese character-pinyin comparison corpus to alternately train the two subtask branches of the deep learning model until the loss functions of the two subtask branches converge or reach a preset number of iterations, and save the current deep learning model as an intermediate model; The second training module is used to obtain Chinese character-universal Braille comparison corpus and divide it into multiple batches of data, each batch of data contains multiple data, each data contains Chinese character text, Braille word segmentation mark corresponding to the Chinese character text, and Pinyin mark with tone corresponding to the Chinese character text, and train the intermediate model with each batch of data in turn. During training, the Chinese character text of the batch data is used as input, and the output results of Braille word segmentation and Pinyin mark are generated through two subtask branches respectively. The loss function of each output result is calculated using the Braille word segmentation mark and the Pinyin mark with tone, and then the intermediate model is updated by the back propagation algorithm until the loss function of each output result converges or reaches a preset number of iterations, and the current intermediate model is saved as the recognition model; A conversion module, used for inputting Chinese character data to be converted into universal Braille into the recognition model, obtaining word segmentation information and tone pinyin information of the Chinese character data, and converting the Chinese character data into corresponding universal Braille text based on the word segmentation information and the tone pinyin information; The bottom layer of this deep learning model is BERT, and two fully connected layers are connected to the upper layer of BERT, one for predicting the word segmentation information of the input text, and the other for predicting the pinyin information of the input text.

8. The Chinese character to universal Braille conversion system as claimed in claim 7, characterized in that: This deep learning model is a language representation model.

9. A storage medium for storing a program for executing the method for converting Chinese characters into universal Braille as claimed in any one of claims 1 to 4.

10. A client, used in any one of the Chinese character to universal Braille conversion systems of claims 5 to 8.

Images