Ancient Zhuang character electronic dictionary system supporting multi-modal retrieval and speech synthesis

Abstract

The invention discloses an ancient Zhuang character electronic dictionary system supporting multi-modal retrieval and speech synthesis. The ancient Zhuang character electronic dictionary system comprises a retrieval module, a pronunciation module, a display module, a database design example and a system architecture. Compared with the prior art, the method has the advantages that a multi-mode input retrieval matching technology is adopted, a standard stroke number matching algorithm is based, a stroke retrieval function of Zhuang Chinese square characters is optimized, a phoneme matching algorithm of pinyin and Zhuang Latin letters is used, a rapid and accurate matching and speech synthesis technology is realized, and a FastSpeech is adopted to generate frequency domain features of a text; and a high-fidelity time domain audio is generated through the HiFi-GAN, so that the naturalness and definition of speech synthesis are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of language and character processing and speech synthesis, and particularly relates to a Zhuang script electronic dictionary system based on multimodal retrieval and speech synthesis. The system integrates natural language processing, speech processing, multimodal data matching, and multimedia display technologies to achieve efficient retrieval, pronunciation, and learning of Zhuang scripts, and is applicable to the fields of language protection, education promotion, and information processing. Background Art

[0002] With the development of informatization and language technologies, electronic dictionaries have been widely used in the field of language learning. For example, electronic dictionary systems for mainstream languages such as Chinese and English have integrated natural language processing (NLP) and text-to-speech (TTS) technologies to achieve multimodal retrieval, efficient query, and standard pronunciation guidance, providing a good user experience for learners. However, in the field of Zhuang language electronic dictionaries, especially in the retrieval and learning of Zhuang square characters, there is a lack of technical support.

[0003] The existing technologies have the following deficiencies:

[0004] 1. Limitations in retrieval methods: Most existing Zhuang language dictionaries are presented in paper form with a low degree of digitization. The retrieval mode only supports text or pinyin input and lacks multimodal support (such as stroke input, fuzzy query, pinyin phoneme matching, etc.), making it difficult to meet the diverse needs of different users.

[0005] 2. Lack of voice output function: Most Zhuang language dictionaries lack the text-to-speech function, and learners are difficult to obtain standard pronunciation guidance for the Zhuang language. Especially for non-native learners, this significantly increases the learning difficulty.

[0006] 3. Insufficient display information: The interface design of existing electronic dictionaries is simple, lacking a clear display of the details of Zhuang square characters and unable to intuitively present the position of entries in traditional paper dictionaries.

[0007] Electronic dictionaries for languages such as Chinese and English have widely adopted technologies such as text-to-speech, multimedia display, and intelligent retrieval, but the application of these technologies in Zhuang language electronic dictionaries is still blank. For the multimodal retrieval and speech generation of Zhuang square characters, a complete technical solution has not yet been formed. This situation is not conducive to the in-depth research of the Zhuang language and the protection and inheritance of ethnic language cultures. Therefore, an electronic dictionary system for Zhuang square characters integrating multimodal retrieval, speech synthesis, and multimedia display functions is urgently needed to meet the needs of modern language learning and information processing. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to overcome the above technical deficiencies and provide an ancient Zhuang script electronic dictionary system that supports multimodal retrieval and speech synthesis.

[0009] To solve the above problems, the technical solution of the present invention is as follows: An ancient Zhuang character electronic dictionary system that supports multimodal retrieval and speech synthesis, including a retrieval module, and the retrieval table module includes stroke retrieval, Zhuang language Latin pinyin retrieval, Chinese character spelling retrieval, Chinese character fuzzy retrieval, special character retrieval, and classification retrieval;

[0010] A pronunciation module, the pronunciation module includes inputting an entry text, loading a speech synthesis model, text-to-spectrum conversion, spectrum-to-audio conversion, and generating a speech output;

[0011] A display module, the display module includes inputting a detection result, data parsing, content classification processing, pronunciation button detection, and outputting a result;

[0012] A database design example, the database design example includes a database table structure, a description of table relationships, and data examples;

[0013] A system architecture, the system architecture includes an input end, a retrieval module, a pronunciation module, a display module, and a database.

[0014] Furthermore, the stroke retrieval module supports users to input the number of strokes or stroke types of Zhuang square characters. The system quickly locates the corresponding words by matching with the standardized stroke data in the dictionary database. The Zhuang language Latin pinyin retrieval is designed for Zhuang language users or users familiar with the Zhuang language pinyin rules. The system designs an input retrieval method based on the Latin letters of the Zhuang language. The Chinese character spelling retrieval module allows users to input complete or partial Chinese character glyphs. The system uses a glyph space structure analysis algorithm to infer possible spelling results and match them with the database. The Chinese character fuzzy retrieval module supports users to perform semantic or glyph matching by inputting Chinese characters. The special character retrieval is to meet the usage needs of users in special scenarios. This module supports inputting custom special characters for retrieval. The classification retrieval module supports users to quickly locate relevant entries according to part-of-speech classification, such as nouns, verbs, adjectives, etc.

[0015] Further, for the input entry text, the text selected by the user for the text input system, such as the Zhuang block characters selected through the retrieval module, will be used as the initial input data for speech generation. The speech synthesis model system loads a pre-trained speech synthesis model, including the FastSpeech and HiFi-GAN modules. The model dynamically selects suitable computing resources through a scheduling mechanism to optimize the response time. The input text is converted from text to spectrogram through the FastSpeech model to generate a Mel spectrogram. In this process, the input Zhuang text features are mapped to the frequency domain features of the time series. The spectrogram is converted from spectrogram to audio through the HiFi-GAN model into a high-fidelity audio waveform. This model combines the training method of the generative adversarial network and has significant advantages in speech naturalness and authenticity. After post-processing, the generated speech outputs an audio signal in the form of an audio file or real-time playback. Users can directly listen to the generated Zhuang pronunciation or save it as an audio file.

[0016] Further, for the input detection result, the user or the system will input the retrieval result as the initial input content for data processing. The input data is processed through the parsing module to extract key information, such as characters, meanings, position information, etc. According to the parsing result, the content classification process classifies the content into the following four categories: displaying block characters, displaying interpretations, displaying in which dictionary, and displaying dictionary location information. The pronunciation button detection is used to determine whether the user needs the pronunciation function. The output result displays the final result, including text information, interpretations, dictionary location information, or voice playback content.

[0017] Further, the database table structure includes a block character main table, an entry table, a Chinese character set table, and an audio resource table. The table relationship description includes the relationship between the block character main table and the entry table, the block character main table and the Chinese character set table, and the block character main table and the audio resource table. The data examples include the data of the block character main table, the entry table, the Chinese character set table, and the audio resource table.

[0018] Further, the input end provides multiple input methods, including strokes, Chinese character spelling, Chinese character fuzzy query, Zhuang Latin letters, special characters, and classification selection. The retrieval module realizes accurate retrieval by matching the user input with the entry data in the database. The pronunciation module receives the entry data selected by the user and generates a speech signal through FastSpeech and HiFi-GAN. The display module presents the search results, including block character pictures, interpretations, and relative positions in the dictionary. The database stores all Zhuang block character entries and their related information, including pinyin, interpretations, voice data, and dictionary page positions, etc.

[0019] Further, the main table of Chinese characters stores the core information of Zhuang Chinese characters, including Chinese character association, spelling composition, Zhuang language pinyin, etc. The entry table stores entry-related information, including interpretation, part of speech, and dictionary location. The Chinese character character set table stores the character set information of Chinese character spelling. The audio resource table stores the link or path information of voice data.

[0020] The advantages of the present invention compared with the existing technology are as follows:

[0021] 1. The multi-mode input retrieval matching technology, based on the standard stroke number matching algorithm, optimizes the stroke retrieval function of Zhuangwen Chinese characters, and uses the phoneme matching algorithm of pinyin and Zhuang language Latin letters to achieve fast and accurate matching.

[0022] 2. The speech synthesis technology uses FastSpeech to generate the frequency domain features (Mel spectrogram) of the text, and then uses HiFi-GAN to generate high-fidelity time domain audio to ensure the naturalness and clarity of speech synthesis.

[0023] 3. The coordinate annotation and relative position calculation utilize the coordinate data generated when splitting Zhuangwen Chinese characters, combined with the dictionary page size and layout, to accurately annotate the position of each entry on the original dictionary page and visually present it through the display module. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the system architecture of the present invention.

[0025] Figure 2 It is a schematic diagram of the retrieval module process of the present invention.

[0026] Figure 3 It is a schematic diagram of the pronunciation module process of the present invention.

[0027] Figure 4 It is a schematic diagram of the display module of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0028] The following will further illustrate the specific implementation manners of the present invention with reference to the accompanying drawings. Among them, the same components are denoted by the same reference numerals.

[0029] It should be noted that the terms "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to the directions in the drawings, and the terms "inner" and "outer" respectively refer to the directions towards or away from the geometric center of a specific component.

[0030] In order to make the content of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] Embodiment 1

[0032] As Figures 1 to 4 shown, an ancient Zhuang character electronic dictionary system that supports multimodal retrieval and speech synthesis includes a retrieval module. The retrieval table module includes stroke retrieval, Zhuang language Latin pinyin retrieval, Chinese character spelling retrieval, Chinese character fuzzy retrieval, special character retrieval, and classification retrieval;

[0033] The stroke retrieval module supports users to input the number of strokes or stroke types of Zhuang square characters. The system quickly locates the corresponding words by matching with the standardized stroke data in the dictionary database. This function is mainly for users who know the characters and want to find the pronunciation. In the module design, the disassembling logic of complex glyphs and the unified processing of simplified and traditional Chinese characters are considered. The Zhuang language Latin pinyin retrieval is for Zhuang language users or users familiar with the Zhuang language pinyin rules. The system designs an input retrieval method based on the Latin letters of the Zhuang language. By inputting the Latin pinyin of Zhuang language words, the system automatically matches the relevant entries in the database according to phonemes. The matching algorithm combines the unique phoneme rules, tone information, and fault tolerance design of common spelling errors in the Zhuang language to ensure the comprehensiveness and accuracy of the retrieval results. The Chinese character spelling retrieval module allows users to input complete or partial Chinese character glyphs. The system uses the glyph space structure analysis algorithm to infer possible spelling results and match them with the database. This module is especially suitable for scenarios where users have a vague memory but have an impression of the glyph, such as only remembering the components, structure, or general outline of a certain character. The core algorithm of the module combines the radical features and structure types of Chinese characters, such as left-right, up-down, and inside-outside structures, as well as the stroke order rules, greatly improving the retrieval efficiency in the case of fuzzy input. The Chinese character fuzzy retrieval module supports users to match the word meaning or glyph by inputting Chinese characters. When preprocessing the picture data, the word meaning of the word is associated with the picture as the label content. The input Chinese character is matched with the labeled word meaning to retrieve the relevant entries; in terms of glyph matching, the similarity calculation of strokes, radicals, and structures is combined to provide the most relevant retrieval results for users. This module can be widely used in scenarios such as finding polysemous words, distinguishing variant characters, and fuzzy positioning of phrases. The special character retrieval is to meet the usage needs of users in special scenarios. This module supports inputting custom special characters for retrieval. Special characters include but are not limited to mathematical symbols, scientific and technological symbols, and specific cultural symbols. The system realizes the accurate comparison between the user input and the database characters through the built-in symbol coding rule library and extended matching algorithm, ensuring the efficiency and reliability of the retrieval results. This module has significant advantages for users in fields such as education, scientific research, and technology. The classification retrieval module supports users to quickly locate relevant entries according to part-of-speech classification, such as nouns, verbs, adjectives, etc. The module design is based on a multi-level classification system. Through the structured data processing of part-of-speech tagging and the optimization of classification indexing, the retrieval time is significantly shortened, which is suitable for scenarios of language learning and efficient language material searching;

[0034] Pronunciation module. The pronunciation module includes inputting entry text, loading a speech synthesis model, text-to-spectrum conversion, spectrum-to-audio conversion, and generating voice output;

[0035] Inputting entry text: The entry text selected by the user is input into the system, such as the Zhuang square characters selected through the retrieval module. This text will serve as the initial input data for voice generation, supporting Zhuang Latin letters, Zhuang pinyin, and standard text input. The input entry is preprocessed standardly, such as removing redundant characters and correcting the pinyin format, to adapt to the model input. Loading the speech synthesis model: The system loads a pre-trained speech synthesis model, including the FastSpeech and HiFi-GAN modules. The model dynamically selects suitable computing resources through a scheduling mechanism to optimize the response time. FastSpeech, as the main model for text-to-spectrum conversion, adopts a non-autoregressive structure, significantly reducing the latency problem of speech synthesis. HiFi-GAN: As the vocoder for spectrum-to-audio conversion, it generates high-fidelity speech waveforms. The model loading supports dynamic adjustment, and a lightweight model is selected according to the performance of the terminal device, such as the version after knowledge distillation, supporting multi-threaded loading, improving parallel processing capabilities, and shortening the loading time. Text-to-spectrum conversion: The input text generates a Mel spectrogram through the FastSpeech model. This process maps the input Zhuang text features into frequency-domain features of a time series, uses the Transformer architecture for text feature modeling, captures the association between semantic context and pronunciation rules, optimizes the unique phoneme characteristics of Zhuang language, and ensures the correct mapping of tones and pinyin. The Mel spectrogram, as an intermediate result, represents the frequency-domain distribution of the generated voice. Spectrum-to-audio conversion: The spectrogram is converted into a high-fidelity audio waveform through the HiFi-GAN model. This model combines the training method of generative adversarial networks and has significant advantages in speech naturalness and authenticity. The HiFi-GAN model decodes the Mel spectrogram to generate a time-domain audio signal, and uses perceptual loss to optimize model training to ensure that the generated speech has an auditory sense close to natural recordings. The high-fidelity audio signal completely retains the intonation, speech rate, and timbre characteristics of Zhuang language. Generating voice output: After post-processing, the audio signal is output in the form of an audio file or real-time playback. The user can directly listen to the generated Zhuang pronunciation or save it as an audio file, with playback function, file saving function, and diversified settings. The playback function supports real-time playback of the generated voice, providing intuitive pronunciation guidance. The file can be saved as an audio in standard formats such as WAV or MP3 files for subsequent learning and reference. The user can adjust the pronunciation speed, tone, and volume according to needs to complete diversified settings;

[0036] Display module. The display module includes inputting detection results, data parsing, content classification processing, pronunciation button detection, and outputting results;

[0037] The user or system inputs the retrieval result as the initial input content for data processing. The input data is processed by the parsing module to extract key information such as characters, meanings, location information, etc. Content classification processing: Based on the parsing result, the system classifies the content into the following four categories: displaying Chinese characters, displaying explanations, indicating which dictionary, and displaying dictionary location information. Displaying Chinese characters is used to extract and display the Chinese characters in the input. Displaying explanations is used to provide explanations or translation information for the Chinese characters. Indicating which dictionary is used to mark the position or source of the Chinese characters in a specific dictionary. Displaying dictionary location information is used to provide more precise specific location information of the Chinese characters in the dictionary, such as page numbers or entry numbers. Pronunciation button detection: It is used to determine whether the user needs the pronunciation function. If the user presses the pronunciation button, the system will output the voice information of the retrieval result. If the pronunciation button is not pressed, it will directly jump to the end of the program. Output result display: The final result includes text information, explanations, dictionary location information, or voice playback content;

[0038] Database design example. The database design example includes database table structures, table relationship descriptions, and data examples. The database table structures include the Chinese character main table, entry table, Chinese character character set table, and audio resource table. The table relationship descriptions include the Chinese character main table, entry table, the relationship between the Chinese character main table and the Chinese character character set table, and the relationship between the Chinese character main table and the audio resource table. The Chinese character main table stores the core information of Zhuang Chinese characters, including Chinese character associations, spelling compositions, Zhuang Chinese pinyin, etc. The entry table stores entry-related information, including explanations, parts of speech, and dictionary locations. The Chinese character character set table stores the character set information of Chinese character spellings. The audio resource table stores the link or path information of voice data, as shown in the following table: Chinese character main table

[0039]

[0040] Entry table

[0041]

[0042] Chinese character character set table

[0043]

[0044]

[0045] Audio resource table

[0046]

[0047] The data example includes data from the Chinese character main table, entry table, Chinese character character set table, and audio resource table;

[0048] Table relationship description: there is a one-to-many relationship between the Chinese character main table and the entry table. One Chinese character may correspond to multiple entries, which are stored in the entry table. There is a many-to-many relationship between the Chinese character main table and the Chinese character set table. One Chinese character may be composed of multiple Chinese characters. The Chinese character set table stores all supported spelling Chinese characters. There is a one-to-one relationship between the Chinese character main table and the audio resource table. One Chinese character usually has a corresponding voice file. The audio resource table stores the specific audio path and related information, as shown in the following table;

[0049] Chinese character main table

[0050]

[0051]

[0052] Chinese character set table

[0053]

[0054] Audio resource table

[0055]

[0056] System architecture, the system architecture includes an input end, a retrieval module, a pronunciation module, a display module and a database;

[0057] The input end provides various input methods, including strokes, Chinese character spelling, Chinese character fuzzy query, Zhuang language Latin letters, special characters and classification selection. The retrieval module realizes accurate retrieval by matching the user input with the entry data in the database. The pronunciation module receives the entry data selected by the user and generates a voice signal through FastSpeech and HiFi-GAN. The display module presents the search results, including Chinese character pictures, interpretations and relative positions in the dictionary. The database stores all Zhuang language Chinese character entries and their related information, including pinyin, interpretations, voice data and dictionary page positions, etc.

[0058] The above describes the present invention and its implementation manners. This description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention. The actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and without departing from the gist of the present invention creation, and design similar structural manners and embodiments to this technical solution without creative efforts, they shall fall within the protection scope of the present invention.

Claims

1. An ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis, characterized in that: It includes a retrieval module, and the retrieval table module includes stroke retrieval, Zhuang language Latin pinyin retrieval, Chinese character spelling retrieval, Chinese character fuzzy retrieval, special character retrieval, and classification retrieval; A pronunciation module, the pronunciation module includes inputting an entry text, loading a speech synthesis model, text-to-spectrum conversion, spectrum-to-audio conversion, and generating a voice output; A display module, the display module includes inputting detection results, data parsing, content classification processing, pronunciation button detection, and outputting results; A database design example, the database design example includes a database table structure, a description of table relationships, and data examples; A system architecture, the system architecture includes an input end, a retrieval module, a pronunciation module, a display module, and a database.

2. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 1, characterized in that: The stroke retrieval module supports the user to input the number of strokes or stroke types of Zhuang language square characters. The system matches with the standardized stroke data in the dictionary database to quickly locate the corresponding words. For the Zhuang language Latin pinyin retrieval, for Zhuang language users or users familiar with the Zhuang language pinyin rules, the system designs an input retrieval method based on the Zhuang language Latin alphabet. The Chinese character spelling retrieval module allows the user to input a complete or partial Chinese character glyph. The system uses a glyph spatial structure analysis algorithm to infer possible spelling results and match them with the database. The Chinese character fuzzy retrieval module supports the user to perform word meaning or glyph matching by inputting Chinese characters. For the special character retrieval, to meet the user's usage requirements in special scenarios, this module supports inputting custom special characters for retrieval. The classification retrieval module supports the user to quickly locate relevant entries according to part-of-speech classification, such as nouns, verbs, adjectives, etc.

3. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 1, characterized in that: The input entry text is the entry text selected by the user and input into the system. For example, the Zhuang language square characters selected through the retrieval module. This text will be used as the initial input data for voice generation. The system loads a pre-trained speech synthesis model, including the FastSpeech and HiFi-GAN modules. The model dynamically selects suitable computing resources through a scheduling mechanism to optimize the response time. The input text is converted into a Mel spectrogram through the FastSpeech model. This process maps the input Zhuang language text features into time-series frequency domain features. The spectrogram is converted into a high-fidelity audio waveform through the HiFi-GAN model. This model combines the training method of a generative adversarial network and has significant advantages in speech naturalness and authenticity. After the audio signal is post-processed, it is output in the form of an audio file or real-time playback. The user can directly listen to the generated Zhuang language pronunciation or save it as an audio file.

4. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 1, characterized in that: The input detection result is that the user or system takes the input retrieval result as the initial input content for data processing. The input data is processed by the parsing module to extract key information such as characters, meanings, position information, etc. According to the parsing result, the content classification process classifies the content into the following four categories: displaying Chinese characters, displaying interpretations, displaying in which dictionary, and displaying dictionary location information. The pronunciation button detection is used to determine whether the user needs the pronunciation function. The output result displays the final result, including text information, interpretations, dictionary location information, or voice playback content.

5. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 1, characterized in that: The database table structure includes the main table of Chinese characters, the entry table, the Chinese character set table, and the audio resource table. The table relationship description includes the relationship between the main table of Chinese characters and the entry table, the main table of Chinese characters and the Chinese character set table, and the main table of Chinese characters and the audio resource table. The data examples include the data of the main table of Chinese characters, the data of the entry table, the data of the Chinese character set table, and the data of the audio resource table.

6. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 1, characterized in that: The input end provides multiple input methods, including strokes, Chinese character spelling, Chinese character fuzzy query, Zhuang language Latin letters, special characters, and classification selection. The retrieval module realizes accurate retrieval by matching the user input with the entry data in the database. The pronunciation module receives the entry data selected by the user and generates a voice signal through FastSpeech and HiFi-GAN. The display module presents the search results, including Chinese character pictures, interpretations, and relative positions in the dictionary. The database stores all Zhuang language Chinese character entries and their related information, including pinyin, interpretations, voice data, and dictionary page positions, etc.

7. The ancient Zhuang character electronic dictionary system supporting multimodal retrieval and speech synthesis according to claim 5, characterized in that: The main table of Chinese characters stores the core information of Zhuang language Chinese characters, including Chinese character associations, spelling compositions, Zhuang language pinyin, etc. The entry table stores entry-related information, including interpretations, parts of speech, and dictionary locations. The Chinese character set table stores the character set information of Chinese character spellings. The audio resource table stores the link or path information of voice data.

Images