Method and system for constructing and interacting Yao language corpus based on artificial intelligence

Abstract

The invention discloses a Yao language corpus construction and interaction method and system based on artificial intelligence, and the method comprises the steps: constructing a multi-modal Yao language data set, and carrying out the data preprocessing, and obtaining a preprocessed multi-modal Yao language data set; a double-encoder structure and a bilingual semantic alignment module are introduced, and a Yao-Chinese bilingual semantic retrieval interaction model is constructed; and based on a Yao-Chinese bilingual semantic retrieval interaction model, performing semantic similarity retrieval and interaction on the preprocessed multi-modal Yao-language data set to realize Yao-Chinese bilingual autonomous interaction. According to the method, manual intervention can be reduced, and the retrieval query accuracy of the Yao language is improved. The Yao language corpus construction and interaction method and system based on artificial intelligence can be widely applied to the technical field of natural language processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a Yao language corpus construction and interaction method and system based on artificial intelligence. BACKGROUND

[0002] As the main language of the Yao nationality in China, Yao language belongs to the Miao-Yao language family and has rich dialect branches (such as Mian Yao and Bunu). It is an important carrier of Yao culture. With the advent of the digital era, the protection and inheritance of minority languages have become increasingly important. Existing technologies mainly rely on manual collection of Yao language texts, audio and video materials to form a corpus, and build a dictionary system through a traditional database. For example, existing digital projects for minority languages (such as the AI revival project for the Nüshu language) implement query functions through rule-based methods or simple database storage. These systems usually include corpus collection, annotation and retrieval modules, supporting basic text search and translation.

[0003] However, in the prior art, corpus construction relies on manual annotation and collection, resulting in low efficiency and limited coverage, and being unable to handle the dialect diversity and non-standard expressions of Yao language. At the same time, digital dictionaries are mostly static storage, lacking semantic search and multi-modal interaction functions based on deep learning, and are difficult to adapt to user individual needs. For example, traditional methods are prone to low accuracy when dealing with phonetic variants or semantic associations of Yao language, and are unable to update the corpus in real time. In addition, existing systems ignore the potential of Transformer models in semantic understanding and cultural inheritance, resulting in poor language protection effects. SUMMARY

[0004] To solve the above technical problems, the purpose of the present application is to provide a Yao language corpus construction and interaction method and system based on artificial intelligence, which can reduce manual intervention and improve the accuracy of Yao language retrieval queries.

[0005] The first technical solution adopted by the present application is: a Yao language corpus construction and interaction method based on artificial intelligence, comprising the following steps: constructing a multi-modal Yao language dataset and performing data preprocessing to obtain a preprocessed multi-modal Yao language dataset; introducing a dual-encoder structure and a bilingual semantic alignment module to construct a Yao-Chinese bilingual semantic retrieval interaction model; based on the Yao-Chinese bilingual semantic retrieval interaction model, performing semantic similarity retrieval and interaction on the preprocessed multi-modal Yao language dataset to realize autonomous interaction between Yao and Chinese.

[0006] Further, the step of constructing a multi-modal Yao language dataset and performing data preprocessing to obtain a preprocessed multi-modal Yao language dataset specifically comprises: Collecting Yao language text data, Yao language voice recording and image data related to Yao language, and constructing a multi-modal Yao language data set; For the multi-modal Yao language data set, text cleaning, voice denoising, uniform conversion and image cropping and standardization are sequentially performed to obtain a preliminary pre-processed multi-modal Yao language data set; The preliminary pre-processed multi-modal Yao language data set is respectively marked with language category, assigned with dialect label and annotated with Chinese explanation to obtain a pre-processed multi-modal Yao language data set.

[0007] Further, the Yao-Chinese bilingual semantic retrieval interaction model specifically includes a double encoder structure, a bilingual semantic alignment module, a vector index module, a knowledge graph construction module, a semantic retrieval module and a Yao-Chinese bilingual interaction module, wherein: The double encoder structure includes a Yao language encoder and a Chinese language encoder, and the Yao language encoder includes a text encoding layer, a dialect selection and combination layer, a voice encoding layer and an image encoding layer. The knowledge graph construction module includes a vocabulary layer, a sentence pattern layer, a knowledge layer and a dynamic update layer.

[0008] Further, based on the Yao-Chinese bilingual semantic retrieval interaction model, the semantic similarity retrieval and interaction of the pre-processed multi-modal Yao language data set are performed to realize the step of Yao-Chinese bilingual autonomous interaction, which specifically includes: The pre-processed multi-modal Yao language data set is input into the Yao-Chinese bilingual semantic retrieval interaction model; Based on the double encoder structure of the Yao-Chinese bilingual semantic retrieval interaction model, the pre-processed multi-modal Yao language data set is semantically encoded to obtain Yao language encoding vectors and Chinese language encoding vectors; Based on the bilingual semantic alignment module of the Yao-Chinese bilingual semantic retrieval interaction model, the Yao language encoding vectors and the Chinese language encoding vectors are subjected to multi-modal semantic alignment processing to obtain aligned semantic term vectors; Based on the vector index module of the Yao-Chinese bilingual semantic retrieval interaction model, the vector index is constructed through the approximate nearest neighbor retrieval structure, the aligned semantic term vectors are subjected to similarity retrieval, and the semantic retrieval matching result is obtained; Based on the knowledge graph construction module of the Yao-Chinese bilingual semantic retrieval interaction model, the dynamic dictionary and knowledge graph are constructed according to the semantic retrieval matching result to obtain a knowledge graph structure continuously evolving with the corpus; Based on the semantic retrieval module of the Yao-Chinese bilingual semantic retrieval interaction model, the semantic retrieval of the user's input is performed based on the knowledge graph structure continuously evolving with the corpus to obtain the semantic retrieval matching result; Based on the Yao-Chinese bilingual interaction module of the Yao-Chinese bilingual semantic retrieval interaction model, the semantic retrieval matching result is interacted with the user to realize Yao-Chinese bilingual autonomous interaction.

[0009] Further, the dual-encoder structure based on the Yao-Han bilingual semantic retrieval interaction model encodes the pre-processed multi-modal Yao language data set to obtain Yao language encoding vectors and Chinese encoding vectors, which specifically includes: Input the pre-processed multi-modal Yao language data set into the dual-encoder structure of the Yao-Han bilingual semantic retrieval interaction model; The Yao language encoder based on the dual-encoder structure encodes the pre-processed multi-modal Yao language data set to obtain Yao language encoding vectors; The Chinese encoder based on the dual-encoder structure encodes the Chinese explanation annotations in the pre-processed multi-modal Yao language data set to obtain Chinese encoding vectors.

[0010] Further, the Yao language encoder based on the dual-encoder structure encodes the pre-processed multi-modal Yao language data set to obtain Yao language encoding vectors, which specifically includes: Input the pre-processed multi-modal Yao language data set into the Yao language encoder of the dual-encoder structure; The text encoding layer based on the Yao language encoder encodes the pre-processed multi-modal Yao language data set to obtain a text encoding structure of the Yao language; The dialect selection and combination layer based on the Yao language encoder automatically identifies the dialect labels of the text encoding structure of the Yao language to obtain a dialect adaptive text vector; The speech encoding layer based on the Yao language encoder encodes the pre-processed multi-modal Yao language data set to obtain a Yao language speech vector; The image encoding layer based on the Yao language encoder extracts image visual features from the pre-processed multi-modal Yao language data set to obtain a Yao language image vector; Map and align the dialect adaptive text vector, the Yao language speech vector, and the Yao language image vector to obtain a Yao language encoding vector.

[0011] Further, the knowledge graph construction module based on the Yao-Han bilingual semantic retrieval interaction model constructs a dynamic dictionary and a knowledge graph according to the semantic retrieval matching result to obtain a knowledge graph structure that evolves with the corpus, which specifically includes: Input the semantic retrieval matching result into the knowledge graph construction module of the Yao-Han bilingual semantic retrieval interaction model; The vocabulary layer based on the knowledge graph construction module splits and counts the vocabulary of the semantic retrieval matching result to obtain basic semantic vector frequency information; The sentence pattern layer based on the knowledge graph construction module connects the sentence pattern nodes and related word node based on the basic semantic vector frequency information to obtain semantic sentence pattern nodes; The knowledge layer and the dynamic updating layer based on the knowledge graph construction module are used for connecting and dynamically updating the semantic sentence nodes and the cultural knowledge nodes, so as to obtain a knowledge graph structure continuously evolved with corpus.

[0012] Further, the semantic retrieval module based on the Yao-Han bilingual semantic retrieval interaction model performs semantic retrieval on the input of the user based on the knowledge graph structure continuously evolved with corpus, and obtains a semantic retrieval matching result, and the step specifically includes: dialect adaptive multi-modal coding is performed on the input of the user to obtain a unified semantic vector; the knowledge graph structure continuously evolved with corpus is input to the semantic retrieval module of the Yao-Han bilingual semantic retrieval interaction model; the unified semantic vector is used as a query vector, and semantic retrieval is performed in combination with the knowledge graph structure continuously evolved with corpus to construct a retrieval knowledge set; the retrieval knowledge set is subjected to retrieval enhancement generation to obtain a semantic retrieval matching result.

[0013] The second technical scheme adopted by the present application is a Yao language corpus construction and interaction system based on artificial intelligence, comprising: a first module for constructing a multi-modal Yao language data set and performing data preprocessing to obtain a preprocessed multi-modal Yao language data set; a second module for introducing a double-encoder structure and a bilingual semantic alignment module to construct a Yao-Han bilingual semantic retrieval interaction model; a third module for performing semantic similarity retrieval and interaction on the preprocessed multi-modal Yao language data set based on the Yao-Han bilingual semantic retrieval interaction model to realize autonomous interaction between Yao and Han languages.

[0014] The present application has the following advantages: the present application constructs a multi-modal Yao language data set and performs data preprocessing, then introduces a double-encoder structure and a bilingual semantic alignment module to construct a Yao-Han bilingual semantic retrieval interaction model, multi-modal coding realizes unified semantic representation of Yao language text, voice and even images on the model structure, and through a dialect expert mechanism, differences between different dialects are explicitly captured, the adaptability to dialect diversity and non-standard expression is improved, finally, based on the Yao-Han bilingual semantic retrieval interaction model, semantic similarity retrieval and interaction are performed on the preprocessed multi-modal Yao language data set to realize autonomous interaction between Yao and Han languages, multi-modal interaction is supported, the query accuracy is improved, the cultural inheritance function is enhanced through the knowledge graph, the problems of staticity and poor adaptability of traditional systems are solved, artificial intervention is reduced, and the retrieval and query accuracy of the Yao language is improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1is a step flow chart of a Yao language corpus construction and interaction method based on artificial intelligence according to the present application; Figure 2 is a structural block diagram of a Yao language corpus construction and interaction system based on artificial intelligence according to the present application. DETAILED DESCRIPTION

[0016] The present application will be further described in detail below in combination with the drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0017] First of all, it needs to be pointed out that the related art, for example, a kind of NLP knowledge graph construction method combined with information quantity and BERT_BiLSTM_CRF, the mode layer structure of the knowledge graph in the field of natural language processing is proposed by analyzing the structure of the CNKI journal paper data and combining the research task of natural language processing;Then a new word discovery algorithm is proposed to obtain the key term entity class in the paper data, and a feature fusion multi-classification algorithm is proposed to obtain the fine-grained NLP research task entity class of the paper. Further, the knowledge extraction module obtains the triple, and finally the content of the natural language processing knowledge graph is constructed.

[0018] Based on this, the present application aims at the problem that the traditional corpus construction relies on manual annotation and collection, resulting in low efficiency, limited coverage, and inability to handle the dialect diversity and non-standard expression of Yao language. At the same time, the digital dictionary is mostly stored statically, lacking semantic search and multi-modal interaction functions based on deep learning, and is difficult to adapt to the personalized needs of users. The present application embodiment collects multi-modal data of Yao language, including text, audio and image;Using natural language processing (NLP) model to preprocess the collected data, including word segmentation, annotation and denoising, to obtain standardized Yao language corpus;Based on deep learning model (such as BERT variant), semantic features are extracted to form corpus index;Based on the processed corpus, a Yao-Han bidirectional dictionary is constructed to support semantic search and translation;Integrating speech recognition and image recognition modules, multi-modal query is realized, and Yao language cultural elements are associated through knowledge graph, the purpose of the present application embodiment is to construct an efficient Yao language corpus and digital dictionary system through Transformer model, to improve the automation degree of corpus collection, the intelligence of dictionary query and the interactivity of cultural heritage, to solve the shortcomings of low efficiency, poor accuracy and weak adaptability of the prior art.

[0019] Referring to Figure 1 , the present application provides a kind of Yao language corpus construction and interaction method based on artificial intelligence, which comprises the following steps: S100, construct a multi-modal Yao language data set and perform data preprocessing to obtain a preprocessed multi-modal Yao language data set; Specifically, Yao language text materials, Yao language audio recordings, and image materials related to Yao language are collected to construct a multi-modal Yao language data set. The multi-modal Yao language data set is sequentially subjected to text cleaning, speech denoising and unified conversion processing, and image cropping and standardization processing to obtain a preliminarily preprocessed multi-modal Yao language data set. The preliminarily preprocessed multi-modal Yao language data set is subjected to language category labeling, dialect labeling, and Chinese explanation labeling processing to obtain a preprocessed multi-modal Yao language data set.

[0020] In the present embodiment, Yao language corpus from multiple channels is collected, including Yao language text materials (folk stories, songs, dialogue scripts, teaching materials, etc.), Yao language audio recordings (daily conversations, story telling, teaching recordings, etc.), and corresponding Chinese explanations, and image materials related to Yao language (traditional document scans, handwritten Yao language, signboard pictures, etc.).

[0021] The collected data is further preprocessed. First, text cleaning is performed to remove garbled codes and duplicate content, unify the coding format, and retain the original dialect spelling or transcription. Then, speech preprocessing is performed, including denoising, segmentation, and sampling rate unification, and (where possible) pairing with preliminary manual transcription or automatic transcription text. Finally, image preprocessing is performed, including cropping and resolution standardization, to prepare for subsequent image coding.

[0022] Based on the dialect labeling rules and recognition model, metadata is added to each sample, including: 1) Dialect category (such as Mian Yao language, Bunu language, etc.); 2) Speaker or source region label; 3) Corresponding Chinese explanation, keywords, or labels.

[0023] In summary, a multi-modal Yao language data set with dialect labels and basic annotations is formed, providing a foundation for subsequent modeling.

[0024] S200, introduce a dual-encoder structure and a bilingual semantic alignment module to construct a Yao-Chinese bilingual semantic retrieval interaction model; Specifically, the Yao-Chinese bilingual semantic retrieval interaction model specifically includes a dual-encoder structure, a bilingual semantic alignment module, a vector index module, a knowledge graph construction module, a semantic retrieval module, and a Yao-Chinese bilingual interaction module. The dual-encoder structure includes a Yao language encoder and a Chinese language encoder. The Yao language encoder includes a text encoding layer, a dialect selection and combination layer, a speech encoding layer, and an image encoding layer. The knowledge graph construction module includes a vocabulary layer, a sentence pattern layer, a knowledge layer, and a dynamic update layer.

[0025] S300, based on the Yao-Han bilingual semantic retrieval interaction model, the preprocessed multi-modal Yao language data set is subjected to semantic similarity retrieval and interaction, and Yao-Han bilingual autonomous interaction is realized.

[0026] S310, the preprocessed multi-modal Yao language data set is input to the Yao-Han bilingual semantic retrieval interaction model; S320, based on the double encoder structure of the Yao-Han bilingual semantic retrieval interaction model, the preprocessed multi-modal Yao language data set is subjected to semantic coding to obtain a Yao language coding vector and a Chinese coding vector; Specifically, the preprocessed multi-modal Yao language data set is input to the double encoder structure of the Yao-Han bilingual semantic retrieval interaction model; based on the Yao language encoder of the double encoder structure, the preprocessed multi-modal Yao language data set is subjected to dialect adaptive multi-modal coding to obtain a Yao language coding vector; based on the Chinese encoder of the double encoder structure, the Chinese explanation annotation in the preprocessed multi-modal Yao language data set is coded to obtain a Chinese coding vector.

[0027] In this embodiment, a Yao language encoder and a Chinese encoder are constructed to code the Yao language input and the Chinese input respectively; the Chinese encoder can use an existing Chinese pre-training model and is adjusted to a unified semantic space through a projection layer.

[0028] The preprocessed multi-modal Yao language data set is input to the Yao language encoder of the double encoder structure; based on the text coding layer of the Yao language encoder, the preprocessed multi-modal Yao language data set is subjected to text coding processing to obtain a text coding structure of the Yao language; based on the dialect selection and combination layer of the Yao language encoder, the text coding structure of the Yao language is subjected to automatic recognition of dialect labels to obtain a dialect adaptive text vector; based on the speech coding layer of the Yao language encoder, the preprocessed multi-modal Yao language data set is subjected to speech segment coding to obtain a Yao language speech vector; based on the image coding layer of the Yao language encoder, the preprocessed multi-modal Yao language data set is subjected to image visual feature extraction to obtain a Yao language image vector; the dialect adaptive text vector, the Yao language speech vector and the Yao language image vector are subjected to mapping alignment processing to obtain a Yao language coding vector.

[0029] In this embodiment, for dialect adaptive multi-modal coding, it includes: 1) Text coding; A Yao language pre-training Transformer encoder is constructed for coding Yao language text and transcribed speech text; on the shared encoder, multiple dialect expert sub-networks or lightweight adaptation layers (Adapter / LoRA) are set for different dialects to form a text coding structure of "shared base + dialect specialization".

[0030] 2) Dialect selection and combination; Through an independent dialect recognition model or a routing network, a corresponding expert sub-network is selected according to a dialect label or an automatic recognition result of an input sample, or multiple expert outputs are combined with a certain weight; and a shared coding output and a dialect expert output are fused to obtain a final dialect adaptive text vector representation.

[0031] 3) Speech coding; An end-to-end Yao language speech encoder (such as an acoustic model based on self-supervised learning) is used to encode an original speech segment into a speech vector; and the speech vector is mapped to the same Yao language semantic space as the text encoder through a projection layer, so that the speech representation of the same speech content is similar to the text transcription representation thereof.

[0032] 4) Image coding; A visual encoder is used to extract a feature vector from a Yao language related image; and an image vector is mapped to a unified semantic space through a cross-modal projection mechanism, so as to facilitate alignment with text and speech vectors.

[0033] Through the above dialect adaptive multi-modal coding, the application realizes unified semantic representation of Yao language text, speech and even images in the model structure, and explicitly captures differences between different dialects through a dialect expert mechanism, thereby improving the adaptability to dialect diversity and non-standard expression.

[0034] In S330, a bilingual semantic alignment module based on a Yao-Chinese bilingual semantic retrieval interaction model performs multi-modal semantic alignment processing on Yao language coding vectors and Chinese coding vectors to obtain aligned semantic entry vectors. In this embodiment, based on Yao-Chinese parallel corpus (including a vocabulary correspondence table, bilingual example sentences, translation dialogue data, etc.), a contrastive learning loss function is used to make corresponding Yao-Chinese vectors close in the semantic space and make non-corresponding samples far apart; and multi-modal contrastive learning is used to align Yao language speech, Yao language text and Chinese interpretation with the same meaning in the semantic space, thereby realizing multi-modal semantic alignment of speech-text-Chinese.

[0035] In S340, a vector indexing module based on the Yao-Chinese bilingual semantic retrieval interaction model constructs a vector index through an approximate nearest neighbor retrieval structure, performs similarity retrieval on the aligned semantic entry vectors, and obtains a semantic retrieval matching result. In this embodiment, the aligned Yao language entry vectors, phrase vectors, example sentence vectors and Chinese interpretation vectors are stored in a high-dimensional vector database; and an approximate nearest neighbor (ANN) retrieval structure is used to construct a vector index, so that subsequent similarity retrieval can be quickly performed on a large-scale corpus.

[0036] In summary, through the bilingual semantic alignment and vector indexing layer, the application can support "Yao-Chinese" and "Yao-Yao dialect" retrieval based on semantic similarity, rather than simple character matching, providing a foundation for multilingual and multi-dialect intelligent query.

[0037] S350, based on the knowledge graph construction module of the Yao-Chinese bilingual semantic retrieval interaction model, the dynamic dictionary and knowledge graph are constructed according to the semantic retrieval matching result, and the knowledge graph structure continuously evolving with the corpus is obtained; Specifically, the semantic retrieval matching result is input to the knowledge graph construction module of the Yao-Chinese bilingual semantic retrieval interaction model; based on the vocabulary layer of the knowledge graph construction module, the semantic retrieval matching result is divided and counted, and the basic semantic vector frequency information is obtained; based on the sentence pattern layer of the knowledge graph construction module, the sentence pattern node and the related word node are connected according to the basic semantic vector frequency information, and the semantic sentence pattern node is obtained; based on the knowledge layer and the dynamic updating layer of the knowledge graph construction module, the semantic sentence pattern node and the cultural knowledge node are connected and dynamically updated, and the knowledge graph structure continuously evolving with the corpus is obtained.

[0038] In this embodiment, for the construction and updating of the dynamic dictionary and the knowledge graph, including: 1) vocabulary layer construction; In the unified semantic space, the coded Yao language text is divided and counted, combined with the word segmentation, sub-word unit and clustering method, the high-frequency word, fixed collocation and common phrase are automatically discovered; a node is established for each word, and its Yao language spelling, phonetic alphabet, dialect label, basic semantic vector and appearance frequency are recorded.

[0039] 2) sentence pattern layer construction; Common sentence patterns, sentence type templates and dialogue modes are mined from the corpus, which are used as sentence pattern nodes; through co-occurrence relationship and semantic similarity, the sentence pattern nodes and the related word nodes are connected, which are used for subsequent example generation and natural interaction.

[0040] 3) knowledge layer construction; Based on text extraction and manual sorting, the place names, personal names, festivals, customs, songs and other related to Yao culture are constructed as cultural knowledge nodes; the cultural knowledge nodes and the corresponding word, sentence are connected to form a knowledge graph reflecting "vocabulary-usage-cultural background".

[0041] 4) dynamic updating; According to the new collected corpus and the model retrieval result, the high-confidence new word, new usage and new cultural fact are automatically identified, and the related nodes and edges are added or updated; the low-frequency or corrected by the user as the wrong node is de-weighted or merged.

[0042] In summary, unlike the traditional "static glossary + fixed database structure" Yao language dictionary system, the dictionary and knowledge graph of the present application are closely coupled with the semantic space, and are dynamic structures that can evolve continuously with corpus and models, providing a rich and expandable knowledge base for subsequent intelligent retrieval and generation.

[0043] S360, a semantic retrieval module based on a Yao-Han bilingual semantic retrieval interaction model, based on a knowledge graph structure that evolves continuously with corpus, performs semantic retrieval on the input of the user to obtain a semantic retrieval matching result; Specifically, the input of the user is dialect-adaptive multi-modal encoded to obtain a unified semantic vector; the knowledge graph structure that evolves continuously with corpus is input to the semantic retrieval module of the Yao-Han bilingual semantic retrieval interaction model; the unified semantic vector is used as a query vector, and semantic retrieval is performed in combination with the knowledge graph structure that evolves continuously with corpus to construct a retrieval knowledge set; retrieval enhancement generation is performed on the retrieval knowledge set to obtain a semantic retrieval matching result.

[0044] In this embodiment, for semantic retrieval and generation interaction, it includes: 1) multi-modal encoding of user input; The user can initiate a query through Chinese text, Yao language text, Yao language voice, or image, etc.; the system inputs the input into the dialect-adaptive multi-modal encoding module to obtain a unified semantic vector.

[0045] 2) retrieval of related knowledge; In the established vector index and the established dictionary / knowledge graph, the user vector is used as a query vector to retrieve a number of most similar entries, example sentences, and knowledge nodes; a "retrieval knowledge set" highly related to the current input is formed.

[0046] 3) retrieval enhancement generation; The semantic vector of the user input and the retrieved number of knowledge vectors are jointly input as conditions to feed a lightweight Yao-Han bilingual generation model or a dialogue model; the generation model generates answer content under the above conditions, including Chinese explanation of a certain Yao language entry and multiple example sentences; multiple Yao language dialect expressions of a certain segment of Chinese; background explanation and usage prompts based on cultural knowledge nodes; at the same time, the output is accompanied by the identification of the corresponding entry, example sentence, or knowledge node to form a traceable answer explanation chain.

[0047] In summary, through the retrieval enhancement generation interaction structure, the present application not only utilizes vector retrieval to ensure the accuracy of semantic matching, but also utilizes the generation model to improve the naturalness and flexibility of the answer, solving the limitation of the traditional dictionary system that can only return a single entry and cannot generate answers according to the context.

[0048] S370, a Yao-Han bilingual interactive module based on the Yao-Han bilingual semantic retrieval interaction model, interacts with the user according to the semantic retrieval matching result, and realizes self-interaction of Yao-Han bilingual.

[0049] In the embodiment, the self-evolution mechanism for user feedback and active learning includes: 1) feedback collection; In the interactive interface, a feedback entry is provided for each answer, and the user can perform "confirmation", "correction" or "supplementation" on the word meaning, translation, pronunciation, example sentence, etc. The user can also upload new Yao language voice, text or picture examples, and attach simple Chinese explanations.

[0050] 2) feedback conversion and sample generation; The system regards the user-confirmed result as a high-confidence sample, and regards the corrected and supplemented content as weakly-labeled training data. These data are distributed to different sub-modules according to the type: dialect recognition / coding samples for fine-tuning the dialect expert selector and the encoder; parallel corpus for fine-tuning the Yao-Han semantic alignment; new nodes and relationships for updating the dictionary / knowledge graph.

[0051] 3) incremental training and updating; The system performs small-step incremental training on the light modules (such as the adaptation layer, expert weight, projection layer, etc.) of the related models according to the set period or trigger condition; under the premise of maintaining the stability of the overall model, the system gradually adapts to new dialects, new vocabularies and new expressions.

[0052] In summary, through the user feedback and active learning mechanism, the Yao language corpus and the interactive system constructed by the present application can continuously improve itself in real use, significantly reduce the dependence on expert centralized labeling and regular large-scale retraining, and improve the maintainability and practical application value of the system in long-term operation.

[0053] Therefore, the embodiment of the present application first collects and cleanses multi-modal data such as Yao language related texts, voices, images, etc., and labels the dialect category, basic transcription and metadata. Then, the Yao language texts, voices and images of different dialects are vectorized and encoded in a unified semantic space, and the self-adaptive modeling of dialect differences is realized through the dialect expert sub-network or the adaptation layer. The Yao language representation and the Chinese representation are aligned to the common semantic space, and a high-dimensional vector index is constructed to realize semantic-level bilingual retrieval. On the basis of the unified semantic space, Yao language entries, sentence patterns and cultural knowledge nodes are automatically extracted and updated to form a dictionary / knowledge graph structure that evolves with the corpus. Further, multi-modal queries from users are received, relevant knowledge is obtained based on vector retrieval, and interactive content such as translation, explanation and example sentences is given through the retrieval-enhanced generation model. Finally, the confirmation, correction and supplement information of the user on the output result of the system is collected, which is converted into weakly-labeled samples to drive the incremental optimization of the encoding layer, the semantic alignment layer and the knowledge graph layer.

[0054] More specifically, in combination with specific embodiments, first, 1000 Yao language text and audio samples (for Mian Yao dialect) are collected from Yao community; further, a word segmentation algorithm similar to Gaussian decomposition is used for text preprocessing to obtain a standardized corpus set; and based on the BERT model, features are extracted, the input is the corpus set, and the output is an index database; further, the corpus index is fused to construct a Yao-Chinese dictionary to support basic search; and a speech recognition module is integrated to realize audio query. The BERT model parameters include embedding dimension 768, and layer number 12.

[0055] Compared with traditional manual construction, the corpus processing time of the embodiment is shortened by 50%, and the accuracy is improved by 20% (compared by F1 score index, traditional method F1=0.75, and the embodiment F1=0.90).

[0056] Embodiment 2: A Yao language corpus construction and digital dictionary system based on artificial intelligence (enhanced version) The implementation method comprises the following steps: (1) Corpus collection: 2000 samples are collected, including Bunu dialect and image data; (2) Corpus processing: 1. Use the Transformer model for semantic annotation, input is extended corpus, and output is a multi-modal feature set; (3) Dictionary construction: add knowledge graph and associate cultural elements; (4) Intelligent interaction: integrate image recognition to realize picture query Yao language interpretation.

[0057] The Transformer model parameters include 12 attention heads and 6 layers.

[0058] Comparison data: The response time of the reconstructed dictionary query is shortened by 40%, and the cultural association accuracy is improved by 25% (compared by recall rate index, traditional method recall rate=0.60, and the method recall rate=0.85) In summary, the embodiment of the application realizes automatic corpus construction through AI technology, reduces manual intervention, and improves efficiency. In addition, the system supports multi-modal interaction, improves query accuracy, and enhances cultural heritage function through knowledge graph, solving the problems of poor adaptability and staticity of traditional systems. Specific beneficial effects include: wider corpus coverage, real-time dictionary update, more interactive user experience, which helps to protect and promote the Yao language.

[0059] Reference Figure 2 A Yao language corpus construction and interaction system based on artificial intelligence, comprising: The first module 201 is configured to construct a multi-modal Yao language data set and perform data preprocessing to obtain a preprocessed multi-modal Yao language data set. The second module 202 is configured to introduce a dual-encoder structure and a bilingual semantic alignment module to construct a Yao-Chinese bilingual semantic retrieval interaction model. The third module 203 is configured to perform semantic similarity retrieval and interaction on the preprocessed multi-modal Yao language data set based on the Yao-Chinese bilingual semantic retrieval interaction model to realize Yao-Chinese bilingual autonomous interaction.

[0060] The content in the method embodiments is applicable to the system embodiments, the system embodiments specifically realize the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0061] The above is a specific description of the preferred embodiments of the application, but the application is not limited to the embodiments described, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A method for constructing and interacting with a Yao language corpus based on artificial intelligence, characterized in that, Includes the following steps: A multimodal Yao language dataset was constructed and preprocessed to obtain the preprocessed multimodal Yao language dataset. A dual encoder structure and a bilingual semantic alignment module are introduced to construct a Yao-Chinese bilingual semantic retrieval interaction model; Based on the Yao-Chinese bilingual semantic retrieval and interaction model, semantic similarity retrieval and interaction are performed on the preprocessed multimodal Yao language dataset to achieve autonomous Yao-Chinese bilingual interaction.

2. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 1, characterized in that, The step of constructing a multimodal Yao language dataset and performing data preprocessing to obtain a preprocessed multimodal Yao language dataset specifically includes: Collect Yao language text data, Yao language audio recordings, and Yao language-related image data to construct a multimodal Yao language dataset; The multimodal Yao language dataset is subjected to text cleaning, speech denoising and unified conversion processing, and image cropping and standardization processing in sequence to obtain the pre-processed multimodal Yao language dataset. The preprocessed multimodal Yao language dataset was subjected to language category labeling, dialect labeling, and Chinese explanation annotation to obtain the preprocessed multimodal Yao language dataset.

3. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 2, characterized in that, The Yao-Chinese bilingual semantic retrieval and interaction model specifically includes a dual encoder structure, a bilingual semantic alignment module, a vector indexing module, a knowledge graph construction module, a semantic retrieval module, and a Yao-Chinese bilingual interaction module, wherein: The dual encoder structure includes a Yao language encoder and a Chinese language encoder. The Yao language encoder includes a text encoding layer, a dialect selection and combination layer, a speech encoding layer, and an image encoding layer. The knowledge graph construction module includes a vocabulary layer, a sentence structure layer, a knowledge layer, and a dynamic update layer.

4. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 3, characterized in that, The step of performing semantic similarity retrieval and interaction on the preprocessed multimodal Yao language dataset based on the Yao-Chinese bilingual semantic retrieval and interaction model to achieve autonomous Yao-Chinese bilingual interaction specifically includes: The preprocessed multimodal Yao language dataset is input into the Yao-Chinese bilingual semantic retrieval interaction model; Based on the dual encoder structure of the Yao-Chinese bilingual semantic retrieval interaction model, the preprocessed multimodal Yao language dataset is semantically encoded to obtain Yao language encoding vectors and Chinese language encoding vectors. The bilingual semantic alignment module based on the Yao-Chinese bilingual semantic retrieval interaction model performs multimodal semantic alignment processing on the Yao language encoding vector and the Chinese language encoding vector to obtain the aligned semantic term vector. The vector index module based on the Yao-Han bilingual semantic retrieval interaction model constructs a vector index through an approximate nearest neighbor retrieval structure, performs similarity retrieval on the aligned semantic term vectors, and obtains semantic retrieval matching results. The knowledge graph construction module based on the Yao-Han bilingual semantic retrieval interaction model constructs a dynamic dictionary and knowledge graph based on the semantic retrieval matching results, resulting in a knowledge graph structure that continuously evolves with the corpus. The semantic retrieval module based on the Yao-Han bilingual semantic retrieval interaction model performs semantic retrieval on user input based on a knowledge graph structure that continuously evolves with the corpus, and obtains semantic retrieval matching results. The Yao-Chinese bilingual interaction module, based on the Yao-Chinese bilingual semantic retrieval and interaction model, interacts with users according to the semantic retrieval matching results, realizing autonomous Yao-Chinese bilingual interaction.

5. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 4, characterized in that, The dual encoder structure based on the Yao-Chinese bilingual semantic retrieval interaction model performs semantic encoding on the preprocessed multimodal Yao language dataset to obtain Yao language encoding vectors and Chinese language encoding vectors. This step specifically includes: The preprocessed multimodal Yao language dataset is input into the dual encoder structure of the Yao-Chinese bilingual semantic retrieval interaction model; A Yao language encoder based on a dual-encoder structure performs dialect-adaptive multimodal coding on the preprocessed multimodal Yao language dataset to obtain Yao language coding vectors. A Chinese encoder based on a dual-encoder structure encodes the Chinese interpretation annotations in the preprocessed multimodal Yao language dataset to obtain Chinese encoding vectors.

6. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 5, characterized in that, The Yao language encoder based on a dual-encoder structure performs dialect-adaptive multimodal coding on the preprocessed multimodal Yao language dataset to obtain the Yao language coding vector. The specific steps include: The preprocessed multimodal Yao language dataset is input into a Yao language encoder with a dual encoder structure; Based on the Yao language encoder, the text encoding layer performs text encoding processing on the preprocessed multimodal Yao language dataset to obtain the Yao language text encoding structure. Based on the dialect selection and combination layer of the Yao language encoder, dialect labels are automatically identified in the Yao language text encoding structure to obtain dialect adaptive text vectors. Based on the speech coding layer of the Yao language encoder, speech segments are encoded on the preprocessed multimodal Yao language dataset to obtain Yao language speech vectors; Based on the image coding layer of the Yao language encoder, image visual features are extracted from the preprocessed multimodal Yao language dataset to obtain Yao language image vectors; The dialect-adaptive text vector, Yao language speech vector, and Yao language image vector are mapped and aligned to obtain the Yao language encoding vector.

7. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 6, characterized in that, The knowledge graph construction module based on the Yao-Han bilingual semantic retrieval interaction model includes the following steps: constructing a dynamic dictionary and knowledge graph based on the semantic retrieval matching results to obtain a knowledge graph structure that continuously evolves with the corpus. Input the semantic retrieval matching results into the knowledge graph construction module of the Yao-Han bilingual semantic retrieval interaction model; Based on the lexical layer of the knowledge graph construction module, the semantic retrieval matching results are segmented and statistically analyzed to obtain basic semantic vector frequency information. Based on the knowledge graph construction module, the sentence structure layer connects sentence structure nodes with related word nodes according to the frequency information of basic semantic vectors to obtain semantic sentence structure nodes. Based on the knowledge layer and dynamic update layer of the knowledge graph construction module, semantic sentence nodes and cultural knowledge nodes are connected and dynamically updated to obtain a knowledge graph structure that continuously evolves with the corpus.

8. The method for constructing and interacting with a Yao language corpus based on artificial intelligence according to claim 7, characterized in that, The semantic retrieval module based on the Yao-Han bilingual semantic retrieval interaction model, which performs semantic retrieval on user input based on a continuously evolving knowledge graph structure, and obtains semantic retrieval matching results, specifically includes the following steps: Dialect-adaptive multimodal encoding is performed on user input to obtain a unified semantic vector; The knowledge graph structure that evolves continuously with the corpus is input into the semantic retrieval module of the Yao-Han bilingual semantic retrieval interaction model; Using a unified semantic vector as the query vector, and combining it with a knowledge graph structure that continuously evolves with the corpus, semantic retrieval is performed to construct a retrieval knowledge set. The retrieval knowledge set is enhanced to generate semantic retrieval matching results.

9. A Yao language corpus construction and interaction system based on artificial intelligence, characterized in that, Includes the following modules: The first module is used to construct a multimodal Yao language dataset and perform data preprocessing to obtain a preprocessed multimodal Yao language dataset. The second module is used to introduce a dual encoder structure and a bilingual semantic alignment module to build a Yao-Chinese bilingual semantic retrieval interaction model. The third module is used to perform semantic similarity retrieval and interaction on the preprocessed multimodal Yao language dataset based on the Yao-Han bilingual semantic retrieval and interaction model, so as to realize autonomous Yao-Han bilingual interaction.

Images