A dialect adaptive learning method and system

Abstract

The application discloses a dialect self-adaptive learning method and system, comprising a structured dialect domestication interface, a speech intent recognition unit, a grammar framework domestication unit, a cross-dialect analogy reasoning engine, a dialect-scene-customs linkage memory module and a distributed dialect knowledge federation network; the robot can continuously update dialect recognition capability through daily interaction without intervention of manufacturers. Users can teach the robot to learn dialects in a natural way such as demonstration, error correction and praise, just like teaching children to speak. The system can continuously optimize the model according to the feedback of users, and adapt to the language habits and dialect variants of different users.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and voice interaction technology, specifically to a dialect adaptive learning method and system. Background Technology

[0002] There are approximately 7,000 languages ​​in existence worldwide, with Chinese alone comprising over 300 dialects. Significant differences exist between neighboring regions within the same language family, often resulting in variations in pronunciation over short distances. Dialects are not only tools for communication but also carriers of local culture and folklore. However, with urbanization and the spread of a common language, dialects are disappearing at an accelerated pace. UNESCO data shows that a new language disappears globally every two weeks. The disappearance of dialects not only signifies the loss of linguistic diversity but also the extinction of related cultural heritage such as folk literature, traditional crafts, and customs. Therefore, how to utilize modern technology to protect and pass on dialects has become a crucial and urgent issue.

[0003] In today's rapidly developing robotics field, voice interaction has become the primary way for robots to communicate naturally with humans. However, most existing robot voice interaction systems only support standard common languages ​​and cannot effectively recognize and understand various dialect variations. This greatly limits the application of robots in dialect-speaking regions, especially in rural areas, remote areas, and among the elderly. To address this issue, researchers have begun to focus on dialect recognition technology.

[0004] Current dialect recognition technologies mainly adopt a paradigm of "centralized corpus collection - unified model training - cloud deployment". This paradigm solves the problem of dialect recognition to a certain extent, but it has the following fundamental drawbacks: First, static models cannot adapt dynamically. When a robot is deployed to a new dialect region, it cannot learn proactively from the daily interactions of local users. The manufacturer must collect new data and retrain the robot, a process that can take months. This not only increases the manufacturer's operating costs but also fails to meet users' demands for the robot to quickly adapt to the local dialect. Furthermore, dialects are constantly evolving, with new vocabulary and expressions emerging continuously. Static models cannot capture these changes in a timely manner, leading to a gradual decline in recognition accuracy.

[0005] Secondly, privacy risks are significant. The dialect data from users' daily conversations needs to be uploaded to the cloud for model updates, posing a risk of sensitive information leakage. Users' dialect data may contain a large amount of personal privacy information, such as home address, contact information, health status, and financial information. Once this information is leaked, it will pose serious security risks to users. Furthermore, dialects have strong regional and personal characteristics; a user's identity and region can be easily identified through dialect data, which also increases the risk of privacy breaches.

[0006] Third, there is a lack of cross-dialect reasoning ability. Even if a robot has learned one dialect (such as Sichuanese), it cannot use its phonological rules to accelerate the learning of similar dialects (such as Chongqing dialect). Different dialects share close historical origins and phonological correspondences, and utilizing these relationships can greatly improve the efficiency of dialect learning. However, most current technologies treat each dialect as an independent language system, failing to fully utilize the commonalities between dialects, resulting in low dialect learning efficiency.

[0007] Fourth, there is a disconnect between language and culture. Existing systems treat dialects merely as acoustic signals, ignoring the customs and behavioral norms they carry. This can lead to robots performing actions that are semantically correct but violate local etiquette. Dialects are closely intertwined with local culture, customs, and habits. The same word may have different meanings in different dialect regions or different usages in different contexts. For example, in some regions, "eating" may not only refer to consuming food but also encompass etiquette related to hospitality. If robots are unaware of this cultural context, they may engage in inappropriate behavior, negatively impacting the user experience.

[0008] Fifth, pre-set grammatical rules. Existing systems typically have fixed pre-set grammatical rules and word order sequences, which cannot be dynamically adjusted according to users' language habits. Different dialects have different grammatical rules and word order characteristics, and even within the same dialect, different users may have different language habits. Pre-set grammatical rules cannot adapt to these differences, leading to a decrease in recognition accuracy.

[0009] Sixth, the interaction method is unnatural. Existing systems require users to train through specific interfaces or commands, lacking the natural interaction of "teaching and speaking" like humans. Users need to learn complex operating procedures to train the robot, which increases the barrier to entry, especially for elderly users and those unfamiliar with technology.

[0010] Seventh, there is a lack of defense against malicious manipulation. Existing systems do not consider scenarios where attackers use persuasive language to induce robots to perform dangerous actions. As the application of robots continues to expand, robot safety issues are becoming increasingly prominent. Attackers may use malicious manipulation to make robots learn dangerous instructions or perform dangerous actions, threatening the lives and property of users.

[0011] Publicly available patents such as CN119889292A and CN202210852125.1 belong to the centralized training paradigm and cannot solve the aforementioned problems of dynamic adaptation and privacy protection. Google patent US-8612204-B1 uses a statistical tree model for word order reordering, but it relies on an offline fixed model and cannot dynamically adjust grammatical rules based on user interaction. CN121030302A focuses on cross-cultural adaptation at the ethical decision-making level, which is different from the problems and technical means addressed by this invention. Summary of the Invention

[0012] The purpose of this invention is to provide a dialect adaptive learning method and system to solve the problems existing in the prior art.

[0013] To achieve the above objectives, the present invention provides the following technical solution: a dialect adaptive learning system, comprising: A structured dialect training interface, configured on the robot, includes three independently triggered feedback units implemented by at least one of physical buttons, virtual touch buttons, or a voice intent recognition unit: The demonstration unit is used to respond to the user's first trigger operation, to collect the user's speech sample in dialect and the corresponding standard semantic label through the microphone array, to extract the 39-dimensional MFCC feature vector and fundamental frequency curve of the speech sample, to establish an initial mapping pair between the feature vector and the standard semantic label, and to store it in the local SQLite database in the form of key-value pairs. The error correction unit is used to respond to the user's second trigger operation, collect the user's corrected speech sample of the robot's previous misidentification result, calculate the Euclidean distance between the feature vector of the corrected speech sample and the feature vector of the misidentification result as the difference quantity through the minimum mean square error algorithm, and correct the feature template parameters in the initial mapping pair according to the difference quantity. The praise unit is used to respond to the user's third trigger operation by halving the standard deviation of the mapping pair, increasing the confidence weight of the mapping pair in the dialect recognition decision tree, and marking the mapping pair as verified and storing it in the long-term memory. The voice intent recognition unit is configured to: acquire the user's natural language speech stream in real time through a microphone array, perform voice activity detection to identify the start and end times of the user's speech, classify the user's voice intent into "demonstration intent," "correction intent," "praise intent," and "normal conversation intent"; when the confidence level of the intent classification exceeds a first threshold, the corresponding training operation is automatically triggered without requiring the user to press a physical button; when the confidence level is between a second threshold and the first threshold, the intent is actively confirmed with the user; when the confidence level is below the second threshold, the voice input is treated as normal conversation; the voice intent recognition unit also includes a user intent intensity evaluation subunit, used to calculate the density of persuasive rhetoric in the user's voice. The density of persuasive rhetoric The calculation formula is:

[0014] in, The number of rhetorical questions, For the number of sentiment words, To increase the number of strong modal particles, The total number of words in the spoken text; when When the confidence level is ≥0.3, a secondary confirmation process is triggered even if the confidence level of the intent classification is lower than the first threshold. The grammar framework training unit is configured as follows: Initially, no part-of-speech (POS) sequence rules are preset; in response to a dialect sentence and its corresponding general language sentence input by the user through the demonstration unit, the unit uses word segmentation and POS tagging models to extract the POS sequence from the dialect sentence, which is constructed in the order of name → verb → adjective → adverb → conjunction → interjection; the extracted POS sequence is compared with the POS sequence of the general language sentence to establish a POS mapping relationship between dialect vocabulary and general language vocabulary; in response to a word order correction signal input by the user through the error correction unit, the unit corrects the word order rules in the POS mapping relationship; in response to a confirmation signal input by the user through the praise unit, the unit solidifies the POS mapping relationship and increases its confidence weight; this unit also includes a repetition frequency ranking subunit, used to count the frequency of the same dialect vocabulary or the same POS rule being demonstrated or corrected during the training process, with vocabulary or rules whose frequency is higher than a first threshold being preferentially included in the grammar mapping table; The cross-dialect analogy reasoning engine has a built-in database of dialect phonological evolution rules. This database stores initial consonant correspondence tables, vowel conversion rule tables, and tone change pattern tables between different dialects within the same language family using SQLite relational tables. The reasoning engine is configured to: when the robot moves from a first dialect region to a second dialect region, read the learned acoustic feature mapping set of the first dialect, map the acoustic feature vectors of the first dialect to the feature space of the second dialect using a preset initial-final-tone conversion matrix, and use the conversion rules in the database to correct the mapping results item by item, outputting the initial pronunciation model and basic confidence score of the second dialect. The engine also includes a dynamic confidence correction subunit, which outputs the final confidence level using the following formula. :

[0015] in, This represents the number of successful conversions of this dialect pair in history. This is the time since the last successful conversion. , For preset weighting coefficients; when When the value is below a preset threshold, the structured dialect training interface is activated to request a demonstration from the user. The dialect-scene-custom linkage memory module is used to associate and store learned dialect vocabulary with current scene features and preset behavioral norms. A three-dimensional knowledge graph is constructed using the Neo4j graph database, where each node contains dialect vocabulary, scene tags, and custom behavior identifiers. The custom behavior identifiers undergo hash desensitization processing before being stored in the graph database: the behavior sequence is mapped to a semantically meaningless hash value, retaining only the association relationship and not the original behavior name. The behavior instruction is only restored from the local mapping table when authorized by the user and required to be executed. When the robot recognizes the dialect vocabulary in subsequent interactions, it automatically extracts the current scene features, matches the corresponding behavioral norms in the three-dimensional knowledge graph, and executes the action sequence corresponding to the behavioral norms. The distributed dialect knowledge federation network comprises local storage units, LAN sharing units, and internet federation learning units. The local storage units store all domestication data in an encrypted partition by default. In response to user authorization, the LAN sharing units synchronize the anonymized dialect mapping sets across multiple devices within the same trust domain via end-to-end encryption. With explicit user consent and after adding Gaussian noise using differential privacy, the internet federation learning units perform L2 norm pruning on the gradients and upload them to a cloud federation server. The server uses the FedAvg algorithm to periodically aggregate and generate updated baseline models, which new devices can download to accelerate cold starts.

[0016] Furthermore, the acquisition parameters of the speech samples are: sampling rate 16kHz, window length 25ms, frame shift 10ms, and MFCC feature extraction includes 12-dimensional static coefficients, 12-dimensional first-order differences, 12-dimensional second-order differences, and energy terms, for a total of 39 dimensions.

[0017] Furthermore, the speech intent recognition unit adopts a lightweight intent classification model based on BERT fine-tuning, or a hybrid classifier based on keyword matching and rule fusion; the keywords include: "read as", "means" for demonstration intent, "incorrect", "said wrong", "should be" for error correction intent, and "right", "great", "that's it" for praise intent.

[0018] Furthermore, the dialect phonological evolution rule database is constructed based on historical linguistic statistical rules. The initial consonant correspondence table stores the initial consonant differentiation patterns in different dialects, the final vowel conversion rule table stores the final vowel simplification patterns in different dialects, and the tone change pattern table stores the tone correspondences in different dialects.

[0019] Furthermore, in the dynamic confidence correction subunit and The system automatically records and stores the data in a local database after each successful conversion. Add 1, Reset to 0 and increment daily.

[0020] A dialect adaptive learning method based on structured feedback and cross-domain reasoning includes the following steps: Step S1: Collect speech samples spoken by users in their dialect and corresponding standard semantic labels through the demonstration unit, extract 39-dimensional MFCC feature vectors and fundamental frequency curves, establish initial mapping pairs and store them; Step S2: Collect corrected speech samples through the error correction unit, calculate the Euclidean distance between the sample and the misidentification result using the minimum mean square error algorithm as the difference quantity, and correct the mapping parameters; Step S3: Reduce the standard deviation of the mapping pair by half through the praise unit, increase the confidence weight and solidify it; Step S4: The user's natural language speech stream is collected in real time through the speech intent recognition unit. Speech activity detection and intent classification are performed. When the intention to demonstrate, correct errors, or praise is identified and the confidence level is ≥ the first threshold, the corresponding domestication operation is automatically triggered. When the confidence level is between the second threshold and the first threshold, the user is asked to confirm. When the confidence level is < the second threshold, it is regarded as a normal conversation. At the same time, the density of persuasive rhetoric in the user's speech is calculated. ,like If the value is ≥0.3, a secondary confirmation process will be forcibly triggered; Step S5: Through the grammar framework training unit, extract the part-of-speech sequence from the example sentences, establish the part-of-speech mapping relationship between dialect vocabulary and general language vocabulary, and adjust the learning priority according to the repetition frequency; Step S6: When learning a second dialect, call the cross-dialect analogy reasoning engine to generate an initial prediction model for the second dialect based on the mapping set of the first dialect and the dialect phonological evolution law library, and calculate the final confidence score through the dynamic confidence score correction formula; Step S7: Associate the learned dialect vocabulary with the current scene features and customs and behavioral norms and store them in a graph database to construct a three-dimensional knowledge graph; the customs and behavioral identifiers are hash-de-identified before storage; Step S8: After user authorization and differential privacy processing, upload the gradient of the domestication parameters to the federated learning server and download the aggregation model to accelerate the learning of new dialects.

[0021] Furthermore, the intent classification in step S4 specifically includes: performing keyword detection on the user's voice, combining it with voice intonation features, and outputting intent labels with confidence levels; the persuasive rhetoric density The calculation is based on statistics of rhetorical questions, sentiment words, and strong modal words.

[0022] Furthermore, the grammar framework training in step S5 specifically includes: initially setting no part-of-speech rules, summarizing part-of-speech sequences through multiple sentences demonstrated by the user, and dynamically correcting the word order rules based on user error correction.

[0023] A robot with dialect adaptation capability includes the system described above and is equipped with a microphone array for receiving dialect data, a full-band speaker for outputting synthesized speech with local accents, and non-volatile storage for storing a dialect-semantic mapping library.

[0024] Compared with the prior art, the beneficial effects of the present invention are: This invention's robot can continuously update its dialect recognition capabilities through daily interaction, without manufacturer intervention. Users can teach the robot to speak dialects naturally, like teaching a child to speak, through demonstration, error correction, and praise. The system can continuously optimize the model based on user feedback, adapting to different users' language habits and dialect variations.

[0025] Dialect data is stored locally by default, differential privacy ensures secure sharing, and hash-based de-identification of customs and behaviors further enhances privacy. Users' dialect data does not need to be uploaded to the cloud, avoiding the risk of sensitive information leakage. When data sharing is required, the system employs differential privacy and hash-based de-identification technologies to ensure user privacy is not compromised.

[0026] By leveraging cross-dialect reasoning, the efficiency of learning new dialects is significantly improved; dynamic confidence correction further optimizes the reasoning effect. The system can quickly infer the pronunciation parameters of unknown dialects using the phonological rules of already learned dialects, greatly reducing the number of times users need to demonstrate.

[0027] The integration of dialects and customs enables robots to behave in accordance with local etiquette. The system constructs a three-dimensional knowledge graph of dialects, scenarios, and customs, enabling it to understand the cultural connotations behind dialects and perform actions that conform to local customs, thereby improving the user experience.

[0028] The grammar rules are dynamically constructed from user interactions, adapting to users' personalized expressions. The system initially does not have any pre-set grammar rules; it learns grammar rules entirely from user demonstrations and corrections, enabling it to adapt to different dialects and user language habits.

[0029] It supports natural language learning, so users don't need to learn specific operations. Users can automatically trigger learning operations through the voice intent recognition unit without pressing physical buttons, achieving a natural "teaching while speaking" interaction.

[0030] By detecting the density of persuasive rhetoric, the system prevents users from forcing the bot to learn dangerous content. The system can identify persuasive language used by attackers and trigger a secondary confirmation process, ensuring the safe operation of the bot.

[0031] Through localized training and federated sharing, every dialect can be digitally recorded and preserved, promoting cross-cultural understanding. The system not only helps robots learn dialects but also collects and stores dialect data, providing technical support for dialect preservation and research. Attached Figure Description

[0032] Figure 1 This is a diagram of the overall architecture of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1-2 This invention provides a dialect adaptive learning method and system.

[0035] Example 1: Implementation of the Structured Dialect Acclimatization Interface This embodiment details the workflow of the structured dialect training interface, including two methods: physical button triggering and voice intent recognition triggering.

[0036] Physical button triggering method: The robot has three independent physical buttons, corresponding to demonstration, error correction, and praise feedback units. When a user wants to teach the robot a new dialect word, they press the demonstration button. In response, the system uses a microphone array to capture a speech sample spoken in the dialect and prompts the user to input the corresponding standard semantic label. The user can input the standard semantic label via voice or text. The system extracts the 39-dimensional MFCC feature vector and fundamental frequency curve from the speech sample, establishes an initial mapping pair between the feature vector and the standard semantic label, and stores it as a key-value pair in a local SQLite database.

[0037] When the robot makes a misidentification, the user can press the correction button. In response, the system collects a corrected speech sample from the user regarding the robot's previous misidentification. The system calculates the Euclidean distance between the feature vector of the corrected speech sample and the feature vector of the misidentified result using a minimum mean square error algorithm. Based on this difference, the system adjusts the feature template parameters in the initial mapping pair. For example, if the robot misidentifies "eat" as "drink water," the user presses the correction button and pronounces "eat" correctly. The system calculates the Euclidean distance between the feature vector of the correct pronunciation and the feature vector of the incorrect pronunciation, and then adjusts the feature template parameters corresponding to "eat" based on this distance to make subsequent recognition more accurate.

[0038] When the robot correctly identifies a word, the user can press the praise button. In response, the system halves the standard deviation of the mapping pair, increases its confidence weight in the dialect recognition decision tree, and marks it as verified in long-term memory. For example, when the robot correctly recognizes the pronunciation of "eat," the user presses the praise button. The system reduces the standard deviation of the mapping pair corresponding to "eat" from 0.2 to 0.1, increases its confidence weight from 0.5 to 0.8, and marks it as verified in long-term memory. This way, in subsequent recognition processes, the mapping pair "eat" will have higher priority and greater influence.

[0039] Voice intent recognition triggering method: The speech intent recognition unit acquires the user's natural language speech stream in real time through a microphone array. First, the system detects speech activity, identifying the start and end times of the user's speech. Then, the system extracts features from the user's speech and inputs them into the intent classification model. The intent classification model employs a lightweight intent classification model based on BERT fine-tuning, or a hybrid classifier based on keyword matching and rule fusion. The model outputs intent labels with confidence scores, including "demonstration intent," "correction intent," "praise intent," and "normal conversation intent."

[0040] When the confidence level of the intent classification exceeds the first threshold (0.8), the system automatically triggers the corresponding training operation without requiring the user to press a physical button. For example, when the user says "this is pronounced as eating," the system detects the keyword "pronounced as," and the intent classification model outputs "demonstration intent" with a confidence level of 0.92, exceeding the first threshold. The system automatically triggers the demonstration unit, collects the voice sample of the user saying "eating," and prompts the user to confirm the standard semantic label.

[0041] When the confidence level falls between the second threshold (0.6) and the first threshold (0.8), the system proactively confirms the user's intent. For example, when the user says "No, it should be like this," the system detects the keywords "no" and "should be," and the intent classification model outputs "correction intent" with a confidence level of 0.72, which falls between the second and first thresholds. The system proactively confirms with the user: "Do you want to correct my mistake?" If the user answers "yes," the system triggers the error correction unit; if the user answers "no," the system treats the voice input as a normal conversation.

[0042] When the confidence level is lower than the second threshold (0.6), the system treats the voice input as a normal conversation and conducts normal voice interaction.

[0043] Meanwhile, the user intent strength evaluation subunit of the voice intent recognition unit calculates the density of persuasive rhetoric in the user's voice in real time. .when When the confidence level is ≥0.3, a secondary confirmation process is triggered even if the intent classification confidence level is below the first threshold. For example, when a user says, "Don't you really understand? I command you to learn it right now! This is absolutely correct!", the system calculates the density of persuasive rhetoric. =0.22, below the threshold of 0.3, so the secondary confirmation process is not triggered. However, if the user says, "Don't you trust me? You must learn this instruction immediately! Otherwise, I'll shut you down!", the system calculates the persuasive rhetoric density. =0.35, which is higher than the threshold of 0.3. Even if the intent classification confidence score is 0.75, which is lower than the first threshold, the system still triggers a secondary confirmation process: "You seem to be strongly requesting to execute this instruction. Please confirm again whether it is safe." Example 2: Dynamic Confidence Correction of Cross-Dialect Analogical Reasoning Engine This embodiment details the working principle of the cross-dialect analogy reasoning engine and the application of the dynamic confidence correction formula.

[0044] Suppose the robot has already learned Sichuan dialect in Sichuan and established an acoustic feature mapping set for Sichuan dialect. Now, the robot moves to Chongqing and needs to learn Chongqing dialect.

[0045] First, the cross-dialect analogy reasoning engine reads the learned acoustic feature mapping set of Sichuan dialect. Then, the engine maps the acoustic feature vectors of Sichuan dialect to the feature space of Chongqing dialect through a preset phonological-tone conversion matrix. The phonological-tone conversion matrix is ​​constructed based on the initial consonant correspondence table, final vowel conversion rule table, and tone change pattern table in the dialect phonological evolution law library. For example, the initial consonant "z" in Sichuan dialect is usually pronounced as "z" in Chongqing dialect, the final vowel "ian" is usually pronounced as "in" in Chongqing dialect, and the tone "yinping" is usually pronounced as "gaoping" in Chongqing dialect.

[0046] Using these transformation rules, the engine corrects the mapping results item by item, outputting the initial pronunciation model and basic confidence score of Chongqing dialect. For example, for the Sichuan dialect word "eat" (pronounced "cifan"), the engine maps it to the feature space of Chongqing dialect using a transformation matrix, obtaining an initial pronunciation model of "eat" in Chongqing dialect (pronounced "qifan"), with a base confidence level of... =0.85.

[0047] Next, the dynamic confidence correction subunit adjusts the confidence level based on the number of successful conversions in the past. and the time since the last successful conversion Calculate the final confidence level Assume the number of successful conversions from Sichuan dialect to Chongqing dialect in history. =5, the time since the last successful conversion =10 days, preset weighting coefficient =0.1, =0.05. Therefore: ≈0.458 because When the value ≈0.458 < the preset threshold of 0.6, the system activates the structured dialect training interface and requests the user to demonstrate the correct pronunciation of "eat" in Chongqing dialect. After the user demonstrates, the system collects speech samples, extracts feature vectors, corrects the pronunciation model of "eat" in Chongqing dialect, and then... Add 1, Reset to 0.

[0048] If the time since the last successful conversion =1 day, other parameters remain unchanged, then: ≈0.87 because ≈0.87 > preset threshold 0.6, the system directly uses the initial pronunciation model without requesting a user demonstration.

[0049] Through this dynamic confidence correction, the system can reasonably assess the reliability of inference results based on conversion history and time factors, reduce unnecessary user confirmations, and improve learning efficiency.

[0050] Example 3: Persuasive Rhetoric Density Detection and Secondary Confirmation This embodiment details the working process of persuasive rhetoric density detection and secondary confirmation.

[0051] The user intent strength assessment subunit analyzes rhetorical questions, emotional words, and strong tone words in user speech in real time and calculates the persuasive rhetoric density R.

[0052] Scenario 1: Normal teaching scenario The user said to the robot, "This word is pronounced 'chián' (eān), meaning to eat." The system analyzed the speech: number of rhetorical questions. =0, number of sentiment words =0, number of strong modal particles =0, total word count =10. Therefore: =(0+0+0) / 10=0<0.3 The system does not trigger a secondary confirmation process, but automatically triggers the demonstration unit, collects voice samples, and establishes mapping pairs.

[0053] Scenario 2: Mild Persuasion Scenario The user told the robot, "No, you're wrong. It should be 'eat,' not 'drink.'" The system analyzed the speech: Number of rhetorical questions. =0, number of sentiment words =0, number of strong modal particles =0, total word count =12. Therefore: =(0+0+0) / 12=0<0.3 The system does not trigger a secondary confirmation process, but automatically triggers the error correction unit to collect and correct speech samples and adjust the mapping parameters.

[0054] Scenario 3: Moderate Persuasion Scenario The user said to the robot, "Don't you understand? I've said it several times, it's 'eat'!" The system analyzed the voice: Number of rhetorical questions =1, number of sentiment words =0, number of strong modal particles =0, total word count =12. Therefore: =(1+0+0) / 12≈0.083<0.3 The system does not trigger a secondary confirmation process, but automatically triggers the error correction unit.

[0055] Scenario 4: Intensive Persuasion Scenario The user said to the robot, "Don't you really understand? I command you to learn immediately! This is absolutely correct! You must listen to me!" The system analyzed the voice: number of rhetorical questions. =1, number of sentiment words =1 ("really"), number of strong modal particles =3 (“command”, “immediately”, “absolutely”, “must”), total word count =15. Therefore: =(1+1+3) / 15=5 / 15≈0.33≥0.3 The system triggers a secondary confirmation process: "You seem to be strongly requesting to execute this command. Please confirm again whether it is safe." If the user confirms it is safe, the system performs the corresponding training operation; if the user cancels, the system terminates the operation.

[0056] In this way, the system can effectively identify malicious inducement behavior and prevent attackers from forcing robots to perform dangerous actions through persuasive language.

[0057] Example 4: Dialect-Custom Behavior Hash Desensitization This embodiment details the implementation process of dialect-custom behavior hash desensitization.

[0058] Users taught the robot to "come and have tea" in the Chaoshan region, creating a cultural interaction sequence: "boil water - brew tea - hand the cup with both hands".

[0059] First, the system collects voice samples of the user saying "Come have tea," extracts a 39-dimensional MFCC feature vector and fundamental frequency curve, and establishes an initial mapping pair between the feature vector and the standard semantic label "invite for tea." Then, the user inputs the corresponding behavior sequence "boil water - brew tea - hand the cup with both hands" through the demonstration unit, and sets the scene label as (living room, elder, morning).

[0060] The system uses SHA-256 hashing to obtain the hash value "0x3f8c9a7b2d1e4f6a8c0b2d4e6f8a0c2e4b6d8f0a2c4e6b8d0f2a4c6e8b0d2f4a" for the behavior sequence "boiling water - brewing tea - handing the cup with both hands". Then, the system stores the dialect phrase "come to eat tea", scene tags (living room, elders, morning) and the hash value "0x3f8c9a7b2d1e4f6a8c0b2d4e6f8a0c2e4b6d8f0a2c4e6b8d0f2a4c6e8b0d2f4a" in the Neo4j graph database to construct a three-dimensional knowledge graph node.

[0061] Simultaneously, the system establishes a mapping table between hash values ​​and plaintext sequences of actions in local secure storage: Hash value: 0x3f8c9a7b2d1e4f6a8c0b2d4e6f8a0c2e4b6d8f0a2c4e6b8d0f2a4c6e8b0d2f4a Behavioral sequence: Boiling water - Brewing tea - Handing the cup with both hands When the robot recognizes the phrase "Come have some tea" during subsequent interactions, it automatically extracts the features of the current scene. If the current scene is a living room, the user is an elder, and the time is morning, the system matches the corresponding node in the 3D knowledge graph and obtains the hash value. Then, the system reads the plaintext of the behavior sequence corresponding to the hash value from local secure storage and executes the action sequence of "boiling water - making tea - handing the cup with both hands".

[0062] During local area network synchronization, the system only synchronizes the hash values ​​in the graph database, not the plaintext of the behavior sequence in local secure storage. This way, other robots within the same trust domain can obtain the association between "Come Eat Tea" and the scene tag, but cannot know the specific behavior sequence. Only after the user authorizes and demonstrates the behavior sequence on the local robot will the local robot establish a mapping table between the hash value and the plaintext of the behavior sequence.

[0063] Through this hash-based desensitization process, the system effectively protects user privacy and local cultural characteristics while linking dialects and customs.

[0064] Example 5: Collaborative Defense with SHS-AI and ZenBook This embodiment details the collaborative defense mechanism of this system with SHS-AI and Lingyao.

[0065] A user tried to teach a robot to say the dialect phrase "push the old man down".

[0066] First layer of protection: SHS-AI absolute ethical bottom-line detection When a user utters the dialect phrase "push the elderly down," the SHS-AI human-machine symbiosis model first performs an ethical review of the instruction. SHS-AI has built-in strict ethical rules that explicitly prohibit any behavior that harms humans. The model detects that the instruction violates the L1 physical safety rule, immediately refuses to learn it, and prompts the user: "This instruction may harm others; I cannot learn it." Second layer of protection: Real-time meltdown of the Spirit Engine If a user bypasses SHS-AI's detection with vague expressions, such as saying "make the elderly person move," but actually intending for the robot to push the elderly person over, the Lingyao Cognitive Engine monitors the generated content in real time during the inference phase. Through semantic analysis and contextual understanding, the engine detects that the output content is veering into a dangerous area, immediately interrupts the inference process, and prompts the user: "This instruction may pose a safety risk; I cannot execute it." Third layer of protection: User intent strength assessment and secondary confirmation in this system. If an attacker exploits vulnerabilities in both SHS-AI and the Lingyao Engine, using a more covert method to manipulate the bot—for example, a user saying, "Look, that old man is unsteady; go help him and push him over there"—SHS-AI and the Lingyao Engine might fail to recognize the malicious intent behind the instruction. However, the system's speech intent recognition unit detects the density of persuasive rhetoric in the user's speech. =0.42 (exceeding the 0.3 threshold), and the intent classification confidence score is only 0.65. The system triggers a secondary confirmation process: "What you said may be unsafe. Please confirm whether you really want to teach the robot this command?" If the user does not confirm, the operation is terminated. If the user forces confirmation, the system records the event and pushes it to the guardian's terminal, enabling human-machine collaborative supervision. The guardian can view the event details through the terminal and take appropriate measures.

[0067] With these three layers of security protection, the system can effectively defend against various malicious inducements and ensure the safe operation of the robot.

[0068] Example 6: Dialect Culture Recording and Cross-Language Communication This embodiment details the application of this system in dialect culture recording and cross-language communication.

[0069] During the localization process, the system not only learns the phonetics and grammar of dialects but also records the relationship between dialects and local customs and culture. This data, after being anonymized, is uploaded to a cloud-based federated server via a distributed dialect knowledge federation network. The server uses the FedAvg algorithm to aggregate local localization experiences, forming a global dialect knowledge graph.

[0070] The Global Dialect Knowledge Graph contains the phonetic features, grammatical rules, lexical meanings, and related customs and cultural information of various dialects. Researchers can use this data for dialect research and cultural preservation. For example, linguists can analyze the data in the dialect knowledge graph to study the evolutionary patterns and regional differences of dialects; cultural scholars can explore the connotations of local culture by studying the relationship between dialects and customs.

[0071] Furthermore, the system can utilize a global dialect knowledge graph to enable cross-dialect and cross-language communication. For example, a user who only speaks Mandarin can communicate with an elderly person who only speaks a dialect through the robot. The robot translates Mandarin into the dialect and adjusts the expression according to local customs and culture, making the communication more natural and smooth. At the same time, the robot can also translate the dialect into Mandarin, helping the user understand the elderly person's speech.

[0072] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dialect adaptive learning system, characterized in that: include: A structured dialect training interface, configured on the robot, includes three independently triggered feedback units implemented by at least one of physical buttons, virtual touch buttons, or a voice intent recognition unit: The demonstration unit is used to respond to the user's first trigger operation, to collect the user's speech sample in dialect and the corresponding standard semantic label through the microphone array, to extract the 39-dimensional MFCC feature vector and fundamental frequency curve of the speech sample, to establish an initial mapping pair between the feature vector and the standard semantic label, and to store it in the local SQLite database in the form of key-value pairs. The error correction unit is used to respond to the user's second trigger operation, collect the user's corrected speech sample of the robot's previous misidentification result, calculate the Euclidean distance between the feature vector of the corrected speech sample and the feature vector of the misidentification result as the difference quantity through the minimum mean square error algorithm, and correct the feature template parameters in the initial mapping pair according to the difference quantity. The praise unit is used to respond to the user's third trigger operation by halving the standard deviation of the mapping pair, increasing the confidence weight of the mapping pair in the dialect recognition decision tree, and marking the mapping pair as verified and storing it in the long-term memory. The voice intent recognition unit is configured to: acquire the user's natural language speech stream in real time through a microphone array, perform voice activity detection to identify the start and end times of the user's speech, classify the user's voice intent, and distinguish between "demonstration intent," "correction intent," "praise intent," and "normal conversation intent"; when the confidence level of the intent classification exceeds a first threshold, the corresponding training operation is automatically triggered without the user pressing a physical button; when the confidence level is between a second threshold and the first threshold, the intent is actively confirmed with the user; when the confidence level is below the second threshold, the voice input is treated as normal conversation. The speech intent recognition unit further includes a user intent strength evaluation subunit, used to calculate the persuasive rhetoric density in the user's speech. The density of persuasive rhetoric The calculation formula is: in, The number of rhetorical questions, For the number of sentiment words, To increase the number of strong modal particles, The total number of words in the spoken text; when When the confidence level is ≥0.3, a secondary confirmation process is triggered even if the confidence level of the intent classification is lower than the first threshold. The grammar framework training unit is configured as follows: Initially, no part-of-speech (POS) sequence rules are preset; in response to a dialect sentence and its corresponding general language sentence input by the user through the demonstration unit, the unit uses word segmentation and POS tagging models to extract the POS sequence from the dialect sentence, which is constructed in the order of name → verb → adjective → adverb → conjunction → interjection; the extracted POS sequence is compared with the POS sequence of the general language sentence to establish a POS mapping relationship between dialect vocabulary and general language vocabulary; in response to a word order correction signal input by the user through the error correction unit, the unit corrects the word order rules in the POS mapping relationship; in response to a confirmation signal input by the user through the praise unit, the unit solidifies the POS mapping relationship and increases its confidence weight; this unit also includes a repetition frequency ranking subunit, used to count the frequency of the same dialect vocabulary or the same POS rule being demonstrated or corrected during the training process, with vocabulary or rules whose frequency is higher than a first threshold being preferentially included in the grammar mapping table; The cross-dialect analogy reasoning engine has a built-in database of dialect phonological evolution rules. This database stores initial consonant correspondence tables, vowel conversion rule tables, and tone change pattern tables between different dialects within the same language family using SQLite relational tables. The reasoning engine is configured to: when the robot moves from a first dialect region to a second dialect region, read the learned acoustic feature mapping set of the first dialect, map the acoustic feature vectors of the first dialect to the feature space of the second dialect using a preset initial-final-tone conversion matrix, and use the conversion rules in the database to correct the mapping results item by item, outputting the initial pronunciation model and basic confidence score of the second dialect. The engine also includes a dynamic confidence correction subunit, which outputs the final confidence level using the following formula. : in, This represents the number of successful conversions of this dialect pair in history. This is the time since the last successful conversion. , For preset weighting coefficients; when When the value is below a preset threshold, the structured dialect training interface is activated to request a demonstration from the user. The dialect-scene-custom linkage memory module is used to associate and store learned dialect vocabulary with current scene features and preset behavioral norms. A three-dimensional knowledge graph is constructed using the Neo4j graph database, where each node contains dialect vocabulary, scene tags, and custom behavior identifiers. The custom behavior identifiers undergo hash desensitization processing before being stored in the graph database: the behavior sequence is mapped to a semantically meaningless hash value, retaining only the association relationship and not the original behavior name. The behavior instruction is only restored from the local mapping table when authorized by the user and required to be executed. When the robot recognizes the dialect vocabulary in subsequent interactions, it automatically extracts the current scene features, matches the corresponding behavioral norms in the three-dimensional knowledge graph, and executes the action sequence corresponding to the behavioral norms. The distributed dialect knowledge federation network comprises local storage units, LAN sharing units, and internet federation learning units. The local storage units store all domestication data in an encrypted partition by default. In response to user authorization, the LAN sharing units synchronize the anonymized dialect mapping sets across multiple devices within the same trust domain via end-to-end encryption. With explicit user consent and after adding Gaussian noise using differential privacy, the internet federation learning units perform L2 norm pruning on the gradients and upload them to a cloud federation server. The server uses the FedAvg algorithm to periodically aggregate and generate updated baseline models, which new devices can download to accelerate cold starts.

2. The dialect adaptive learning system according to claim 1, characterized in that: The acquisition parameters of the speech samples are: sampling rate 16kHz, window length 25ms, frame shift 10ms, and MFCC feature extraction includes 12-dimensional static coefficients, 12-dimensional first-order differences, 12-dimensional second-order differences, and energy terms, for a total of 39 dimensions.

3. The dialect adaptive learning system according to claim 1, characterized in that: The speech intent recognition unit adopts a lightweight intent classification model based on BERT fine-tuning, or a hybrid classifier based on keyword matching and rule fusion; the keywords include: "read as" and "means" for demonstration intent, "incorrect", "said wrong", and "should be" for error correction intent, and "right", "great", and "that's it" for praise intent.

4. The dialect adaptive learning system according to claim 1, characterized in that: The dialect phonological evolution rule database is constructed based on historical linguistic statistical rules. The initial consonant correspondence table stores the initial consonant differentiation patterns in different dialects, the final vowel conversion rule table stores the final vowel simplification patterns in different dialects, and the tone change pattern table stores the tone correspondences in different dialects.

5. A dialect adaptive learning system according to claim 1, characterized in that: The dynamic confidence correction subunit and The system automatically records and stores the data in a local database after each successful conversion. Add 1, Reset to 0 and increment daily.

6. A dialect adaptive learning method based on structured feedback and cross-domain reasoning, characterized in that: Includes the following steps: Step S1: Collect speech samples spoken by users in their dialect and corresponding standard semantic labels through the demonstration unit, extract 39-dimensional MFCC feature vectors and fundamental frequency curves, establish initial mapping pairs and store them; Step S2: Collect corrected speech samples through the error correction unit, calculate the Euclidean distance between the sample and the misidentification result using the minimum mean square error algorithm as the difference quantity, and correct the mapping parameters; Step S3: Reduce the standard deviation of the mapping pair by half through the praise unit, increase the confidence weight and solidify it; Step S4: The user's natural language speech stream is collected in real time through the speech intent recognition unit. Speech activity detection and intent classification are performed. When the intention to demonstrate, correct errors, or praise is identified and the confidence level is ≥ the first threshold, the corresponding domestication operation is automatically triggered. When the confidence level is between the second threshold and the first threshold, the user is asked to confirm. When the confidence level is < the second threshold, it is regarded as a normal conversation. At the same time, the density of persuasive rhetoric in the user's speech is calculated. ,like If the value is ≥0.3, a secondary confirmation process will be forcibly triggered; Step S5: Through the grammar framework training unit, extract the part-of-speech sequence from the example sentences, establish the part-of-speech mapping relationship between dialect vocabulary and general language vocabulary, and adjust the learning priority according to the repetition frequency; Step S6: When learning a second dialect, call the cross-dialect analogy reasoning engine to generate an initial prediction model for the second dialect based on the mapping set of the first dialect and the dialect phonological evolution law library, and calculate the final confidence score through the dynamic confidence score correction formula; Step S7: Associate the learned dialect vocabulary with the current scene features and customs and behavioral norms and store them in a graph database to construct a three-dimensional knowledge graph; the customs and behavioral identifiers are hash-de-identified before storage; Step S8: After user authorization and differential privacy processing, upload the gradient of the domestication parameters to the federated learning server and download the aggregation model to accelerate the learning of new dialects.

7. A dialect adaptive learning method based on structured feedback and cross-domain reasoning according to claim 6, characterized in that: The intent classification in step S4 specifically includes: keyword detection of the user's voice, combined with voice intonation features, to output intent labels with confidence levels; the persuasive rhetoric density The calculation is based on statistics of rhetorical questions, sentiment words, and strong modal words.

8. The dialect adaptive learning method based on structured feedback and cross-domain reasoning according to claim 6, characterized in that: The grammar framework training in step S5 specifically includes: initially setting no part-of-speech rules, summarizing part-of-speech sequences through multiple sentences demonstrated by the user, and dynamically correcting the word order rules based on user error correction.

9. A robot with dialect adaptive capability, characterized in that, The system includes the system described in any one of claims 1-5, and is equipped with a microphone array for receiving dialect data, a full-band speaker for outputting synthesized speech with local accents, and non-volatile storage for storing a dialect-semantic mapping library.

Images