An AI chat robot dynamic prompt word adaptive interaction system and method

Abstract

The application discloses an AI chat robot dynamic prompt word adaptive interaction system and method, and the system comprises: a double-path acquisition module, which synchronously acquires a face image and a human voice; a biological data desensitization preprocessing module, which pre-processes and destroys original data locally on a device end; a visual face recognition module and a voiceprint recognition module, which independently output age, gender pre-labels and confidence, respectively; a double-mode fusion calibration module, which dynamically adjusts weights according to an environment and outputs a final identity label; a two-dimensional prompt word template storage library, which stores multiple sets of bottom-layer prompt words divided according to age and gender; a dynamic prompt word scheduling engine, which matches and replaces default prompt words of a large model according to the identity label; and a large model interaction and display module, which generates personalized dialogue; through double-mode fusion recognition and environment adaptive weights, the application realizes high-precision, full-scene and non-intrusive personalized interaction, and processes data locally on the device end, thereby ensuring user privacy security.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an AI chatbot dynamic prompt word adaptive interaction system and method based on the fusion recognition of human facial vision and voiceprint dual modality. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, AI chatbots have been widely used in various fields such as intelligent customer service, virtual assistants, education and tutoring, and emotional companionship. Existing AI chatbots usually rely on a fixed set of system prompts preset by developers to control their dialogue behavior, role setting, tone style, and content security policies. However, this "one-size-fits-all" approach cannot adapt to the personalized interaction needs of users of different ages and genders, resulting in poor user experience and a stiff interaction process.

[0003] To address the above problems, the following improvements have emerged in the prior art: The first category is interaction solutions based on visual recognition: Some high-end AI products (such as GPT-4o, Gemini, etc.) integrate cameras, which can perform facial emotion recognition or image content understanding; however, these solutions are limited to recognizing the user's instantaneous emotions or scene content, and cannot use the camera to automatically and accurately identify the age and gender information of the human body, let alone dynamically switch the underlying system prompts of the AI ​​robot accordingly; in other words, no product can currently automatically identify the age and gender of the human body through the camera, and automatically switch the underlying system prompts of the AI ​​and adapt to exclusive polite titles.

[0004] The second type is the manually selected interaction scheme: There are a few products on the market that allow users to manually select their age, gender, or AI persona; however, this type of scheme relies entirely on the user's manual operation and cannot achieve seamless automated interaction, which increases the user's usage threshold, especially when facing elderly or child users, the problem of inconvenience in operation is particularly prominent.

[0005] The third category is biometric recognition technology: existing voiceprint recognition and facial recognition technologies are mainly used in security fields such as user authentication, account login, liveness detection, or differentiation of family member devices; however, the purpose of these technologies is to confirm "who you are", rather than to identify "what kind of person you are" (i.e., age, gender attributes); therefore, they have not been used to drive personalized interactions of AI chatbots.

[0006] Furthermore, there is no technical solution in the industry that relies solely on cameras, voiceprints, or dual-modal fusion to identify age and gender, and then automatically changes the entire set of system prompts in conjunction with a large model.

[0007] In summary, existing solutions, whether single-camera, single-voiceprint, or dual-modal fusion, lack a complete technical chain for sensing the user's age and gender, along with seamless automatic switching of prompts and human-machine address. Furthermore, existing technologies have the following shortcomings: 1. Lack of fully automatic personalized perception capabilities: No solution can automatically and seamlessly identify a user's age and gender using only a camera or microphone, and adjust the AI's dialogue logic accordingly; existing visual capabilities are only used to view images and recognize emotions, without segmenting people by age and gender or linking with prompts.

[0008] 2. Insufficient accuracy of single-modal recognition: Relying solely on facial visual recognition is easily affected by lighting and occlusion (such as masks and sunglasses); relying solely on voiceprint recognition is easily affected by environmental noise and user physiological state (such as colds or voice changes), resulting in large errors in age and gender recognition, making it difficult to achieve refined user segmentation; while the publicly available face + voiceprint dual-modal technology only performs account security verification, does not recognize age or gender, and does not connect to large model prompt word systems. The application scenarios, technical purposes, and technical links of the technology are completely unrelated to this invention.

[0009] 3. The linkage of prompt words is too shallow: Even if there are solutions that can modify the way people address each other in the dialogue, most of them are just simple replacements at the front-end UI level, failing to reach the underlying system prompt words of the large model; this means that the core dialogue logic, content security filtering standards and response style of AI have not changed substantially, and the personalized interaction effect is greatly reduced; moreover, some solutions require users to manually select age, gender and dialogue persona, which cannot achieve fully automatic and seamless interaction, and the user threshold is high.

[0010] 4. Significant privacy risks: Most biometric identification solutions require uploading users' raw biometric data, such as facial images and voiceprint audio, to cloud servers for processing and storage. This seriously violates the requirements of the Personal Information Protection Law and other relevant laws and regulations, posing a huge risk of privacy leaks and data misuse.

[0011] Therefore, how to provide a personalized interaction solution for AI chatbots that can overcome the above-mentioned defects and achieve high precision, high privacy, full automation, and deep understanding has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0012] The present invention aims to solve the above-mentioned problems existing in the prior art and provides an AI chatbot dynamic prompt word adaptive interaction system and method to achieve accurate, seamless and privacy-secure recognition of user age and gender, and automatically and comprehensively replace the prompt words of the underlying system of the large model based on the recognition results, thereby significantly improving the personalization and intelligence level of human-computer interaction.

[0013] To achieve the above objectives, the present invention provides an AI chatbot dynamic prompt word adaptive interaction system, the system comprising: Dual-channel acquisition module, used to simultaneously acquire the user's facial image and voice audio; The biological data desensitization preprocessing module is connected to the dual-channel acquisition module and is used to perform noise reduction and feature extraction preprocessing on the face image and the human voice audio locally on the device, and destroy the original data immediately after processing. The visual face recognition module is connected to the biometric data desensitization preprocessing module and is used to output the age pre-label, gender pre-label and their corresponding first confidence level from the visual end based on the preprocessed face image through a lightweight visual recognition model. The voiceprint recognition module is connected to the biometric data desensitization preprocessing module and is used to output the age pre-label, gender pre-label and their corresponding second confidence level of the voiceprint based on the preprocessed human voice audio through a lightweight voiceprint recognition model. A dual-modal fusion calibration module is connected to the visual face recognition module and the voiceprint recognition module respectively. It is used to dynamically adjust the weights of the outputs of the visual end and the voiceprint end according to the current environmental parameters, and to perform weighted fusion calculation on the first confidence and the second confidence, and output a unique final user identity label. The final user identity label includes at least a final age label and a final gender label. A two-dimensional prompt word template repository is used to store multiple sets of underlying system prompt word templates categorized by age and gender dimensions; The dynamic prompt word scheduling engine is connected to the dual-modal fusion calibration module and the two-dimensional prompt word template repository, respectively. It is used to receive the end user identity tag and match and call the corresponding underlying system prompt word template from the two-dimensional prompt word template repository to replace the default system prompt word of the large model. The large model interaction and display module is connected to the dynamic prompt word scheduling engine and is used to generate personalized dialogue content based on the replaced underlying system prompt words and display it.

[0014] Furthermore, The dual-modal fusion calibration module is configured to: in response to detecting sufficient ambient light and an unobstructed user face, set the weight of the visual end to a first weight value and the weight of the voiceprint end to a second weight value, wherein the first weight value is greater than the second weight value; in response to detecting ambient light below a preset threshold or an obstruction of the user face, set the weight of the visual end to a third weight value and the weight of the voiceprint end to a fourth weight value, wherein the third weight value is less than the fourth weight value.

[0015] FurtherThe dual-modal fusion calibration module is also equipped with a conflict fallback mechanism: when the difference between the age pre-label of the visual end and the age pre-label of the voiceprint end exceeds a preset age difference threshold, and the first confidence level of the visual end and the second confidence level of the voiceprint end are both lower than a preset confidence threshold, the fusion result is abandoned and the dynamic prompt word scheduling engine is instructed to call the default general prompt word template.

[0016] Further The two-dimensional prompt word template repository stores prompt word templates divided into four age groups: children, youth, middle-aged, and elderly, and into three gender categories: male, female, and neutral, forming 12 independent underlying system prompt word templates in a 4×3 format. Each prompt word template is bound to a unique dialogue address, AI persona parameters, dialogue tone parameters, and content security filtering level.

[0017] This invention also provides an adaptive interaction method for dynamic prompts in AI chatbots, the method comprising the following steps: Step S1, Authorization Initialization: Obtain authorization to use the camera and microphone. If either authorization is denied, enable the default prompt word and disable the biometric function. Step S2, Dual-channel synchronous acquisition: In response to the detection of a user initiating a conversation, the user's facial image and voice audio are acquired simultaneously; Step S3, Data desensitization preprocessing: The face image and the voice audio are preprocessed locally on the device, and the original data is destroyed immediately after preprocessing; Step S4, Dual-path independent inference: The preprocessed face image is inferred through the visual recognition model, and the age pre-label, gender pre-label, and first confidence score are output from the visual end; the preprocessed human voice audio is inferred through the voiceprint recognition model, and the age pre-label, gender pre-label, and second confidence score are output from the voiceprint end. Step S5, Dual-modal weighted fusion calibration: The weights of the visual end and the voiceprint end are dynamically adjusted according to the current environmental parameters. The first confidence and the second confidence are weighted and fused to calculate and output a unique final user identity label. Step S6, Dynamic matching and replacement of prompt words: Based on the end user identity tag, match and call the corresponding underlying system prompt word template from the two-dimensional prompt word template repository to replace the default system prompt word of the large model; Step S7, Personalized Dialogue Output: The large model generates dialogue content based on the replaced system prompts and displays it through the front-end interface; Step S8, Data Destruction: Immediately destroy all facial images, audio recordings, and intermediate feature data collected during this conversation after the conversation ends, retaining only temporary digital identity tags.

[0018] Further In step S5, the process of dynamically adjusting the weights includes: obtaining the light intensity parameters of the current environment and / or the user's face occlusion detection results; when the light intensity is higher than a preset light threshold and no face occlusion is detected, assigning a first weight to the visual end and a second weight to the voiceprint end, wherein the first weight is greater than the second weight; when the light intensity is lower than the preset light threshold or face occlusion is detected, assigning a third weight to the visual end and a fourth weight to the voiceprint end, wherein the third weight is less than the fourth weight.

[0019] Further In step S5, a conflict fallback process is also included: if the difference between the age pre-label of the visual end and the age pre-label of the voiceprint end is greater than the preset age difference threshold, and the first confidence and the second confidence are both lower than the preset confidence threshold, then it is determined to be an extreme conflict scenario, the fusion result is abandoned, and the process jumps to step S6': calling the default general prompt word template.

[0020] The present invention achieves the following beneficial effects through the above-described system and method: 1. Significantly improved recognition accuracy: By using dual-modal fusion recognition of facial vision and voiceprint, the shortcomings of single-modal recognition in complex environments are effectively compensated, resulting in an overall improvement of over 35% in the accuracy of age and gender recognition.

[0021] 2. Achieve seamless interaction across all scenarios: The system can adaptively adjust the weights of the two recognition channels based on parameters such as ambient light and noise. It can work stably regardless of day or night, indoors or outdoors, or whether a mask is being worn, without requiring any manual operation from the user.

[0022] 3. Deep Personalized Interaction: This invention directly replaces the underlying system prompts of the large model, rather than merely modifying the front-end address; this means that the AI's role, tone, knowledge boundaries, and security strategies will all undergo fundamental changes based on the user profile, achieving true personalization.

[0023] 4. Extremely high privacy and security: All biometric data collection, processing, and identification are completed locally on the device. The original facial images and voiceprint audio are destroyed immediately after processing, fundamentally eliminating the risk of data leakage and fully complying with privacy protection regulations.

[0024] This invention can be directly adapted to existing general-purpose AI chatbots, smart speakers, and companion robots with cameras and microphones without requiring significant changes to the hardware structure, resulting in low implementation costs and high industrialization value. Attached Figure Description

[0025] Figure 1This is a block diagram illustrating the overall system principle of the AI ​​chatbot dynamic prompt word adaptive interaction system of the present invention; Figure 2a and 2b This is an overall flowchart of the AI ​​chatbot dynamic prompt word adaptive interaction method of the present invention; Figure 2a It's the upper part. Figure 2b It's the lower half; Figure 3a and 3b This is a schematic diagram of the age-gender two-dimensional prompt word matrix matching structure used in an embodiment of the present invention; Figure 3a It's the left half. Figure 3b It's the right half. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely for explaining the invention and not for limiting it. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] To address the technical pain points of existing single visual recognition and single voiceprint recognition, such as poor environmental adaptability, low recognition accuracy, shallow adaptation level of prompt words, and high privacy risks, this invention proposes an AI chatbot dynamic prompt word adaptive interaction system and method based on dual-modal fusion recognition of face vision and voiceprint.

[0028] This invention simultaneously collects user facial images and voice audio, and independently outputs age and gender pre-labels through visual recognition models and voiceprint recognition models, respectively. Then, a weighted fusion algorithm is used for comprehensive calibration to output the final accurate user identity label. Based on the final identity label, a preset two-dimensional prompt word template library is automatically retrieved to fully replace the prompt words of the underlying system of the large model, and exclusive dialogue address, tone of voice, content risk control level and personality logic are matched simultaneously. At the same time, all biometric data are processed offline locally on the device. After recognition is completed, the original facial image and original voiceprint audio are destroyed immediately, without uploading to the cloud or storing the original biometric data.

[0029] Furthermore, this invention significantly improves the accuracy of user age and gender recognition in complex environments through dual-modal complementary recognition, achieving seamless human-computer interaction across all scenarios. It optimizes the large model dialogue logic from the bottom layer, taking into account both interactive experience and biometric privacy and security, effectively solving the technical problems of traditional AI robots such as stiff interaction, weak scenario adaptability, and low recognition error tolerance.

[0030] ( Example 1 System Architecture Reference Figure 1 As shown, this embodiment provides an AI chatbot dynamic prompt word adaptive interaction system. The system can be logically divided into six layers: collection layer, preprocessing layer, recognition layer, fusion layer, scheduling layer and interaction layer, containing eight core functional modules. The calculation and processing of all modules are executed locally on the user device.

[0031] 1. Dual-channel acquisition module; This module is located in the acquisition layer and serves as the data entry point for this system. Acquisition only begins after user authorization, and it integrates two sub-modules: 1) Camera face capture submodule: After user authorization, when the system detects the user's intention to initiate a conversation (e.g., triggered by a voice wake-up word or proximity sensor), it quickly captures one or more frames of the user's face; to ensure a seamless experience, the entire capture process should be completed in milliseconds.

[0032] 2) Microphone Voiceprint Acquisition Submodule: Works synchronously with the camera, responsible for collecting the first sentence spoken by the user or a continuous speech as a voiceprint sample; this submodule will monitor the microphone input in real time and start recording when a valid voice signal is detected.

[0033] 2. Biometric Data Desensitization Preprocessing Module; This module is located in the preprocessing layer and connects to the dual-channel acquisition module. Its core task is to ensure data security and provide high-quality input for the subsequent identification module. Specific tasks include: 1) Preprocessing of face images: First, noise reduction is performed to remove noise introduced by the image sensor; then, face detection algorithms (such as MTCNN) are used to locate and crop the face region to eliminate background interference; finally, the color image is converted to grayscale to reduce the amount of computation; after this process is completed, the original, unprocessed image data is immediately erased from memory.

[0034] 2) Preprocessing of human voice audio: First, noise reduction filtering is performed to eliminate environmental background noise; then, human voice separation is performed to try to separate the pure human voice signal from the mixed audio; finally, silence truncation is performed to remove the silence segments before and after the speech, eliminate invalid environmental data, and retain only the valid speech segments; after this process is completed, the original audio file is immediately erased from memory.

[0035] 3. Visual Face Recognition Module; This module, located in the recognition layer, is a lightweight visual recognition engine. Internally, it incorporates a pre-trained MTCNN (Multi-task Cascaded Convolutional Networks) face detection algorithm and a lightweight ResNet (Residual Network, also known as an image feature classification network) classification model. This ResNet model has been fine-tuned specifically for age and gender classification tasks. The module receives a pre-processed grayscale image of a face. First, MTCNN accurately identifies the facial landmarks, and then the aligned face region is fed into the ResNet model. The model outputs two branches: an age pre-label (e.g., "youth") and a gender pre-label (e.g., "female"), each with its first confidence level (e.g., 94%).

[0036] 4. Voiceprint Recognition Module; This module, also located in the recognition layer, is a lightweight voiceprint analysis engine. It incorporates a pre-trained voiceprint spectrum analysis model, which can extract various features of the human voice, including but not limited to fundamental frequency (F0), formant frequencies, Mel-frequency cepstral coefficients (MFCCs), timbre features, etc. The model infers the speaker's age range and gender through statistical analysis of these features. This module outputs age pre-labels (e.g., "youth") and gender pre-labels (e.g., "female") from the voiceprint, along with their respective second confidence levels (e.g., 87%).

[0037] 5. Dual-modal fusion calibration module; This module, located in the fusion layer, dynamically allocates weights according to environmental conditions and is one of the core innovative modules of this invention. It receives pre-labels and confidence levels from both the visual and voiceprint modules and fuses them according to the following preset dynamic weighting algorithm: 1) Dynamic weight allocation: The module integrates an ambient light sensor interface and / or a face occlusion detection algorithm. When sufficient ambient light is detected (above the preset threshold, such as 50 lux) and the user's face is unobstructed (e.g., not wearing a mask, glasses without reflection), the visual recognition conditions are considered excellent. At this time, a higher visual weight (e.g., 0.7) and a lower voiceprint weight (e.g., 0.3) are assigned. Conversely, when the light is dim (below the preset threshold) or when the user is detected wearing a mask, sunglasses, or other obstructions, the reliability of visual recognition is considered to have decreased. The weights are then automatically switched, reducing the visual weight (e.g., 0.2) and increasing the voiceprint weight (e.g., 0.8).

[0038] 2) Weighted Fusion Calculation: As a preferred fusion method, this module uses a weighted average method to calculate the final age. Assuming the confidence level of the visual age label A_v is C_v, the confidence level of the voiceprint age label A_a is C_a, the visual weight is W_v, and the voiceprint weight is W_a; the final age label A_final can be determined through weighted voting or a weighted average. For example, for discrete age intervals, weighted voting can be used, and the interval with the highest score is the final result; for continuous age prediction, a weighted average can be used: A_final = (W_v... C_v A_v + W_a C_a A_a) / (W_v C_v+ W_a C_a); Similarly, gender identification can be determined by weighted voting, that is, by comparing the weighted total scores of the two categories 'male' and 'female', and the one with the higher score wins; finally, a unique and accurate age label and gender label are output.

[0039] 3) Conflict fallback mechanism: The module has a built-in logic judgment unit; if the age labels output by the visual end and the voiceprint end differ significantly (for example, the visual end considers it "child" and the voiceprint end considers it "elderly"), the fusion calibration module retains the recognition result of the one with higher confidence; if the confidence of both is lower than a preset low threshold (for example, 40%), the system determines it as an extreme conflict or a scenario where both modalities fail. At this time, the fusion calibration module abandons the fusion result and sends a "use default prompt word" signal to the scheduling layer to avoid a poor user experience caused by incorrect recognition.

[0040] 6. Two-dimensional prompt word template repository; This module, located in the scheduling layer, is a structured database that internally maintains a 4×3 two-dimensional matrix. Rows represent the age dimension, divided into four ranges: [0-12 years old] children, [13-35 years old] youth, [36-60 years old] middle-aged, and [60 years and older] elderly. Columns represent the gender dimension, divided into three types: male, female, and neutral (suitable for scenarios where gender is undesirable or cannot be clearly distinguished). This forms 12 cells, each storing a complete set of underlying system prompt word templates. Each template not only includes the AI's role definition (e.g., "You are a gentle and patient young female friend"), but also binds to: 1) Use specific terms of address in conversations, such as "little friend," "little sister," "sir," or "madam." 2) AI character parameters: such as level of knowledge, sense of humor, patience, etc.; 3) Dialogue tone parameters: such as lively, formal, friendly, concise; 4) Content security filtering level: For children, the strictest filtering is enabled to block violent, pornographic and terrifying content; for adults, the filtering is appropriately relaxed.

[0041] 7. Dynamic Prompt Word Scheduling Engine; This module, also located in the scheduling layer, serves as a bridge connecting the recognition layer and the interaction layer. It receives the end-user identity tag from the dual-modal fusion calibration module. Then, using the tag as an index, it quickly locates the corresponding cell in the two-dimensional prompt word template repository through a matrix matching algorithm and extracts the complete set of underlying system prompt words within that cell. Subsequently, the scheduling engine sends instructions to the large model interaction and display module to completely cover and replace the original default system prompt words of the large model with this new set of prompt words.

[0042] 8. Large Model Interaction and Display Module; This module is located in the interaction layer and is the interface through which the system and the user finally interact. After receiving new system prompts, it injects them into the context of the large language model. Subsequently, for every dialogue request initiated by the user, the large model will generate a response based on this new set of personalized system prompts. The front-end interface will also be updated synchronously, displaying a dialogue title and UI style that matches the user's identity.

[0043] It is important to note that the "underlying system prompts" replaced by this invention refer to the initial instructions directly injected into the context of the large language model. They define the model's role, behavioral guidelines, knowledge boundaries, and output format throughout the dialogue process. This is fundamentally different from the "dialogue titles" displayed only on the front end of the user interface. The former fundamentally changes the AI's "thinking" and "behavior," while the latter is merely a superficial text replacement. This invention achieves a personalized leap from "formal resemblance" to "spiritual resemblance."

[0044] In this embodiment, the technical effects of the aforementioned modules and mechanisms will be further explained: Dual-path (parallel) acquisition module: eliminates the need for time-sharing image and audio acquisition, does not increase system interaction latency, and ensures a seamless interactive experience; Dual-modal fusion calibration module: overcomes the limitations of single-camera recognition due to light and occlusion, and single-voiceprint recognition due to noise and voice status, improving the accuracy of recognition across all scenarios by more than 35%; Adaptive mechanism for dynamic weight allocation: without manual intervention, the device can automatically sense the environment and adjust the recognition weights, adapting to all usage scenarios such as day and night, indoor and outdoor, and mask obstruction; Local end-to-end biometric data desensitization and destruction mechanism: eliminates the leakage of original facial and voiceprint biometric data from the source, fully complying with personal information protection regulations; Underlying dynamic prompt word full-coverage replacement mechanism: unlike superficial address modification, it directly rewrites the underlying instructions for running the large model, fundamentally changing the AI's answer logic and risk control standards, significantly improving personalized interaction effects.

[0045] ( Example 2 Interaction Method Flow Reference Figure 2a and 2b As shown in the figure, this embodiment describes in detail the complete method and process of using the system described in Embodiment 1 for dynamic prompt word adaptive interaction.

[0046] Step S1, Authorization Initialization: Upon first system startup or after a reset, the system simultaneously obtains authorization for both the camera and microphone. A permission request window will pop up, explicitly requesting access to the device's camera and microphone. If the user denies either authorization, the system will enter "Basic Mode," directly enabling a pre-set, universal default system prompt and disabling all biometric-related functional modules to protect user privacy. If both authorizations are granted, the system will enter "Personalized Mode."

[0047] Step S2, Dual-channel synchronous acquisition: In "Personalized Mode", the system continuously monitors the user's interaction intent; once the system detects that the user has started speaking (e.g., through VAD voice activity detection) or initiates a conversation through other means (such as buttons or gestures), the dual-channel acquisition module is immediately activated; the camera captures a face image of the current frame, and the microphone simultaneously begins recording the user's voice, completing the acquisition of two channels of data in parallel; the two are almost simultaneous to ensure temporal correlation and reduce inconsistencies caused by changes in the user's facial expressions or voice.

[0048] Step S3, Data Desensitization Preprocessing: The acquired original face image and original human voice audio are immediately transmitted to the biometric data desensitization preprocessing module; this module performs local operations on the device's CPU or NPU (Neural Processing Unit), sequentially executing noise reduction, cropping / separation, grayscale conversion / truncation, and other operations; after all preprocessing steps are completed, the system immediately calls the secure erase function to overwrite the memory area storing the original image and audio with zeros, ensuring that the original biometric data is not left on the device in any form; the entire process is processed locally, and the original biometric data is not transmitted externally.

[0049] Step S4, Dual-path independent inference: The preprocessed image data and audio data are respectively sent to the visual face recognition module and the voiceprint recognition module; these two modules work in parallel without interfering with each other; the visual module outputs the age pre-label (e.g., "youth") and gender pre-label (e.g., "female") from the visual end, along with a first confidence level (e.g., 0.92); the voiceprint module outputs the age pre-label (e.g., "youth") and gender pre-label (e.g., "female") from the voiceprint end, along with a second confidence level (e.g., 0.85).

[0050] Step S5, Dual-modal weighted fusion calibration: The dual-modal fusion calibration module receives the output from step S4, combines it with the current ambient light parameters, dynamically adjusts the weights of the two recognition paths, and fuses them to obtain the final user identity tag. First, it reads the ambient light intensity Lux using its built-in ambient light sensor. Simultaneously, this module can obtain the signal-to-noise ratio (SNR) of the input audio from the voiceprint recognition module or preprocessing module as a parameter to measure the ambient noise level. Assuming Lux > 50 and no facial occlusion is detected, it is determined to be a "visual priority" scene, setting the visual weight W_v = 0.7 and the voiceprint weight W_a = 0.3. Then… The system performs weighted fusion calculations. For example, for gender, both visual and voiceprint predictions are "female," and the final gender label is determined to be "female." Similarly, for age, visual prediction is "young" (confidence 0.92), and voiceprint prediction is also "young" (confidence 0.85). Through weighted voting, the final age label is determined to be "young." The confidence level of the entire process is very high. If Lux < 50 in another scenario, W_v = 0.2 and W_a = 0.8 are set, and the voiceprint result is given priority. If the aforementioned extreme conflict situation is encountered, a conflict fallback mechanism is executed to generate an "unknown" or "general" label.

[0051] Step S6, Dynamic Matching and Replacement of Prompt Words: The dynamic prompt word scheduling engine receives the end-user identity tags (e.g., "youth", "female") from step S5; it uses these tags as keys to search in the two-dimensional prompt word template repository, quickly locates the cell "youth-female", and extracts the complete set of underlying system prompt words stored therein; subsequently, the scheduling engine injects these prompt words into the session context of the large model, completely replacing any previously used default prompt words.

[0052] Combination Figure 3a and 3b As shown in the figure, a 4x3 matrix structure is clearly displayed. The rows of the matrix represent the age dimension, from top to bottom: children (0-12 years old), youth (13-35 years old), middle-aged (36-60 years old), and elderly (60 years old and above). The columns of the matrix represent the gender dimension, from left to right: male, female, and neutral. Each cell in the matrix (a total of 12) stores a complete and independent set of underlying system prompt word templates. When the dynamic prompt word scheduling engine receives the end user identity tag (e.g., "youth-female") from the dual-modal fusion calibration module, the engine uses this tag as an index to quickly locate the cell in the 2nd row and 2nd column of the matrix through a matrix matching algorithm, and extracts all the prompt word content stored in that cell to replace the default system prompt word of the large model. Figure 3a and 3bThis intuitively demonstrates the logical relationship between structured storage and fast matching, enabling those skilled in the art to clearly understand the organization of the two-dimensional prompt word template repository and the working mechanism of the scheduling engine.

[0053] Step S7, Personalized Dialogue Output: The big model now has a brand new "personality" and "rules"; when the user asks a question again, the big model will generate a reply based on the new system prompts; for example, it will address the user as "Miss Sister", the tone will become lively and cute, and the content of the answer will be closer to the interests of young people; the front-end interface will simultaneously display this personalized dialogue content and address to the user.

[0054] Step S8, Data Destruction: When the current conversation ends (e.g., the user does not speak for a long time or actively ends the conversation), the system enters the cleanup phase; in addition to the original data destroyed in step S3, all intermediate feature data generated during the current conversation (such as facial feature vectors and voiceprint feature vectors) are also destroyed; the only thing left in the system is a highly abstract, irreversible digital identity tag (e.g., an enumeration value representing "young woman"), which is used to quickly match prompt words when the next conversation begins, but this tag itself cannot restore any of the user's biometric features to ensure privacy and security.

[0055] ( Example 3 Specific application scenarios To better illustrate the technical effects of the present invention, several typical application scenarios are listed below.

[0056] (Scenario Example 1) Normal Indoor Interaction With good lighting, an unobstructed face, and low ambient noise, the system primarily uses visual recognition, supplemented by voiceprint recognition, with both results cross-validated for maximum accuracy. For example, imagine Xiaoming interacting with an AI chatbot at home. The room is well-lit and quiet. After system startup, the camera captures a clear image of Xiaoming's face, and the microphone picks up his clear greeting. The visual recognition module outputs: Age - Young (94% confidence), Gender - Male (98% confidence); the voiceprint recognition module outputs: Age - Young (90% confidence), Gender - Male (95% confidence). The dual-modal fusion calibration module, detecting good lighting, sets W_v=0.7 and W_a=0.3. After fusion, the final label is determined to be "Young - Male." The scheduling engine retrieves the corresponding prompts from the repository, including the term "brother," a "cheerful and straightforward" tone, and "standard" content risk control, allowing for normal casual conversation. The large model generates a response based on this, engaging in a relaxed and pleasant dialogue with Xiaoming.

[0057] (Scenario Example 2) Visual Failure Scenario (Wearing a Mask) In scenarios where users wear masks or are in low-light or backlit conditions, the system reduces visual weighting and primarily uses voiceprint recognition to determine identity, ensuring the system can switch prompts normally. For example, when Grandma Li is chatting with the AI ​​while wearing a mask in winter, the camera captures her face with the mask on, resulting in a significant loss of key features. The visual recognition module outputs: Age - Middle-aged (45% confidence), Gender - Female (60% confidence). The voiceprint recognition module captures Grandma Li's clear voice and outputs: Age - Elderly (91% confidence), Gender - Female (97% confidence). The fusion module detects facial occlusion and automatically switches the weights to W_v=0.2, W_a=0.8. After weighted fusion, the high-confidence result of the voiceprint dominates, and the final label is determined to be "Elderly - Female". The system then retrieves the prompt corresponding to "Elderly - Female", changes the address to "Grandma", adjusts the tone to "respectful, gentle, and slightly slower", and raises the content risk control level to "strict", automatically blocking internet slang and potentially offensive topics to suit the usage habits of elderly users. As a result, Grandma Li has a very suitable communication experience.

[0058] (Scenario Example 3) Voiceprint Failure Scenario (Outdoor Noisy Environment) In noisy environments with no user voice input, the visual weighting is increased, relying solely on facial visual recognition for identity determination. For example, Xiao Zhang is chatting with AI on a noisy street; the ambient noise is extremely high, and the signal-to-noise ratio of the microphone's voice is very low. The voiceprint recognition module outputs: Age - Child (38% confidence), Gender - Male (55% confidence). The visual recognition module captures a clear image of Xiao Zhang's face and outputs: Age - Young Adult (88% confidence), Gender - Male (95% confidence). The fusion module detects abnormal ambient noise parameters (estimated through signal-to-noise ratio), and despite adequate lighting, the visual weighting is increased to W_v=0.9, W_a=0.1. The final label is determined to be "Young Adult - Male," and the personalized prompts are switched normally. The system successfully avoids incorrect adaptation caused by voiceprint recognition failure.

[0059] (Scenario Example 4) Extreme Conflict Scenario When face and voiceprint recognition results show significant conflict and both have low confidence levels, the system automatically reverts to the default general prompt word to avoid incorrect word matching. For example, in a special case, the system recognizes a user whose voice sounds like an "elderly male" (80% confidence) but whose face looks like a "young female" (70% confidence). The two results are vastly different, and neither has a very high confidence level. The fusion module triggers the conflict fallback mechanism, determines it as an unreliable scenario, abandons fusion, and instructs the scheduling engine to directly call the "general-default" prompt word template. Although personalization is lost, it avoids the embarrassment or offense that may be caused by incorrect recognition, ensuring the basic usability and robustness of the system.

[0060] Compared with existing technologies, this invention innovatively adopts differentiated reuse of mature bimodal capabilities, opening up entirely new application scenarios: While the industry currently only uses face + voiceprint bimodality for identity verification and device grouping, this invention is the first to use bimodal fusion technology for accurate identification of user age and gender, and it connects the perception layer and the large model application layer, filling the industry gap in bimodal biometric recognition linked to the dynamic switching of prompt words in the large model; it also designs an environment-adaptive dynamic weight allocation algorithm, requiring no manual adjustment, with the device automatically switching the two recognition weights according to real-time operating conditions, balancing recognition speed and accuracy; simultaneously, the bimodal recognition results directly link to the prompt words of the underlying system of the large model, rather than merely modifying the front-end surface address, changing the AI ​​dialogue persona, risk control rules, and response logic from the bottom up; the entire chain is computed locally on the end-device side, with sensitive biometric data such as face and voiceprint not being uploaded to the cloud, and the original data being destroyed immediately after recognition, meeting privacy compliance requirements in two dimensions; and it builds a two-dimensional matrix prompt word library of age + gender, combined with accurate bimodal recognition, to achieve refined and undifferentiated personalized adaptation for 12 groups of people.

[0061] Furthermore, compared to the time-sharing acquisition scheme that collects face and voiceprint sequentially, this invention uses synchronous parallel acquisition, resulting in lower interaction latency and a significant technical barrier; and compared to the fixed-weight bimodal recognition, this invention has stronger adaptive dynamic weight fault tolerance.

[0062] It should be understood that the above description is only a preferred embodiment of the present invention and is not sufficient to limit the technical solution of the present invention. For those skilled in the art, within the spirit and principles of the present invention, additions, subtractions, substitutions, transformations, or improvements can be made based on the above description. For example, a third modality, such as iris recognition and human body posture recognition, can be added to further improve the recognition accuracy in extreme scenarios, which is an equivalent variation of the multimodal fusion recognition of the present invention. For another example, the fixed dynamic weights can be optimized into machine learning adaptive weights based on historical recognition data, continuously optimizing the recognition accuracy based on user usage records, which is a conventional replacement of the weight algorithm. For yet another example, the existing fixed prompt word templates can be replaced with a lightweight cloud template library that can be incrementally updated (only synchronizing text prompt words, not biometric data), without changing the core logic of local biometric recognition, which is also a variation of the prompt word storage module. All these technical solutions with additions, subtractions, substitutions, transformations, or improvements should fall within the protection scope of the appended claims of the present invention.

Claims

1. An AI chatbot dynamic prompt word adaptive interaction system, characterized in that, include: Dual-channel acquisition module, used to simultaneously acquire the user's facial image and voice audio; The biometric data desensitization preprocessing module is connected to the dual-channel acquisition module and is used to preprocess the face image and the human voice audio locally on the device, and destroy the original data after preprocessing. The visual face recognition module is connected to the biometric data desensitization preprocessing module and is used to output the age pre-label, gender pre-label and first confidence score from the visual end based on the preprocessed face image. The voiceprint recognition module is connected to the biometric data desensitization preprocessing module and is used to output the age pre-label, gender pre-label and second confidence level of the voiceprint based on the preprocessed human voice audio. The dual-modal fusion calibration module is connected to the visual face recognition module and the voiceprint recognition module respectively. It is used to dynamically adjust the weights of the outputs of the visual end and the voiceprint end according to the current environmental parameters, and to perform weighted fusion calculation on the first confidence and the second confidence to output a unique final user identity label. A two-dimensional prompt word template repository is used to store multiple sets of underlying system prompt word templates categorized by age and gender dimensions; The dynamic prompt word scheduling engine is connected to the dual-modal fusion calibration module and the two-dimensional prompt word template repository, respectively. It is used to receive the end user identity tag and match and call the corresponding underlying system prompt word template from the two-dimensional prompt word template repository to replace the default system prompt word of the large model. The large model interaction and display module is connected to the dynamic prompt word scheduling engine and is used to generate personalized dialogue content based on the replaced underlying system prompt words and display it.

2. The AI ​​chatbot dynamic prompt word adaptive interaction system according to claim 1, characterized in that, The dual-modal fusion calibration module is configured as follows: In response to detecting sufficient ambient light and an unobstructed user face, the weight of the visual terminal is set to a first weight value, and the weight of the voiceprint terminal is set to a second weight value, wherein the first weight value is greater than the second weight value; In response to detecting that the ambient light is below a preset threshold or that the user's face is obstructed, the weight of the visual terminal is set to a third weight value, and the weight of the voiceprint terminal is set to a fourth weight value, wherein the third weight value is less than the fourth weight value.

3. The AI ​​chatbot dynamic prompt word adaptive interaction system according to claim 1, characterized in that, The dual-modal fusion calibration module is also equipped with a conflict fallback mechanism: when the difference between the age pre-label of the visual end and the age pre-label of the voiceprint end exceeds a preset age difference threshold, and both the first confidence and the second confidence are lower than a preset confidence threshold, the dynamic prompt word scheduling engine is instructed to call the default general prompt word template.

4. The AI ​​chatbot dynamic prompt word adaptive interaction system according to claim 1, characterized in that, The two-dimensional prompt word template repository stores prompt word templates divided into four age groups: children, youth, middle-aged, and elderly, and into three gender categories: male, female, and neutral, forming 12 independent underlying system prompt word templates in a 4×3 format. Each prompt word template is bound to a unique dialogue address, AI persona parameters, dialogue tone parameters, and content security filtering level.

5. A method for adaptive interaction using dynamic prompts in an AI chatbot, characterized in that, Includes the following steps: Step S1, Authorization Initialization: Obtain authorization to use the camera and microphone. If either authorization is denied, enable the default prompt word and disable the biometric function. Step S2, Dual-channel synchronous acquisition: In response to the detection of a user initiating a conversation, the user's facial image and voice audio are acquired simultaneously; Step S3, Data desensitization preprocessing: The face image and the voice audio are preprocessed locally on the device, and the original data is destroyed immediately after preprocessing; Step S4, Dual-path independent inference: The preprocessed face image is inferred through the visual recognition model, and the age pre-label, gender pre-label and first confidence score are output from the visual end; The preprocessed human voice audio is inferred using a voiceprint recognition model, and the pre-label of age, pre-label of gender, and second confidence level of the voiceprint are output. Step S5, Dual-modal weighted fusion calibration: The weights of the visual end and the voiceprint end are dynamically adjusted according to the current environmental parameters. The first confidence and the second confidence are weighted and fused to calculate and output a unique final user identity label. Step S6, Dynamic matching and replacement of prompt words: Based on the end user identity tag, match and call the corresponding underlying system prompt word template from the two-dimensional prompt word template repository to replace the default system prompt word of the large model; Step S7, Personalized Dialogue Output: The large model generates dialogue content based on the replaced system prompts and displays it through the front-end interface; Step S8, Data Destruction: Immediately destroy all facial images, audio recordings, and intermediate feature data collected during this conversation after the conversation ends, retaining only temporary digital identity tags.

6. The AI ​​chatbot dynamic prompt word adaptive interaction method according to claim 5, characterized in that, In step S5, the process of dynamically adjusting the weights includes: Obtain the current ambient light intensity parameters and / or the user's face occlusion detection results; When the light intensity is higher than a preset light threshold and no facial occlusion is detected, a first weight is assigned to the visual end and a second weight is assigned to the voiceprint end, with the first weight being greater than the second weight. When the light intensity is lower than a preset light threshold or when facial occlusion is detected, a third weight is assigned to the visual end and a fourth weight is assigned to the voiceprint end, wherein the third weight is less than the fourth weight.

7. The AI ​​chatbot dynamic prompt word adaptive interaction method according to claim 5, characterized in that, In step S5, a conflict fallback process is also included: if the difference between the age pre-label of the visual end and the age pre-label of the voiceprint end is greater than a preset age difference threshold, and both the first confidence and the second confidence are lower than a preset confidence threshold, then it is determined to be an extreme conflict scenario, the fusion result is abandoned, and the process jumps to step S6': calling the default general prompt word template.

8. The AI ​​chatbot dynamic prompt word adaptive interaction method according to claim 5, characterized in that, The two-dimensional prompt word template repository is divided into four age groups: children, youth, middle-aged, and elderly, and into three gender categories: male, female, and neutral, forming 12 independent underlying system prompt word templates in a 4×3 format.

9. The AI ​​chatbot dynamic prompt word adaptive interaction method according to claim 5, characterized in that, The preprocessing of the face image in step S3 includes noise reduction, face region cropping, and grayscale processing; the preprocessing of the voiceprint audio includes noise reduction filtering, voice separation, and mute truncation processing.

10. The AI ​​chatbot dynamic prompt word adaptive interaction system according to claim 1, characterized in that, The visual face recognition module is equipped with the MTCNN face detection algorithm and a lightweight ResNet classification model; the voiceprint recognition module is equipped with a lightweight voiceprint spectrum analysis model.

Images