An intelligent display terminal and control method for integrated cloud AI model invocation
By using multimodal noise reduction and parallel verification at the edge terminal, combined with knowledge interpolation autoencoders and large language models in the cloud, the problems of multi-source noise interference and privacy leakage in the home environment are solved, achieving efficient noise filtering, privacy protection and interaction optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN OSTAR DISPLAY ELECTRONIC CO LTD
- Filing Date
- 2026-04-30
- Publication Date
- 2026-06-02
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies struggle to effectively handle multi-source interference and high self-noise in noisy home environments, leading to false triggers and invalid interactions, excessive cloud inference resources and terminal power consumption, and easy leakage of privacy data.
Multimodal selective state-space model processing is performed on the edge terminal, and noise reduction is performed by combining visual prior features and UI context. User identity and intent are verified in parallel. Only the desensitized high-dimensional feature matrix is uploaded to the cloud. The cloud uses knowledge interpolation conditional variational autoencoder and large language model to generate sentiment responses.
It accurately filters out ambient noise in the home environment, ensuring user privacy and security, saving cloud inference computing power and terminal network power consumption, and improving the accuracy and stickiness of the interactive experience.
Smart Images
Figure CN122132002A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of voice terminal control technology, and in particular to an intelligent display terminal and control method that integrates cloud-based AI model invocation. Background Technology
[0002] With the development of voice assistants and large language model technology, devices such as smart central control screens, smart speaker terminals with screens, and TV voice remote controls are gradually becoming more common in home settings.
[0003] In open environments such as living rooms, devices often coexist with multiple sound sources, including televisions and speakers, and the content played by the devices' own speakers can generate strong echoes and self-noise. Current noise reduction technologies mostly rely on traditional acoustic methods such as echo cancellation (AEC), beamforming, and VAD / keyword wake-up, or employ single-modal depth models. Because household noise is characterized by high volume, near-field, multiple sources, and content-dependent variations, relying solely on audio-side processing can easily lead to problems such as false wake-up word triggers, voice recognition being contaminated by television sound, and users casually chatting with their backs to the screen triggering interactions, resulting in a degraded user experience and increased risk of accidental operation.
[0004] Meanwhile, home devices with cameras and microphones involve sensitive privacy data. Existing solutions generally upload raw or near-raw audio waveforms, image frames, or reversibly compressed data to the cloud for processing. Even with encryption such as TLS / HTTPS on the transmission link, this mainly addresses the issue of eavesdropping during transmission. If terminal-side cache management is inadequate, devices are compromised, or there is a risk of permission abuse / leakage on the cloud side, privacy information such as the user's appearance, home environment, and real voice may still be exposed. In addition, authentication and intent verification in many existing systems are performed in the cloud or serially, making it impossible to intercept invalid requests in a timely manner at the edge, resulting in unnecessary network power consumption and cloud inference costs.
[0005] In addition, existing response generation is mostly based on text semantics, and there is a lack of a consistent linkage mechanism between emotional tags, terminal UI visual presentation, and TTS tone parameters. This leads to problems such as correct content but inappropriate emotions and mismatch between display and voice style, making it difficult to meet the needs of family companionship interaction experience. Summary of the Invention
[0006] The technical problem solved by this invention is that existing technologies have difficulty solving the problems of high self-noise and multi-source interference in noisy home environments, false triggers and invalid interactions cannot be effectively intercepted at the edge, and cloud inference resources and terminal power consumption are too high.
[0007] To address the aforementioned technical problems, this invention provides the following technical solution: an intelligent display control method integrating cloud-based AI model invocation, comprising the following steps:
[0008] Step S1: Acquire the user's original visual image and audio commands in real time, perform noise reduction and feature extraction on the original visual image and audio commands, and generate desensitized interactive feature data.
[0009] Step S2: Parallel verification of interaction feature data to confirm user identity and interaction intent, and uploading the de-identified interaction feature data to the cloud via encryption;
[0010] Step S3: Receive interactive feature data and reconstruct the semantic vector. Use a knowledge interpolation conditional variational autoencoder to jointly model the semantic vector and generate candidate text.
[0011] Step S4: Call the cloud-based large language model and use a multi-dimensional enhanced sentiment response generation method to extract multi-dimensional sentiment tags from the interaction feature data. The large language model reconstructs and infers based on candidate texts and sentiment tags to generate a response instruction stream.
[0012] Step S5: The terminal receives the response command stream sent from the cloud, uses the display component interface function to perform UI visual rendering that matches the emotion tag, and simultaneously matches the intonation parameters of the emotion tag to synthesize audio signals through speech.
[0013] Preferably, step S1 includes the following sub-steps:
[0014] Step S11: Acquire the user's original visual image and audio commands in real time, and synchronously read the context feature vector of the display screen interface at the current moment. The context feature vector is used to represent the current active state of the UI interface and the local media playback volume level.
[0015] Step S12: Scale and normalize the original visual image to obtain a standardized visual image; use a lightweight spatial feature network to extract the user gaze direction vector and lip motion tensor of the standardized visual image and concatenate them into a visual prior feature vector.
[0016] Step S13: Perform a short-time Fourier transform on the audio command to extract the acoustic features at each time step and construct a time-frequency feature sequence;
[0017] Step S14: The context feature vector, visual prior feature vector, and time-frequency feature sequence are synchronously input into the multimodal selective state-space model for streaming noise reduction. The dynamic step size and projection matrix are calculated through a multimodal fusion gating network to obtain the acoustic feature matrix.
[0018] Step S15: The visual prior feature vector, context feature vector, and acoustic feature matrix are used as desensitized interaction feature data, cached in a local secure memory area, and then encrypted and uploaded to the cloud after identity and intent verification is passed.
[0019] Preferably, the specific processing logic for inputting the context feature vector, the visual prior feature vector, and the time-frequency feature sequence into the multimodal selective state-space model for streaming denoising is as follows:
[0020] By using a multimodal fusion gating network, the context feature vector, visual prior feature vector, and current time step time-frequency feature sequence are reduced in dimensionality and concatenated to construct a multimodal joint representation vector;
[0021] The dynamic step size parameter, dynamic input projection matrix, and dynamic output projection matrix for the current time step are generated using a multimodal joint representation vector. The basic transition matrix of the continuous state space is zero-order preserved discretization using the dynamic step size parameter to obtain a discrete transition matrix and a discrete input matrix. Based on the discrete transition matrix, the discrete input matrix, and the time-frequency feature sequence of the current time step, the hidden state vector is updated and the denoised feature output vector is calculated. The denoised feature output vectors for all time steps are extracted, and Mel-band mapping is performed on the denoised feature output vectors for all time steps to obtain the acoustic feature matrix.
[0022] Preferably, the specific processing logic for performing encryption encapsulation and uploading to the cloud after identity and intent verification is passed is as follows:
[0023] The RGB pixel data of the original visual image and the original PCM waveform data of the audio command are removed from the local memory, and only the visual prior feature vector, context feature vector and acoustic feature matrix are retained. Hardware encryption components are used for packaging and asymmetric encryption to obtain the desensitized interactive feature data.
[0024] Preferably, step S2 includes the following sub-steps:
[0025] Step S21: The desensitized interaction feature data is parsed at the edge computing terminal, the acoustic feature matrix and visual prior feature vector are extracted, and the data are synchronously input into the lightweight multi-task cross-attention network.
[0026] Step S22: Parallel verification is performed using a lightweight multi-task cross-attention network to confirm the user's identity and interaction intent. The specific processing logic for parallel verification is as follows:
[0027] The acoustic feature matrix is encoded by the voiceprint recognition branch in a lightweight multi-task cross-attention network, the voiceprint embedding vector of the current user is extracted, and the cosine similarity between the voiceprint embedding vector and the registered voiceprint vector of the preset owner in the local secure storage area is calculated to obtain the identity verification confidence.
[0028] By using the intent detection branch in a lightweight multi-task cross-attention network, multimodal fusion analysis is performed on the acoustic feature matrix and visual prior feature vector to identify wake-up acoustic phonemes. The user's gaze direction vector in the visual prior feature vector is used to determine whether the user is in an effective screen-gazing state. The interaction intent confidence is calculated comprehensively.
[0029] Step S23: Compare the authentication confidence level with the preset identity threshold, and compare the interaction intent confidence level with the preset intent threshold to determine the interception, and obtain the determination result.
[0030] Preferably, step S3 includes the following sub-steps:
[0031] Step S31: The cloud server receives the uploaded encrypted interactive data packet and uses the speech recognition network to perform textual parsing on the acoustic feature matrix in the desensitized interactive feature data. Combined with the current dialogue context stored in the cloud, the dialogue context semantic vector is extracted. Based on the visual prior feature vector in the desensitized interactive feature data, the basic sentiment tag vector is extracted.
[0032] Step S32: Using the retrieval enhancement generation module, retrieve objective fact information from the external fact database based on the dialogue context semantic vector C, and encode the objective fact information using a knowledge encoding network to obtain the fact knowledge vector K;
[0033] Step S33: Input the dialogue context semantic vector C, the basic sentiment label vector E, and the factual knowledge vector K into the variational autoencoder based on knowledge interpolation conditions. The variational autoencoder includes a semantic encoder, a knowledge encoder, and a latent variable fusion layer.
[0034] In the latent space of the variational autoencoder, factual knowledge interpolation and sentiment variable fusion are performed:
[0035] Using the semantic encoder and the knowledge encoder, the mean and variance of the dialogue semantics and the mean and variance of the factual knowledge are calculated respectively. Semantic latent variables and knowledge latent variables are obtained through reparameterization techniques, and learnable interpolation coefficients are used. semantic latent variables and knowledge latent variables Latent space knowledge interpolation is performed to generate fact-aligned latent variables. The calculation expression for the fact-aligned latent variables is as follows:
[0036] ;
[0037] in, For fact-aligned latent variables, the basic sentiment label vectors are mapped to sentiment control latent variables, and the sentiment control latent variables are concatenated with the fact-aligned latent variables to obtain multidimensional joint latent variables;
[0038] Step S34: Based on the knowledge interpolation conditional variational autoencoder decoder, the multidimensional joint latent variables are autoregressively decoded to generate candidate text.
[0039] Preferably, step S4 specifically includes:
[0040] Step S41: Call the cloud-based large language model, use the multi-dimensional enhanced emotional response generation method to obtain the enhanced computation graph, and perform joint analysis on the visual prior feature vector and acoustic feature matrix in the desensitized interaction feature data to extract the multi-dimensional emotional tags of the target user. The multi-dimensional emotional tags include the user's instantaneous emotional features, physical fatigue features, and interaction urgency features.
[0041] Step S42: Extract the system's preset dynamic personality feature vector and the common sense knowledge base related to the semantic vector of the dialogue context;
[0042] Step S43: The candidate text is used as factual benchmark material, and the multi-dimensional sentiment tags, dynamic personality feature vectors and common sense knowledge base are fused and constructed as sentiment constraints to obtain multi-dimensional personalized enhanced prompt words;
[0043] Step S44: The cloud-based large language model reads multi-dimensional personalized enhanced prompt words and performs reconstruction reasoning to transform the candidate text into the final response text;
[0044] Step S45: Obtain the current user's historical interaction feedback score, package and encrypt the final reply text with the UI visual rendering parameters that match the multi-dimensional emotional tags, and generate a reply command stream.
[0045] Preferably, step S5 specifically includes:
[0046] The system receives the reply command stream and parses out the final reply text, multi-dimensional emotion tags, and corresponding UI visual rendering parameters. The terminal calls the display component interface function, and based on the UI visual rendering parameters and the multi-dimensional emotion tags, drives the system's underlying graphics engine to perform background color mapping and dynamic interactive effects rendering of the current UI interface. Simultaneously, the terminal's built-in speech synthesis component is called to dynamically adjust the speech rate and pitch parameters of the speech synthesis based on the multi-dimensional emotion tags, synthesize the final reply text into an audio signal, and broadcast it through the speaker.
[0047] Preferably, the processing logic for comparing the confidence level of the interaction intent with a preset intent threshold to determine interception is as follows:
[0048] If the authentication confidence level Preset identity threshold or confidence level of the interaction intent If the preset intent threshold is set, the current interaction is determined to be an invalid trigger, the locally cached desensitized interaction feature data is immediately cleared and the network upload request is blocked.
[0049] If the authentication confidence level The preset identity threshold and the confidence level of the interaction intent A preset intent threshold is set, and the verification is confirmed and the edge-cloud collaboration mechanism is triggered. The network communication device of the embedded device is invoked to establish a secure transmission layer protocol link with the cloud server. The desensitized interaction feature data is then attached with a local device timestamp and a device physical anti-counterfeiting signature and uploaded to the cloud server through an encrypted channel.
[0050] In a second aspect, a control terminal includes a processor and a memory, the memory storing a computer program, and the processor executing any one of the intelligent display control methods for integrating cloud-based AI model invocation in the computer program.
[0051] The beneficial effects of this invention are as follows: This invention introduces a multimodal selective state space model into edge terminals, creatively utilizing visual prior features and UI context to guide auditory attention, enabling precise filtering of high-volume self-noise in the home environment. More importantly, this invention forcibly overwrites and destroys the original RGB pixel and PCM waveform data in local memory, only packaging and encrypting the highly abstract feature matrix for cloud transmission. Even if the data is intercepted by hackers or the cloud is compromised, it is absolutely impossible to reverse engineer the user's true appearance and voice, thus completely breaking through the privacy and security bottleneck of smart devices with screens from both physical and algorithmic perspectives.
[0052] This invention constructs a lightweight multi-task cross-attention network on the terminal side, and performs parallel computation and dual threshold comparison of voiceprint authentication and multimodal intent verification. This mechanism can accurately identify and reject invalid chatter caused by accidental touch of TV sound or when the user is not looking at the screen. It can also clear the cache and intercept cloud requests directly on the edge side, which greatly saves the inference computing power of large cloud models and the power consumption of terminal network.
[0053] This invention creatively proposes a dual-drive logic based on a knowledge interpolation conditional variational autoencoder and a large language model in the cloud. In the knowledge interpolation conditional variational autoencoder stage, by introducing dynamic interpolation coefficients, objective factual knowledge and colloquial semantics are mathematically bound together in the latent variable space. This curbs the possibility of random AI generation from the lowest level of the generation logic, ensuring the absolute accuracy of high-quality fact queries such as schedules and health. On the basis of ensuring factual accuracy, this invention extracts multi-dimensional emotional tags such as instantaneous emotions, fatigue, and urgency of users through the MERL method, and constructs enhanced prompt words by combining common sense base and dynamic personality. The large language model reconstructs and infers based on this, transforming factual text into personalized anthropomorphic expressions. Through reinforcement learning, it continuously evolves, greatly improving the emotional value and interaction stickiness of users. This invention synchronously sends multi-dimensional emotional tags and reply text to the terminal. The underlying graphics engine dynamically maps the background color and interactive effects of the UI interface accordingly, while the TTS engine dynamically adjusts the speech rate and pitch frequency, optimizing the natural human-computer interaction experience of the display screen. Attached Figure Description
[0054] Figure 1 The flowchart illustrates the steps of an intelligent display control method that integrates cloud-based AI model invocation, as provided in one embodiment of the present invention. Detailed Implementation
[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0056] Example, refer to Figure 1 This paper provides an intelligent display control method that integrates cloud-based AI model invocation, comprising the following steps:
[0057] Step S1: Acquire the user's original visual image and audio commands in real time, perform noise reduction and feature extraction on the original visual image and audio commands, and generate desensitized interactive feature data.
[0058] Step S2: Parallel verification of interaction feature data to confirm user identity and interaction intent, and uploading the de-identified interaction feature data to the cloud via encryption;
[0059] Step S3: Receive interactive feature data and reconstruct the semantic vector. Use a knowledge interpolation conditional variational autoencoder to jointly model the semantic vector and generate candidate text.
[0060] Step S4: Call the cloud-based large language model and use a multi-dimensional enhanced sentiment response generation method to extract multi-dimensional sentiment tags from the interaction feature data. The large language model reconstructs and infers based on candidate texts and sentiment tags to generate a response instruction stream.
[0061] In step S5, the terminal receives the response command stream sent from the cloud, uses the display component interface function to perform UI visual rendering that matches the emotion tag, and simultaneously matches the intonation parameters of the emotion tag to synthesize audio signals through speech.
[0062] This embodiment introduces a multimodal selective state space model at the edge terminal, creatively utilizing visual prior features and UI context to guide auditory attention, accurately filtering out high-volume self-noise in the home environment. More importantly, this invention forcibly overwrites and destroys the original RGB pixel and PCM waveform data in local memory, only packaging and encrypting the highly abstract feature matrix for cloud upload. Even if the data is intercepted by hackers or the cloud is compromised, it is absolutely impossible to reverse engineer the user's true appearance and voice, completely breaking through the privacy and security bottleneck of smart devices with screens from both physical and algorithmic levels.
[0063] A lightweight multi-task cross-attention network was built on the terminal side, which performs parallel computation and dual threshold comparison of voiceprint authentication and multimodal intent verification. This mechanism can accurately identify and reject invalid chatter caused by accidental touch of TV sound or when the user is not looking at the screen. The cache is cleared directly on the edge side and cloud requests are blocked, which greatly saves the inference computing power of the large cloud model and the power consumption of the terminal network.
[0064] Step S1 includes the following sub-steps:
[0065] Step S11: Acquire the user's original visual image and audio commands in real time, and synchronously read the context feature vector of the display screen interface at the current moment. The context feature vector is used to represent the current active state of the UI interface and the local media playback volume level.
[0066] Step S12: Scale and normalize the original visual image to obtain a standardized visual image; use a lightweight spatial feature network to extract the user gaze direction vector and lip motion tensor of the standardized visual image and concatenate them into a visual prior feature vector.
[0067] Step S13: Perform a short-time Fourier transform on the audio command to extract the acoustic features at each time step and construct a time-frequency feature sequence;
[0068] Step S14: The context feature vector, visual prior feature vector, and time-frequency feature sequence are synchronously input into the multimodal selective state-space model for streaming noise reduction. The dynamic step size and projection matrix are calculated through a multimodal fusion gating network to obtain the acoustic feature matrix.
[0069] Step S15: The visual prior feature vector, context feature vector, and acoustic feature matrix are used as desensitized interaction feature data, cached in a local secure memory area, and then encrypted and uploaded to the cloud after identity and intent verification is passed.
[0070] In home scenarios, smart control screens are typically in sleep mode to reduce power consumption (e.g., standby current as low as 5mA). When a user approaches or makes a sound, an infrared sensor or sound level triggers a hardware interrupt signal, instantly waking the system. At this point, the system immediately activates the camera and microphone. Assuming the living room TV is loudly playing news and the user is saying "Check tomorrow's schedule" to the screen, the system first reads the current display's contextual feature vector (e.g., the system senses it is in a screen-off state, but there is TV noise in the environment). To absolutely protect user privacy, the system does not store the user's face locally. Instead, it quickly extracts the user's gaze direction vector and lip movement tensor (visual prior feature vector) through a lightweight network. For audio noise reduction, an innovative DC-MSSM multimodal state space model is introduced. This model combines visual prior feature vectors and contextual feature vectors, using dynamic step sizes... Visually guided auditory attention is achieved. The model automatically locks its attention onto the user's voice, which is consistent with the lip movement frequency, while the sound of the TV is judged as invalid noise and forgotten, thus outputting an extremely pure acoustic feature matrix. Finally, the original RGB image and PCM recording are completely destroyed in physical memory, and only these abstract feature matrices are packaged and encrypted, thus achieving desensitization of user privacy data.
[0071] The specific processing logic for encryption encapsulation and cloud upload after successful identity and intent verification is as follows:
[0072] By using a multimodal fusion gating network, the context feature vector, visual prior feature vector, and current time step time-frequency feature sequence are reduced in dimensionality and concatenated to construct a multimodal joint representation vector;
[0073] The dynamic step size parameter, dynamic input projection matrix, and dynamic output projection matrix for the current time step are generated using a multimodal joint representation vector. The basic transition matrix of the continuous state space is discretized with zero-order preservation using the dynamic step size parameter to obtain a discrete transition matrix and a discrete input matrix. Based on the discrete transition matrix, discrete input matrix, and time-frequency feature sequence of the current time step, the hidden state vector is updated and the denoised feature output vector is calculated. The denoised feature output vectors of all time steps are extracted, and Mel-band mapping is performed on the denoised feature output vectors of all time steps to obtain the acoustic feature matrix.
[0074] In this embodiment, when the smart terminal with a screen is in a noisy living room environment and its own speaker is playing loud music, the user looks at the screen and speaks a voice command. At the current time step t, a time-frequency feature sequence containing ambient noise and the user's voice is acquired. Through a multimodal fusion gating network, the context feature vector representing that the device is playing loud music is then processed. Visual prior feature vectors representing the user's gaze at the screen and subtle lip movements. and the current time step frequency feature sequence mixed with music and human voices. Feature dimensionality reduction and concatenation are performed to construct a multimodal joint representation vector, the calculation expression of which is:
[0075] ;
[0076] in, This is a multimodal joint representation vector. This is the vector concatenation operator. These represent the learnable dimensionality reduction weight matrices for the corresponding time-frequency features, visual features, and contextual features, respectively. These matrices are used to align data from different modalities to the same dimensional space. The dynamic step size parameter for the current time step t is dynamically generated using the multimodal joint representation vector. Dynamic input projection matrix and dynamic output projection matrix The specific formula is as follows:
[0077] ;
[0078] ;
[0079] ;
[0080] in, For smooth activation function, These are the corresponding learnable projective weight matrices. For bias terms;
[0081] The zero-order preserved discretization of the pre-defined basic transition matrix in the continuous state space is performed using the calculated dynamic step size parameter to obtain the discrete transition matrix and the discrete input matrix.
[0082] ;
[0083] ;
[0084] in, It is a discrete transition matrix. For discrete input matrices, In this embodiment, if the multimodal selective state-space model detects through visual prior feature vectors that the user is not speaking (i.e., their lips are not moving), the dynamic step size parameter will be automatically reduced by the network and approach zero. At this time, the discrete transition matrix approaches the identity matrix, thus maintaining the historical hidden state unchanged. Simultaneously, the discrete input matrix approaches the zero matrix, thus ignoring noise input in the current frame and enabling noise reduction and interference resistance.
[0085] When the visual prior feature vector detects that the user's lips are moving, the dynamic step size parameter is amplified, enabling the model to acquire the speech signal in the current frame that is consistent with the lip movement frequency.
[0086] Based on the discrete transition matrix, discrete input matrix, and time-frequency feature sequence of the current time step, the hidden state vector of the system is updated in linear time complexity, and the denoising feature output vector of the current frame is calculated. The specific formula is as follows:
[0087] ;
[0088] ;
[0089] in, For the hidden state vector, Denoising feature output vector The hidden state vector stored in the previous time step is used for linear recursive calculation. The output noise reduction feature vector has filtered out environmental noise such as background music.
[0090] Finally, perform Mel-band mapping to generate a desensitization matrix and extract all time steps. Generate a sequence of noise-reduced feature output vectors. The noise reduction feature output vectors at all time steps are processed by frequency band mapping using a Mel filter bank to obtain a two-dimensional acoustic feature matrix. At this point, the acoustic feature matrix is no longer the original acoustic waveform data, but a desensitized high-dimensional feature array, thus achieving extreme noise reduction while eliminating the privacy risk of leakage of the original speech data.
[0091] The specific processing logic for secondary encryption and encapsulation of visual prior feature vectors, context feature vectors, and acoustic feature matrices is as follows:
[0092] The RGB pixel data of the original visual image and the original PCM waveform data of the audio command are removed from the local memory, and only the visual prior feature vector, context feature vector and acoustic feature matrix are retained. Hardware encryption components are used for packaging and asymmetric encryption to obtain the desensitized interactive feature data.
[0093] In this embodiment, assuming the smart terminal with a screen has just completed a preliminary perception and noise reduction of the user's "open the curtains" command, such as a home control screen, at this time, the system's local memory (such as SRAM or DRAM) simultaneously contains two types of data: RGB pixel data of the original visual image captured by the camera, containing the user's facial features and the home background, and original PCM waveform data of the audio command containing the user's real voice recorded by the microphone array; after the algorithm extracts the dimensionality-reduced visual prior feature vector (containing only the coordinate values of the gaze direction and the scalar of lip opening), context feature vector, and the two-dimensional acoustic feature matrix after model noise reduction and Mel mapping; and then performs thorough desensitization and removal at the physical memory level:
[0094] To eliminate the risk of privacy leakage at the physical source, the system's underlying memory management unit is triggered to execute explicit data overwrite instructions. The system directly releases and overwrites (such as zero-filling operations) the RGB pixel data of the original visual image and the original PCM waveform data of the audio instructions in local memory. At this time, the terminal device completely loses any intuitive physical medium that can reproduce the user's appearance, the background of the living room, and the user's original vocal timbre. The system cache only retains highly abstract visual prior feature vectors, context feature vectors, and acoustic feature matrices.
[0095] Secondly, the system calls hardware-level security components for asymmetric encryption encapsulation: Although the visual prior feature vector, context feature vector, and acoustic feature matrix cannot be restored to intuitive audio and video, in order to prevent malicious tampering or sniffing during network transmission, these three are spliced and packaged according to a preset data frame format. Subsequently, the system calls the hardware encryption components on the motherboard of the smart terminal to perform asymmetric encryption calculations on the spliced data packets.
[0096] Specifically, after the identity and intent verification at the edge is successful, the terminal calls the hardware encryption component to encrypt and encapsulate the de-identified interaction feature data. The hardware encryption component uses the cloud server public key (such as RSA-2048 or ECC elliptic curve public key) pre-burned in the device's secure area to perform asymmetric encryption on the encryption session key, and uses the session key to perform symmetric encryption (such as AES-GCM) on the de-identified interaction feature data, thereby generating an encrypted interaction feature data packet for uploading. After encryption, the originally structured feature matrix is converted into a ciphertext byte stream, achieving confidentiality and tamper resistance on the transmission link.
[0097] In this embodiment, due to the use of one-way feature extraction and immediate deletion of the original RGB and PCM data locally, coupled with a public key encryption mechanism, it is evident that when the desensitized interactive feature data is uploaded to the cloud via home Wi-Fi or a public network router, even if it is intercepted by a hacker using packet sniffing tools, the hacker will not be able to decrypt it. Even if the hacker breaks into the cloud server, obtains the private key, and decrypts the ciphertext, what they obtain is merely a set of numerical matrices, which cannot be reversed to reconstruct the user's photos or actual recordings. This mechanism perfectly complies with the strictest data privacy compliance requirements, dispelling users' privacy concerns about installing smart devices with cameras at home.
[0098] Step S2 includes the following sub-steps:
[0099] Step S21: Read and parse the desensitized interaction feature data from the local secure memory area, extract the acoustic feature matrix and visual prior feature vector, and input them synchronously into the lightweight multi-task cross-attention network.
[0100] Step S22: Parallel verification is performed using a lightweight multi-task cross-attention network to confirm the user's identity and interaction intent. The specific processing logic for parallel verification is as follows:
[0101] The acoustic feature matrix is encoded by the voiceprint recognition branch in a lightweight multi-task cross-attention network, the voiceprint embedding vector of the current user is extracted, and the cosine similarity between the voiceprint embedding vector and the registered voiceprint vector of the preset owner in the local secure storage area is calculated to obtain the identity verification confidence.
[0102] By using the intent detection branch in a lightweight multi-task cross-attention network, multimodal fusion analysis is performed on the acoustic feature matrix and visual prior feature vector to identify wake-up acoustic phonemes. The user's gaze direction vector in the visual prior feature vector is used to determine whether the user is in an effective screen-gazing state. The interaction intent confidence is calculated comprehensively.
[0103] Step S23: Compare the authentication confidence level with the preset identity threshold, and compare the interaction intent confidence level with the preset intent threshold to intercept and determine the result.
[0104] Step S24: When the verification is successful, the hardware encryption component is invoked, and the desensitized interactive feature data is asymmetrically encrypted and encapsulated using the pre-programmed cloud server public key to generate an encrypted interactive data packet. After attaching a device timestamp and a device physical anti-counterfeiting signature to the encrypted interactive data packet, it is uploaded to the cloud server through a secure transport layer protocol link.
[0105] In this embodiment, the desensitized interaction feature data is parsed in the local cache of the edge computing terminal, and a two-dimensional acoustic feature matrix (e.g., a Mel mapping matrix with a dimension of 80×T) and a one-dimensional visual prior feature vector (containing polar coordinate data of line-of-sight yaw angle and pitch angle) are extracted. The acoustic feature matrix and the visual prior feature vector are synchronously input into the locally preloaded lightweight multi-task cross-attention network.
[0106] By using the voiceprint recognition branch in a lightweight multi-task cross-attention network, deep convolutional encoding is performed on the acoustic feature matrix to extract a 256-dimensional voiceprint embedding vector for the current user. The registered voiceprint vector of the preset owner is then read, and the matching degree between the voiceprint embedding vector and the registered voiceprint vector is calculated using the cosine similarity formula to obtain the identity verification confidence level.
[0107] ;
[0108] in, For authentication confidence, For voiceprint embedding vectors, For the registered voiceprint vector, the authentication confidence score ranges from [0, 1]. An authentication confidence score close to 1 indicates that the current speaker is closer to the preset registered voiceprint vector.
[0109] By using the intent detection branch in a lightweight multi-task cross-attention network, phoneme decoding is performed on the acoustic feature matrix to output the acoustic confidence of the wake word. Simultaneously analyze the user's gaze direction vector from the visual prior feature vector, calculate the spatial angle between the user's gaze focus and the center point of the smart display screen, and output the visual gaze confidence score after Gaussian mapping:
[0110] ;
[0111] in, For visual fixation confidence, The spatial angle between the user's focal point and the center point of the smart display screen. The view frustum tolerance coefficient is set to 15 degrees in this embodiment. The confidence level of the interaction intent is calculated by comprehensively using the multimodal fusion formula. :
[0112] ;
[0113] In this embodiment, a visual gating function is introduced to implement a veto interception mechanism for visual gaze. :
[0114] ;
[0115] Among them, visual fixation threshold When set to 0.7, the visual gating function outputs 0 when the user fails to effectively look at the screen, thus directly rejecting the interaction and blocking the cloud request. When the user effectively looks at the screen, the gating function outputs 1, and the interaction intent is determined by the acoustic wake-up confidence, which ensures the robustness of wake-up and protects against accidental touches through visual gating.
[0116] The authentication confidence level is compared with the preset authentication threshold, and the interaction intent confidence level is compared with the preset intent threshold.
[0117] Step S3 includes the following sub-steps:
[0118] Step S31: The cloud server receives the uploaded encrypted interactive data packet and uses the speech recognition network to perform textual parsing on the acoustic feature matrix in the desensitized interactive feature data. Combined with the current dialogue context stored in the cloud, the dialogue context semantic vector is extracted. Based on the visual prior feature vector in the desensitized interactive feature data, the basic sentiment tag vector is extracted.
[0119] Step S32: Using the retrieval enhancement generation module, retrieve objective factual information from the external factual database based on the semantic vector C of the dialogue context, and encode the objective factual information using a knowledge encoding network to obtain the factual knowledge vector K;
[0120] Step S33: Input the dialogue context semantic vector C, the basic sentiment label vector E, and the factual knowledge vector K into the variational autoencoder based on knowledge interpolation conditions. The variational autoencoder includes a semantic encoder, a knowledge encoder, and a latent variable fusion layer.
[0121] In the latent space of the variational autoencoder, factual knowledge interpolation and sentiment variable fusion are performed:
[0122] Using a semantic encoder and a knowledge encoder, the mean and variance of dialogue semantics and factual knowledge are calculated respectively. Semantic latent variables and knowledge latent variables are obtained through reparameterization techniques, and learnable interpolation coefficients are utilized. semantic latent variables and knowledge latent variables Latent space knowledge interpolation is performed to generate fact-aligned latent variables. The calculation expression for the fact-aligned latent variables is as follows:
[0123]
[0124] in, For fact-aligned latent variables, the basic sentiment label vectors are mapped to sentiment control latent variables, and the sentiment control latent variables are concatenated with the fact-aligned latent variables to obtain multidimensional joint latent variables;
[0125] Step S34: Based on the knowledge interpolation conditional variational autoencoder decoder, the multidimensional joint latent variables are autoregressively decoded to generate candidate text.
[0126] After the cloud server receives the encrypted uploaded data, it uses hardware decryption components to parse the acoustic feature matrix and visual prior feature vector. The cloud then calls the speech recognition network to convert the acoustic feature matrix into the text "Please check my schedule for tomorrow." Simultaneously, the cloud reads the previous dialogue "How's the weather today? It's 25 degrees," and encodes it into a dialogue context semantic vector.
[0127] Based on visual prior feature vectors, the cloud analyzes the user's gaze direction (looking at the screen) and lip movement frequency (stationary), encoding the basic sentiment tag vector as: valence 0.6, arousal 0.4, dominance 0.7, and the sentiment category is determined to be "neutral". Subsequently, the cloud retrieval enhancement generation module retrieves "meeting at 9 am tomorrow and client visit at 2 pm" from the user's bound calendar API, and obtains the fact knowledge vector through the knowledge encoding network.
[0128] In the latent space of the variational autoencoder, the semantic encoder calculates the mean and log-variance of the semantic latent variables and samples them to obtain the semantic latent variables.
[0129] The knowledge encoder calculates the mean and logarithmic variance of the knowledge latent variables and samples them to obtain the knowledge latent variables;
[0130] The system's built-in intent classification multilayer perceptron detects that the current query contains the strong factual keyword "schedule," thereby dynamically predicting a learnable interpolation coefficient of 0.45 (meaning that 45% of the dialogue semantics are retained and 55% of the objective facts are relied upon).
[0131] The system calculates fact-aligned latent variables, maps the 32-dimensional basic sentiment label vector to sentiment control latent variables, and concatenates them to obtain multi-dimensional joint latent variables. The decoder finally generates candidate text: "You have a meeting at 9 a.m. tomorrow and a client visit at 2 p.m.".
[0132] Step S4 specifically includes:
[0133] Step S41: Call the cloud-based large language model and use the multi-dimensional enhanced emotional response generation method to obtain the enhanced computation graph. Based on the basic emotional tag vector extracted in step S31, perform joint analysis on the visual prior feature vector and acoustic feature matrix in the desensitized interaction feature data to extract the multi-dimensional emotional tags of the target user. The multi-dimensional emotional tags include the user's instantaneous emotional features, physical fatigue features, and interaction urgency features.
[0134] Step S42: Extract the system's preset dynamic personality feature vector and common sense knowledge base related to the semantic vector of the dialogue context;
[0135] Step S43: The candidate text is used as the factual benchmark material, and multi-dimensional sentiment tags, dynamic personality feature vectors and common sense knowledge base are fused and constructed as sentiment constraints to obtain multi-dimensional personalized enhanced prompt words;
[0136] Step S44: The cloud-based large language model reads multi-dimensional personalized enhanced prompts and performs reconstruction reasoning to transform candidate texts into final response texts;
[0137] Step S45: Obtain the current user's historical interaction feedback score, package and encrypt the final reply text with the UI visual rendering parameters that match multi-dimensional sentiment tags, and generate a reply command stream.
[0138] In this embodiment, the cloud calls a large language model to extract multi-dimensional sentiment tags from the visual prior feature vector and acoustic feature matrix:
[0139] (1) Instantaneous emotional characteristics: The fundamental frequency (F0) showed no abruptness and the lips opened and closed smoothly, which was judged as "neutral to positive", with a coding valence of 0.6 and an arousal of 0.4;
[0140] (2) Fatigue characteristics: Calculate the duration of continuous screen viewing and blinking frequency of the user. If no fatigue is found, the value is coded as 0.2.
[0141] (3) Urgency characteristics: Based on the analysis of speech rate (approximately 180 words / minute) and the time keyword "tomorrow", it was determined to be normal and coded as 0.3;
[0142] The system extracts preset dynamic personality trait vectors and common sense knowledge base embedding vectors from the cloud to construct multi-dimensional personalized enhanced prompts: "[System Role]: Intelligent central control screen assistant, prefers simplicity. [Fact Baseline]: Meeting at 9 am tomorrow, visit at 2 pm. [Emotional Constraint]: Neutral to positive emotions, no fatigue, non-urgent interaction. [Output Requirements]: Generate final reply text."
[0143] The large language model reconstructs and infers based on the above prompts, and outputs the final response text: "There is a meeting at 9 am tomorrow and a client visit at 2 pm. Remember to prepare in advance~".
[0144] Step S5 specifically includes:
[0145] The system receives the reply command stream and parses out the final reply text, multi-dimensional emotion tags, and corresponding UI visual rendering parameters. The terminal calls the display component interface function, and based on the UI visual rendering parameters and multi-dimensional emotion tags, drives the underlying graphics engine of the system to perform background color mapping and dynamic interactive effects of the current UI interface. Simultaneously, it calls the terminal's built-in speech synthesis component, dynamically adjusts the speech rate and pitch parameters of the speech synthesis based on the multi-dimensional emotion tags, synthesizes the final reply text into an audio signal, and broadcasts it through the speaker.
[0146] In this embodiment, the response instruction stream sent from the cloud is received and transmitted to the local secure execution environment for decryption. After decryption, the data packet is deserialized and three different types of data structures are extracted from memory: the final response text (string format) for carrying semantic information, the multi-dimensional emotion tag (multi-dimensional feature vector format) for representing emotional state, and the UI visual rendering parameters for controlling the display (a structure containing color configuration matrix and animation control protocol).
[0147] On the visual presentation side, the terminal's main control process calls the display component interface function to pass the UI visual rendering parameters and multi-dimensional emotion tags to the system's underlying graphics engine (such as OpenGL or Vulkan engine). The underlying graphics engine first performs background tone mapping: by reading the multi-dimensional data in the multi-dimensional emotion tags and combining it with the preset color space conversion matrix, it dynamically interpolates the global basic color value of the current UI interface, thereby smoothly transitioning the interface background to the target tone that matches the current emotion tag.
[0148] Subsequently, the underlying graphics engine performs visual rendering of dynamic interactive effects: by reading the state dimension data in the multi-dimensional emotion tags through the graphics card shader program, dynamically modifying the Bezier time curve function and animation frame rate of dynamic components in the UI interface (such as breathing light effects, ripple animations or component transition animations), generating rendered images and pushing them into the video memory frame buffer, and finally presenting them on the physical display screen.
[0149] Finally, the speech synthesis component is synchronously invoked to execute audio playback: While performing visual rendering, the terminal's built-in speech synthesis component (TTS engine) is synchronously invoked through an internal multi-threaded clock synchronization mechanism. After obtaining the final response text, the speech synthesis component bypasses the default static pronunciation configuration and uses multi-dimensional emotion tags as the dynamic adjustment input for the acoustic vocoder. The specific logic is as follows:
[0150] The speech synthesis component extracts feature values from multi-dimensional emotion tags, maps the feature values to adjustment factors in the acoustic feature generation model, and dynamically adjusts the speech rate control parameters (changing the duration of global pronunciation) and pitch fundamental frequency contour parameters (changing the fluctuation amplitude and reference frequency of sound waves) of the generated audio.
[0151] After dynamic intervention and waveform restoration of the aforementioned acoustic parameters, the final reply text is synthesized into a digital audio signal with corresponding empathic characteristics. This digital audio signal is then transmitted to the audio amplifier chip via the terminal's underlying Audio Hardware Abstraction Layer (HAL) and I2S digital audio interface, driving the speaker to complete the broadcast. Through this clock synchronization mechanism, the system achieves strict alignment between the dynamic rendering of the visual interface and the emotionally nuanced voice broadcast on the physical timeline.
[0152] The logic for comparing the confidence level of the interaction intent with a preset intent threshold to determine whether to block is as follows:
[0153] If the confidence level of identity verification Preset identity threshold or confidence level of interaction intent If a preset intent threshold is set, the current interaction is determined to be an invalid trigger, the locally cached de-identified interaction feature data is immediately cleared, and the network upload request is blocked.
[0154] If the confidence level of identity verification Preset identity threshold and interaction intent confidence level. The system presets an intent threshold, confirms successful verification, and triggers the edge-cloud collaboration mechanism. It then calls the network communication device of the embedded device to establish a secure transmission layer protocol link with the cloud server. After attaching the de-identified interaction feature data with the local device timestamp and the device's physical anti-counterfeiting signature, it uploads the data to the cloud server through an encrypted channel.
[0155] In this embodiment, the preset identity threshold is set to 0.82, and the preset intent threshold is set to 0.75; if <0.82 or If the value is less than 0.75, the current interaction is determined to be an invalid trigger (such as a guest speaking, accidental touch of the TV sound, or the host not looking at the screen while speaking), the local feature is immediately cleared and the upload is blocked; only when both conditions are met at or above the corresponding threshold will encrypted cloud upload be allowed.
[0156] Based on the same inventive concept, embodiments of this application provide a control terminal, a processor, and a memory. The memory stores a computer program, and the memory stores a computer program that can be loaded and executed by the processor to integrate a cloud AI model invocation method.
[0157] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.
Claims
1. A smart display control method integrating cloud-based AI model invocation, characterized in that, Includes the following steps: Step S1: Acquire the user's original visual image and audio commands in real time, perform noise reduction and feature extraction on the original visual image and audio commands, and generate desensitized interactive feature data. Step S2: Parallel verification of interaction feature data to confirm user identity and interaction intent, and uploading the de-identified interaction feature data to the cloud via encryption; Step S3: Receive interactive feature data and reconstruct the semantic vector. Use a knowledge interpolation conditional variational autoencoder to jointly model the semantic vector and generate candidate text. Step S4: Call the cloud-based large language model and use a multi-dimensional enhanced sentiment response generation method to extract multi-dimensional sentiment tags from the interaction feature data. The large language model reconstructs and infers based on candidate texts and sentiment tags to generate a response instruction stream. Step S5: The terminal receives the response command stream sent from the cloud, uses the display component interface function to perform UI visual rendering that matches the emotion tag, and simultaneously matches the intonation parameters of the emotion tag to synthesize audio signals through speech.
2. The intelligent display control method integrating cloud-based AI model invocation as described in claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Acquire the user's original visual image and audio commands in real time, and synchronously read the context feature vector of the display screen interface at the current moment. The context feature vector is used to represent the current active state of the UI interface and the local media playback volume level. Step S12: Scale and normalize the original visual image to obtain a standardized visual image; use a lightweight spatial feature network to extract the user gaze direction vector and lip motion tensor of the standardized visual image and concatenate them into a visual prior feature vector. Step S13: Perform a short-time Fourier transform on the audio command to extract the acoustic features at each time step and construct a time-frequency feature sequence; Step S14: The context feature vector, visual prior feature vector, and time-frequency feature sequence are synchronously input into the multimodal selective state-space model for streaming noise reduction. The dynamic step size and projection matrix are calculated through a multimodal fusion gating network to obtain the acoustic feature matrix. Step S15: The visual prior feature vector, context feature vector, and acoustic feature matrix are used as desensitized interaction feature data, cached in a local secure memory area, and then encrypted and uploaded to the cloud after identity and intent verification is passed.
3. The intelligent display control method integrating cloud-based AI model invocation as described in claim 2, characterized in that, The specific processing logic for inputting the context feature vector, the visual prior feature vector, and the time-frequency feature sequence into the multimodal selective state-space model for streaming denoising is as follows: By using a multimodal fusion gating network, the context feature vector, visual prior feature vector, and current time step time-frequency feature sequence are reduced in dimensionality and concatenated to construct a multimodal joint representation vector; The dynamic step size parameter, dynamic input projection matrix, and dynamic output projection matrix of the current time step are generated using the multimodal joint representation vector. The basic transition matrix of the continuous state space is discretized with zero-order preservation using the dynamic step size parameter to obtain the discrete transition matrix and discrete input matrix. Based on the discrete transition matrix, the discrete input matrix, and the time-frequency feature sequence of the current time step, the hidden state vector is updated and the denoising feature output vector is calculated. The denoising feature output vectors of all time steps are extracted, and Mel-band mapping is performed on the denoising feature output vectors of all time steps to obtain the acoustic feature matrix.
4. The intelligent display control method integrating cloud-based AI model invocation as described in claim 3, characterized in that, The specific processing logic for encryption encapsulation and cloud upload after successful identity and intent verification is as follows: The RGB pixel data of the original visual image and the original PCM waveform data of the audio command are removed from the local memory, and only the visual prior feature vector, context feature vector and acoustic feature matrix are retained. Hardware encryption components are used for packaging and asymmetric encryption to obtain the desensitized interactive feature data.
5. The intelligent display control method integrating cloud-based AI model invocation as described in claim 4, characterized in that, Step S2 includes the following sub-steps: Step S21: The desensitized interaction feature data is parsed at the edge computing terminal, the acoustic feature matrix and visual prior feature vector are extracted, and the data are synchronously input into the lightweight multi-task cross-attention network. Step S22: Parallel verification is performed using a lightweight multi-task cross-attention network to confirm the user's identity and interaction intent. The specific processing logic for parallel verification is as follows: The acoustic feature matrix is encoded by the voiceprint recognition branch in a lightweight multi-task cross-attention network, the voiceprint embedding vector of the current user is extracted, and the cosine similarity between the voiceprint embedding vector and the registered voiceprint vector of the preset owner in the local secure storage area is calculated to obtain the identity verification confidence. By using the intent detection branch in a lightweight multi-task cross-attention network, multimodal fusion analysis is performed on the acoustic feature matrix and visual prior feature vector to identify wake-up acoustic phonemes. The user's gaze direction vector in the visual prior feature vector is used to determine whether the user is in an effective screen-gazing state. The interaction intent confidence is calculated comprehensively. Step S23: Compare the authentication confidence level with the preset identity threshold, and compare the interaction intent confidence level with the preset intent threshold to intercept and determine the result. Step S24: When the verification is successful, the hardware encryption component is invoked, and the desensitized interactive feature data is asymmetrically encrypted and encapsulated using the pre-programmed cloud server public key to generate an encrypted interactive data packet. After attaching a device timestamp and a device physical anti-counterfeiting signature to the encrypted interactive data packet, it is uploaded to the cloud server through a secure transport layer protocol link.
6. The intelligent display control method integrating cloud-based AI model invocation as described in claim 5, characterized in that, Step S3 includes the following sub-steps: Step S31: The cloud server receives the uploaded encrypted interactive data packet and uses the speech recognition network to perform textual parsing on the acoustic feature matrix in the desensitized interactive feature data. Combined with the current dialogue context stored in the cloud, the dialogue context semantic vector is extracted. Based on the visual prior feature vector in the desensitized interactive feature data, the basic sentiment tag vector is extracted. Step S32: Using the retrieval enhancement generation module, retrieve objective fact information from the external fact database based on the dialogue context semantic vector C, and encode the objective fact information using a knowledge encoding network to obtain the fact knowledge vector K; Step S33: Input the dialogue context semantic vector C, the basic sentiment label vector E, and the factual knowledge vector K into the variational autoencoder based on knowledge interpolation conditions. The variational autoencoder includes a semantic encoder, a knowledge encoder, and a latent variable fusion layer. In the latent space of the variational autoencoder, factual knowledge interpolation and sentiment variable fusion are performed: Using the semantic encoder and the knowledge encoder, the mean and variance of the dialogue semantics and the mean and variance of the factual knowledge are calculated respectively. Semantic latent variables and knowledge latent variables are obtained through reparameterization techniques, and learnable interpolation coefficients are used. semantic latent variables and knowledge latent variables Latent space knowledge interpolation is performed to generate fact-aligned latent variables. The calculation expression for the fact-aligned latent variables is as follows: ； in, For fact-aligned latent variables, the basic sentiment label vectors are mapped to sentiment control latent variables, and the sentiment control latent variables are concatenated with the fact-aligned latent variables to obtain multidimensional joint latent variables; Step S34: Based on the knowledge interpolation conditional variational autoencoder decoder, the multidimensional joint latent variables are autoregressively decoded to generate candidate text.
7. The intelligent display control method integrating cloud-based AI model invocation as described in claim 6, characterized in that, Step S4 specifically includes: Step S41: Call the cloud-based large language model, use the multi-dimensional enhanced emotional response generation method to obtain the enhanced computation graph, and perform joint analysis on the visual prior feature vector and acoustic feature matrix in the desensitized interaction feature data to extract the multi-dimensional emotional tags of the target user. The multi-dimensional emotional tags include the user's instantaneous emotional features, physical fatigue features, and interaction urgency features. Step S42: Extract the system's preset dynamic personality feature vector and the common sense knowledge base related to the semantic vector of the dialogue context; Step S43: The candidate text is used as factual benchmark material, and the multi-dimensional sentiment tags, dynamic personality feature vectors and common sense knowledge base are fused and constructed as sentiment constraints to obtain multi-dimensional personalized enhanced prompt words; Step S44: The cloud-based large language model reads multi-dimensional personalized enhanced prompt words and performs reconstruction reasoning to transform the candidate text into the final response text; Step S45: Obtain the current user's historical interaction feedback score, package and encrypt the final reply text with the UI visual rendering parameters that match the multi-dimensional emotional tags, and generate a reply command stream.
8. The intelligent display control method integrating cloud-based AI model invocation as described in claim 7, characterized in that, Step S5 specifically includes: The system receives the reply command stream and parses out the final reply text, multi-dimensional emotion tags, and corresponding UI visual rendering parameters. The terminal calls the display component interface function, and based on the UI visual rendering parameters and the multi-dimensional emotion tags, drives the system's underlying graphics engine to perform background color mapping and dynamic interactive effects rendering of the current UI interface. Simultaneously, the terminal's built-in speech synthesis component is called to dynamically adjust the speech rate and pitch parameters of the speech synthesis based on the multi-dimensional emotion tags, synthesize the final reply text into an audio signal, and broadcast it through the speaker.
9. The intelligent display control method integrating cloud-based AI model invocation as described in claim 8, characterized in that, The processing logic for comparing the confidence level of the interaction intent with a preset intent threshold to determine whether to block is as follows: If the authentication confidence level Preset identity threshold or confidence level of the interaction intent If the preset intent threshold is set, the current interaction is determined to be an invalid trigger, the locally cached desensitized interaction feature data is immediately cleared and the network upload request is blocked. If the authentication confidence level The preset identity threshold and the confidence level of the interaction intent Set a preset intent threshold, confirm successful verification, and trigger the edge-cloud collaboration mechanism; The embedded device's network communication device is invoked to establish a secure transmission layer protocol link with the cloud server. The de-identified interactive feature data is then attached with a local device timestamp and a device physical anti-counterfeiting signature and uploaded to the cloud server through an encrypted channel.
10. A control terminal, characterized in that, The control terminal includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the intelligent display control method for integrated cloud AI model invocation as described in any one of claims 1-9.