Multi-mode induction display screen response method fusing voice interaction

Abstract

The invention discloses a multi-mode induction display screen response method fusing voice interaction, and relates to the technical field of man-machine interaction, and the method comprises the following steps: in a user interaction process, calling a built-in sensor through an interface component real-time state monitoring mechanism, continuously monitoring a rendering process of a target interface component at a millisecond-level time granularity, and displaying the rendering process of the target interface component; collecting multi-dimensional availability state parameter information of the interface component; the method comprises the following steps: preprocessing collected interface component multi-dimensional availability state parameter information, and extracting key indexes reflecting potential asynchronous triggering risks from preprocessed data through a feature engineering method; by sensing the state of the interface component in real time, extracting the asynchronous risk index and combining machine learning recognition and TCN prediction modeling, dynamic regulation and control of the induction triggering window and the animation rhythm are achieved, the problem of mispointing or dislocation of the induction component is avoided, the accuracy and safety of interaction response are improved, and the user experience is improved. The method is suitable for high-precision scenes such as vehicle-mounted, medical and industrial control.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a multimodal induction display screen response method integrated with voice interaction. Background Art

[0002] "Multimodal induced display screen response integrated with voice interaction" refers to an intelligent interaction method that integrates voice recognition technology and multimodal perception means (such as gesture recognition, facial recognition, gaze tracking, etc.) in information display devices (such as smart screens) to jointly perceive, judge and respond based on the user's voice commands and other behavioral characteristics. This method dynamically adjusts the screen display content, display method or interaction strategy by identifying the user's voice intention and combining it with their current visual focus, action behavior or emotional state, so as to guide the user to complete the task more naturally and accurately under the synergy of different perception signals, thereby improving the intuitiveness, responsiveness and personalized experience of human-computer interaction. For example, when the user says "recommend a movie to me", the system not only recognizes the voice content, but also selects the screen area for recommendation display based on the user's gaze direction, realizing a multimodal fusion mechanism of active perception and precise response.

[0003] The existing technology has the following deficiencies: In the multimodal induction display screen response process integrated with voice interaction, the existing technology generally adopts a strategy generation mechanism based on the fusion of semantic intent and behavioral characteristics to achieve dynamic responsive content guidance. However, in actual applications, the triggering speed of the induction response strategy is often ahead of the complete loading of the interface display content. The system generates and displays the induction components (such as highlight prompt boxes, guide arrows or action prompt labels) in advance before completing the rendering of the interface components and the positioning of the content. This asynchronous triggering problem can easily lead to the positioning offset, mispointing of the target or hovering in a blank area of ​​the induction component, thereby seriously damaging the user's operation consistency and information understanding consistency. Especially in in-vehicle interaction, medical information display or industrial control interface, erroneous induction may cause the misselection of key buttons, misreading of prompt information or execution of unexpected operations, which has a high security risk and hidden danger of task failure.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a multimodal induced display screen response method that integrates voice interaction. By real-time perception of interface component status, extraction of asynchronous risk indicators, and combining machine learning recognition and TCN predictive modeling, dynamic regulation of the induced trigger window and animation rhythm is achieved, avoiding the problem of mis-pointing or misplacement of the induced components, and improving the accuracy and safety of the interactive response. It is suitable for high-precision scenarios such as automotive, medical and industrial control, to solve the problems in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a multimodal induction display screen response method integrating voice interaction, comprising the following steps:

[0007] During user interaction, the real-time status monitoring mechanism of the interface components is used to call built-in sensors to continuously monitor the rendering process of the target interface components at millisecond-level time granularity and collect multi-dimensional availability status parameter information of the interface components;

[0008] Preprocess the collected multi-dimensional availability status parameter information of interface components and extract key indicators reflecting potential asynchronous triggering risks from the preprocessed data through feature engineering methods;

[0009] Conduct comprehensive analysis on the extracted key indicators to quantify potential asynchronous trigger risks;

[0010] The key indicators after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk status annotation data. The model then intelligently evaluates the rendering status of the current target interface component to determine whether there is an asynchronous trigger risk.

[0011] When the assessment shows that there is a risk of asynchronous triggering, the temporal correlation between the user voice triggering behavior sequence and the interface component loading delay is modeled through a prediction model based on a time convolutional network. The historical interaction sequence and the current system load situation are integrated to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and presentation animation duration of the induced component are dynamically adjusted to achieve self-calibration of the response rhythm.

[0012] Preferably, during the user interaction process, the specific process of the interface component real-time status monitoring mechanism is as follows:

[0013] After initializing the interactive session, start the interface status monitoring module and bind the target interface component node to be monitored;

[0014] By calling the built-in system sensors or the rendering hook interface of the UI framework, low-level events related to the component rendering are captured in real time;

[0015] Continuously poll or monitor these event states at millisecond-level time granularity to build a multi-dimensional state parameter sequence that reflects component availability;

[0016] The collected raw state data is structured and encapsulated as the basic input for subsequent asynchronous risk assessment and induction strategy control, achieving high-precision dynamic perception of component rendering status.

[0017] Preferably, key indicators reflecting potential asynchronous triggering risks are extracted from the preprocessed data through feature engineering methods. The extracted indicators include the frequency of the monitoring component state switching from "loaded" → "unloaded" → "loaded" multiple times in a short period of time and the response delay fluctuation amplitude of the interface interaction component in different time periods. After comprehensive analysis under the detection window, the frequency of the monitoring component state switching from "loaded" → "unloaded" → "loaded" multiple times in a short period of time and the response delay fluctuation amplitude of the interface interaction component in different time periods are generated to generate a state rebound reference value and a response fuzzy reference value respectively. The potential asynchronous triggering risk is quantified by the state rebound reference value and the response fuzzy reference value.

[0018] Preferably, the specific steps of comprehensively analyzing the frequency of switching the status of the monitoring component from "loaded" → "unloaded" → "loaded" multiple times in a short period of time in the detection window to generate the status rebound reference value are as follows:

[0019] Within the detection window, the loading state of the target interface component is continuously sampled, where the state "loaded" is marked with a value of 1 and the state "unloaded" is marked with a value of 0. When the component state is detected to switch from "loaded → unloaded → loaded" in a short period of time, it is identified as a rebound event. The number of all rebound events within the detection window is counted, and the loading state recovery time span corresponding to each rebound event is recorded. The state switching density factor is constructed using the following formula:

[0020]

[0021] Where N r is the total number of rebound events, D i is the duration of the i-th rebound event, α is the density amplification factor, and Λ is the state switching density factor;

[0022] After obtaining the state switching density factor Λ, the abnormal recognition ability of short-term high-frequency rebound behavior is improved. A bilateral normalized nonlinear adjustment function is introduced to transform the density factor Λ to generate a state rebound reference value. The specific calculation method is as follows:

[0023] SRRV=tanh(·Λ)

[0024] , where SRRV is the state rebound reference value, δ is the state response amplification coefficient, and tanh(·) is the hyperbolic tangent function.

[0025] Preferably, the specific steps of generating a response fuzzy reference value after comprehensively analyzing the response delay fluctuation amplitude of the interface interaction component in different time periods under the detection window are as follows:

[0026] In the detection window, the continuous response events of the target interface interaction component are sampled in real time to obtain the response delay value sequence D of adjacent response events, where D = {d j}={d1, d2, d3, ... d n}, where d j is the response delay of the jth response event, n is the total number of response events, and for two adjacent response events, the response delay change gradient value is calculated. The calculation expression is as follows:

[0027] Δ j =tanh(λ·(d j+1 -d j ))

[0028] , where d j+1 is the response delay of the j+1th response event, that is, the response delay of the next response event, λ is the disturbance sensitivity adjustment coefficient, ranh is the hyperbolic tangent function, Δ j is the response delay change gradient value;

[0029] After obtaining all response delay change gradient values, nonlinear amplitude amplification and weighted summation are performed on each response delay change gradient value to obtain the final response fuzzy reference value. The calculation formula is as follows:

[0030]

[0031] Where RBRV is the response fuzzy reference value, β is the trend amplification coefficient, and γ is the disturbance amplitude power adjustment factor.

[0032] Preferably, the state rebound reference value and response fuzzy reference value after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk state annotation data. The asynchronous trigger risk coefficient is generated by the model, and the rendering state of the current target interface component is intelligently evaluated based on the asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk.

[0033] Preferably, the asynchronous trigger risk coefficient generated by intelligently evaluating the rendering state of the current target interface component using a machine learning model trained based on historical interaction data and asynchronous risk state annotation data is compared with a pre-set asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk. The judgment logic is as follows:

[0034] If the asynchronous trigger risk coefficient is greater than the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is an asynchronous trigger risk; if the asynchronous trigger risk coefficient is less than or equal to the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is no asynchronous trigger risk.

[0035] Preferably, when the assessment shows the risk of asynchronous triggering, a prediction model based on a temporal convolutional network is used to model the temporal correlation between the user voice triggering behavior sequence and the interface component loading delay. This model integrates the historical interaction sequence with the current system load situation to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and animation presentation duration of the induced component are dynamically adjusted. The specific steps for achieving self-calibration of the response rhythm are as follows:

[0036] After determining that there is an asynchronous trigger risk, that is, the asynchronous trigger risk coefficient ATRC is greater than the asynchronous trigger risk coefficient reference threshold, the prediction mechanism based on the time convolutional network is activated to construct a temporal relationship mapping between the user voice trigger behavior sequence and the interface loading history state sequence. The formula is as follows:

[0037] Y load =TCN(X voice ||X load ||X context )

[0038] , where Y load is the predicted output, X voice is the user voice-triggered behavior sequence, X load is the state sequence of the interface loading process, X context is the current system load situation vector, TCN is the temporal convolutional network model, and || is the sequence concatenation operation;

[0039] Root test prediction output Y load , combined with the deviation between the current asynchronous trigger risk coefficient ATRC and the preset asynchronous trigger risk coefficient reference threshold, the trigger delay window of the final induced component is determined. The trigger delay window is calculated as follows:

[0040]

[0041] Where, Δ induce is the trigger delay window of the inducing component, ATRC th is the reference threshold of the asynchronous trigger risk coefficient, and θ is the risk sensitivity adjustment coefficient;

[0042] After the delay window calculation is completed, the trigger delay window Δ based on the induced component induce Adjust the animation rendering parameters of the induced component and calculate the rendering duration of the induced animation. The calculation expression is as follows:

[0043]

[0044] Where, Ω anim is the duration of the induced animation. is a rhythm adjustment factor used to control the response amplitude amplification relationship between the delay window and the animation rhythm, and ω is the minimum basic presentation duration.

[0045] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0046] The present invention introduces a real-time perception mechanism for the rendering state of interface components and multi-dimensional usability state parameter modeling, supplemented by key risk indicators extracted by feature engineering and a risk identification model driven by machine learning, to effectively identify and predict asynchronous risks that may exist when inducing components are triggered; further combining the temporal convolutional network (TCN) to predict the temporal coupling relationship between user voice triggering and interface response, and achieve dynamic adaptive adjustment of the inducing trigger window and animation rhythm. This solution not only avoids problems such as inducing components mistakenly pointing to blank areas or misplaced highlights, but also significantly improves the system's response accuracy, logical coherence, and operational safety during human-computer interaction. It is particularly suitable for scenarios such as in-vehicle systems, medical terminals, and industrial control that require extremely high interaction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0048] Figure 1 This is a flow chart of the method for inducing display screen response using multimodal technology that integrates voice interaction. DETAILED DESCRIPTION

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0050] The present invention provides Figure 1 The multimodal induction display screen response method integrating voice interaction shown includes the following steps:

[0051] During user interaction, the real-time status monitoring mechanism of the interface components is used to call built-in sensors to continuously monitor the rendering process of the target interface components at millisecond-level time granularity and collect multi-dimensional availability status parameter information of the interface components;

[0052] "Multi-dimensional availability status parameters" refer to status information in multiple different dimensions used to comprehensively describe whether an interface component is currently "available." These dimensions cover various key indicators from the initial loading of a component to its final interactivity. Common examples include: component mounting completion flag, rendering progress percentage, first responsiveness time, click response delay, whether a layer is activated, animation execution status, whether the style has been rendered, whether dependent resources (such as images or scripts) have been loaded successfully, etc. The purpose of collecting these multi-dimensional parameters is to more accurately determine whether the component is truly "inducible," that is, whether it can be highlighted, respond to user operations, or display guidance information. A single dimension (such as mounting completion) often cannot truly reflect the overall availability of a component, and can easily lead to the induction strategy being mistakenly triggered when the content is not fully loaded or is not interactive. Therefore, by collecting and integrating multi-dimensional availability status parameters, the system can evaluate the true status of the component from a more comprehensive and detailed perspective, significantly improving the accuracy of the judgment of the timing of the induction response and the stability of the system interaction.

[0053] During the user's interaction with the system, a monitoring mechanism is activated to track the status of interface components in real time. By calling built-in software and hardware resources (such as rendering engine callbacks, GPU rendering status, UI framework lifecycle hooks, or dedicated performance sensors, etc.), the status information of the target interface components is continuously collected at millisecond intervals. The collected parameters include but are not limited to: whether the component has been mounted, whether it has completed rendering, whether it has interactive response capabilities, whether the layer is activated, whether the animation has been completed, etc. These multi-dimensional availability status parameters are used to dynamically determine whether a certain interface component is truly "available" at the moment, that is, whether it is ready to be highlighted, indicated, or interactively called by the induced component.

[0054] The core role of this step is to provide the system with a real-time, fine-grained rendering status perception foundation, thereby supporting subsequent asynchronous trigger risk assessment and induced response strategy regulation, ensuring that induced prompts are not triggered prematurely when the component is not ready.

[0055] During user interaction, the specific process of the real-time status monitoring mechanism for interface components is as follows: First, after initializing the interactive session, the interface status monitoring module is started and bound to the target interface component node to be monitored. Second, by calling the built-in system sensor or the UI framework's rendering hook interface, low-level events related to the component's rendering, such as mounting completion, layer activation, and animation loading progress, are captured in real time. Next, these event states are continuously polled or monitored at millisecond time granularity to construct a multidimensional state parameter sequence reflecting component availability. Finally, the collected raw state data is structured and encapsulated as the basic input for subsequent asynchronous risk assessment and induced strategy control, achieving high-precision dynamic perception of component rendering status. This process ensures that the system can grasp the real-time availability of components at the initial stage of interaction, avoiding premature triggering of induced responses.

[0056] Preprocess the collected multi-dimensional availability status parameter information of interface components and extract key indicators reflecting potential asynchronous triggering risks from the preprocessed data through feature engineering methods;

[0057] Preprocessing the collected multi-dimensional availability status parameter information of interface components means that after completing real-time collection, the system performs a series of data cleaning and standardization operations on these raw status data (such as component loading delay, mounting status flag, response delay, animation execution progress, etc.) to improve its quality and availability. Specifically, it includes removing outliers, filling missing data, smoothing time series fluctuations, unifying data formats, and normalization processing. The core role of this preprocessing process is to eliminate noise interference and systematic errors in the data, making subsequent feature extraction, risk modeling, and intelligent judgment more accurate and stable, thereby ensuring that the asynchronous trigger risk assessment model has good robustness and generalization capabilities in high-dimensional parameter space.

[0058] After preprocessing the interface component state data, the system further applies feature engineering techniques to identify and extract core features that are sensitive to the risk of asynchrony between induced responses and interface loading. These features serve as "information variables" for model training and evaluation. In this scenario, they may include: component rendering completion timestamp, the time interval between voice triggering and component response (latency difference), loading rate gradient, component state fluctuation frequency, layer activation duration, and first interactivity time. Common feature engineering methods include statistical feature extraction (mean, variance, maximum and minimum values), time series analysis (delayed series differencing, moving average), signal smoothing and trend extraction (such as sliding window filtering and weighted averaging of reference values), principal component analysis (PCA) for dimensionality reduction and extraction of global trend features, and tree-based feature importance ranking to select the most predictive risk indicators. Through these methods, the system constructs a set of key features with high relevance, low redundancy, and strong interpretability for subsequent intelligent assessment of asynchronous triggering risks.

[0059] Conduct comprehensive analysis on the extracted key indicators to quantify potential asynchronous trigger risks;

[0060] Through feature engineering methods, key indicators reflecting potential asynchronous triggering risks are extracted from the preprocessed data. The extracted indicators include the frequency of monitoring component status switching from "loaded" → "unloaded" → "loaded" multiple times in a short period of time and the fluctuation range of the response delay of the interface interaction component in different time periods. After comprehensive analysis within the detection window, the frequency of monitoring component status switching from "loaded" → "unloaded" → "loaded" multiple times in a short period of time and the fluctuation range of the response delay of the interface interaction component in different time periods are generated to generate a state rebound reference value and a response fuzzy reference value, respectively. The potential asynchronous triggering risk is quantified by the state rebound reference value and the response fuzzy reference value.

[0061] Quantifying potential asynchronous triggering risks using state bounce reference values ​​and response fuzziness reference values ​​enables highly accurate identification of UI components in the "apparently available but unstable" state, providing a more robust risk assessment basis for induction strategies. The state bounce reference value characterizes the instability of components frequently switching between "loaded" and "unloaded" within a short period of time, reflecting volatility or abnormal refresh behavior in the system's rendering path. The response fuzziness reference value captures the fluctuations in component response latency over different time periods, measuring the consistency and reliability of its interactive responses. When these two reference values ​​rise significantly within a certain threshold, it indicates that while the component appears to be rendered, its state exhibits potential timing drift and unpredictable interactions. This can easily lead to the inducing component triggering at the wrong time, pointing to invalid areas, or even failing to trigger. Therefore, quantifying asynchronous risk using these two reference values ​​not only enhances the system's awareness of potential misaligned inductions but also provides a calculable and traceable risk foundation for subsequent response rhythm self-calibration and dynamic policy adjustment.

[0062] When it is detected that the component status switches frequently from "loaded" → "not loaded" → "loaded" multiple times in a short period of time, it does indicate that there is a potential risk of asynchronous triggering of the induced response. This phenomenon usually reflects that there are problems such as rendering jitter, unstable DOM mounting, or repeated reconstruction of the view hierarchy during the loading process of the component, indicating that the system's recognition of the availability of the component is in an uncertain state. In this state, the system may mistakenly believe that the component is ready and trigger the induced component (such as a highlight box, a guide arrow) in advance, but in fact the target component is still in a non-interactive state, the content is not stably rendered, or it is temporarily uninstalled, resulting in induced dislocation, guidance failure, or even misoperation. Especially in fast multi-round voice-driven interactions, the risk of the induced response rhythm being out of sync with the component status will be further amplified.

[0063] The specific steps for generating a status rebound reference value by comprehensively analyzing the frequency of component status switching from "loaded" to "unloaded" to "loaded" multiple times in a short period of time in the detection window are as follows:

[0064] Within the detection window, the loading state of the target interface component is continuously sampled, where the state "loaded" is marked with a value of 1 and the state "unloaded" is marked with a value of 0. When the component state is detected to switch from "loaded → unloaded → loaded" in a short period of time, it is identified as a rebound event. The number of all rebound events within the detection window is counted, and the loading state recovery time span corresponding to each rebound event is recorded. The state switching density factor is constructed using the following formula:

[0065]

[0066] Where N r is the total number of rebound events, which indicates the total number of occurrences of the “loaded → unloaded → loaded” state switching sequence detected, D i is the duration of the i-th rebound event, which represents the number of consecutive sampling frames, time segments, or event cycles from the time the component state first enters the "loaded" state to the time it returns to the "loaded" state. α is the density amplification factor, which is used to control the weight of each rebound event in the overall density factor and is used to adjust the sensitivity to short-term frequent rebound behaviors. Its value range is 1-3. Λ is the state switching density factor. A high Λ value indicates that the state switching is highly intensive and concentrated, which is a strong warning signal for potential asynchronous induced response error triggering.

[0067] By constructing a state switching density factor, we can accurately quantify the structural concentration of repeated fluctuations in the loading state of interface components within the detection window, thereby capturing potential short-term and high-frequency unstable behaviors.

[0068] After obtaining the state switching density factor Λ, the abnormal recognition ability of short-term high-frequency rebound behavior is improved. A bilateral normalized nonlinear adjustment function is introduced to transform the density factor Λ to generate a state rebound reference value. The specific calculation method is as follows:

[0069] SRRV=tanh(·Λ)

[0070] , where SRRV is the state rebound reference value, δ is the state response amplification coefficient, which is used to adjust the response amplitude of the state switching density factor Λ and control the amplification degree before nonlinear transformation. Its value range is 1-10, and tanh(·) is the hyperbolic tangent function, which can compress the output to the interval (-1, 1), suppress extreme value interference, and realize nonlinear amplification of the response value in the high-risk section.

[0071] Tanh, the hyperbolic tangent function, is a commonly used nonlinear activation function with an output range of (-1, 1) and a smooth, S-shaped curve. When the input value is small, the output approaches 0; as the input value increases, the output rapidly approaches 1; and when the input is negative, the output approaches -1. The tanh function serves as a nonlinear adjustment function in the state rebound reference value formula. Its core function is to normalize and compress the state switching density factor and enhance risk sensitivity. Specifically, it can suppress the interference of extreme outliers on the system and prevent excessive exponential growth from leading to policy misjudgments. At the same time, it retains the ability to nonlinearly amplify responses to dense rebound areas. This allows the exponential output to rise rapidly when the rebound density is high, allowing the system to promptly perceive potential asynchronous risks. Compared to linear functions, the tanh function has a stronger response gradient and risk warning capabilities when approaching critical states, making it more suitable for describing the boundary effects of potential sudden instabilities in induced responses.

[0072] This regulation mechanism can ensure that even if the total number of rebound events is small, if the rebound behavior occurs intensively in a short period of time, a highly sensitive amplification response can still be presented in the value.

[0073] A higher state bounce reference value, generated by comprehensively analyzing the frequency of component status changes from "loaded" to "not loaded" to "loaded" within a short period of time within the monitoring window, can be used as a key indicator of potential asynchronous triggering risks for induced responses. The state bounce reference value quantifies the frequency of repeated state changes from "loaded" to "not loaded" to "loaded" within the monitoring window, reflecting significant state instability during the rendering process. A high reference value indicates that while the component may appear "loaded" at a certain point in time, its rendering state is extremely unstable, potentially affected by resource jitter, hierarchy reconstruction, or mounting delays. In this case, triggering the induced component based on a "presumed loaded" state is prone to asynchronous issues such as misalignment, empty triggering, or misdirection. Therefore, an increase in the state bounce reference value should be considered a signal that the interface is not yet fully ready, providing a clear warning of potential asynchronous risks. Conversely, a low state bounce reference value indicates a stable component loading process and reliable, continuous state, allowing the system to safely trigger the induced response.

[0074] When the response delay of the interface interaction component fluctuates frequently in different time periods, and the fluctuation period is extremely short, it usually indicates that the system currently has strong operational instability or rendering resource competition, which will directly lead to the potential risk of asynchronous triggering of the induced response. From the perspective of the induced response mechanism, its core relies on the interface component being in a "stable and interactive" state before being highlighted, guided or prompted, and the drastic fluctuation of the response delay means that although the surface rendering of the component is completed at certain moments, the actual interaction processing path may not be fully ready or is in a temporarily blocked state. At this time, if the induced component is triggered in advance based on surface visibility, it is very easy to cause the user's click to be unresponsive, trigger delays, or misjudgment of command execution failure, further causing confusion in interface operations or repeated input.

[0075] The specific steps for generating a response fuzzy reference value after comprehensively analyzing the response delay fluctuation amplitude of the interface interaction component in different time periods within the detection window are as follows:

[0076] In the detection window, the continuous response events of the target interface interaction component are sampled in real time to obtain the response delay value sequence D of adjacent response events, where D = {d j}={d1, d2, d3, ... d n}, where d jis the response delay of the jth response event, which means the time it takes for the interface component to generate interface feedback after the user initiates the jth interaction (such as click, voice command, gesture sliding, etc.). n is the total number of response events. For two adjacent response events, the response delay change gradient value is calculated. The calculation expression is as follows:

[0077] Δ j =tanh(λ·(d j+1 -d j ))

[0078] , where d j+1 is the response delay of the j+1th response event, that is, the response delay of the next response event. λ is the disturbance sensitivity adjustment coefficient, which is used to control the sensitivity of the system to the response delay difference and adjust the reaction speed of the delay gradient to extreme values, so that the model can not only identify small disturbances but also suppress the excessive influence of occasional spikes on the overall judgment. The value range is 0.1-1. tanh is the hyperbolic tangent function. Δ j is the gradient value of the response delay change, which indicates the change trend and severity of the response delay between the jth and j+1th user interaction events;

[0079] In the formula, tanh (hyperbolic tangent function) plays the role of smoothing, compressing and nonlinearly mapping the difference between two adjacent response delays, thereby enhancing the system's sensitivity to subtle fluctuations and suppressing the interference of extreme outliers on the overall judgment results. Since the output value of tanh is always limited to the interval (-1,1), its S-shaped curve shape can maintain a linear response when the delay difference is small, and gradually tends to saturation when the delay difference is large, effectively preventing drastic fluctuations in response in a short period of time (such as sudden freezes or too fast responses) from causing indicator deviations. At the same time, the tanh function retains the positive and negative signs of the input data, thereby retaining the directionality of the response trend (whether it is getting faster or slower), which is critical for the subsequent identification of whether the induced response is synchronized with the interface loading state. Therefore, when constructing the response fuzzy index, the introduction of tanh not only improves numerical stability, but also improves the ability to model nonlinear system behavior. It is an important balancing mechanism for control sensitivity and robustness.

[0080] After obtaining all response delay change gradient values, nonlinear amplitude amplification and weighted summation are performed on each response delay change gradient value to obtain the final response fuzzy reference value. The calculation formula is as follows:

[0081]

[0082] Where RBRV is the response fuzzy reference value, β is the trend amplification factor, which is used to amplify the original fuzzy reference value to an interpretable range so that it can be used by downstream models. It is usually 10, 50, or 100. γ is the perturbation amplitude power adjustment factor, which is used to nonlinearly amplify the amplitude of the offset gradient. It ranges from 1.2 to 2.5.

[0083] The larger the response fuzzy reference value RBRV, the worse the system response stability and the more intense the fluctuation, and the higher the possibility of asynchronous risk between the triggering timing of the induction strategy and the interface interaction state. Conversely, it can be considered that the interface interaction state is relatively stable and suitable for synchronous induction control.

[0084] A larger response fuzzy reference value, generated by comprehensively analyzing the fluctuations in the response delay of interface interaction components over different time periods within the monitoring window, indicates a potential asynchronous triggering risk for the induced response. Conversely, a smaller reference value indicates a safer and more controllable induced response. The response fuzzy reference value is derived by analyzing the fluctuations in the response delay of interface interaction components over different time periods within the monitoring window. It reflects the consistency and stability of the system's interactive performance. A high reference value indicates that the component's response time fluctuates frequently within a short period of time, exhibiting significant uncertainty or transient performance jitter. In this state, the system struggles to accurately determine whether the component is truly interactive. If the induced component triggers prematurely, problems such as misdirection, unresponsiveness, or interface failure may occur, posing a typical asynchronous triggering risk. Conversely, a low response fuzzy reference value indicates that the component's response delay is stable and fluctuates minimally, making the system's interactive judgment more reliable. Based on this, the induced response triggers with greater synchronization and security.

[0085] The key indicators after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk status annotation data. The model then intelligently evaluates the rendering status of the current target interface component to determine whether there is an asynchronous trigger risk.

[0086] The state rebound reference value and response fuzzy reference value after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk state annotation data. The asynchronous trigger risk coefficient is generated by the model, and the rendering state of the current target interface component is intelligently evaluated based on the asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk.

[0087] "A machine learning model pre-trained based on historical interaction data and asynchronous risk state annotation data" refers to the construction of an intelligent model capable of identifying asynchronous trigger risks through offline training on a large amount of historical interaction sample data before the system is deployed. During this training process, the system first accumulates a large number of interaction data samples related to interface induction, including user voice triggering timing, changes in the rendering state of interface components, sequences of usability status parameters, and corresponding system response behavior records. Furthermore, to build a high-quality training set, labels generated manually or by a rule engine are introduced to classify and annotate each interaction as exhibiting an "asynchronous trigger risk" state (e.g., "risk present" or "no risk"), where the induction strategy completes before the interface rendering. Based on this, the system extracts multi-dimensional input features such as the state rebound index, response fuzziness index, and the time offset between voice triggering and response through feature engineering. These features are then fed into a supervised machine learning algorithm, such as random forest, gradient boosting tree (XGBoost), support vector machine (SVM), or lightweight neural network (MLP) for model training. The result is a model that can learn the nonlinear mapping relationship between input features and risk states.

[0088] After training, the model has the ability to intelligently evaluate unknown input data. That is, when the system obtains characteristic values ​​such as the state rebound index and response fuzziness index extracted from a round of interaction during actual operation, it can quickly pass these input features into the trained model for inference, and output a continuous value or probability value as the asynchronous trigger risk coefficient indicating whether there is a risk of asynchronous triggering. This coefficient can be used to determine whether the induction component should be triggered late, or whether the system needs to adjust the animation rhythm, reconfirm the interface loading status, and other strategies to ensure a high degree of synchronization between the induction logic and the display logic, thereby effectively avoiding misleading guidance, interaction interruption, or user error caused by incorrect timing judgment. Through this machine learning model, the system has transformed from "rule-based static control" to "data-driven dynamic decision-making", realizing the intelligent, adaptive, and continuous optimization of the induction rhythm control, which helps to improve the induction accuracy and user experience consistency in complex interactive environments.

[0089] The machine learning model is not limited here. Any machine learning model that can perform a comprehensive analysis of the state rebound reference value SRRV and the response fuzzy reference value RBRV to generate the asynchronous trigger risk coefficient ATRC is acceptable. To implement the technical solution of the present invention, the present invention provides a specific implementation method:

[0090] The asynchronous trigger risk coefficient ATRC is generated by the following formula: ATRC = k1·SRRV+k2·RBRV, where k1 and k2 are the preset proportional coefficients of the state rebound reference value SRRV and the response fuzzy reference value RBRV, respectively, and both k1 and k2 are greater than 0.

[0091] "Preset proportional coefficient" refers to the weighting factors set by the system for the two core reference values ​​- the state rebound reference value SRRV and the response fuzzy reference value RBRV - when generating the asynchronous trigger risk coefficient ATRC, namely k1 and k2 in the formula. The role of these two proportional coefficients is to adjust the relative importance of the state rebound reference value SRRV and the response fuzzy reference value RBRV in the overall risk assessment. Specifically, k1 reflects the contribution of state instability to asynchronous risk, while k2 reflects the impact of interactive response volatility on risk. Since the importance of these two dimensions may vary in different systems and scenarios, by pre-setting (or model training optimization) these two coefficients, it is possible to more flexibly and accurately reflect which risk factor dominates the current environment, thereby making the final generated ATRC value more discriminative and adaptable. In short, the preset proportional coefficient is a key parameter for realizing weighted analysis and risk proportion control in risk fusion modeling.

[0092] It can be seen from the asynchronous trigger risk coefficient that the larger the state rebound reference value generated after comprehensive analysis of the frequency of switching the monitoring component state from "loaded" → "unloaded" → "loaded" multiple times in a short period of time under the detection window, and the larger the response fuzzy reference value generated after comprehensive analysis of the response delay fluctuation amplitude of the interface interaction component in different time periods under the detection window, the larger the asynchronous trigger risk coefficient generated when the rendering state of the current target interface component is intelligently evaluated by the machine learning model trained based on historical interaction data and asynchronous risk state annotation data, indicating a greater probability of asynchronous trigger risk, and vice versa.

[0093] The asynchronous trigger risk coefficient generated by intelligently evaluating the rendering state of the current target interface component using a machine learning model trained based on historical interaction data and asynchronous risk status annotation data is compared with the pre-set asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk. The judgment logic is as follows:

[0094] If the asynchronous trigger risk coefficient is greater than the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is an asynchronous trigger risk; if the asynchronous trigger risk coefficient is less than or equal to the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is no asynchronous trigger risk.

[0095] When the assessment indicates the risk of asynchronous triggering, a prediction model based on a temporal convolutional network (TCN) is used to model the temporal correlation between the user's voice triggering behavior sequence and the interface component loading delay. This model then integrates historical interaction sequences with the current system load situation to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and animation duration of the induced component are dynamically adjusted to achieve self-calibration of the response rhythm, thereby improving system response synchronization and user operation accuracy during multimodal interaction.

[0096] After the system identifies the asynchronous risk that the inducing component may be triggered before the interface loads, it uses a temporal convolutional network (TCN) to construct a temporal mapping relationship between voice behavior triggering and interface loading delay, and dynamically adjusts the response strategy based on the prediction results to achieve precise adaptive adjustment of the system response rhythm. Specifically, the TCN model can effectively capture the changes in the behavioral characteristics of user voice commands in the temporal dimension, and combined with the delay pattern between voice and interface rendering during historical interactions, it predicts the time required for interface components to reach a stable and usable state under the current system load status (such as CPU utilization, thread resource occupancy, and network bandwidth fluctuations). Based on this, the system generates a reasonable trigger window range and dynamically adjusts the delayed startup time and animation presentation rhythm of the inducing component to ensure that the inducing element is not displayed until the interface is truly visible and interactive, thereby significantly improving the synchronization between voice commands and visual feedback, and preventing users from being misled when the interface is not ready. At the same time, based on model prediction rather than static delay setting, it can achieve adaptive processing of different device performance and scene changes, effectively reduce the interactive interference caused by problems such as induced dislocation, empty guidance or visual frame skipping, comprehensively enhance the system's multimodal coordination capabilities and user operation accuracy, and build a highly robust and intelligent dynamic rhythm control mechanism for complex human-computer interaction systems.

[0097] When the assessment indicates the risk of asynchronous triggering, a prediction model based on a temporal convolutional network is used to model the temporal correlation between the user's voice triggering behavior sequence and the interface component loading delay. This model then integrates historical interaction sequences with the current system load situation to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and animation duration of the induced component are dynamically adjusted. The specific steps for achieving self-calibration of the response rhythm are as follows:

[0098] After determining that there is an asynchronous trigger risk, that is, the asynchronous trigger risk coefficient ATRC is greater than the asynchronous trigger risk coefficient reference threshold, the prediction mechanism based on the time convolutional network is activated to construct a temporal relationship mapping between the user voice trigger behavior sequence and the interface loading history state sequence. The formula is as follows:

[0099] Y load =TCN(X voice||X load ||X context )

[0100] , where Y load is the predicted output, which indicates the window interval length required for the interface component to reach a stable and responsive state. voice It is the user voice trigger behavior sequence (such as semantic intensity, instruction continuity, trigger frequency coding), which describes the behavior sequence information in the process of the user issuing voice commands. load It is the state sequence of the interface loading process (such as the latest rendering delay gradient, component activation progress change sequence), which is used to record the loading state change trajectory of the target interface component in the recent period of time. context It is the current system load situation vector (such as CPU load level, memory availability encoding, GPU occupancy distribution), which is used to describe the context information vector of the current device or system operating status. TCN is a temporal convolutional network model, and || is a sequence concatenation operation that concatenates multiple input sequences according to the time dimension or feature dimension to form a unified input tensor.

[0101] The Temporal Convolutional Network (TCN) model in this solution is used to construct temporal dependencies between user voice-triggered behavior and interface component loading dynamics, and to predict the time window required for components to reach a stable, responsive state. By integrating voice behavior sequences, component loading state sequences, and the current system load situation, TCN is able to capture the long-term dependencies and local dynamic changes between multimodal inputs while maintaining the integrity of the temporal structure. Its architecture, which utilizes dilated convolutions and residual connections, boasts excellent contextual memory and computational parallelism, making it particularly well-suited for modeling the complex time-delay mapping between semantic triggers and interface responses in interactive systems. Leveraging the prediction window output by the TCN, the system can dynamically regulate the triggering rhythm of induced components, avoiding misaligned, misdirected, or premature responses caused by asynchronous loading, thereby achieving more precise and intelligent human-computer interaction control strategies.

[0102] The role of the sequence splicing operation (usually denoted as ||) in this solution is to integrate multimodal temporal inputs from different sources - including voice trigger behavior sequences, interface component loading state sequences, and system load situation vectors - in the time dimension or feature dimension, and construct a multi-channel input tensor with consistent structure for unified modeling and processing by the temporal convolutional network (TCN). This operation ensures that various types of heterogeneous information can be considered synchronously under the same time index, thereby preserving the relative position and time correspondence between the features of each modality, and effectively supporting the TCN model to capture cross-modal temporal dependencies and contextual association features. Through sequence splicing, the system can not only model the user's intention expression (voice) and component availability status in a linked manner, but also introduce current environmental factors (system load) as contextual adjustment input to achieve accurate prediction of interface response time and dynamic regulation of induced rhythm.

[0103] This step uses TCN to model the potential dependency structure between the user's voice intent behavior and the loading path, and predicts the window required for the system to complete the "from visual to interactive" process.

[0104] Root test prediction output Y load , combined with the deviation between the current asynchronous trigger risk coefficient ATRC and the preset asynchronous trigger risk coefficient reference threshold, the trigger delay window of the final induced component is determined. The trigger delay window is calculated as follows:

[0105]

[0106] , where Δ induce It is the trigger delay window of the induction component, which indicates how long the system should delay the presentation of the induction component (such as highlight box, guide arrow, prompt) after detecting the user's intention. It is used to dynamically adjust the output timing of the induction strategy to ensure that the interface component has been rendered and the state is stable before starting the induction, avoiding the asynchronous triggering risk of "induction component loading earlier than content loading". th is the reference threshold of the asynchronous trigger risk factor, and θ is the risk sensitivity adjustment coefficient, which is used to adjust the impact of risk intensity on the delay window expansion, with a value range of 0-2;

[0107] Based on the deviation between the predicted loading completion time and the current asynchronous trigger risk factor, the trigger delay window of the inducement component is dynamically adjusted to synchronize the inducement timing with the interface's available state. This mechanism effectively prevents the inducement component from triggering prematurely before the interface has stabilized and loaded, thereby reducing the risk of misleading users and improving the reliability of system responses.

[0108] After the delay window calculation is completed, the trigger delay window Δ based on the induced component induce Adjust the animation rendering parameters of the induced component and calculate the rendering duration of the induced animation. The calculation expression is as follows:

[0109]

[0110] Where, Ω anim It is the duration of the induced animation presentation, which indicates the animation execution time required for the induced component during the display process. It is used to control dynamic presentation processes such as fade-in effects, prompt track drawing, highlight diffusion, arrow sliding, etc. It is a rhythm adjustment factor, which is used to control the response amplitude amplification relationship between the delay window and the animation rhythm. The value range is 0.1-1. ω is the minimum basic presentation duration, which is used to ensure that the presentation duration of the induced component animation is not less than the benchmark value under any circumstances. The value range is 150-500, and the unit is milliseconds.

[0111] The above-mentioned adjustment mechanism can ensure that the induction prompts are presented in a progressive manner when the risk is high, thereby improving the user's perceived consistency of the guidance path and the accuracy of operation, thereby achieving dynamic self-calibration of the system's induction response rhythm.

[0112] The present invention introduces a real-time perception mechanism for the rendering state of interface components and multi-dimensional usability state parameter modeling, supplemented by key risk indicators extracted by feature engineering and a risk identification model driven by machine learning, to effectively identify and predict asynchronous risks that may exist when inducing components are triggered; further combining the temporal convolutional network (TCN) to predict the temporal coupling relationship between user voice triggering and interface response, and achieve dynamic adaptive adjustment of the inducing trigger window and animation rhythm. This solution not only avoids problems such as inducing components mistakenly pointing to blank areas or misplaced highlights, but also significantly improves the system's response accuracy, logical coherence, and operational safety during human-computer interaction. It is particularly suitable for scenarios such as in-vehicle systems, medical terminals, and industrial control that require extremely high interaction accuracy.

[0113] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0114] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

[0115] It should be noted that, in this document, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0116] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0117] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0121] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0122] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. A multi-modal display screen response method integrating voice interaction, characterized in that: The following steps are involved: During user interaction, the real-time status monitoring mechanism of the interface components is used to call built-in sensors to continuously monitor the rendering process of the target interface components at millisecond-level time granularity and collect multi-dimensional availability status parameter information of the interface components; Preprocess the collected multi-dimensional availability status parameter information of interface components and extract key indicators reflecting potential asynchronous triggering risks from the preprocessed data through feature engineering methods; Conduct comprehensive analysis on the extracted key indicators to quantify potential asynchronous trigger risks; The key indicators after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk status annotation data. The model then intelligently evaluates the rendering status of the current target interface component to determine whether there is an asynchronous trigger risk. When the assessment shows that there is a risk of asynchronous triggering, the temporal correlation between the user voice triggering behavior sequence and the interface component loading delay is modeled through a prediction model based on a time convolutional network. The historical interaction sequence and the current system load situation are integrated to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and presentation animation duration of the induced component are dynamically adjusted to achieve self-calibration of the response rhythm.

2. The multimodal induction display screen response method integrating voice interaction according to claim 1 is characterized in that: During user interaction, the specific process of the real-time status monitoring mechanism of the interface components is as follows: After initializing the interactive session, start the interface status monitoring module and bind the target interface component node to be monitored; By calling the built-in system sensors or the rendering hook interface of the UI framework, low-level events related to the component rendering are captured in real time; Continuously poll or monitor these event states at millisecond-level time granularity to build a multi-dimensional state parameter sequence that reflects component availability; The collected raw state data is structured and encapsulated as the basic input for subsequent asynchronous risk assessment and induction strategy control, achieving high-precision dynamic perception of component rendering status.

3. The multimodal induction display screen response method integrating voice interaction according to claim 1 is characterized in that: Through feature engineering methods, key indicators reflecting potential asynchronous triggering risks are extracted from the preprocessed data. The extracted indicators include the frequency of monitoring component status switching from "loaded" → "unloaded" → "loaded" multiple times in a short period of time and the fluctuation range of the response delay of the interface interaction component in different time periods. After comprehensive analysis within the detection window, these indicators are used to generate state rebound reference values ​​and response fuzzy reference values, respectively. The state rebound reference value and response fuzzy reference value are used to quantify the potential asynchronous triggering risk.

4. The multimodal induction display screen response method integrating voice interaction according to claim 3 is characterized in that: The specific steps for generating a status rebound reference value by comprehensively analyzing the frequency of component status switching from "loaded" to "unloaded" to "loaded" multiple times in a short period of time in the detection window are as follows: Within the detection window, the loading state of the target interface component is continuously sampled, where the state "loaded" is marked with a value of 1, and the state "unloaded" is marked with a value of 0. When the component state is detected to switch from "loaded → unloaded → loaded" in a short period of time, it is identified as a rebound event. The number of all rebound events within the detection window is counted, and the loading state recovery time span corresponding to each rebound event is recorded. The state switching density factor is constructed using the following formula: ， Where N r is the total number of rebound events, D i is the duration of the i-th rebound event, α is the density amplification factor, and Λ is the state switching density factor; After obtaining the state switching density factor Λ, the abnormal recognition ability of short-term high-frequency rebound behavior is improved. A bilateral normalized nonlinear adjustment function is introduced to transform the density factor Λ to generate a state rebound reference value. The specific calculation method is as follows: SRRV=tanh(·Λ), Where SRRV is the state rebound reference value, δ is the state response amplification factor, and tanh(·) is the hyperbolic tangent function.

5. The multi-modal induction display screen response method integrating voice interaction according to claim 3 is characterized in that: The specific steps for generating a response fuzzy reference value after comprehensively analyzing the response delay fluctuation amplitude of the interface interaction component in different time periods within the detection window are as follows: In the detection window, the continuous response events of the target interface interaction component are sampled in real time to obtain the response delay value sequence D of adjacent response events, where D = {d j }={d1, d2, d3, ... d n }, where d j is the response delay of the jth response event, n is the total number of response events, and for two adjacent response events, the response delay change gradient value is calculated. The calculation expression is as follows: Δ j =tanh(λ·(d j+1 -d j )), Where, d j+1 is the response delay of the j+1th response event, that is, the response delay of the next response event, λ is the disturbance sensitivity adjustment coefficient, tanh is the hyperbolic tangent function, Δ j is the response delay change gradient value; After obtaining all response delay change gradient values, nonlinear amplitude amplification and weighted summation are performed on each response delay change gradient value to obtain the final response fuzzy reference value. The calculation formula is as follows: ， Where RBRV is the response fuzzy reference value, β is the trend amplification coefficient, and γ is the disturbance amplitude power adjustment factor.

6. The multi-modal induction display screen response method integrating voice interaction according to claim 3 is characterized in that: The state rebound reference value and response fuzzy reference value after quantitative analysis are input into a machine learning model that has been pre-trained based on historical interaction data and asynchronous risk state annotation data. The asynchronous trigger risk coefficient is generated by the model, and the rendering state of the current target interface component is intelligently evaluated based on the asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk.

7. The multimodal induction display screen response method integrating voice interaction according to claim 6 is characterized in that: The asynchronous trigger risk coefficient generated by intelligently evaluating the rendering state of the current target interface component using a machine learning model trained based on historical interaction data and asynchronous risk status annotation data is compared with the pre-set asynchronous trigger risk coefficient to determine whether there is an asynchronous trigger risk. The judgment logic is as follows: If the asynchronous trigger risk coefficient is greater than the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is an asynchronous trigger risk; if the asynchronous trigger risk coefficient is less than or equal to the preset asynchronous trigger risk coefficient reference threshold, it is judged that there is no asynchronous trigger risk.

8. The multimodal induction display screen response method integrating voice interaction according to claim 7 is characterized in that: When the assessment indicates the risk of asynchronous triggering, a prediction model based on a temporal convolutional network is used to model the temporal correlation between the user's voice triggering behavior sequence and the interface component loading delay. This model then integrates historical interaction sequences with the current system load situation to predict the time interval required for the interface to enter a stable and responsive state. Based on this, the trigger delay window and animation duration of the induced component are dynamically adjusted. The specific steps for achieving self-calibration of the response rhythm are as follows: After determining that there is an asynchronous trigger risk, that is, the asynchronous trigger risk coefficient ATRC is greater than the asynchronous trigger risk coefficient reference threshold, the prediction mechanism based on the time convolutional network is activated to construct a temporal relationship mapping between the user voice trigger behavior sequence and the interface loading history state sequence. The formula is as follows: Y load =BC(X voice ||X load ||X context ), Where Y load is the predicted output, X voice is the user voice-triggered behavior sequence, X load is the state sequence of the interface loading process, X context is the current system load situation vector, TCN is the temporal convolutional network model, and || is the sequence concatenation operation; Root test prediction output Y load , combined with the deviation between the current asynchronous trigger risk coefficient ATRC and the preset asynchronous trigger risk coefficient reference threshold, the trigger delay window of the final induced component is determined. The trigger delay window is calculated as follows: ， Where, Δ induce is the trigger delay window of the inducing component, ATRC th is the reference threshold of the asynchronous trigger risk coefficient, and θ is the risk sensitivity adjustment coefficient; After the delay window calculation is completed, the trigger delay window Δ based on the induced component induce Adjust the animation rendering parameters of the induced component and calculate the rendering duration of the induced animation. The calculation expression is as follows: ， Where, Ω anim is the duration of the induced animation. is a rhythm adjustment factor used to control the response amplitude amplification relationship between the delay window and the animation rhythm, and ω is the minimum basic presentation duration.

Images