Conversational AI low-delay response control method and system based on semantic analysis

Abstract

The invention relates to the technical field of voice interaction, in particular to a dialogue type AI low-delay response control method and system based on semantic analysis. The method comprises the following steps: firstly, generating a driving intensity correlation factor of a current time window according to the change of CAN bus data; further monitoring the output of the NLU, analyzing the acoustic intonation raising characteristics, and obtaining an interaction suppression coefficient in combination with the driving intensity correlation factor; further acquiring a time pressure coefficient based on the mute duration after the user stops sounding; further comparing the interaction suppression coefficient and the time pressure coefficient of the current time window to obtain a response judgment value; further analyzing the change trend of the interaction inhibition coefficient, and updating an environment improvement flag bit; and finally, according to the driving intensity correlation factor, the environment improvement flag bit and the response decision value of the current time window, carrying out multi-level response decision control, and carrying out selective reset of an interaction state, thereby solving the problems of wrong truncation and response delay of voice interaction under dynamic driving.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, specifically to a conversational AI low-latency response control method and system based on semantic analysis. Background Technology

[0002] The timing of response in in-vehicle voice interaction systems is crucial for the smart cockpit experience. Existing technologies typically employ endpoint detection (VAD) strategies based on fixed thresholds to determine the end of user commands, such as setting a fixed silence duration threshold; a response is triggered when the pause exceeds this threshold. However, this static mechanism struggles to adapt to dynamically changing driving scenarios. In actual driving, drivers often experience involuntary pauses or slowed speech due to maneuvers such as turning, changing lanes, or swerving. A fixed short threshold can easily misinterpret these pauses as command termination, causing the system to "intercept" or interrupt the driver. Conversely, setting the threshold too long to accommodate such pauses can lead to sluggish responses under stable conditions, reducing interaction efficiency.

[0003] Although some solutions attempt to introduce vehicle speed or gear information to simply switch interaction modes, they often lack detailed modeling of the continuous changes in driving load, cannot accurately perceive the lag effect of attention release after driving actions, and are difficult to establish a dynamic balance between "risk of accidental interruption" and "user waiting anxiety," resulting in the interaction strategy being difficult to synchronize with the driver's actual state. Summary of the Invention

[0004] To address the technical problems of accidental truncation in voice interaction under complex driving conditions and sluggish response under stable conditions, the present invention aims to provide a conversational AI low-latency response control method and system based on semantic analysis. The specific technical solution adopted is as follows: A low-latency response control method for conversational AI based on semantic analysis, the method comprising: After the system restarts, it initializes, acquires the vehicle's CAN bus data, and divides it into preset time windows; based on the changes in the CAN bus data, it acquires the vehicle handling intensity of the current time window; and generates a driving intensity correlation factor for the current time window based on the vehicle handling intensity. After detecting the first voice signal, the NLU output is monitored, and the acoustic pitch rise feature is analyzed to obtain the instruction incompleteness degree of the current time window; the instruction incompleteness degree and the driving intensity correlation factor are fused to obtain the interaction suppression coefficient; the time pressure coefficient is obtained based on the silence duration after the user stops speaking; the interaction suppression coefficient and the time pressure coefficient of the current time window are compared to obtain the response judgment value; the changing trend of the interaction suppression coefficient is analyzed to update the environment improvement flag; Based on the driving intensity correlation factor, the environmental improvement flag, and the response judgment value of the current time window, multi-level response decision control is performed, and the interactive state is selectively reset.

[0005] Furthermore, the method for obtaining the vehicle handling strength includes: The CAN bus data includes at least longitudinal acceleration, steering wheel angle, and real-time vehicle speed; For the current time window, a first control intensity is obtained based on the rate of change of longitudinal acceleration and the rate of change of steering wheel angle between adjacent time windows; a second control intensity is obtained by fusing the absolute value of the steering wheel angle and the real-time vehicle speed. The vehicle control strength is obtained by combining the first control strength and the second control strength.

[0006] Furthermore, the method for obtaining the incompleteness of the instruction includes: Monitor the output of NLU to obtain semantic continuous probabilities; Extract the fundamental frequency sequence of speech data from the preset historical neighborhood of the current time window, and obtain the acoustic pitch rise based on the changing trend of the fundamental frequency sequence; By combining the semantic continuity probability and the acoustic intonation rise, the incompleteness of the instruction in the current time window is obtained.

[0007] Furthermore, the method for obtaining the semantic continuity probability includes: Monitor the NLU output. If the duration of output or stop output within the current time window does not exceed a preset first time limit, mark the semantic continuity probability as a preset first parameter. If the duration of stop output corresponding to the current time window exceeds the preset first time limit, obtain the semantic continuity probability based on the preset first parameter which is linearly decayed based on the duration exceeding the first time limit.

[0008] Furthermore, the method for obtaining the environmental improvement flag bit includes: The interaction inhibition coefficient of the current time window is compared with that of the preset historical period to obtain the trend gradient; the trend gradient is compared with the preset gradient improvement threshold to update the environmental improvement flag.

[0009] Furthermore, methods for multi-level response decision control include: If the driving intensity correlation factor of the current time window is greater than the preset high-risk circuit breaker threshold, the preset safety circuit breaker process is triggered. If the preset safety circuit breaker procedure is not triggered, the system enters a suspended state when the environment improvement flag is in a valid state. If the preset safety circuit breaker process is not triggered and the device is not in a suspended state, an endpoint detection completion signal is generated based on the response judgment value.

[0010] Furthermore, when in a suspended state, if the environment improvement flag of the current time window is invalid or the duration of the suspended state exceeds a preset maximum greedy threshold, the suspended state is terminated.

[0011] Furthermore, the method for performing the selective reset includes: The silence duration is reset to 0, while the driving intensity correlation factor is retained.

[0012] Furthermore, the method for obtaining the driving intensity correlation factor includes: The maximum value of the vehicle handling intensity in the previous time window and the preset attenuation coefficient, and the vehicle handling intensity in the current time window, is taken as the driving intensity correlation factor.

[0013] The present invention also proposes a semantic analysis-based conversational AI low-latency response control system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the steps of the semantic analysis-based conversational AI low-latency response control method.

[0014] The present invention has the following beneficial effects: This invention first generates a driving intensity correlation factor for the current time window based on changes in CAN bus data, characterizing the level of cognitive resource occupancy of the driver in current and recent driving actions. It further monitors the NLU output and analyzes the rising vocal tone characteristics to obtain the instruction incompleteness of the current time window, achieving complementary fusion of semantics and acoustics to improve the accuracy of endpoint judgment and the rationality of response timing in dynamic driving scenarios. It further integrates the instruction incompleteness and driving intensity correlation factor to obtain an interaction suppression coefficient, coupling driving safety and language understanding into a unified cost parameter, providing support for subsequent updates to the environment improvement flag. It further obtains a time pressure coefficient based on the silence duration after the user stops speaking. It further compares the interaction suppression coefficient and time pressure coefficient of the current time window to obtain a response judgment value, serving as a direct criterion for response triggering. It further analyzes the changing trend of the interaction suppression coefficient to update the environment improvement flag, providing more basis for subsequent decisions. Finally, based on the driving intensity correlation factor, environment improvement flag, and response judgment value of the current time window, it performs multi-level response decision control and selectively resets the interaction state. By integrating driving control and voice features, the system calculates the interruption and waiting costs under load modulation and incorporates trend-based multi-level control. It leverages the risk of interruption due to driving load modulation, weighs waiting costs against environmental changes, and achieves adaptive decision-making for response timing, thus resolving the issues of truncation and response delay in voice interaction under dynamic driving conditions. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a semantic analysis-based conversational AI low-latency response control method provided in one embodiment of the present invention; Figure 2 This is a flowchart of a method for obtaining the incompleteness of an instruction, provided as an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a semantic analysis-based conversational AI low-latency response control method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details the specific scheme of the conversational AI low-latency response control method and system based on semantic analysis provided by this invention.

[0020] Please see Figure 1 The diagram illustrates a flowchart of a semantic analysis-based conversational AI low-latency response control method according to an embodiment of the present invention, specifically including: Step S1: After the system restarts, initialize the system, acquire the vehicle's CAN bus data and divide it into preset time windows; based on the changes in the CAN bus data, acquire the vehicle handling intensity of the current time window; and generate the driving intensity correlation factor of the current time window based on the vehicle handling intensity.

[0021] Voice interaction needs to be synchronized with the driving state, which is a continuous time-varying process. A unified time reference must be established. At the same time, in order to ensure the effectiveness of the recursive calculation logic, the system must perform a strict initialization process after power-on or reset. Therefore, initialization is performed after the system restarts, the vehicle's CAN bus data is acquired and divided into preset time windows to provide a basis for subsequent analysis.

[0022] In one embodiment of the present invention, the preset time window length is 10ms, the time window index is k, and the system restart includes startup methods such as power-on startup or reset; initialization includes resetting the status register: setting the driving intensity association factor Set to zero to represent the duration of silence (interactive silence duration) after the user stops speaking. Set the time window index k to zero, and set the semantic continuous probability to zero. Set it to the default high bit (e.g., 1.0) to assume that the system is in a high semantic protection state in the initial state.

[0023] To perform sliding window filling, for calculations requiring historical data, a queue of length N is pre-established (N=5). This queue is used during the initial runtime (i.e., the cold start phase). Due to insufficient historical data, the system adopts a "zero-fill" or "copy-fill" strategy, that is, it assumes... All state values ​​at any given time are 0 or equal to the value of the time window k=0 to prevent computational anomalies.

[0024] It should be noted that the control method of this invention is mainly applied to voice interaction scenarios between the driver and the vehicle system. Before performing the interaction determination, the system will verify the identity of the speaker of the voice signal based on existing cabin sound source localization technology (Sound Zone Localization) or voiceprint recognition technology. Only when the speaker is identified as the driver (driver's seat) will the response control logic based on the driving intensity correlation factor be activated; if the speaker is a passenger, the conventional interaction logic will be used to avoid unnecessary interference with the passenger's normal experience.

[0025] Furthermore, considering that even with Advanced Driver Assistance Systems (ADAS) enabled, the vehicle's drastic movements (such as automatic obstacle avoidance and sharp cornering) can still trigger driver attention or a need for takeover, this embodiment does not exclude the possibility of the vehicle being in ADAS mode. This ensures consistent safety interaction and protection for the driver under any highly dynamic conditions. In other embodiments of the present invention, the implementer can adjust the length of the preset time window and the queue length N according to the specific implementation scenario, which will not be elaborated further.

[0026] The intensity of driving operations directly reflects the degree to which the driver's attention is occupied. In order to quantify the driver's current physical workload, the system collects CAN bus data (data from the chassis control system). Based on the changes in CAN bus data, the system obtains the vehicle control intensity of the current time window, quantifies the physical workload of the current driving task, and abstracts the underlying vehicle dynamics information into engineering features that can participate in interactive decision-making. Then, based on the vehicle control intensity, the system generates a driving intensity correlation factor for the current time window, which characterizes the level of cognitive resource occupation of the driver in the current and recent driving actions. This provides key input for the subsequent construction of interaction inhibition coefficients, the realization of interaction inhibition under high-risk conditions, and the ensuring of the continuity of state memory in continuous dialogue.

[0027] Preferably, in one embodiment of the present invention, the acceleration, deceleration and steering behaviors of the vehicle will cause the driver's attention to shift from the interactive task to the driving task (i.e., cognitive resources are occupied), reflecting the intensity of vehicle control. Therefore, the CAN bus data includes at least longitudinal acceleration, steering wheel angle and real-time vehicle speed. Considering that the actual road environment (such as speed bumps, gravel roads) may introduce high-frequency noise, the system first performs low-pass filtering on the raw data (for example, using a Butterworth filter with a cutoff frequency of 5Hz) to obtain a smoothed signal.

[0028] Considering that the greater the longitudinal acceleration change and steering wheel angle change between adjacent time windows, the more intense the instantaneous control action, and although the steering wheel angle may be smaller in high-speed scenarios, it results in a more sensitive vehicle attitude response (greater lateral centrifugal force) and higher operational risk; Based on this, for the current time window, the first control intensity is obtained according to the rate of change of longitudinal acceleration and the rate of change of steering wheel angle between adjacent time windows; the second control intensity is obtained by combining the absolute value of steering wheel angle and real-time vehicle speed. The vehicle control strength is obtained by combining the first control strength and the second control strength.

[0029] As an example, the current time window is compared with the adjacent previous time window. The longitudinal acceleration and steering wheel angle are taken as the dimensions to be analyzed. The absolute value of the difference between the data of the dimensions to be analyzed in the two windows is taken as the numerator, the time window length is taken as the denominator, and the ratio of the fractions is taken as the corresponding rate of change. The rate of change is standardized using a preset limiting constant, which involves dividing the real-time data by the limiting constant. In this example, the limiting constant for the rate of change of longitudinal acceleration is 0.8. (Based on the physical limits of acceleration for conventional vehicles, adjustments may be made based on specific vehicle models), the limiting constant corresponding to the rate of change of steering wheel angle is 500. (Based on the maximum statistical value or the physical limit speed of human hand turning the steering wheel).

[0030] The standardized rate of change of longitudinal acceleration and the rate of change of steering wheel angle are weighted and summed, with each weight being 0.5, corresponding to equal emphasis on longitudinal acceleration / deceleration and lateral steering operation. The weighted sum is used as the first control intensity. Compare the real-time vehicle speed with a preset reference vehicle speed constant (e.g., 120). The ratio of the steering wheel angle to the maximum steering wheel angle (e.g., 540°) is then added to a constant of 1, and the sum is used as a speed weighting factor. The ratio of the absolute values ​​of the two values ​​is used as the steering angle coefficient. The product of the steering angle coefficient and the vehicle speed weighting coefficient is linearly normalized, and the normalized result is used as the second handling intensity. The first and second control intensities are weighted and summed, with each weight being 0.5, corresponding to equal emphasis on the intensity of instantaneous operation and the risk of steady-state operation. The weighted sum is taken as the vehicle control intensity.

[0031] Among them, longitudinal acceleration refers to the acceleration data in the direction of vehicle travel; steering wheel angle refers to the angle data of the driver turning the steering wheel relative to the center position.

[0032] It should be noted that when there are multiple data points within a time window, if only one parameter is needed, the maximum value within the window is taken. When comparing two windows, the maximum difference (absolute value of the difference) between the data in the two windows is taken. In this example, the vehicle handling intensity is allowed to be greater than 1, so that it can break through the upper limit under extreme conditions, providing a stronger suppression of interactive response for subsequent calculations. Linear normalization is performed under the corresponding data dimension. The normalization method used in the embodiments of this invention can all adopt this method. The data dimension is determined by the independent data set composed of historical measured data or each type of data collected in the test vehicle / program. The specific technical means are well known to those skilled in the art and will not be described in detail here.

[0033] In other embodiments of the present invention, the preset limit constant, preset reference speed constant, and maximum steering wheel angle can be adjusted according to the specific vehicle model and implementation scenario.

[0034] Preferably, in one embodiment of the present invention, the release of cognitive resources by the human body has a physiological lag relative to physical movement (i.e., "the brain continues even after the hand stops"). To simulate this characteristic in engineering, this embodiment employs an asymmetric recursive filtering algorithm to update the driving intensity correlation factor. ; As an example, the maximum value of the product of the vehicle handling intensity of the previous adjacent time window and the preset attenuation coefficient, and the vehicle handling intensity of the current time window, is taken as the driving intensity correlation factor.

[0035] In this example, the preset decay coefficient is 0.95, corresponding to a half-life of 200ms in a period of about 10ms. When the input vehicle control intensity increases, the output follows without delay; when the input decreases, the output decays slowly according to an exponential law. The value has an asymmetric dynamic characteristic of "rapid rise and slow decay" to simulate the physiological lag effect of "the brain does not stop even when the hand stops".

[0036] It should be noted that the driving intensity correlation factor is used to characterize the risk of attention shift that may accompany intense physical manipulation. It is an engineering correlation modeling, rather than a direct measurement of the driver's physiological cognitive load. In other embodiments of the present invention, the implementer may adjust the preset attenuation coefficient.

[0037] Step S2: After detecting the first voice signal, monitor the NLU output and analyze the acoustic pitch rise feature to obtain the instruction incompleteness of the current time window; fuse the instruction incompleteness and driving intensity correlation factor to obtain the interaction suppression coefficient; obtain the time pressure coefficient based on the silence duration after the user stops speaking; compare the interaction suppression coefficient and time pressure coefficient of the current time window to obtain the response judgment value; analyze the changing trend of the interaction suppression coefficient and update the environment improvement flag.

[0038] The system reads the status bits of the Voice Activity Detection (VAD) in real time to bridge the gap when no voice signal is detected. After the first voice signal is detected, since the data stream of the voice interaction system (ASR / NLU) is usually event-driven, monitoring the output of the NLU can obtain the semantic continuity probability (i.e., whether the current sentence is not finished). However, since semantic recognition has processing delays and cannot reflect the acoustic changes at the end of the speech (such as interrogative intonation) in real time, the system also analyzes the acoustic intonation rise feature to obtain the degree of incompleteness of the instruction in the current time window, so as to achieve complementary fusion of semantics and acoustics and improve the accuracy of endpoint judgment and the rationality of response timing in dynamic driving scenarios.

[0039] In a preferred embodiment of the present invention, please refer to Figure 2 The flowchart illustrates a method for obtaining instruction incompleteness according to an embodiment of the present invention, specifically including: Step S201: Monitor the output of NLU and obtain the semantic continuous probability.

[0040] The first detected voice signal includes the first detection after the system powers on and the first detection after the last interaction ends. The VAD status bit is: VAD=1, indicating sound; VAD=0, indicating no sound. The system includes a timer to record the elapsed time after the user stops speaking. During the window period when no voice signal was detected, The timer remains at 0. After the first voice signal is detected, if VAD=1, the timer is reset. If VAD=0, the accumulator timer is activated. , The time window length.

[0041] Monitor the NLU output; if there is output or no output duration within the current time window ( If the timeout period is less than the preset first timeout (500ms in this example), it means that the NLU believes that the user's current statement has not ended or is being processed. In this case, the semantic continuity probability is marked as the preset first parameter (1 in this example). If the timeout period corresponding to the current time window exceeds the preset first timeout, it means that the NLU service has not responded for a long time. The system determines that the semantic continuity is reduced due to the timeout (fallback mechanism). In this case, the semantic continuity probability is obtained by linearly decaying the preset first parameter based on the timeout period.

[0042] Among them, the preset first parameter corresponds to the default high bit, indicating that the current state is highly likely to be incomplete (such as thinking or pausing), and the semantic continuity probability remains high; the linear decay process includes: setting the decay step size to 0.05 (decreasing by 0.05 in each time window). Triggered at time . If the duration exceeds the limit, the semantic continuity probability is at least 0, and if it is less than 0, it is truncated.

[0043] It should be noted that, since the driving intensity correlation factor in step S1 is calculated using asymmetric recursive filtering (which has the memory characteristics of rapid rise and slow decay), the value of the driving intensity correlation factor at the current moment naturally continues and covers the driving load state during the NLU processing delay, thereby achieving implicit alignment between driving state and semantic result in the time domain.

[0044] In another embodiment of the present invention, the system is configured with a state backtracking buffer (such as a circular queue) to store the sequence of driving intensity correlation factors for the most recent preset duration (such as 500ms). When the NLU outputs a result, the driving intensity correlation factor at the corresponding time is retrieved from the buffer based on the timestamp carried by the NLU result, and used to calculate the interaction inhibition coefficient. In other embodiments of the present invention, the implementer may adjust the preset first time limit and decay step size as needed, which will not be elaborated further.

[0045] Step S202: Extract the fundamental frequency sequence of the speech data of the preset historical neighborhood of the current time window, and obtain the acoustic pitch rise based on the changing trend of the fundamental frequency sequence.

[0046] Since a single window cannot capture the overall characteristics of intonation changes (such as the rise at the end of an interrogative sentence), the fundamental frequency sequence of speech data from a preset historical neighborhood is extracted. The trend of the fundamental frequency sequence reflects the prosodic characteristics of the user's pronunciation (such as whether there is lingering meaning or interrogative tone), so this is used to obtain the acoustic intonation rise. .

[0047] As an example, a linear regression is performed on the fundamental frequency sequence to obtain the original slope. Then, the original slope is divided by a preset maximum fundamental frequency change rate (e.g., 500 Hz / s) and normalized (if it exceeds this value, it is truncated to 1) to obtain the slope. Map the slope S to the interval [0,1]. Specifically, a variant of the Sigmoid function can be used: ; in, The sensitivity coefficient, This is the center offset. Slope. The larger the value (the steeper the intonation rise), the closer the output value is to 1, and the higher the likelihood that it represents a question or an unfinished sentence.

[0048] It should be noted that in this example, the preset historical neighborhood is 150ms, corresponding to 15 time windows (including the current time window). , (Engine experience values ​​derived from the statistical distribution characteristics and ROC curve optimization of a large amount of real driving voice data); Because there may be brief clear or silent intervals (such as plosive sounds) in the middle of the speech, there is no fundamental frequency at this time. Directly setting it to zero will cause a sharp fluctuation in the rising pitch. Therefore, if the current frame is silent (no fundamental frequency), the calculated value of the previous frame with sound is used. The fundamental frequency sequence refers to the set of data points in the speech signal in which the frequency of vocal cord vibration changes over time.

[0049] When k=0 or there are no historical audio frames, the default is... The value is 0.5 (neutral); if there is no fundamental frequency in the preset historical neighborhood, then... Set to zero.

[0050] Step S203: Combine semantic continuity probability and acoustic intonation rise to obtain the instruction incompleteness of the current time window.

[0051] As an example, a linearly weighted fusion of the semantic continuity probability and the vocal intonation rise of the current time window is used, with a weight of 0.6 for the semantic continuity probability and a weight of 0.4 for the vocal intonation rise. The weighted sum is then used as the instruction incompleteness of the current time window. .

[0052] It should be noted that in other embodiments of the present invention, the implementer may adjust the preset historical neighborhood and the linearly weighted weight allocation.

[0053] The risk of interruption depends not only on the completeness of the semantics, but also on the modulation of driving load. Even if the semantics are complete, interruption should be approached with caution under high load. Therefore, we integrate the incompleteness of instructions and the correlation factor of driving intensity to obtain the interaction inhibition coefficient. This couples driving safety and language understanding into a unified cost parameter, providing support for subsequent updates to the environment improvement flag.

[0054] While quantifying the interaction inhibition coefficient, the user's waiting cost must also be considered. Therefore, a time pressure coefficient is obtained based on the silence duration after the user stops speaking, quantifying the interaction pressure and forming a game variable equivalent to the interruption risk. The interaction inhibition coefficient and time pressure coefficient of the current time window are then compared to obtain a response judgment value. The response timing is determined by the real-time trade-off between risk and cost, compressing multi-dimensional decision-making into a single-dimensional continuous signal as a direct criterion for triggering the response.

[0055] Furthermore, considering the dynamic evolution of the driving environment, in order to capture optimization opportunities in the transition state, the changing trend of the interaction inhibition coefficient was analyzed, and the environmental improvement flag was updated to provide more basis for subsequent decision-making.

[0056] Preferably, in one embodiment of the present invention, relying solely on semantic probability or static thresholds cannot adapt to dynamic driving scenarios. Under high load, the driver's involuntary pause is easily misjudged as the end of the command. Therefore, the driving intensity correlation factor is used as a gain coefficient to dynamically adjust the system's requirements for semantic integrity. By introducing a nonlinear gain, the system can automatically increase the requirements for semantic integrity when it senses increased driving stress, thus building an interactive protection barrier.

[0057] As an example, the interaction inhibition coefficient The calculation formulas include: ; In the formula, This represents the incompleteness of instructions in the current window. The driving intensity correlation factor for the current time window; This is the operating condition penalty gain coefficient.

[0058] in, The basic risk item originates from semantic and acoustic features. It is a nonlinear gain term that amplifies the basic risk using the square of the driving load. Under stable operating conditions... Approaching 0, the nonlinear gain term approaches 1, and the system degenerates into conventional semantic judgment, ensuring response sensitivity under low load conditions. Under high load conditions, Approaching 1, the nonlinear gain term approaches This means that during aggressive driving, even if the semantics appear relatively complete (with low basic risk factors), the amplified interaction inhibition coefficient may still be very high, thus forcing the system to remain silent. This not only tolerates the incompleteness of the semantics but also effectively constructs a "soft barrier" that blocks interaction.

[0059] In this example, It is obtained through offline data calibration, collecting historical real-world driving data containing different driving intensities (from smooth to aggressive) and interaction scenarios (from brief commands to complex queries). The calibration aims to jointly optimize "minimizing the false interruption rate under high load conditions" while "limiting the response latency increment under low load conditions". . The recommended value range is For example, take This means that under extreme operating conditions, the system's risk assessment value is 3 times that under normal operating conditions.

[0060] Next, the system determines the duration of silence after the user stops speaking. Obtaining the time pressure coefficient Users' tolerance for system silence increases non-linearly over time; that is, they are more patient initially, but their anxiety rises sharply later. Based on this, the formula for calculating the time pressure coefficient includes: ; In the formula, This represents the preset user tolerance limit, which is 2000ms in this example, representing the maximum silent window that the user can psychologically accept. To preset an anxiety growth index, In this example, we take 2, which makes the waiting cost exhibit a "concave function" growth characteristic to simulate the psychological characteristic of user patience decaying nonlinearly over time. This is a function to find the minimum value, used for truncation. The upper limit is used for boundary protection.

[0061] It should be noted that in other embodiments of the present invention, and The implementer can adjust the settings themselves, for example, by setting three interaction modes for the user to choose from, such as faster response, default, and more accurate interaction modes. The values ​​are 1200, 2000, and 3000 (ms).

[0062] After quantifying the risks and costs, a game is played between the interruption risk and the waiting cost. As an example, the difference between the interaction inhibition coefficient of the current time window and the time pressure coefficient is used as the response judgment value.

[0063] The response judgment value directly quantifies the outcome of the trade-off between "misunderstanding risk" and "waiting cost": if it is greater than 0, it means that "misunderstanding risk" currently dominates, and the system believes that interrupting the user at this time is not worthwhile in terms of risk-benefit ratio, so it tends to continue waiting. If it is less than or equal to 0, it means that "waiting cost" has outweighed "misunderstanding risk." The system believes that the user has waited long enough, or that the semantics are complete enough and the risk is very low, so it tends to trigger a response.

[0064] Finally, the system compares the interaction inhibition coefficient of the current time window with that of the preset historical period to obtain the trend gradient. Compare the trend gradient with the preset gradient improvement threshold, and update the environmental improvement flag.

[0065] As an example, ; like A negative value indicates a decreasing trend in the interaction inhibition coefficient. A preset gradient improvement threshold of -0.05 is set to filter out minor numerical noise and capture only significant improvement trends. When the trend gradient is determined to be less than the preset gradient improvement threshold, it indicates a significant decrease in objective risk, and the system updates the environmental improvement flag. (Valid status), otherwise update the environment improvement flag. (Invalid state).

[0066] The preset gradient improvement threshold is set based on a comprehensive consideration of the system control cycle and noise tolerance. In this embodiment, the control cycle (time window length) is 10ms, N=5, and the preset gradient improvement threshold is -0.05, which means that the risk cost has decreased by at least 0.05, that is, the rate of change per unit time is about 1.0 / s. This value is significantly higher than the random fluctuations caused by the sensor noise floor and the small jitter of semantic probability. This ensures that the system only responds to clear and drastic trends in operating conditions (such as the driver quickly straightening the steering wheel or a large jump in semantic probability), thereby effectively filtering numerical noise and avoiding decision oscillations in the system.

[0067] This step eliminates interference from the natural increase in waiting costs, accurately identifies the moment when the environment truly improves (such as when the driver straightens the steering wheel), provides a reliable basis for subsequent "timing optimization" decisions, and avoids premature decision-making leading to misjudgment.

[0068] It should be noted that, in other embodiments of the present invention, the implementer may adjust the preset gradient improvement threshold independently.

[0069] Step S3: Based on the driving intensity correlation factor, environmental improvement flag and response judgment value of the current time window, perform multi-level response decision control and selectively reset the interaction state.

[0070] A single numerical judgment is insufficient to cover extreme dangerous working conditions and dynamic transition scenarios, and the physiological load after the interaction ends is continuous. Therefore, based on the driving intensity correlation factor, environmental improvement flag and response judgment value of the current time window, multi-level response decision control is carried out, and the interaction state is selectively reset. A hierarchical arbitration mechanism is constructed, thereby achieving an adaptive optimal balance between driving safety and interaction experience on a global scale.

[0071] Preferably, in one embodiment of the present invention, a multi-level priority state machine logic is adopted to balance driving safety, interaction quality and response speed in real time, and to manage the state lifecycle of the entire system.

[0072] First and foremost, driving safety is the absolute bottom line for in-vehicle systems. Regardless of the completeness of the current semantics or how long the user has waited, once the vehicle enters an extremely dangerous situation, the interaction system must unconditionally yield. Therefore, if the driving intensity correlation factor of the current time window exceeds the preset high-risk circuit breaker threshold, the preset safety circuit breaker process is triggered, specifically including: The preset high-risk circuit breaker threshold is 0.9, which represents that cognitive resources are close to saturation. When the preset safety circuit breaker process is triggered, it indicates that the driver is in an extremely tense control state (such as emergency avoidance or sudden braking). In order to avoid confusion for the user due to the system suddenly "losing its hearing" (which may in turn cause the driver to be distracted by checking), this embodiment optimizes the feedback mechanism after the circuit breaker is triggered: Interaction terminated: Immediately discard all currently cached speech data and semantic intermediate results, and reset the recognition engine.

[0073] Flexible feedback: Instead of playing lengthy voice prompts (to avoid interfering with driving), the system selectively plays a very short non-voice prompt (earcon) (such as a "ding" sound or a buzzer of a specific frequency) to inform the user through auditory symbols: "The system is online, but it is currently unavailable to respond."

[0074] Silent Lock: Within a short period of time (e.g., 500ms) after the circuit breaker is triggered, the system is forced into a silent lock state, ignoring any new voice input to prevent accidental triggering caused by the driver's unconscious shouts or noise. In the forced silent state, the multi-level response decision logic is skipped directly. During this period, the system only maintains state updates but does not perform any output actions until the silent lock timer ends.

[0075] Secondly, if the preset safety circuit breaker procedure is not triggered, the environmental improvement flag will be in a valid state ( This indicates that the environment is improving. At this point, even if the response decision value allows for a response, the system performs timing optimization to suppress decision oscillations, and the system enters a suspended state. Specifically: If the environmental improvement flag is active, the system is forced into a suspended state, temporarily delaying the triggering of a response. This leverages the predictive power of gradients, attempting to trade a small delay for a lower risk of misunderstanding (e.g., waiting for the steering wheel to fully straighten, or waiting for the last syllable of a sentence to land).

[0076] To dynamically adapt to the uncertainty of environmental changes, and to prevent the system from being suspended indefinitely (deadlock) due to the environment being in a state of slight improvement for a long time (such as a slow deceleration process lasting several seconds), when in a suspended state, if the environmental improvement flag of the current time window is invalid, it means that the original improvement trend has disappeared or has become insignificant, and the suspended state needs to be ended. Alternatively, if the duration of the suspended state exceeds the preset maximum greedy threshold, the suspended state is ended, and the final judgment stage based on numerical game theory is entered.

[0077] In this example, the preset maximum greedy threshold is 200ms. This threshold is set based on an engineering trade-off between intent recognition accuracy and interaction fluency. In complex scenarios involving dynamic driving and voice interaction, the additional 200ms waiting window is sufficient to cover the transient recovery process of most driving operations (such as quick steering wheel corrections) and short pauses at the end of voice commands (such as elision or hesitation). Although this delay may be perceived by the user, in high-risk or high-uncertainty scenarios, a "slightly slower but correct" interaction experience is far superior to an "extremely fast but incorrect / interrupted" experience. Therefore, 200ms was chosen as an empirical upper limit, aiming to use this brief time to significantly reduce the risk of misunderstanding while avoiding an unbearable sense of interaction interruption.

[0078] Finally, if the preset safety circuit breaker procedure is not triggered and the system is not in a suspended state, the system performs a final decision based on numerical game theory and generates an endpoint detection completion signal based on the response decision value.

[0079] When the response judgment value is less than or equal to 0, it indicates that the time pressure coefficient has overwhelmed the interaction inhibition coefficient. At this point, the system immediately generates an End of Utterance (EOU) signal. This signal is sent to the downstream Dialogue Management System (DM) to trigger the final submission of the speech recognition result, completing a full interaction loop.

[0080] And perform a selective reset: reset the silence duration to 0 and set the semantic continuity probability to the default high bit, because a new round of dialogue means user waiting and the restart of the semantic context.

[0081] Retaining the driving intensity correlation factor is crucial because the driver's physiological state and vehicle movement are objective physical processes that don't instantly disappear after the voice interaction ends. Retaining the driving intensity correlation factor means that if the driver has just experienced a high-load operation (high driving intensity correlation factor), even after the previous dialogue ends, the system still "remembers" the driver is in a recovery period at the moment the next dialogue begins (k+1 time window). This allows the high threshold protection to be directly inherited at the start of the next round of interaction, without needing to wait for vehicle data accumulation again, thus achieving seamless safety protection for continuous dialogue scenarios.

[0082] After selective reset, the system enters standby listening state. In this state, the system only performs VAD detection and analysis of driving intensity correlation factors, and suspends various cost parameter calculations, flag updates, and multi-level decision-making.

[0083] It should be noted that, considering that the vehicle may be driving on a long curve or bumpy road, the driving intensity correlation factor may be high, but not enough to trigger the circuit breaker. Furthermore, because the waiting cost is limited to the upper limit, the response judgment value will be greater than 0 and no response will be triggered. Therefore, an absolute timeout threshold (e.g., 5000ms) is set. When the silence time after the user stops speaking exceeds the absolute timeout threshold, it means that the user has been waiting for too long, which has seriously exceeded the psychological expectation of normal dialogue. At this time, the system's protection mechanism becomes an obstacle to interaction, forcing a response or forcing an error to avoid logical deadlock.

[0084] An embodiment of the present invention also provides a semantic analysis-based conversational AI low-latency response control system. The system includes a memory, a processor, and a computer program, wherein the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and the computer program, when running in the processor, can implement the semantic analysis-based conversational AI low-latency response control method described in steps S1-S3.

[0085] In summary, to address the technical problems of accidental truncation in voice interaction under complex driving conditions and sluggish response under stable conditions, this invention provides a conversational AI low-latency response control method and system based on semantic analysis. First, this invention generates a driving intensity correlation factor for the current time window based on changes in CAN bus data. Next, it monitors the NLU output and analyzes the rising vocal pitch characteristics to obtain the instruction incompleteness degree for the current time window. Then, it fuses the instruction incompleteness degree and the driving intensity correlation factor to obtain an interaction suppression coefficient. Finally, it obtains a time pressure coefficient based on the silence duration after the user stops speaking. Next, it compares the interaction suppression coefficient and the time pressure coefficient for the current time window to obtain a response judgment value. Finally, it analyzes the changing trend of the interaction suppression coefficient and updates the environment improvement flag. Finally, based on the driving intensity correlation factor, the environment improvement flag, and the response judgment value for the current time window, it performs multi-level response decision control and selectively resets the interaction state. It integrates driving control and voice features, calculates interruption and waiting costs modulated by load, and combines trend-based multi-level control. By leveraging the risk of interruption due to driving load modulation and balancing waiting costs with environmental changes, adaptive decision-making on response timing can be achieved, thus solving the problems of truncation and delayed response in voice interaction under dynamic driving.

[0086] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A low-latency response control method for conversational AI based on semantic analysis, characterized in that, The method includes: After the system restarts, it initializes, acquires the vehicle's CAN bus data, and divides it into preset time windows; based on the changes in the CAN bus data, it acquires the vehicle handling intensity of the current time window; and generates a driving intensity correlation factor for the current time window based on the vehicle handling intensity. After detecting the first voice signal, the NLU output is monitored, and the acoustic pitch rise feature is analyzed to obtain the instruction incompleteness degree of the current time window; the instruction incompleteness degree and the driving intensity correlation factor are fused to obtain the interaction suppression coefficient; the time pressure coefficient is obtained based on the silence duration after the user stops speaking; the interaction suppression coefficient and the time pressure coefficient of the current time window are compared to obtain the response judgment value; the changing trend of the interaction suppression coefficient is analyzed to update the environment improvement flag; Based on the driving intensity correlation factor, the environmental improvement flag, and the response judgment value of the current time window, multi-level response decision control is performed, and the interactive state is selectively reset.

2. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, The method for obtaining the vehicle handling strength includes: The CAN bus data includes at least longitudinal acceleration, steering wheel angle, and real-time vehicle speed; For the current time window, a first control intensity is obtained based on the rate of change of longitudinal acceleration and the rate of change of steering wheel angle between adjacent time windows; a second control intensity is obtained by fusing the absolute value of the steering wheel angle and the real-time vehicle speed. The vehicle control strength is obtained by combining the first control strength and the second control strength.

3. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, The method for obtaining the incompleteness of the instruction includes: Monitor the output of NLU to obtain semantic continuous probabilities; Extract the fundamental frequency sequence of speech data from the preset historical neighborhood of the current time window, and obtain the acoustic pitch rise based on the changing trend of the fundamental frequency sequence; By combining the semantic continuity probability and the acoustic intonation rise, the incompleteness of the instruction in the current time window is obtained.

4. The semantic analysis-based conversational AI low-latency response control method according to claim 3, characterized in that, The method for obtaining the semantic continuity probability includes: Monitor the NLU output. If the duration of output or stop output within the current time window does not exceed a preset first time limit, mark the semantic continuity probability as a preset first parameter. If the duration of stop output corresponding to the current time window exceeds the preset first time limit, obtain the semantic continuity probability based on the preset first parameter which is linearly decayed based on the duration exceeding the first time limit.

5. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, The method for obtaining the environmental improvement flag bit includes: The interaction inhibition coefficient of the current time window is compared with that of the preset historical period to obtain the trend gradient; the trend gradient is compared with the preset gradient improvement threshold to update the environmental improvement flag.

6. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, Methods for multi-level response decision control include: If the driving intensity correlation factor of the current time window is greater than the preset high-risk circuit breaker threshold, the preset safety circuit breaker process is triggered. If the preset safety circuit breaker procedure is not triggered, the system enters a suspended state when the environment improvement flag is in a valid state. If the preset safety circuit breaker process is not triggered and the device is not in a suspended state, an endpoint detection completion signal is generated based on the response judgment value.

7. The method for low-latency response control of conversational AI based on semantic analysis according to claim 6, characterized in that, If the environment improvement flag of the current time window is invalid or the duration of the suspension exceeds the preset maximum greedy threshold when the window is in a suspended state, the suspension state will end.

8. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, The method for performing the selective reset includes: The silence duration is reset to 0, while the driving intensity correlation factor is retained.

9. The method for low-latency response control of conversational AI based on semantic analysis according to claim 1, characterized in that, The method for obtaining the driving intensity correlation factor includes: The maximum value of the vehicle handling intensity in the previous time window and the preset attenuation coefficient, and the vehicle handling intensity in the current time window, is taken as the driving intensity correlation factor.

10. A semantic analysis-based conversational AI low-latency response control system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the semantic analysis-based conversational AI low-latency response control method as described in any one of claims 1 to 9.

Images