Conversational ai low latency response control method and system based on semantic analysis

Abstract

The present application relates to the technical field of voice interaction, in particular to a dialogue AI low-delay response control method and system based on semantic analysis. The present application first generates a driving intensity correlation factor of the current time window according to the change of CAN bus data; further monitors the output of NLU and analyzes the acoustic tone lifting feature, combines the driving intensity correlation factor to obtain an interaction inhibition coefficient; further obtains a time pressure coefficient based on the silence duration after the user stops speaking; further compares the interaction inhibition coefficient and the time pressure coefficient of the current time window to obtain a response decision value; further analyzes the change trend of the interaction inhibition coefficient to update the environment improvement flag; finally, based on the driving intensity correlation factor of the current time window, the environment improvement flag and the response decision value, multi-level response decision control is performed, and the selective reset of the interaction state is performed, solving the problems of voice interaction mis-truncation and response delay under dynamic driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, in particular to a dialogue AI low-latency response control method and system based on semantic analysis. BACKGROUND

[0002] The response timing control of the vehicle-mounted voice interaction system is the key to the intelligent cockpit experience. The existing technology usually adopts a fixed threshold-based voice activity detection (VAD) strategy to determine the end of the user's instruction, for example, a fixed silence duration threshold is set, and when the pause exceeds the threshold, the response is triggered. However, this static mechanism is difficult to adapt to the dynamically changing driving scenarios. In actual driving, the driver often has involuntary speech pauses or slows down due to turning, lane changing or avoiding operations, and a fixed short threshold is easy to misjudge as the end of the instruction, resulting in "stealing" or interrupting the system; in order to be compatible with such pauses, if the threshold is set too long, it will also cause response delay in smooth working conditions, reducing the interaction efficiency.

[0003] Although some schemes try to introduce vehicle speed or gear information to simply switch interaction modes, they often lack fine modeling of the continuous change process of driving load, cannot accurately perceive the lag effect of attention release after the end of driving actions, and are difficult to establish a dynamic balance between "misinterrupt risk" and "user waiting anxiety", resulting in difficulty in synchronizing the interaction strategy with the actual state of the driver. SUMMARY

[0004] In order to solve the technical problems of easy mis-truncation of voice interaction in complex driving conditions and response delay in smooth conditions, the purpose of the present application is to provide a dialogue AI low-latency response control method and system based on semantic analysis, and the technical solution adopted is as follows:

[0005] A dialogue AI low-latency response control method based on semantic analysis, the method comprising:

[0006] After the system is restarted, initialization is performed, CAN bus data of the vehicle is obtained and divided into preset time windows; according to the changes of the CAN bus data, the vehicle control strength of the current time window is obtained; based on the vehicle control strength, the driving strength correlation factor of the current time window is generated;

[0007] After detecting the first voice signal, the output of NLU is monitored, and the acoustic intonation lifting feature is analyzed to obtain the instruction incomplete degree of the current time window; the interaction inhibition coefficient is obtained by fusing the instruction incomplete degree and the driving strength correlation factor; the time pressure coefficient is obtained based on the silence duration after the user stops speaking; the response determination value is obtained by comparing the interaction inhibition coefficient and the time pressure coefficient of the current time window; the change trend of the interaction inhibition coefficient is analyzed, and the environment improvement flag is updated;

[0008] According to the driving intensity correlation factor of the current time window, the environment improvement flag and the response determination value, multi-level response decision control is performed, and selective reset of the interactive state is performed.

[0009] Further, the vehicle control intensity acquisition method comprises:

[0010] The CAN bus data at least comprises longitudinal acceleration, steering wheel angle and real-time vehicle speed;

[0011] For the current time window, according to the rate of change of the longitudinal acceleration and the rate of change of the steering wheel angle between adjacent time windows, a first control intensity is obtained; and by fusing the absolute value of the steering wheel angle and the real-time vehicle speed, a second control intensity is obtained;

[0012] The first control intensity and the second control intensity are fused to obtain the vehicle control intensity.

[0013] Further, the instruction incomplete degree acquisition method comprises:

[0014] The output of the NLU is monitored to obtain a semantic continuity probability;

[0015] The fundamental frequency sequence of the voice data of the preset historical neighborhood of the current time window is extracted, and according to the change trend of the fundamental frequency sequence, an acoustic intonation rising degree is obtained;

[0016] The semantic continuity probability and the acoustic intonation rising degree are fused to obtain the instruction incomplete degree of the current time window.

[0017] Further, the semantic continuity probability acquisition method comprises:

[0018] The output of the NLU is monitored, and if there is output within the current time window or the duration of stopping output does not exceed a preset first time limit, the semantic continuity probability is marked as a preset first parameter; if the duration of stopping output corresponding to the current time window exceeds the preset first time limit, the semantic continuity probability is obtained based on linear attenuation of the preset first parameter by the exceeding duration.

[0019] Further, the environment improvement flag acquisition method comprises:

[0020] The interactive inhibition coefficient of the current time window is compared with that of a preset historical period to obtain a determination trend gradient; the environment improvement flag is updated by comparing the determination trend gradient with a preset gradient improvement threshold.

[0021] Further, the multi-level response decision control method comprises:

[0022] If the driving intensity correlation factor of the current time window is greater than a preset high-risk fuse threshold, a preset safety fuse process is triggered;

[0023] If the preset safety fuse process is not triggered, and the environment improvement flag is in the valid state, the system enters a suspended state.

[0024] If the preset safety fuse process is not triggered, and the system is not in the suspended state, an endpoint detection completion signal is generated based on the response determination value.

[0025] Further, when in the suspended state, if the environment improvement flag of the current time window is in the invalid state or the duration of the suspended state exceeds a preset maximum greed threshold, the suspended state is ended.

[0026] Further, the method of selectively resetting includes:

[0027] The mute duration is reset to 0, and the driving intensity correlation factor is retained.

[0028] Further, the method of obtaining the driving intensity correlation factor includes:

[0029] The maximum value of the product of the vehicle control intensity of the adjacent previous time window and a preset attenuation coefficient, and the vehicle control intensity of the current time window, is taken as the driving intensity correlation factor.

[0030] The application further provides a dialog AI low-latency response control system based on semantic analysis, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any one of the dialog AI low-latency response control methods based on semantic analysis.

[0031] The application has the following beneficial effects:

[0032] The application first generates a driving intensity correlation factor of the current time window according to the change of CAN bus data, representing the occupation level of cognitive resources of the driver in the current and recent driving actions; further monitors the output of NLU and analyzes the acoustic intonation lifting feature to obtain the instruction incompleteness degree of the current time window, so as to realize the complementary fusion of semantics and acoustics, improve the accuracy of endpoint judgment and the rationality of response timing in the dynamic driving scene; further fuse the instruction incompleteness degree and the driving intensity correlation factor to obtain the interaction inhibition coefficient, couple driving safety and language understanding into a unified cost parameter, and provide support for subsequent updating of environment improvement flag; further obtain the time pressure coefficient based on the silence duration after the user stops speaking; further compare the interaction inhibition coefficient and the time pressure coefficient of the current time window to obtain the response decision value as the direct criterion for response triggering; further analyze the trend of the interaction inhibition coefficient to update the environment improvement flag, providing more basis for subsequent decision-making; finally, according to the driving intensity correlation factor, the environment improvement flag and the response decision value of the current time window, multi-level response decision control is performed, and the interactive state is selectively reset. The driving control and voice features are fused, the interruption cost and waiting cost modulated by the load are calculated, and the multi-level control is combined with the trend. The driving control and voice features are fused, the interruption cost and waiting cost are calculated, and the multi-level control is combined with the trend flag. The interruption risk modulated by the driving load is used to balance the waiting cost and environmental change, realize adaptive decision-making of the response timing, and solve the problems of mis-truncation and response lag in voice interaction under dynamic driving. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, a brief introduction will be given to the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0034] Figure 1 A flowchart of a dialogue AI low-latency response control method based on semantic analysis provided by an embodiment of the present application;

[0035] Figure 2 A flowchart of an instruction incompleteness degree acquisition method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the specific implementation, structure, features and effects of the semantic analysis-based conversational AI low-latency response control method and system according to the present application are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0038] The specific scheme of the semantic analysis-based conversational AI low-latency response control method and system provided by the present application is described in detail below in combination with the drawings.

[0039] Please refer to Figure 1 , which shows a flowchart of a semantic analysis-based conversational AI low-latency response control method according to an embodiment of the present application, which specifically includes:

[0040] Step S1, after system restart, initialization is performed, CAN bus data of the vehicle is acquired and divided into preset time windows; according to the change of the CAN bus data, the vehicle control strength of the current time window is acquired; and the driving strength correlation factor of the current time window is generated based on the vehicle control strength.

[0041] Voice interaction needs to be synchronized with the driving state, and the driving state is a continuous time-varying process, so a unified time reference must be established. At the same time, in order to ensure the effectiveness of the recursive calculation logic, the system must perform a strict initialization process after power-on start or reset, so initialization is performed after system restart, CAN bus data of the vehicle is acquired and divided into preset time windows, providing a basis for subsequent analysis.

[0042] In an embodiment of the present application, the length of the preset time window is 10 ms, the index of the time window is k, and the system restart includes power-on start or reset and other start modes; the initialization includes resetting the state register: setting the driving strength correlation factor to zero, setting the mute duration after the user stops speaking (interaction mute duration) to zero, setting the semantic continuity probability to the default high bit (such as 1.0) to assume that the system is in a high semantic protection state in the initial state.

[0043] Sliding window filling is performed, and for the calculation of historical data, a queue with a length of N is established in advance, N=5, in the initial running period (i.e. cold start phase, ), due to lack of historical data, the system adopts a "zero padding" or "copy padding" strategy, i.e. all state values at the moment are 0 or equal to the value of the window at k = 0 to prevent calculation abnormalities.

[0044] It should be noted that the control method of the embodiment of the application is mainly applied to the voice interaction scene of the driver and the vehicle-mounted system. Before executing the interaction judgment, the system will identify the identity of the voice signal speaker based on the existing cabin sound source positioning technology (Sound Zone Localization) or voiceprint recognition technology, and only when the speaker is identified as the driver (main driver position), the response control logic based on the driving intensity correlation factor is activated; if the speaker is a passenger, the conventional interaction logic is used to avoid unnecessary interference to the normal experience of the passenger.

[0045] In addition, considering that in the state where the assisted driving (ADAS) is started, the violent movement of the vehicle (such as automatic avoidance and large-curvature cornering) will still cause the driver's supervision attention or takeover demand, therefore, the embodiment does not exclude whether the vehicle is in the assisted driving mode, so as to ensure that the driver is provided with consistent safety interaction protection in any high dynamic working condition. In other embodiments of the application, the implementer can adjust the length of the preset time window and the queue length N according to the specific implementation scene, and will not be repeated here.

[0046] The degree of driving operation directly reflects the degree of driver's attention being occupied, in order to quantify the current physical operation load of the driver, the system collects CAN bus data (data of the chassis control system), according to the change of the CAN bus data, the vehicle control strength of the current time window is obtained, the physical load of the current driving task is quantified, the underlying vehicle dynamics information is abstracted as an engineering feature that can participate in the interaction decision, and then the driving intensity correlation factor of the current time window is generated based on the vehicle control strength, representing the occupation level of the driver's cognitive resources in the current and recent driving actions, providing a key input for subsequent construction of an interaction suppression coefficient, implementation of interaction suppression in a high-risk working condition, and ensuring the coherence of state memory in continuous dialogue.

[0047] Preferably, in an embodiment of the application, the acceleration and deceleration behavior and the steering behavior of the vehicle will cause the driver's attention to shift from the interaction task to the driving task (i.e. the cognitive resources are occupied), reflecting the vehicle control strength, so the CAN bus data at least includes longitudinal acceleration, steering wheel angle and real-time vehicle speed; considering that the actual road environment (such as a speed bump and a gravel road) may introduce high-frequency noise, the system first performs low-pass filtering processing (for example, using a Butterworth filter with a cutoff frequency of 5 Hz) on the original data to obtain a smoothed signal.

[0048] The greater the longitudinal acceleration change and steering wheel angle change of adjacent time windows, the more intense the instantaneous steering action, and the smaller the steering wheel angle value in high-speed scenarios, but the vehicle posture response is more sensitive (the lateral centrifugal force is greater), and the operation risk is higher;

[0049] Based on this, for the current time window, the first steering strength is obtained according to the change rate of longitudinal acceleration and the change rate of steering wheel angle between adjacent time windows; the second steering strength is obtained by fusing the absolute value of the steering wheel angle and the real-time vehicle speed;

[0050] The vehicle steering strength is obtained by fusing the first steering strength and the second steering strength.

[0051] As an example, the current time window is compared with the adjacent previous time window, the longitudinal acceleration and the steering wheel angle are taken as the dimensions to be analyzed respectively, the absolute value of the difference between the data of the two windows is taken as the numerator, the time window length is taken as the denominator, and the fractional ratio is taken as the corresponding change rate;

[0052] The change rates are standardized by means of preset limit constants, that is, the real-time data is divided by the limit constant. In this example, the limit constant corresponding to the change rate of longitudinal acceleration is 0.8 (based on the physical limit acceleration of a conventional vehicle, which can be adjusted according to the specific vehicle model), and the limit constant corresponding to the change rate of the steering wheel angle is 500 (based on the maximum statistical value or the physical limit speed of a human hand turning the steering wheel).

[0053] The standardized change rates of longitudinal acceleration and steering wheel angle are weighted and summed, and the weighted weight is 0.5, corresponding to the same importance of longitudinal acceleration and deceleration and lateral steering operation, and the weighted sum result is taken as the first steering strength;

[0054] The ratio of the real-time vehicle speed to the preset reference vehicle speed constant (such as 120 ) is added by a constant 1, and the sum is taken as the vehicle speed weighting coefficient. The ratio of the absolute value of the steering wheel angle to the absolute value of the maximum steering angle of the vehicle steering wheel (such as 540 ) is taken as the steering angle coefficient. The product of the steering angle coefficient and the vehicle speed weighting coefficient is linearly normalized, and the normalized result is taken as the second steering strength;

[0055] The first steering strength and the second steering strength are weighted and summed, and the weighted weight is 0.5, corresponding to the same importance of the instantaneous operation intensity and the steady-state operation risk, and the weighted sum result is taken as the vehicle steering strength.

[0056] The longitudinal acceleration refers to the acceleration data of the vehicle traveling direction; the steering wheel angle refers to the angle data of the steering wheel turned by the driver relative to the neutral position.

[0057] It should be noted that when there are multiple data in a time window, only one parameter is required to take the maximum value in the window, and when comparing two windows, the maximum difference value (absolute value of difference) between the data of the two windows is taken; in this example, the vehicle control strength is allowed to be greater than 1, so that it can break through the upper limit in extreme working conditions, and provide stronger inhibition of interaction for subsequent calculation; linear normalization is performed under the corresponding data dimension, and the normalization adopted in the embodiments of the present application can all adopt this method, and the data dimension is determined by the independent data set formed by each type of data collected from historical measured data or test vehicles / programs, which is a technical means familiar to those skilled in the art and will not be repeated here.

[0058] In other embodiments of the present application, the preset limit constant, the preset reference vehicle speed constant and the maximum steering angle of the vehicle steering wheel can be adjusted according to specific vehicle models and implementation scenarios.

[0059] Preferably, in an embodiment of the present application, the release of cognitive resources of the human body has a physiological lag relative to the limb movement (i.e. "hand stops brain does not stop"). In order to simulate this characteristic in engineering, the asymmetric recursive filtering algorithm is adopted in this embodiment to update the driving strength correlation factor ;

[0060] As an example, the product of the vehicle control strength of the adjacent last time window and the preset decay coefficient, and the maximum value in the vehicle control strength of the current time window, are taken as the driving strength correlation factor.

[0061] In this example, the preset decay coefficient is 0.95, corresponding to a half-life of 200ms under a period of about 10ms, when the input vehicle control strength rises, the output follows without delay; when the input decreases, the output decays slowly according to the exponential law, and the numerical value has the asymmetric dynamic characteristic of "fast rise, slow decay", which is used to simulate the physiological lag effect of "hand stops brain does not stop".

[0062] It should be noted that the driving strength correlation factor is used to represent the risk of attention shift that may be accompanied by intense physical manipulation, which belongs to the correlation modeling in engineering, and is not a direct measurement of the physiological cognitive load of the driver; in other embodiments of the present application, the implementer can adjust the preset decay coefficient.

[0063] Step S2, after detecting the first voice signal, monitoring the output of NLU and analyzing the acoustic intonation lifting feature to obtain the instruction incompleteness degree of the current time window; fusing the instruction incompleteness degree and the driving strength correlation factor to obtain the interaction inhibition coefficient; obtaining the time pressure coefficient based on the silence duration after the user stops speaking; comparing the interaction inhibition coefficient and the time pressure coefficient of the current time window to obtain the response decision value; analyzing the trend of the interaction inhibition coefficient to update the environment improvement flag.

[0064] The system reads the status bits of the Voice Activity Detection (VAD) in real time to bridge the gap when no voice signal is detected. After the first voice signal is detected, since the data stream of the voice interaction system (ASR / NLU) is usually event-driven, monitoring the output of the NLU can obtain the semantic continuity probability (i.e., whether the current sentence is not finished). However, since semantic recognition has processing delays and cannot reflect the acoustic changes at the end of the speech (such as interrogative intonation) in real time, the system also analyzes the acoustic intonation rise feature to obtain the degree of incompleteness of the instruction in the current time window, so as to achieve complementary fusion of semantics and acoustics and improve the accuracy of endpoint judgment and the rationality of response timing in dynamic driving scenarios.

[0065] In a preferred embodiment of the present invention, please refer to Figure 2 The flowchart illustrates a method for obtaining instruction incompleteness according to an embodiment of the present invention, specifically including:

[0066] Step S201: Monitor the output of NLU and obtain the semantic continuous probability.

[0067] The first detected voice signal includes the first detection after the system powers on and the first detection after the last interaction ends. The VAD status bit is: VAD=1, indicating sound; VAD=0, indicating no sound. The system includes a timer to record the elapsed time after the user stops speaking. During the window period when no voice signal was detected, The timer remains at 0. After the first voice signal is detected, if VAD=1, the timer is reset. If VAD=0, the accumulator timer is incremented. , The time window length.

[0068] Monitor the NLU output; if there is output or no output duration within the current time window ( If the timeout period is less than the preset first timeout (500ms in this example), it means that the NLU believes that the user's current statement has not ended or is being processed. In this case, the semantic continuity probability is marked as the preset first parameter (1 in this example). If the timeout period corresponding to the current time window exceeds the preset first timeout, it means that the NLU service has not responded for a long time. The system determines that the semantic continuity is reduced due to the timeout (fallback mechanism). In this case, the semantic continuity probability is obtained by linearly decaying the preset first parameter based on the timeout period.

[0069] Among them, the preset first parameter corresponds to the default high bit, indicating that the current state is highly likely to be incomplete (such as thinking or pausing), and the semantic continuity probability remains high; the linear decay process includes: setting the decay step size to 0.05 (decreasing by 0.05 in each time window). Triggered at time . The semantic continuity probability is minimum 0 corresponding to the exceeding time length, and is truncated when less than 0.

[0070] It should be noted that, since the driving intensity correlation factor in step S1 adopts an asymmetric recursive filtering calculation (with a memory characteristic of fast rising and slow decay), the driving intensity correlation factor value at the current time naturally continues and covers the driving load state during the NLU processing delay, thereby realizing implicit alignment of the driving state and the semantic result in the time domain.

[0071] In another embodiment of the present application, a state backtracking buffer (such as a ring queue) is provided for the system to store the driving intensity correlation factor sequence of the recent preset time length (such as 500 ms). When the NLU output result is output, the driving intensity correlation factor at the corresponding time is found back in the buffer according to the time stamp carried by the NLU result, which is used to calculate the interaction inhibition coefficient. In other embodiments of the present application, the implementer can adjust the preset first time limit and the decay step as needed, which will not be described again.

[0072] Step S202, extract the fundamental frequency sequence of the voice data of the preset historical neighborhood of the current time window, and obtain the acoustic tone rising degree according to the variation trend of the fundamental frequency sequence.

[0073] Since a single window cannot capture the overall change characteristics of the tone (such as the rising at the end of a question sentence), the fundamental frequency sequence of the voice data of the preset historical neighborhood is extracted, and the variation trend of the fundamental frequency sequence reflects the prosodic characteristics of the user's pronunciation (such as whether it has an unfinished meaning or a questioning tone), so the acoustic tone rising degree is obtained .

[0074] As an example, linear regression fitting is performed on the fundamental frequency sequence to obtain the original slope, and then the original slope is divided by the preset maximum fundamental frequency change rate (such as 500 Hz / s) for normalization (exceeding 1 is truncated), to obtain the slope . The slope S is mapped to the [0, 1] interval. Specifically, the Sigmoid variant function can be used:

[0075] ;

[0076] Wherein, is a sensitivity coefficient, is a center offset. The larger the slope , the closer the output value is to 1, indicating a higher possibility of a questioning or unfinished tone.

[0077] It should be noted that, in the present example, the preset historical neighborhood is 150 ms, corresponding to 15 time windows (including the current time window), , (based on the statistical distribution characteristics of a large amount of real driving voice data and the ROC curve optimization engineering experience value); because there may be a short pause or silent gap (such as a burst) in the voice, at this time there is no fundamental frequency, and directly setting to zero will cause the sharp fluctuation of the degree of rising, so if the current is a silent frame (no fundamental frequency), the calculated value of the last voiced frame is followed. The fundamental frequency sequence refers to the data point set of the frequency of vocal cord vibration in the voice signal changing with time.

[0078] When k = 0 or there is no history of voiced frame, the default is 0.5 (neutral); if there is no fundamental frequency in the preset history neighborhood, set to zero.

[0079] Step S203, fuse the semantic continuity probability and the acoustic tone rising degree to obtain the instruction incomplete degree of the current time window.

[0080] As an example, linearly weighted fusion of the semantic continuity probability and the acoustic tone rising degree of the current time window, the weight of the semantic continuity probability is 0.6, the weight of the acoustic tone rising degree is 0.4, and the weighted sum result is taken as the instruction incomplete degree of the current time window .

[0081] It should be noted that in other embodiments of the present application, the implementer can adjust the preset history neighborhood and the weight distribution of linear weighting.

[0082] The interruption risk not only depends on whether the semantics is complete, but also is modulated by driving load - even if the semantics is complete, it should be cautious to interrupt under high load, so the instruction incomplete degree and the driving intensity correlation factor are fused to obtain the interaction inhibition coefficient, and the driving safety and language understanding are coupled as a unified cost parameter to provide support for subsequent updating of the environment improvement flag.

[0083] While quantifying the interaction inhibition coefficient, the user's waiting cost also needs to be considered, so the time pressure coefficient is obtained based on the silent duration after the user stops speaking to quantify the interaction pressure, and a game variable equivalent to the interruption risk is formed. Compare the interaction inhibition coefficient and the time pressure coefficient of the current time window to obtain the response decision value, and the real-time trade-off between risk and cost determines the response opportunity, which compresses the multi-dimensional decision into a single-dimensional continuous signal as a direct criterion for response triggering.

[0084] In addition, considering that the driving environment has dynamic evolution characteristics, in order to capture the optimization opportunity in the transition state, the change trend of the interaction inhibition coefficient is also analyzed to update the environment improvement flag to provide more basis for subsequent decision-making.

[0085] ​Preferably, in an embodiment of the present application, the semantic probability or static threshold alone cannot adapt to dynamic driving scenarios, and the involuntary pause of the driver under high load is easily misjudged as the end of the instruction, so the driving intensity correlation factor is used as a gain coefficient to dynamically adjust the system's requirement for semantic integrity;

[0086] By introducing a nonlinear gain, the system can automatically increase the requirement for semantic integrity when it perceives an increase in driving pressure, and build an interactive protection barrier.

[0087] As an example, the interactive suppression coefficient is calculated as follows:

[0088] ;

[0089] In the formula, is the instruction completion degree of the current time window, is the driving intensity correlation factor of the current time window; is the working condition penalty gain coefficient.

[0090] wherein, is the basic risk term, derived from semantic and acoustic features. is the nonlinear gain term, which amplifies the basic risk by the square of the driving load. Under smooth working conditions, is close to 0, and the nonlinear gain term tends to 1, degrading the system to a regular semantic judgment, ensuring the response sensitivity under low load. Under high load conditions, is close to 1, and the nonlinear gain term tends to . This means that during intense driving, even if the semantics look relatively complete (the basic risk term is low), the amplified interactive suppression coefficient may still be high, forcing the system to remain silent, not only tolerating incomplete semantics, but also essentially building a "soft barrier" to block interaction.

[0091] In this example, is obtained by offline data calibration, collecting historical real driving data containing different driving intensities (from smooth to intense) and interactive scenarios (from short instructions to complex queries), and taking "minimizing the misinterrupt rate under high load" and "limiting the response delay increment under low load" as joint optimization objectives to calibrate . The recommended value range of is . For example, taking means that in extreme conditions, the system's evaluation of risk is 3 times that of normal conditions.

[0092] Then, the system obtains the time pressure coefficient , the user's tolerance to system silence increases non-linearly over time, that is, the patience is better at the beginning, and the anxiety increases sharply at the later stage, based on this, the calculation formula of the time pressure coefficient includes:

[0093] ;

[0094] In the formula, , the preset user tolerance limit is 2000ms in this example, representing the maximum silence window that the user can accept psychologically; is a preset anxiety growth index, , which is 2 in this example, so that the waiting cost presents a "concave function" growth characteristic, to simulate the psychological characteristics of the user's patience non-linearly decaying over time; is a minimum function, which is used to truncate the upper limit of , for boundary protection.

[0095] It should be noted that in other embodiments of the present application, and can be adjusted by the implementer, for example, setting three interaction modes for the user to choose, such as faster response, default, and more accurate interaction, 1200, 2000 and 3000 (ms) respectively.

[0096] After the risk and cost are quantified, the game between the interruption risk and the waiting cost is carried out, as an example, the difference between the interaction suppression coefficient of the current time window and the time pressure coefficient is taken as the response decision value.

[0097] The response decision value directly quantifies the game result of "misunderstanding risk" and "waiting cost": if greater than 0, it means that "misunderstanding risk" currently dominates, and the system considers that the risk benefit brought by interrupting the user is not cost-effective, and tends to continue waiting. If less than or equal to 0, it means that "waiting cost" has overwhelmed "misunderstanding risk". The system considers that the user has waited long enough, or the semantics is complete enough and the risk is low, and tends to trigger the response.

[0098] Finally, the system compares the interaction suppression coefficient of the current time window with the preset historical period to obtain the decision trend gradient ; compare the decision trend gradient with the preset gradient improvement threshold to update the environment improvement flag.

[0099] As an example, ;

[0100] If If the result is negative, it means that the interaction inhibition coefficient shows a downward trend, and the preset gradient improvement threshold is set to -0.05, which is used to filter small numerical noise and only capture significant improvement trends. When the trend gradient is less than the preset gradient improvement threshold, it means that the objective risk is significantly decreasing, and the system updates the environment improvement flag to (effective state), otherwise the environment improvement flag is updated to (inactive state).

[0101] The preset gradient improvement threshold is set based on the comprehensive consideration of the system control period and the noise tolerance. In the embodiment, the control period (time window length) is 10 ms, N=5, and the preset gradient improvement threshold is -0.05, which means that the risk cost decreases by at least 0.05, i.e. the unit time change rate is about 1.0 / s, which is significantly higher than the random fluctuations caused by sensor noise and small semantic probability fluctuations, ensuring that the system only responds to clear and dramatic work condition improvement trends (such as the driver quickly returning the steering wheel or the semantic probability jumping significantly), thereby effectively filtering numerical noise and avoiding system decision shock.

[0102] This step excludes the interference caused by the natural growth of the waiting cost, accurately identifies the moment when the environment really improves (such as the driver returning the steering wheel), and provides a reliable basis for subsequent "timing optimization" decisions, avoiding premature decisions that lead to mis-truncation.

[0103] It should be noted that in other embodiments of the present application, the implementer can adjust the preset gradient improvement threshold.

[0104] Step S3, according to the driving intensity correlation factor of the current time window, the environment improvement flag and the response decision value, multi-level response decision control is performed, and selective reset of the interaction state is performed.

[0105] Single numerical determination cannot cover extreme dangerous working conditions and dynamic transition scenarios, and the physiological load after the interaction ends has continuity, so according to the driving intensity correlation factor of the current time window, the environment improvement flag and the response decision value, multi-level response decision control is performed, and selective reset of the interaction state is performed, thereby constructing a hierarchical arbitration mechanism, and achieving adaptive optimal balance of driving safety and interaction experience globally.

[0106] Preferably, in an embodiment of the present application, a multi-level priority state machine logic is used to real-time weigh driving safety, interaction quality and response speed, and is responsible for managing the state life cycle of the whole system.

[0107] First, driving safety is the absolute bottom line of in-vehicle systems. Regardless of whether the current semantics are complete or how long the user has been waiting, once the vehicle enters an extremely dangerous working condition, the interactive system must unconditionally yield, so if the driving intensity correlation factor of the current time window is greater than the preset high-risk fuse threshold, the preset safety fuse process is triggered, which specifically includes:

[0108] The preset high-risk fuse threshold is 0.9, which represents that the cognitive resources have approached saturation, and when the preset safety fuse process is triggered, it means that the driver is in an extremely tense control state (such as emergency avoidance or emergency braking). In order to avoid the user's confusion caused by the system's sudden "deafness" (which in turn may cause the driver's distraction check), the feedback mechanism after the fuse is optimized in this embodiment:

[0109] Interaction termination: immediately discard all cached voice data and semantic intermediate results, and reset the recognition engine.

[0110] Flexible feedback: the system does not play a long voice prompt (to avoid interfering with driving), but selectively plays a very short non-voice prompt sound (Earcon) (such as a "ding" sound or a specific frequency beep) to inform the user in the form of auditory symbol: "The system is online, but it is not convenient to respond now."

[0111] Silent lock: within a short time (such as 500ms) after triggering the fuse, the system is forced into a silent lock state, ignoring any new voice input, to prevent false triggering caused by the driver's unintentional exclamation or noise. The forced silent state directly skips the multi-level response decision logic, and the system only maintains state updates but does not perform any output actions during this period until the silent lock timer ends.

[0112] Second, if the preset safety fuse process is not triggered, when the environment improvement flag is in the active state (the driving intensity correlation factor of the current time window is less than the preset high-risk fuse threshold, and the driving intensity correlation factor of the previous time window is greater than the preset high-risk fuse threshold) ), it means that the environment is getting better, so even if the response decision value allows response, the system performs time window optimization, suppresses decision shock, and enters the pending state, which specifically includes:

[0113] If the environment improvement flag is in the active state, the system is forced into the pending (PENDING) state, and the response is temporarily suspended. This takes advantage of the predictive ability of the gradient, trying to exchange a small delay for a lower risk of misunderstanding (for example, waiting for the steering wheel to fully return to normal, or waiting for the last syllable of the sentence to land).

[0114] To dynamically adapt to the uncertainty of environmental changes, and also to prevent the system from being suspended indefinitely (deadlock) due to the environment being in a state of slight improvement for a long time (e.g. a slow deceleration process lasting for several seconds), when in the suspended state, if the environment improvement flag of the current time window is in the invalid state, it means that the original improvement trend has disappeared or become no longer significant, the suspended state needs to be ended, or if the duration of the suspended state exceeds the preset maximum greed threshold, the suspended state is ended and the final decision stage based on numerical game is entered.

[0115] In this example, the preset maximum greed threshold is 200 ms, and the setting of the threshold is based on the engineering trade-off between intent recognition accuracy and interaction fluency. In the complex scene of dynamic driving and voice interaction, an additional waiting window of 200 ms is sufficient to cover most of the transient recovery process of driving operations (such as rapid steering correction) and the short pause at the end of speech (such as swallowing or hesitation). Although this delay may be perceived by the user, in high-risk or high-uncertainty scenarios, the interaction experience of "slower but correct" is much better than that of "extremely fast but wrong / interrupted". Therefore, 200 ms is selected as an empirical upper limit, aiming to use this short time to significantly reduce the risk of misunderstanding, while avoiding causing an unbearable sense of interaction interruption.

[0116] Finally, if the preset safety fuse process is not triggered and the system is not in the suspended state, the system performs the final decision based on numerical game, and generates an end point detection completion signal based on the response decision value.

[0117] When the response decision value is less than or equal to 0, it means that the time pressure coefficient has overwhelmed the interaction inhibition coefficient, and the system immediately generates an end point detection completion signal (EOU, End of Utterance). This signal is sent to the downstream dialogue management system (DM) to trigger the final submission of the speech recognition result, completing a complete interaction loop.

[0118] And perform selective reset: reset the silence duration to 0 and set the semantic continuity probability to the default high bit, because a new round of dialogue means the restart of user waiting and semantic context.

[0119] The driving intensity correlation factor is retained, because the physiological state of the driver and the vehicle motion are objective physical processes that do not disappear instantly just because the voice interaction has ended. Retaining the driving intensity correlation factor means that if the driver has just experienced a high-load operation (the driving intensity correlation factor is high), even if the previous dialogue has ended, the system still "remembers" that the driver is in the recovery period at the moment when the next dialogue starts (k+1 time window). This allows the starting stage of the next round of interaction to directly inherit the high threshold protection without the need to accumulate vehicle data again, thereby achieving seamless safety protection for continuous dialogue scenarios.

[0120] After the selective reset, the system enters a standby listening state, in which the system only performs VAD detection and analyzes the driving intensity correlation factor, suspends various cost parameter calculations, flag updates, and multi-level decision-making.

[0121] It should be noted that, considering that the vehicle may continue to drive in a long curve or a bumpy road section, the driving intensity correlation factor may be high at this time, but it is not enough to trigger the fuse, and since the waiting cost is limited to an upper limit, the response decision value continues to be greater than 0, and the response is not triggered. Therefore, an absolute timeout threshold (for example, 5000 ms) is set, and when the silence duration after the user stops speaking exceeds the absolute timeout threshold, it means that the user has been waiting for too long, which has seriously exceeded the psychological expectation of normal conversation. At this time, the protection mechanism of the system becomes an interactive obstacle, and forced response or forced error is avoided to avoid logical deadlock.

[0122] One embodiment of the application also provides a semantic analysis-based conversational AI low-latency response control system, which comprises a memory, a processor and a computer program, wherein the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and the computer program can realize the semantic analysis-based conversational AI low-latency response control method described in steps S1-S3 when running in the processor.

[0123] In summary, in order to solve the technical problems of easy mis-truncation of voice interaction in complex driving conditions and response delay in smooth conditions, the application provides a semantic analysis-based conversational AI low-latency response control method and system. The application first generates a driving intensity correlation factor for the current time window according to the change of CAN bus data; further monitors the output of NLU and analyzes the acoustic intonation lifting feature to obtain the instruction incompleteness degree of the current time window; further fuses the instruction incompleteness degree and the driving intensity correlation factor to obtain an interaction suppression coefficient; further obtains a time pressure coefficient based on the silence duration after the user stops speaking; further compares the interaction suppression coefficient and the time pressure coefficient of the current time window to obtain a response decision value; further analyzes the trend of the interaction suppression coefficient to update the environment improvement flag; and finally, based on the driving intensity correlation factor of the current time window, the environment improvement flag and the response decision value, multi-level response decision control is performed, and selective reset of the interaction state is performed. The driving control and voice features are fused, the interruption cost and the waiting cost modulated by the load are calculated, and the multi-level control is combined with the trend. The driving control and voice features are fused, the interruption cost and the waiting cost are calculated, and the multi-level control is combined with the trend flag. The interruption risk modulated by the driving load is used to balance the waiting cost and the environmental change, the adaptive decision of the response opportunity is realized, and the mis-truncation and response delay problems of voice interaction in dynamic driving are solved.

[0124] It is to be noted that the sequential order of the above-described embodiments of the present application only for the purpose of description, but not the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0125] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments.

Claims

1. A semantic analysis-based conversational AI low-latency response control method, characterized by, The method comprises: After system restart, initialization is performed, CAN bus data of the vehicle is acquired and divided in preset time windows; according to changes in the CAN bus data, vehicle operation intensity of the current time window is acquired; based on the vehicle operation intensity, a driving intensity correlation factor of the current time window is generated; After detecting a first voice signal, the output of NLU is monitored, and the acoustic tone rising feature is analyzed to acquire instruction incompleteness of the current time window; the instruction incompleteness and the driving intensity correlation factor are fused to acquire an interaction inhibition coefficient; based on the silence duration after the user stops speaking, a time pressure coefficient is acquired; the interaction inhibition coefficient and the time pressure coefficient of the current time window are compared to acquire a response decision value; the change trend of the interaction inhibition coefficient is analyzed to update an environment improvement flag; According to the driving intensity correlation factor, the environment improvement flag and the response decision value of the current time window, multi-level response decision control is performed, and selective resetting of the interaction state is performed; The method for acquiring the vehicle operation intensity comprises: the CAN bus data at least includes longitudinal acceleration, steering wheel angle and real-time vehicle speed; for the current time window, a first operation intensity is acquired according to the change rate of the longitudinal acceleration and the change rate of the steering wheel angle between adjacent time windows; a second operation intensity is acquired by fusing the absolute value of the steering wheel angle and the real-time vehicle speed; the first operation intensity and the second operation intensity are fused to acquire the vehicle operation intensity; The method for acquiring the instruction incompleteness comprises: the output of NLU is monitored to acquire semantic continuity probability; the fundamental frequency sequence of the voice data of a preset historical neighborhood of the current time window is extracted, and according to the change trend of the fundamental frequency sequence, acoustic tone rising degree is acquired; the semantic continuity probability and the acoustic tone rising degree are fused to acquire the instruction incompleteness of the current time window; The method for acquiring the driving intensity correlation factor comprises: taking the product of the vehicle operation intensity of the adjacent last time window and a preset attenuation coefficient, and the maximum value in the vehicle operation intensity of the current time window, as the driving intensity correlation factor.

2. The low-latency conversational AI response control method based on semantic analysis according to claim 1, characterized in that, The method for acquiring the semantic continuity probability comprises: The output of NLU is monitored, if there is output in the current time window or the duration of stopping output does not exceed a preset first time limit, the semantic continuity probability is marked as a preset first parameter; if the duration of stopping output corresponding to the current time window exceeds the preset first time limit, the semantic continuity probability is acquired by linearly attenuating the preset first parameter based on the exceeding duration.

3. The low-latency conversational AI response control method based on semantic analysis according to claim 1, characterized in that, The method for acquiring the environment improvement flag comprises: The interaction inhibition coefficient of the current time window is compared with that of a preset historical period to acquire a decision trend gradient; the decision trend gradient is compared with a preset gradient improvement threshold to update the environment improvement flag.

4. The low-latency conversational AI response control method based on semantic analysis according to claim 1, wherein, The method for performing multi-level response decision control comprises: If the driving intensity correlation factor of the current time window is greater than a preset high-risk fuse threshold, a preset safety fuse process is triggered; If the preset safety fuse process is not triggered, when the environment improvement flag is in an effective state, the system enters a suspended state; If the preset security fusing process is not triggered and the system is not in the suspended state, an endpoint detection completion signal is generated based on the response determination value.

5. The low latency response control method of conversational AI based on semantic analysis according to claim 4, characterized in that, If the environment improvement flag of the current time window is in the invalid state or the duration of the suspended state exceeds the preset maximum greed threshold, the suspended state is ended.

6. The low-latency conversational AI response control method based on semantic analysis according to claim 1, wherein, The method for performing the selective reset comprises: The mute duration is reset to 0, and the driving intensity correlation factor is retained.

7. A semantic analysis based conversational AI low latency response control system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the dialog AI low-latency response control method based on semantic analysis according to any one of claims 1-6 when executing the computer program.

Images