A vehicle-mounted voice interaction control method and device, a vehicle, and a storage medium

Abstract

This invention belongs to the field of in-vehicle intelligent interaction technology, and particularly relates to an in-vehicle voice interaction control method, device, vehicle, and storage medium. The method includes: responding to a voice interaction event to be executed, acquiring the current scene risk level in real time; the voice interaction event is a passive interaction event or an active interaction event; for the passive interaction event, determining execution parameters based on the scene risk level and the importance level of the user's voice command; for the active interaction event, generating a voice reminder based on the scene risk level and driving behavior analysis results, and determining reminder parameters; controlling the execution of the voice interaction event based on the execution parameters or the reminder parameters. This invention solves the problem in the prior art that voice interaction systems cannot dynamically adjust interaction strategies according to real-time driving conditions, improving driving safety and interactive experience.

Description

Technical Field

[0001] This invention relates to the field of in-vehicle intelligent interaction technology, and in particular to an in-vehicle voice interaction control method, device, vehicle, and computer-readable storage medium. Background Technology

[0002] Currently, existing in-vehicle voice interaction systems mainly employ the following implementation method: After receiving a user's voice command, the system recognizes the command content through a voice recognition module and then executes the corresponding operation according to the preset command category; for the system's proactive reminder function, it generates and broadcasts voice prompts based on preset trigger conditions. Throughout the interaction process, the system's response strategy is relatively fixed and cannot dynamically adjust the voice interaction strategy according to the real-time driving status. This may lead to the execution of non-critical commands in high-risk scenarios, distracting the driver and posing safety hazards. Summary of the Invention

[0003] In view of this, the present invention provides an in-vehicle voice interaction control method, device, vehicle, and computer-readable storage medium to realize dynamic control of voice interaction based on real-time driving status perception, thereby improving driving safety and interactive experience.

[0004] To achieve the above objectives, a first aspect of the present invention provides an in-vehicle voice interaction control method, comprising: In response to pending voice interaction events, the current scenario risk level is obtained in real time; the voice interaction event is a passive interaction event or an active interaction event, the passive interaction event is an interaction event that responds to the user's voice command, and the active interaction event is a system-initiated voice reminder interaction event triggered based on driving behavior analysis results; For the passive interaction event, the execution parameters are determined based on the scenario risk level and the importance level of the user's voice command; For the aforementioned proactive interaction event, a voice prompt is generated based on the scenario risk level and driving behavior analysis results, and the prompt parameters are determined; The execution of the voice interaction event is controlled according to the execution parameters or the reminder parameters.

[0005] Preferably, obtaining the current scenario risk level includes: collecting vehicle dynamics data, driver status data, and environmental data; and determining the scenario risk level based on the vehicle dynamics data, the driver status data, and the environmental data.

[0006] Preferably, the triggering of the proactive interaction event includes: detecting poor driving behavior based on driving behavior data and determining the confidence level of the behavior; when the confidence level of the behavior meets a preset threshold, triggering a proactive voice reminder interaction event of the system.

[0007] Preferably, determining the importance level of the instruction includes: receiving a user's voice instruction and recognizing the user's intent; and determining the importance level of the instruction based on at least one of the intent category, voice urgency characteristics, and a preset instruction level mapping relationship.

[0008] Preferably, the execution parameter is immediate execution, conditional triggering, or rejection of execution; if the execution parameter is conditional triggering, the user voice command is stored in the first cache queue; the scenario risk level is continuously monitored, and when the scenario risk level meets the preset conditions, it is extracted from the first cache queue and executed.

[0009] Preferably, the reminder parameters include a reminder decision and a reminder format; the reminder decision includes immediate triggering or conditional triggering; the reminder format includes at least one of the following: broadcast speed, tone, and reminder duration; if the reminder decision is conditional triggering, the system's proactive voice reminder is stored in a second cache queue; the scenario risk level is continuously monitored, and when the scenario risk level meets preset conditions, it is retrieved from the second cache queue; reminders of the same type are merged and output, and reminders that have exceeded a preset validity period are discarded.

[0010] Preferably, the closed-loop optimization includes: employing a reinforcement learning model, using the scenario risk level, driving behavior category, user satisfaction, and system response quality as state inputs, and the adjustment amount of the execution parameters and the reminder parameters as action outputs; and continuously optimizing the policy network through a reward function.

[0011] A second aspect of the present invention provides an in-vehicle voice interaction control device, comprising: The risk identification module is configured to respond to pending voice interaction events and obtain the current scene risk level in real time; the voice interaction event is a passive interaction event or an active interaction event, the passive interaction event is an interaction event that responds to the user's voice command, and the active interaction event is a system active voice reminder interaction event triggered based on driving behavior analysis results; The parameter determination module is configured to determine execution parameters for the passive interaction event based on the scenario risk level and the importance level of the user's voice command; and to generate a voice reminder and determine reminder parameters for the active interaction event based on the scenario risk level and driving behavior analysis results. The execution control module is configured to control the execution of the voice interaction event based on the execution parameters or the reminder parameters.

[0012] A third aspect of the invention provides a vehicle including a processor and a memory, the memory storing computer instructions that, when executed by the processor, cause the vehicle to perform the method described in any of the first aspects above.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the first aspects above.

[0014] One or more of the above technical solutions achieve precise responses by sensing the risk level of the scene in real time and determining the execution parameters based on the importance level of the instruction and the risk level of the scene for passive interaction events. For active interaction events, the solutions determine the reminder parameters based on the driving behavior analysis results and the risk level of the scene, ensuring that the timing and form of the reminder are appropriate for the driving scene. In high-risk scenarios, the solutions can automatically reject or delay the execution of non-critical voice commands, avoid distracting the driver's attention with voice interaction, and reduce the risk of traffic accidents. Attached Figure Description

[0015] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0016] Figure 1 This is a schematic diagram illustrating an example environment that can be implemented according to the embodiments of the present invention; Figure 2 A flowchart illustrating the in-vehicle voice interaction control method provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the process of determining execution / reminder parameters based on the scenario risk level for both passive and active interaction events, as described in this embodiment of the invention. Figure 4 This is a schematic diagram illustrating the process of adjusting the voice interaction strategy in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the in-vehicle voice interaction control device provided in an embodiment of the present invention. Detailed Implementation

[0017] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0018] In the description of embodiments of this application, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] Existing in-vehicle voice interaction systems receive user voice commands, recognize the command content through a voice recognition module, and then execute corresponding operations based on preset command categories. For system-initiated reminder functions, voice prompts are generated and broadcast based on preset trigger conditions. Throughout the interaction process, the system's response strategy is relatively fixed and does not dynamically adjust according to the current driving state. This may lead to the execution of non-critical commands in high-risk scenarios, distracting the driver. For example, it may still perform operations such as music switching during emergency braking, high-speed driving, or when the driver is distracted.

[0020] Based on this, this application provides an in-vehicle voice interaction control method, which distinguishes between passive interaction events and active interaction events, and establishes an independent dynamic control mechanism for each type of event, thereby realizing dynamic control of voice interaction strategies based on real-time driving state perception and solving the above-mentioned technical defects.

[0021] Figure 1 A schematic diagram of an example environment 100 in which various embodiments of the present invention can be implemented is shown. For example... Figure 1 As shown, the example environment 100 includes a vehicle 110, a data acquisition module 120, and an on-board domain controller 130.

[0022] Vehicle 110 is a new energy vehicle equipped with a data acquisition module 120. The data acquisition module 120 includes an in-vehicle array microphone, a DMS driver monitoring system camera, a CAN bus interface, and environmental sensors, used to collect voice signals, driver status data, vehicle dynamics data, and environmental data. Specifically, the in-vehicle array microphone is used to collect driver voice commands and environmental noise, with a sampling rate of 16kHz, quantization accuracy of 16bit, support for active noise cancellation, and a pickup distance of 0.5–2 meters; the DMS camera is used to monitor the driver's facial orientation, line of sight, fatigue level, and distraction level in real time; the CAN bus interface is used to acquire vehicle dynamics data, including vehicle speed, acceleration, steering wheel angle, and braking force. For example, the vehicle speed sampling frequency is 10Hz, the acceleration sampling frequency is 50Hz, the steering wheel angle sampling frequency is 20Hz, and the braking force sampling frequency is 20Hz; the environmental sensors are used to acquire environmental data such as weather type, road condition level, and light intensity, with a sampling frequency of 5Hz.

[0023] The vehicle domain controller 130 is an in-vehicle edge computing unit, which can also be expanded into a cloud server or edge computing node. It receives data collected by the data acquisition module 120 and executes the in-vehicle voice interaction control method of this application. The vehicle domain controller 130 is equipped with an in-vehicle AI chip for locally deploying deep learning models and achieving real-time inference. For example, the in-vehicle AI chip can be an NVIDIA Xavier with a computing power of 30 TOPS; the deep learning models include a speech recognition model, a driving behavior analysis model, and a reinforcement learning model. The data acquisition module 120 communicates with the vehicle domain controller 130 via in-vehicle Ethernet, Wi-Fi, or Bluetooth to ensure the real-time performance and reliability of data transmission.

[0024] In this application embodiment, data collection and processing strictly comply with relevant laws and regulations. The collection of driver voice, facial images, vehicle location, and other data requires explicit user authorization and is used solely for in-vehicle voice interaction control and driving behavior optimization, not for any other unauthorized purposes. Voice data and driver status data are processed locally, and differential privacy technology is used to anonymize data uploaded to the cloud. Sensitive data is stored encrypted. For example, voiceprint information is stored encrypted, complying with GDPR and other regulatory requirements.

[0025] It should be understood that the example environment 100 is described for illustrative purposes only and does not imply any limitation on the scope of the invention.

[0026] Figure 2 A flowchart illustrating the in-vehicle voice interaction control method provided in an embodiment of this application is shown. Figure 2 As shown, the method includes the following steps: In box 210, in response to pending voice interaction events, the current scenario risk level is obtained in real time. Voice interaction events refer to events that trigger the voice interaction process in the in-vehicle voice interaction system, including passive and active interaction events. Passive interaction events are those that respond to user voice commands, initiated by the user and passively responded to by the system. For example, if the user says "Navigate to the nearest charging station," the system will recognize the command and perform navigation adjustments. Active interaction events are those triggered by the system's proactive voice reminders based on driving behavior analysis results, initiated by the system and passively received by the user. For example, if the system detects frequent rapid acceleration by the driver, it will proactively announce, "Frequent rapid acceleration detected; smooth driving is recommended."

[0027] For example, the in-vehicle array microphones collect user voice commands, perform voice recognition, and trigger passive interaction events; when the driving behavior analysis module detects bad driving behavior and the behavior confidence level meets a preset threshold, it triggers active interaction events.

[0028] In some embodiments, after receiving a voice interaction event trigger signal, the vehicle domain controller 130 first determines the event type. If the signal originates from voice acquisition by the in-vehicle array microphone, it is determined to be a passive interaction event; if the signal originates from the result of poor driving behavior detection by the driving behavior analysis module, it is determined to be an active interaction event.

[0029] The scenario risk level refers to the degree of risk of the current driving scenario as determined by a comprehensive assessment of vehicle dynamics data, driver status data, and environmental data.

[0030] In some embodiments, the vehicle domain controller 130 can acquire vehicle dynamics data via the CAN bus, driver status data via the DMS, and environmental data via onboard sensors. The vehicle dynamics data includes vehicle speed, acceleration, steering wheel angle, and braking force; the driver status data includes facial orientation, line of sight, fatigue level, and distraction level; and the environmental data includes weather type, road condition level, and light intensity. Based on multiple conditions from the above data, a scenario risk level is determined; when multiple conditions are simultaneously met, the highest risk level is taken as the current scenario risk level.

[0031] For example, a low-risk scenario requires one of the following conditions to be met: vehicle speed between 60-90 km / h and smooth road conditions; the vehicle is stationary or parked, i.e., speed = 0; the driver is attentive and there is no sudden acceleration / braking. A medium-risk scenario requires one of the following conditions to be met: vehicle speed between 30-60 km / h and slow-moving road conditions; moderate driver distraction; low visibility. A high-risk scenario requires one of the following conditions to be met: vehicle speed ≥ 90 km / h; braking force ≥ 0.7G, i.e., emergency braking; high driver distraction; traffic congestion or construction; severe weather.

[0032] This enables the determination of scenario risk levels based on multi-source data fusion, covering vehicle dynamics, driver status, and environmental factors, thereby improving the comprehensiveness of risk perception.

[0033] In box 220, the execution / alert parameters are determined based on the scenario risk level for both passive and active interaction events.

[0034] Specifically, in box 2201, the vehicle domain controller 130 can determine the importance level of a voice command issued by the user 120, and determine the execution parameters based on the current scenario risk level. These execution parameters include immediate execution, conditionally triggered execution, or rejection of execution.

[0035] In some embodiments, the vehicle domain controller 130 first performs acoustic recognition of the user's voice commands using a locally deployed speech recognition model and outputs the command text. This speech recognition model employs a Transformer end-to-end architecture and consists of an encoding network, a decoding network, and a noise suppression subnetwork.

[0036] The encoding network consists of a multi-layer encoder, with each layer configured with a multi-head attention mechanism, hidden layers, and a feedforward neural network. Layers are connected via residual connections and layer normalization. The decoding network consists of a multi-layer decoder, with a structure largely consistent with the encoder, but with the addition of a masked multi-head attention layer to achieve autoregressive decoding. The noise suppression subnetwork employs a hybrid CNN and LSTM architecture. The CNN is used to extract noise features, and the LSTM is used to model temporal dependencies, connecting to the encoding network through feature concatenation. Specifically, the encoding network is a 6-layer encoder, with each layer containing 8 multi-head attention heads, a 512-dimensional hidden layer, and a feedforward neural network. The feedforward neural network has a dimension of 2048 and uses ReLU activation. The decoding network is a 6-layer decoder, with each layer having the same structure as the encoder, but with the addition of a masked multi-head attention layer. In the noise suppression subnetwork, the CNN has 3 layers with a 3×3 kernel size, a stride of 1, and padding of 1; the LSTM has 2 layers with a hidden layer dimension of 256.

[0037] The model training utilizes a large-scale in-vehicle voice dataset, covering various noise scenarios and new energy vehicle-specific command categories. The training process includes data preprocessing, model initialization, optimizer configuration, and scenario-adaptive fine-tuning to ensure the model's recognition performance in driving scenarios. The model is deployed locally on the in-vehicle AI chip to meet the real-time interaction needs during driving. For example, the training dataset contains 100,000 hours of in-vehicle voice data, covering eight noise scenarios such as high-speed wind noise and urban traffic noise, and annotating more than 2,000 new energy vehicle-specific command categories. Data preprocessing includes: pre-emphasis of the voice signal with a filter coefficient of 0.97; frame segmentation with a frame length of 20ms and a frame shift of 10ms; Mel-frequency conversion to output 40-dimensional Mel coefficients; and normalization of the noise data. The model training used Xavier uniform distribution for parameter initialization, with AdamW as the optimizer, an initial learning rate of 1e-4, weight decay of 0.01, a batch size of 32, and 100 training epochs. An early stopping strategy was employed: training stopped if the validation set accuracy showed no improvement for five consecutive epochs. The loss function was cross-entropy loss. In the scenario adaptation fine-tuning phase, 50,000 voice data points specific to new energy vehicle driving scenarios were added, with a fine-tuning learning rate of 1e-5 and 20 training epochs.

[0038] The speech signal is first input into the noise suppression subnetwork. The CNN performs local perception of the time-frequency features of the speech signal, extracting noise pattern features at different scales layer by layer; the LSTM receives the feature sequence output by the CNN, models the dynamic changes of noise in time, and outputs the noise estimation features of the current frame. This noise estimation feature is concatenated with the original speech features in the channel dimension to form an enhanced input representation, which is then fed into the encoding network.

[0039] The encoding network receives the enhanced input representation and processes it layer by layer through a multi-layer encoder. Each encoder layer first uses a multi-head attention mechanism to model the global dependencies of the input sequence, capturing long-distance acoustic correlations in the speech signal; then, a feedforward neural network performs a nonlinear transformation on the attention output to extract high-order semantic features; residual connections between layers preserve the original information pathways, and layer normalization stabilizes the training process. The encoding network ultimately outputs a high-level feature representation containing global contextual information.

[0040] The decoding network generates text sequences autoregressively through multiple decoders, conditioned on the high-level feature representations output by the encoding network. Each decoder layer first performs self-attention modeling on the generated text sequence using a masked multi-head attention mechanism, ensuring that the decoding process relies only on historical information. Then, a cross-attention mechanism is used to introduce the high-level features of the encoding network into the decoding process, achieving a mapping from acoustic features to text features. Finally, a feedforward neural network performs feature transformation, outputting the text prediction probability distribution for the current step. The decoding process iterates progressively until a complete instruction text is generated.

[0041] The model is deployed locally on the vehicle's AI chip. During inference, the speech signal is input in a streaming manner, the noise suppression subnetwork and the encoding network are processed in parallel in a frame-by-frame manner, and the decoding network outputs the text in an incremental manner. The overall recognition latency is no more than 300ms.

[0042] Subsequently, the vehicle domain controller 130 determines the importance level of the command based on the intent category and / or voice urgency characteristics. After obtaining the command text, the system first determines the user's intent category through an intent recognition model. This intent recognition model is implemented based on a text classification network, exemplarily using a BERT or TextCNN architecture. The input is the command text output from speech recognition, and the output is an intent category label. Exemplarily, intent categories include safety, navigation, entertainment, communication, and comfort.

[0043] The model training includes the following steps: Dataset construction: collecting user command texts from new energy vehicle driving scenarios, covering five intent categories: safety, navigation, entertainment, communication, and comfort, and labeling over 2000 exclusive command samples; Data preprocessing: segmenting the command text, removing stop words, and converting word embeddings, for example using Word2Vec or BERT pre-trained word vectors; Model training: initializing network parameters, configuring the optimizer and learning rate, setting the batch size and training epochs, using the cross-entropy loss function, and updating parameters through backpropagation; Scenario adaptation: adding colloquial expressions from driving scenarios for fine-tuning to improve the ability to recognize non-standard commands. Specifically, the dataset contains over 2000 new energy vehicle-specific commands, such as "emergency assistance" and "collision warning confirmation" for safety, "navigate to the nearest charging station" and "avoid congested roads" for navigation, "switch to the next song" and "play Jay Chou's songs" for entertainment, "answer a call" and "dial 10086" for communication, and "turn on the air conditioner" and "adjust the seat heating" for comfort. Data preprocessing used BERT pre-trained word vectors with a dimension of 768. Model training employed the Adam optimizer with an initial learning rate of 2e-5, a batch size of 16, 10 training epochs, and a validation set comprising 20% ​​of the dataset, using an early stopping strategy. During the scene adaptation phase, 500 colloquial samples were added, the learning rate was fine-tuned to 1e-5, and training continued for 5 epochs.

[0044] Subsequently, the system determines the importance level of the instruction based on intent category labels, voice urgency features, and a preset instruction level mapping relationship. Specifically, the system first determines the basic level based on intent category labels: safety-related intents correspond to level one, navigation, communication, and comfort-related intents correspond to level two, and entertainment and casual conversation intents correspond to level three. Based on this, the system extracts the urgency features of the original voice signal, including speech rate, volume, and keywords. If the speech rate is greater than 5 words per second, the volume is greater than 70 decibels, or the system contains keywords such as "urgent," "quick," or "help me," the basic level is increased by one level. Simultaneously, the system queries a preset instruction level mapping table, which contains pre-labeled levels for over 2000 new energy vehicle-specific instructions. If the mapping table result differs from the aforementioned result, the highest level is taken as the final instruction importance level.

[0045] Finally, the vehicle domain controller 130 queries a preset execution parameter matrix based on the combination of scenario risk level and instruction importance level to determine the execution parameters: in low-risk scenarios, all levels of instructions are executed immediately; in medium-risk scenarios, safety-critical instructions are executed immediately, and non-critical instructions are executed conditionally; in high-risk scenarios, safety-critical instructions are executed immediately, and non-critical instructions are refused execution. See Table 1 for details.

[0046] Table 1 Example of an instruction parameter matrix

[0047] If the execution parameter is set to immediate execution, the system immediately executes the user's voice command and announces the complete execution result. For example, "Navigation has been adjusted to the nearest charging station."

[0048] If the execution parameter is conditionally triggered, the system stores the user's voice command in the first cache queue and continuously monitors the scene risk level. When the scene risk level meets a preset condition, the system retrieves the command from the first cache queue and executes it. The preset condition is a downgrade from a medium-risk or high-risk scene to a safe scene. The capacity of the first cache queue is set to 5 entries, and the validity period of each command is set to 60 seconds, after which it is automatically discarded. The system checks the scene risk level every 500ms. When the condition is met, commands are sorted according to their priority level, with level 1 commands taking precedence over level 2 commands, and then dequeued and executed in sequence.

[0049] If the execution parameter is "refuse to execute," the system silently discards the instruction without making any sound to avoid distracting the driver. Once the scenario's risk level has been reduced to a safe level, the system silently prompts on the in-vehicle display, "The music switching request was ignored. Do you want to execute it now?" The user can confirm whether to execute the request via touchscreen or voice.

[0050] Therefore, by establishing a two-dimensional decision-making mechanism, the system can prioritize driving safety in high-risk scenarios, refuse or delay the execution of non-critical instructions, and at the same time respond normally to user needs in safe scenarios, thereby improving the accuracy of interaction strategies and driving safety.

[0051] In frame S2202, the vehicle domain controller 130 generates a voice prompt based on the scenario risk level and driving behavior analysis results for proactive interaction events, and determines the prompt parameters. These prompt parameters include the prompt decision and the prompt format.

[0052] In some embodiments, the vehicle domain controller 130 performs temporal feature modeling on driving behavior data through a driving behavior analysis module, detects undesirable driving behaviors, and outputs the behavior category and confidence level. This module uses a sliding window mechanism to organize the input data, combines a temporal neural network for feature modeling, and maps specific undesirable driving behavior types through a rule engine.

[0053] For example, vehicle dynamics data is continuously received with an analysis window of 30 seconds and a step size of 10 seconds. Each window contains 600 frames of data (30 seconds × 20Hz), covering six types of raw driving parameters: vehicle speed, longitudinal acceleration, lateral acceleration, steering wheel angle, steering wheel angular velocity, and brake pedal travel. The system simultaneously calculates six types of derived statistical features, including the rate of change of longitudinal acceleration, the rate of change of lateral acceleration, the rate of change of steering wheel angular velocity, the standard deviation of vehicle speed, the standard deviation of acceleration, and the standard deviation of steering wheel angle, forming a 12-dimensional input vector.

[0054] For example, the temporal neural network adopts an LSTM network architecture with an input layer dimension of 12, receiving a temporal sequence composed of the original parameters and derived features mentioned above; the hidden layer has two layers, each with 256 dimensions, a dropout rate of 0.3, and fully connected between layers; the output layer has three labels, including good, average, and poor, and the activation function is Softmax. The network updates the hidden state frame by frame for 600 frames of temporal data within a window, and finally outputs the window-level behavior classification result. The dataset contains driving behavior data of 5000 drivers, each data point being a continuous 30-second sequence of driving parameters, with a sampling frequency of 20Hz, and the behavior categories are manually labeled. Training uses the Adam optimizer with an initial learning rate of 1e-3, a batch size of 64, 80 training epochs, and a cross-entropy loss function, with a validation set accounting for 20%.

[0055] When the probability of the "bad" category in the Softmax output is ≥0.7, the system triggers the rule engine to map specific behavior types. The rule engine determines the behavior based on feature thresholds within the window: longitudinal acceleration ≥0.3G lasting more than 2 seconds is considered rapid acceleration; lane changes ≥3 times per minute are considered frequent lane changes; braking force ≥0.6G is considered emergency braking; and continuous driving time ≥4 hours without significant breaks is considered fatigued driving. If the "bad" probability is between 0.5 and 0.7, the system marks it as a suspected state and includes it in the next window for cumulative judgment.

[0056] The vehicle domain controller 130 dynamically matches reminder content based on the type of undesirable driving behavior and the risk level of the scenario. In some embodiments, the system queries a preset voice reminder text library based on the type of undesirable driving behavior and generates corresponding voice reminder content. The voice reminder text library establishes a one-to-one correspondence between "undesirable driving behavior type - voice reminder text". For example: When acceleration ≥ 0.3G, in a safe scenario, the reminder is "Frequent rapid acceleration detected, it is recommended to drive smoothly, which can save about 15% of battery power"; in a medium-risk scenario, the reminder is "It is recommended to reduce rapid acceleration, pay attention to safety in the current road conditions". Frequent lane changes: When lane changes ≥ 3 times / minute, in a safe scenario, the reminder is "Frequent lane changes detected, it is recommended to stay in your lane and reduce fatigue"; in a medium-risk scenario, the reminder is "The road conditions ahead are complex, it is recommended to reduce lane changes". When braking force ≥ 0.6G, in a safe scenario, the reminder is "Sudden braking detected, it is recommended to maintain a safe distance"; in a medium-risk scenario, the reminder is "Please pay attention to maintaining a distance and reduce sudden braking". Driving ≥ 4 hours, in a safe scenario, the reminder is "You have been driving continuously for 4 hours, it is recommended to rest nearby"; in a medium-risk scenario, the reminder is "Long-term driving can easily lead to fatigue, pay attention to safety".

[0057] In some embodiments, the vehicle domain controller 130 determines alert parameters based on the scenario risk level and driving behavior analysis results. The alert parameters include alert decision and alert format.

[0058] The alert decision includes immediate triggering, conditional triggering, and temporary non-triggering. Low-risk scenarios trigger immediately, medium-risk scenarios trigger conditionally, and high-risk scenarios do not trigger temporarily.

[0059] If the reminder decision is to be triggered immediately, the system will immediately broadcast a voice reminder, which will last for ≤8 seconds and contain complete information.

[0060] If the alert decision is conditionally triggered, the system stores the voice alert in a second cache queue, continuously monitoring the scenario's risk level. The second cache queue has a capacity of 3 entries, and each alert has a validity period of 5 minutes, automatically discarding those that expire to avoid broadcasting outdated information. The system continuously monitors the scenario's risk level, and when the scenario's risk level meets preset conditions, it retrieves the alert from the second cache queue. The preset conditions are a reduction from a medium-risk or high-risk scenario to a safe scenario, and a vehicle speed fluctuation of ≤5km / h for 3 consecutive seconds. If similar alerts exist in the queue, the system merges them to generate a comprehensive alert. For example, if multiple instances of rapid acceleration are detected, it is merged into "Multiple instances of rapid acceleration and sudden braking have been detected during this trip; smooth driving is recommended." Alerts exceeding the preset validity period of 5 minutes are automatically discarded and will not be broadcast again.

[0061] If the notification decision is conditionally triggered, it will be entered into the second cache queue. The system will not broadcast it until the scenario risk level is reduced to medium risk or safe.

[0062] The reminder format includes at least one of the following: broadcast speed, tone, and reminder duration.

[0063] In safe scenarios, the broadcast duration is ≤8 seconds, the speech rate is a normal speech rate of 4 words / second, the tone is a gentle suggestion, and the content is complete. For example, "Frequent rapid acceleration has been detected. We recommend driving smoothly, which can save approximately 15% of battery power."

[0064] In medium-risk scenarios, the broadcast duration should be ≤4 seconds, the speaking speed slightly faster at 5 words per second, the tone should be a prompt, and the content should be the core prompt. For example, "It is recommended to reduce sudden acceleration and pay attention to safety."

[0065] In high-risk scenarios, the announcement will not be broadcast until the risk decreases.

[0066] This allows the timing of proactive interactive events to be matched with the risks of driving scenarios, and the content and form of reminders to be dynamically generated, thereby improving the acceptance of reminders and avoiding interference from broadcasts during dangerous moments.

[0067] In box 230, the execution of voice interaction events is controlled according to execution parameters or reminder parameters. Specifically, this includes: in box 2301, the vehicle domain controller executes corresponding instructions based on the execution parameters determined in box 2201.

[0068] If the execution parameter is set to immediate execution, the system will immediately execute the user's voice command and announce the complete execution result. For example, if the user issues the command "Navigate to the nearest charging station", the system will immediately adjust the navigation route and announce "Navigation adjusted to the nearest charging station, 3 kilometers away, estimated arrival time 5 minutes".

[0069] If the execution parameter is conditionally triggered, the system stores the user's voice command in the first cache queue and continuously monitors the scene risk level. When the scene risk level meets the preset conditions, it is downgraded from a medium-risk or high-risk scene to a safe scene. The system checks every 500ms, sorts the commands in the first cache queue according to their priority level (first-level commands take precedence over second-level commands), and then retrieves and executes the commands sequentially. During execution, the system announces a simplified execution result to avoid lengthy announcements that might distract the user. For example, "Navigation has been adjusted."

[0070] If the execution parameter is "refuse to execute," the system silently discards the instruction without making any sound. Once the scenario's risk level has been reduced to a safe scenario, the system silently prompts on the in-vehicle display screen, "The music switching request was ignored. Do you want to execute it now?" The user can confirm whether to execute via screen touch or voice.

[0071] In box 2302, the vehicle domain controller 130 outputs a reminder voice based on the reminder parameters determined in box 2202.

[0072] If the alert decision is triggered immediately, the system will immediately broadcast a voice reminder, with the broadcast duration, speed, and tone dynamically adjusted according to the risk level of the scenario. For example, in a safe scenario, the system will broadcast "Frequent rapid acceleration detected; smooth driving is recommended, which can save approximately 15% of battery power," with a broadcast duration of ≤8 seconds, a speed of 4 words per second, and a gentle tone.

[0073] If the alert decision is conditionally triggered, the system stores the voice alert in a second cache queue and continuously monitors the scenario's risk level. When the scenario's risk level meets preset conditions—that is, it is downgraded from a medium-risk or high-risk scenario to a safe scenario—and the vehicle speed fluctuation is ≤5km / h for 3 consecutive seconds, the system retrieves the alert from the second cache queue. If similar alerts exist in the queue, the system merges them and outputs a comprehensive alert. For example, "This trip has detected multiple instances of rapid acceleration and sudden braking; we recommend driving smoothly." Alerts exceeding the preset validity period of 5 minutes are automatically discarded and will not be broadcast again.

[0074] This enables voice interaction event execution control based on execution parameters and reminder parameters, ensuring that the voice interaction strategy is adapted to the driving scenario.

[0075] In some embodiments, the above method further includes: adjusting the voice interaction strategy based on real-time behavioral feedback, such as... Figure 5 As shown, this method can be implemented based on the above embodiments to achieve continuous optimization of the voice interaction strategy.

[0076] Specifically, in box 2401, a voice interaction parameter group is set for each driving scenario type. The driving scenario types include at least highway cruising, urban congestion, and general driving. Each scenario type corresponds to an independent parameter group, which includes at least a response duration parameter, a broadcast content parameter, a recognition threshold parameter, and a response priority parameter. This isolates the adjustments for different scenarios, avoiding policy distortion caused by global averaging.

[0077] In box 2402, real-time behavioral feedback metrics are collected during voice interaction. These metrics are automatically calculated based on system interaction logs. The metrics include at least: Voice conflict rate: the percentage of times the vehicle is detected to accelerate suddenly, brake suddenly, or turn sharply during or within a preset time after voice broadcast; Command repetition rate: the percentage of times the user repeatedly issues the same type of command in the current driving scenario out of the total number of such commands; Active interruption rate: the percentage of times the user interrupts the broadcast by pressing a button or using voice within the total number of broadcasts; Silence timeout rate: the percentage of times the interaction times out due to the user not responding within a preset waiting time after the system issues a voice command out of the total number of voice outputs.

[0078] In box 2403, when the accumulated interaction data in a certain driving scenario reaches a preset statistical value, the associated parameters in the parameter group corresponding to that scenario are fine-tuned stepwise based on the deviation between each real-time behavioral feedback indicator and its corresponding threshold. The preset statistical value can be the number of valid interaction sessions accumulated in the same scenario, such as 10 sessions. The stepwise fine-tuning means adjusting only one associated parameter at a preset step size each time to avoid parameter jumps, as shown in Table 2.

[0079] Table 2 Examples of Adjustment Actions and Adjustment Steps

[0080] In other embodiments, the adjustment action may further include: when the voice conflict rate is greater than a first threshold, increasing the response priority threshold for that scenario, making it more difficult for non-critical voice requests to be triggered; and when the active interruption rate is greater than a third threshold, shortening the response time parameter for that scenario. This allows for precise problem localization and targeted adjustment based on real-time behavioral feedback, avoiding confusion in attribution.

[0081] In box 2404, the adjusted parameters are validated using scene safety constraint rules. These rules include: when the current driving state is emergency braking, all non-critical voice outputs are forcibly blocked, and no parameter adjustments are permitted in this state; each parameter has preset absolute upper and lower limits, and step adjustments must not exceed these limits; if the previous session executed an adjustment in the first direction due to an abnormal first indicator, then in the next session, an adjustment in the opposite direction must not be executed immediately due to an abnormal opposite indicator, and at least one session cycle must be allowed between adjustments.

[0082] In box 2405, the parameters that have passed the verification are updated to the parameter group corresponding to this scene and used for subsequent voice interaction control in the same scene.

[0083] It should be understood that the above-mentioned real-time behavioral feedback metrics are merely examples. In specific implementations, other interactive performance metrics that can be automatically extracted from system logs may also be included, such as voice wake-up false trigger rate and multi-turn dialogue interruption rate. The embodiments of this disclosure are not limited to specific metric types.

[0084] In some embodiments, the method further includes anti-jitter design: adjustment is only performed when the same metric exceeds a corresponding threshold in multiple consecutive sessions. This avoids system jitter caused by abnormal data on a single day.

[0085] In other embodiments, the method further includes a parameter rollback mechanism: when the adjusted parameters cause the immediate behavioral feedback indicators in the scenario to deteriorate, the method automatically rolls back to the previously effective parameter value and reduces the subsequent adjustment step size. This ensures the robustness of the strategy adjustment.

[0086] Using the voice interaction strategy adjustment method based on real-time behavioral feedback according to embodiments of this disclosure, on the one hand, the strategy adjustment can be triggered immediately after the session ends through automatically calculated real-time behavioral indicators, significantly shortening the response cycle. On the other hand, through scenario-specific independent parameter groups and single-indicator single-parameter mapping, the strategies for high-speed cruising scenarios and urban congestion scenarios can each evolve towards the optimal direction.

[0087] Figure 5 A schematic block diagram of an in-vehicle voice interaction control device 500 according to some embodiments of the present invention is shown. The device 500 may be implemented as or included in... Figure 1 In the vehicle domain controller 130. Device 500 may include multiple modules that can be used to perform, for example... Figure 2 The corresponding actions in method 500 discussed herein. Device 500 includes: a risk identification module 510, configured to acquire the current scene risk level in real time in response to a voice interaction event to be executed. The voice interaction event is a passive interaction event or an active interaction event; the passive interaction event is an interaction event responding to a user's voice command, and the active interaction event is a system-triggered active voice reminder interaction event based on driving behavior analysis results. A parameter determination module 520 is configured to determine execution parameters for the passive interaction event based on the scene risk level and the importance level of the user's voice command; and for the active interaction event, to generate a voice reminder based on the scene risk level and driving behavior analysis results, and determine reminder parameters. Specifically, the parameter determination module 520 executes step 220 in the above method embodiment. For the passive interaction event, the parameter determination module 520 receives the user's voice command, identifies the user's intent, determines the importance level of the command based on the intent category, voice urgency characteristics, and a preset command level mapping relationship, and then queries the execution parameter decision matrix based on the scene risk level and the command importance level to determine the execution parameters, including immediate execution, conditional triggering, or refusal to execute. For proactive interaction events, the parameter determination module 520 queries a voice reminder text library based on the type of undesirable driving behavior to generate reminder content, and then determines reminder parameters based on the scenario risk level, including reminder decision and reminder format. The execution control module 530 is configured to control the execution of the voice interaction event based on the execution parameters or the reminder parameters.

[0088] Specifically, the execution control module performs step 230 in the above method embodiment. For passive interaction events, if the execution parameter is immediate execution, the execution control module immediately executes the user's voice command and announces the result; if the execution parameter is conditional triggering, the execution control module stores the command in a first buffer queue, continuously monitors the scene risk level, and retrieves and executes the command from the queue when the condition is met; if the execution parameter is rejection, the execution control module silently discards the command. For active interaction events, if the reminder decision is immediate triggering, the execution control module immediately announces a voice reminder; if the reminder decision is conditional triggering, the execution control module stores the reminder in a second buffer queue, continuously monitors the scene risk level, and retrieves and announces the reminder from the queue when the condition is met.

[0089] In addition, the device may also include a closed-loop optimization module, not shown in the figure, configured to perform step 250 in the above method embodiment to adjust the voice interaction strategy based on real-time behavioral feedback.

[0090] Some embodiments of the present invention also provide an electronic device including a computing unit that can perform various appropriate actions and processes according to computer program instructions stored in random access memory (RAM) and / or read-only memory (ROM) or computer program instructions loaded from a storage unit into RAM and / or ROM. Various programs and data required for device operation may also be stored in the RAM and / or ROM. The computing unit and the RAM and / or ROM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0091] Multiple components within the device are connected to I / O interfaces, including: input units such as a steering wheel angle sensor, pedal position sensor, and ECG acquisition module; output units such as a central control display, speaker, and hazard warning lights; storage units such as a solid-state drive and embedded memory; and communication units such as an in-vehicle Ethernet controller, Bluetooth module, and V2X communication module. The communication units allow the device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunications networks.

[0092] The computing unit can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing units include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via RAM and / or ROM and / or a communication unit. When the computer program is loaded into RAM and / or ROM and executed by the computing unit, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).

[0093] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a server or terminal, they generate, in whole or in part, the processes or functions described in the embodiments of this application. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to the server or terminal, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, and magnetic tape), an optical medium (e.g., digital video disk (DVD), etc.), or a semiconductor medium (e.g., solid-state drive).

[0094] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0095] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A vehicle-mounted voice interaction control method, characterized in that, include: In response to pending voice interaction events, the risk level of the current scenario can be obtained in real time; The voice interaction event is a passive interaction event or an active interaction event. The passive interaction event is an interaction event that responds to the user's voice command, and the active interaction event is a system-initiated voice reminder interaction event triggered based on the driving behavior analysis results. For the passive interaction event, the execution parameters are determined based on the scenario risk level and the importance level of the user's voice command; For the aforementioned proactive interaction event, a voice prompt is generated based on the scenario risk level and driving behavior analysis results, and the prompt parameters are determined; The execution of the voice interaction event is controlled according to the execution parameters / reminder parameters.

2. The method according to claim 1, characterized in that, Obtaining the current scenario risk level includes: Collect vehicle dynamics data, driver status data, and environmental data; Based on the vehicle dynamics data, the driver state data, and the environmental data, the scenario risk level is determined.

3. The method according to claim 1, characterized in that, The triggering of the active interaction event includes: Detecting undesirable driving behaviors based on driving behavior data and determining the confidence level of the behaviors; When the confidence level of the behavior meets the preset threshold, the system triggers an active voice reminder interaction event.

4. The method according to claim 1, characterized in that, The determination of the importance level of the instruction includes: Receive user voice commands and recognize user intent; and The importance level of the instruction is determined based on at least one of the following: intent category, voice urgency characteristics, and preset instruction level mapping relationship.

5. The method according to claim 1, characterized in that, The execution parameters are immediate execution, conditional triggering, or rejection of execution; If the execution parameter is a condition-triggered execution, the user's voice command is stored in the first cache queue; the scenario risk level is continuously monitored, and when the scenario risk level meets the preset conditions, it is extracted from the first cache queue and executed.

6. The method according to claim 3, characterized in that, The reminder parameters include reminder decision and reminder format; the reminder decision includes immediate triggering or conditional triggering; the reminder format includes at least one of the following: broadcast speed, tone, and reminder duration; If the reminder decision is conditionally triggered, the system's proactive voice reminder will be stored in the second cache queue; The system continuously monitors the risk level of the scenario. When the risk level of the scenario meets the preset conditions, it retrieves the data from the second cache queue. It merges and outputs reminders of the same type and discards reminders that have exceeded the preset validity period.

7. The method according to claim 1, characterized in that, The method further includes: collecting real-time behavioral feedback indicators during voice interaction; when the accumulated interaction data in a certain driving scenario reaches a preset statistical quantity, adjusting the associated parameters in the parameter group corresponding to the scenario step by step based on the deviation between each real-time behavioral feedback indicator and the corresponding threshold in that scenario.

8. A vehicle-mounted voice interaction control device, characterized in that, include: The risk identification module is configured to respond to pending voice interaction events and obtain the current scene risk level in real time. The voice interaction event is a passive interaction event or an active interaction event. The passive interaction event is an interaction event that responds to the user's voice command, and the active interaction event is a system-initiated voice reminder interaction event triggered based on the driving behavior analysis results. The parameter determination module is configured to determine execution parameters for the passive interaction event based on the scenario risk level and the importance level of the user's voice command; and to generate a voice reminder and determine reminder parameters for the active interaction event based on the scenario risk level and driving behavior analysis results. The execution control module is configured to control the execution of the voice interaction event according to the execution parameters / reminder parameters.

9. A vehicle, characterized in that, It includes a processor and a memory, the memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 7.

Images