Interaction control method, device and equipment for man-machine co-driving and storage medium

Abstract

The invention discloses an interaction control method, device and equipment for man-machine co-driving and a storage medium, and relates to the technical field of intelligent driving and man-machine interaction, and the method comprises the steps: obtaining multi-mode sensor data of a vehicle, driver interaction data in a cabin, and vehicle-road cooperation data; generating a unified driving scene tensor according to the multi-modal sensor data, the driver interaction data and the vehicle-road cooperation data; analyzing the driving scene tensor according to a preset intention collaborative analysis strategy to obtain a driver target operation intention; the target operation intention of the driver and the driving scene tensor are analyzed according to a preset man-machine co-driving large model, and a vehicle control instruction is obtained; and controlling the vehicle to perform corresponding instruction response according to the vehicle control instruction. According to the invention, through multi-mode perception fusion, intention collaborative analysis and man-machine co-driving large model decision making, the scene adaptability and processing efficiency of man-machine co-driving are improved.

Description

Technical Field

[0001] This application relates to the technical field of intelligent driving and human-machine interaction, and in particular to an interactive control method, device, equipment and storage medium for human-machine co-driving. Background Technology

[0002] With the development of intelligent connected vehicle technology, C-V2X (Cellular Vehicle-to-Everything), as a core communication technology, has been widely applied in vehicle-road cooperative systems. It enables real-time information interaction between vehicles and roadside facilities, other vehicles, pedestrians, and the cloud through LTE-V2X and 5G-V2X, providing crucial support for high-level autonomous driving. Currently, city-level demonstration projects have implemented functions such as traffic light warnings and blind spot detection, and some automakers have begun to pre-install C-V2X modules in their mass-produced models. However, existing C-V2X-based vehicle-road cooperative control systems still have significant limitations. First, the system typically relies on roadside computing units for environmental fusion and command issuance, resulting in high requirements for infrastructure coverage and insufficient service continuity in non-demonstration areas or remote road sections. Second, existing solutions largely rely on preset rules for driving decisions, lacking an understanding of the driver's operational intentions, making it difficult to achieve efficient human-machine collaboration. In addition, the system has a single interaction method, which mainly transmits information through the screen or voice. It is not adaptable to complex scenarios such as sudden road conditions and severe weather, and is prone to response delays or decision-making errors.

[0003] Therefore, how to improve the adaptability and efficiency of human-computer interaction in complex driving scenarios is an urgent problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide an interactive control method, device, equipment, and storage medium for human-machine co-driving, aiming to solve the technical problem of how to improve the adaptability and efficiency of human-machine interaction in complex driving scenarios.

[0005] To achieve the above objectives, this application proposes an interactive control method for human-machine co-driving, the method comprising: Acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-to-infrastructure (V2I) data; A unified driving scenario tensor is generated based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; The driving scenario tensor is parsed according to a preset intention collaborative parsing strategy to obtain the driver's target operation intention; Based on the preset human-machine co-driving model, the driver's target operation intention and the driving scenario tensor are analyzed to obtain vehicle control commands; The vehicle is controlled to respond to the corresponding commands according to the vehicle control commands.

[0006] In one embodiment, the sensors on the vehicle include a surround-view camera, radar, and lidar; The steps for acquiring multimodal sensor data of a vehicle include: Acquire image sequence data collected by the surround-view camera, obstacle distance and velocity sequence data collected by the radar, and point cloud sequence data collected by the lidar; The image sequence data, the obstacle distance and velocity sequence data, and the point cloud sequence data are aligned in time sequence to obtain multimodal sensor data.

[0007] In one embodiment, the sensors on the vehicle include a microphone array, a cockpit camera, and biosensors; The steps for acquiring driver interaction data within the cockpit include: Acquire voice signals collected by the microphone array, gesture images collected by the cockpit camera, and physiological state signals of the driver collected by the biosensor; The speech signal is converted into text instruction information according to a preset natural language processing model; The gesture image is converted into gesture command information according to a preset image recognition strategy; The text command information, the gesture command information, and the physiological state signals are fused to obtain the in-cabin driver interaction data.

[0008] In one embodiment, the step of generating a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-to-infrastructure (V2I) data includes: Based on the feature extraction network, features are extracted from the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data to obtain environmental feature vectors, driver feature vectors, and cooperative feature vectors. The environmental feature vector, the driver feature vector, and the collaborative feature vector are aligned and fused using a cross-modal attention mechanism to generate a fused feature tensor. Temporal dynamic feature extraction and spatial structure feature extraction are performed on the fused feature tensor to obtain a unified driving scene tensor.

[0009] In one embodiment, the step of parsing the driving scenario tensor according to a preset intent collaborative parsing strategy to obtain the driver's target operation intent includes: Based on a preset intention-based collaborative parsing strategy, feature extraction is performed on the driving scenario tensor to obtain text interaction instructions and action behavior instructions. The text interaction instructions and action instructions are parsed based on the pre-trained language model and operation behavior sequence model to obtain the driver's explicit intention and the driver's implicit intention. The driver's target operational intention is obtained by quantifying the difference between the driver's explicit intention and the driver's implicit intention.

[0010] In one embodiment, the step of obtaining the driver's target operational intention by differential quantification of the driver's explicit intention and the driver's implicit intention includes: Calculate the intention based on the driver's explicit intention and the driver's implicit intention. Figure 1 Consistency; In the meaning Figure 1 When the degree of consistency is greater than the preset intention deviation threshold, the driver's explicit intention is regarded as an intention to be confirmed; In the meaning Figure 1 When the consistency is less than or equal to the preset intention deviation threshold, the preset conflict arbitration sub-model is invoked to perform weighted fusion and risk scoring on the driver's explicit intention and the driver's implicit intention, to obtain multiple candidate intentions with scores, and the candidate intention with the highest score is taken as the intention to be confirmed. The driver's target operational intent is determined based on the intent to be confirmed and the environmental constraints in the current driving scenario tensor.

[0011] In one embodiment, the step of determining the driver's target operational intention based on the intent to be confirmed and the environmental constraints in the current driving scenario tensor includes: The intent to be confirmed is matched and verified with the environmental constraints in the current driving scenario tensor to obtain the matching result; When the matching result is a successful match, the intent to be confirmed is taken as the driver's target operational intent; When the matching result is a failure, the preset default operation intention will be used as the driver's target operation intention, which is an operation intention that prioritizes scenario safety.

[0012] Furthermore, to achieve the above objectives, this application also proposes an interactive control device for human-machine co-driving, the device comprising: The data acquisition module is used to acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data; The data processing module is used to generate a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; The intent parsing module is used to parse the driving scenario tensor according to a preset intent collaborative parsing strategy to obtain the driver's target operation intent; The intent conversion module is used to analyze the driver's target operation intent and the driving scenario tensor based on the preset human-machine co-driving model to obtain vehicle control commands; The interactive response module is used to control the vehicle to respond to the corresponding commands based on the vehicle control commands.

[0013] Furthermore, to achieve the above objectives, this application also proposes an interactive control device for human-machine co-driving, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the interactive control method for human-machine co-driving as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the interactive control method for human-machine co-driving described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the interactive control method for human-machine co-driving described above.

[0016] This application provides an interactive control method for human-machine co-driving. The method includes: acquiring multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-to-infrastructure (V2I) data; generating a unified driving scene tensor based on the multimodal sensor data, driver interaction data, and V2I data; parsing the driving scene tensor according to a preset intent-based collaborative parsing strategy to obtain the driver's target operation intent; analyzing the driver's target operation intent and the driving scene tensor according to a preset human-machine co-driving big model to obtain vehicle control commands; and controlling the vehicle to perform corresponding command responses according to the vehicle control commands. In summary, this application, through multimodal perception fusion, intent-based collaborative parsing, and human-machine co-driving big model decision-making, achieves accurate understanding of the driver's intent, improving the system's adaptability and processing efficiency in human-machine co-driving scenarios. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the interactive control method for human-machine co-driving in this application; Figure 2 This is an overall algorithm architecture diagram of an embodiment of the interactive control method for human-machine co-driving in this application; Figure 3 This is a flowchart illustrating the second embodiment of the interactive control method for human-machine co-driving in this application; Figure 4 This is a flowchart illustrating the third embodiment of the interactive control method for human-machine co-driving in this application; Figure 5 This is a schematic diagram of the module structure of the interactive control device for human-machine co-driving according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the interactive control method for human-machine co-driving in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: acquiring multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data; generating a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; parsing the driving scene tensor according to a preset intention collaborative parsing strategy to obtain the driver's target operation intention; analyzing the driver's target operation intention and the driving scene tensor according to a preset human-machine co-driving model to obtain vehicle control commands; and controlling the vehicle to perform corresponding command responses according to the vehicle control commands.

[0024] With the development of intelligent connected vehicle technology, C-V2X, as a core communication technology, has been widely applied in vehicle-road cooperative systems. It enables real-time information interaction between vehicles and roadside facilities, other vehicles, pedestrians, and the cloud through LTE-V2X and 5G-V2X, providing crucial support for high-level autonomous driving. Currently, city-level demonstration projects have implemented functions such as traffic light warnings and blind spot detection, and some automakers have begun to pre-install C-V2X modules in their mass-produced models. However, existing C-V2X-based vehicle-road cooperative control systems still have significant limitations. First, the system typically relies on roadside computing units for environmental fusion and command issuance, resulting in high requirements for infrastructure coverage and insufficient service continuity in non-demonstration areas or remote road sections. Second, existing solutions largely rely on preset rules for driving decisions, lacking understanding of the driver's operational intentions and making it difficult to achieve efficient human-machine collaboration. Furthermore, the system's interaction methods are limited, primarily relying on screens or voice for information transmission, making it less adaptable to complex scenarios such as sudden road conditions and inclement weather, prone to response delays or decision-making errors. Therefore, how to improve the adaptability and efficiency of human-computer interaction in complex driving scenarios is an urgent problem to be solved.

[0025] This application achieves accurate understanding of the driver's intent through multimodal perception fusion, intention collaborative parsing, and human-machine co-driving big model decision-making, thereby improving the system's adaptability and processing efficiency in human-machine co-driving scenarios.

[0026] It should be noted that the executing entity in this embodiment can be an interactive control system for human-machine co-driving, a computing service device with data processing, network communication, and program execution functions, or an electronic device capable of implementing the aforementioned interactive control functions for human-machine co-driving, etc. This embodiment does not specifically limit it in this way. The following uses an interactive control system for human-machine co-driving as an example to describe this embodiment and the following embodiments.

[0027] Based on this, embodiments of this application provide an interactive control method for human-machine co-driving, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the interactive control method for human-machine co-driving according to this application.

[0028] In this embodiment, the interactive control method for human-machine co-driving includes steps S10 to S50: Step S10: Acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data.

[0029] It should be noted that this step is the data acquisition and preprocessing stage, aiming to obtain comprehensive, real-time multi-source data from the vehicle's external environment, driver behavior, and vehicle-to-infrastructure (V2I) systems. This data provides the foundational input for subsequent driving scenario modeling and intent parsing. Specifically, this step includes three data sources: the vehicle's multimodal sensors (such as surround-view cameras, radar, and lidar), in-cabin driver interaction devices (such as microphone arrays, cabin cameras, and biosensors), and the V2I system (such as V2X roadside units). Multimodal sensor data refers to physical quantity data reflecting the vehicle's external environment collected by different types of sensors, including visual, radar, and lidar data; driver interaction data refers to data reflecting the driver's operational intentions, physiological state, and interaction commands; and V2I data refers to traffic signals, road condition warnings, and map information obtained from roadside units (RSUs) or other vehicles via C-V2X communication.

[0030] In one feasible implementation, the sensors on the vehicle include surround-view cameras, radar, and lidar; The specific steps for acquiring multimodal sensor data from a vehicle include: Step A10: Acquire image sequence data collected by the surround-view camera, obstacle distance and velocity sequence data collected by the radar, and point cloud sequence data collected by the lidar.

[0031] It should be noted that in this step, the system continuously acquires multi-view images of the vehicle's surroundings using surround-view cameras, forming an image sequence; it detects the relative distance and speed of obstacles by emitting and receiving electromagnetic waves using radar (such as millimeter-wave radar), generating time-series data; and it obtains high-precision 3D point clouds by emitting laser scans using lidar, describing the environmental geometry. These data collectively constitute the raw perception information of the vehicle's external environment, used for subsequent environmental modeling and target detection.

[0032] Additionally, it should be noted that the image sequence data consists of image frames with consecutive timestamps, used for visual target recognition and scene understanding; the obstacle distance and velocity sequence data is a time series output by the radar, including the position, relative velocity, and direction of motion of the obstacles; and the point cloud sequence data is a set of three-dimensional coordinates obtained from each frame scan by the lidar, used for high-precision environment reconstruction.

[0033] Step A20: Align the image sequence data, the obstacle distance and velocity sequence data, and the point cloud sequence data according to time sequence to obtain multimodal sensor data.

[0034] It should be noted that in this step, the system uses a time synchronization protocol (such as PTP or NTP) to assign a unified timestamp to the data from each sensor, and aligns data with different sampling rates to the same time reference through interpolation or resampling methods. The aligned data forms multimodal sensor data, ensuring spatiotemporal consistency during subsequent fusion processing and improving the accuracy and reliability of environmental perception. Furthermore, it should be noted that the time alignment refers to synchronizing the data from different sensors in time through timestamp matching and data interpolation, avoiding inconsistencies caused by differences in sampling times.

[0035] In one feasible implementation, the sensors on the vehicle include a microphone array, a cockpit camera, and biosensors; The steps for acquiring driver interaction data within the cockpit include: Step B10: Acquire the voice signal collected by the microphone array, the gesture image collected by the cockpit camera, and the physiological state signal of the driver collected by the biosensor.

[0036] It should be noted that in this step, the system will collect the driver's voice commands through a microphone array to form the raw audio signal; capture the driver's hand gestures through an in-cabin camera to generate an image sequence; and collect the driver's physiological indicators (such as heart rate monitor and skin conductance sensor) in real time through biosensors (such as heart rate monitor and skin conductance sensor).

[0037] Additionally, it should be noted that the voice signal is audio waveform data, which needs to be processed and converted into text commands; the gesture image is a video frame containing hand movements, used to recognize predefined gestures; and the physiological state signal is a bioelectrical signal reflecting the driver's fatigue, tension, and other states.

[0038] Step B20: Convert the speech signal into text instruction information according to the preset natural language processing model.

[0039] It's important to note that in this step, the system uses a pre-trained Natural Language Processing (NLP) model (such as the Transformer-based Qwen3 model) to perform speech recognition and semantic parsing on the speech signal, outputting structured text instruction information. For example, the speech "Turn right and close the window" is parsed as the action: "Turn right"; the control: "Close the window." This step converts unstructured speech into machine-understandable instructions, facilitating subsequent intent parsing.

[0040] Step B30: Convert the gesture image into gesture command information according to a preset image recognition strategy.

[0041] It should be noted that in this step, the system employs a gesture recognition model based on a convolutional neural network (CNN) or visual Transformer to analyze gesture images captured by the cockpit camera in real time, identify predefined gestures (such as pointing, waving, and clenching a fist), and map them to corresponding control commands (such as changing music or confirming navigation). This step achieves contactless interaction, enhancing ease of operation. Furthermore, it should be noted that the preset image recognition strategy includes gesture detection, key point localization, and classification models, supporting dynamic gesture sequence recognition.

[0042] Step B40: The text instruction information, the gesture instruction information, and the physiological state signal are fused to obtain in-cabin driver interaction data.

[0043] It should be noted that in this step, the system integrates text commands, gesture commands, and physiological state signals into unified driver interaction data using feature fusion methods (such as concatenation or attention weighting). For example, when a voice command to accelerate occurs simultaneously with a gesture of waving forward, and physiological signals indicate that the driver is awake, the fused data represents a strong intention to accelerate. Understandably, this step is used to provide a comprehensive description of the driver's state, offering multi-dimensional input for intent parsing.

[0044] Step S20: Generate a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data.

[0045] It's important to note that in this step, the system aligns multi-source data features through a cross-modal attention mechanism and introduces a spatiotemporal encoder to handle temporal dependencies, generating a unified driving scene tensor. This tensor integrates environmental perception, driver intent, and traffic state information to form a comprehensive and structured scene representation, providing a unified input for subsequent intent parsing and decision-making. Furthermore, it's crucial to understand that the driving scene tensor is a multi-dimensional data structure containing spatial dimensions (such as the vehicle's surrounding environment), temporal dimensions (such as historical state sequences), and feature dimensions (such as object categories and driver commands), used to describe the complete state of the current driving scene.

[0046] Step S30: Parse the driving scenario tensor according to the preset intention collaborative parsing strategy to obtain the driver's target operation intention.

[0047] It should be noted that in this step, the system employs a hybrid expert architecture-based intent collaborative parsing engine to perform multi-level parsing of the driving scenario tensor. Specifically, this includes: explicit intent parsing, which extracts semantic tokens from voice commands (such as right turn, avoidance) based on a pre-trained language model; implicit intent inference, which analyzes driver operating habits through behavioral sequence modeling (such as frequent checks of the rearview mirror implying lane change intentions); and a conflict arbitration module, which quantifies the differences between human and machine operations and downgrades permissions when the differences exceed a threshold. This results in a structured output of the driver's target operating intent (e.g., target lane: right lane; operation: decelerate).

[0048] Additionally, it should be noted that the driver's target operation intention is a machine-understandable semantic representation, including operation type, target parameters, and priority; the preset intention collaborative parsing strategy is a parsing model based on multi-task learning, which supports dynamic scene adaptation.

[0049] Step S40: Analyze the driver's target operation intention and the driving scenario tensor according to the preset human-machine co-driving model to obtain vehicle control commands.

[0050] It should be noted that, as Figure 2 As shown, in this step, the system uses a pre-defined human-machine co-driving model (H-JointDrive). This model employs a layered hybrid architecture, combining Transformer and MoE networks to collaboratively analyze and make decisions regarding the driver's target operational intent and the driving scenario tensor. The model aligns intent and scenario information through a multimodal fusion layer, outputting specific vehicle control commands (such as steering angle, acceleration, and braking pressure). Simultaneously, the model utilizes knowledge distillation technology for lightweight deployment, ensuring inference latency on automotive-grade chips is less than 100ms.

[0051] Additionally, it should be noted that the preset human-machine co-driving model is a deep learning model pre-trained in a large number of driving scenarios, which supports the generation of end-to-end control commands; vehicle control commands include low-level actuator commands (such as steering, braking, and throttle control) and high-level behavioral commands (such as following, lane changing, and parking).

[0052] Step S50: Control the vehicle to respond to the corresponding command according to the vehicle control command.

[0053] It should be noted that the command response includes both mechanical control response and human-machine interaction feedback to ensure a smooth transition for the driver between different command controls and system automation. In this step, the system sends vehicle control commands to vehicle actuators (such as the electronic stability control system, electric power steering system, and low-voltage control system) to achieve vehicle action response. Simultaneously, the perception and visualization module updates environmental information (such as obstacles and road structures) and target heatmaps to the in-vehicle HMI interface, and the emotion interaction module dynamically adjusts the prompting strategy (such as gentle voice reminders or forced takeover) based on the driver's physiological state. It can be understood that the purpose of this step is to provide complete execution and feedback for different commands, improving driving safety and user experience.

[0054] This embodiment provides an interactive control method for human-machine co-driving. The method includes: acquiring multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-to-infrastructure (V2I) data; generating a unified driving scene tensor based on the multimodal sensor data, driver interaction data, and V2I data; parsing the driving scene tensor according to a preset intent-based collaborative parsing strategy to obtain the driver's target operation intent; analyzing the driver's target operation intent and the driving scene tensor according to a preset human-machine co-driving big model to obtain vehicle control commands; and controlling the vehicle to perform corresponding command responses according to the vehicle control commands. In summary, this embodiment, through multimodal perception fusion, intent-based collaborative parsing, and human-machine co-driving big model decision-making, achieves accurate understanding of the driver's intent, improving the system's adaptability and processing efficiency in human-machine co-driving scenarios.

[0055] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the interactive control method for human-machine co-driving according to this application. Step S20 specifically includes: Step S201: Based on the feature extraction network, perform feature extraction on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data respectively to obtain environmental feature vectors, driver feature vectors, and cooperative feature vectors.

[0056] It should be noted that in this step, the system employs dedicated feature extraction networks for parallel processing of data from three different sources. For multimodal sensor data, a convolutional neural network (CNN) is used to process surround-view camera images to extract visual features (such as road edges and vehicle outlines), a PointNet++ network is used to process LiDAR point clouds to extract 3D geometric features, and a recurrent neural network (RNN) is used to process radar sequence data to capture the motion trends of dynamic targets. Finally, these features are concatenated and dimensionality reduced to form a unified environmental feature vector. For driver interaction data, a pre-trained language model (such as Qwen3) is used to encode textual instructions into semantic vectors, a gesture recognition model is used to output gesture feature vectors, and physiological state signals are standardized and embedded. Subsequently, these features are fused into a driver feature vector through a fully connected layer, which comprehensively reflects the driver's explicit instructions and implicit states. For vehicle-to-infrastructure (V2I) data, an encoder network is used to process structured information received from V2X roadside units (such as traffic light status, road condition warnings, and forward accident information) to generate a cooperative feature vector, which provides beyond-line-of-sight global traffic situation information.

[0057] Step S202: The environmental feature vector, the driver feature vector, and the collaborative feature vector are aligned and fused using a cross-modal attention mechanism to generate a fused feature tensor.

[0058] It should be noted that a cross-modal attention mechanism is introduced in this step to eliminate the semantic and dimensional differences between different feature vectors and to establish their intrinsic relationship. Specifically, the environmental feature vector is used as the query, and the driver feature vector and the collaborative feature vector are used as the key and value to calculate the attention weight. For example, when the driver issues a voice command to turn right (belonging to the driver feature vector), the attention mechanism will assign higher weights to the visual and radar features of the right front area of ​​the vehicle (belonging to the environmental feature vector), while also paying attention to information in the collaborative feature vector regarding the accessibility of the right-turn lane. Through this mechanism, the system can achieve context-aware feature alignment, fusing the three feature vectors into a more closely related and semantically richer fusion feature tensor.

[0059] Step S203: Perform temporal dynamic feature extraction and spatial structure feature extraction on the fused feature tensor to obtain a unified driving scene tensor.

[0060] It's important to note that driving scenarios are dynamically changing, necessitating deep spatiotemporal modeling of the fused features. This is achieved through a spatiotemporal encoder. This encoder typically consists of a temporal module (such as a Transformer encoder or LSTM) and a spatial module (such as a GCN or 3D CNN). The temporal dynamic feature extraction module analyzes the changes in the fused feature tensor over time, such as predicting the future trajectory of vehicles ahead and recognizing the driver's continuous operating patterns. The spatial structure feature extraction module understands the spatial distribution and structural relationships of the features, such as determining the relative positions of the vehicle and surrounding obstacles, and the road topology. Ultimately, the tensor output by the spatiotemporal encoder is the unified driving scene tensor defined in this scheme, which simultaneously contains the scene's spatial layout, temporal evolution, and multimodal semantic information.

[0061] In this embodiment, by extracting multimodal features, fusion of cross-modal attention, and extracting temporal and spatial features, a unified representation and efficient integration of environmental, driver, and vehicle-road cooperative data are achieved. This solves the problems of insufficient multimodal data fusion and one-sided scene understanding in traditional solutions, and improves the accuracy of driving decisions and the system's adaptability to complex traffic environments.

[0062] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the interactive control method for human-machine co-driving according to this application. Step S30 specifically includes: Step S301: Extract features from the driving scene tensor according to the preset intent collaborative parsing strategy to obtain text interaction instructions and action behavior instructions.

[0063] It should be noted that in this step, the system decodes key behavioral commands from the tensor using a specific feature head. Specifically, a fully connected layer classifier is used to decode structured text interaction commands (such as turning off the air conditioning or navigating to the company) from feature channels related to cockpit interaction; simultaneously, a recurrent neural network (RNN) is used to decode continuous action commands (such as steering wheel angle sequences and accelerator pedal opening rate changes) from temporal features related to driver operation behavior.

[0064] Additionally, it should be noted that the text interaction instructions are the final text commands converted from the driver's voice and modified by the scene context; the action behavior instructions are quantitative descriptions of the driver's physical operations (such as turning the steering wheel and pressing the pedals), reflecting the driver's real-time control behavior.

[0065] Step S302: Parse the text interaction instructions and the action behavior instructions according to the pre-trained language model and operation behavior sequence model to obtain the driver's explicit intention and the driver's implicit intention.

[0066] It should be noted that this step employs a dual-path parsing architecture. One path uses a pre-trained language model (such as Qwen3) to perform deep semantic understanding of the text interaction commands, parsing out the driver's explicit intentions, i.e., the needs clearly expressed by the driver through language. For example, "I feel a little hot" is parsed as "Intention: Adjust the air conditioning; Parameter: Lower the temperature." The other path uses an operation behavior sequence model (such as an LSTM-based neural network) to model the action behavior commands. This model, combined with historical operation data, infers the driver's implicit intentions, i.e., the driver's unspoken but potential needs revealed through operation behaviors. For example, if the driver is detected to be continuously and slightly turning to the right at an increased frequency, it may be inferred that their implicit intention is "Preparing to change lanes to the right."

[0067] Understandably, a driver's explicit intent is a direct and easily understood user command; a driver's implicit intent is an indirect user tendency that can only be inferred through behavioral pattern analysis.

[0068] Step S303: Quantify the difference between the driver's explicit intention and the driver's implicit intention to obtain the driver's target operation intention.

[0069] It should be noted that, since explicit and implicit intentions may coincide or conflict, a quantitative arbitration mechanism is introduced in this step. First, the intention-arbitration relationship between the two intentions is calculated. Figure 1 Consistency. If the consistency is high, the explicit intent is adopted first; if the consistency is low, a conflict arbitration algorithm is activated, comprehensively considering safety, comfort, and driver habits, performing a weighted fusion and risk assessment of the two intents, and finally outputting an optimal, conflict-free driver target operation intent. It can be understood that the purpose of this step is to eliminate comprehension bias and ensure that the system's response intent reflects the driver's most genuine and reasonable needs.

[0070] In one feasible implementation, step S303 specifically includes: Step C10: Calculate the intention based on the driver's explicit intention and the driver's implicit intention. Figure 1 To a certain degree.

[0071] It should be noted that in this step, the system embeds both explicit and implicit intents into the same high-dimensional semantic space, forming intent vectors. Then, it calculates the cosine similarity between these two vectors as the meaning vector. Figure 1Consistency. The closer this value is to 1, the more consistent the two intentions are; the closer it is to 0, the greater the conflict.

[0072] Step C20: In the meaning Figure 1 When the degree of consistency is greater than the preset intention deviation threshold, the driver's explicit intention is regarded as an intention to be confirmed.

[0073] It should be noted that when the calculated meaning Figure 1 When the consistency exceeds a preset intent deviation threshold (e.g., 0.8), the system considers the intent conveyed by the driver's words and actions to be basically consistent, and the system can trust its explicit linguistic instructions. Therefore, the system will directly treat the driver's explicit intent as the intent to be confirmed and proceed to the next step of environmental verification.

[0074] Additionally, it should be noted that the preset intent deviation threshold is a configurable parameter, and its value can be adjusted according to the driving scenario (such as highway or urban area) and safety requirements. The intent to be confirmed is an intermediate state, indicating that it still needs to undergo environmental feasibility verification before becoming the final target operation intent.

[0075] Step C30: In the context of Figure 1 When the consistency is less than or equal to the preset intention deviation threshold, the preset conflict arbitration sub-model is invoked to perform weighted fusion and risk scoring on the driver's explicit intention and the driver's implicit intention, resulting in multiple candidate intentions with scores, and the candidate intention with the highest score is taken as the intention to be confirmed.

[0076] It should be noted that when the intention Figure 1 When the consistency score is lower than a preset intention deviation threshold (e.g., 0.8), it indicates a contradiction between the driver's words and actions (e.g., saying "go straight" but showing a tendency to turn the steering wheel left). In this case, the system invokes a preset conflict arbitration sub-model. This model generates multiple candidate intentions (e.g., maintaining straight, initiating a left turn, deceleration warning, etc.) and calculates a risk score for each candidate intention. This score integrates factors such as traffic rule compliance, collision risk, and comfort impact. Ultimately, the candidate intention with the lowest risk score (i.e., the safest) and the highest consistency score is selected as the intention to be confirmed. Understandably, the purpose of this step is to introduce safety and rationality judgments when human-machine intentions conflict, leading to the optimal decision.

[0077] Additionally, it should be noted that the preset conflict arbitration sub-model is a decision-making model that prioritizes safety and rules. The risk score is a numerical value; a higher score indicates a greater potential risk in carrying out the intended action.

[0078] Step C40: Determine the driver's target operation intention based on the intent to be confirmed and the environmental constraints in the current driving scenario tensor.

[0079] It's important to note that environmental constraints refer to all physical and rule-based conditions that restrict vehicle behavior in the current driving scenario, such as road boundaries, red lights, and safe distances from vehicles ahead. In this step, even after internal arbitration, the resulting intent to be confirmed must be feasible within the current physical environment. The system matches and verifies the intent against the environmental information (such as lane lines, traffic lights, and the positions of surrounding vehicles) contained in the driving scenario tensor. Only if the verification passes will the intent be ultimately confirmed as the driver's target operation intent; if the verification fails, the system will employ a preset safety strategy to ensure the vehicle remains in a safe state.

[0080] In one feasible implementation, step C40 specifically includes: Step C401: Match and verify the intent to be confirmed with the environmental constraints in the current driving scenario tensor to obtain the matching result.

[0081] It should be noted that in this step, the system verifies the intent to be confirmed based on a rule base or feasibility prediction network. For example, if the intent to be confirmed is to change lanes to the left, it checks whether the current left lane line is a dashed line and whether there is a rapidly approaching vehicle from the left rear. The matching result is a binary output: successful match or failed match.

[0082] Step C402: When the matching result is a successful match, the intent to be confirmed is taken as the driver's target operation intent.

[0083] It should be noted that if the environmental verification passes, the system will adopt the intent to be confirmed, officially recognize it as the driver's target operation intent, and prepare to pass it to the subsequent human-machine co-driving model for generating control commands.

[0084] Step C403: When the matching result is a failure, the preset default operation intention is taken as the driver's target operation intention, and the preset default operation intention is the operation intention that prioritizes scenario safety.

[0085] It should be noted that if the environmental verification fails (e.g., the intention is to turn right but right turns are prohibited at the current intersection), the system will abandon the intended action and adopt a preset default action, prioritizing safety. This default action is usually conservative and safe, such as maintaining a safe following distance within the current lane or slowing down until the danger has passed. Simultaneously, the system will provide the driver with the reason for the rejected action through the HMI interface.

[0086] In this embodiment, text interaction and operation commands are comprehensively analyzed, and intention is introduced. Figure 1The consistency verification and conflict arbitration mechanism enables accurate identification and conflict resolution of the driver's explicit and implicit intentions, solving the problem of traditional systems having a one-sided understanding of the driver's intentions and rigid responses when intentions conflict, thus improving the accuracy of human-machine collaborative decision-making and driving safety.

[0087] This application also provides an interactive control device for human-machine co-driving, please refer to... Figure 5 The interactive control device for human-machine co-driving includes: Data acquisition module 10 is used to acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data; The data processing module 20 is used to generate a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data. The intent parsing module 30 is used to parse the driving scenario tensor according to a preset intent collaborative parsing strategy to obtain the driver's target operation intent; The intent conversion module 40 is used to analyze the driver's target operation intent and the driving scenario tensor according to the preset human-machine co-driving model to obtain vehicle control commands; The interactive response module 50 is used to control the vehicle to respond to the corresponding command according to the vehicle control command.

[0088] The interactive control device for human-machine co-driving provided in this application, employing the interactive control method for human-machine co-driving in the above embodiments, can solve the technical problem of how to improve the adaptability and efficiency of human-machine interaction in complex driving scenarios. Compared with the prior art, the beneficial effects of the interactive control device for human-machine co-driving provided in this application are the same as the beneficial effects of the interactive control method for human-machine co-driving provided in the above embodiments, and other technical features in the interactive control device for human-machine co-driving are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0089] In one embodiment, the data acquisition module 10 is further configured to acquire image sequence data acquired by the surround-view camera, obstacle distance and velocity sequence data acquired by the radar, and point cloud sequence data acquired by the lidar; and to align the image sequence data, obstacle distance and velocity sequence data, and point cloud sequence data in time sequence to obtain multimodal sensor data.

[0090] In one embodiment, the data acquisition module 10 is further configured to acquire voice signals acquired through the microphone array, gesture images acquired by the cockpit camera, and physiological state signals of the driver acquired by the biosensor; convert the voice signals into text command information according to a preset natural language processing model; convert the gesture images into gesture command information according to a preset image recognition strategy; and fuse the text command information, the gesture command information, and the physiological state signals to obtain in-cabin driver interaction data.

[0091] In one embodiment, the data processing module 20 is further configured to extract features from the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data according to the feature extraction network to obtain environmental feature vectors, driver feature vectors, and cooperative feature vectors; perform feature alignment and fusion on the environmental feature vectors, driver feature vectors, and cooperative feature vectors through a cross-modal attention mechanism to generate a fused feature tensor; and perform temporal dynamic feature extraction and spatial structure feature extraction on the fused feature tensor to obtain a unified driving scene tensor.

[0092] In one embodiment, the intent parsing module 30 is further configured to extract features from the driving scenario tensor according to a preset intent collaborative parsing strategy to obtain text interaction instructions and action behavior instructions; parse the text interaction instructions and action behavior instructions according to a pre-trained language model and operation behavior sequence model to obtain the driver's explicit intent and the driver's implicit intent; and perform differential quantization based on the driver's explicit intent and the driver's implicit intent to obtain the driver's target operation intent.

[0093] In one embodiment, the intent parsing module 30 is further configured to calculate the intent based on the driver's explicit intent and the driver's implicit intent. Figure 1 Consistency; in the meaning Figure 1 When the degree of consistency is greater than a preset intention deviation threshold, the driver's explicit intention is taken as an intention to be confirmed; in the intention Figure 1 When the consistency is less than or equal to the preset intention deviation threshold, the preset conflict arbitration sub-model is invoked to perform weighted fusion and risk scoring on the driver's explicit intention and the driver's implicit intention, resulting in multiple candidate intentions with scores, and the candidate intention with the highest score is taken as the intention to be confirmed; the driver's target operation intention is determined according to the intention to be confirmed and the environmental constraints in the current driving scenario tensor.

[0094] In one embodiment, the intent parsing module 30 is further configured to match and verify the intent to be confirmed with the environmental constraints in the current driving scenario tensor to obtain a matching result; when the matching result is a successful match, the intent to be confirmed is used as the driver's target operation intent; when the matching result is a failed match, a preset default operation intent is used as the driver's target operation intent, wherein the preset default operation intent is an operation intent that prioritizes scenario safety.

[0095] This application provides an interactive control device for human-machine co-driving, the interactive control device for human-machine co-driving includes: at least one processor; and a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to execute the interactive control method for human-machine co-driving in the above embodiment 1.

[0096] The following is for reference. Figure 6 This document illustrates a structural schematic diagram of an interactive control device suitable for implementing embodiments of this application for human-machine co-driving. The interactive control device for human-machine co-driving in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The interactive control device for human-machine co-driving shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0097] like Figure 6As shown, the interactive control device for human-machine co-driving can include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the interactive control device for human-machine co-driving. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the interactive control device for human-machine co-driving to exchange data wirelessly or via wired communication with other devices. Although interactive control devices for human-machine co-driving with various systems are shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0098] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0099] The interactive control device for human-machine co-driving provided in this application, employing the interactive control method for human-machine co-driving in the above embodiments, can solve the technical problem of how to improve the adaptability and efficiency of human-machine interaction in complex driving scenarios. Compared with the prior art, the beneficial effects of the interactive control device for human-machine co-driving provided in this application are the same as the beneficial effects of the interactive control method for human-machine co-driving provided in the above embodiments, and other technical features in the interactive control device for human-machine co-driving are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0100] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0102] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the interactive control method for human-machine co-driving in the above embodiments.

[0103] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0104] The aforementioned computer-readable storage medium may be included in the interactive control device for human-machine co-driving; or it may exist independently and not assembled into the interactive control device for human-machine co-driving.

[0105] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a human-machine co-driving interactive control device, cause the human-machine co-driving interactive control device to: acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data; generate a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; parse the driving scene tensor according to a preset intent collaborative parsing strategy to obtain the driver's target operation intent; analyze the driver's target operation intent and the driving scene tensor according to a preset human-machine co-driving big model to obtain vehicle control commands; and control the vehicle to perform corresponding command responses according to the vehicle control commands.

[0106] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0109] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned interactive control method for human-machine co-driving, thereby solving the technical problem of how to improve the adaptability and efficiency of human-machine interaction in complex driving scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the interactive control method for human-machine co-driving provided in the above embodiments, and will not be repeated here.

[0110] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the interactive control method for human-machine co-driving described above.

[0111] The computer program product provided in this application can solve the technical problem of how to improve the adaptability and efficiency of human-machine interaction in complex driving scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the interactive control method for human-machine co-driving provided in the above embodiments, and will not be repeated here.

[0112] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An interactive control method for human-machine co-driving, characterized in that, The method includes: Acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-to-infrastructure (V2I) data; A unified driving scenario tensor is generated based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; The driving scenario tensor is parsed according to a preset intention collaborative parsing strategy to obtain the driver's target operation intention; Based on the preset human-machine co-driving model, the driver's target operation intention and the driving scenario tensor are analyzed to obtain vehicle control commands; The vehicle is controlled to respond to the corresponding commands according to the vehicle control commands.

2. The method as described in claim 1, characterized in that, The sensors on the vehicle include surround-view cameras, radar, and lidar; The steps for acquiring multimodal sensor data of a vehicle include: Acquire image sequence data collected by the surround-view camera, obstacle distance and velocity sequence data collected by the radar, and point cloud sequence data collected by the lidar; The image sequence data, the obstacle distance and velocity sequence data, and the point cloud sequence data are aligned in time sequence to obtain multimodal sensor data.

3. The method as described in claim 1, characterized in that, The sensors on the vehicle include microphone arrays, cockpit cameras, and biosensors; The steps for acquiring driver interaction data within the cockpit include: Acquire voice signals collected by the microphone array, gesture images collected by the cockpit camera, and physiological state signals of the driver collected by the biosensor; The speech signal is converted into text instruction information according to a preset natural language processing model; The gesture image is converted into gesture command information according to a preset image recognition strategy; The text command information, the gesture command information, and the physiological state signals are fused to obtain the in-cabin driver interaction data.

4. The method as described in claim 1, characterized in that, The step of generating a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data includes: Based on the feature extraction network, features are extracted from the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data to obtain environmental feature vectors, driver feature vectors, and cooperative feature vectors. The environmental feature vector, the driver feature vector, and the collaborative feature vector are aligned and fused using a cross-modal attention mechanism to generate a fused feature tensor. Temporal dynamic feature extraction and spatial structure feature extraction are performed on the fused feature tensor to obtain a unified driving scene tensor.

5. The method as described in claim 1, characterized in that, The step of parsing the driving scenario tensor according to a preset intention collaborative parsing strategy to obtain the driver's target operation intention includes: Based on a preset intention-based collaborative parsing strategy, feature extraction is performed on the driving scenario tensor to obtain text interaction instructions and action behavior instructions. The text interaction instructions and action instructions are parsed based on the pre-trained language model and operation behavior sequence model to obtain the driver's explicit intention and the driver's implicit intention. The driver's target operational intention is obtained by quantifying the difference between the driver's explicit intention and the driver's implicit intention.

6. The method as described in claim 5, characterized in that, The step of obtaining the driver's target operational intention by differential quantification based on the driver's explicit intention and the driver's implicit intention includes: The degree of intent consistency is calculated based on the driver's explicit intent and the driver's implicit intent; When the consistency of intent is greater than a preset intent deviation threshold, the driver's explicit intent is taken as an intent to be confirmed. When the consistency of intent is less than or equal to a preset intent deviation threshold, a preset conflict arbitration sub-model is invoked to perform weighted fusion and risk scoring on the driver's explicit intent and the driver's implicit intent, resulting in multiple candidate intents with scores, and the candidate intent with the highest score is taken as the intent to be confirmed. The driver's target operational intent is determined based on the intent to be confirmed and the environmental constraints in the current driving scenario tensor.

7. The method as described in claim 6, characterized in that, The step of determining the driver's target operational intent based on the intent to be confirmed and the environmental constraints in the current driving scenario tensor includes: The intent to be confirmed is matched and verified with the environmental constraints in the current driving scenario tensor to obtain the matching result; When the matching result is a successful match, the intent to be confirmed is taken as the driver's target operational intent; When the matching result is a failure, the preset default operation intention will be used as the driver's target operation intention, which is an operation intention that prioritizes scenario safety.

8. An interactive control device for human-machine co-driving, characterized in that, The device includes: The data acquisition module is used to acquire multimodal sensor data of the vehicle, driver interaction data in the cockpit, and vehicle-road cooperative data; The data processing module is used to generate a unified driving scene tensor based on the multimodal sensor data, the driver interaction data, and the vehicle-road cooperative data; The intent parsing module is used to parse the driving scenario tensor according to a preset intent collaborative parsing strategy to obtain the driver's target operation intent; The intent conversion module is used to analyze the driver's target operation intent and the driving scenario tensor based on the preset human-machine co-driving model to obtain vehicle control commands; The interactive response module is used to control the vehicle to respond to the corresponding commands based on the vehicle control commands.

9. An interactive control device for human-machine co-driving, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the interactive control method for human-machine co-driving as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the interactive control method for human-machine co-driving as described in any one of claims 1 to 7.

Images