Vehicle control method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides a vehicle control method and device, equipment, a medium and a program product, and relates to the technical field of intelligent driving. The method comprises the steps that driver identity information is obtained based on information collected by a biological feature recognition module, a driving file corresponding to the driver identity information is obtained, the driving file is used for indicating driver driving preference, a source model is adjusted based on the driving file, a target model is obtained, and the target model is made to adapt to the driver driving preference. Multi-modal sensor data are obtained, the multi-modal sensor data comprise steering wheel holding state information and vehicle dynamic state information, and the intention recognition result is recognized based on the multi-modal sensor data and the target model; according to the method, the intention recognition model based on big data training is adjusted in a personalized manner by fusing the biological characteristics of the driver and the multi-modal sensing data, so that the individual driving preference is accurately adapted, and the accuracy and adaptability of driving intention recognition are improved.

Description

Technical Field

[0001] This application relates to intelligent driving technology, and more particularly to a vehicle control method, device, equipment, medium, and program product. Background Technology

[0002] With the rapid development of intelligent vehicle technology, the electric power steering (EPS) system, as a key interface connecting the driver and the vehicle, plays a decisive role in driving safety and handling experience. In real-world road environments, driving scenarios are complex and varied, and the driver's intentions are highly dynamic and uncertain.

[0003] Current vehicle control strategies typically rely on only a torque sensor and vehicle speed signal to calculate and adjust the steering assist, supplemented by preset driving modes (such as comfort mode or sport mode) to provide a certain degree of personalized experience.

[0004] However, most existing driving modes require drivers to actively and manually switch between them, lacking the ability to perceive and respond to operational intentions in real time. This makes it difficult to automatically and accurately match the assistance characteristics in different driving situations, thus limiting the level of intelligence in human-vehicle collaboration and further improvement of the driving experience. Summary of the Invention

[0005] This application provides a vehicle control method, apparatus, device, medium, and program product to address the problem of low accuracy in vehicle driving intention recognition in the prior art.

[0006] In a first aspect, embodiments of this application provide a vehicle control method applied to a vehicle, the vehicle including a steering wheel equipped with a biometric recognition module for verification, the method comprising:

[0007] Based on the information collected by the biometric identification module, the driver's identity information is obtained;

[0008] Obtain the driving profile corresponding to the driver's identity information; the driving profile is used to indicate the driver's driving preferences.

[0009] The source model is adjusted based on the driving record to obtain the target model, which is adapted to the driver's driving preferences. The source model is a model trained on big data for intent recognition.

[0010] Acquire multimodal sensor data, including steering wheel grip status information and vehicle dynamic status information;

[0011] Based on the multimodal sensor data and the target model, the intent recognition result is identified;

[0012] The electric power steering (EPS) motor in the vehicle is controlled based on the intent recognition result.

[0013] In one possible implementation, the source model includes a multi-module encoding layer and a temporal fusion Transformer layer. The multi-module encoding layer is used to convert multimodal data into feature vectors, and the temporal fusion Transformer layer is used to interact with the feature vectors in real time through a cross-modal self-attention mechanism to generate a unified context vector. Adjusting the source model based on the driving record to obtain the target model includes:

[0014] Based on the driving record, the source model is adjusted using a partial layer freezing fine-tuning strategy to obtain the target model; the partial layer unfreezing includes at least: an encoding layer for driver state processing and a layer for intent prediction.

[0015] In one possible implementation, controlling the electric power steering (EPS) motor in the vehicle based on the intent recognition result includes:

[0016] If the intent recognition result indicates a risk of lane departure, a command is sent to the EPS to instruct the adjustment of the EPS motor's assist curve to increase assist and reduce damping.

[0017] In one possible implementation, the steering wheel is further provided with a vibration module, and the method further includes:

[0018] When the intent recognition result indicates a risk of lane departure, the vibration module is controlled to generate vibration at the position corresponding to the deviating side. The greater the risk of lane departure, the higher the vibration frequency and the greater the intensity of the vibration module.

[0019] In one possible implementation, the multimodal sensor data is stored in the vehicle's electronic control unit (ECU), the target model is set in the vehicle's neural network processing unit (NPU), and the vehicle also includes a privacy gateway that is the only path for external requests to access the multimodal sensor data, including requests sent by devices other than the vehicle.

[0020] In one possible implementation, the multimodal sensor data includes publicly available data and sensitive data, and the method further includes:

[0021] Upon receiving the external request, the public data and desensitized data are transmitted through the privacy protection gateway. The desensitized data is obtained by desensitizing the sensitive data, and the desensitization process includes: dynamic encryption, anonymization, and / or aggregation.

[0022] If the external request is determined to be abnormal, the data link of the external request is blocked, the steering wheel vibration is controlled, and / or a prompt message is displayed.

[0023] In one possible implementation, the method further includes:

[0024] The gradient update amount of the source model is calculated locally in the vehicle using the driver's posture data;

[0025] The gradient update amount is encrypted and transmitted to the cloud, so that the cloud updates the source model based on the gradient update amounts from multiple vehicles;

[0026] Receive the updated source model from the cloud.

[0027] Secondly, embodiments of this application provide a vehicle control device applied to a vehicle, the device comprising:

[0028] The acquisition module is used to acquire driver identity information based on the information collected by the biometric recognition module;

[0029] The acquisition module is also used to acquire the driving profile corresponding to the driver's identity information, and the driving profile is used to indicate the driver's driving preferences;

[0030] The determination module is used to adjust the source model based on the driving record to obtain the target model, so that the target model is adapted to the driver's driving preferences. The source model is a model trained on big data for intention recognition.

[0031] The acquisition module is also used to acquire multimodal sensor data, which includes steering wheel grip status information and vehicle dynamic status information.

[0032] The identification module is used to identify the intent recognition result based on the multimodal sensor data and the target model;

[0033] A control module is used to control the electric power steering (EPS) motor in the vehicle based on the intent recognition result.

[0034] In one possible implementation, the vehicle control device further includes: an adjustment module;

[0035] The adjustment module is used to adjust the source model by combining the driving file and adopting a fine-tuning strategy of partial layer freezing to obtain the target model; the partial layer unfreezing includes at least: an encoding layer for driver state processing and a layer for intent prediction.

[0036] In one possible implementation, the vehicle control device further includes: a transmitting module;

[0037] The sending module is used to send a command to the EPS when the intent recognition result indicates that there is a risk of lane departure. The command is used to instruct the adjustment of the assist curve of the EPS motor to increase the assist and reduce the damping.

[0038] In one possible implementation, the vehicle control device further includes: a control module;

[0039] The control module is used to control the position of the vibration module corresponding to the deviating side to vibrate when the intent recognition result indicates that there is a risk of lane departure. The greater the risk of lane departure, the higher the vibration frequency and the greater the intensity of the vibration module.

[0040] In one possible implementation, the determining module is further configured to store the multimodal sensor data in the vehicle's electronic control unit (ECU), the target model is set in the vehicle's neural network processing unit (NPU), and the vehicle further includes a privacy protection gateway, which is the only path for external requests to access the multimodal sensor data, including requests sent by devices other than the vehicle.

[0041] In one possible implementation, the vehicle control device further includes: a transmission module;

[0042] The transmission module is used to transmit the public data and desensitized data through the privacy protection gateway when the external request is received. The desensitized data is obtained by desensitizing the sensitive data. The desensitization process includes dynamic encryption, anonymization and / or aggregation processing.

[0043] If the external request is determined to be abnormal, the data link of the external request is blocked, the steering wheel vibration is controlled, and / or a prompt message is displayed.

[0044] In one possible implementation, the vehicle control device further includes: a calculation module, an update module, and a receiving module;

[0045] The calculation module is used to calculate the gradient update amount of the source model locally on the vehicle using the driver's posture data.

[0046] The update module is used to encrypt and transmit the gradient update amount to the cloud, so that the cloud updates the source model based on the gradient update amount from multiple vehicles.

[0047] The receiving module is used to receive the updated source model from the cloud.

[0048] Thirdly, embodiments of this application provide a vehicle, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0050] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0051] The vehicle control method, device, equipment, medium, and program products provided in this application integrate driver biometric recognition with multimodal sensor data (such as steering wheel grip state and vehicle dynamic information). Based on a general intent recognition source model trained on big data, they are personalized to generate a target model that adapts to the specific driving preferences of drivers, thereby improving the accuracy and real-time adaptability of individual driving intent recognition. This not only overcomes the problem that traditional general models cannot take into account individual differences, but also enhances the intelligent driving system's ability to understand different driver behavior patterns, improving the naturalness and safety of human-vehicle interaction. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] Figure 1 Flowchart of the vehicle control method provided in this application Figure 1 ;

[0054] Figure 2 Flowchart of the vehicle control method provided in this application Figure 2 ;

[0055] Figure 3 A schematic diagram of the vehicle control device provided in this application;

[0056] Figure 4 This is a structural diagram of the vehicle provided in this application.

[0057] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0058] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0060] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0061] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0062] In the embodiments of this application, the use of terms such as "first" and "second" is to distinguish between identical or similar items that have essentially the same function and purpose. For example, "first vehicle" and "second vehicle" are merely used to distinguish different vehicles and do not limit their order of execution. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0063] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0064] First, let me explain the terms used in this application:

[0065] EPS (Electric Power Steering) is an electronic control system for automotive chassis that uses an electric motor to provide auxiliary steering force, reducing the difficulty for the driver to operate the steering wheel and improving steering precision.

[0066] ECU (Electronic Control Unit): It is the core control component of various electronic systems in a car. It receives signals from various sensors, processes them according to preset programs, and outputs commands to drive actuators to complete corresponding actions, thereby achieving precise control of systems such as the engine, chassis, and body.

[0067] NPU (Neural Processing Unit): A dedicated chip designed specifically for neural network computing tasks, capable of efficiently processing matrix operations related to deep learning in parallel, and widely used in artificial intelligence inference, image recognition and other scenarios.

[0068] CPU (Central Processing Unit): The core computing and control component of a computer or intelligent device, responsible for executing various program instructions, processing data, and coordinating the work of other hardware components in the system.

[0069] GPU (Graphics Processing Unit): A dedicated processor with a large number of parallel computing cores. Originally used for rendering computer graphics and images, it is now widely used in large-scale parallel data processing scenarios such as deep learning training and high-performance computing.

[0070] OBD (On-Board Diagnostics) is a standardized electronic system integrated into a vehicle that monitors the operating status of engine and emission-related systems and components in real time. When a fault is detected, a fault light is illuminated and a fault code is stored. The fault code can be read by professional equipment through a standard diagnostic interface to assist in repair and ensure emission compliance.

[0071] With the rapid advancement of intelligent vehicle technology, the electric power steering (EPS) system, as a crucial interaction interface between the driver and the vehicle, directly impacts driving safety and handling comfort. In actual road driving, the driving environment is complex and ever-changing, and the driver's intentions are highly dynamic and uncertain.

[0072] Current vehicle control strategies primarily rely on a single torque sensor and vehicle speed signal to calculate and adjust steering assist, and provide a limited personalized experience through preset driving modes (such as comfort or sport). While this approach satisfies different preferences to some extent, its perception dimension is relatively singular, making it difficult to fully capture the driver's true intentions.

[0073] However, existing driving modes typically require manual switching by the user, lacking the ability to perceive and adaptively adjust to changes in driving behavior and scenarios in real time. This prevents the system from automatically matching the optimal assistance characteristics in diverse and dynamic driving situations, thus hindering further improvements in the level of intelligence in human-vehicle collaboration and the overall driving experience.

[0074] To address the aforementioned issues, this application provides a vehicle control method that accurately obtains the current driver's identity information through a biometric identification module (such as fingerprint, face, or iris scan) and retrieves their unique driving profile, which records the driver's historical driving preferences (such as acceleration habits, lane change frequency, following distance, etc.). Subsequently, the driving profile is used to fine-tune or adjust the parameters of a source intent recognition model trained on large-scale general data to generate a target model specific to the driver. Based on this, the system collects multimodal sensor data in real time (including dynamic information such as steering wheel grip state, vehicle acceleration, steering angle, and speed) and inputs this data into the personalized target model, thereby outputting a driving intent recognition result that better matches the driver's behavioral habits. This method effectively integrates individual biometrics and real-time sensor information, improving the accuracy and adaptability of judging the intent of a specific driver while retaining the generalization ability of the big data model.

[0075] This application can be applied to automobiles and mobile terminals, and is suitable for solving the problem of low accuracy in vehicle driving intention recognition in the prior art.

[0076] The implementing entity of this application can be the control system of the aforementioned vehicle, or it can be the control system of a mobile terminal. This application does not impose any special restrictions in this regard.

[0077] The technical solutions of this application will be described in detail below with reference to specific embodiments. The specific embodiments described below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0078] Figure 1 Flowchart of the vehicle control method provided in this application Figure 1 .like Figure 1 As shown, the method is applied to a vehicle, which includes a steering wheel equipped with a biometric recognition module for verification. The method includes:

[0079] S101: Obtain driver identity information based on information collected by the biometric recognition module.

[0080] Understandably, driver identity information refers to the characteristic data and associated information collected based on the biometric identification module to uniquely identify the vehicle operator, including biometric information and linked personal registration information.

[0081] The biometric identification module collects information including fingerprints, palm veins, faces, voiceprints, finger veins, and multiple biometric features.

[0082] The driver's identity information can be obtained through a biometric recognition module integrated into the steering wheel, which collects the driver's unique biometric data, matches and verifies it against a pre-stored feature template, and retrieves the bound identity information after a successful match.

[0083] For example, a fingerprint recognition sensor is built into the steering wheel. When the driver holds the steering wheel, the sensor collects the fingerprint texture features and compares the collected fingerprint data with the owner's fingerprint template pre-stored in the vehicle system. If the comparison matches, the system obtains the driver's name, driver's license registration information, and vehicle access level and other identity data.

[0084] A near-infrared palm vein scanner is integrated into a specific area of ​​the steering wheel. When the driver's palm rests on the steering wheel, near-infrared light penetrates the skin of the palm and captures an image of the distribution of veins. The system extracts feature points from the vein image and compares them with a template library. Once verified, the driver's identity information is obtained. This method has high anti-counterfeiting capabilities and cannot be forged by copying fingerprints or other methods.

[0085] Miniature cameras, either above or integrated into the steering wheel, can capture the driver's facial features (such as facial contours and key facial coordinates). Combined with in-vehicle light compensation technology, facial feature acquisition is completed under different lighting conditions. The system matches the collected data with pre-stored facial templates, confirms identity, and retrieves driver information. It can also be combined with liveness detection (such as blinking and mouth opening verification) to prevent photo and 3D model forgery.

[0086] The steering wheel has a built-in microphone that can capture the driver's voice commands or preset voice snippets (such as "start the vehicle"). The system extracts the voiceprint features (such as frequency, timbre, and intonation) and compares them with pre-stored voiceprint templates. If a match is found, the driver's identity information is obtained. This method can be combined with other biometric methods to achieve multi-factor authentication.

[0087] The steering wheel integrates a finger vein scanning module. The driver places their finger in a designated area, and the module uses near-infrared light to capture the distribution characteristics of veins inside the finger. The system extracts the vein feature data and compares it with a template library. Once verified, it is associated with the driver's identity information. This method is less affected by external environmental interference and has a high degree of uniqueness.

[0088] The steering wheel integrates two or more biometric modules (such as fingerprint + facial recognition, or fingerprint + palm vein recognition). The system simultaneously collects multiple biometric data and performs multi-dimensional cross-verification; only when all features match successfully will the driver's identity information be retrieved. This method improves the accuracy and security of identity recognition and is suitable for scenarios with high access control requirements (such as commercial vehicles and special vehicles).

[0089] Obtaining driver identity information can prevent unauthorized personnel from driving vehicles and reduce the risk of vehicle theft; at the same time, if an unregistered driver (such as someone without a driver's license) is identified, the system can restrict the vehicle from starting, avoiding potential risks of illegal driving.

[0090] Different vehicle usage permissions can also be assigned to different drivers, such as differentiating the operating permissions of vehicle owners, family members, and temporary users (e.g., whether or not they are allowed to adjust core vehicle system parameters).

[0091] It can also automatically load personalized settings based on the driver's identity, such as seat position, rearview mirror angle, in-car navigation preferences, music playlists, etc., to improve driving convenience.

[0092] S102: Obtain the driving profile corresponding to the driver's identity information. The driving profile is used to indicate the driver's driving preferences.

[0093] Understandably, a driver's profile refers to a structured data set that is associated with the driver's identity information, records and stores the driver's personalized driving habits and preferences, and is used to support the vehicle system in automatically adapting to the driving experience.

[0094] The driver's identity information can be obtained through manual preset entry. Drivers can manually set and save their personal driving preferences via the in-vehicle central control screen, a mobile phone-vehicle interconnection APP, or steering wheel buttons. The system then binds these settings with the driver's identity information to generate a driving profile. For example, if a driver manually adjusts the seat back angle, steering wheel height, and in-vehicle air conditioning temperature to a comfortable state, and clicks "Save Preferences" on the central control screen, the system will associate these parameters with the driver's fingerprint information to generate the corresponding driving profile.

[0095] Alternatively, an automatic learning-based generation method can be used. The vehicle's onboard driving data acquisition module continuously records the driver's operational behaviors and settings adjustments during driving. After algorithmic analysis to extract fixed preferences, a driving profile is automatically generated. For example, the system can record the driver's accelerator pedal response sensitivity adjustment habits, braking force preferences, and navigation route selection preferences (such as preference for highways or shortest paths). Through machine learning, random operations are eliminated, stable preference features are extracted, and driving profiles are generated and updated.

[0096] It can also be achieved through cross-device synchronization, relying on the vehicle-to-everything (V2X) cloud platform, to synchronize the driver's driving profile established on other terminals or vehicles to the current vehicle, enabling the sharing of preferences across multiple devices and vehicles. For example, if a driver has already established a driving profile including seat memory and music playlists in another family vehicle of the same brand, the current vehicle can synchronize this profile from the cloud by logging into the vehicle brand's cloud account, without needing to set it up again.

[0097] The system can also collect data through intelligent interactive methods. The vehicle can proactively ask the driver for their preferences via voice assistant or touch interaction, and then record the feedback information into the driver's profile. For example, when a driver who has logged in for the first time starts the vehicle, the voice assistant will proactively ask "Do you want to set the usual air conditioning temperature?" and "What is your preferred route type for navigation?" After the driver responds by voice, the system will store this information in the driver's profile.

[0098] Accessing driving profiles enhances the personalization and convenience of driving, optimizes the driving experience, and helps improve driving safety.

[0099] The vehicle can automatically adapt to user preferences based on the driver's profile, eliminating the need for manual adjustments each time. For example, the seat automatically adjusts to the memory position after the driver gets in the car, the navigation system automatically recommends frequently used routes, and the music app automatically plays the favorite playlist, greatly improving ease of use.

[0100] The system can match the driver's operating habits based on the driving profile. For example, it can adjust the throttle response sensitivity and increase the braking assistance for novice drivers to reduce the risks caused by the mismatch between the vehicle's operating characteristics and the driver's habits.

[0101] Driving profiles can enable vehicle manufacturers to provide customized services, such as pushing information on services along the way based on the driver's travel route preferences and providing energy-saving driving suggestions based on energy consumption habits.

[0102] S103: Adjust the source model based on the driving record to obtain the target model, so that the target model is adapted to the driver's driving preferences. The source model is a model trained on big data and used for intent recognition.

[0103] Understandably, adjusting the source model based on the driving record to obtain the target model can be achieved by using personalized driving operation data (such as command triggering habits and intention expression preferences) in the driving record to perform small-batch gradient updates on the parameters of the fully connected layer or attention mechanism layer of the source model, thereby adapting to the individual driver's intention recognition preferences while retaining the big data generalization ability of the source model.

[0104] It can also be used to analyze the features corresponding to the driver's high-frequency intentions in the driving file (such as voice command keywords and operation action features), adjust the weight ratio of different input features in the source model, strengthen the feature weights that are strongly correlated with the driver's preferences, and weaken the interference of irrelevant features to obtain the target model.

[0105] It can also extract driver-specific intent samples (such as custom voice commands or operational intents in specific scenarios) from driving records, and use them as incremental datasets to retrain the source model, enabling the target model to learn and recognize the driver's personalized intent expression patterns.

[0106] Furthermore, by combining the driver's intent type distribution in the driving profile (such as high-frequency navigation intent and low-frequency entertainment control intent), the network branches in the source model related to the driver's low-frequency intent can be pruned, and a lightweight sub-network adapted to the driver's high-frequency intent can be added to obtain a simplified and personalized target model.

[0107] It can also dynamically adjust the confidence threshold of the source model's output intent based on the historical accuracy data of driver intent recognition in the driving file. It can directly output the intent with high confidence for the driver and trigger a secondary confirmation mechanism for intent with low confidence, thereby obtaining a target model that adapts to the driver's intent expression habits.

[0108] A target model adapted to the driver's driving preferences is obtained to improve the accuracy, real-time performance, and personalized adaptation capabilities of intent recognition.

[0109] The source model, trained on general big data, has a low accuracy rate in recognizing the personalized intentions of individual drivers (such as dialect voice commands and custom operating habits). The target model, adapted to the driver's profile, can accurately capture the driver's intention characteristics and reduce misidentification. For example, it avoids misinterpreting "turn down the air conditioning" as "open the sunroof".

[0110] The target model, after pruning or lightweighting, eliminates redundant computational branches irrelevant to the driver, resulting in higher computational efficiency, faster response to driver commands, and reduced interaction latency. For example, after a voice command is issued, the target model can execute the operation in a shorter time.

[0111] The target model can recognize and adapt to the driver's unique intention expression habits, without requiring the driver to deliberately use standardized commands. For example, it supports the driver's dialect and verbal commands, while prioritizing responses to their high-frequency intention needs, making vehicle intention interaction more in line with personal usage habits. Examples include commuting route navigation and frequently used music playback.

[0112] S104: Acquire multimodal sensor data, including steering wheel grip status information and vehicle dynamic status information.

[0113] Understandably, the way to obtain multimodal sensor data is to use pressure sensors, capacitance sensors, and angle sensors integrated inside the steering wheel to collect real-time information on the driver's grip, such as pressure distribution, contact position, steering wheel rotation angle, and speed.

[0114] It can also collect vehicle dynamic status information such as driving speed, acceleration, turning radius, and vehicle posture by relying on acceleration sensors, gyroscopes, vehicle speed sensors, and steering angle sensors mounted on the vehicle chassis and body.

[0115] It can also receive raw data from steering wheel grip status sensor and vehicle dynamic sensor simultaneously through automotive-grade data fusion module, and integrate them into structured multimodal sensor data after timestamp alignment and noise filtering.

[0116] It can also collect raw data on steering wheel grip status and vehicle dynamic status from vehicle terminal sensors, and upload the data to the cloud server through the vehicle networking module. The cloud then completes the aggregation and format standardization of multi-source data to generate a unified multimodal sensor dataset.

[0117] It can also connect the data streams from steering wheel sensors and vehicle dynamic sensors to the edge computing nodes deployed on vehicle terminals, and perform preliminary data screening, feature extraction and modal fusion locally to obtain lightweight multimodal sensor data.

[0118] The purpose of acquiring multimodal sensor data, including steering wheel grip status information and vehicle dynamic status information, is to provide multi-dimensional and highly reliable perception basis for intelligent vehicle decision-making. By combining the changes in steering wheel grip pressure and contact position with the matching degree of vehicle speed and steering action, it is possible to determine whether the driver is in a state of fatigued driving or distracted driving (for example, holding the steering wheel lightly with one hand for a long time + frequent vehicle deviation), and trigger safety warnings in a timely manner.

[0119] By fusing and analyzing multimodal data, the driver's driving intentions can be accurately identified (for example, increased grip on the steering wheel and changes in steering angle indicate an upcoming lane change), making the responses of driver assistance functions such as adaptive cruise control and lane keeping more in line with the driver's operating habits.

[0120] Multimodal data can cross-verify the authenticity of vehicle operation status. For example, when the steering wheel grip status shows no one is holding it, but the vehicle dynamic data shows that it is in motion, the system can determine the abnormality and initiate emergency measures (such as deceleration and alarm) to avoid the risks of autonomous driving.

[0121] S105: Based on multimodal sensor data and target model, identify the intent recognition result.

[0122] Understandably, there are multiple ways to identify the driving intention based on multimodal sensor data and target models: extracting features and aligning dimensions of steering wheel grip state information (pressure distribution, contact position) and vehicle dynamic state information (vehicle speed, steering angle), inputting them into a target model that adapts to driver preferences, strengthening the feature weights that are strongly correlated with the intention through the model's attention mechanism, and outputting the driver's driving intention recognition result.

[0123] The time-series data collected by multimodal sensors (e.g., a 5-second sequence of steering wheel grip force changes + vehicle acceleration changes) are segmented and analyzed. The analyzed time-series features are input into the target model and matched with the pre-stored time-series feature library of typical driver intentions in the model. The intention recognition result with the highest matching degree is output.

[0124] First, the steering wheel grip state information is used to make a preliminary intention prediction through the target model. Then, the prediction result is verified by combining the vehicle dynamic state information. When the intention judgments corresponding to the two types of data are consistent, the final result is output. When they are inconsistent, the model’s refined reasoning process is triggered to complete the judgment.

[0125] The target model prioritizes detecting abnormal features in multimodal sensor data (e.g., sudden release of the steering wheel + rapid vehicle acceleration), matches them with preset dangerous intent feature templates in the model, and quickly outputs special intent recognition results such as "emergency avoidance" and "sudden loss of control," before performing subsequent analysis on regular intents.

[0126] The target model performs independent intent reasoning on steering wheel grip state information and vehicle dynamic state information respectively and outputs their respective confidence scores. Based on the correlation weight between the two types of data in the driver's driving record and the intent, a weighted confidence score is calculated, and the intent with the highest weighted confidence score is selected as the final recognition result.

[0127] The aim is to improve the accuracy, real-time performance, and scene adaptability of driving intention recognition by using multimodal sensor data and target models adapted to driver preferences.

[0128] Relying solely on single data points such as steering wheel grip status or vehicle dynamics can easily lead to misjudgments due to environmental interference or accidental operations (e.g., holding the steering wheel with one hand may be misjudged as distracted driving). Cross-validation of multimodal data can effectively eliminate interference and improve the reliability of intent recognition.

[0129] The target model is optimized based on the driver's driving profile and can accurately capture the correlation pattern between the driver's operating habits and intentions (such as a driver's habit of lightly turning the steering wheel to change lanes). Compared with the general model, it can reduce the recognition bias caused by personalized operation.

[0130] Accurate intent recognition results can directly drive the vehicle's intelligent assistance functions (such as adjusting the lane keeping strategy in advance after predicting lane change intent, and triggering a warning after recognizing fatigue intent), achieving seamless connection between driving intent and vehicle operation, and improving the safety and comfort of intelligent driving.

[0131] S106: Control the EPS motor of the electric power steering system in the vehicle based on the intent recognition result.

[0132] Understandably, controlling the EPS motor based on intent recognition results can involve adjusting the power assist torque output by the EPS motor in real time according to the intent recognition results (such as "high-speed lane change" or "low-speed parking"). In high-speed scenarios, the power assist torque is reduced to improve steering stability, while in low-speed scenarios, the power assist torque is increased to reduce steering effort.

[0133] It can also control the EPS motor to adjust the damping coefficient of the steering system when it recognizes the driver's intention to "avoid emergency" or "make a sharp turn", thereby increasing the steering return force, avoiding oversteering of the vehicle, and improving the handling safety in emergency situations.

[0134] Based on the intent recognition results (such as "smooth cruise" and "aggressive overtaking"), the power response rate of the EPS motor can be adjusted to match the driver's steering operation rhythm. Under aggressive driving intent, the motor response speed is increased, and under smooth driving intent, the response speed is slowed down to ensure smoothness.

[0135] It can also combine the intent recognition results with the driver's driving profile to control the EPS motor to output torque according to the driver's exclusive assist characteristic curve. For example, it can provide linearly increasing assist torque for novice drivers and provide a sportier low assist setting for experienced drivers.

[0136] It can also detect abnormal intentions such as "driver distraction" and "steering wheel detachment" and control the EPS motor to temporarily take over the steering assist function, output a small correction torque to maintain the stability of the vehicle's driving trajectory, and trigger a safety warning at the same time.

[0137] Controlling the EPS motor of the electric power steering system based on intent recognition results enables intelligent, personalized, and safe adaptation of the steering system.

[0138] Traditional EPS motors mostly use a fixed assist strategy, which cannot match different driving scenarios and driver operation needs; dynamic control based on intent recognition can adjust the assist torque as needed, making steering operations in different working conditions such as low-speed parking and high-speed lane changing more effortless and convenient.

[0139] By recognizing dangerous intentions such as emergency avoidance or loss of control of the steering wheel, the EPS motor can actively intervene to adjust steering damping or torque, assisting the driver in maintaining vehicle handling stability and reducing the risk of loss of control due to operational errors or sudden situations.

[0140] By combining the driver-specific intent features output by the target model, the EPS motor can match the driving habits of different drivers (such as the high assistance needs of novice drivers and the low assistance preference of experienced drivers), making the steering feel more in line with the personal operating style and enhancing the personalization of the driving experience.

[0141] The vehicle control method provided in this application embodiment obtains driver identity information based on information collected by a biometric recognition module, obtains a driving profile corresponding to the driver identity information, and uses the driving profile to indicate the driver's driving preferences. Based on the driving profile, the source model is adjusted to obtain a target model, making the target model adaptable to the driver's driving preferences. The source model is a model trained on big data for intention recognition. Multimodal sensor data is acquired, including steering wheel grip state information and vehicle dynamic state information. Based on the multimodal sensor data and the target model, the intention recognition result is identified. This method integrates driver biometrics and multimodal sensor data, and personalizes the intention recognition model trained on big data, thereby accurately adapting to individual driving preferences and improving the accuracy and adaptability of driving intention recognition.

[0142] Figure 2 Flowchart of the vehicle control method provided in this application Figure 2 .like Figure 2 As shown. This embodiment is... Figure 1Based on the embodiments, the implementation process of the vehicle control method is described in detail. The vehicle control method provided in this embodiment includes:

[0143] S201: Obtain driver identity information based on information collected by the biometric recognition module.

[0144] Step S101 is similar to step S201, and will not be described again here.

[0145] S202: Obtain the driver's identity information corresponding to the driver's driving profile, which is used to indicate the driver's driving preferences.

[0146] Step S102 is similar to step S202, and will not be described again here.

[0147] S203: Combine driving records and use a fine-tuning strategy of partial layer freezing to adjust the source model to obtain the target model; partial layer unfreezing includes at least: an encoding layer for driver state processing and a layer for intent prediction.

[0148] The source model includes a multi-module encoding layer and a temporal fusion Transformer layer. The multi-module encoding layer is used to transform multimodal data into feature vectors, and the temporal fusion Transformer layer is used to interact with the feature vectors in real time through a cross-modal self-attention mechanism to generate a unified context vector.

[0149] Understandably, the underlying feature extraction layer in the frozen source model is trained based on general big data, the encoding layer and intent prediction layer used for driver state processing are unfrozen, personalized driver operation data and state labels in the driving file are input for mini-batch gradient descent training, and the parameters of the unfrozen layer are updated to obtain the target model.

[0150] The non-core network layers of the source model are frozen and the learning rate is set to 0. Differential learning rates are set for the unfrozen driver state processing encoding layer and intention prediction layer (low learning rate for encoding layer and high learning rate for prediction layer). Iterative training is performed based on the time-series driving preference data in the driving file to obtain the target model adapted to driver intention recognition.

[0151] Extract driver-specific state-intent association samples from driving records, freeze the general feature layer of the source model, input incremental samples into the unfrozen encoding and prediction layers for targeted training, constrain the unfrozen layer to learn the driver's personalized feature mapping relationship through a loss function, and generate the target model.

[0152] The basic network layer of the frozen source model is introduced into the driver state processing encoding layer, which introduces attention weights based on the driving profile to strengthen the feature encoding that is strongly correlated with the driver's driving habits. The intention prediction layer is trained simultaneously to complete the mapping from features to intentions, thus obtaining a personalized target model.

[0153] The general layers in the source model that are irrelevant to the driving scenario are frozen, the driver state processing encoding layer and the intent prediction layer are unfrozen, personalized data in the driving file are used as target domain samples, and transfer learning is performed by combining the general domain knowledge pre-trained in the source model. The parameters of the unfrozen layer are optimized through domain adaptation loss to obtain the target model.

[0154] While retaining the general intent recognition capabilities of the source model, it efficiently adapts to the driver's personalized driving habits, while also taking into account the model training efficiency and parameter optimization accuracy.

[0155] The source model is trained on massive general big data, and its bottom feature extraction layer has a stable general feature recognition capability. Freezing these layers can reduce the number of parameters and avoid overfitting due to the small number of driving file samples. Only unfreezing the core layers that are directly related to driver state encoding and intent prediction can achieve efficient training with small samples.

[0156] The driving profile contains the driver's unique operating habits, state characteristics, and intention association patterns. By specifically unfreezing the driver's state processing encoding layer, the model can learn the driver's state feature encoding rules (such as the correlation between steering wheel grip pressure and fatigue state). Unfreezing the intention prediction layer can optimize the mapping relationship between features and intentions, making the intention recognition of the target model more consistent with the driver's behavior patterns.

[0157] Freezing the non-core layers preserves the source model's ability to recognize general driving intentions, while unfreezing the core task layers gives the model the ability to recognize the driver's personalized intentions (e.g., recognizing the cautious steering intentions of a novice driver and the aggressive lane-changing intentions of an experienced driver), achieving the dual effect of "general foundation + personalized adaptation" and improving the comprehensiveness and accuracy of intention recognition.

[0158] S204: Acquire multimodal sensor data, including steering wheel grip status information and vehicle dynamic status information.

[0159] Step S104 is similar to step S204, and will not be described again here.

[0160] S205: Based on multimodal sensor data and target model, identify the intent recognition result.

[0161] Step S105 is similar to step S205, and will not be described again here.

[0162] S206: If the intent recognition result indicates a risk of lane departure, a command is sent to the EPS to instruct the adjustment of the EPS motor's assist curve to increase assist and reduce damping.

[0163] Understandably, adjusting the EPS motor assist curve under lane departure risk can be achieved by sending a linearly increasing torque command to the EPS when the intent recognition result indicates lane departure risk, dynamically raising the torque output slope of the assist curve, and simultaneously reducing the steering damping coefficient, thus realizing the instantaneous coordinated adjustment of increased steering assist and reduced damping.

[0164] It can also send the corresponding level of assist curve parameter package to EPS according to the lane departure risk level (mild / moderate / severe). For mild risk, the assist is slightly increased and the damping is reduced. For severe risk, the upper limit of assist torque is significantly increased and the damping threshold is simultaneously reduced.

[0165] It can also combine the driver's steering habits in the driving file, and send a personalized power assist curve adjustment command to the EPS when the risk of lane departure is detected. For drivers who prefer heavy steering, the power assist can be increased appropriately, and for drivers who prefer light steering, the power assist and damping parameters can be adjusted smoothly.

[0166] It can also send segmented instructions to the EPS within the time window when lane departure risk is detected. First, it instantly reduces the damping to reduce steering resistance, and then it increases the assist torque in stages to ensure the smoothness and continuity of the driver's operation when correcting the direction.

[0167] It can also perform cross-validation based on the intent recognition results and combined with vehicle dynamic status data (such as offset angle and vehicle speed). After confirming the risk of lane departure, it sends a precise adjustment command to the EPS, and adjusts the slope of the assist curve and the damping coefficient according to the vehicle speed (the assist increase is more gradual in high-speed scenarios and more obvious in low-speed scenarios).

[0168] Adjusting the EPS motor assist curve (increasing assist and reducing damping) when lane departure risk is present reduces the difficulty for the driver to correct the direction, improves the timeliness and accuracy of lane departure correction, and ensures driving safety.

[0169] When lane departure occurs, the driver needs to quickly turn the steering wheel to correct the driving trajectory. Increasing the EPS power assist torque and reducing the steering damping can reduce the force required to turn the steering wheel, allowing the driver to complete the direction adjustment more easily and quickly, and avoiding the delay in correction due to excessive steering resistance.

[0170] Lane departure risk is often accompanied by sudden situations (such as driver distraction or crosswind interference). In such situations, the driver's reaction time is limited. Adjusting the power steering curve can make the steering system more sensitive, match the need for rapid steering in emergency situations, and reduce the probability of the vehicle deviating from the lane.

[0171] By adjusting the power assist and damping parameters differently, the "kickback" phenomenon of the steering wheel caused by sudden changes in power assist can be avoided. At the same time, it ensures a smooth steering feel when correcting the direction, which not only guarantees the effectiveness of emergency corrections, but also takes into account the comfort of the driving process and prevents secondary risks caused by jerky operation.

[0172] The vehicle control method provided in this application first collects the driver's identity information through a biometric recognition module and retrieves the corresponding driving file, which records the driver's personalized driving preferences. Then, based on the driving file, a fine-tuning strategy of partial layer freezing is used to adjust the pre-trained source model, wherein at least the encoding layer used to process the driver's state and the output layer used to predict intent are unfrozen, thereby generating a target model adapted to the current driver. Next, the system integrates multimodal sensor data such as steering wheel grip state and vehicle dynamics, and uses the target model to identify driving intentions in real time. Once a lane departure risk is detected, a command is immediately sent to the electric power steering system (EPS) to dynamically adjust the assistance curve of the EPS motor, increasing assistance and reducing damping, thereby providing a more driver-friendly driving experience while ensuring safety.

[0173] In one possible implementation, the steering wheel is further provided with a vibration module, and the method further includes:

[0174] When the intent recognition result indicates that there is a risk of lane departure, the vibration module is controlled to generate vibration at the position corresponding to the side of departure. The greater the risk of lane departure, the higher the vibration frequency and the greater the intensity of the vibration module.

[0175] Understandably, the steering wheel also features a vibration module, specifically a high-fidelity tactile feedback module integrated inside the steering wheel. This module consists of a uniformly distributed array of piezoelectric ceramic actuators and can be precisely controlled by the ECU. It can generate vibration feedback with different frequencies, intensities, and patterns according to different scenarios. For example, when lane departure risk is detected, a clear "bumpy" vibration will be generated on the corresponding side of the steering wheel to intuitively warn the driver. When privacy and security risks such as abnormal data access from the OBD interface are detected, a warning can also be issued to the driver through a specific vibration pattern. At the same time, this module can also cooperate with the identity authentication process to provide feedback on the authentication results, forming a precise tactile interaction in multiple scenarios.

[0176] The way to control the steering wheel vibration module under lane departure risk is to map the lane departure risk level (low / medium / high) output by the intent recognition result to a preset vibration parameter library, and control the vibration module on the deviating side to output vibration according to the rule of "the higher the risk level, the more linearly the vibration frequency and intensity increases", so as to achieve graded warning.

[0177] It can also combine the intent recognition results with vehicle dynamic status data (such as deviation angle and vehicle speed) to calculate the risk weighting value, and dynamically adjust the frequency and intensity of the deviation side vibration module according to the weighting value. The larger the risk weighting value, the greater the improvement of vibration parameters.

[0178] It can also be based on the driver's sensitivity preference for vibration warning in the driving file. When lane departure risk is identified, a vibration strategy of "low frequency and low intensity start and step-by-step increase" is adopted for highly sensitive drivers, while high frequency and high intensity vibration is directly activated for low sensitive drivers, while ensuring that the vibration parameters are higher when the risk is higher.

[0179] It can also control the vibration module on the deviating side to output the basic frequency and intensity of vibration in the initial stage of identifying lane departure risk; if the risk continues to rise and no driver correction is detected, the vibration frequency and intensity will be gradually increased until the risk is eliminated.

[0180] Based on the partitioned layout of the steering wheel vibration module (independent vibration units on the left and right sides), after the intent recognition result clearly indicates a deviation from the direction, only the vibration unit corresponding to the deviation side is activated, and the vibration frequency and intensity of the unit are adjusted in real time according to the risk level, while the non-deviation side remains silent.

[0181] By using tactile warnings, drivers can be quickly and accurately reminded to correct their direction, thus improving the effectiveness of lane departure warnings and driving safety.

[0182] Compared to visual warnings (such as dashboard prompts) or auditory warnings (such as beeping sounds), the tactile vibrations of the steering wheel can be directly transmitted to the driver's hands, unaffected by blind spots or noise interference during driving. Furthermore, the precise vibrations on the deviating side can clearly indicate the direction of deviation, allowing the driver to quickly determine and correct the course of the vehicle.

[0183] By adopting the rule of "the greater the risk, the higher the vibration frequency and intensity", drivers can intuitively perceive the severity of the risk through the intensity of the vibration. When there is a slight deviation, a gentle reminder is given to avoid interfering with driving, while when there is a severe deviation, a strong warning is given to forcefully awaken the driver's attention, thus balancing the effectiveness of the warning and driving comfort.

[0184] When lane departure risk occurs, tactile warning can work in conjunction with EPS motor power assist adjustment to form a coordinated strategy. Vibration alerts the driver to intervene and correct the direction in time, while the EPS motor adjusts the power assist curve to reduce steering resistance. The combination of the two shortens the driver's reaction time, improves the timeliness of steering correction, and effectively reduces the probability of accidents such as vehicles leaving the lane or scraping with adjacent vehicles.

[0185] In one possible implementation, the multimodal sensor data is stored in the vehicle's electronic control unit (ECU), the target model is set in the vehicle's neural network processing unit (NPU), and the vehicle also includes a privacy gateway, which is the only path for external requests to access the multimodal sensor data. External requests include requests sent by devices other than the vehicle.

[0186] Understandably, there are various ways to store multimodal sensor data, deploy target models, and set up privacy protection gateways. For example, multimodal sensor data can be partitioned into "steering wheel grip data / vehicle dynamic data" and stored in the encrypted storage area of ​​the vehicle ECU. The target model can be permanently deployed in the dedicated neural network computing core of the NPU. The privacy protection gateway, as the only entry point for external requests, verifies the legitimacy of external requests through a preset device whitelist. Only after the verification is passed can the specified data access permissions be granted.

[0187] The ECU can also perform noise reduction and time alignment preprocessing on the raw data from multimodal sensors before storage. The target model is dynamically loaded into the NPU's computing memory based on the driving scenario requirements. The privacy protection gateway serves as the only access path, and bidirectional encryption is performed on external request instructions and transmitted data to prevent data from being stolen or tampered with during transmission.

[0188] Multimodal sensor data can also be distributed and stored in sub-modules of multiple vehicle ECUs. The target model is coordinated by the main NPU to perform parallel inference operations on multiple auxiliary NPUs. The privacy protection gateway serves as the sole channel for external requests and performs hierarchical authorization based on the type of external device (such as manufacturer maintenance equipment / third-party service equipment). Different devices can only access sensor data with corresponding permissions.

[0189] The ECU can also cache the real-time multimodal sensor data for a short time. The target model is deployed in the NPU after lightweight pruning to reduce computing power consumption. The privacy protection gateway serves as the only path for external requests, monitors the request frequency and data access range in real time, and intercepts abnormal requests (such as high-frequency batch data retrieval) to prevent the risk of data leakage.

[0190] The ECU can also store the hot data of multimodal sensor data locally and upload the cold data to the vehicle network cloud. The target model is deployed in a heterogeneous NPU (compatible with CPU+GPU collaborative computing). The privacy protection gateway serves as the only entry point for external requests, completing the conversion between external device communication protocols and vehicle bus protocols, while verifying the identity of the requester and recording access logs.

[0191] It achieves secure storage of vehicle data, efficient computation of models, and strict control of privacy data, balancing the real-time nature of intelligent driving with data security compliance.

[0192] Multimodal sensor data is stored in the ECU, which can leverage the short-range communication advantage between the ECU and the vehicle sensors to achieve low-latency data acquisition, preprocessing, and storage, avoiding delays caused by long-distance data transmission. The target model is deployed in a dedicated NPU, which has the hardware advantage of parallel processing of neural network operations, which can significantly improve the inference speed of intent recognition and meet the core requirement of "real-time response" of intelligent driving systems.

[0193] Multimodal sensor data contains sensitive information such as driver grip habits and vehicle driving status. The privacy protection gateway, as the only access path for external requests, can achieve "unified entry and unified management". Through mechanisms such as permission verification, encrypted transmission, and anomaly interception, it prevents external devices from accessing sensitive data without authorization, avoids the risk of data leakage and abuse, and complies with the relevant regulatory requirements for vehicle network data security.

[0194] The layered architecture of data storage (ECU), model computation (NPU), and privacy protection (gateway) enables the decoupling of functions of each module, facilitating individual upgrades and maintenance (such as iterative updates of the target model in the NPU and optimization of gateway protection strategies). At the same time, layered management can clarify the safety responsibility boundaries of each module, reduce the risk to the entire vehicle system caused by the failure of a single module, and improve the stability and reliability of the in-vehicle intelligent system.

[0195] In one possible implementation, the multimodal sensor data includes both publicly available data and sensitive data, and the method further includes:

[0196] Upon receiving an external request, public data and de-identified data are transmitted through a privacy protection gateway. De-identified data is obtained by de-identifying sensitive data, and the de-identification process includes: dynamic encryption, anonymization, and / or aggregation.

[0197] If an external request is determined to be abnormal, the data link of the external request is blocked, the steering wheel vibration is controlled, and / or a prompt message is displayed.

[0198] Understandably, the multimodal sensor data classification and transmission + abnormal request handling methods can preclassify multimodal sensor data into public data (such as the current vehicle speed range) and sensitive data (such as the driver's steering wheel grip pressure curve). Upon receiving an external request, the privacy protection gateway first verifies the requester's permissions, directly transmits public data, and only performs dynamic encryption and desensitization processing on sensitive data before transmission. If the request is determined to be abnormal, the data link is immediately blocked and a steering wheel vibration warning is triggered.

[0199] It can also classify and match data access permissions according to the type of external requester (such as manufacturer operation and maintenance / third-party services). The privacy protection gateway uses a combination of "anonymization + aggregation processing" to generate de-identified data for sensitive data, which is transmitted together with public data. If abnormal situations such as exceeding the request frequency limit are detected, the link is blocked and an abnormal prompt message is displayed on the central control screen at the same time.

[0200] The privacy protection gateway can also identify public and sensitive fields in multimodal sensor data in real time when receiving external requests, and dynamically switch the desensitization strategy according to the type of sensitive data (such as anonymizing driver identity-related data and aggregating driving behavior time series data), and transmit public and desensitized data. If the request is determined to be an illegal and unauthorized access, the link is blocked and the steering wheel vibration and the central control screen prompt are triggered at the same time.

[0201] The privacy protection gateway can also adapt the anonymization strength of sensitive data according to the business needs of external requests (e.g., high-privilege requests use light dynamic encryption, and low-privilege requests use deep anonymization + aggregation), and transmit the corresponding anonymized data and public data. If an abnormal signature of the request data is detected, the data link is blocked, and the steering wheel vibration or central control screen prompt is triggered according to the preset priority.

[0202] It can also perform full-process auditing of external request access behavior through a privacy protection gateway, directly transmit public data, and perform dynamic encryption, anonymization, and aggregation processing on sensitive data in sequence to generate multi-layer desensitized data, which is transmitted in a hierarchical manner as needed. If the audit finds abnormal access trajectory, the data link will be blocked immediately, and a linkage warning will be triggered to the steering wheel vibration and the display of prompt information.

[0203] While ensuring the legal external circulation of vehicle-mounted multimodal sensor data, it also achieves privacy protection for sensitive data and timely early warning of abnormal access, balancing the value of data utilization with the information security of vehicles and drivers.

[0204] By dividing multimodal sensor data into public and sensitive data, differentiated processing strategies can be implemented accordingly. Public data is directly transmitted to meet the routine business needs of external devices (such as vehicle maintenance data reporting). Sensitive data (such as driver grip habits and driving status data) is dynamically encrypted, anonymized, and aggregated to eliminate personal identification information and behavioral privacy features in the data, preventing the leakage of driver privacy.

[0205] As the sole entry point for data transmission, the privacy protection gateway identifies abnormal requests (such as unauthorized access or high-frequency batch data theft) through permission verification and behavior auditing, and promptly blocks the data link, thus preventing malicious attacks and data theft by external devices at the source. In addition, combined with warnings such as steering wheel vibration and central control screen prompts, it can promptly remind the driver that the vehicle is being subjected to abnormal data access.

[0206] Current regulations on vehicle-to-everything (V2X) data security clearly require the protection of sensitive vehicle and driver data. This mechanism, through full-process control of data classification, de-identification, and anomaly interception, can ensure the compliance of data transmission to external parties. At the same time, transparent de-identification strategies and perceptible anomaly warnings can enhance drivers' trust in the security of in-vehicle system data and promote the compliant implementation of intelligent driving technology.

[0207] In one possible implementation, the gradient update amount of the source model is calculated locally on the vehicle using the driver's stance data.

[0208] The gradient update values ​​are encrypted and transmitted to the cloud, so that the cloud updates the source model based on the gradient update values ​​from multiple vehicles.

[0209] Receive the updated source model from the cloud.

[0210] Understandably, the NPU in the vehicle calls the driver's driving data locally, calculates the gradient update of the source model based on the gradient descent algorithm, encrypts the gradient data using asymmetric encryption technology through a privacy protection gateway, and then uploads it to the cloud. The cloud aggregates the gradient update of multiple vehicles to complete the iteration of the source model, and the vehicle then receives the updated source model from the cloud through an encrypted channel.

[0211] The vehicle ECU first performs noise reduction and feature extraction preprocessing on the driver's driving data, calculates the gradient update amount of the source model locally and performs gradient compression (removing redundant gradient information), and transmits it to the cloud after encryption. The cloud aggregates the gradient update amounts of multiple vehicles based on the federated learning framework to update the source model, and the vehicle receives the updated source model through a secure communication protocol.

[0212] The gradient update is calculated locally on the vehicle only for the unfrozen layers (driver state coding layer and intent prediction layer) of the source model. The gradient of the frozen layer is not included in the calculation. The gradient data is encrypted with a dynamic key and then uploaded to the cloud. The cloud aggregates the gradient update amounts of multiple vehicles to perform incremental iterative updates on the source model. The vehicle receives the model incremental update package sent by the cloud and completes the local model update.

[0213] After the vehicle calculates the gradient update amount of the source model locally, it first performs local verification (verifies the validity of the gradient), then writes the gradient data into the blockchain block to complete the encrypted signature and transmits it to the cloud. The cloud completes the source model update based on the consensus mechanism of multi-vehicle gradient data, and the vehicle synchronizes the updated source model from the cloud blockchain node.

[0214] After accumulating a certain amount of driver driving data locally, the vehicle calculates the gradient update amount of the source model in batches. After being relayed and encrypted by the edge computing node, it is transmitted to the cloud. The distributed server cluster in the cloud aggregates the gradient update amounts of multiple vehicles to complete the source model update. The vehicle receives the updated source model from the cloud through the edge node.

[0215] While ensuring the security of driver privacy data, the source model is optimized through collaborative iteration, taking into account both personalized model adaptation and improved global generalization capabilities.

[0216] Driving data (such as grip habits and operating preferences) is sensitive and private information of drivers. Calculating gradient update amounts locally in the vehicle, rather than directly uploading raw driving data to the cloud, avoids the risk of leakage of raw privacy data. Encrypting the transmission of gradient data further prevents the data from being stolen or tampered with during transmission, thus building a strong privacy protection line from the data source and the transmission link.

[0217] By aggregating gradient updates from multiple vehicles in the cloud to update the source model, the driving behavior characteristics of different drivers can be integrated, allowing the source model to learn more comprehensive driving scenario patterns and improve the model's global generalization ability. Compared with models trained on single vehicle data, the collaboratively updated model has higher accuracy in tasks such as intent recognition and risk warning, and is more adaptable to a wider range of scenarios.

[0218] The vehicle only calculates the gradient update amount, without having to bear the computational load of large-scale model training, and can adapt to the computational power limitations of vehicle hardware; after the cloud completes the global model update, it is sent to the vehicle. The vehicle then makes fine adjustments based on the updated source model and local driving profiles, which can both maintain the global universality of the model and meet the personalized adaptation needs of individual drivers, achieving the dual goals of "global optimization + local customization".

[0219] Figure 3 A schematic diagram of the vehicle control device provided in this application is shown below. Figure 3 As shown, this application provides a vehicle control device applied to a vehicle, the device comprising:

[0220] The acquisition module 301 is used to acquire driver identity information based on the information collected by the biometric recognition module.

[0221] The acquisition module 301 is also used to acquire the driving profile corresponding to the driver's identity information, and the driving profile is used to indicate the driver's driving preferences;

[0222] The determination module 302 is used to adjust the source model based on the driving record to obtain the target model, so that the target model is adapted to the driver's driving preferences. The source model is a model trained based on big data and used for intent recognition.

[0223] The acquisition module 301 is also used to acquire multimodal sensor data, which includes steering wheel grip status information and vehicle dynamic status information.

[0224] The recognition module 303 is used to recognize the intent recognition result based on multimodal sensor data and target model;

[0225] Control module 304 is used to control the electric power steering (EPS) motor in a vehicle based on intent recognition results.

[0226] In one possible implementation, the vehicle control device further includes: an adjustment module 305;

[0227] The adjustment module 305 is used to adjust the source model by combining the driving file and adopting a fine-tuning strategy of partial layer freezing to obtain the target model; the partial layer unfreezing includes at least: an encoding layer for driver state processing and a layer for intent prediction.

[0228] In one possible implementation, the vehicle control device further includes: a transmitting module 306;

[0229] The sending module 306 is used to send a command to the EPS when the intent recognition result indicates that there is a risk of lane departure. The command is used to instruct the adjustment of the EPS motor's assist curve to increase assist and reduce damping.

[0230] In one possible implementation, the vehicle control device further includes: a control module 307;

[0231] The control module 307 is used to control the vibration module to generate vibration at the position corresponding to the deviating side when the intent recognition result indicates that there is a risk of lane departure. The greater the risk of lane departure, the higher the vibration frequency and the greater the intensity of the vibration module.

[0232] In one possible implementation, the determining module 302 is further configured to store the multimodal sensor data in the vehicle's electronic control unit (ECU), set the target model in the vehicle's neural network processing unit (NPU), and the vehicle also includes a privacy protection gateway, which is the only path for external requests to access the multimodal sensor data, including requests sent by devices other than the vehicle.

[0233] In one possible implementation, the vehicle control device further includes: a transmission module 308;

[0234] The transmission module 308 is used to transmit public data and desensitized data through a privacy protection gateway when receiving an external request. The desensitized data is obtained by desensitizing sensitive data. The desensitization process includes dynamic encryption, anonymization and / or aggregation processing.

[0235] If an external request is determined to be abnormal, the data link of the external request is blocked, the steering wheel vibration is controlled, and / or a prompt message is displayed.

[0236] In one possible implementation, the vehicle control device further includes: a computing module 309, an updating module 310, and a receiving module 311;

[0237] The calculation module 309 is used to calculate the gradient update amount of the source model locally using the driver's posture data.

[0238] The update module 310 is used to encrypt and transmit the gradient update amount to the cloud, so that the cloud updates the source model based on the gradient update amount from multiple vehicles.

[0239] The receiving module 311 is used to receive the updated source model from the cloud.

[0240] The vehicle control device provided in this application embodiment can be used to execute the technical solution of the vehicle control method in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again here.

[0241] Figure 4 This is a structural diagram of the vehicle provided in this application. Figure 4 As shown, the vehicle 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 40 also includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0242] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0243] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0244] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0245] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0246] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0247] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0248] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0249] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0250] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0251] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0252] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0253] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0254] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0255] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0256] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A vehicle control method, characterized in that, Applied to a vehicle, the vehicle including a steering wheel equipped with a biometric recognition module for verification, the method includes: Based on the information collected by the biometric identification module, the driver's identity information is obtained; Obtain the driving profile corresponding to the driver's identity information; the driving profile is used to indicate the driver's driving preferences. The source model is adjusted based on the driving record to obtain the target model, which is adapted to the driver's driving preferences. The source model is a model trained on big data for intent recognition. Acquire multimodal sensor data, including steering wheel grip status information and vehicle dynamic status information; Based on the multimodal sensor data and the target model, the intent recognition result is identified; The electric power steering (EPS) motor in the vehicle is controlled based on the intent recognition result.

2. The method according to claim 1, characterized in that, The source model includes a multi-module encoding layer and a temporal fusion Transformer layer. The multi-module encoding layer is used to transform multimodal data into feature vectors, and the temporal fusion Transformer layer is used to interact with the feature vectors in real time through a cross-modal self-attention mechanism to generate a unified context vector. The process of adjusting the source model based on the driving record to obtain the target model includes: Based on the driving record, the source model is adjusted using a partial layer freezing fine-tuning strategy to obtain the target model; the partial layer unfreezing includes at least: an encoding layer for driver state processing and a layer for intent prediction.

3. The method according to claim 1, characterized in that, The control of the electric power steering (EPS) motor in the vehicle based on the intent recognition result includes: If the intent recognition result indicates a risk of lane departure, a command is sent to the EPS to instruct the adjustment of the EPS motor's assist curve to increase assist and reduce damping.

4. The method according to claim 3, characterized in that, The steering wheel is also equipped with a vibration module, and the method further includes: When the intent recognition result indicates a risk of lane departure, the vibration module is controlled to generate vibration at the position corresponding to the deviating side. The greater the risk of lane departure, the higher the vibration frequency and the greater the intensity of the vibration module.

5. The method according to any one of claims 1-4, characterized in that, The multimodal sensor data is stored in the vehicle's electronic control unit (ECU), the target model is set in the vehicle's neural network processing unit (NPU), and the vehicle also includes a privacy protection gateway, which is the only path for external requests to access the multimodal sensor data. The external requests include requests sent by devices other than the vehicle.

6. The method according to claim 5, characterized in that, The multimodal sensor data includes publicly available data and sensitive data, and the method further includes: Upon receiving the external request, the public data and desensitized data are transmitted through the privacy protection gateway. The desensitized data is obtained by desensitizing the sensitive data, and the desensitization process includes: dynamic encryption, anonymization, and / or aggregation. If the external request is determined to be abnormal, the data link of the external request is blocked, the steering wheel vibration is controlled, and / or a prompt message is displayed.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: The gradient update amount of the source model is calculated locally in the vehicle using the driver's posture data; The gradient update amount is encrypted and transmitted to the cloud, so that the cloud updates the source model based on the gradient update amounts from multiple vehicles; Receive the updated source model from the cloud.

8. A vehicle, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images