A system and method for driving intention prediction and active service dynamic scheduling

Abstract

The application discloses a driving intention prediction and active service dynamic scheduling system and method based on end-side multi-modal fusion, and relates to the technical field of intelligent networked vehicles.The system comprises a multi-source heterogeneous data acquisition module, an end-side privacy calculation protection module, an end-side lightweight multi-modal intention prediction module, an active service strategy matching module, a dynamic priority scheduling module and a cabin execution module.The application collects multi-dimensional data such as driver biological characteristics, vehicle behavior and environment, and completes privacy protection processing at the end side;lightweight scheduling module Transformer scheduling module and a scenario-based attention mechanism are used to realize accurate prediction of driving intention;based on the driving safety level, multi-service conflict resolution and dynamic scheduling are realized, and safe warning, comfortable adjustment and information entertainment services are sequentially executed;the application realizes the upgrade of the cabin from passive response to active prediction, improves driving safety, driving experience and data privacy protection level, and is suitable for various intelligent cabin platforms.

Description

Technical Field

[0001] This invention relates to the field of intelligent cockpit technology for automobiles, and in particular to a system and method for predicting driving intentions and dynamically scheduling proactive services. Background Technology

[0002] With the rapid development of intelligent connected vehicles, the intelligent cockpit has become the core carrier of automotive intelligence. Currently, mainstream intelligent cockpits rely primarily on passive voice wake-up and touch operation, unable to proactively anticipate the driver's intentions. Cockpit service triggering is delayed, making it difficult to match real-time driving needs. In complex scenarios, concurrent operation of multiple services can easily lead to task conflicts, distracting the driver and affecting driving safety.

[0003] The existing technology has the following drawbacks:

[0004] 1. Low accuracy and poor real-time performance in intent prediction: It relies heavily on a single data source and does not dynamically adapt to the scenario, resulting in inaccurate prediction results and high latency;

[0005] 2. Disordered service scheduling: The lack of a priority mechanism based on security levels leads to frequent conflicts between multiple services, interfering with driving.

[0006] 3. Significant privacy risks: Uploading sensitive biometric data such as eye movement and heart rate to the cloud poses a risk of leakage during transmission and storage, and cloud computing exacerbates latency.

[0007] 4. Fragmented system architecture: Intent prediction and proactive services are separated, and there is no end-to-end integration and privacy protection.

[0008] Therefore, the industry urgently needs a system that can perform high-precision intention prediction, proactive intelligent scheduling, and protect driver privacy throughout the entire process, all locally on the vehicle side. Summary of the Invention

[0009] This invention aims to solve at least one of the technical problems existing in the prior art, and proposes a system and method for predicting driving intentions and dynamically scheduling proactive services.

[0010] In a first aspect, embodiments of the present invention provide a system for predicting driving intentions and dynamically scheduling proactive services, comprising: a multi-source heterogeneous data acquisition module, an edge-side privacy computing protection module, an edge-side lightweight multimodal intention prediction module, a proactive service strategy matching module, a dynamic priority scheduling module, and a cockpit execution module; wherein:

[0011] The multi-source heterogeneous data acquisition module is used to collect multi-source heterogeneous data of the driver, including biometric data, vehicle operation behavior data, vehicle operating status data and external environment data.

[0012] The edge privacy computing protection module is used to perform local desensitization processing on driver biometric data to protect driver data privacy.

[0013] The edge-side lightweight multimodal intent prediction module is used to perform feature fusion and temporal reasoning on multi-source heterogeneous data and output the driver's driving intent at a preset time in the future.

[0014] The active service strategy matching module is used to match active cockpit services based on driving intention, external environment data and vehicle operating status.

[0015] The dynamic priority scheduling module is used to detect conflicts among multiple service tasks, prioritize them according to driving safety levels, and generate scheduling instructions.

[0016] The cockpit execution module is used to execute scheduling commands and complete the cockpit's proactive service output.

[0017] Furthermore, the hardware of the multi-source heterogeneous data acquisition module includes: a heart rate sensor and eye movement acquisition module integrated into the steering wheel, a multi-point pressure sensor arranged in the seat, a driver's forward vision camera, a vehicle operation sensor, an in-vehicle camera, a millimeter-wave radar, a navigation module, and a weather sensor.

[0018] Furthermore, the biometric data includes at least eye movement data and heart rate data, the vehicle operation behavior data assistant includes pedal data and steering wheel data, the vehicle operating status data includes at least vehicle speed and RPM, and the external environment data includes at least road conditions and weather data.

[0019] Furthermore, the edge-side privacy computing protection module is used to perform local de-identification processing on the driver's biometric data, specifically including:

[0020] After the multi-source heterogeneous data acquisition module acquires the driver's biometric data, the data is first de-identified locally by the edge privacy computing protection module to remove the driver's unique identification information from the data. Then, differential privacy noise is added to the de-identified data to protect the driver's data privacy. Therefore, the feature extraction, fusion, and inference calculation of biometric data are all completed in the vehicle's edge inference chip, and no data is transmitted back to the cloud throughout the process.

[0021] Furthermore, the edge-side lightweight multimodal intent prediction model adopts a lightweight Transformer architecture to preprocess the collected multi-source heterogeneous data and extract feature vectors from each data source. Then, through the model's built-in contextual attention mechanism, dynamic weights are assigned to each feature vector according to the current driving scenario.

[0022] Furthermore, current driving scenarios include at least highway scenarios, fatigue scenarios, and parking scenarios; among them, highway scenarios increase the weight of vehicle operation and environmental data, fatigue scenarios increase the weight of eye movement, heart rate, and facial posture, and parking scenarios increase the weight of steering wheel and surrounding environment.

[0023] Furthermore, the proactive service strategy matching module is used to match proactive cockpit services based on driving intention, external environment data, and vehicle operating status. The specific method includes: the proactive service strategy matching module receives the driving intention prediction result, combines the real-time driving scenario and vehicle status, retrieves a preset service strategy library, intelligently matches and outputs suitable proactive cockpit services; when the predicted driving intention is charging and the external environment indicates that there is a charging station nearby, it matches charging station reservation and navigation route preloading; when the predicted driving intention is fatigue driving, it matches fatigue warning, air conditioning ventilation, and seat massage services.

[0024] Furthermore, the priorities of the dynamic priority scheduling module, from high to low, are: driving safety warning services, driving comfort adjustment services, and infotainment services.

[0025] Secondly, embodiments of the present invention provide a method for predicting driving intentions and dynamically scheduling proactive services, including:

[0026] Collect multi-source heterogeneous data of drivers, including biometric data, vehicle operation behavior data, vehicle operating status data, and external environment data.

[0027] Local anonymization processing is performed on driver biometric data to protect driver data privacy;

[0028] Feature fusion and temporal reasoning are performed on multi-source heterogeneous data to output the driver's driving intention at a future preset time;

[0029] Active cockpit services are matched based on driving intentions, external environmental data, and vehicle operating status.

[0030] Conflict detection is performed on multiple service tasks, and priorities are assigned according to driving safety levels to generate scheduling instructions;

[0031] Execute dispatch instructions to complete the active cabin service output.

[0032] Thirdly, embodiments of the present invention provide an electronic device, characterized in that it includes:

[0033] One or more processors;

[0034] Memory, used to store one or more programs;

[0035] When the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0036] This invention discloses a driving intention prediction and proactive service dynamic scheduling system and method based on edge-side multimodal fusion, relating to the field of intelligent connected vehicle technology. The system includes a multi-source heterogeneous data acquisition module, an edge-side privacy computing protection module, an edge-side lightweight multimodal intention prediction module, a proactive service strategy matching module, a dynamic priority scheduling module, and a cockpit execution module. This invention collects multi-dimensional data such as driver biometrics, vehicle behavior, and environmental data, and performs privacy protection processing on the edge. It employs a lightweight scheduling module (Transformer scheduling module) and a contextualized attention mechanism to achieve accurate prediction of driving intentions. Based on driving safety levels, it resolves and dynamically schedules multiple service conflicts, enabling the orderly execution of safety warnings, comfort adjustments, and infotainment services. All data is computed locally, preventing privacy leaks. This invention upgrades the cockpit from passive response to proactive prediction, improving driving safety, driving experience, and data privacy protection, and is applicable to various intelligent cockpit platforms.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This invention uses the fusion analysis of multi-source heterogeneous data and combines a contextual attention mechanism to assign dynamic weights to each data source. Compared with the intention prediction of a single data source, it greatly improves the accuracy and real-time performance of driving intention prediction and can accurately predict the driver's future driving intention.

[0039] 2. This invention deploys the intent prediction model on the vehicle side to achieve local data computation, avoiding cloud network latency. At the same time, the edge-side privacy computation protection module completes the desensitization and differential privacy processing of biometric data, thus eliminating the risk of privacy leakage from a technical perspective.

[0040] 3. The dynamic priority scheduling module set up in this invention resolves conflicts and dynamically arranges multiple service tasks based on driving safety levels, ensuring that driving safety warning services are executed first, effectively avoiding the problem of multiple service conflicts interfering with driving safety, and improving the safety of the driving process;

[0041] 4. This invention realizes the interactive upgrade of the cockpit from "passive response" to "active prediction". The seamless active service based on driving intention greatly improves the driver's cockpit experience. Moreover, the system architecture is lightweight and easy to deploy and apply in passenger cars and commercial vehicles, and has broad industrial application value. Attached Figure Description

[0042] Figure 1A structural block diagram of a system for predicting driving intentions and dynamically scheduling proactive services provided in an embodiment of the present invention;

[0043] Figure 2 A flowchart illustrating a method for predicting driving intentions and dynamically scheduling proactive services provided in an embodiment of the present invention;

[0044] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0046] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0047] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0048] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0049] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0050] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0051] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a driving intention prediction and proactive service dynamic scheduling system and method based on edge multimodal fusion.

[0052] This implementation discloses a system for predicting driving intentions and dynamically scheduling proactive services, such as Figure 1 It includes: a multi-source heterogeneous data acquisition module, an edge-side privacy computing protection module, an edge-side lightweight multimodal intent prediction module, a proactive service strategy matching module, a dynamic priority scheduling module, and a cockpit execution module; among which:

[0053] The multi-source heterogeneous data acquisition module is used to collect multi-source heterogeneous data of the driver, including biometric data, vehicle operation behavior data, vehicle operating status data and external environment data.

[0054] In this embodiment, the hardware of the multi-source heterogeneous data acquisition module includes: a heart rate sensor and eye movement acquisition module integrated into the steering wheel, a multi-point pressure sensor arranged in the seat, a driver's forward vision camera, a vehicle operation sensor, an in-vehicle camera, a millimeter-wave radar, a navigation module, and a weather sensor.

[0055] Specifically, the hardware deployment of this system is all based on the vehicle side, including: integrating a heart rate sensor and eye-tracking acquisition module into the steering wheel to collect driver heart rate and eye-tracking data; arranging multiple pressure sensors on the seat cushion and backrest to collect driver posture pressure distribution data; placing a vision camera in front of the driver to collect driver facial posture data; arranging sensors on the vehicle pedals, steering wheel, and gear shift lever to collect vehicle operation behavior data; equipping the vehicle with an edge-side inference chip (such as an automotive-grade AI chip) to deploy a lightweight multimodal intent prediction model and achieve local data computation; and also equipping the vehicle with an in-vehicle camera, millimeter-wave radar, vehicle navigation module, and weather sensor to collect external environmental data; and the vehicle's main unit, as the core control unit of the system, connects the various acquisition modules and execution modules to realize data interaction and command issuance.

[0056] In this embodiment, the biometric data includes at least eye movement data and heart rate data, the vehicle operation behavior data assistant includes pedal data and steering wheel data, the vehicle operating status data includes at least vehicle speed and engine speed, and the external environment data includes at least road conditions and weather data.

[0057] Specifically, the multi-source heterogeneous data acquisition module uses the aforementioned hardware devices to achieve real-time synchronous acquisition of data across all dimensions. The acquisition frequency is dynamically adjusted according to the data type: biometric data (eye movement, heart rate, etc.) is acquired at a frequency of 10Hz, vehicle operation behavior data (pedal, steering wheel, etc.) is acquired at a frequency of 20Hz, vehicle operating status data (vehicle speed, RPM, etc.) is acquired at a frequency of 15Hz, and external environmental data (road conditions, weather, etc.) is acquired at a frequency of 5Hz. All acquired data is transmitted to the edge inference chip in real time and is not uploaded to the cloud.

[0058] The edge privacy computing protection module is used to perform local desensitization processing on driver biometric data to protect driver data privacy.

[0059] In this embodiment, the edge privacy computing protection module is used to perform local de-identification processing on the driver's biometric data, and the specific method includes:

[0060] After the multi-source heterogeneous data acquisition module acquires the driver's biometric data, the data is first locally anonymized by the edge-side privacy computing protection module to remove the driver's unique identification information. Then, differential privacy noise is added to the anonymized data to protect the driver's data privacy. Therefore, feature extraction, fusion, and inference calculation of biometric data are all completed in the vehicle's edge inference chip, without any data transmission to the cloud. The edge-side privacy computing protection module's calculation process avoids data transmission to the cloud throughout the entire process, preventing the risk of privacy leakage of biometric data from the source.

[0061] The lightweight multimodal intent prediction module on the edge is used to perform feature fusion and temporal reasoning on multi-source heterogeneous data and output the driver's driving intent at a preset time in the future.

[0062] In this embodiment, the edge-side lightweight multimodal intent prediction model adopts a lightweight Transformer architecture to preprocess the collected multi-source heterogeneous data and extract feature vectors from each data source. Then, through the model's built-in contextual attention mechanism, dynamic weights are assigned to each feature vector based on the current driving scenario. The current driving scenario includes at least highway, fatigue, and parking scenarios. Specifically, highway scenarios increase the weight of vehicle operation and environmental data, fatigue scenarios increase the weight of eye movement, heart rate, and facial posture, and parking scenarios increase the weight of steering wheel and surrounding environment data.

[0063] Specifically, the lightweight multimodal intent prediction model on the edge adopts a lightweight Transformer architecture to preprocess the collected multi-source heterogeneous data and extract feature vectors from each data source. Then, through the model's built-in contextual attention mechanism, dynamic weights are assigned to each feature vector according to the current driving scenario (such as highway driving, parking, fatigue driving, etc.). For example, in the parking scenario, the weights of steering wheel rotation data and surrounding environment data are increased, and in the fatigue driving scenario, the weights of eye movement, heart rate, and facial posture data are increased. Finally, through temporal correlation analysis, the preprocessed fused features are inferred to output the driver's driving intent prediction results for the next 3-10 seconds (overtaking, parking, charging, resting, changing lanes, etc.). The prediction results are transmitted to the active service strategy matching module in real time.

[0064] The active service strategy matching module is used to match active cockpit services based on driving intention, external environment data and vehicle operating status.

[0065] In this embodiment, the proactive service strategy matching module is used to match proactive cabin services based on driving intention, external environment data, and vehicle operating status. The specific method includes: the proactive service strategy matching module receives the driving intention prediction result, combines the real-time driving scenario and vehicle status, retrieves a preset service strategy library, intelligently matches and outputs suitable proactive cabin services; when the predicted driving intention is charging and the external environment is that there is a charging station nearby, it matches charging station reservation and navigation route preloading; when the predicted driving intention is fatigue driving, it matches fatigue warning, air conditioning ventilation, and seat massage services.

[0066] For example, when parking intention is predicted, services such as seat adjustment, HUD switching, and 360° panoramic imaging are matched; when fatigue intention is predicted, services such as voice warning, air conditioning external circulation, seat massage, and window slightly opening are matched; when charging intention is predicted, services such as charging pile selection, reservation, navigation preloading, and low power mode are matched.

[0067] The dynamic priority scheduling module is used to detect conflicts among multiple service tasks, prioritize them according to driving safety levels, and generate scheduling instructions.

[0068] Specifically, if the dynamic priority scheduling module matches multiple active service tasks, it transmits all tasks to the dynamic priority scheduling module. The scheduling module first performs conflict detection on the tasks. For example, "HUD display content switching" and "driving fatigue warning" do not conflict and can be executed in parallel; "entertainment audio playback" and "traffic sign voice reminder" conflict and need to be executed serially.

[0069] The dispatch module prioritizes service tasks based on driving safety levels, from highest to lowest: driving safety warning services (such as fatigue warnings and avoidance reminders) → driving comfort adjustment services (such as air conditioning adjustment and seat adjustment) → infotainment services (such as navigation broadcasts and audio playback).

[0070] The scheduling module dynamically arranges the service execution order based on the priority division results. Conflicting tasks are executed sequentially according to priority, while non-conflicting tasks are executed in parallel. Then, the scheduling instructions are sent to each execution component in the cockpit to complete the triggering of proactive services.

[0071] The cockpit execution module is used to execute scheduling commands and complete the cockpit's proactive service output. Specifically, when there are conflicting tasks, they are executed sequentially according to priority; when there are no conflicting tasks, service tasks are executed in parallel. Subsequently, the scheduling commands are sent to each execution component of the cockpit to trigger the proactive service.

[0072] This embodiment discloses a driving intention prediction and proactive service dynamic scheduling system based on edge-side multimodal fusion. The system includes a multi-source heterogeneous data acquisition module, an edge-side privacy computing protection module, an edge-side lightweight multimodal intention prediction module, a proactive service strategy matching module, a dynamic priority scheduling module, and a cockpit execution module. This invention collects multi-dimensional data such as driver biometrics, vehicle behavior, and environment, and performs privacy protection processing on the edge. It employs a lightweight scheduling module (Transformer scheduling module) and a contextual attention mechanism to achieve accurate prediction of driving intentions. Based on driving safety levels, it resolves and dynamically schedules multiple service conflicts, enabling the orderly execution of safety warnings, comfort adjustments, and infotainment services. All data is computed locally, preventing privacy leaks. This invention upgrades the cockpit from passive response to proactive prediction, improving driving safety, driving experience, and data privacy protection, and is applicable to various intelligent cockpit platforms.

[0073] To better understand this embodiment, this embodiment provides examples of various specific application scenarios of this system:

[0074] Scenario 1: Parking Intent Prediction When a vehicle approaches a residential parking lot, its speed drops below 5 km / h, the steering wheel angle increases, and surrounding environmental data indicates a parking area. The edge-side intent prediction model, based on this data, predicts the driver's intention for the next 5 seconds as "parking." The proactive service strategy matching module matches the services of "automatic seat posture adjustment (lowering seat height to improve visibility) + HUD display content switching (displaying parking assist lines) + 360-degree panoramic imaging activation." The dynamic priority scheduling module determines that the above services do not conflict and executes them in parallel, realizing proactive cabin services in the parking scenario.

[0075] Scenario 2: Predicting Fatigue Driving Intent. After driving at high speed for more than 2 hours, the driver's eye movement data shows a decrease in blinking frequency and a shift in gaze, heart rate data shows a slowed heart rate, and facial posture data shows a drooping head. The model predicts that the driver has the intention to "drive while fatigued." The proactive service strategy matching module matches the services of "fatigue driving voice reminder + air conditioning external circulation + seat back massage + slightly open window." The dynamic priority scheduling module sets "fatigue driving voice reminder" as the highest priority, and the other comfort services are executed in parallel to promptly remind the driver and ensure driving safety.

[0076] Scenario 3: When the vehicle's remaining battery level is less than 20%, the driver uses the vehicle's navigation system to search for "charging stations." The model predicts that the driver's driving intention in the next 10 seconds is "charging." The proactive service strategy matching module matches the "nearby charging station filtering + charging station reservation + navigation route preloading + air conditioning low-power operation (saving power)" services. The module determines that there is no task conflict and executes them in parallel to improve the driver's charging experience.

[0077] Based on the same inventive concept, embodiments of the present invention also provide a method for predicting driving intentions and dynamically scheduling proactive services, such as... Figure 2 ,include:

[0078] S100. Collect multi-source heterogeneous data of the driver, including biometric data, vehicle operation behavior data, vehicle operating status data and external environment data;

[0079] S200. Perform local de-identification processing on driver biometric data to protect driver data privacy;

[0080] S300. Perform feature fusion and temporal reasoning on multi-source heterogeneous data to output the driver's driving intention at a future preset time;

[0081] S400. Matches proactive cockpit services based on driving intentions, external environmental data, and vehicle operating status;

[0082] S500 performs conflict detection on multiple service tasks, prioritizes them according to driving safety levels, and generates scheduling instructions.

[0083] S600 executes dispatch instructions to complete the active cockpit service output.

[0084] The specific working method of the driving intention prediction and proactive service dynamic scheduling method has been described in detail in a driving intention prediction and proactive service dynamic scheduling system, and will not be repeated here.

[0085] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 3This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0086] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0087] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0088] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0089] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described in the above embodiments. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0090] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method.

[0091] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0092] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0093] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0094] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0095] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0096] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0100] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A system for predicting driving intentions and dynamically scheduling proactive services, characterized in that, include: The system comprises a multi-source heterogeneous data acquisition module, an edge-side privacy computing protection module, an edge-side lightweight multimodal intent prediction module, a proactive service strategy matching module, a dynamic priority scheduling module, and a cockpit execution module; among which: The multi-source heterogeneous data acquisition module is used to collect multi-source heterogeneous data of the driver, including biometric data, vehicle operation behavior data, vehicle operating status data and external environment data. The edge privacy computing protection module is used to perform local desensitization processing on driver biometric data to protect driver data privacy. The edge-side lightweight multimodal intent prediction module is used to perform feature fusion and temporal reasoning on multi-source heterogeneous data and output the driver's driving intent at a preset time in the future. The active service strategy matching module is used to match active cockpit services based on driving intention, external environment data and vehicle operating status. The dynamic priority scheduling module is used to detect conflicts among multiple service tasks, prioritize them according to driving safety levels, and generate scheduling instructions. The cockpit execution module is used to execute scheduling commands and complete the cockpit's proactive service output.

2. The system according to claim 1, characterized in that, The hardware of the multi-source heterogeneous data acquisition module includes: a heart rate sensor and eye movement acquisition module integrated into the steering wheel, a multi-point pressure sensor arranged in the seat, a driver's forward vision camera, a vehicle operation sensor, an in-vehicle camera, a millimeter-wave radar, a navigation module, and a weather sensor.

3. The system according to claim 1, characterized in that, The biometric data includes at least eye movement data and heart rate data; the vehicle operation behavior data assistant includes pedal data and steering wheel data; the vehicle operating status data includes at least vehicle speed and engine speed; and the external environment data includes at least road conditions and weather data.

4. The system according to claim 1, characterized in that, The edge-side privacy computing protection module is used to perform local de-identification processing on driver biometric data. Specific methods include: After the multi-source heterogeneous data acquisition module acquires the driver's biometric data, the data is first de-identified locally by the edge privacy computing protection module to remove the driver's unique identification information from the data. Then, differential privacy noise is added to the de-identified data to protect the driver's data privacy. Therefore, the feature extraction, fusion, and inference calculation of biometric data are all completed in the vehicle's edge inference chip, and no data is transmitted back to the cloud throughout the process.

5. The system according to claim 1, characterized in that, The edge-side lightweight multimodal intent prediction model adopts a lightweight Transformer architecture to preprocess the collected multi-source heterogeneous data and extract the feature vectors of each data source. Then, through the model's built-in contextual attention mechanism, dynamic weights are assigned to each feature vector based on the current driving scenario.

6. The system according to claim 5, characterized in that, Current driving scenarios include at least highway scenarios, fatigue scenarios, and parking scenarios. Among them, highway scenarios increase the weight of vehicle operation and environmental data, fatigue scenarios increase the weight of eye movement, heart rate, and facial posture, and parking scenarios increase the weight of steering wheel and surrounding environment.

7. The system according to claim 1, characterized in that, The proactive service strategy matching module is used to match proactive cabin services based on driving intention, external environment data, and vehicle operating status. The specific method includes: the proactive service strategy matching module receives the driving intention prediction result, combines the real-time driving scenario and vehicle status, retrieves the preset service strategy library, intelligently matches and outputs the appropriate proactive cabin services; when the predicted driving intention is charging and the external environment is that there is a charging station nearby, it matches charging station reservation and navigation route preloading; when the predicted driving intention is fatigue driving, it matches fatigue warning, air conditioning ventilation and seat massage services.

8. The system according to claim 1, characterized in that, The priorities of the dynamic priority scheduling module, from highest to lowest, are: driving safety warning services, driving comfort adjustment services, and infotainment services.

9. A method for predicting driving intentions and dynamically scheduling proactive services, applied to the system described in any one of claims 1-8, characterized in that, include: Collect multi-source heterogeneous data of drivers, including biometric data, vehicle operation behavior data, vehicle operating status data, and external environment data. Local anonymization processing is performed on driver biometric data to protect driver data privacy; Feature fusion and temporal reasoning are performed on multi-source heterogeneous data to output the driver's driving intention at a future preset time; Active cockpit services are matched based on driving intentions, external environmental data, and vehicle operating status. Conflict detection is performed on multiple service tasks, and priorities are assigned according to driving safety levels to generate scheduling instructions; Execute dispatch instructions to complete the active cabin service output.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in claim 9.

Images