Vehicle control method and device, electronic equipment and storage medium

Abstract

The invention discloses a vehicle control method and device, electronic equipment and a storage medium, and relates to the technical field of vehicle auxiliary driving control. The method comprises the steps that built-in sensors are installed on the two sides of a steering wheel to obtain the grip strength, temperature and humidity of the hands of a driver and the heart rate, blood pressure and other data of the driver, and the data can be continuously detected and accurately obtained; the driving state of the driver is obtained according to the change degree of the data in combination with the information obtained by the driver monitoring system and the road condition information obtained by the navigation system, and the interaction strategy is executed according to the driving state to ensure that the driver is timely emotional guidance, so that the driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle driver assistance control technology, and more specifically, to a vehicle method, apparatus, electronic device, and storage medium. Background Technology

[0002] Every year, 37% of traffic accidents are caused by drivers' emotional problems, and more than 20% in China. Consumers' demand for vehicle safety and emotional management is increasing, and the popularity of related content such as psychological counseling and mental health assistance continues to rise. The DMS (Driver Monitoring System) has a facial recognition error rate of up to 30% in strong light and occluded scenes. The health detection data of DMS or OMS (Occupancy Monitoring System) is not accurate enough, especially in the recognition of users' emotions.

[0003] According to the National Highway Traffic Safety Administration (NHTSA), in 2024, drivers held the steering wheel with both hands naturally for 92% of the time. However, among the existing detection technologies, only the Direct Motion Monitor (DMS) can detect driver fatigue or dangerous driving behavior, and the Optical Motion Monitor (OMS) can detect passenger age or emotions. Both of these technologies use 2D or 3D cameras to detect eye movements and head positions, and cannot actually detect driver emotions through contact. Therefore, their accuracy is insufficient, and they cannot judge emotional fluctuations caused by changes in the environment. Summary of the Invention

[0004] In view of this, embodiments of this application propose a vehicle method, apparatus, electronic device, and storage medium that can promptly detect and effectively manage the driver's emotional fluctuations, thereby improving driving safety.

[0005] The following technical solution is adopted in this application.

[0006] In a first aspect, embodiments of this application provide a vehicle control method applied to a vehicle's driving system. The driving system includes a steering wheel, with built-in sensors disposed on both the left and right sides of the steering wheel. The method includes: During vehicle operation, multiple primary data points of the driver are acquired through built-in sensors. These primary data points include the driver's hand grip strength, temperature, humidity, heart rate, and blood pressure. Based on these primary data points, the driver's driving state is determined. The driving state indicates the degree of change between different primary data points in chronological order. Interaction strategies are executed based on the driving state to obtain control commands. These interaction strategies indicate whether the vehicle initiates voice interaction, voice reminders, or voice warnings to the driver. The vehicle is then controlled based on the control commands.

[0007] In some embodiments, before acquiring multiple first data points of the driver via built-in sensors during vehicle operation, the process includes: During vehicle idling, multiple secondary data points of the driver are acquired through built-in sensors. These secondary data points include the driver's hand grip strength, temperature, humidity, heart rate, and blood pressure. Based on these secondary data points acquired by the built-in sensors, reference data for the driver is determined.

[0008] In some embodiments, the driver's driving state is obtained based on multiple first data acquired by built-in sensors, including: Based on the time sequence of multiple first data points, determine the rate of change of each data point in the first data points; based on the first data points and reference data at the same time, determine the change value of each data point in the first data points; based on the rate of change and the change value, determine the driving state; the driving state is a first state, a second state, or a third state; different driving states are used to determine different interaction strategies.

[0009] In some embodiments, the driving system further includes a driver monitoring system and a navigation system, which execute interaction strategies based on the driving state and obtain control commands, including: During vehicle operation, third-party data of the driver is acquired. This third-party data is used to indicate: the driver's facial information obtained from the driver monitoring system, and the road condition information obtained from the navigation system. Based on different driving states, the interaction items in the interaction strategy are determined according to the third-party data.

[0010] In some embodiments, based on different driving states, the interaction items in the interaction strategy are determined according to third data, including: If the driving state is in the first state, the interaction items in the interaction strategy determined by the third data include voice interaction; if the driving state is in the second state, the interaction items in the interaction strategy determined by the third data include voice reminders; if the driving state is in the third state, the interaction items in the interaction strategy determined by the third data include voice warnings.

[0011] In some embodiments, the steering wheel is controlled by one side or both sides. During vehicle operation, multiple first data points of the driver are acquired via built-in sensors, including: If the steering wheel is controlled by one side, the data obtained from the built-in sensor of the one-side control is determined as the first data; if the steering wheel is controlled by both hands, the data with the higher value among the two sets of data obtained from the built-in sensor of the two-side control is determined as the first data.

[0012] According to a second aspect of the embodiments of this application, a vehicle control device is provided for a vehicle driving system. The driving system includes a steering wheel, and built-in sensors are respectively disposed on the left and right sides of the steering wheel. The device includes: The acquisition module is used to acquire multiple first data points of the driver through built-in sensors during vehicle operation; the first data points include the driver's hand grip strength, temperature, humidity, and the driver's heart rate and blood pressure; the first processing module is used to acquire the driver's driving state based on the multiple first data points acquired by the built-in sensors; the driving state is used to indicate the degree of change between different first data points along the time sequence of the multiple first data points; the second processing module is used to execute an interaction strategy based on the driving state to obtain control commands; the interaction strategy is used to indicate that the vehicle initiates voice interaction, voice reminders, or voice warnings to the driver; the control module is used to control the vehicle based on the control commands.

[0013] According to a third aspect of the embodiments of this application, an electronic device is provided, which includes a processor and a memory, wherein the memory stores instructions that are executed by the processor to implement the vehicle control method described above.

[0014] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein instructions are stored thereon, which, when executed by a processor or a computer device, implement the vehicle control method described above.

[0015] According to a fifth aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the vehicle control method described above.

[0016] In this application's solution, firstly, during vehicle operation, multiple primary data points are acquired in real time via built-in sensors, including the driver's hand grip strength, temperature, humidity, heart rate, and blood pressure. Data acquisition via a contact-based sensor on the steering wheel ensures data continuity and accuracy. Secondly, analysis of the rate or value of change in multiple dimensions of the primary data allows for the assessment of the driver's emotional fluctuations, fatigue, or sudden health issues, enabling timely identification of the driver's emotional state during driving. Finally, different interaction strategies are implemented based on different driving states, including voice interaction, voice reminders, or other methods to effectively manage the driver's emotions and improve driving safety.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 This is a schematic diagram of a vehicle control method provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram illustrating the installation of a built-in sensor in a vehicle, as provided in an embodiment of this application.

[0021] Figure 3 This is a schematic flowchart of a vehicle control method provided in an embodiment of this application.

[0022] Figure 4 This is a flowchart illustrating a method for obtaining driving status provided in an embodiment of this application.

[0023] Figure 5 This is a flowchart illustrating a steering wheel emotion perception method provided in an embodiment of this application.

[0024] Figure 6 This is a flowchart illustrating a method for executing an interaction strategy, as provided in an embodiment of this application.

[0025] Figure 7 This is a schematic diagram of the structure of a vehicle control device provided in an embodiment of this application.

[0026] Figure 8This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0027] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through specific embodiments. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] In conventional technologies, DMS primarily uses cameras to detect the driver's state and behavior in real time, while OMS also uses cameras to detect eye movements, head position, and other data to obtain information such as the passenger's emotions and age. However, in scenarios with strong light or when the driver's or passenger's face is obscured, the facial recognition error of DMS or OMS is large, resulting in inaccurate data related to emotion detection. Therefore, there are many potential traffic accident hazards caused by the driver's emotional fluctuations.

[0030] The vehicle control method provided in this application aims to solve the above-mentioned technical problems of the prior art.

[0031] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0032] Figure 1 This is a schematic diagram illustrating a vehicle control method provided in an embodiment of this application. Figure 1 As shown, the vehicle control method execution body provided in this application embodiment includes a first module 101 and a second module 102.

[0033] In one alternative implementation, the first module 101 and the second module 102 refer to software units or modules. Specifically, the first module 101 generates control instructions based on acquired first data, and the second module 102 executes the received control instructions.

[0034] In another alternative implementation, the first module 101 and the second module 102 refer to hardware devices.

[0035] For example, the first module 101 may include, but is not limited to, electronic devices with data processing capabilities such as computers, host computers, servers, or data centers.

[0036] For example, the second module 102 may include, but is not limited to, an application device with functions such as voice interaction and music playback.

[0037] As an example, the second module 102 may include, but is not limited to, the vehicle's central control display screen, air conditioning system, etc.

[0038] Optionally, the first module 101 and the second module 102 can communicate via wired or wireless connections. Wired connections may include, but are not limited to, buses, fiber optic cables, or network cables. Wireless connections may include, for example, transmission control protocol / internet protocol (TCP / IP), wireless local area network (WLAN) protocols, and remote direct memory access (RDMA) over converged ethernet (RoCE) protocols.

[0039] The following is combined Figure 1 The first module 101 and the second module 102 shown herein provide an illustrative description of the vehicle control method provided in this application embodiment: The first module 101 acquires multiple first data points of the driver, obtains the driver's driving state based on the multiple first data points, and executes an interaction strategy based on the driving state to obtain control commands. The first module 101 transmits the control commands to the second module 102 wirelessly or via wired connection, and the second module 102 executes the control commands.

[0040] Below Figure 1 Based on the first module 101 and the second module 102 shown, the vehicle control method provided in the embodiments of this application will be further described, such as... Figure 3 The diagram shown illustrates a vehicle control method. In a specific embodiment, this vehicle control method can be applied to, for example... Figure 7 The vehicle control device 700 and the electronic equipment 800 equipped with the vehicle control device 700 are shown. Figure 8 The specific process of the embodiments of this application will be described below. Of course, it is understood that this method can be executed by a cloud server with computing power.

[0041] like Figure 2The diagram illustrates the installation of a built-in sensor in a vehicle. In a specific embodiment, this vehicle control method can also be applied to the vehicle's driving system, which includes a steering wheel, a driver monitoring system, and a navigation system. Built-in sensors 201 and 202 are respectively installed on the left and right sides of the steering wheel, positioned below the leather layer. The built-in sensors 201 and 202 are connected to a power supply line via the center of the steering wheel, with power supplied by wires pre-embedded in the steering wheel frame. The built-in sensors combine multiple sensing modes and can detect changes in hand temperature, hand sweat, heart rate, and blood pressure, among other things.

[0042] like Figure 3 The diagram shown is a flowchart of a vehicle control method. The following will focus on... Figure 3 The process shown is described in detail. The vehicle control method may specifically include the following steps 301 to 304.

[0043] Step 301: During the vehicle's operation, several first data points of the driver are acquired through built-in sensors; the first data points include the driver's hand grip strength, temperature, humidity, as well as the driver's heart rate and blood pressure.

[0044] In this embodiment of the application, the first data is the physiological data of the driver during the driving process, which is detected by contact through a built-in sensor.

[0045] For example, by placing the built-in sensors on the left and right sides of the steering wheel, the driver can maintain continuous contact with the built-in sensors while holding the steering wheel, thus ensuring the continuity of the data detected by the built-in sensors. Moreover, compared to wearable contact sensors that require the driver to wear additional equipment, placing the sensors under the leather layer of the steering wheel makes it more convenient to obtain the driver's data, and the detection data will not be interrupted if the driver forgets to wear the device.

[0046] For example, multiple small ultrasonic transducers in the built-in sensors send ultrasonic waves through the steering wheel leather to the driver's palms where he is holding the steering wheel. After detecting signals such as hand temperature and humidity, the signals are transmitted back, thereby realizing real-time detection of the driver's grip strength, palm temperature, and humidity caused by sweating. This can be used to assess the driver's stress, tension, or fatigue level. At the same time, combined with the real-time detection of the driver's heart rate and blood pressure, the driver's condition can be judged more comprehensively and accurately, realizing driver health and emotional feedback.

[0047] Step 302: Based on multiple first data points acquired by the built-in sensors, the driver's driving state is obtained; the driving state is used to indicate the degree of change between different first data points along the time sequence of the multiple first data points.

[0048] In this embodiment of the application, the driving status is determined by analyzing the first data to identify the driver's emotional fluctuations, fatigue, or health problems.

[0049] In the first optional example, when a driver's emotions are affected by a sudden traffic situation while driving, such as a traffic accident ahead while driving on the highway, multiple vehicles brake suddenly, causing the driver to grip the steering wheel tightly and step on the brakes. At this time, the driver may experience changes in palm temperature, sweaty palms, rapid heartbeat, and increased blood pressure. The built-in sensors in the steering wheel detect the increase in the driver's grip strength, temperature and humidity, as well as the increase in the driver's heart rate and blood pressure, and determine that the driver's driving state is an emotional fluctuation.

[0050] In the second optional example, when a driver drives a vehicle for a long time, there may be situations such as sweaty hands and changes in palm temperature. The built-in sensors in the steering wheel detect the increase in temperature and humidity of the driver's hands. Combined with the driver monitoring system (DMS) capturing features such as the driver's eyelid closure, gaze direction, and head posture, the driver's driving state is determined to be fatigued driving.

[0051] In the third optional example, when a driver is driving, they may suddenly experience some physical discomfort. Due to tension, the driver's palms may sweat, and the hand temperature may suddenly rise or fall. At this time, the built-in sensor in the steering wheel detects changes in the driver's heart rate and blood oxygen concentration, as well as abnormal hand temperature and humidity and abnormal electrocardiogram detection, and judges that the driver's driving status is a health problem.

[0052] Step 303: Execute the interaction strategy according to the driving status to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminder or voice warning to the driver.

[0053] In this embodiment of the application, the interaction strategy is the interaction between the vehicle and the driver determined based on the driver's driving status.

[0054] In this embodiment, the control command refers to the instruction determined by the artificial intelligence system that can complete the execution action specified by the interaction strategy. These instructions can control various functions of the vehicle system, such as navigation, information query, air conditioning adjustment, and audio-visual entertainment, to improve the user's safety and experience during driving and using the vehicle.

[0055] In the first optional example, when the driver is experiencing emotional fluctuations and obtains road condition information about a traffic accident ahead on the highway from the navigation system, the vehicle's artificial intelligence system initiates voice interaction with the driver. For example, it first suggests that the driver remain calm to reduce stress hormone levels and minimize overactivation of the sympathetic nervous system, thus creating favorable conditions for subsequent emotional calming; secondly, it broadcasts navigation information explaining that the emergency braking was caused by the traffic accident and advises the driver to remain calm; finally, the vehicle plays soothing music and automatically adjusts the air conditioning temperature to alleviate the driver's high level of tension, providing a more comfortable in-car environment, thereby reducing the driver's fear and anxiety and relieving psychological stress.

[0056] In the second alternative example, when the driver is fatigued and obtains distance information to a highway service area from the navigation system, the vehicle's artificial intelligence system issues a voice reminder to the driver. For example, it may remind the driver that they are currently fatigued and suggest that they pull into a highway service area to rest.

[0057] In the third optional example, when the driver's driving condition indicates a health problem, the first data detected by the built-in sensors is transmitted to the vehicle's artificial intelligence system for analysis via LVDS (Low-Voltage Differential Signaling) or CAN (Controller Area Network) signals. Simultaneously, the vehicle's AI system issues a voice warning to the driver. For example, if the severity of the driver's health problem is assessed as minor, basic health guidance is provided; if the severity is assessed as significant, a one-button call for emergency medical services or emergency contacts is triggered; if the driver's health problem is assessed as potentially worsening, the vehicle will automatically activate its hazard lights, safely slow down, and pull over to the emergency lane, while automatically calling for emergency assistance to prevent the driver from becoming incapacitated due to a sudden health emergency and going unnoticed.

[0058] Step 304: Control the vehicle based on control commands.

[0059] For example, when the driver experiences emotional fluctuations and the vehicle's AI system initiates a voice interaction with the driver, the vehicle's voice interaction response program is triggered, issuing several specific in-vehicle commands, including setting the air conditioning temperature to 19°C and the fan speed to level 3, and playing soothing music on the vehicle's audio system at 25% volume. The vehicle's air conditioning, audio system, and other related actuators then execute these commands.

[0060] In this embodiment, firstly, the driver's hand temperature, hand humidity, heart rate, blood pressure, and other primary data are acquired in real time through the built-in sensors of the steering wheel. When changes in temperature or humidity, or abnormalities in heart rate or blood pressure are detected, the driver's driving state is determined, thereby enabling real-time detection of the driver's emotional state during driving. Secondly, corresponding interaction strategies are executed according to different driving states, triggering voice interaction, voice reminders, or other forms of communication between the vehicle and the driver, ensuring effective management of the driver's emotions and improving vehicle driving safety.

[0061] Building upon the above, this application provides an optional implementation method for obtaining the driver's driving state based on multiple first data points acquired by built-in sensors, such as... Figure 4 The flowchart shown is a method for obtaining driving status, which may specifically include the following steps 401 to 408.

[0062] Step 401: During the vehicle's stop, multiple second data points of the driver are acquired through built-in sensors; the second data points include the driver's hand grip strength, temperature, humidity, as well as the driver's heart rate and blood pressure.

[0063] In this embodiment of the application, the second data is physiological data of the driver when not driving the vehicle, which is obtained by contact through a built-in sensor.

[0064] For example, while the vehicle is stopped or getting in and out of the vehicle, the driver can perform basic personal health data checks by repeatedly gripping the steering wheel to maintain body and grip strength stability. This includes, but is not limited to, secondary data from multiple dimensions such as heart rate, blood pressure, blood oxygen concentration, hand grip strength, hand temperature, and humidity.

[0065] Step 402: Determine the driver's reference data based on multiple second data acquired by the built-in sensors.

[0066] For example, the acquired second data is automatically analyzed to generate personal health information which is stored in the vehicle's storage unit and can be used as reference data for the driver while driving. At the same time, in order to improve the accuracy of the second data, it can also be combined with the measurement results of DMS for data analysis to provide more accurate personal health information.

[0067] The steering wheel can be controlled by one side or both sides. In view of how to obtain multiple first data of the driver through built-in sensors during the driving process, this application embodiment provides an optional implementation method, which includes the following steps.

[0068] If the steering wheel is controlled by one side, the data obtained from the built-in sensor of the one-side control is determined as the first data; if the steering wheel is controlled by both hands, the data with the higher value among the two sets of data obtained from the built-in sensor of the two-side control is determined as the first data.

[0069] For example, while the driver is driving, an automatic adaptation algorithm is used based on the driver's hand grip on the steering wheel to ensure continuous contact and uninterrupted detection data. When the driver controls the steering wheel with one hand, the built-in sensor on the side of the steering wheel where the driver is holding the wheel detects the driver's first data in real time. When the driver controls the steering wheel with both hands, the built-in sensors on both sides of the steering wheel detect the driver's first data in real time, and the first data with the higher value is used to determine the driver's driving state.

[0070] Step 403: During the vehicle's operation, several first data points of the driver are acquired through built-in sensors; the first data points include the driver's hand grip strength, temperature, humidity, as well as the driver's heart rate and blood pressure.

[0071] Step 404: Determine the rate of change of each data point in the first data based on the time sequence of the multiple first data points.

[0072] For example, the rate of change of each data in the first data can be reflected as an increase in hand grip strength for a period of time, a brief increase in hand grip strength, or a gradual decrease in hand grip strength until the steering wheel is removed, a continuous decrease or abnormal increase in hand temperature, an increase in hand humidity, an excessively fast or slow heart rate, a continuous increase in blood pressure, or a sudden increase or decrease in blood pressure.

[0073] Step 405: Determine the change value of each data point in the first data based on the first data and the reference data at the same time.

[0074] For example, reference data is detected when the driver is in a normal or relaxed driving state. The reference data is characterized by stable hand grip strength that changes naturally with steering wheel turning, relatively stable hand temperature that is close to the individual's basal body temperature, dry hand humidity, heart rate that matches the resting heart rate and is stable, and blood pressure that fluctuates within the individual's normal range.

[0075] Step 406: Determine the driving state based on the rate of change and the value of change; the driving state is a first state, a second state, or a third state; different driving states are used to determine different interaction strategies.

[0076] In this embodiment, the driver's grip strength, temperature, humidity, heart rate, and blood pressure are analyzed over time to determine whether the data changes continuously or suddenly, based on the magnitude of the changes compared to the normal values ​​of each data point in the reference data, so as to comprehensively assess the driver's driving status.

[0077] For example, when a driver's heart rate and blood pressure rise rapidly, the humidity in their hands increases significantly due to sweating, their grip becomes unconsciously tense, and their hand temperature may drop due to vasoconstriction. At this time, the driver is in a state of emotional fluctuation due to tension or stress.

[0078] For example, when a driver's heart rate gradually slows down or an abnormal rhythm occurs, grip strength continuously weakens or there is a brief moment of hand release, and hand temperature may drop slightly due to decreased metabolism, the driver is in a state of drowsy driving.

[0079] Optionally, such as Figure 5 The diagram shows a flowchart of a steering wheel emotion perception method. When the steering wheel emotion perception detection function is activated for the first time, a reminder to the driver is sent through the vehicle display screen asking whether they agree to the collection of personal information. When the driver chooses to agree, the steering wheel emotion perception detection function is enabled by default by the vehicle's driving system, and the reminder to agree to the collection of personal information is no longer sent separately when the driver starts the vehicle thereafter.

[0080] Optionally, a one-button on / off function control can be provided, allowing the driver to select whether to enable or disable the linkage function on the in-vehicle display screen according to actual needs. When the linkage function is enabled, the steering wheel's built-in sensors dynamically detect first data, monitor the driver's driving status in real time, and generate corresponding interaction strategies based on the driving status to execute voice interaction or other interactive feedback. Simultaneously, the steering wheel's built-in sensors also statically detect second data after the linkage function is enabled. When the driver chooses not to enable the linkage function, the steering wheel emotion perception-related linkage functions are disabled, but this does not affect the independent operation of other module functions.

[0081] When the linkage function is enabled, the artificial intelligence system is activated to enable more human-like voice interaction between the vehicle and the driver, and permissions are granted to access other applications, such as navigation and music, to better achieve functional linkage. When the linkage function is disabled, the steering wheel emotion perception-related functions are disabled without affecting the independent operation of other modules.

[0082] Optionally, after the built-in sensor in the steering wheel completes the static detection of the second data, a pop-up window on the in-vehicle display screen prompts whether to generate information; after the driver agrees, the second data is analyzed, the analysis results are recorded, and personal health information is generated. Shareable content can be generated monthly, quarterly, or annually for easy access at any time; after the driver disagrees, a pop-up window on the in-vehicle display screen prompts whether to turn off the function.

[0083] Optionally, based on the pop-up prompt on the in-vehicle display screen, if the driver selects no, dynamic or static detection will continue until the detection is completed; if the driver selects yes, the system will redirect to the control interface, where the driver can actively disable the steering wheel emotion perception-related functions without affecting the independent operation of other modules.

[0084] Step 407: Execute the interaction strategy according to the driving status to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminder or voice warning to the driver.

[0085] Step 408: Control the vehicle based on control commands.

[0086] The specific steps of steps 403, 407 to 408 can be found in steps 301, 303 to 304, and will not be repeated here.

[0087] In this embodiment, the driver's condition is comprehensively judged by considering the magnitude and rate of change of multiple dimensions of physiological data, such as grip strength, temperature, humidity, heart rate, and blood pressure, under normal and abnormal conditions. This can improve the accuracy of the driving condition judgment.

[0088] Based on the above, this application provides an optional implementation method for executing interaction strategies and obtaining control commands according to the driving state, such as... Figure 6 The flowchart shown is a method for executing an interaction strategy, which may specifically include the following steps 601 to 606.

[0089] Step 601: During the vehicle's operation, several first data points of the driver are acquired through built-in sensors; the first data points include the driver's hand grip strength, temperature, humidity, as well as the driver's heart rate and blood pressure.

[0090] Step 602: Based on multiple first data points acquired by the built-in sensors, the driver's driving state is obtained; the driving state is used to indicate the degree of change between different first data points along the time sequence of the multiple first data points.

[0091] Step 603: During vehicle operation, acquire third data of the driver; the third data is used to indicate: the driver's facial information acquired from the driver monitoring system, and the road condition information of the vehicle during operation acquired from the navigation system.

[0092] In this embodiment of the application, the third data is the driver's personal data and trip data obtained from the vehicle's driver assistance system.

[0093] Optionally, the driver monitoring system and navigation system, as commonly used driver assistance systems, can use the third data they acquire to determine the interaction items that need to be executed in the interaction strategy based on the driver's state as determined by the first and second data.

[0094] Step 604: Based on different driving states, determine the interaction items in the interaction strategy according to third data.

[0095] For example, when the navigation system detects a traffic accident causing congestion ahead, it uses voice interaction to briefly explain the cause of the sudden braking and the estimated travel time to the driver, thus alleviating the driver's anxiety. When the driver monitoring system detects facial features such as yawning, frequent blinking, or a deviation from the road, it increases the volume of the voice reminders, lowers the air conditioning temperature, or plays upbeat music. When the driver monitoring system detects abnormal facial color, it automatically activates the hazard lights or switches to autonomous driving mode in addition to providing a voice warning to the driver.

[0096] Specifically, regarding how to determine the interaction items in the interaction strategy based on third data according to different driving states, this application embodiment provides an optional implementation method, which includes the following steps 614 to 634.

[0097] Step 614: If the driving state is the first state, determine the interactive items in the interaction strategy, including voice interaction, based on the third data.

[0098] In this embodiment, the first state is characterized by the driver experiencing emotional fluctuations. By combining the driver's facial information or the vehicle's road condition information, the emotion regulation function of the artificial intelligence system is triggered. The system interacts with the driver via voice according to the magnitude of the emotional fluctuations. When the emotions fluctuate significantly, music is played or voice chat is conducted to help the driver soothe their emotions and reduce the accident rate. When the emotions fluctuate slightly, music is played to alleviate the driver's emotions.

[0099] Step 624: If the driving state is the second state, determine the interactive items in the interaction strategy, including voice reminders, based on the third data.

[0100] In this embodiment, the second state is characterized by the driver being fatigued. Combining the driver's facial information and the vehicle's road condition information, the voice prompt function of the artificial intelligence system is triggered. For example, the voice broadcast content is: Fatigue detected. There is a service area 15 kilometers ahead. We have added you to the navigation waypoint.

[0101] Step 634: If the driving state is the third state, determine the interactive items in the interaction strategy, including voice warnings, based on the third data.

[0102] In this embodiment, the third state is characterized by the driver experiencing a health problem, directly triggering the voice warning function of the artificial intelligence system. This function combines the driver's facial information to determine if the driver has fainted, or guides the vehicle to stop based on road conditions. For example, the voice prompt might ask: "Are you feeling unwell? Do you need help?" If there is no positive response or the situation is detected to be worsening, the vehicle will automatically activate its hazard lights, safely slow down, and pull over to the emergency lane, while simultaneously calling for emergency assistance.

[0103] Step 605: Execute the interaction strategy according to the driving status to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminder or voice warning to the driver.

[0104] Step 606: Control the vehicle based on control commands.

[0105] The specific steps of steps 601 to 602 and steps 605 to 606 can be found in steps 301 to 304, and will not be repeated here.

[0106] In this embodiment of the application, by triggering different interactive events corresponding to different emergencies during driving, it is possible to avoid the failure to detect the driver's emotional fluctuations in time, which would cause the driver to be in a state of high pressure. It is also possible to avoid the failure to detect driver fatigue or health problems in time, which could lead to accidents.

[0107] To achieve the functions of the above embodiments, the vehicle control method includes hardware structures and / or software modules corresponding to each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0108] exist Figures 2 to 6 Based on the vehicle control method shown, the embodiments of this application also provide a vehicle control device for further explanation, such as... Figure 7 The schematic diagram of the vehicle control device shown includes: an acquisition module 710, a first processing module 720, a second processing module 730, and a control module 740.

[0109] The acquisition module 710 is used to acquire multiple first data points of the driver during vehicle operation via built-in sensors; the first data points include the driver's hand grip strength, temperature, humidity, and the driver's heart rate and blood pressure; wherein, the acquisition module 710 may include, for example, Figure 1 The first module shown.

[0110] The first processing module 720 is used to acquire the driver's driving state based on multiple first data acquired by built-in sensors; the driving state is used to indicate the degree of change between different first data along the time sequence of the multiple first data; wherein, the first processing module 720 may include, for example, Figure 1 The first module shown.

[0111] The second processing module 730 is used to execute an interaction strategy based on the driving state to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminders, or voice warnings to the driver; wherein, the second processing module 730 may include, for example, Figure 1 The first module shown.

[0112] Control module 740 is used to control the vehicle based on control commands; wherein, control module 740 may include, for example, Figure 1 The second module shown includes the vehicle's central control display screen, air conditioning system, etc.

[0113] In some embodiments, the acquisition module 710 includes: acquiring multiple second data of the driver via built-in sensors during vehicle pausing; the second data includes the driver's hand grip strength, temperature, humidity, and the driver's heart rate and blood pressure; and determining the driver's reference data based on the multiple second data acquired by the built-in sensors.

[0114] In some embodiments, the acquisition module 710 further includes: if the steering wheel control mode is single-sided control, determining the data acquired from the built-in sensor of single-sided control as the first data; if the steering wheel control mode is two-sided control, determining the data with the higher value from the two sets of data acquired from the built-in sensor of dual-sided control as the first data.

[0115] In some embodiments, the first processing module 720 includes: determining the rate of change of each data in the first data according to the time sequence of a plurality of first data; determining the change value of each data in the first data according to the first data and reference data at the same time; determining a driving state according to the rate of change and the change value; the driving state is a first state, a second state or a third state; different driving states are used to determine different interaction strategies.

[0116] In other embodiments, the second processing module 730 includes: acquiring third data of the driver during vehicle operation; the third data is used to indicate: facial information of the driver acquired from the driver monitoring system and road condition information of the vehicle during operation acquired from the navigation system; and determining interactive items in the interaction strategy based on different driving states according to the third data.

[0117] In some embodiments, the second processing module 730 further includes: if the driving state is a first state, determining, based on third data, that the interaction items in the interaction strategy include voice interaction; if the driving state is a second state, determining, based on third data, that the interaction items in the interaction strategy include voice reminders; and if the driving state is a third state, determining, based on third data, that the interaction items in the interaction strategy include voice warnings.

[0118] According to one aspect of the embodiments of this application, Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 800 includes a processor 810 and one or more memories 820. The one or more memories 820 are used to store program instructions executed by the processor 810. When the processor 810 executes the program instructions, it implements the above-described interface processing method.

[0119] Furthermore, the processor 810 may include one or more processing cores. The processor 810 runs or executes instructions, programs, code sets, or instruction sets stored in the memory 820, and retrieves data stored in the memory 820. Optionally, the processor 810 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 810 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented using a separate communication chip.

[0120] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the computer device described in the above embodiments; or it may exist independently and not assembled into the computer device. The computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0121] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0122] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0124] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0125] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A vehicle control method, characterized in that, A driving system applied to a vehicle, the driving system including a steering wheel, with built-in sensors respectively disposed on the left and right sides of the steering wheel, the method comprising: During the operation of the vehicle, several first data points of the driver are acquired through the built-in sensors; the first data points include the driver's grip strength, temperature, humidity, heart rate, and blood pressure. The driver's driving state is obtained based on multiple first data acquired by the built-in sensors; the driving state is used to indicate the degree of change between different first data along the time sequence of the multiple first data. The interaction strategy is executed according to the driving state to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminder or voice warning to the driver. The vehicle is controlled based on the control commands.

2. The method according to claim 1, characterized in that, Before acquiring multiple first data points of the driver via the built-in sensors during the vehicle's operation, the method further includes: During the vehicle's stop, the driver's second data is acquired via the built-in sensors; the second data includes the driver's hand grip strength, temperature, humidity, heart rate, and blood pressure. The driver's reference data is determined based on multiple second data acquired by the built-in sensors.

3. The method according to claim 2, characterized in that, The method for obtaining the driver's driving state based on multiple first data acquired by the built-in sensors includes: Based on the temporal order of the plurality of first data, determine the rate of change of each data point in the first data; Based on the first data and the reference data at the same time, determine the change value of each data in the first data; The driving state is determined based on the rate of change and the value of change; the driving state is a first state, a second state, or a third state; different driving states are used to determine different interaction strategies.

4. The method according to claim 3, characterized in that, The driving system also includes a driver monitoring system and a navigation system. The step of executing an interaction strategy based on the driving state to obtain control commands includes: During the vehicle's operation, third data of the driver is acquired; the third data is used to indicate: the driver's facial information acquired from the driver monitoring system, and the road condition information of the vehicle during its operation acquired from the navigation system. Based on the different driving states, the interaction items in the interaction strategy are determined according to the third data.

5. The method according to claim 4, characterized in that, The method for determining interaction items in the interaction strategy based on different driving states and according to the third data includes: If the driving state is the first state, the interaction items in the interaction strategy are determined to include voice interaction based on the third data; If the driving state is the second state, the interactive items in the interaction strategy are determined to include voice reminders based on the third data; If the driving state is the third state, the interactive items in the interaction strategy are determined to include voice warnings based on the third data.

6. The method according to claim 1, characterized in that, The steering wheel can be controlled by one side or both sides. During vehicle operation, the method involves acquiring multiple first data points of the driver via the built-in sensors. If the steering wheel is controlled by a single-sided control, the data obtained from the built-in sensor of the single-sided control is determined to be the first data; If the steering wheel is controlled by both hands, the first data is determined to be the one with the higher value from the two sets of data obtained from the built-in sensors of the dual-side control.

7. A vehicle control device, characterized in that, A driving system for a vehicle, the driving system including a steering wheel, with built-in sensors respectively disposed on the left and right sides of the steering wheel, the device including: The acquisition module is used to acquire multiple first data of the driver through the built-in sensors during the driving of the vehicle; the first data includes the driver's grip strength, temperature, humidity, heart rate, and blood pressure. The first processing module is used to obtain the driver's driving state based on multiple first data acquired by the built-in sensor; the driving state is used to indicate the degree of change between different first data along the time sequence of the multiple first data. The second processing module is used to execute an interaction strategy based on the driving state to obtain control commands; the interaction strategy is used to instruct the vehicle to initiate voice interaction, voice reminder or voice warning to the driver. A control module is used to control the vehicle based on the control commands.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing instructions which are executed by the processor to implement the vehicle control method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores instructions that are executed by a processor of a computer device to implement the vehicle control method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the vehicle control method as described in any one of claims 1 to 6.

Images