Cockpit scene recommendation method, system, device and storage medium

Abstract

The present invention discloses a cockpit scene recommendation method, system, device and storage medium, the method comprising: when a scene trigger signal is monitored, obtaining a recommended scene matching the scene trigger signal; controlling corresponding vehicle-mounted components of the vehicle cockpit to perform corresponding actions according to vehicle control instructions specified by the recommended scene; obtaining a driver's facial image in real time, and identifying the driver's emotional state based on the facial image; obtaining the driver's grip on the steering wheel in real time; and performing execution interruption arbitration on the recommended scene based on the driver's emotional state and the grip on the steering wheel; the present invention determines whether to interrupt the execution of the current recommended scene by detecting the driver's emotional state and the grip on the steering wheel in real time during the driving process of executing the recommended scene, so as to avoid triggering the execution of the recommended scene based on the trigger condition at an inappropriate opportunity, which may cause driving disturbance to the car owner, thereby improving the driving experience.

Description

Technical Field

[0001] The present invention relates to the field of vehicle intelligent cockpit control technology, and in particular to a cockpit scene recommendation method, system, device and storage medium. Background Art

[0002] With the development of smart cars, users' demand for vehicles has changed from a means of transportation to a demand for a full-life scenario experience during the car-using process. Therefore, providing users with intelligent and personalized in-vehicle scenes is a key factor in triggering users' pleasant driving experience.

[0003] In the current field of smart car smart cockpit products, the method of in-vehicle scene control and recommendation mainly includes: when a new in-vehicle scene meets the preset trigger conditions, calculating the scene score of the new in-vehicle scene; judging whether the scene score is greater than the scene score of the current in-vehicle scene, if so, controlling the execution of the new in-vehicle scene, if not, adding the new in-vehicle scene to the pre-stored in-vehicle scene list for queue execution.

[0004] From the perspective of the user experience of in-vehicle scenarios, the trigger conditions for in-vehicle recommended scenarios are currently biased towards the configurable trigger conditions set by the product, which are a combination of conditions set by the system. However, since the trigger condition parameters have been set in the car system before, before and during driving, if the recommended in-vehicle scenario is triggered, it will cause driving interruptions to the car owner. Moreover, some in-vehicle scenario recommendation solutions on the market do not support interruption. For some in-vehicle scenario recommendation solutions, car owners need to interrupt the in-vehicle scenario process through voice, car HMI interface interaction, etc., which will cause secondary driving interruptions to the car owner. Summary of the Invention

[0005] Embodiments of the present invention provide a cockpit scene recommendation method, system, device, and storage medium, which can avoid driving interruptions to car owners caused by triggering the execution of recommended scenes based on trigger conditions at inappropriate opportunities, thereby improving the driving experience.

[0006] In a first aspect, an embodiment of the present invention provides a cockpit scene recommendation method, comprising:

[0007] When a scene trigger signal is detected, a recommended scene matching the scene trigger signal is obtained;

[0008] Controlling corresponding on-board components of the vehicle cabin to perform corresponding actions according to the vehicle control instructions specified by the recommended scenario;

[0009] Acquire a driver's facial image in real time, and identify the driver's emotional state based on the facial image;

[0010] obtaining the driver's grip on the steering wheel in real time;

[0011] The recommended scenario is subjected to execution interrupt arbitration according to the driver's emotional state and grip on the steering wheel.

[0012] As an improvement to the above solution, the execution interrupt arbitration of the recommended scenario based on the driver's emotional state and grip on the steering wheel includes:

[0013] detecting the driver's driving state according to the driver's emotional state and grip strength on the steering wheel;

[0014] When the driving state of the driver is abnormal, it is determined to interrupt the execution of the recommended scenario, and corresponding vehicle-mounted components in the vehicle cabin are controlled to interrupt the execution of corresponding actions.

[0015] As an improvement to the above solution, detecting the driver's driving state based on the driver's emotional state and grip on the steering wheel includes:

[0016] Determining whether a preset negative emotion constraint condition is satisfied based on the driver's emotional state;

[0017] determining whether a preset grip force constraint condition is satisfied based on the driver's grip force on the steering wheel;

[0018] When the negative emotion constraint condition and the grip strength constraint condition are satisfied, it is determined that the driver's driving state is abnormal.

[0019] As an improvement to the above solution, the negative emotion constraint condition is: the driver's emotional state is at least one of indifference, boredom, disgust, panic, sadness, fear, anger, pain, anxiety, tension, and frustration.

[0020] As an improvement to the above solution, the grip force constraint condition is: the driver's grip force on the steering wheel is greater than a set grip force pressure threshold.

[0021] As an improvement to the above solution, the cockpit scene recommendation method further includes:

[0022] obtaining the driver's grip on the steering wheel detected within a set time period;

[0023] Calculating an average grip force value based on the driver's grip force on the steering wheel detected within a set time period;

[0024] The grip force pressure threshold is updated according to the grip force average value.

[0025] In a second aspect, an embodiment of the present invention provides a cockpit scene recommendation system, comprising: a cloud server and a vehicle computer;

[0026] The vehicle computer is used to perform the following steps:

[0027] When a scene trigger signal is detected, a recommended scene matching the scene trigger signal is obtained;

[0028] Controlling corresponding on-board components of the vehicle cabin to perform corresponding actions according to the vehicle control instructions specified by the recommended scenario;

[0029] The cloud controller is used to perform the following steps:

[0030] Acquire a driver's facial image in real time, and identify the driver's emotional state based on the facial image;

[0031] obtaining the driver's grip on the steering wheel in real time;

[0032] The recommended scenario is subjected to execution interrupt arbitration according to the driver's emotional state and grip on the steering wheel.

[0033] As an improvement to the above solution, the cloud server includes:

[0034] a driving state detection module, configured to detect the driver's driving state based on the driver's emotional state and grip on the steering wheel;

[0035] The scenario interruption control module is used to determine to interrupt the execution of the recommended scenario when the driver's driving state is abnormal, and control the corresponding vehicle-mounted components in the vehicle cabin to interrupt the execution of the corresponding action.

[0036] In a third aspect, an embodiment of the present invention provides a cabin scene recommendation device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the cabin scene recommendation method as described in any one of the first aspects.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the cockpit scene recommendation method as described in any one of the first aspects.

[0038] Compared to the prior art, the beneficial effects of the embodiments of the present invention are as follows: when a scene trigger signal is monitored, a recommended scene matching the scene trigger signal is obtained; according to the vehicle control instructions specified by the recommended scene, corresponding vehicle components in the vehicle cabin are controlled to perform corresponding actions; a facial image of the driver is obtained in real time, and the driver's emotional state is identified based on the facial image; the driver's grip on the steering wheel is obtained in real time; and execution interruption arbitration of the recommended scene is performed based on the driver's emotional state and grip on the steering wheel. The embodiments of the present invention determine whether to interrupt execution of the current recommended scene by detecting the driver's emotional state and grip on the steering wheel in real time during the driving process of executing the recommended scene, so as to avoid driving interruptions caused to the driver by triggering the execution of the recommended scene based on the trigger conditions at inappropriate opportunities, thereby improving the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings used in the implementation methods. Obviously, the drawings described below are only some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a flowchart of a cockpit scene recommendation method provided by an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of a cockpit scene recommendation system provided by an embodiment of the present invention;

[0042] Figure 3 2 is a schematic diagram of a cockpit scene recommendation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] Example 1

[0045] See Figure 1 , which is a flow chart of a cockpit scene recommendation method provided by an embodiment of the present invention. The cockpit scene recommendation method includes:

[0046] S1: When a scene trigger signal is monitored, obtaining a recommended scene matching the scene trigger signal;

[0047] For example, the vehicle computer pre-sets trigger conditions for different recommendation scenarios, for example, the trigger conditions include but are not limited to:

[0048] (1) Based on the date and time, the vehicle is started for the first time;

[0049] (2) Navigate to the company or home based on the navigation instructions;

[0050] (3) Based on special voice instructions;

[0051] Vehicle control includes but is not limited to:

[0052] (1) The window is opened or closed or adjusted by a certain percentage;

[0053] (2) The sunroof is opened or closed or adjusted by a certain percentage;

[0054] (3) Ambient light turns on or off or adjusts the percentage;

[0055] Vehicle computer signal source control includes but is not limited to:

[0056] (1) Open the music source and recommend users to listen to songs by xx singer in the music app or today's recommended playlist;

[0057] (2) Open the radio source and recommend the user to listen to the xx radio program of the radio source;

[0058] Among them, the recommended scene can be obtained by freely combining the above-mentioned vehicle control and vehicle computer signal source control. For example, set April 14, 2023, at 8 o'clock, when the vehicle computer is started for the first time, the trigger recommended scene is: the car window is open, the ambient light is turned on, the music source is turned on, and today's recommended playlist is recommended. It should be noted that the trigger conditions stored in the vehicle computer and the matching recommended scenes can be the ones originally set by the vehicle computer, or they can be set by the owner himself on the vehicle computer or mobile phone APP. For example, different trigger conditions and their matching recommended scenes can be recorded in the form of a mapping table. When any trigger condition is met, a corresponding scene trigger signal is generated. When the vehicle computer monitors the scene trigger signal, the corresponding matching recommended scene is obtained.

[0059] S2: controlling corresponding vehicle components in the vehicle cabin to perform corresponding actions according to the vehicle control instructions specified by the recommended scenario;

[0060] After obtaining the recommended scenario that needs to be executed, based on the vehicle control instructions specified by the recommended scenario, the vehicle computer controls the corresponding vehicle components in the vehicle cabin to perform corresponding actions, such as controlling the windows to open, the ambient lights to turn on, the music source to open, and recommend today's recommended playlist.

[0061] S3: Acquire a facial image of the driver in real time, and identify the driver's emotional state based on the facial image;

[0062] S4: obtaining the driver's grip on the steering wheel in real time;

[0063] S5: performing execution interrupt arbitration on the recommended scenario according to the driver's emotional state and grip on the steering wheel.

[0064] The vehicle is equipped with an in-car camera OMS / DMS for real-time filming of the driver, and the steering wheel is provided with a pressure sensor for real-time detection of the driver's grip on the steering wheel. During driving, the in-car camera will monitor the driver's facial image in real time and identify the driver's emotional state. The identified emotional state of the driver is uploaded to the cloud server, and the pressure sensor will also upload the real-time detected driver's grip on the steering wheel to the cloud server. Based on the uploaded driver's emotional state and grip on the steering wheel, the cloud server will perform execution interruption arbitration on the recommended scenario and determine whether to interrupt the execution of the current recommended scenario to avoid triggering the execution of the recommended scenario based on the trigger conditions at inappropriate opportunities, which will cause driving interruptions to the car owner, thereby improving the driving experience.

[0065] In other embodiments, under the premise that the computing power of the vehicle computer meets the requirements, the vehicle computer can also be used to execute the cabin scene recommendation method of the above steps S1-S5 without uploading to the cloud server for recommended scene execution interrupt arbitration.

[0066] In an optional embodiment, the interruption arbitration of the recommended scenario according to the driver's emotional state and steering wheel grip includes:

[0067] detecting the driver's driving state according to the driver's emotional state and grip strength on the steering wheel;

[0068] Furthermore, the detecting the driving state of the driver according to the driver's emotional state and grip strength of the steering wheel includes:

[0069] Determining whether a preset negative emotion constraint condition is satisfied based on the driver's emotional state;

[0070] The negative emotion constraint condition is that the driver's emotional state is at least one of indifference, boredom, disgust, panic, sadness, fear, anger, pain, anxiety, tension, and frustration.

[0071] determining whether a preset grip force constraint condition is satisfied based on the driver's grip force on the steering wheel;

[0072] The grip force constraint condition is that the driver's grip force on the steering wheel is greater than a set grip force pressure threshold.

[0073] When the negative emotion constraint condition and the grip strength constraint condition are satisfied, it is determined that the driver's driving state is abnormal.

[0074] When the driving state of the driver is abnormal, it is determined to interrupt the execution of the recommended scenario, and corresponding vehicle-mounted components in the vehicle cabin are controlled to interrupt the execution of corresponding actions.

[0075] In an embodiment of the present invention, when it is detected that the driver is in a negative emotion and the driver's current grip on the steering wheel is greater than the grip pressure threshold under normal driving conditions, the cloud service sends a scene interruption command to the vehicle computer. After receiving the scene interruption command, the vehicle computer will no longer execute the recommended scene matched based on the trigger conditions, and will no longer complete the execution of a series of collaborative instructions of the vehicle control & vehicle computer signal source, thereby interrupting the execution of the recommended scene to avoid excessively disturbing the driver during driving when it is sensed that the driver is in a negative emotion.

[0076] In an optional embodiment, the cockpit scene recommendation method further includes:

[0077] obtaining the driver's grip on the steering wheel detected within a set time period;

[0078] Calculating an average grip force value based on the driver's grip force on the steering wheel detected within a set time period;

[0079] The grip force pressure threshold is updated according to the grip force average value.

[0080] In an embodiment of the invention, the cloud server periodically calculates the average grip force values ​​uploaded by the steering wheel pressure sensor within a set time period before the current time, using this value as the grip pressure threshold for the corresponding driver during their daily driving state. It should be noted that the cloud server identifies the driver's identity information based on a facial image uploaded by the in-car camera, and constructs and stores a user profile for the driver based on the driver's facial image, emotional state, and grip pressure threshold during their daily driving state. This user profile can be updated in real time based on the driver's emotional state and steering wheel grip force monitored in real time. At this point, the cloud server can directly obtain the grip pressure threshold for daily driving conditions from the latest user profile and compare it with the driver's current steering wheel grip force.

[0081] Compared with the existing technology, the embodiments of the present invention determine whether to interrupt the execution of the current recommended scenario by detecting the driver's emotional state and grip on the steering wheel in real time during the driving process of executing the recommended scenario, so as to avoid triggering the execution of the recommended scenario based on trigger conditions at inappropriate opportunities, which may cause driving interruptions to the car owner, thereby improving the driving experience.

[0082] Example 2

[0083] See also Figure 2 ,The embodiment of the present invention provides a cockpit scene recommendation system, including: a cloud service ,1, a vehicle computer 2;

[0084] The vehicle computer 2 is used to perform the following steps:

[0085] When a scene trigger signal is detected, a recommended scene matching the scene trigger signal is obtained;

[0086] Controlling corresponding on-board components of the vehicle cabin to perform corresponding actions according to the vehicle control instructions specified by the recommended scenario;

[0087] The cloud controller 1 is configured to perform the following steps:

[0088] Acquire a driver's facial image in real time, and identify the driver's emotional state based on the facial image;

[0089] obtaining the driver's grip on the steering wheel in real time;

[0090] The recommended scenario is subjected to execution interrupt arbitration according to the driver's emotional state and grip on the steering wheel.

[0091] In an optional embodiment, the cloud server includes:

[0092] a driving state detection module, configured to detect the driver's driving state based on the driver's emotional state and grip on the steering wheel;

[0093] The scenario interruption control module is used to determine to interrupt the execution of the recommended scenario when the driver's driving state is abnormal, and control the corresponding vehicle-mounted components in the vehicle cabin to interrupt the execution of the corresponding action.

[0094] In an optional embodiment, the driving state detection module includes:

[0095] An emotion judgment unit, configured to judge whether a preset negative emotion constraint condition is satisfied based on the driver's emotional state;

[0096] A grip force determination unit, configured to determine whether a preset grip force constraint condition is satisfied based on the driver's grip force on the steering wheel;

[0097] A driving state judgment unit is used to judge that the driver's driving state is abnormal when the negative emotion constraint condition and the grip strength constraint condition are met.

[0098] In an optional embodiment, the negative emotion constraint condition is: the driver's emotional state is at least one of indifference, boredom, disgust, panic, sadness, fear, anger, pain, anxiety, tension, and frustration.

[0099] In an optional embodiment, the grip force constraint condition is: the driver's grip force on the steering wheel is greater than a set grip force pressure threshold.

[0100] In an optional embodiment, the cloud service further includes: a grip pressure threshold calculation module, configured to:

[0101] obtaining the driver's grip on the steering wheel detected within a set time period;

[0102] Calculating an average grip force value based on the driver's grip force on the steering wheel detected within a set time period;

[0103] The grip force pressure threshold is updated according to the grip force average value.

[0104] It should be noted that the technical principles and technical effects achieved by the cockpit scene recommendation system described in this embodiment of the present invention are the same as those in Example 1 and will not be repeated here.

[0105] Example 3

[0106] See also Figure 3 , is a schematic diagram of a cockpit scene recommendation device according to an embodiment of the present invention. The cockpit scene recommendation device according to this embodiment includes: a processor 100, a memory 200, and a computer program stored in the memory 200 and executable on the processor 100, such as a cockpit scene recommendation program. When the processor 100 executes the computer program, the steps of each of the above cockpit scene recommendation method embodiments are implemented, such as Figure 1 Steps S1-S5 are shown.

[0107] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the cockpit scene recommendation device.

[0108] The cockpit scene recommendation device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of the cockpit scene recommendation device and does not limit the scope of the device. The device may include more or fewer components than shown, or a combination of certain components or different components. For example, the device may also include input and output devices, network access devices, buses, and the like.

[0109] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor serves as the control center of the cockpit scene recommendation device, connecting various parts of the entire cockpit scene recommendation device using various interfaces and lines.

[0110] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the cockpit scene recommendation device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0111] Wherein, if the module / unit integrated in the cockpit scene recommendation device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0112] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0113] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, many improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A cockpit scene recommendation method, characterized in that: include: When a scene trigger signal is detected, a recommended scene matching the scene trigger signal is obtained; Controlling corresponding on-board components of the vehicle cabin to perform corresponding actions according to the vehicle control instructions specified by the recommended scenario; Acquire a driver's facial image in real time, and identify the driver's emotional state based on the facial image; obtaining the driver's grip on the steering wheel in real time; The recommended scenario is subjected to execution interrupt arbitration according to the driver's emotional state and grip on the steering wheel.

2. The cockpit scene recommendation method according to claim 1, wherein: The execution interruption arbitration of the recommended scenario according to the driver's emotional state and the driver's grip on the steering wheel includes: detecting the driver's driving state according to the driver's emotional state and grip strength on the steering wheel; When the driving state of the driver is abnormal, it is determined to interrupt the execution of the recommended scenario, and corresponding vehicle-mounted components in the vehicle cabin are controlled to interrupt the execution of corresponding actions.

3. The cockpit scene recommendation method according to claim 2, wherein: The detecting the driver's driving state according to the driver's emotional state and the grip strength of the steering wheel includes: Determining whether a preset negative emotion constraint condition is satisfied based on the driver's emotional state; determining whether a preset grip force constraint condition is satisfied based on the driver's grip force on the steering wheel; When the negative emotion constraint condition and the grip strength constraint condition are satisfied, it is determined that the driver's driving state is abnormal.

4. The cockpit scene recommendation method according to claim 3, wherein: The negative emotion constraint condition is that the driver's emotional state is at least one of indifference, boredom, disgust, panic, sadness, fear, anger, pain, anxiety, tension, and frustration.

5. The cockpit scene recommendation method according to claim 3, wherein: The grip force constraint condition is that the driver's grip force on the steering wheel is greater than a set grip force pressure threshold.

6. The cockpit scene recommendation method according to claim 5, characterized in that: Also includes: obtaining the driver's grip on the steering wheel detected within a set time period; Calculating an average grip force value based on the driver's grip force on the steering wheel detected within a set time period; The grip force pressure threshold is updated according to the grip force average value.

7. A cockpit scene recommendation system, characterized in that: include: Cloud server, car computer; The vehicle computer is used to perform the following steps: When a scene trigger signal is detected, a recommended scene matching the scene trigger signal is obtained; According to the vehicle control instructions specified by the recommended scenario, control the corresponding vehicle components in the vehicle cabin to perform corresponding actions; The cloud server is used to perform the following steps: Acquire a driver's facial image in real time, and identify the driver's emotional state based on the facial image; obtaining the driver's grip on the steering wheel in real time; The recommended scenario is subjected to execution interrupt arbitration according to the driver's emotional state and grip on the steering wheel.

8. The cockpit scene recommendation system according to claim 7, characterized in that: The cloud server includes: a driving state detection module, configured to detect the driver's driving state based on the driver's emotional state and grip on the steering wheel; The scenario interruption control module is used to determine to interrupt the execution of the recommended scenario when the driver's driving state is abnormal, and control the corresponding vehicle-mounted components in the vehicle cabin to interrupt the execution of the corresponding action.

9. A cockpit scene recommendation device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the cockpit scene recommendation method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the cockpit scene recommendation method according to any one of claims 1 to 6.

Images