Automobile cabin control method, computer device and storage medium

Abstract

The invention discloses an automobile cabin control method and device and a storage medium. The automobile cabin control method comprises the steps of detecting first emotion information of a driver, detecting second emotion information of other detection objects, determining an active interaction strategy for the driver according to the first emotion information and the second emotion information, executing the active interaction strategy and the like. According to the emotion interaction result between the driver and each traffic participant (other detection objects), active interaction can be carried out on the driver, so that the driver is guided to drive carefully, possible driving risks are effectively coped, and traffic safety is guaranteed; by processing the first emotion information corresponding to the driver and the second emotion information corresponding to the other detection objects, simulation can be carried out close to the real interaction condition of the driving process of the driver, so that good active interaction experience is provided for the driver. The invention is widely applied to the technical field of automobiles.

Description

Technical Field

[0001] The present invention relates to the technical field of automobiles, and in particular to an automobile cockpit control method, a computer device, and a storage medium. Background Art

[0002] One development direction of automobiles is cockpit intelligentization, so as to provide a better driving experience for the people in the vehicle, especially the driver. The current automobile cockpit technology focuses on the implementation of basic functions such as navigation, atmosphere, and multimedia entertainment, and still needs to be improved.

[0003] With the improvement of the computing power of in-vehicle computers, some technical attempts rely on high computing power to achieve cockpit intelligentization. For example, some technical attempts try to sense the driver's emotions and then perform interactions with the driver in order to relieve bad emotions and fatigue states and achieve the effect of ensuring traffic safety. However, the current related technologies for automobile cockpit interaction consider single factors, and the interaction model is quite different from the real situation of the driver's driving process, so the interaction experience is poor, and it is difficult to improve the driver's bad emotions and fatigue states. Summary of the Invention

[0004] Aiming at the technical problems existing in the current related technologies for automobile cockpit interaction, such as the large difference between the interaction model and the real situation of the driver's driving process, poor interaction experience, and difficulty in relieving the driver's bad emotions and fatigue states, the purpose of the present invention is to provide an automobile cockpit control method, device, and storage medium.

[0005] On the one hand, an embodiment of the present invention includes an automobile cockpit control method, and the automobile cockpit control method includes the following steps:

[0006] Detect the first emotion information of the driver;

[0007] Detect the second emotion information of other detection objects; the other detection objects are detection objects other than the driver;

[0008] Determine an active interaction strategy for the driver according to the first emotion information and the second emotion information;

[0009] Execute the active interaction strategy.

[0010] Further, the detecting the first emotion information of the driver includes:

[0011] Perform multimodal detection on the driver to obtain first multimodal data;

[0012] Perform emotion recognition on the first multimodal data to obtain the first emotion information.

[0013] Further, the detection of the second emotional information of other detection objects includes:

[0014] Determine a list of objects to be detected; the list of objects to be detected includes at least one of the other detection objects;

[0015] For any one of the other detection objects in the list of objects to be detected, perform multimodal detection on the other detection object to obtain second multimodal data;

[0016] Perform emotion recognition on the second multimodal data to obtain the second emotional information.

[0017] Further, the determination of the list of objects to be detected includes:

[0018] Obtain a candidate list; the candidate list includes multiple other detection objects, and each of the other detection objects has a corresponding priority. For any one of the other detection objects, the priority of the other detection object is negatively correlated with the interaction distance between the other detection object and the driver;

[0019] Detect the current road condition information;

[0020] Determine the number of objects to be detected according to the current road condition information;

[0021] According to each priority, preferentially select the number of other detection objects equal to the number of objects to be detected from the candidate list;

[0022] Establish the list of objects to be detected according to the selected other detection objects.

[0023] Further, the obtaining of the candidate list includes:

[0024] Detect the interior of the vehicle cockpit. When a passenger is detected, determine the passenger as the other detection object;

[0025] Detect the exterior of the vehicle cockpit. When a person outside the vehicle is detected, determine the person outside the vehicle as the other detection object;

[0026] Detect the exterior of the vehicle cockpit. When a neighboring vehicle is detected, determine the neighboring vehicle as the other detection object;

[0027] Perform a self-check on the sensor system of the vehicle. When the sensor system is detected to be normal, determine the vehicle itself as the other detection object;

[0028] Establish the candidate list according to all the determined other detection objects.

[0029] Further, determining an active interaction strategy for the driver according to the first emotion information and the second emotion information includes:

[0030] Configuring a first dialogue model according to the first emotion information;

[0031] Configuring a second dialogue model according to the second emotion information;

[0032] Performing a dialogue between the first dialogue model and the second dialogue model to obtain virtual dialogue information;

[0033] Determining the active interaction strategy according to the virtual dialogue information.

[0034] Further, performing the dialogue between the first dialogue model and the second dialogue model includes:

[0035] When there is one second dialogue model, using the first dialogue model and the second dialogue model as the dialogue parties in the same dialogue process to perform the dialogue;

[0036] When there are multiple second dialogue models, detecting the interaction distance distribution of each other detection object corresponding to each second dialogue model, grouping each second dialogue model according to the interval where the corresponding interaction distance is located to obtain at least one dialogue group, where one dialogue group includes at least one second dialogue model with the corresponding interaction distance in the same interval;

[0037] Establishing at least one dialogue process, the number of dialogue processes being the same as the number of dialogue groups;

[0038] For any one of the dialogue processes, using the first dialogue model and a corresponding dialogue group as the dialogue parties to perform the dialogue.

[0039] Further, determining the active interaction strategy according to the virtual dialogue information includes:

[0040] Generating driving advice information according to the virtual dialogue information.

[0041] On the other hand, an embodiment of the present invention further includes a computer device, including a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the vehicle cockpit control method in the embodiment.

[0042] On the other hand, an embodiment of the present invention further includes a computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the vehicle cockpit control method in the embodiment when executed by the processor.

[0043] The beneficial effects of the present invention are as follows: The vehicle cockpit control method in the embodiment can simulate the emotional interaction and evolution results between the driver and each traffic participant (other detection objects), and actively interact with the driver according to the emotional interaction and evolution results, so as to guide the driver to drive carefully, effectively cope with possible driving risks, and ensure traffic safety; By processing the first emotional information corresponding to the driver and the second emotional information corresponding to other detection objects, it can simulate the real interaction situation during the driver's driving process, thus providing a good active interaction experience for the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of a vehicle system to which the vehicle cockpit control method can be applied in the embodiment;

[0045] Figure 2 It is a schematic diagram of the steps of the vehicle cockpit control method in the embodiment;

[0046] Figure 3 It is a schematic diagram of the principle of emotion information recognition in the embodiment;

[0047] Figure 4 It is a schematic diagram of the principle of obtaining the list to be detected in the embodiment;

[0048] Figure 5 It is a schematic diagram of the principle of obtaining virtual dialogue information in the embodiment;

[0049] Figure 6 It is a schematic diagram of the user emotion knowledge base in the embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In this embodiment, the vehicle cockpit control method can be applied to Figure 1 the vehicle system shown in Figure 1 . Referring to

[0051] In this embodiment, each sensor can be a single sensor component or a combination of multiple sensor components. For example, Sensor 1 (driver behavior sensor) can be a combination of sensor components such as a camera, a pressure sensor, a microphone, a blood pressure sensor, and a carbon dioxide concentration sensor installed at the driver's position in the vehicle, so as to be able to detect data such as the driver's face and limb movement images, speaking voice, sitting posture, blood pressure, and breathing intensity; Sensor 3 (adjacent vehicle behavior sensor) can be a combination of a distance sensor, a camera, and a microphone installed outside the vehicle, so as to be able to detect data such as the relative position between the adjacent vehicle and the vehicle itself, as well as the movement images and sounds of the adjacent vehicle; Sensor 4 (vehicle driving state sensor) can be a combination of an engine speed sensor, an engine oil temperature sensor, a driving speed sensor, a driving acceleration sensor, an inclination sensor, a steering angle sensor, etc., so as to be able to detect data such as engine speed, engine oil temperature, driving speed, driving acceleration, inclination, and steering angle; Sensor 5 (vehicle component state sensor) can be a combination of a component failure sensor, an engine oil quantity sensor, and a brake wear sensor, etc., so as to be able to detect data such as component failure status, engine oil quantity, and brake wear degree; Sensor 6 (environmental parameter sensor) can be a combination of a temperature sensor, a humidity sensor, a lidar, and a satellite navigation module, etc., so as to be able to detect data such as the temperature outside the vehicle, the humidity outside the vehicle, vehicle condition information (specifically including traffic flow, pedestrian flow, obstacle position distribution, etc.), and route planning.

[0052] In this embodiment, the vehicle installed with the Figure 1 system shown and executing the vehicle cockpit control method is referred to as the vehicle itself. The vehicle cockpit control method is executed by an in-vehicle computer. Referring to Figure 2 , the vehicle cockpit control method includes the following steps:

[0053] S1. Detect the first emotion information of the driver;

[0054] S2. Detect the second emotion information of other detection objects;

[0055] S3. Determine an active interaction strategy for the driver according to the first emotion information and the second emotion information;

[0056] S4. Execute the active interaction strategy.

[0057] In this embodiment, when performing step S1, that is, the step of detecting the first emotion information of the driver, the following steps can be specifically executed:

[0058] S101. Perform multimodal detection on the driver to obtain first multimodal data;

[0059] S102. Perform emotion recognition on the first multimodal data to obtain the first emotion information.

[0060] In step S101, the in-vehicle computer can call Sensor 1 (driver behavior sensor) to perform multimodal detection on the driver and obtain first multimodal data. Among them, multimodal detection refers to performing detections in multiple different aspects, and the obtained multimodal data respectively represent the state of the detection object from multiple different aspects. For example, when step S101 is executed, the camera in Sensor 1 (driver behavior sensor) takes pictures of the driver to obtain images containing actions such as the facial movements (expressions), head movements, and limb movements of the driver, the microphone collects the voice of the driver to obtain data such as the speaking voice of the driver, the pressure sensor detects the pressure distribution generated by the driver on the seat, so as to determine the sitting posture of the driver, the blood pressure sensor detects the blood pressure of the driver's hand or buttocks, and can detect the heart rate of the driver, and the carbon dioxide concentration sensor detects the breathing intensity of the driver. These detected data such as images, speaking voice, pressure distribution, blood pressure, heart rate, and breathing intensity constitute the first multimodal data.

[0061] In step S102, the in-vehicle computer can run an emotion recognition algorithm to perform emotion recognition on the first multimodal data, and the obtained first emotion information represents the emotion state or emotion type of the driver. For example, referring to Figure 3 , the in-vehicle computer can run a trained artificial intelligence model to recognize the first multimodal data, so as to obtain the first emotion information. The first emotion information can represent the classification result of the first multimodal data in terms of emotion type, such as excited, happy, angry, normal, low, etc.

[0062] In this embodiment, when step S2, that is, the step of detecting the second emotion information of other detection objects, is executed, the following steps can be specifically executed:

[0063] S201. Determine the list of objects to be detected;

[0064] S202. For any other detection object in the list of objects to be detected, perform multimodal detection on the other detection object to obtain second multimodal data;

[0065] S203. Perform emotion recognition on the second multimodal data to obtain second emotion information.

[0066] In step S201, one or more detection objects other than the driver, that is, other detection objects, can be selected to form the list of objects to be detected.

[0067] In this embodiment, the list of objects to be detected can be formed in the manner shown by Figure 4 .

[0068] Referring to Figure 4, the in-vehicle computer can first determine a candidate list. The candidate list includes all other detection objects that the vehicle can sense. For example, the in-vehicle computer calls the sensor system to detect whether there are occupants inside the vehicle cockpit, whether the working state of the vehicle's sensor system is normal, and whether there are people outside the vehicle and adjacent vehicles outside the vehicle cockpit. The detected occupants, people outside the vehicle, and adjacent vehicles, as well as the normal vehicle sensor system, are respectively determined as an other detection object. In this embodiment, it is assumed that 7 other detection objects such as occupant 1, occupant 2, occupant 3, the vehicle itself, adjacent vehicle 1, person outside the vehicle 1, and adjacent vehicle 2 are detected. As Figure 4 shown, a candidate list is established, and the candidate list includes all these existing other detection objects.

[0069] In this embodiment, there is a priority order among the various other detection objects in the candidate list. In this embodiment, according to the interaction distance between each other detection object and the driver, the priority of each other detection object is determined negatively correlated. Specifically, the interaction distance between an other detection object and the driver refers to the distance faced in the process of the other detection object actively interacting with the driver or objectively existing interaction. In this embodiment, the spatial physical distance between this other detection object and the driver can be used as the interaction distance. In this embodiment, other properties of each other detection object can also be considered, and the interaction distance is adjusted on the basis of the spatial physical distance. For example, even if the spatial physical distance between person outside the vehicle 1 and the driver is smaller (i.e., closer to the driver in space), and the spatial physical distance between adjacent vehicle 1 and the driver is larger (i.e., farther from the driver in space), but since the driver is more likely to notice adjacent vehicle 1, the objectively existing interaction between adjacent vehicle 1 and the driver is stronger. Therefore, the interaction distance between adjacent vehicle 1 and the driver can also be determined to be smaller than the interaction distance between person outside the vehicle 1 and the driver.

[0070] In this embodiment, referring to Figure 4 , the closer the interaction distance between an other detection object and the driver, the higher the priority of this other detection object in the candidate list.

[0071] After determining the content and priority of the candidate list in Figure 4 , the in-vehicle computer can call the sensor 6 (environmental parameter sensor) to detect the current road condition information. The current road condition information represents the traffic flow, pedestrian flow, and congestion degree of the environment where the vehicle is located. The in-vehicle computer can quantify and classify data such as traffic flow, pedestrian flow, and congestion degree, so as to determine the current road condition information as values such as "complex", "medium", and "simple".

[0072] After determining the current road condition information, the in-vehicle computer can positively correlate and determine the number of objects to be detected according to the complexity of the current road condition information. For example, referring toFigure 4 If the complexity level of the current road condition information is "complex" which is relatively high, then the in-vehicle computer can determine the quantity to be detected as a relatively large value 6.

[0073] After determining the quantity to be detected, the in-vehicle computer selects, from the candidate list, other detection objects with the highest priority level and the same quantity as the quantity to be detected, that is, selects 6 other detection objects with the highest priority level, namely Occupant 1, Occupant 2, Occupant 3, the vehicle itself, Vehicle 1 beside, and Pedestrian 1 outside the vehicle, to establish a list of objects to be detected.

[0074] In this embodiment, the principle of performing step S201 is as follows: The candidate list contains all existing and detectable other detection objects. The interaction distance of other detection objects indicates the intensity of the interaction between other detection objects and the driver. For example, the shorter the interaction distance, the stronger the possibility that other detection objects can interact with the driver and the greater the impact of the interaction. Therefore, a higher priority is set, and there is a greater possibility of being selected when establishing the list of objects to be detected, so that the other detection objects in the list of objects to be detected are those other detection objects with the shortest interaction distance; the current road condition information reflects the road conditions of the environment where the vehicle is currently located. In the case of more complex current road condition information, more other detection objects are selected to form the list of objects to be detected, which can process more other detection objects in the subsequent steps S202 - S203 and steps S3 - S4, thereby considering the interaction between more other detection objects and the driver, which is beneficial to dealing with more complex road conditions.

[0075] In step S202, the in-vehicle computer traverses each other detection object in the list of objects to be detected, performs multimodal detection on each other detection object, and obtains the corresponding second multimodal data for each other detection object.

[0076] Specifically, in steps S202 - S203, for Figure 4 other detection objects such as Occupant 1, Occupant 2, and Occupant 3 who are all vehicle occupants in the vehicle itself, the in-vehicle computer can refer to the principle of step S1, use the data obtained from multimodal detection of each occupant as their corresponding second multimodal data, and detect the second emotion information 1 of Occupant 1, the second emotion information 2 of Occupant 2, and the second emotion information 3 of Occupant 3.

[0077] For Figure 4For other detection objects such as the vehicle itself in [the text], the in-vehicle computer can call sensors such as Sensor 4 (the vehicle driving state sensor), Sensor 5 (the vehicle component state sensor), and Sensor 6 (the environmental parameter sensor) to collect data, perform anthropomorphic processing on the data, obtain corresponding somatosensory data, and then perform emotion recognition on the somatosensory data. For example, the in-vehicle computer can call Sensor 4 (the vehicle driving state sensor) to anthropomorphically convert the detected engine speed into breathing intensity, the detected driving speed into running speed, the detected engine oil temperature into body temperature, and the detected data such as driving acceleration, inclination angle, and steering angle into limb movements; the in-vehicle computer can call Sensor 5 (the vehicle component state sensor) to anthropomorphically convert the detected component failure state and brake wear degree into pain degree, and the detected engine oil volume into remaining physical strength; the in-vehicle computer can call Sensor 6 (the environmental parameter sensor) to anthropomorphically convert the detected outside vehicle temperature and outside vehicle humidity into somatosensory temperature and somatosensory humidity, etc. The above anthropomorphic conversion can be carried out by querying data tables and other methods.

[0078] Through anthropomorphic conversion, the in-vehicle computer can convert the data detected for the vehicle itself into data equivalent to that detected for a person, such as somatosensory data like breathing intensity and body temperature. These data are the second multi-modal data detected for the vehicle itself, so that an emotion recognition algorithm with a person as the processing object can be run to recognize the data detected from the vehicle itself and obtain the "emotion information" of the vehicle itself, that is, the second emotion information 4.

[0079] For Figure 4 other detection objects such as the adjacent vehicle 1 and the outside vehicle person 1 in the outside environment of the vehicle itself, the in-vehicle computer can call Sensor 3 (the adjacent vehicle behavior sensor) to detect data such as the distance between the adjacent vehicle / outside vehicle person and the vehicle itself, the driving actions / limb movements / facial expression images of the adjacent vehicle / outside vehicle person, and the driving sounds / talking sounds of the adjacent vehicle / outside vehicle person. These data are also obtained through multi-modal detection like the first multi-modal data, and are the second multi-modal data corresponding to other detection objects such as the adjacent vehicle 1 and the outside vehicle person 1. The in-vehicle computer can run a trained artificial intelligence model to recognize data such as the distance between the adjacent vehicle 1 and the vehicle itself, the driving actions (including specific actions such as speed, acceleration, turning amplitude, headlight usage, and cutting in line) images of the adjacent vehicle 1, and the driving sounds of the adjacent vehicle 1, and obtain the "emotion information" of the adjacent vehicle 1 (or the emotion information of the driver of the adjacent vehicle 1), that is, the second emotion information 5; the in-vehicle computer can run a trained artificial intelligence model to recognize data such as the distance between the outside vehicle person 1 and the vehicle itself, the limb movements / facial expression images of the outside vehicle person 1, and the talking sounds of the outside vehicle person 1, and obtain the emotion information of the outside vehicle person 1, that is, the second emotion information 6.

[0080] After steps S201 - S203 are completed, step S3 is then executed.

[0081] In this embodiment, when executing step S3, that is, determining the active interaction strategy for the driver according to the first emotion information and the second emotion information, the following steps can be specifically executed:

[0082] S301. Configure the first dialogue model according to the first emotion information;

[0083] S302. Configure the second dialogue model according to the second emotion information;

[0084] S303. Execute the dialogue between the first dialogue model and the second dialogue model to obtain virtual dialogue information;

[0085] S304. Determine the active interaction strategy according to the virtual dialogue information.

[0086] The principles of steps S301 - S304 are as Figure 5 shown.

[0087] In steps S301 - S302, the in - vehicle computer can deploy locally (when the computing power is sufficient) or remotely call (when the computing power is insufficient) the large language model (LLM), and use the emotion information to configure the large language model to obtain the corresponding dialogue model.

[0088] Specifically, referring to Figure 5 , use the first emotion information to configure the large language model to obtain the first dialogue model; use the second emotion information 1 to configure the large language model to obtain the second dialogue model 1; use the second emotion information 2 to configure the large language model to obtain the second dialogue model 2; use the second emotion information 3 to configure the large language model to obtain the second dialogue model 3; use the second emotion information 4 to configure the large language model to obtain the second dialogue model 4; use the second emotion information 5 to configure the large language model to obtain the second dialogue model 5; use the second emotion information 6 to configure the large language model to obtain the second dialogue model 6.

[0089] By executing steps S301 - S302, the obtained first dialogue model and each second dialogue model have the ability to conduct a dialogue. For example, taking the first dialogue model as the dialogue object, editing the content to be told to the first dialogue model into a prompt word (token) and inputting it into the first dialogue model, the first dialogue model will understand and generate the corresponding output content for output according to the first emotion information; taking the second dialogue model 1 as the dialogue object, editing the content to be told to the second dialogue model 1 into a prompt word (token) and inputting it into the second dialogue model 1, the second dialogue model 1 will understand and generate the corresponding output content for output according to the first emotion information.

[0090] When performing step S303, if there is only one other detection object, that is, only one second dialogue model, then the first dialogue model and the second dialogue model are used as the dialogue parties in the same dialogue process to perform the dialogue. For example, the output content of the first dialogue model is used as the prompt word input into the second dialogue model to trigger the second dialogue model to generate the corresponding output content, and the output content of the second dialogue model is used as the prompt word input into the first dialogue model to trigger the first dialogue model to generate the corresponding output content... Thus, a one-on-one dialogue between the first dialogue model and the second dialogue model is realized.

[0091] When performing step S303, if there are multiple other detection objects, that is, multiple second dialogue models, then each second dialogue model can be respectively established with a one-on-one dialogue with the first dialogue model, that is, the first dialogue model establishes a one-on-one dialogue with the second dialogue model 1, the first dialogue model establishes a one-on-one dialogue with the second dialogue model 2...

[0092] When performing step S303, if there are multiple second dialogue models, then as Figure 5 shown, the second dialogue models are grouped according to the distribution of the interaction distances of the other detection objects corresponding to each second dialogue model. For example, referring to Figure 5 , the second dialogue model 1, the second dialogue model 2, and the second dialogue model 3 respectively correspond to occupant 1, occupant 2, and occupant 3, and their interaction distances with the driver are all in the "near" interval, then the second dialogue model 1, the second dialogue model 2, and the second dialogue model 3 are divided into dialogue group 1; the second dialogue model 4 corresponds to the vehicle itself, and its interaction distance with the driver is in the "medium" interval, then the second dialogue model 4 is divided into dialogue group 2; the second dialogue model 5 and the second dialogue model 6 respectively correspond to vehicle 1 and the person outside the vehicle 1, and their interaction distances with the driver are all in the "far" interval, then the second dialogue model 5 and the second dialogue model 6 are divided into dialogue group 3.

[0093] After dividing the dialogue groups, referring to Figure 5 , each dialogue group can be respectively established with a multi-party dialogue with the first dialogue model, that is, the first dialogue model establishes a multi-party dialogue with dialogue group 1 (the second dialogue model 1, the second dialogue model 2, and the second dialogue model 3) to form dialogue process 1; the first dialogue model establishes a multi-party (one-on-one) dialogue with dialogue group 2 (the second dialogue model 4) to form dialogue process 2; the first dialogue model establishes a multi-party dialogue with dialogue group 3 (the second dialogue model 5 and the second dialogue model 6) to form dialogue process 3.

[0094] Taking the multi-party conversation of conversation process 1 as an example, the output content of the first conversation model can be used as the prompt words respectively input into the second conversation model 1, the second conversation model 2, and the second conversation model 3, triggering the second conversation model 1, the second conversation model 2, and the second conversation model 3 to generate corresponding output contents respectively; taking the output content of the second conversation model 1 as the prompt words respectively input into the first conversation model, the second conversation model 2, and the second conversation model 3, triggering the first conversation model, the second conversation model 2, and the second conversation model 3 to generate corresponding output contents respectively... Thus, a multi-party conversation between the first conversation model and multiple second conversation models is realized.

[0095] In step S303, referring to Figure 5 , in conversation process 1, the first conversation model and the second conversation model 1, the second conversation model 2, and the second conversation model 3 simulate a conversation carried out by multiple conversation parties with the first emotion information, the second emotion information 1, the second emotion information 2, and the second emotion information 3 respectively, so as to obtain virtual conversation information 1, and the virtual conversation information 1 contains the respective speeches of the first conversation model, the second conversation model 1, the second conversation model 2, and the second conversation model 3. Similarly, conversation process 2 and conversation process 3 also generate virtual conversation information 2 and virtual conversation information 3 respectively.

[0096] By executing step S303, the obtained virtual conversation information is the conversation information obtained by using the conversation model to simulate multiple conversation parties with emotions the same as those of the driver and other detection objects respectively, and the conversation parties carry out conversations with each other. For example, the conversation between the first conversation model and the second conversation model 1 is a conversation simulated by a conversation party with an emotion the same as that of the driver and a conversation party with an emotion the same as that of occupant 1; the conversation between the first conversation model and the second conversation model 4 is a conversation simulated by a conversation party with an emotion the same as that of the driver and a conversation party with an emotion the same as that of the anthropomorphic image of the vehicle itself; the conversation between the first conversation model and the second conversation model 5 is a conversation simulated by a conversation party with an emotion the same as that of the driver and a conversation party with an emotion the same as that of vehicle 1 (or its driver). Therefore, each virtual conversation information simulates the possible results when the parties related to the traffic process participated by the vehicle interact according to their respective emotions.

[0097] When executing step S303, by grouping the second conversation models according to the proximity of the interaction distance, the interaction between other detection objects with the same interaction distance and the driver can be simulated, and the interaction between other detection objects with different interaction distances can be isolated, so that the obtained virtual conversation information is closer to the real interaction between the driver and each other detection object.

[0098] For example, for Figure 5For each of the other detection objects shown in the list to be detected, according to the interaction distance interval, in addition to the emotional interaction between the driver and the occupants 1, 2, and 3, there will also be interactions among them. However, the interactions between these occupants and other detection objects such as the adjacent vehicle 1 tend to be non-existent. Therefore, the second dialogue models 1, 2, and 3 corresponding to the occupants 1, 2, and 3 are formed into a dialogue group 1 and carry out a dialogue process 1 together with the first dialogue model; similarly, dialogue processes 2 and 3 are carried out. The dialogue processes 1, 2, and 3 are independent of each other. Through such a dialogue process design, the emotional interaction between the driver and various other detection objects in a real driving environment can be well simulated.

[0099] In this embodiment, when performing step S304, driving advice information can be generated according to the virtual dialogue information obtained in step S303. Specifically, the in-vehicle computer can perform an emotion detection on the speech of each second dialogue model at the end stage of each dialogue process in the virtual dialogue information, so as to determine the emotion type of the speech of the second dialogue model, and generate driving advice information according to the emotion type.

[0100] For example, if in the virtual dialogue information 1 generated by the dialogue process 1, the emotion type of the speech of the second dialogue model 2 (corresponding to the occupant 2) at the end stage is "low", then the in-vehicle computer can generate driving advice information with the content of "Your emotion may make the co-passengers unhappy. Please keep silent and concentrate on driving" or "Everyone, stop talking and play a song"; if in the virtual dialogue information 2 generated by the dialogue process 2, the emotion type of the speech of the second dialogue model 4 (corresponding to the vehicle itself) at the end stage is "excited", then the in-vehicle computer can generate driving advice information with the content of "Your driving operation may make your car too excited. Please don't drive too fast"; if in the virtual dialogue information 3 generated by the dialogue process 3, the emotion type of the speech of the second dialogue model 5 (corresponding to the adjacent vehicle 1) at the end stage is "angry", then the in-vehicle computer can generate driving advice information with the content of "The behavior of the adjacent vehicle is dangerous. Please don't approach or imitate it. Keep calm and drive in your own lane" or "You've worked hard. Turn on the air conditioner to cool down".

[0101] The driving advice information obtained by performing steps S301 - S304 can be used as an active interaction strategy and executed in step S4. Specifically, when the in-vehicle computer performs step S4, it can call the interaction module and play the driving advice information through the speaker of the interaction module or display it through the HMI interface (Human Machine Interface) of the display screen of the interaction module, so as to provide driving reference for the driver.

[0102] According to Figure 3 、 Figure 4 andFigure 5 As can be seen from the embodiments such as , by performing steps S1 - S4, the emotional interaction caused by the language interaction and driving operation interaction between the driver and various traffic participants (other detected objects) such as passengers, the vehicle itself, people outside the vehicle, and adjacent vehicles can be simulated according to the emotional information of the driver and other detected objects, that is, the first emotional information and the second emotional information. Specifically, the simulation of the emotional interaction can be achieved by generating a dialogue model based on the emotional information, so as to project the driver and each other detected object into each dialogue model and conduct a dialogue. Finally, in the obtained virtual dialogue information, it contains the emotional development result generated by the interaction of the emotional information of the driver and each other detected object in the current driving environment. Therefore, corresponding driving advice information can be generated according to the emotional development result contained in the virtual dialogue information as the active interaction strategy of the in - vehicle computer for the driver. That is, the in - vehicle computer transmits the driving advice information to the driver as a way for the in - vehicle computer to execute the active interaction strategy, so that the driver can obtain the driving advice information to cope with the emotional development result that may be caused by the interaction between the driver and various traffic participants (other detected objects). For example, if the finally obtained virtual dialogue information contains a negative emotional development result (for example, the speech of the dialogue model of a certain other detected object shows a negative emotion type), then driving advice information with the content of "drive carefully" can be generated to guide the driver to drive carefully, so as to effectively cope with the possible driving risks.

[0103] In this embodiment, if the content of the driving advice information is related to the control of in - vehicle functional components (such as the audio - visual entertainment system and air conditioner), such as "everyone, stop talking and play a song" or "you've worked hard, turn on the air conditioner to cool down", then when performing step S4, in addition to playing or displaying the driving advice information, the in - vehicle computer can also automatically execute a control instruction according to the driving advice information. For example, if the content of the driving advice information is "everyone, stop talking and play a song", then the in - vehicle computer can generate a control instruction to play a song on the audio - visual entertainment system, so as to achieve automatic song playing without the driver's operation. If the content of the driving advice information is "you've worked hard, turn on the air conditioner to cool down", then the in - vehicle computer can generate a control instruction to control the air conditioner to cool and send air, so as to achieve automatic air conditioner turning - on without the driver's operation.

[0104] In summary, the vehicle cockpit control method in this embodiment has the following advantages:

[0105] 1. More comprehensive active interaction: Combining multi - party information such as specific road conditions, the driver, co - passengers, and people outside the vehicle for interaction decision - making can more comprehensively meet various needs in the driving scenario and improve the driving experience;

[0106] 2. Wide interaction range: Considering the influence of co - passengers and people outside the vehicle on the driver, multi - party interaction is realized, enabling the cockpit system to provide more personalized and user - friendly services, enhancing the overall driving comfort and safety.

[0107] 3. Improve the level of emotion recognition and interaction design: Multi - modal data fusion improves the accuracy of emotion recognition, optimizes the interaction design according to different scenarios and user differences, reduces the driver's cognitive load, and enhances driving safety and comfort.

[0108] In this embodiment, referring to Figure 6 , it is also possible to Figure 5 extract the emotion type information corresponding to the speeches of each party in the virtual conversation information from Figure 6 and store it in the user emotion knowledge base. Therefore, the user emotion knowledge base contains the emotion type information that may appear for multiple traffic participants during the driving process of this vehicle. The in - vehicle computer pre - generates driving advice information and corresponding trigger scenarios as active interaction strategies based on this emotion type information. For example, for the emotion type information of "anger", driving advice information with the content of "How about playing your favorite relaxing music to relax?" can be generated, and the corresponding trigger scenario is determined as "the current road condition information is congestion"; for the emotion type information of "irritability", driving advice information with the content of "Don't worry, just drive safely" can be generated, and the corresponding trigger scenario is determined as "the current road condition information is heavy traffic flow". In this way, when it is detected that the current road condition information is "congestion", the in - vehicle computer can determine that the driver's emotion type information is "anger", and thus read out the driving advice information with the content of "How about playing your favorite relaxing music to relax?" and conduct active interaction with the driver; when it is detected that the current road condition information is "heavy traffic flow", the in - vehicle computer can determine that the driver's emotion type information is "irritability", and thus read out the driving advice information with the content of "Don't worry, just drive safely" and conduct active interaction with the driver. Therefore, by establishing the

[0109] user emotion knowledge base shown in

[0110] , it is possible to achieve active interaction with the driver when specific trigger conditions (such as specific real - time road condition information) are met, thereby achieving effects such as alleviating the driver's bad emotions and ensuring driving safety. The computer program for implementing the vehicle cockpit control method in this embodiment can be written and stored in a computer device or storage medium. When the computer program is read and run, it executes the vehicle cockpit control method in this embodiment, thereby achieving the same technical effects as the vehicle cockpit control method in the embodiment.

[0110] It should be noted that, unless otherwise specified, when a certain feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. In addition, the descriptions such as up, down, left, and right used in this disclosure are only relative to the mutual positional relationship of the components of this disclosure in the drawings. The singular forms of "a", "an", and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as commonly understood by those skilled in the technical field of this technology. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this embodiment includes any and all combinations of one or more of the related listed items.

[0111] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of this disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as", etc.) provided in this embodiment is only intended to better illustrate the embodiments of the present invention and will not impose a limitation on the scope of the present invention unless otherwise required.

[0112] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method can be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with the computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can run on a dedicated integrated circuit programmed for this purpose.

[0113] In addition, the operations of the processes described in this embodiment may be performed in any suitable order, unless this embodiment otherwise indicates or is clearly inconsistent with the context in other detected objects. The processes described in this embodiment (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executed commonly on one or more processors, by hardware, or a combination thereof. A computer program includes a plurality of instructions executable by one or more processors.

[0114] Furthermore, the method may be implemented in any type of computing platform operably connected, including but not limited to personal computers, minicomputers, mainframes, workstations, network or distributed computing environments, separate or integrated computer platforms, or communicating with charged particle tools or other imaging devices, etc. Aspects of the present invention may be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into the computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer and, when the storage medium or device is read by the computer, can be used to configure and operate the computer to execute the processes described herein. In addition, the machine-readable code, or portions thereof, may be transmitted via a wired or wireless network. When such media include instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0115] The computer program can be applied to input data to perform the functions of this embodiment, thereby converting the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on a display.

[0116] The above are only the preferred embodiments of the present invention. The present invention is not limited to the above embodiments. As long as it achieves the technical effects of the present invention by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Within the scope of protection of the present invention, its technical solutions and / or implementation manners may have various different modifications and variations.

Claims

1. A method for controlling a car cabin, characterized in that: The automobile cockpit control method comprises: Detecting the driver's first emotional information; Detecting second emotion information of other detection objects; the other detection objects are detection objects other than the driver; determining an active interaction strategy for the driver according to the first emotion information and the second emotion information; The active interaction strategy is executed.

2. The vehicle cabin control method according to claim 1, characterized in that: The detecting the first emotion information of the driver comprises: Performing multimodal detection on the driver to obtain first multimodal data; Perform emotion recognition on the first multimodal data to obtain the first emotion information.

3. The vehicle cabin control method according to claim 1, characterized in that: The detecting the second emotion information of other detection objects includes: Determine a list of people to be tested; the list of people to be tested includes at least one of the other test objects; For any of the other detection objects in the to-be-detected list, perform multimodal detection on the other detection object to obtain second multimodal data; Perform emotion recognition on the second multimodal data to obtain the second emotion information.

4. The vehicle cockpit control method according to claim 3, characterized in that: Determining the list of people to be tested includes: Acquire a candidate list; the candidate list includes a plurality of other detection objects, each of which has a corresponding priority, and for any other detection object, the priority of the other detection object is negatively correlated with the interaction distance between the other detection object and the driver; Detect current road condition information; Determine the number to be detected according to the current road condition information; According to each of the priorities, preferentially selecting the other detection objects of the number to be detected from the candidate list; The list to be tested is established based on the other selected test objects.

5. The vehicle cockpit control method according to claim 4, characterized in that: The obtaining of the candidate list comprises: Detecting the interior of the vehicle cabin, and when an occupant is detected, determining the occupant as the other detection object; Detecting the outside of the vehicle cabin, and when a person outside the vehicle is detected, determining the person outside the vehicle as the other detection object; Detecting the exterior of the vehicle cabin, and when a nearby vehicle is detected, determining the nearby vehicle as the other detection object; Performing a sensor system self-check on the vehicle, and when it is detected that the sensor system is normal, determining the vehicle as the other detection object; The candidate list is established based on all the other detection objects that have been determined.

6. The vehicle cockpit control method according to claim 1, characterized in that: The determining, according to the first emotion information and the second emotion information, an active interaction strategy for the driver includes: configuring a first dialogue model according to the first emotion information; configuring a second dialogue model according to the second emotion information; executing a dialogue between the first dialogue model and the second dialogue model to obtain virtual dialogue information; The active interaction strategy is determined according to the virtual dialogue information.

7. The vehicle cockpit control method according to claim 6, characterized in that: The executing the dialogue between the first dialogue model and the second dialogue model includes: When there is one second dialogue model, the first dialogue model and the second dialogue model are used as dialogue parties in the same dialogue process to perform a dialogue; When there are multiple second dialogue models, detecting the distribution of interaction distances of the other detection objects corresponding to the second dialogue models, grouping the second dialogue models according to the intervals in which the corresponding interaction distances are located, and obtaining at least one dialogue group, wherein one dialogue group includes at least one second dialogue model whose corresponding interaction distances are in the same interval; Establishing at least one dialogue process, the number of the dialogue processes being the same as the number of the dialogue groups; For any of the dialogue processes, the dialogue is performed with the first dialogue model and a corresponding one of the dialogue groups as dialogue parties.

8. The vehicle cockpit control method according to claim 6, characterized in that: Determining the active interaction strategy according to the virtual dialogue information includes: Driving suggestion information is generated according to the virtual dialogue information.

9. A computer device, characterized in that: It comprises a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the automobile cockpit control method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the automobile cockpit control method described in any one of claims 1 to 8 when executed by the processor.

Images