Environment control method and device based on emotional state, equipment and storage medium
By identifying the emotional state of passengers in driverless vehicles and using virtual reality technology and environmental parameters to adjust, negative emotional problems caused by long-term driving are solved and the riding experience is improved.
Patent Information
- Application Number
- CN202411773056.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-04-04
AI Technical Summary
During the long driving of an unmanned vehicle, passengers in the cabin are prone to negative emotions such as anxiety and fatigue due to the closed and isolated environment, which affects the riding experience.
By identifying that the passenger's emotional state is negative, the cockpit domain controller is used to generate suitable emotional virtual environment images and background audio, and display them to the passenger through virtual reality technology, while adjusting the environmental parameters in the cockpit to create a comfortable riding environment.
Effectively improve passengers' emotional state, improve their riding experience, and provide a more relaxed and pleasant riding environment through the combined effects of vision, hearing and somatosensory.
Smart Images
Figure CN119749444A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of unmanned driving, and in particular to an emotional state-based environment control method, device, equipment and storage medium. Background Art
[0002] With the rapid development of vehicle technology, the vehicle's cockpit system has made significant progress. It can provide basic functions such as safety monitoring, navigation and information display for passengers in the cabin, such as drivers and / or passengers, and integrates advanced human-computer interaction interfaces and real-time environmental monitoring systems to ensure the safety and reliability of the vehicle in various complex environments.
[0003] However, when the driverless vehicle is driving for a long time, the passengers in the cabin will experience negative emotions such as anxiety and fatigue due to being in a closed and isolated environment for a long time. Therefore, how to effectively adjust the emotional state of the passengers in the cabin to improve the riding experience has become an important issue that needs to be solved urgently. Summary of the invention
[0004] The present application provides an emotional state-based environment control method, device, equipment and storage medium for adjusting the emotional state of passengers in a cabin and improving the riding experience.
[0005] In a first aspect, the present application provides an emotional state-based environment control method, which is applied to a cockpit domain controller of a vehicle, and the method comprises:
[0006] If it is identified that the emotional state of the passenger in the cabin is a negative state, determining a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin;
[0007] Determining, according to the virtual environment image, background audio and target environment parameters that match the virtual environment image;
[0008] The virtual environment image and the background audio are displayed to the passenger in a virtual reality manner, and the environment in the cabin is controlled according to the target environment parameters.
[0009] In a possible implementation manner, determining the virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin includes:
[0010] determining an adjusted target emotional state according to the emotional state;
[0011] Using a pre-acquired environment image generation model, the environment parameters and the target emotional state are processed to determine the virtual environment image;
[0012] The environmental image generation model is obtained by training a deep learning model based on multiple groups of environmental parameters and the emotional state corresponding to each group of environmental parameters.
[0013] In a possible implementation manner, determining, according to the virtual environment image, background audio and target environment parameters that match the virtual environment image includes:
[0014] Generating audio matching the virtual environment image to obtain the background audio;
[0015] Using a pre-acquired environmental parameter determination model, the virtual environment image and the target emotional state are processed to determine the target environmental parameters;
[0016] The environment parameter determination model is obtained by training a deep learning model based on multiple virtual environment images and the emotional state corresponding to each virtual environment image.
[0017] In a possible implementation manner, presenting the virtual environment image and the background audio to the passenger in a virtual reality manner includes:
[0018] Performing fusion processing on the virtual environment image and the background audio to generate a virtual reality video;
[0019] The virtual reality video is input into the brain-computer interface worn by the passenger.
[0020] In a possible implementation, inputting the virtual reality video into the brain-computer interface worn by the passenger includes:
[0021] encoding the virtual reality video to obtain a neural signal;
[0022] The neural signal is input into a brain-computer interface worn by the passenger.
[0023] In a possible implementation, the method further includes:
[0024] collecting brain wave information of the passenger in real time through the brain-computer interface;
[0025] Using a pre-acquired emotion recognition model, performing emotion recognition on the brain wave information to determine the passenger's emotional state, where the emotional state includes a positive state or a negative state;
[0026] The emotion recognition model is obtained by training a deep learning model based on brain wave information and preset emotional state labels.
[0027] In a possible implementation manner, the environmental parameters include cabin temperature, cabin humidity, and cabin light intensity, and the method further includes:
[0028] The temperature in the cabin, the humidity in the cabin, and the light intensity in the cabin are respectively acquired according to the temperature sensor, the humidity sensor, and the light sensor arranged in the cabin.
[0029] In a second aspect, the present application provides an environment control device based on emotional state, comprising:
[0030] A first processing module, for determining a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin if the emotional state of the passenger in the cabin is identified as a negative state;
[0031] A second processing module is used to determine background audio and target environment parameters matching the virtual environment image according to the virtual environment image;
[0032] The third processing module is used to display the virtual environment image and the background audio to the passenger in a virtual reality manner, and control the environment in the cabin according to the target environment parameters.
[0033] In a third aspect, the present application provides a cockpit domain controller, including: a processor, a memory;
[0034] The memory stores computer-executable instructions;
[0035] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the emotional state-based environment control method as described in any one of the first aspects.
[0036] In a fourth aspect, the present application provides a vehicle, comprising: a vehicle body and a cockpit domain controller as described in the third aspect.
[0037] In a fifth aspect, the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the emotional state-based environment control method described in any one of the first aspects.
[0038] In a sixth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it is used to implement the emotional state-based environment control method described in any one of the first aspects.
[0039] The emotional state-based environment control method, device, equipment and storage medium provided in the present application can determine the virtual environment image according to the negative emotional state of the passenger and the environmental parameters in the cabin, and match the virtual environment image with appropriate background audio, and display the image and sound to the passenger through virtual reality technology, so that the passenger can immerse in a comfortable scene. At the same time, the environment in the cabin can also be controlled according to the target environmental parameters determined by the virtual environment image, thereby comprehensively improving the passenger's riding experience in terms of vision, hearing and body sensation. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0041] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application;
[0042] Figure 2 A flowchart of Embodiment 1 of the method for controlling an environment based on an emotional state provided in the present application;
[0043] Figure 3 A flow chart of Embodiment 2 of the method for controlling an environment based on an emotional state provided in the present application;
[0044] Figure 4 A schematic diagram of the structure of a first embodiment of an environment control device based on emotional state provided by the present application;
[0045] Figure 5 A schematic diagram of the structure of Embodiment 2 of the environment control device based on emotional state provided by the present application;
[0046] Figure 6 A schematic diagram of the structure of a cockpit domain controller provided in an embodiment of the present application.
[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0049] With the continuous advancement of vehicle technology, the cockpit system of existing driverless vehicles can already provide a variety of basic functions, such as safety monitoring, which can monitor the traffic conditions outside the vehicle, identify pedestrians, vehicles and obstacles, and take automatic braking or avoidance measures when necessary, navigation information can provide accurate navigation information, and information display can let passengers know the status of the vehicle in real time. In addition, the cockpit system also integrates an advanced human-computer interaction interface, allowing passengers to interact with the system through touch screens, voice commands, etc., which improves the convenience of operation and the pleasure of riding.
[0050] However, in driverless vehicles, passengers often feel uncomfortable because they are in a small cabin. This discomfort is caused by the combined influence of multiple factors. First, the continuous noise during driving will disturb the passengers. In addition, the closed and isolated environment in the cabin limits the passengers' activity space, which can easily cause loneliness and boredom. These factors work together, usually increasing the negative emotions of passengers, such as anxiety, irritability and fatigue, thereby reducing the passengers' riding experience.
[0051] In response to the above problems, the inventors considered that the emotional state of the passengers could be identified, and when it is determined that the emotional state of the passengers is negative, the environment in the cabin can be adjusted to improve the emotional state of the passengers and enhance the passenger's riding experience. Specifically, when it is determined that the emotional state of the passengers is negative, suitable virtual environment images and background audio can be generated and displayed to the passengers through virtual reality, creating a relaxing and pleasant virtual space for the passengers. In addition, the physical environment in the cabin can also be adjusted, such as adjusting the brightness and color temperature of the lighting to create a more comfortable visual environment. Through the coordination of multiple senses, the passenger's riding experience is comprehensively improved. Based on this, the present application proposes an environmental control method based on emotional state to further improve the riding experience of passengers in the cabin.
[0052] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 The unmanned vehicle 10 includes a cockpit domain controller 11 and a cockpit 12 that can carry passengers. During the driving process of the unmanned vehicle 10, the cockpit domain controller 11 can adjust the environment in the cockpit 12 to improve the emotional state of the passengers, thereby improving the riding experience of the passengers.
[0053] It should be noted that the application scenarios applicable to this application are not limited to unmanned vehicles, but can also be extended to manned aircraft, unmanned flying cars and other fields.
[0054] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0055] Figure 2 This is a flow chart of the first embodiment of the emotional state-based environment control method provided in this application. Figure 2 , the method comprising:
[0056] S201: If it is identified that the emotional state of the passenger in the cabin is a negative state, a virtual environment image is determined according to the emotional state of the passenger and the environmental parameters in the cabin.
[0057] The execution subject of the embodiment of the present application may be a cockpit domain controller, or an emotional state-based environment control device set in the cockpit domain controller. The emotional state-based environment control device may be implemented by software, or by a combination of software and hardware. The emotional state-based environment control device may be a processor in the cockpit domain controller. For ease of understanding, the following description is made by taking the cockpit domain controller as an example.
[0058] In this step, the cockpit domain controller can obtain the environmental parameters in the cockpit through sensors installed in the cockpit. When it is identified that the emotional state of the passengers in the cockpit is negative, the virtual environment image can be determined based on the emotional state of the passengers and the environmental parameters in the cockpit.
[0059] Optionally, negative emotional states may include sadness, grief, sadness, anger, fear, anxiety, disappointment, loneliness, etc. Positive emotional states corresponding to negative emotional states may include happiness, excitement, satisfaction, gratitude, confidence, etc.
[0060] Environmental parameters may include cabin temperature, cabin humidity, and cabin light intensity. Optionally, the cabin temperature, cabin humidity, and cabin light intensity may be acquired by a temperature sensor, a humidity sensor, and a light sensor disposed in the cabin.
[0061] For virtual environment images, they can be visual content used to improve negative emotional states, and can include natural scenery, tranquil beaches, starry night scenes, gorgeous flowers, or other scenes with soothing effects.
[0062] For example, when the driverless vehicle is driving, the cockpit domain controller can collect the cabin temperature of 10 degrees Celsius (℃) through the temperature sensor set in the cabin, collect the cabin humidity of 5%, and collect the cabin light intensity of 100 lux through the light sensor. When the emotional state of the passenger in the cabin is identified as anxious, the virtual environment image can be determined as a quiet beach based on the emotional state of anxiety and the environmental parameters in the cabin.
[0063] S202: Determine background audio and target environment parameters that match the virtual environment image according to the virtual environment image.
[0064] In this step, after determining the virtual environment image, the cockpit domain controller can generate audio matching the virtual environment image to obtain background audio; and determine target environment parameters for adjusting the environment parameters in the cockpit according to the virtual environment image.
[0065] In an optional implementation, a pre-acquired background audio generation model may be used to process the virtual environment image to generate audio matching the virtual environment image to obtain background audio. The background audio generation model is obtained by training a deep learning model based on multiple virtual environment images and the background audio corresponding to each virtual environment image. The deep learning model may be a convolutional neural network (CNN).
[0066] In another optional implementation, a mapping relationship between the virtual environment image and the background audio is pre-established in the database of the cockpit domain controller, and the background audio matching the virtual environment image can be determined from the database based on the pre-established mapping relationship.
[0067] Background audio may be auditory content used to improve negative emotional states, and may include birdsong, breeze, waves, flowing water, or other sounds with a soothing effect.
[0068] For example, according to the pre-established mapping, it can be determined that the background audio that matches the natural scenery is the sound of birdsong; the background audio that matches the quiet beach is the sound of breeze; and the background audio that matches the starry night view is the sound of flowing water.
[0069] Optionally, the target environment parameter may include at least one of a target temperature, a target humidity, and a target light intensity in the cabin. For example, according to the virtual environment image of a quiet beach, the target environment parameters may be determined to include: a target temperature of 18° C., a target humidity of 15%, and a target light intensity of 150 Lux.
[0070] S203: Display the virtual environment image and background audio to the passengers in a virtual reality manner, and control the environment in the cabin according to the target environment parameters.
[0071] In this step, virtual reality technology can be used to display virtual environment images and background audio to passengers in an immersive manner, and the environment in the cabin can be adjusted according to target environmental parameters to enhance the riding experience.
[0072] In an optional implementation, the virtual environment image and background audio may be fused to generate a virtual reality video, which is then input into a brain-computer interface (BCI) worn by the passenger to enhance the passenger's sense of reality.
[0073] Optionally, the temperature in the cabin can be adjusted by a temperature control system according to the target temperature in the target environmental parameters; the humidity in the cabin can be adjusted by a humidity control system according to the target humidity in the target environmental parameters; and the light intensity in the cabin can be adjusted by a light adjustment system according to the target light intensity in the target environmental parameters.
[0074] For example, the tranquil beach and the sound of the breeze can be fused through virtual reality technology to generate a highly immersive virtual reality video, which can be input into the brain-computer interface worn by passengers to enhance the passengers' sense of reality. In addition, the cabin temperature of 10°C, the cabin humidity of 5%, and the cabin light intensity of 100Lux can be adjusted according to the target environmental parameters, including the target temperature of 18°C, the target humidity of 15%, and the target light intensity of 150Lux, to ensure that the environment is consistent with the virtual reality video.
[0075] In the embodiment of the present application, if the cockpit domain controller identifies that the emotional state of the passenger in the cockpit is negative, the virtual environment image can be determined according to the emotional state of the passenger and the environmental parameters in the cockpit; the background audio and target environmental parameters matching the virtual environment image can be determined according to the virtual environment image, and then the virtual environment image and background audio can be presented to the passenger through virtual reality technology. At the same time, the environment in the cockpit is adjusted according to the target environmental parameters to improve the emotional state of the passenger and enhance the riding experience.
[0076] exist Figure 2 Based on the embodiment shown below, combined with Figure 3 , the above-mentioned emotional state-based environment control method is further described in detail.
[0077] Figure 3 This is a flow chart of the second embodiment of the emotional state-based environment control method provided in this application. Figure 3, the method may include:
[0078] S301. Collect passengers' brain wave information in real time through brain-computer interface.
[0079] In this step, the cockpit domain controller can communicate with the BCI to obtain the passengers' brain wave information collected by the BCI in real time.
[0080] In a specific implementation, when an unmanned vehicle is traveling for a long time, passengers can wear BCI to interact with the vehicle, and the cockpit domain controller can communicate with the BCI to obtain the passenger's brain wave information collected in real time by the BCI.
[0081] Optionally, brain wave information generally refers to brain activity data recorded and analyzed by electroencephalogram (EEG) technology, where EEG is an electrical signal generated by brain neuron activity, which can be detected by an EEG sensor. The EEG signal sampling frequency can be preset. For example, the EEG signal sampling frequency can be 200 Hz.
[0082] Specifically, the brain's electrical activity signals can be collected through the electroencephalogram sensor array in the BCI, and the electrical activity signals can be pre-amplified to enhance the strength of the electrical activity signals. Subsequently, a filter can be used to remove noise and other unnecessary frequency components, thereby improving the quality and accuracy of the signal. Finally, the filtered electrical activity signals can be digitized to extract differential entropy features and obtain brain wave information.
[0083] S302: Use the pre-acquired emotion recognition model to perform emotion recognition on the brain wave information to determine the emotional state of the passenger.
[0084] In this step, the emotion recognition model acquired in advance can be used to perform emotion recognition processing on the brain wave information. By analyzing the collected brain wave information, the model can identify the emotional state of the passenger and determine whether the current emotion is in a positive state or a negative state. The emotion recognition model is obtained by training a deep learning model based on brain wave information and preset emotional state labels.
[0085] Optionally, the emotion recognition model can be obtained through the following steps.
[0086] Step ①: Construct multiple training samples, each of which includes a brain wave information and a preset emotional state label. The emotional state label can be multi-classified, such as happy, sad, angry, anxious, etc.
[0087] Step ②: Based on multiple training samples, train the deep learning model to obtain an emotion recognition model. The deep learning model may be a convolutional neural network.
[0088] For example, a pre-acquired emotion recognition model may be used to perform emotion recognition on brain wave information to determine that the passenger's emotional state is an anxious state.
[0089] In an optional implementation, the emotional state of the passenger may also be determined by any one of the following methods.
[0090] Method 1: Collecting the passenger's facial expression and determining the passenger's emotional state based on the facial expression. Specifically, the passenger's facial expression can be collected by an image acquisition device such as a camera installed inside the vehicle.
[0091] Method 2: Collecting the passenger's voice or tone, and determining the passenger's emotional state based on the voice or tone. Specifically, the passenger's voice or tone can be collected by a voice collection device such as an audio sensor installed inside the vehicle.
[0092] It should be noted that a comprehensive analysis may be performed on at least two of the acquired brain wave information, facial expressions, and voice or intonation information to improve the accuracy of emotional state recognition.
[0093] S303: If it is identified that the emotional state of the passenger in the cabin is a negative state, determine an adjusted target emotional state according to the emotional state.
[0094] In this step, a target emotional state can be determined based on identifying that the emotional state of the passenger in the cabin is a negative state, wherein the target emotional state can be a positive emotional state opposite to the negative emotional state.
[0095] For example, if the emotional state of the passenger in the cabin is identified as anxious, the target emotional state may be relaxed; if the emotional state of the passenger in the cabin is identified as sad, the target emotional state may be happy.
[0096] In an optional implementation, an emotion mapping relationship between a negative state and a target emotion state is pre-established in a database of the cockpit domain controller, and a positive emotion state opposite to the negative emotion state can be determined from the database based on the pre-established emotion mapping relationship.
[0097] S304: Using the pre-acquired environment image generation model, process the environment parameters and the target emotional state to determine the virtual environment image.
[0098] In this step, the pre-acquired environment image generation model can be used to process the environment parameters acquired by the sensors in the cabin and the determined target emotional state to generate a virtual environment image that can enable the passenger to reach the target emotional state. The environment image generation model is obtained by training a deep learning model based on multiple sets of environment parameters and the emotional state corresponding to each set of environment parameters.
[0099] Optionally, the environment image generation model can be obtained through the following steps.
[0100] Step ①: Construct multiple training samples. Each training sample includes a set of environmental parameters, the emotional state corresponding to the set of environmental parameters, and the corresponding virtual environment image.
[0101] Step ②: Based on multiple training samples, the deep learning model is trained to obtain an environmental image generation model. The deep learning model may be a Generative Adversarial Network (GAN).
[0102] For example, a pre-acquired environmental image generation model can be used to process the environmental parameters acquired by the cabin sensor and the determined target emotional state as a relaxed state to generate a virtual environmental image that can make passengers reach a relaxed state. The virtual environmental image can be a quiet beach; the environmental parameters acquired by the cabin sensor can include: the cabin temperature is 10°C, the cabin humidity is 5%, and the cabin light intensity is 100Lux.
[0103] S305: Generate audio that matches the virtual environment image to obtain background audio.
[0104] For example, a mapping relationship between a virtual environment image and background audio is pre-established in the database of the cockpit domain controller, and according to the pre-established mapping relationship, it can be determined from the database that the background audio matching the quiet beach is the sound of breeze.
[0105] S306: Using the pre-acquired environment parameter determination model, the virtual environment image and the target emotional state are processed to determine the target environment parameters.
[0106] In this step, the pre-acquired environment parameter determination model can be used to process the generated virtual environment image and the determined target emotional state to determine the target environment parameters for adjusting the environment parameters in the cabin. The environment parameter determination model is obtained by training a deep learning model based on multiple virtual environment images and the emotional state corresponding to each virtual environment image.
[0107] Optionally, the environmental parameters can be obtained by the following steps to determine the model.
[0108] Step ①: Construct multiple training samples. Each training sample includes a virtual environment image, the emotional state corresponding to the virtual environment image, and the corresponding environment parameters.
[0109] Step ②: Based on multiple training samples, train the deep learning model to obtain an environment image generation model. The deep learning model may be a convolutional neural network.
[0110] For example, the model for determining the environmental parameters acquired in advance can be used to process the generated virtual environment image as a quiet beach and the determined target emotional state as a relaxed state, and determine the target environmental parameters for adjusting the environmental parameters in the cabin. The target environmental parameters may include: the target temperature in the cabin is 18°C, the target humidity is 15%, and the target light intensity is 150 Lux.
[0111] S307: Fusing the virtual environment image and the background audio to generate a virtual reality video.
[0112] In this step, the virtual environment image and the background audio can be fused to generate an immersive virtual reality video. In a specific implementation, the virtual environment image and the background audio can be synchronously fused to ensure the coordination of vision and hearing. Spatial audio technology is used to enhance the directionality and depth of sound, so that passengers can perceive the source and distance of the sound, thereby enhancing the overall immersion. Then, the processed audio can be combined with the virtual environment image through rendering technology to generate a highly immersive virtual reality video.
[0113] S308. Input the virtual reality video into the brain-computer interface worn by the passenger, and control the environment in the cabin according to the target environment parameters.
[0114] In this step, the generated virtual reality video can be input into the brain-computer interface worn by the passenger to achieve a deeper interactive experience. At the same time, the environmental parameters in the cabin can be adjusted to match the target environmental parameters to enhance the authenticity of the virtual reality experience.
[0115] In an optional implementation, the virtual reality video can be encoded and processed to obtain neural signals so as to match the natural signal pattern of the human brain. Then, the neural signals are input into the brain-computer interface worn by the passenger. In a specific implementation, the virtual reality video can be encoded into neural signals through a memristor array, and the processed neural signals are input into the brain through the brain-computer interface worn by the passenger to achieve a true perception of the virtual reality video.
[0116] In another optional implementation, the virtual reality video can also be displayed to passengers in a more immersive and interactive way through high-resolution display devices, such as holographic images and mixed reality (MR) devices. These devices can provide realistic visual effects and dynamic environmental interactions, allowing passengers to not only watch virtual reality videos, but also interact with the video content in real time, enhancing the passenger experience.
[0117] In a specific implementation, when adjusting the light intensity in the cabin according to the target light intensity included in the target environment parameter, the cockpit domain controller can achieve this by setting a programmable LED light in the light adjustment system. Specifically, the cockpit domain controller can adjust the color and brightness of the LED light in the light adjustment system according to the target light intensity, thereby adjusting the light intensity in the cabin.
[0118] For example, there are 256 programmable LED lights in the cockpit. The cockpit domain controller can calculate the difference between the current light intensity of 100 Lux and the target light intensity of 150 Lux based on the target light intensity of 150 Lux, and adjust the brightness output of each LED lamp in the lighting control system according to the difference to achieve the target light intensity of 150 Lux.
[0119] In an embodiment of the present application, the cockpit domain controller can collect the passenger's brain wave information through the brain-computer interface and use the emotion recognition model to identify the emotional state. According to the emotional state being a negative state, the target emotional state is determined, and a virtual environment image and matching background audio are generated. Then, the environmental parameter determination model is adopted to process the virtual environment image and the target emotional state, determine the target environmental parameters, fuse the virtual environment image and the background audio, generate a virtual reality video, input the virtual reality video into the brain-computer interface worn by the passenger, and adjust the cabin environment according to the target environmental parameters. In the above process, by identifying the negative emotional state of the passenger, a virtual reality video for improving the emotion can be generated, and the target environmental parameters of the cabin environment can be adjusted, thereby improving the passenger's riding experience.
[0120] Furthermore, the emotional state-based environment control method provided in this embodiment, combined with the brain-computer interface, can convert virtual reality videos into neural signals, directly acting on the passenger's brain, creating an immersive feeling for the passenger and enhancing the sense of immersion; combined with the adjustment of the cabin environment parameters, it can improve the passenger's riding comfort, further help passengers improve their negative emotional states, and comprehensively improve the riding experience.
[0121] Figure 4 This is a schematic diagram of the structure of the first embodiment of the environment control device based on emotional state provided by this application. Figure 4 , the environment control device 20 based on the emotional state includes:
[0122] A first processing module 21 is used to determine a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin if the emotional state of the passenger in the cabin is identified as a negative state;
[0123] A second processing module 22, configured to determine background audio and target environment parameters matching the virtual environment image according to the virtual environment image;
[0124] The third processing module 23 is used to display the virtual environment image and the background audio to the passenger in a virtual reality manner, and control the environment in the cabin according to the target environment parameters.
[0125] The emotional state-based environment control device provided in this embodiment can execute the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.
[0126] In a possible implementation manner, the first processing module 21 is specifically configured to:
[0127] determining an adjusted target emotional state according to the emotional state;
[0128] Using a pre-acquired environment image generation model, the environment parameters and the target emotional state are processed to determine the virtual environment image;
[0129] The environmental image generation model is obtained by training a deep learning model based on multiple groups of environmental parameters and the emotional state corresponding to each group of environmental parameters.
[0130] In a possible implementation manner, the second processing module 22 is specifically configured to:
[0131] Generating audio matching the virtual environment image to obtain the background audio;
[0132] Using a pre-acquired environmental parameter determination model, the virtual environment image and the target emotional state are processed to determine the target environmental parameters;
[0133] The environment parameter determination model is obtained by training a deep learning model based on multiple virtual environment images and the emotional state corresponding to each virtual environment image.
[0134] In a possible implementation manner, the third processing module 23 is specifically configured to:
[0135] Performing fusion processing on the virtual environment image and the background audio to generate a virtual reality video;
[0136] The virtual reality video is input into the brain-computer interface worn by the passenger.
[0137] In a possible implementation manner, the third processing module 23 is specifically configured to:
[0138] encoding the virtual reality video to obtain a neural signal;
[0139] The neural signal is input into a brain-computer interface worn by the passenger.
[0140] Figure 5 This is a schematic diagram of the structure of the second embodiment of the environment control device based on emotional state provided by this application. Figure 5 , the emotional state-based environment control device 20 further includes:
[0141] A collection module 24, used for collecting brain wave information of the passenger in real time through the brain-computer interface;
[0142] The recognition module 25 is used to use a pre-acquired emotion recognition model to perform emotion recognition on the brain wave information to determine the passenger's emotional state, which includes a positive state or a negative state;
[0143] The emotion recognition model is obtained by training a deep learning model based on brain wave information and preset emotional state labels.
[0144] In a possible implementation manner, the environmental parameters include the temperature in the cabin, the humidity in the cabin, and the light intensity in the cabin, and the acquisition module 24 is further used for:
[0145] The temperature in the cabin, the humidity in the cabin, and the light intensity in the cabin are respectively acquired according to the temperature sensor, the humidity sensor, and the light sensor arranged in the cabin.
[0146] The emotional state-based environment control device provided in this embodiment can execute the technical solution shown in the above method embodiment. Its implementation principles and beneficial effects are similar and will not be repeated here.
[0147] Figure 6 This is a schematic diagram of the structure of the cockpit domain controller provided in the embodiment of the present application. Figure 6 The cockpit domain controller 30 provided in this embodiment includes: at least one processor 31 and a memory 32. Optionally, the cockpit domain controller 30 also includes a communication component 33. The processor 31, the memory 32 and the communication component 33 are connected via a bus 34.
[0148] In a specific implementation process, at least one processor 31 executes the computer-executable instructions stored in the memory 32, so that at least one processor 31 executes the above method.
[0149] The specific implementation process of the processor 31 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0150] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.
[0151] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
[0152] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.
[0153] The present application provides a vehicle, including a vehicle body and Figure 6 The cockpit domain controller shown is used to implement the emotional state-based environment control method in the above embodiment. The vehicle may be an unmanned vehicle or an unmanned flying car.
[0154] In addition, the present application embodiment can also provide a manned aircraft, including an aircraft body and Figure 6 The cockpit domain controller shown is used to implement the emotional state-based environment control method in the above embodiment.
[0155] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0156] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0157] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0158] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0159] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0161] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0162] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0163] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0164] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. An environment control method based on emotional state, characterized in that: Applied to a cockpit domain controller of a vehicle, the method comprises: If it is identified that the emotional state of the passenger in the cabin is a negative state, determining a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin; Determining, according to the virtual environment image, background audio and target environment parameters that match the virtual environment image; The virtual environment image and the background audio are displayed to the passenger in a virtual reality manner, and the environment in the cabin is controlled according to the target environment parameters.
2. The method according to claim 1, characterized in that The step of determining a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin includes: determining an adjusted target emotional state according to the emotional state; Using a pre-acquired environment image generation model, the environment parameters and the target emotional state are processed to determine the virtual environment image; The environmental image generation model is obtained by training a deep learning model based on multiple groups of environmental parameters and the emotional state corresponding to each group of environmental parameters.
3. The method according to claim 2, characterized in that The determining, according to the virtual environment image, background audio and target environment parameters that match the virtual environment image includes: Generating audio matching the virtual environment image to obtain the background audio; Using a pre-acquired environmental parameter determination model, the virtual environment image and the target emotional state are processed to determine the target environmental parameters; The environment parameter determination model is obtained by training a deep learning model based on multiple virtual environment images and the emotional state corresponding to each virtual environment image.
4. The method according to any one of claims 1 to 3, characterized in that: The step of presenting the virtual environment image and the background audio to the passenger in a virtual reality manner includes: Performing fusion processing on the virtual environment image and the background audio to generate a virtual reality video; The virtual reality video is input into the brain-computer interface worn by the passenger.
5. The method according to claim 4, characterized in that The step of inputting the virtual reality video into the brain-computer interface worn by the passenger comprises: encoding the virtual reality video to obtain a neural signal; The neural signal is input into a brain-computer interface worn by the passenger.
6. The method according to claim 4, characterized in that The method further comprises: collecting brain wave information of the passenger in real time through the brain-computer interface; Using a pre-acquired emotion recognition model, performing emotion recognition on the brain wave information to determine the passenger's emotional state, where the emotional state includes a positive state or a negative state; The emotion recognition model is obtained by training a deep learning model based on brain wave information and preset emotional state labels.
7. The method according to any one of claims 1 to 3, characterized in that: The environmental parameters include cabin temperature, cabin humidity, and cabin light intensity. The method further includes: The temperature in the cabin, the humidity in the cabin, and the light intensity in the cabin are respectively acquired according to the temperature sensor, the humidity sensor, and the light sensor arranged in the cabin.
8. An environment control device based on emotional state, characterized in that: include: A first processing module, for determining a virtual environment image according to the emotional state of the passenger and the environmental parameters in the cabin if it is identified that the emotional state of the passenger in the cabin is a negative state; A second processing module, configured to determine background audio and target environment parameters matching the virtual environment image according to the virtual environment image; The third processing module is used to display the virtual environment image and the background audio to the passenger in a virtual reality manner, and control the environment in the cabin according to the target environment parameters.
9. A cockpit domain controller, characterized in that: include: Processor, memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the environment control method based on emotional state according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the emotional state-based environment control method according to any one of claims 1 to 7.