Vehicle control method, electronic equipment and vehicle

By collecting real-time ambient sound and biometric information from the vehicle, the system comprehensively assesses the driver's emotional state and plays adaptive, soothing music, thus solving the problem of poor emotional relief in existing technologies and improving driving safety.

CN119018078APending Publication Date: 2024-11-26GREAT WALL MOTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411381433.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Current technology cannot effectively determine whether a driver's emotional state has been alleviated, which affects driving safety.

Method used

By collecting real-time environmental sound information and biometric information of passengers, the system comprehensively judges their emotional state and plays appropriate soothing music based on their emotional state, adjusting playback parameters in real time to meet the biometric characteristics of a relaxed state.

Benefits of technology

It improves the accuracy of emotional state assessment, enables effective monitoring of emotional changes among passengers, ensures stress relief and relaxation, and enhances driving safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119018078A_ABST
    Figure CN119018078A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle control method, electronic equipment and a vehicle. The method comprises the steps that environment sound information of the vehicle and biological characteristic information of people in the vehicle are collected in real time; determining the emotional state of the person in the vehicle based on the environment sound information and the biological characteristic information; and in response to the emotional state being a high-voltage state, controlling a sound playing device on the vehicle to play soothing music corresponding to the high-voltage state, and adjusting playing parameters of the soothing music in real time based on the biological characteristic information, so that the biological characteristic information of the person in the vehicle meets the biological characteristic corresponding to the relaxing state. According to the vehicle control method, the electronic equipment and the vehicle, the emotion state of the person in the vehicle can be accurately judged, the pressure releasing and relaxing effect of the person in the vehicle is effectively monitored, the pressure releasing and relaxing effect is effectively ensured, and the driving safety is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle control method, electronic equipment and a vehicle. Background Art

[0002] During vehicle driving, the driver's emotional state has an important impact on driving safety. When the driver cannot deal with emotions and relieve stress in time, it is easy to cause traffic accidents, thereby threatening the safety of people in the car and pedestrians on the road. In the prior art, when some measures are used to relieve the driver's stress, it is impossible to effectively judge whether the driver's emotions have been effectively relieved. Summary of the invention

[0003] In view of this, the purpose of this application is to propose a vehicle control method, an electronic device and a vehicle to accurately judge the emotional state of people on the vehicle, effectively monitor their emotional relief, and thus improve vehicle driving safety.

[0004] Based on the above objectives, the present application provides a vehicle control method, comprising:

[0005] Collect the vehicle's environmental sound information and the biometric information of the people on board in real time;

[0006] Determining the emotional state of the person on the vehicle based on the ambient sound information and the biometric information;

[0007] In response to the emotional state being a high-pressure state, the sound playback device on the vehicle is controlled to play soothing music corresponding to the high-pressure state, and the playback parameters of the soothing music are adjusted in real time based on the biometric information so that the biometric information of the people on the vehicle meets the biometrics corresponding to the relaxation state.

[0008] Optionally, the method further includes:

[0009] Calculate the duration of time that the biometric information of the person on the vehicle satisfies the biometric information corresponding to the relaxation state;

[0010] In response to the duration exceeding a first duration threshold, a sound playback device on the vehicle is controlled to play wake-up music for waking up the person, and the playback parameters of the wake-up music are adjusted in real time based on the biometric information so that the biometric information of the person on the vehicle meets the biometric characteristics corresponding to the awake state.

[0011] Optionally, the playback parameters include tracks and / or volume, and the real-time adjustment of the playback parameters of the soothing music based on the biometric information includes:

[0012] Calculate the continuous playing time of the soothing music;

[0013] In response to the continuous playing time of the soothing music exceeding the second time threshold, the biometric information of the person on the vehicle still does not meet the biometric information corresponding to the relaxation state, and the sound playback device is controlled to switch the track of the soothing music or adjust the volume of the soothing music.

[0014] Optionally, the biometric information includes facial image information and heart rate information, and the determining the emotional state of the person on the vehicle based on the ambient sound information and the biometric information includes:

[0015] Extracting features of the environmental sound information, the facial image information and the heart rate information respectively to obtain corresponding environmental features, facial expression features and heart rate features;

[0016] Performing fusion processing on the facial expression feature and the heart rate feature to obtain a fusion feature;

[0017] Performing classification prediction on the fusion features to obtain a first emotion classification result;

[0018] Classify and predict the environmental features to obtain a second emotion classification result;

[0019] Perform a linear weighted summation on the first emotion classification result and the second emotion classification result to obtain the emotional state of the occupants of the vehicle.

[0020] Optionally, the environmental sound information includes the volume of environmental sound outside the vehicle, and the method further includes:

[0021] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle exceeding a preset decibel, controlling the vehicle windows to close;

[0022] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle not exceeding a preset decibel, the vehicle windows are controlled to open.

[0023] Optionally, it also includes: controlling an odor release device on the vehicle to release an odor that is compatible with the soothing music.

[0024] Optionally, it also includes: controlling the seats of people on the vehicle to adjust from the current angle to a first preset angle range.

[0025] Optionally, it also includes: controlling the air conditioning equipment on the vehicle to adjust from the current temperature to a preset temperature.

[0026] A second aspect of the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0027] A third aspect of the present application also provides a vehicle, which includes the electronic device as described above.

[0028] From the above, it can be seen that the vehicle control method, electronic device and vehicle provided by the present application comprehensively determine the emotional state of the people on the vehicle by collecting the ambient sound information of the vehicle and the biometric information of the people on the vehicle, thereby effectively improving the accuracy of determining the emotional state of the people on the vehicle, realizing effective monitoring of the emotional changes of the people on the vehicle, and effectively avoiding the problem of inaccurate emotional judgment due to single information, and providing a prerequisite for subsequent efficient pressure relief; when it is determined that the emotional state of the people on the vehicle is a high-pressure state that seriously affects driving safety, soothing music is played so that the biometric information of the people on the vehicle meets the biometrics corresponding to the relaxation state, so as to achieve efficient pressure relief and relaxation for the people on the vehicle; during the playback of soothing music, the playback parameters of the soothing music are adjusted in real time based on the biometric information, so that the playback of soothing music is more targeted and more in line with the personal needs of the current people on the vehicle, thereby better improving the pressure relief and relaxation effect; at the same time, by collecting the biometric information of the people on the vehicle in real time, it is possible to effectively monitor whether the people on the vehicle have truly entered a relaxed state, effectively evaluate the pressure relief effect, and achieve effective monitoring of the pressure relief effect. It can be seen that the present application can not only realize the accurate judgment of the emotional state of the people on the vehicle, but also can monitor the stress relief and relaxation of the people on the vehicle in real time, and through the real-time adjustment of the soothing music playback parameters, effectively improve the stress relief and relaxation effect, and ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A schematic flow chart of a vehicle control method according to an embodiment of the present application;

[0031] Figure 2 A schematic diagram of the structure of a vehicle control device according to an embodiment of the present application;

[0032] Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0034] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0035] As described in the background technology, the emotional state of the driver will directly affect the safety of vehicle driving. The pressure in modern life is everywhere, and the driver also has certain pressure during the vehicle driving process. During the vehicle driving process, the driver often needs to deal with various emergencies, such as vehicle tire blowout, breakdown, traffic jam, etc., and may also encounter other people's malicious car-cutting, malicious provocation and other adverse events during driving. When encountering these situations, the emotional state of the driver will change greatly, and this emotional change, if not handled in time, will not only affect the driver's attention, but also further affect the driver's driving operation stability, and appear bad driving behaviors such as driving in anger and speeding. At the same time, for drivers who drive alone for a long time and a long distance, long-term high-concentration driving will make people physically and mentally exhausted, and constantly dealing with various emergencies on the road will also double the pressure of the driver, thereby seriously affecting driving safety. Therefore, if the emotional problems of the driver cannot be alleviated in time, it will bring serious threats to the personal safety of the people on the car and pedestrians on the road.

[0036] In the prior art, vehicles are generally provided with playback devices. When the driver feels the pressure doubled, he can actively choose to use the playback device or other mobile devices (such as mobile phones, music players, etc.) to play soothing music to relieve his emotions. However, this method requires the driver to take the initiative to operate, and usually, when the driver's emotional problems are more serious, he will not take the initiative to take measures to alleviate the emotional problems, but will fall into the difficulty of alleviating the emotional problems, which will lead to more serious emotional problems and form a vicious circle. On this basis, the prior art also takes certain measures to monitor the emotions of the driver, or takes certain reminder measures to remind the driver that he is currently in a bad mood and needs to relieve pressure. However, these technologies are not only difficult to accurately judge the emotional state of the driver, but also unable to accurately judge whether the driver's emotions have been effectively relieved, and the pressure relief and relaxation effect is limited.

[0037] In view of this, the present application provides a vehicle control method to accurately judge the emotional state of the people in the vehicle, effectively monitor the emotional relief, and adjust the playback parameters in real time according to the monitoring results, so as to effectively improve the stress relief and relaxation effect. Figure 1 As shown, including:

[0038] S101, collecting the environmental sound information of the vehicle and the biometric information of the people in the vehicle in real time;

[0039] Specifically, the sound collection device on the vehicle is used to collect the environmental sound information of the vehicle, and the environmental sound information includes the environmental sound information inside the vehicle and the environmental sound information outside the vehicle. Biometric information is the information generated when various organs of the human body perform different activities, and changes with the emotions of the person. For example, when a person is angry, facial expressions will change, heartbeats will speed up, blood pressure and blood sugar will rise, breathing rate will increase, movement amplitude will increase, and so on. Therefore, the biometric information of the people on the vehicle can be collected by using a biometric collection device, and the biometric collection device includes a wearable smart device, a camera device, etc., and has the function of transmitting information by wire or wireless means. When the biometric information of the people on the vehicle is collected by using a camera device, for example, the camera device can be installed in front of the main driver to shoot a real-time video of the people on the vehicle, and then the image frame containing the biometrics of the people on the vehicle is intercepted from the video stream as the biometric information, or the camera device can be used to take photos of the people on the vehicle in real time to obtain the biometric information of the people on the vehicle, and there is no specific limitation. When using wearable smart devices to collect biometric information of people on the vehicle, for example, the wearable smart device can be worn on the wrist, ankle or head of the person on the vehicle to collect the biometric information of the person on the vehicle in real time, such as heart rate, blood pressure, brain wave information, etc.

[0040] S102: Determine the emotional state of the occupants of the vehicle based on the ambient sound information and the biometric information.

[0041] Specifically, during the driving process of the vehicle, the emotions of the people on board will change with the road conditions, driving conditions, etc. In order to achieve accurate stress relief for the people on board, it is necessary to monitor the emotional changes of the people on board in real time. The sounds of the environment outside the car and inside the car will have an impact on the emotional state of the people on board, and can also represent the emotional state of the people. For example, when there is friction or quarrel with people outside the car, being insulted by others, or quarreling with other people in the car, the volume of the sound will increase significantly, the number of sound types will increase, and more negative and abusive bad words will appear in the language. Changes in human emotions can usually be represented by changes in the activities of human organs. For example, when a person is angry, the facial expression may show features such as eyebrows pressed down, wrinkled, upper eyelids lifted, lower eyelids tightened, lips stretched, nose lifted, etc., while heart rate, blood pressure, blood sugar, etc. will show an upward trend, and brain wave frequency will increase; when a person is tired, the facial expression may show features such as eyes opening less, eyebrows drooping, mouth corners drooping, etc., heart rate and brain wave frequency will show a downward trend, etc.; when a person faces greater pressure, the facial expression may show features such as frowning, face tightening, lips tightening, etc., while heart rate, blood pressure, brain wave frequency will increase, etc. Therefore, environmental sound information and biometric information can effectively reflect the emotional changes of people on the bus. However, if only one kind of information is used to judge the corresponding emotional state of a person, the judgment accuracy is not high. For example, when a person on the bus is happy or excited, the environmental sound in the car may also be loud, and blood pressure and blood sugar may also increase; frowning and tightening facial expressions do not necessarily mean anger, but may also be in a state of tension, disgust, etc., or exaggerated facial expressions may be used to communicate with people on the bus, etc. Therefore, the present application integrates multiple types of information that can characterize the emotional changes of people on the vehicle (i.e., environmental sound information and biometric information) to make a comprehensive judgment on the emotional state of people on the vehicle, thereby achieving accurate detection of emotional changes and a higher accuracy rate in judging the emotional state.

[0042] S103. In response to the emotional state being a high-pressure state, controlling the sound playing device on the vehicle to play soothing music corresponding to the high-pressure state, and adjusting the playing parameters of the soothing music in real time based on the biometric information so that the biometric information of the person on the vehicle meets the biometrics corresponding to the relaxation state.

[0043] Specifically, the emotional state includes a high-pressure state, a low-pressure state and a normal state. When the person on the vehicle is in a highly tense, angry, tired or other situation that may seriously affect driving safety, it can be determined as a high-pressure state; when the person on the vehicle is in a relatively tense, angry and tired situation or other situation that may affect driving safety, it can be determined as a low-pressure state; when the person on the vehicle is in a situation that is not easy to affect driving safety, such as peace, happiness, and relaxation, it can be determined as a normal state. When the current emotional state of the person on the vehicle is determined to be a high-pressure state according to the step of S102, it means that the person on the vehicle needs to release pressure at this time, otherwise it will seriously affect driving safety. Therefore, when the emotional state of the person on the vehicle is a high-pressure state, the sound playback device on the vehicle is controlled to play soothing music corresponding to the high-pressure state. Soothing music can be light music, white noise music, peaceful pure music, such as the sound of flowing water, the sound of waves, the sound of birds, the sound of rain, the sound of singing bowls, etc., or it can be a song with a calm tune and a brisk melody. Optionally, it can be determined according to the personal preferences of the person on the vehicle, and there is no specific restriction. Different people have different feelings and preferences for music. When the person on the bus does not like the current soothing music or the current soothing music does not make him feel relaxed, this information will be fed back in the biometric information. Therefore, in this application, the playback parameters of the soothing music are adjusted in real time according to the biometric information, so that the playback of the soothing music can be more targeted and more in line with the personal needs of the current person on the bus, thereby better improving the pressure relief and relaxation effect.

[0044] In the present application, based on steps S101 to S103, the emotional state of the people on the vehicle is comprehensively determined by collecting the ambient sound information of the vehicle and the biometric information of the people on the vehicle, thereby effectively improving the accuracy of the emotional state determination, realizing effective monitoring of the emotional changes of the people on the vehicle, and effectively avoiding problems such as inaccurate emotional determination caused by single information, thereby providing a prerequisite for subsequent efficient stress relief; when it is determined that the emotional state of the people on the vehicle is a high-pressure state that seriously affects driving safety, soothing music is played so that the biometric information of the people on the vehicle meets the biometrics corresponding to the relaxation state, thereby realizing efficient stress relief and relaxation for the people on the vehicle; during the playback of soothing music, the playback parameters of the soothing music are adjusted in real time based on the biometric information, so that the playback of soothing music is more targeted and more in line with the personal needs of the current people on the vehicle, thereby better improving the stress relief and relaxation effect; at the same time, by collecting the biometric information of the people on the vehicle in real time, it is possible to effectively monitor whether the people on the vehicle are truly in a relaxed state, effectively evaluate the stress relief effect, and realize effective monitoring of the stress relief effect. It can be seen that the present application can not only realize the accurate judgment of the emotional state of the people on the vehicle, but also can monitor the stress relief and relaxation of the people on the vehicle in real time, and through the real-time adjustment of the soothing music playback parameters, effectively improve the stress relief and relaxation effect, and ensure driving safety.

[0045] Optionally, the people on the vehicle include the driver, passengers, etc. Since the emotions of people can affect each other, during the driving of the vehicle, not only the driver's emotional changes will affect driving safety, but also the emotional changes of passengers may affect the driver's emotions, thereby affecting driving safety. Therefore, the present application can be used not only to monitor and relieve the emotional changes of the driver, but also to monitor and relieve the emotional changes of passengers.

[0046] When the passengers on the vehicle are effectively relieved of stress and relaxed, they are awakened to ensure driving efficiency. In some embodiments, the method further includes:

[0047] Calculate the duration of time that the biometric information of the person on the vehicle satisfies the biometric information corresponding to the relaxation state;

[0048] In response to the duration exceeding a first duration threshold, a sound playback device on the vehicle is controlled to play wake-up music for waking up the person, and the playback parameters of the wake-up music are adjusted in real time based on the biometric information so that the biometric information of the person on the vehicle meets the biometric characteristics corresponding to the awake state.

[0049] Specifically, when a person is under high pressure, it usually takes a certain amount of time to achieve true relaxation of the brain. Therefore, the duration of the biometric information corresponding to the relaxation state is monitored to determine the relaxation state. The first duration threshold is a preset duration threshold, which can be set by the user or automatically, without specific restrictions. Optionally, the first time threshold can be 10 to 30 minutes, such as 10, 15, 20, 25, 30 minutes, or other time within 10 to 30 minutes, or other values ​​less than 10 minutes or greater than 30 minutes, such as 5, 6, 7, 8, 9, 35, 40, 45, 50 minutes, etc., without specific restrictions. The wake-up music can be cheerful, fast-paced, exciting, high-spirited music that can effectively mobilize the emotions of the personnel, and can be determined according to the personal preferences of the personnel on the vehicle, without specific restrictions. In addition, while playing the wake-up music, a voice reminder can also be played to wake up the personnel on the vehicle. When the people in the car are awakened, they can be guided by voice to perform stretching activities, such as stretching the body or gently moving the limbs, to speed up their awakening.

[0050] In this embodiment, by calculating the duration of the biometric information satisfying the biometric corresponding to the relaxation state, it is determined whether to wake up the person on the vehicle, which can effectively ensure the relaxation effect of the person on the vehicle. When the duration exceeds the first duration threshold, that is, the duration of the person on the vehicle being in the relaxation state exceeds the first duration threshold, it means that the current person is fully relaxed, and at this time, he can be woken up so that the corresponding person is in a sober state, and can continue driving to ensure driving efficiency. At the same time, when the wake-up music played is not enough to wake up the person on the vehicle, the playback parameters of the music can be adjusted in real time, so that the playback of the wake-up music is more targeted and more in line with the personal needs of the current person on the vehicle, thereby better improving the wake-up effect.

[0051] During the playback of soothing music, the playback parameters of the soothing music can be adjusted according to the change of the playback duration and the corresponding biometric information. In some embodiments, the playback parameters include tracks and / or volume, and the real-time adjustment of the playback parameters of the soothing music based on the biometric information includes:

[0052] Calculate the continuous playing time of the soothing music;

[0053] In response to the continuous playing time of the soothing music exceeding the second time threshold, the biometric information of the person on the vehicle still does not meet the biometric information corresponding to the relaxation state, and the sound playback device is controlled to switch the track of the soothing music or adjust the volume of the soothing music.

[0054] Specifically, soothing music corresponds to tracks that make people feel comfortable, calm, peaceful, and relaxed, and the volume is a volume range that makes people feel comfortable. When the continuous playing time of soothing music exceeds the second time threshold, and the biometric information of the people on the bus still does not meet the biometrics corresponding to the relaxation state, it means that the soothing music currently being played is not enough for the people on the bus to relax, and the sound playback device can be further controlled to switch the tracks and adjust the volume of soothing music, such as switching from white noise music to light music, or from light music to calm songs, etc.; for example, when the current volume is low, the volume can be appropriately increased, and when the current volume is high, the volume can be appropriately reduced, etc. In this embodiment, by monitoring the continuous playing of soothing music and the biometric information of the people on the bus, the pressure relief and relaxation effects are effectively monitored. When the pressure relief and relaxation effects are not good, the soothing music tracks or volume are adjusted in time, thereby effectively ensuring the pressure relief and relaxation effects.

[0055] Exemplarily, the types of soothing music include light music, white noise, and pure music, and there are dozens of songs in each type. The currently playing song is a song in the light music type, and the playback volume is 10. When the continuous playback time exceeds the second time threshold, it is detected that the biometric information of the person on the bus still does not meet the biometric characteristics corresponding to the relaxation state. At this time, the track is switched and the volume is adjusted, that is, switching from a song in the currently playing light music type to another song in the current type, or switching from the currently playing light music type to the white noise type or pure music type, and gradually increasing the playback volume from 10 to 13 or decreasing it to 8, so as to better guide the people on the bus into a state of relaxation.

[0056] The second duration threshold is a preset duration threshold, which can be set by the user or automatically, without specific restrictions. Optionally, the second duration threshold can be 5 to 10 minutes, such as 5, 6, 7, 8, 9, 10 minutes, or other values ​​less than 5 minutes or greater than 10 minutes, such as 3, 4, 15, 20, 25 minutes, etc., without specific restrictions.

[0057] The brain is the control center of the entire body and the most important organ of the human body. The key to efficient relaxation and stress relief is to give the brain enough rest. When the brain is comfortable, peaceful, and relaxed, the brain wave frequency will change significantly. Under anxiety and high pressure, the brain wave frequency is usually higher, while when relaxed, the brain wave frequency will gradually decrease, and when entering a state of meditation or sleep, the brain wave frequency will further decrease. Therefore, the brain wave frequency can accurately, quickly, and intuitively reflect whether the current person's brain is truly resting and de-stressing.

[0058] Determine whether to wake up the person on the vehicle through brain wave information. In some embodiments, the biometric information includes brain wave information; when the person is in a relaxed state, the frequency range corresponding to the brain wave information is a first frequency range; when the person is awake, the frequency range corresponding to the brain wave information is a second frequency range;

[0059] Calculating the duration of the first frequency range corresponding to the relaxation state in which the brain wave information of the person on the vehicle meets the requirements;

[0060] In response to the duration exceeding a first duration threshold, a sound playback device on the vehicle is controlled to play wake-up music for waking up the person, and the playback parameters of the wake-up music are adjusted in real time based on the brain wave information so that the brain wave information of the person on the vehicle meets the second frequency range.

[0061] Specifically, the brain wave information of the people on the vehicle is collected using a brain wave collection device. The brain wave collection device includes an electrode unit and a data amplification unit. The electrode unit is used to collect the brain wave signals of the people on the vehicle. The collected brain wave signals are sent to the data amplification unit for amplification to obtain brain wave information. The human brain mainly has five different brain waves: δ waves (delta waves), θ waves (theta waves), α waves (alpha waves), β waves (beta waves) and γ waves (gamma waves). The frequency of δ waves is less than 4Hz, which is the lowest frequency brain wave and appears in deep sleep and some abnormal processes; the frequency of θ waves is 4 to 8Hz, which is called "suggestion waves" because it is commonly found in people's trance or hypnotic state; the frequency of α waves is 8 to 13Hz, which is the main brain wave observed in normal relaxed adults. A good and healthy alpha can promote mental alertness, help mental coordination, and enhance overall relaxation. When alpha is dominant, most people feel relaxed and calm; beta waves, with a frequency of 14 to 30 Hz, are the most common high-frequency waves in people's wakefulness, which have the effect of stimulation and awakening; gamma waves, with a frequency of more than 30 Hz, are involved in healthy cognitive functions and the processing of more complex tasks. When people are in a peaceful and calm mood, or enter a state of meditation, light sleep, or deep sleep, they can all be considered to be in a state of relaxation, with no specific restrictions. When the brain is in a relaxed state, the brain waves may be alpha waves, theta waves, or delta waves, and the frequency is usually below 13 Hz. When in an awake state, the brain waves are beta waves or gamma waves. Therefore, the first frequency range can be set to below 13 Hz, and the second frequency range can be set to greater than 13 Hz. When the brain wave information is in the first frequency range for a certain period of time, it means that the people on the car have been effectively relaxed, and they can be awakened at this time to ensure driving efficiency.

[0062] The sound playback device is controlled to adjust the playback parameters of the soothing music through the brain wave information. In some embodiments, when the continuous playback time of the soothing music exceeds the second time threshold, and the brain wave information of the occupant still does not meet the first frequency range corresponding to the relaxation state, the sound playback device is controlled to switch the soothing music track or adjust the volume of the soothing music.

[0063] In this embodiment, by controlling the sound playback device to switch the track of soothing music or adjust the volume of soothing music through brain wave information, the relaxation of the brains of the people on the bus can be effectively monitored, thereby accurately determining the stress-relieving and relaxing effect of the current soothing music on the people on the bus, thereby effectively ensuring the stress-relieving and relaxing effect.

[0064] During the playing of the wake-up music, the playing parameters of the wake-up music can be adjusted according to the change of the playing time and the corresponding biometric information. In some embodiments, the real-time adjustment of the playing parameters of the wake-up music based on the biometric information includes:

[0065] Calculate the continuous playing time of the wake-up music;

[0066] In response to the continuous playing time of the wake-up music exceeding the third time threshold, the biometric information of the person on the vehicle does not meet the biometric information corresponding to the awake state, and the sound playback device is controlled to switch the track of the wake-up music or adjust the volume of the wake-up music.

[0067] Specifically, when the continuous playing time of the wake-up music exceeds the third time threshold, and the biometric information of the person on the bus still does not meet the biometrics corresponding to the awake state, it means that the currently played wake-up music is not enough to wake up the person on the bus, and the sound playback device can be further controlled to switch the tracks and adjust the volume of the wake-up music, such as switching from fast-paced light music to rock music, etc., and appropriately increase the volume, etc. In this embodiment, by monitoring the continuous playing of the wake-up music and the biometric information of the person on the bus, the wake-up effect is effectively monitored. When the wake-up effect is not good, the wake-up music track or volume is adjusted in time, thereby effectively ensuring the wake-up effect, so that the person on the bus can re-enter the driving state and ensure driving efficiency.

[0068] Exemplarily, the types of wake-up music include rock music and cheerful music, and each type has dozens of songs. The currently playing track is a song in the cheerful music type, and the playback volume is 15. When the continuous playback time exceeds the third time threshold, it is detected that the biometric information of the person in the car still does not meet the biometric characteristics corresponding to the awake state. At this time, the track of the wake-up music is switched and the volume is adjusted, that is, from a song in the currently playing cheerful music type to another song in the current type, or from the currently playing cheerful music type to the rock music type, and the playback volume is gradually increased from 15 to 20, so as to better guide the people in the car into the awake state. Optionally, the playback track of the wake-up music can be the same as or different from the track of the soothing music. When the wake-up music is the same as the track of the soothing music, after the people in the car have finished relaxing, the people in the car are directly awakened by increasing the playback volume of the current music. When the wake-up music and the soothing music are different, after the people on the bus have finished relaxing, the current music is directly switched to the wake-up music to wake up the people on the bus, without any specific restrictions.

[0069] The third duration threshold is a preset duration threshold, which can be set by the user or automatically, without specific limitation. Optionally, the third duration threshold can be 5 to 10 minutes, such as 5, 6, 7, 8, 9, 10 minutes, or other values ​​less than 5 minutes or greater than 10 minutes, such as 3, 4, 15, 20, 25 minutes, etc., without specific limitation.

[0070] The sound playback device can be controlled by brain wave information to adjust the playback parameters of the wake-up music. In some embodiments, when the continuous playback time of the wake-up music exceeds the third time threshold, the brain wave information of the person on the vehicle still does not meet the second frequency range corresponding to the awake state, and the sound playback device is controlled to switch the track of the wake-up music or adjust the volume of the wake-up music. In this embodiment, by controlling the sound playback device to switch the track of the wake-up music or adjust the volume of the wake-up music through brain wave information, the awakeness of the brain of the person on the vehicle can be effectively monitored, so as to accurately determine the awakening effect of the current wake-up music on the person on the vehicle, thereby effectively ensuring the awakening effect and improving driving efficiency.

[0071] When a person's emotions change, their facial expressions will subconsciously change slightly, and this change is usually rapid and intuitive. There is also a close connection between the human heart rate and emotions. Different emotional states will have different effects on the heart rate. Excitement, tension, and high pressure will increase the heart rate, while calmness and relaxation will slow down the heart rate. Both heart rate and facial expressions can intuitively and quickly reflect the emotional changes of people on the bus. Therefore, facial image information and heart rate information can be used to comprehensively judge the emotional state of people on the bus. In some embodiments, the biometric information includes facial image information and heart rate information, and the determination of the emotional state of people on the bus based on the ambient sound information and the biometric information includes:

[0072] Extracting features of the environmental sound information, the facial image information and the heart rate information respectively to obtain corresponding environmental features, facial expression features and heart rate features;

[0073] Performing fusion processing on the facial expression feature and the heart rate feature to obtain a fusion feature;

[0074] Performing classification prediction on the fusion features to obtain a first emotion classification result;

[0075] Classify and predict the environmental features to obtain a second emotion classification result;

[0076] Perform a linear weighted summation on the first emotion classification result and the second emotion classification result to obtain the emotional state of the occupants of the vehicle.

[0077] Multimodal fusion technology is a technology that predicts results by integrating information from two or more modalities, which can improve information processing and understanding capabilities. Modality can be used to characterize the source or form of information, that is, multimodal fusion technology can fuse information from different sources to achieve the complementation and expansion of multiple information. Multimodal fusion technology can be divided into three types according to its specific implementation process, namely, feature-level fusion type, decision-level fusion type and hybrid-level fusion type. Feature-level fusion type, also known as early fusion, means that after the extraction of different modal features, the extracted features are immediately connected into a single high-dimensional feature vector, and then the single high-dimensional feature vector is classified and decided, with high decision efficiency and fast operation speed. Decision-level fusion type, also known as late fusion, is to obtain the decision results based on each modality, and then perform an integration step on these decision results by applying multiple prediction class label algebraic combination rules (for example, maximum value, minimum value, sum, average value, etc.) to obtain the final decision result. When the correlation between the modalities is low, the decision efficiency is higher and the accuracy is higher. The hybrid-level fusion type is a combination of feature-level fusion and decision-level fusion. It combines the output results of feature-level fusion and single modality prediction to make the final result prediction. It has a wider range of applications and can handle more types of models.

[0078] Specifically, in this embodiment, a multimodal fusion technology of a hybrid fusion type is used to extract and fuse the vehicle environment information, facial image information and heart rate information to obtain the emotional state of the person on the vehicle. That is, after obtaining the environmental features, facial expression features and heart rate features, the facial expression features and heart rate features are first fused to obtain the fusion features, and then the fusion features and the environmental features are classified and predicted to obtain the first emotion classification result and the second emotion classification result. Finally, the first emotion classification result and the second emotion classification result are linearly weighted summed to obtain the final emotional state. In this embodiment, a multimodal fusion technology of a hybrid fusion type is used to achieve efficient fusion of multiple information, effectively improving the efficiency of information processing; for the facial expression features and heart rate features that belong to the biological features of the person on the vehicle, the correlation between them is higher, so after fusing them and then performing classification prediction, the correlation between the facial expression features and the heart rate features can be effectively explored; finally, the classification prediction results based on the fusion features (i.e., the first emotion classification result) and the classification prediction results based on the environmental features (i.e., the second emotion classification result) are used to perform linear weighted summation to accurately judge the emotional state of the person on the vehicle.

[0079] In addition, in addition to the hybrid-level fusion type of multimodal fusion technology, the feature-level fusion type of multimodal fusion technology can also be applied to the present application. In some embodiments, when the feature-level fusion type of multimodal fusion technology is applied to extract and fuse the vehicle environment information, facial image information, and heart rate information, the specific processing process is as follows:

[0080] Extracting features of vehicle environment information, facial image information and heart rate information respectively, and obtaining corresponding environment features, facial expression features and heart rate features;

[0081] fusing the environmental features, the facial expression features, and the heart rate features to obtain fused features;

[0082] The fused features are classified and predicted to obtain the emotional state of the people on the bus.

[0083] In this implementation, after obtaining the features of various types of information, they are directly fused. On the basis of realizing the fusion of multiple types of information, the processing efficiency is higher and the judgment speed is faster, and the emotional state of the people on the car can be judged more quickly.

[0084] The multimodal fusion technology of the decision-level fusion type can also be applied to the present application. In some embodiments, when the multimodal fusion technology of the decision-level fusion type is applied to perform feature extraction and fusion processing on vehicle environment information, facial image information, and heart rate information, the specific processing process is as follows:

[0085] Extract the features of vehicle environment information, facial image information, and heart rate information respectively, and obtain corresponding environment features, facial expression features, and heart rate features;

[0086] Classify and predict the environmental features, the facial expression features, and the heart rate features respectively to obtain a third emotion classification result, a fourth emotion classification result, and a fifth emotion classification result;

[0087] The third emotion classification result, the fourth emotion classification result and the fifth emotion classification result are linearly weighted summed to obtain a final fusion result, that is, the emotional state of the people on the bus.

[0088] In this implementation, after obtaining the characteristics of each type of information, each type of feature is classified and predicted respectively, and then multiple classification prediction results are linearly weighted, which can effectively explore the correlation between various types of information, effectively improve the accuracy of judging emotional states, and thus better achieve stress relief and relaxation effects.

[0089] When the sound playing device is controlled to play soothing music, the opening and closing of the vehicle windows can be further controlled based on the sound level of the vehicle's external environment. In some embodiments, the environmental sound information includes the volume of the vehicle's external environmental sound, and the method further includes:

[0090] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle exceeding a preset decibel, controlling the vehicle windows to close;

[0091] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle not exceeding a preset decibel, the vehicle windows are controlled to open.

[0092] Specifically, the environmental sound information includes the in-vehicle environmental sound information and the out-vehicle environmental sound information. The in-vehicle environmental sound information includes the volume of the in-vehicle environmental sound, the voice content of the in-vehicle environmental sound, etc., and the out-vehicle environmental sound information includes the volume of the in-vehicle environmental sound, the voice content of the in-vehicle environmental sound, etc. When the emotional state of the occupants is in a high-pressure state and they need to play soothing music to relieve stress, the sound outside the vehicle will affect the decompression effect of the occupants. Therefore, when the volume of the external environmental sound of the vehicle exceeds the preset decibel, it means that the current external environmental sound interference of the vehicle is too large. The windows can be controlled to close to ensure that the occupants can be in a quiet environment, so as to better enter a relaxed state for decompression and relaxation. When the volume of the external environmental sound of the vehicle does not exceed the preset decibel, it means that the current external environmental sound interference of the vehicle is too small or even no interference. The windows can be controlled to open, the air circulation effect is good, and the occupants feel more comfortable, which can help them to relieve stress and relax better.

[0093] Optionally, if the current vehicle has a sunroof, the sunroof and windows can be controlled to close or open together while the windows are controlled to close or open, so that the air circulation in the vehicle is better. The preset decibel is determined according to the comfort of human ears, and can be 50 decibels. It can be automatically set by the user or set to other values ​​according to user habits. For example, if the user is sensitive to sound, it can be set to 40 or 45 decibels, etc. There is no specific restriction.

[0094] While playing soothing music, a certain smell can be released in the vehicle to assist the vehicle personnel to enter a relaxed state more quickly. In some embodiments, the method further includes: controlling the smell release device on the vehicle to release a smell that is compatible with the soothing music.

[0095] Specifically, the correspondence between smells and soothing music can be preset in advance. One smell can correspond to multiple soothing music, one soothing music can correspond to multiple smells, one soothing music can correspond to only one smell, and one smell can correspond to only one soothing music, without any specific limitation. The sense of smell has an important influence on positive emotions and mentality. Natural smells and specific fragrances can improve mood and relieve stress. In this embodiment, when soothing music is played, the smell release device releases the corresponding smell. For example, when the sound of the waves is played, the smell with the smell of sea salt can be released. When the sound of the wind is played, the smell with the fragrance of flowers and grass can be released, which can stimulate the sense of smell of the people on the car, thereby helping the people on the car to better enter a state of relaxation and achieve a better pressure relief effect.

[0096] While playing soothing music, the seat angle of the person in the car can be adjusted to make the body in a comfortable state. In some embodiments, it also includes: controlling the seat of the person in the car to adjust from the current angle to a first preset angle range.

[0097] Specifically, the first preset angle range is a seat angle range that makes the human body feel comfortable, which can be set by the user or automatically set according to the user's personal habits, etc., without specific restrictions. Optionally, the first preset angle range can be 100 to 110 degrees, such as 100, 102, 105, 108, 110 degrees, etc., can be other values ​​of 100 to 110 degrees, or can be other values ​​less than 100 degrees or greater than 110 degrees, such as 95, 98, 115, 120, 125, 130 degrees, etc., without specific restrictions. In this embodiment, when the emotional state of the person on the vehicle is a high-pressure state and needs to relax and release pressure, the seat angle can be adjusted to the first preset angle while playing soothing music, so that the body of the person on the vehicle feels more comfortable, which can better help the person on the vehicle to relax, thereby improving the pressure relief effect.

[0098] When the user has finished relieving stress and needs to re-enter the awake state to drive the vehicle, while playing the wake-up music, the seats of the people in the car can be controlled to adjust from the first preset angle range to the second preset angle range. Specifically, the second preset angle range is a seat angle range suitable for human body work, which can be customized by the user or automatically set according to the user's personal habits, etc., without specific restrictions. Optionally, the second preset angle range is 95 to 100 degrees, such as 95, 96, 97, 98, 99, 100 degrees, etc., can be other values ​​of 95 to 100 degrees, or can be other values ​​less than 95 degrees or greater than 100 degrees, such as 93, 94, 101, 102, 103, 104, 105, 110 degrees, etc., without specific restrictions.

[0099] The human body has a comfortable temperature range, and when in this temperature range, the human body's subjective feelings, emotional state, etc. are all good. In some embodiments, it also includes: controlling the air conditioning equipment on the vehicle to adjust from the current temperature to the preset temperature.

[0100] Specifically, the preset temperature is a temperature that makes the human body feel comfortable, which can be set by the user or automatically according to the user's personal habits, etc., without specific restrictions. Optionally, the preset temperature range can be 22 to 26 degrees, such as 22, 23, 24, 25, 26 degrees, etc., can be other values ​​within the range of 22 to 26 degrees, or can be other values ​​less than 22 degrees or greater than 26 degrees, such as 18, 19, 20, 21, 27, 28 degrees, etc., without specific restrictions. In this embodiment, when the emotional state of the people in the car is in a high-pressure state and needs to relax and release pressure, the air-conditioning equipment can be used to adjust the temperature in the vehicle to the preset temperature while playing soothing music, so that the people in the car feel more comfortable, which can better help the people in the car to relax, thereby improving the pressure relief effect.

[0101] When the people on the vehicle need to relieve pressure and relax, the people on the vehicle can be guided to perform parking operations. In some embodiments, in response to the emotional state being a high-pressure state, the sound playback device on the vehicle is controlled to play a parking reminder voice. Specifically, when the people on the vehicle are in a high-pressure state and need to relieve pressure and relax, it can be determined by voice whether the people on the vehicle need to relieve pressure and relax. After getting a definite answer, the parking reminder voice can be played to guide the people on the vehicle to park the vehicle in a safe place, and then start playing soothing music, releasing odors and other operations related to pressure release, so as to ensure the safety of the vehicle and the people on the vehicle. Optionally, after getting a definite answer that pressure relief and relaxation are needed, the vehicle can also be operated to automatically drive to a safe area to park according to the current environmental conditions. Since the people on the vehicle are in a high-pressure state at this time, continuing to operate the vehicle may endanger driving safety. Therefore, automatic parking can be used instead of manual operation by the people on the vehicle, thereby improving the convenience of operation and further improving driving safety.

[0102] When the people on the vehicle are still in a low-pressure state, in order to avoid further deterioration of their emotional problems, some operations can be performed in advance to gradually stabilize the emotions of the people on the vehicle. In some embodiments, in response to the emotional state being a low-pressure state, a voice guidance device on the vehicle is used to play a guidance voice to guide the people on the vehicle to soothe their emotions. When the people on the vehicle are in a low-pressure state, it indicates that although the people on the vehicle have certain emotional problems, the problems are not serious and will not seriously affect driving safety for the time being. At this time, a guidance voice guidance voice can be played to guide the people on the vehicle to soothe their emotions, such as playing deep breathing, spiritual comfort quotations, etc., to help the people on the vehicle relax effectively during driving and calm down their emotions as soon as possible.

[0103] In order to better guide the passengers to relax, the concentration of the odor can be linked with the volume of the soothing music. In some embodiments, the concentration of the odor released by the odor release device increases as the volume of the soothing music increases.

[0104] In this embodiment, the smell is linked with the soothing music, so that the people on the car can immerse themselves in the soothing music more quickly, thereby accelerating their entry into a relaxed state and improving the efficiency of stress relief and relaxation.

[0105] In order to achieve a better relaxation effect, the brain wave information of the occupants can be linked with soothing music, the vehicle window status, etc. In some embodiments, in response to the brain wave information of the occupants meeting the third frequency range corresponding to the sleep state, the sound playback device is controlled to adjust the volume of the soothing music to the minimum, the vehicle windows are controlled to be closed, the odor release device is controlled to release a sleep-inducing odor, and the air conditioning device is controlled to adjust the current temperature to the sleep temperature.

[0106] Specifically, when people are in a high-pressure state such as high pressure and fatigue that seriously affects driving safety, if the brain can be completely rested for a period of time (i.e., sleep for a period of time), it can play a better role in relieving pressure. When playing soothing music to relieve pressure and relax the people on the car, the people on the car are immersed in the current atmosphere of soothing music and smell, and can gradually enter a sleep state. When the people on the car enter a sleep state, adjust the volume of the soothing music to the minimum and close the vehicle windows to create a quiet environment, release the smell of sleep, and improve their sleep quality. At the same time, adjust the temperature of the vehicle environment to a comfortable sleeping temperature during sleep, such as 24 to 26 degrees, so that the people on the car can stay in a sleeping state for a period of time, so as to get enough rest, so as to improve the pressure relief effect and help the people on the car better relieve their emotional conditions. Optionally, the third frequency range is less than 8Hz, such as 8.5, 7.5, 7, 6.5, 6Hz, etc., without specific restrictions; the sleeping temperature can be 24 to 26 degrees, such as 24, 24.5, 25, 25.5, 26, etc., without specific restrictions.

[0107] In some embodiments, the volume of the soothing music and the concentration of the smell are proportional to the frequency of the brain wave information.

[0108] Specifically, the volume of soothing music and the concentration of scent increase as the frequency of brain wave information increases, and decrease as the frequency of brain wave information decreases. When the frequency of brain wave information increases, it means that the emotions of the people on the bus have not been effectively relieved. In order to appease the emotions of the people on the bus as soon as possible, the volume of soothing music and the concentration of scent can be increased to distract the attention of the people on the bus and help them calm down as soon as possible, thereby improving the efficiency and effect of stress relief and relaxation. When the frequency of brain wave information decreases, it means that the emotions of the people on the bus are gradually calmed down and they enter a relaxed state or a sleeping state. At this time, the volume of soothing music and the concentration of scent can be reduced to avoid interfering with the relaxation and sleep of the people on the bus, thereby ensuring the stress relief and relaxation effect.

[0109] The vehicle control method, electronic device and vehicle provided by the present application collect environmental sound information, facial image information, heart rate information and brain wave information in real time, and use multimodal fusion technology to extract and fuse the features of the environmental sound information, facial image information and heart rate information to obtain the emotional state of the people in the car; when the emotional state is a high-pressure state, it means that the emotional pressure of the people in the car is relatively large, which will seriously affect driving safety. Therefore, soothing music can be played, smells suitable for soothing music can be released, the seats can be adjusted from the current angle to a comfortable first preset angle range, and the temperature of the vehicle can be adjusted to a comfortable preset temperature, so as to help the people in the car relieve stress and relax; in addition, it is also possible to determine whether to open the windows according to the volume of the external environmental sound of the vehicle, so as to better help the people in the car to quickly relax; in the process of relieving stress and relaxing the people in the car, the brain wave information of the people in the car is monitored to determine whether they are The decompression and relaxation are truly carried out to ensure the decompression and relaxation effect; when the continuous playing time of the soothing music exceeds the second time threshold, the brain wave information of the people on the car still does not meet the first frequency range, which means that the current music cannot effectively guide the people on the car to enter a relaxed state, so the tracks of the soothing music can be switched or the volume of the soothing music can be adjusted within the first preset condition, and the decompression and relaxation effect is better; when the continuous playing time of the brain wave information meeting the first frequency range exceeds the first time threshold, it means that the people on the car have effectively relaxed, and at this time, the wake-up music can be played to wake up the people on the car, so that the people on the car can continue to drive and drive, ensuring driving efficiency; when the continuous playing time of the wake-up music exceeds the third time threshold, it means that the current music cannot effectively guide the people on the car to enter a sober state, so the tracks of the wake-up music can be switched or the volume of the wake-up music can be adjusted to achieve efficient awakening of the people on the car and ensure driving efficiency. The vehicle control method, electronic equipment and vehicle of the present application can realize accurate judgment of the emotional state of the people on the car, effectively monitor the decompression and relaxation effect of the people on the car, ensure the decompression and relaxation effect, and improve driving safety.

[0110] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0111] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a vehicle control device, referring to Figure 2 , the vehicle control device comprises:

[0113] The acquisition module 201 is used to collect the environmental sound information of the vehicle and the biometric information of the passengers in the vehicle in real time;

[0114] An emotion determination module 202, for determining the emotional state of the occupants of the vehicle based on the ambient sound information and the biometric information;

[0115] The first emotion adjustment monitoring module 203 is used to control the sound playback device on the vehicle to play soothing music corresponding to the high-pressure state in response to the emotional state being a high-pressure state, and adjust the playback parameters of the soothing music in real time based on the biometric information so that the biometric information of the people on the vehicle meets the biometrics corresponding to the relaxation state.

[0116] In some embodiments, it also includes:

[0117] A calculation module, used for calculating the duration of time that the biometric information of the person on the vehicle satisfies the biometric information corresponding to the relaxation state;

[0118] The second emotion adjustment monitoring module is used to control the sound playing device on the vehicle to play wake-up music for waking up the person in response to the duration exceeding the first duration threshold, and adjust the playing parameters of the wake-up music in real time based on the biometric information so that the biometric information of the person on the vehicle meets the biometric characteristics corresponding to the awake state.

[0119] In some embodiments, the playback parameters include tracks and / or volume, and the first emotion adjustment monitoring module 203 further includes:

[0120] A calculation unit, used for calculating the continuous playing time of the soothing music;

[0121] A switching unit is used to control the sound playback device to switch the track of the soothing music or adjust the volume of the soothing music in response to the continuous playback time of the soothing music exceeding a second time threshold and the biometric information of the person on the vehicle still does not meet the biometric information corresponding to the relaxation state.

[0122] In some embodiments, the biometric information includes facial image information and heart rate information, and the emotion determination module 202 is further configured to:

[0123] Extracting features of the environmental sound information, the facial image information and the heart rate information respectively to obtain corresponding environmental features, facial expression features and heart rate features;

[0124] Performing fusion processing on the facial expression feature and the heart rate feature to obtain a fusion feature;

[0125] Performing classification prediction on the fusion features to obtain a first emotion classification result;

[0126] Classify and predict the environmental features to obtain a second emotion classification result;

[0127] Perform a linear weighted summation on the first emotion classification result and the second emotion classification result to obtain the emotional state of the occupants of the vehicle.

[0128] In some embodiments, the environmental sound information includes the volume of environmental sound outside the vehicle, and the first emotion adjustment monitoring module 203 is further used to:

[0129] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle exceeding a preset decibel, controlling the vehicle windows to close;

[0130] In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle not exceeding a preset decibel, the vehicle windows are controlled to open.

[0131] In some embodiments, the first emotion adjustment monitoring module 203 is further used to control an odor release device on the vehicle to release an odor that is compatible with the soothing music.

[0132] In some embodiments, the first emotion adjustment monitoring module 203 is further used to control the seat of the person in the vehicle to adjust from the current angle to a first preset angle range.

[0133] In some embodiments, the first emotion adjustment monitoring module 203 is also used to control the air conditioning equipment on the vehicle to adjust from the current temperature to a preset temperature.

[0134] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0135] The device of the above embodiment is used to implement the corresponding vehicle control method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0136] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle control method described in any of the above embodiments is implemented.

[0137] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0138] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0139] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0140] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0141] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0142] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0143] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0144] The electronic device of the above embodiment is used to implement the corresponding vehicle control method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0145] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the vehicle control method described in any of the above embodiments.

[0146] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0147] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the vehicle control method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0148] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the vehicle control method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0149] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0150] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0151] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0152] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0153] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0154] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0155] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0156] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: include: Collect the vehicle's environmental sound information and the biometric information of the people on board in real time; Determining the emotional state of the person on the vehicle based on the ambient sound information and the biometric information; In response to the emotional state being a high-pressure state, the sound playback device on the vehicle is controlled to play soothing music corresponding to the high-pressure state, and the playback parameters of the soothing music are adjusted in real time based on the biometric information so that the biometric information of the people on the vehicle meets the biometrics corresponding to the relaxation state.

2. A vehicle control method according to claim 1, characterized in that: The method further comprises: Calculate the duration of time that the biometric information of the person on the vehicle satisfies the biometric information corresponding to the relaxation state; In response to the duration exceeding a first duration threshold, a sound playback device on the vehicle is controlled to play wake-up music for waking up the person, and the playback parameters of the wake-up music are adjusted in real time based on the biometric information so that the biometric information of the person on the vehicle meets the biometric characteristics corresponding to the awake state.

3. A vehicle control method according to claim 1, characterized in that: The playing parameters include tracks and / or volume, and the real-time adjustment of the playing parameters of the soothing music based on the biometric information includes: Calculate the continuous playing time of the soothing music; In response to the continuous playing time of the soothing music exceeding the second time threshold, the biometric information of the person on the vehicle still does not meet the biometric information corresponding to the relaxation state, and the sound playback device is controlled to switch the track of the soothing music or adjust the volume of the soothing music.

4. A vehicle control method according to claim 1, characterized in that: The biometric information includes facial image information and heart rate information, and determining the emotional state of the person on the vehicle based on the ambient sound information and the biometric information includes: Extracting features of the environmental sound information, the facial image information and the heart rate information respectively to obtain corresponding environmental features, facial expression features and heart rate features; Performing fusion processing on the facial expression feature and the heart rate feature to obtain a fusion feature; Performing classification prediction on the fusion features to obtain a first emotion classification result; Classify and predict the environmental features to obtain a second emotion classification result; Perform a linear weighted summation on the first emotion classification result and the second emotion classification result to obtain the emotional state of the occupants of the vehicle.

5. A vehicle control method according to claim 1, characterized in that: The environmental sound information includes the volume of the environmental sound outside the vehicle, and the method further includes: In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle exceeding a preset decibel, controlling the vehicle windows to close; In response to the emotional state being a high-pressure state and the volume of the ambient sound outside the vehicle not exceeding a preset decibel, the vehicle windows are controlled to open.

6. A vehicle control method according to claim 1, characterized in that: Also includes: Controlling the scent release device on the vehicle to release a scent that matches the soothing music.

7. A vehicle control method according to claim 1, characterized in that: Also includes: Control the seats of the people in the vehicle to adjust from the current angle to the first preset angle range.

8. A vehicle control method according to claim 1, characterized in that: Also includes: Controls the vehicle's air conditioning system from the current temperature to a preset temperature.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A vehicle, characterized in that: The vehicle comprises the electronic device as claimed in claim 9.