Vehicle-mounted environment intelligent control method based on emotion recognition and related equipment

Abstract

The invention provides a vehicle-mounted environment intelligent control method based on emotion recognition and related equipment, and relates to the technical field of intelligent control. The method comprises the following steps: acquiring face image information of a driver based on an in-vehicle camera, inputting the face image information into an emotion recognition model trained by a large number of face images with emotion tags, and accurately judging the emotion of the user; and according to a preset equipment parameter adjusting relation table, intelligently adjusting and controlling the operation parameters of the target equipment in the vehicle according to the emotion result. Through the series of processes, conversion from emotion recognition to vehicle-mounted environment intelligent adaptation is achieved, emotion fluctuation of a driver can be sensed in real time, operation parameters of equipment in a vehicle can be automatically adjusted, the emotion of the driver is improved, and vehicle-mounted environment control tends to be more intelligent and humanized.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to an in-vehicle environment intelligent control method and related equipment based on emotion recognition. Background Art

[0002] In today's era of booming automobile industry, cars have become an indispensable means of transportation in people's daily lives. As consumers' requirements for automobile quality continue to increase, the optimization of driving experience has become a key area of ​​concern for major automobile manufacturers and related companies. Among them, the driver's emotional state during driving has a key impact on driving safety, comfort and overall driving experience. Positive emotions help improve the driver's reaction speed and decision-making ability, while negative emotions may lead to safety hazards such as inattention. Therefore, how to effectively pay attention to and improve the driver's emotional state has become an urgent problem to be solved.

[0003] Existing technical means mainly monitor the driver's physiological state by installing various sensors in the car, and then indirectly judge his emotional state. Common examples include equipping the steering wheel with a high-precision heart rate sensor to obtain heart rate data by sensing the driver's hand pulse; installing a high-sensitivity skin conductivity sensor on the seat back to detect changes in electrical signals on the body surface. The data collected by these sensors is transmitted to the on-board computer system for analysis. Taking the normal heart rate range of 60-100 beats per minute as an example, when the heart rate sensor detects that the driver's heart rate continues to deviate from this range, and the skin conductivity sensor detects abnormal fluctuations, it is determined that the driver may be in a state of fatigue or stress. Once an abnormality is determined, the on-board system immediately activates the alarm mechanism, sounds an alarm, and displays a warning icon on the dashboard.

[0004] However, the existing technology has many drawbacks. The contact design of the sensor brings a sense of restraint to the driver, affecting the driving operation. For the in-vehicle environment control, the fixed preset mode is difficult to meet the personalized needs of the driver under different emotions. Summary of the invention

[0005] The present application provides an in-vehicle environment intelligent control method and related equipment based on emotion recognition, which are used to accurately perceive the driver's emotional state and intelligently adjust the operating parameters of various devices in the vehicle accordingly, thereby creating an in-vehicle environment atmosphere that matches the driver's emotions, effectively alleviating the driver's negative emotions, and improving their comfort, concentration and safety during driving.

[0006] In the first aspect, the present application provides an intelligent control method for an in-vehicle environment based on emotion recognition, which is applied to a server, and the method includes: obtaining facial image information of a driver in a vehicle through a camera in the vehicle; inputting the facial image information into a preset emotion recognition model to obtain a user emotion judgment result, and the emotion recognition model is trained in advance based on multiple facial images with emotion labels; adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and a preset device parameter adjustment relationship table, and the device parameter adjustment relationship table stores device control parameters corresponding to different emotional states.

[0007] By adopting the above technical solution, the driver's facial image information is collected with the help of the in-car camera to provide a data basis for emotion recognition. The emotion recognition model is trained based on a large number of facial images with emotion labels, has accurate analysis capabilities, and can quickly and accurately judge the driver's emotions. After that, the operating parameters of the target equipment in the car are adjusted according to the equipment parameter adjustment relationship table, realizing the intelligent linkage between emotion and equipment control. This method can automatically optimize the in-car environment according to the driver's real-time emotional state, such as adjusting the temperature and music when the driver is irritable, effectively alleviating the driver's bad emotions, improving their driving comfort and concentration, thereby improving driving safety and the overall driving experience, and making the in-vehicle environment control more intelligent and humane.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of adjusting the operating parameters of the corresponding target device in the car according to the user emotion judgment result and the preset device parameter adjustment relationship table specifically includes: if it is determined that the user emotion judgment result is a negative emotion, then determining the target device operating parameters according to the device parameter adjustment relationship table, and the target device operating parameters at least include controlling the color of the interior ambient light to the target color and controlling the aromatherapy device to emit a set fragrance.

[0009] By adopting the above technical solutions, the color changes of the ambient lights and the fragrance emitted by the aromatherapy can directly stimulate the driver's visual and olfactory senses, soothe his negative emotions, create a comfortable driving atmosphere, reduce the potential impact of negative emotions on driving safety, and improve driving experience and safety.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of obtaining facial image information of the driver in the car through a camera inside the car, it also includes: comparing the facial image information with a facial image library when the driver is in a fatigue state to determine whether the driver is in a fatigue state; if in a fatigue state, controlling the interior ambient light of the car to emit red light to warn the driver.

[0011] By adopting the above technical solution, once it is determined that the driver is in a fatigue state, the interior ambient light is controlled to emit red light. The red light is eye-catching and has a warning effect. It directly acts on the driver's visual perception, making him quickly aware of his fatigue state, thereby prompting the driver to take corresponding measures, such as resting or adjusting the driving state, effectively reducing the risk of traffic accidents caused by fatigue driving and ensuring driving safety.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, it also includes: receiving scene mode selection information sent by the user through the display end, and determining new operating parameters of the target device under the target scene mode according to the scene mode selection information; controlling the target device to operate according to the new operating parameters.

[0013] By adopting the above technical solutions, users can flexibly select scene modes according to their own needs. This interactive method makes the in-vehicle environment control more in line with the personalized needs of users. Different device operating parameters in different scene modes can create a diverse in-vehicle environment experience for users and improve user satisfaction and controllability of the in-vehicle system.

[0014] In combination with some embodiments of the first aspect, in some embodiments, after the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, it also includes: determining the use environment when the user uses the vehicle through the camera; when it is determined that the use environment is at night, determining through the camera whether there is any user getting on or off the vehicle; if there is any getting on or off the vehicle action, controlling the ambient light on the side corresponding to the getting on or off the vehicle action to light up.

[0015] By adopting the above technical solution, the environmental monitoring function of the camera is coordinated with the ambient light control. When the light is dim at night, the ambient light on the corresponding side can light up to provide local lighting when getting on and off the vehicle, making it easier for users to see the surrounding environment and avoid accidents such as bumps caused by insufficient light, thereby improving the convenience and safety of getting on and off the vehicle, and also improving the humanization of vehicle use.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of adjusting the operating parameters of the corresponding target device in the car according to the user emotion judgment result and the preset device parameter adjustment relationship table, it also includes: when the camera detects that there is a child in the car, combining with the sound sensor to detect whether there is crying sound information of the child; if there is crying sound information, determining the position of the child in the car through the camera; controlling the atmosphere light in the area corresponding to the child's position to adjust to a color gradient mode to soothe the child's emotions.

[0017] By adopting the above technical solution, the camera and sound sensor work together to accurately locate crying children, and the colorful gradient ambient light attracts children's attention through visual stimulation, diverts their crying emotions, helps to soothe children, and reduces the interference of children's crying on driving, allowing drivers to focus more on driving, improving driving safety and the comfort of the in-car environment.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, it also includes: receiving the user's adjustment result feedback information on the target device through the display end; determining the user's personalized lighting mode based on the adjustment result feedback information; and updating the personalized lighting mode to the device parameter adjustment relationship table.

[0019] By adopting the above technical solution, user feedback information reflects their personalized preferences, and this information is used to generate personalized lighting patterns and update the relationship table. The subsequent system can automatically adjust the lighting according to this pattern. This process realizes the adaptive learning of the vehicle system to the personalized needs of users, so that the in-vehicle lighting environment can better meet the unique needs of users and enhance the user's favorability and dependence on the vehicle environment control system.

[0020] In a second aspect, the present application provides a server, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the server to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a server, causes the server to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product, which, when executed on a server, enables the server to execute the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Due to the technical means of collecting facial images based on cameras and judging emotions with the help of pre-trained emotion recognition models to drive equipment parameter adjustment, the technical problem that the vehicle environment cannot be intelligently adapted according to the driver's emotions in the existing technology is effectively solved, thereby achieving the technical effect of accurately sensing the driver's emotions and automatically optimizing the in-vehicle environment, improving driving comfort, concentration and safety.

[0024] 2. The use of cameras and sound sensors to collaboratively detect the child's status and soothe the child through ambient light visual stimulation effectively solves the technical problem in the existing technology that it is difficult to deal with the crying of children in the car interfering with driving, thereby achieving the technical effect of timely soothing the child's emotions, reducing driving interference, and ensuring driving safety and comfort in the car.

[0025] 3. Due to the technical means of receiving user feedback to generate personalized lighting modes and updating the equipment parameter adjustment relationship table, the technical problem that the vehicle-mounted equipment in the existing technology is difficult to meet the personalized needs of users is effectively solved, thereby achieving the technical effect of adaptively learning user preferences of the vehicle-mounted system and enhancing users' favorability and dependence on the vehicle-mounted environment control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a method for intelligently controlling an in-vehicle environment based on emotion recognition in an embodiment of the present application; Figure 2 is another flow chart of the vehicle environment intelligent control method based on emotion recognition in an embodiment of the present application; Figure 3 It is a schematic diagram of a physical device structure of a server in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the following describes the process of the method provided by this implementation. Figure 1 , which is a flow chart of an intelligent control method of an in-vehicle environment based on emotion recognition in an embodiment of the present application.

[0030] S101, obtaining facial image information of the driver in the car through a camera in the car; In the process of modern automobile intelligence, obtaining the driver's facial image information through the in-car camera is the key first step to realize emotion recognition and then intelligently control the in-car environment. In order to ensure that the acquired image information is accurate and comprehensive, the server needs to perform a series of configurations and optimizations on the in-car camera. First of all, the installation position of the camera is crucial. It should be installed in a position where the driver's facial features can be clearly captured, usually above the dashboard, near the rearview mirror in the car, or in the center of the roof. At the same time, factors such as the height, sitting posture and driving habits of different drivers should be taken into consideration to ensure that the camera's viewing angle can cover the facial area of ​​most drivers. Secondly, the camera parameter settings also need to be carefully adjusted. This includes parameters such as resolution, frame rate, contrast, and brightness. A higher resolution can provide clearer image details, which helps to more accurately recognize facial expressions; a suitable frame rate can ensure that image information can be obtained smoothly in different driving scenarios to avoid losing key frames due to stuttering; the adjustment of contrast and brightness is to adapt to different light conditions, whether in bright daytime or dim night, to obtain clear and recognizable facial images. In addition, in order to improve the quality and stability of the image, some image processing technologies can also be used. For example, image filtering algorithms are used to remove noise from images to make the images smoother; image enhancement algorithms are used to highlight facial features and improve recognition accuracy.

[0031] In actual applications, when the driver enters the vehicle and starts the engine, the server will immediately activate the camera inside the vehicle and start collecting the driver's facial image information in real time. The camera will continuously capture the driver's face according to the preset parameters and frequency, and transmit the image data to the server for subsequent processing.

[0032] In some embodiments, after the server obtains the facial image information of the driver in the car, it will compare it with the facial image library when fatigued. The facial image library when fatigued stores a large number of facial feature images of drivers at different levels of fatigue, which may include the degree of eye closure, the relaxation of facial muscles, the dullness of eyes, etc. The server analyzes the acquired facial image information through image recognition technology and feature extraction algorithm, extracts features related to fatigue, and matches and compares them with the features in the facial image library when fatigued. If the matching degree reaches a certain threshold, it can be judged that the driver is in a fatigued state. Once it is judged that the driver is in a fatigued state, in order to remind the driver in time and ensure driving safety, the server will control the atmosphere light in the car to emit red light. The red light has the eye-catching characteristics, which can attract the attention of the driver and make him realize that he is currently in a fatigued state, thereby prompting the driver to take corresponding measures, such as stopping to rest, adjusting the driving state, etc., to avoid traffic accidents caused by fatigue driving. This method of alerting the driver to fatigue by changing the color of the atmosphere light is simple, direct and effective, and can convey important information in time without interfering with the driver's normal driving.

[0033] S102, inputting the facial image information into a preset emotion recognition model to obtain a user emotion judgment result, wherein the emotion recognition model is trained in advance based on a plurality of facial images with emotion labels; The emotion recognition model is a convolutional neural network (CNN) model based on deep learning technology. During the training phase, a large amount of face image data covering various emotional states (such as happiness, sadness, anger, calmness, surprise, etc.) was collected. In order to ensure the accuracy and generalization ability of the model, these image data come from people of different ages, genders, and races, and are taken under different lighting conditions, angles, and expression intensities. Each image is annotated with detailed emotion labels. During the training process, the CNN model automatically extracts features from the image through the convolution layer. The convolution kernel slides on the image to extract local features, such as features of the eyes, mouth, etc. As the number of network layers increases, the abstraction level of the features gradually increases. The pooling layer is used to reduce the amount of data, avoid overfitting, and retain important features. The fully connected layer maps the extracted features to different emotion categories and outputs the probability distribution of each emotion category through machine learning related functions.

[0034] In actual applications, the server preprocesses the facial image information obtained in real time, including normalization and cropping of the image, so that it meets the input requirements of the model. Then, the preprocessed image is input into the emotion recognition model, and the model quickly calculates and outputs a probability distribution, indicating the possibility that the facial image belongs to each emotion category. The server selects the emotion category with the highest probability based on the preset threshold as the final user emotion judgment result. In order to further improve the accuracy and robustness of emotion recognition, the attention mechanism can be introduced into the emotion recognition model. The attention mechanism enables the model to automatically focus on the areas in the image that are most relevant to emotion recognition, such as the areas around the eyes and mouth, which usually contain rich expression information. By adding an attention module to the convolutional neural network, the model can focus more on key features and reduce the interference of irrelevant information.

[0035] S103: adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and a preset device parameter adjustment relationship table, wherein the device parameter adjustment relationship table stores device control parameters corresponding to different emotional states.

[0036] When the server obtains the result of the user's emotion judgment, it will accurately adjust the corresponding target device in the car according to the preset device parameter adjustment relationship table to create an in-car environment that matches the driver's emotions. The device parameter adjustment relationship table is obtained through a large number of experiments and data analysis. It records in detail the specific operating parameters that should be adjusted for each target device under different emotional states. For example, when it is judged that the driver is in a happy mood, the volume of the music in the car may be appropriately increased to play cheerful music, and the atmosphere light in the car may be adjusted to a bright color; when the driver is in an angry mood, in addition to adjusting the color of the atmosphere light to a cool tone (such as blue) to soothe the mood, the air conditioning temperature can also be adjusted to make it cooler and reduce the noise in the car. In actual operation, the server monitors the changes in user emotions in real time. Once the emotion changes, it immediately searches for the corresponding device control parameters according to the device parameter adjustment relationship table and sends these parameters to the corresponding target device. After receiving the parameters, the target device adjusts them as required. In order to ensure the smoothness and comfort of the adjustment, the adjustment of the parameters can be gradual to avoid sudden changes that cause discomfort to the driver. At the same time, the server will continue to monitor the operating status of the device to ensure that the device operates as expected.

[0037] In some embodiments, when the server adjusts the corresponding target device operating parameters in the car according to the user emotion judgment result and the preset device parameter adjustment relationship table, if the user emotion judgment result is determined to be a negative emotion, the negative emotion can be set in advance. For example, when the driver is in an emotional state such as irritability, anxiety, anger, etc., the server will determine the corresponding target device operating parameters according to the device parameter adjustment relationship table. Among them, the target device operating parameters at least include controlling the color of the atmosphere light in the car to the target color. For example, the color of the atmosphere light is adjusted to cool colors such as blue and green. These colors usually have a soothing and relaxing visual effect, which helps to alleviate the driver's negative emotions. By changing the color atmosphere in the car, a relatively quiet and peaceful environment is created, thereby alleviating the driver's bad mood to a certain extent. At the same time, the aromatherapy device will also be controlled to emit a set fragrance. Different fragrances have different effects on people's emotions. For example, lavender fragrance has a calming and soothing effect, and lemon fragrance can refresh the mind. The server will select a suitable fragrance and control the aromatherapy device to emit it according to the type and degree of negative emotions, and further adjust the driver's emotions through olfactory stimulation, so that it can recover from negative emotions faster and improve driving comfort and safety. Such a design can work in combination from the visual and olfactory aspects to provide a more comfortable and emotion-regulating in-car environment for drivers who are in a negative mood.

[0038] In the embodiment of the present application, due to the technical means of obtaining the driver's facial image information through the in-vehicle camera, inputting a preset emotion recognition model to judge the emotion, and then intelligently adjusting the operating parameters of the target device in the vehicle according to the device parameter adjustment relationship table, the problem that the vehicle environment in the prior art cannot be intelligently adapted according to the driver's emotions is effectively solved, thereby achieving the technical effect of accurately sensing the driver's emotions and automatically optimizing the in-vehicle environment, improving driving comfort, concentration and safety.

[0039] In some embodiments, after adjusting the corresponding target device operating parameters in the car according to the user's emotion judgment results and the preset device parameter adjustment relationship table, the system also has the function of receiving scene mode selection information sent by the user through the display terminal. The display terminal can be a device such as the central control display screen in the car. The user can select different scene modes on the display terminal according to their preferences and needs, such as "comfort mode", "sports mode", "romantic mode", etc. When the server receives the scene mode selection information sent by the user, it will determine the target scene mode from the preset multiple scene modes based on the information. Each scene mode corresponds to a specific set of target device operating parameters. Then, the server will search and determine the new operating parameters of the target device in the scene mode according to the determined target scene mode. These new operating parameters may be different from the parameters previously adjusted according to the emotion judgment results, and are specially designed for this scene mode.

[0040] Finally, the server will control the target device to operate according to the newly determined operating parameters. For example, in "Comfort Mode", the seat may be adjusted to a more comfortable angle, the temperature in the car may be adjusted to a more suitable range, the volume of the audio may be adjusted to an appropriate level, etc. In "Sports Mode", the engine speed response may be increased, the hardness of the suspension system may be adjusted, etc. In this way, users can flexibly choose different scene modes according to their needs and moods, so that the in-vehicle environment is more in line with their expectations and improve the driving experience.

[0041] In some embodiments, the system uses the camera inside the car to determine the use environment when the user uses the vehicle. The camera can capture the image information inside and outside the car in real time, and judge whether the current use environment is daytime or nighttime by analyzing and processing these image information. When it is determined that the use environment is at night, in order to ensure the safety and convenience of the user getting on and off the car, the system will further use the camera to determine whether there is a user getting on and off the car. The camera will continue to monitor the activities of the people in the car. When it detects that someone is approaching the door or there are actions such as opening the door and getting off the car, it can be judged that there is a user getting on and off the car. Once it is determined that there is a user getting on and off the car, the server will immediately control the atmosphere light on the corresponding side of the getting on and off action. For example, if there is a person getting on and off the car at the left door, the atmosphere light on the left will light up. The lit atmosphere light can provide a certain amount of lighting, allowing the user to see the situation around the door more clearly at night, avoiding accidents such as bumps and falls due to insufficient light, and improving the safety and convenience of getting on and off the car. At the same time, this function of automatically controlling the lighting of the atmosphere light according to the user's getting on and off the car also reflects the intelligent and humanized design of the vehicle, and improves the user's experience.

[0042] After combining the above content, the following is a more detailed description of the process of the method provided by this implementation. Figure 2 , which is another flow chart of the method for intelligent control of the in-vehicle environment based on emotion recognition in an embodiment of the present application.

[0043] S201, when the camera detects that there is a child in the car, the sound sensor is used to detect whether there is crying sound information of the child; In the intelligent development of modern cars, monitoring the status of children in the car is an important part of improving the driving experience. When the server detects that there are children in the car through the in-car camera, in order to further understand the emotional state of the children, it will combine the sound sensor for comprehensive judgment. First of all, the in-car camera uses advanced image recognition technology to capture image information in the car in real time. It can not only detect the presence of children, but also recognize the general movements and expressions of children. For example, by analyzing the facial features of children, it can determine whether the children are in a happy, calm or restless state. The camera's field of view covers the main areas in the car to ensure that various behaviors of children can be discovered in time. At the same time, the sound sensor is carefully arranged in a suitable position in the car to ensure that the sound information in the car can be accurately captured. It has the characteristics of high sensitivity and low noise, and can clearly receive the sounds made by children, including crying, laughing, talking, etc. The sound sensor converts the received sound signal into a digital signal and transmits it to the server for processing.

[0044] After receiving the data from the camera and sound sensor, the server will conduct real-time analysis. Through the preset algorithm, it will determine whether the sound signal contains crying sound characteristics. If crying sound information is detected, the server will further determine parameters such as the intensity and duration of the crying so that corresponding measures can be taken later.

[0045] If it is determined that there is no crying sound information, executing step S202; If it is determined that there is crying sound information, step S203 is executed.

[0046] S202, maintaining the normal operation state of the current target device in the vehicle; When the server detects that there is a child in the car through the camera and the sound sensor does not detect crying sound information, the normal operation state of the target device in the current car is maintained.

[0047] S203, determining the position of the child in the car through a camera; After the server detects the presence of a child in the car through the camera and the sound sensor determines the presence of crying sound information, accurately determining the child's position becomes the key. First, the camera needs to have high resolution and wide viewing angle to ensure that it can clearly capture images of every corner of the car. The server processes the images obtained by the camera in real time and uses image recognition algorithms to identify the characteristics of the child. This includes analyzing and comparing the child's facial features, body contours, etc. In order to improve the accuracy of positioning, the server can use multi-frame image analysis technology. By continuously acquiring multiple frames of images, the position changes of the child at different times are tracked to more accurately determine its current position. At the same time, combined with deep learning algorithms, the system can continuously learn and optimize its ability to judge the child's position. In addition, considering that there may be multiple children in the car or the child's position may be partially blocked, the server can also use image segmentation technology to separate the child's image area from the complex background to further improve the accuracy of positioning.

[0048] S204, controlling the ambient light in the area corresponding to the child's position to be adjusted to a color gradient mode to soothe the child's emotions; When the server determines the position of the child in the car through the camera and detects the crying sound of the child, it is an effective means of soothing to control the ambient light in the area corresponding to the child's position to adjust to the color gradient mode. The server first establishes a communication connection with the ambient light control system in the car to obtain the current status information of the ambient light, including brightness, color, etc. Then, according to the preset soothing strategy, it sends instructions to the ambient light control system to adjust the ambient light in the corresponding area to the color gradient mode. In the color gradient mode, the color of the ambient light will change slowly in a certain order and time interval, such as gradually transitioning from red to blue, and then to green. In order to make the color gradient mode more soothing, the server can be personalized according to factors such as the child's age and gender. For example, for younger children, you can choose a mode with softer colors and slower changes; for older children, you can appropriately speed up the color change speed to increase the richness of the color. At the same time, the server can also automatically adjust the brightness of the ambient light based on the light conditions in the car to ensure that the best soothing effect can be achieved in different environments.

[0049] S205, receiving, via the display terminal, feedback information on the adjustment result of the target device by the user; After the server adjusts the ambient light in the area corresponding to the child's position, the user may provide feedback on the adjustment results of the target device based on the actual situation. The server provides the user with a feedback interface through the vehicle's display terminal (such as the central control display, instrument panel, etc.), on which the user can evaluate and modify the adjustment results of the ambient light. For example, the user can select options such as "satisfied" and "unsatisfied" by touching the screen, or directly drag the slider to adjust the brightness, color and other parameters of the ambient light. The server receives the user's feedback information in real time and analyzes and processes it. If the user is not satisfied with the adjustment of the ambient light, the server will readjust the parameters of the ambient light according to the user's feedback until the user is satisfied. At the same time, the server will also record the user's feedback information for subsequent optimization and improvement of the appeasement strategy.

[0050] Artificial intelligence technology can also be used to intelligently analyze and learn user feedback information. The server can automatically identify user preferences and needs by analyzing a large amount of user feedback data and establishing a user feedback model. For example, when a user repeatedly expresses satisfaction with a certain ambient light mode, the server will mark the mode as the user's preferred mode and automatically recommend it to the user in similar situations next time. In addition, the server can also adjust the parameters of other related target devices in real time based on user feedback information to provide a more coordinated in-car environment.

[0051] S206, determining the user's personalized lighting mode according to the adjustment result feedback information; After receiving the feedback information on the adjustment results of the target device by the user, the server needs to determine the user's personalized lighting mode based on this information. The server first classifies and organizes the user's feedback information and extracts the characteristic information related to the lighting mode, such as brightness, color, change speed, etc. Then, through data analysis and pattern recognition algorithms, these characteristic information are matched and compared with the preset lighting mode to find the personalized lighting mode that best meets the user's needs. In order to improve the accuracy and adaptability of personalized lighting modes, the server can use machine learning algorithms to continuously learn and optimize user feedback information. For example, by continuously collecting user feedback data, updating and improving personalized lighting modes, it can better meet the user's personalized needs.

[0052] S207: Update the personalized lighting mode to the device parameter adjustment relationship table.

[0053] After determining the user's personalized lighting mode, the server needs to update it to the device parameter adjustment relationship table so that the mode can be automatically applied in the subsequent control process. The server first opens the device parameter adjustment relationship table and finds the record related to the lighting mode. Then, the user's personalized lighting mode parameters are updated to the corresponding record, overwriting the original default parameters.

[0054] In order to ensure the accuracy and completeness of the updated device parameter adjustment relationship table, the server will verify and test the updated table. For example, by simulating different scenarios and user needs, check whether the personalized lighting mode can work properly and whether it is coordinated with other device parameters. A cloud-based device parameter adjustment relationship library can be established to store all users' personalized lighting modes and other device parameter adjustment information in the cloud. When the vehicle is connected to the Internet, the server can synchronize data with the cloud in real time to obtain the latest device parameter adjustment relationship table. In this way, when users use it on different vehicles, they only need to log in to their accounts to automatically apply their personalized settings, improving the convenience and consistency of the user experience. At the same time, the cloud database can also provide a large amount of data support for vehicle manufacturers and related research institutions, which will help further optimize the intelligent control algorithms and strategies for the in-vehicle environment.

[0055] In the embodiment of the present application, a camera is used to detect children in the car and a sound sensor is used to determine the crying situation, and then the ambient light mode is adjusted according to the position of the child. At the same time, user feedback is received to determine the personalized lighting mode and update the equipment parameter adjustment relationship table. Therefore, the problem that the existing technology cannot timely soothe the emotions of children in the car and the vehicle environment control is difficult to meet the personalized needs of users is effectively solved, thereby achieving the technical effect of quickly and accurately soothing children's emotions, reducing the interference of children's crying on driving, and improving the personalization and intelligence level of vehicle environment control, providing drivers and passengers with a more comfortable and safe in-car environment.

[0056] The following describes the server in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of a physical device structure of a server in an embodiment of the present application.

[0057] It should be noted that Figure 3 The structure of the server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0058] like Figure 3As shown, the server includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0059] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0060] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0061] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0062] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0063] Specifically, the server of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the method for intelligent control of the in-vehicle environment based on emotion recognition provided in the above embodiment is implemented.

[0064] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the server described in the above embodiment; or may exist independently without being assembled into the server. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the server, the server implements the vehicle-mounted environment intelligent control method based on emotion recognition provided in the above embodiment.

[0065] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0066] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0067] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A vehicle environment intelligent control method based on emotion recognition, applied to a server, characterized in that: The method comprises: Acquire facial image information of the driver in the car through the camera in the car; Inputting the facial image information into a preset emotion recognition model to obtain a user emotion judgment result, wherein the emotion recognition model is trained in advance based on a plurality of facial images with emotion labels; The operating parameters of the corresponding target device in the car are adjusted according to the user emotion judgment result and a preset device parameter adjustment relationship table, wherein the device parameter adjustment relationship table stores device control parameters corresponding to different emotional states.

2. The method according to claim 1, characterized in that The step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table specifically includes: If it is determined that the user emotion judgment result is a negative emotion, the target device operating parameters are determined according to the device parameter adjustment relationship table, and the target device operating parameters at least include controlling the color of the interior ambient light to a target color and controlling the aromatherapy device to emit a set fragrance.

3. The method according to claim 1, characterized in that After the step of obtaining the facial image information of the driver in the car through the camera in the car, the method further includes: Comparing the facial image information with a fatigue facial image library to determine whether the driver is in a fatigue state; If the driver is in a fatigue state, the interior ambient light is controlled to emit a red light to warn the driver.

4. The method according to claim 1, characterized in that: After the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, the method further includes: Receiving scene mode selection information sent by the user through the display terminal, and determining new operating parameters of the target device in the target scene mode according to the scene mode selection information; The target device is controlled to operate according to the new operating parameters.

5. The method according to claim 1, characterized in that After the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, the method further includes: Determine the usage environment of the user when using the vehicle through the camera; When it is determined that the use environment is at night, determining whether there is a user getting on or off the vehicle through the camera; If the getting on and off the vehicle action occurs, the ambient light on the side corresponding to the getting on and off vehicle action is controlled to light up.

6. The method according to claim 1, characterized in that After the step of adjusting the operating parameters of the corresponding target device in the vehicle according to the user emotion judgment result and the preset device parameter adjustment relationship table, the method further includes: When the camera detects that there is a child in the car, the sound sensor is used to detect whether there is crying sound information of the child; If the crying sound information exists, the position of the child in the car is determined by using a camera; The ambient light in the area corresponding to the child's position is controlled to be adjusted to a color gradient mode to soothe the child's emotions.

7. The method according to claim 6, characterized in that After the step of controlling the ambient light in the area corresponding to the child's position to be adjusted to a color gradient mode to soothe the child's emotions, the method further includes: receiving, via the display terminal, feedback information on adjustment results of the target device by the user; Determine the user's personalized lighting mode according to the adjustment result feedback information; The personalized lighting mode is updated to the device parameter adjustment relationship table.

8. A server, characterized in that: The server includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the server to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a server, the server is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a server, the server is caused to execute the method according to any one of claims 1 to 7.

Images