Control method of automobile cabin, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an automobile cabin control method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target video of cabin user image data in a target time period through a ship berth domain controller, and performing face recognition and feature extraction on the target video to obtain attribute information corresponding to a user and state information of the corresponding user in the target time period, and selecting a target scene matched with the user from a plurality of preset scenes according to the attribute information and the state information of the user, determining a corresponding target scene engine based on the target scene, and controlling an actuator of the cabin to operate to realize the target scene. The method is used for achieving the effects of improving the accuracy and convenience of cabin scene regulation and control, enhancing the reliability of cabin scene regulation and control and improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of automotive electronics, and in particular to a control method, electronic equipment and storage medium for an automotive cockpit. Background Art

[0002] During the start-up of a car, passengers in the car cabin usually have different riding experiences and different needs according to the actual situation.

[0003] The existing technology provides users with a function selection entrance through a scene engine module in the cockpit. Users can comprehensively determine the target scene that needs to be adjusted in the current car cabin based on their own needs in the current scenario or the needs of other passengers obtained after inquiry. After determining the target scene, the target scene engine corresponding to the target scene is found and triggered from the scene engine module.

[0004] However, during driving, it is difficult for users to take into account the needs of various users and to formulate a target scene engine suitable for the current scenario in a timely manner, making it difficult to provide targeted and accurate services to users in the cabin, reducing the accuracy and convenience of cabin scene control, and further reducing the user experience. In addition, when user movements are restricted, it is difficult to adjust the target scene engine according to the actual scenario in a timely manner, reducing the reliability of cabin scene control. Summary of the invention

[0005] The embodiments of the present application provide a control method, electronic device and storage medium for a car cabin, which are used to improve the accuracy and convenience of cabin scene control, enhance the reliability of cabin scene control, and improve the user experience.

[0006] In a first aspect, an embodiment of the present application provides a method for controlling a vehicle cockpit, comprising:

[0007] Acquire a target video in a target time period, where the target video includes image data of a user in the cockpit;

[0008] Determine the user's attribute information according to the target video;

[0009] According to the target video, determine the user's status information within the target time period;

[0010] Based on the user's attribute information and state information, determine a target scene that matches the user from a plurality of pre-set scenes;

[0011] At least one actuator in the cockpit is controlled to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

[0012] In a possible implementation, determining the state information of the user within the target time period according to the target video includes:

[0013] According to the target video, determine the user's eye movement information and head position information at each time stamp;

[0014] Determine whether the eye closure angle at each timestamp is greater than the closure angle threshold according to the eye movement information; if the eye closure angle is greater than the closure angle threshold, determine that the user's eyes are closed at the corresponding timestamp; otherwise, determine that the user's eyes are open at the corresponding timestamp;

[0015] Determine whether the posture angle at each timestamp is greater than the posture angle threshold according to the head position information; if the posture angle is greater than the posture angle threshold, determine that the user posture at the corresponding timestamp is a normal orientation; otherwise, determine that the user posture at the corresponding timestamp is an abnormal orientation;

[0016] The user's state information in the target time period is determined based on the information of the user's eyes and user posture at each time stamp.

[0017] In a possible implementation, determining the state information of the user in the target time period according to the information of the user's eyes and the user's posture at each time stamp includes:

[0018] Based on the information of the user's eyes at each time stamp, determine the time interval from each time the user's eyes are closed to the next time the eyes are opened, and based on the information of the user's posture at each time stamp, determine the time interval from each time the user's abnormal orientation state to the next normal orientation state;

[0019] The state information of the user in the target time period is determined according to the time intervals corresponding to the closure of each eye and the time intervals corresponding to the abnormal orientation states.

[0020] In a possible implementation, based on the attribute information and state information of the user, determining a target scene matching the user from a plurality of pre-set scenes includes:

[0021] Based on the attribute information of the user, determining an associated scene matching the attribute information of the user from a plurality of pre-set scenes;

[0022] According to the user's state information, a target scene matching the state information is determined in the associated scenes.

[0023] In a possible implementation, controlling at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene includes:

[0024] Based on the target scenario of moderate fatigue of a young female user as the main driver, a target scenario engine corresponding to moderate fatigue of a young female user as the main driver is determined to drive the headrest speakers in the cabin to play pre-set music and turn on the fragrance to release a pre-set scent;

[0025] Based on the target scenario of moderate fatigue of the child user on the front passenger seat, a target scenario engine corresponding to moderate fatigue of the child user on the front passenger seat is determined to drive the screen actuator in the cockpit to play the video and drive the speakers in the car to play the pre-set music;

[0026] Based on the target scenario of moderate fatigue of an elderly user on the front passenger seat, a target scenario engine corresponding to the moderate fatigue of an elderly user on the front passenger seat is determined to control the seat actuator to open the footrest, and to control the air-conditioning actuator to adjust the cabin temperature to a preset temperature value.

[0027] In a possible implementation, before determining a target scene matching the user from a plurality of preset scenes, the method includes:

[0028] Obtaining a configuration instruction, the configuration instruction including indicated attribute information and a corresponding associated scenario;

[0029] Based on the configuration instruction, a matching relationship is established between the attribute information and the associated scene, wherein the associated scene includes scenes corresponding to the attribute information under different state information.

[0030] In a second aspect, an embodiment of the present application provides a control device for a car cabin, comprising:

[0031] An acquisition module, used to acquire a target video in a target time period, where the target video includes image data of a user in the cockpit;

[0032] A processing module, used to determine attribute information of a user based on a target video;

[0033] A processing module, used to determine the status information of the user within the target time period according to the target video;

[0034] A processing module, used to determine a target scene matching the user from a plurality of pre-set scenes based on the user's attribute information and state information;

[0035] The control module is used to control at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

[0036] In a possible implementation, the processing module is used to determine the eye movement information and head position information of the user at each time stamp according to the target video;

[0037] Determine whether the eye closure angle at each timestamp is greater than the closure angle threshold according to the eye movement information; if the eye closure angle is greater than the closure angle threshold, determine that the user's eyes are closed at the corresponding timestamp; otherwise, determine that the user's eyes are open at the corresponding timestamp;

[0038] Determine whether the posture angle at each timestamp is greater than the posture angle threshold according to the head position information; if the posture angle is greater than the posture angle threshold, determine that the user posture at the corresponding timestamp is a normal orientation; otherwise, determine that the user posture at the corresponding timestamp is an abnormal orientation;

[0039] The user's state information in the target time period is determined based on the information of the user's eyes and user posture at each time stamp.

[0040] In a possible implementation, the processing module is used to determine the time interval between each time when the user's eyes are closed and the next time when the eyes are opened based on the information of the user's eyes at each time stamp, and determine the time interval between each time when the user's abnormal orientation state is reached and the next time when the user's normal orientation state is reached based on the information of the user's posture at each time stamp;

[0041] The state information of the user in the target time period is determined according to the time intervals corresponding to the closure of each eye and the time intervals corresponding to the abnormal orientation states.

[0042] In a possible implementation, the processing module is used to determine, based on the attribute information of the user, an associated scene that matches the attribute information of the user from a plurality of preset scenes;

[0043] According to the user's state information, a target scene matching the state information is determined in the associated scenes.

[0044] In a possible implementation, the control module is used to determine a target scenario engine corresponding to moderate fatigue of a young female user driving the vehicle based on the target scenario of moderate fatigue of a young female user driving the vehicle, so as to drive a headrest speaker in the cabin to play preset music and turn on a fragrance to release a preset smell;

[0045] Based on the target scenario of moderate fatigue of the child user on the front passenger seat, a target scenario engine corresponding to moderate fatigue of the child user on the front passenger seat is determined to drive the screen actuator in the cockpit to play the video and drive the speakers in the car to play the pre-set music;

[0046] Based on the target scenario of moderate fatigue of an elderly user on the front passenger seat, a target scenario engine corresponding to the moderate fatigue of an elderly user on the front passenger seat is determined to control the seat actuator to open the footrest, and to control the air-conditioning actuator to adjust the cabin temperature to a preset temperature value.

[0047] In a possible implementation, the processing module is used to obtain a configuration instruction, where the configuration instruction includes indicated attribute information and a corresponding associated scenario;

[0048] Based on the configuration instructions, a matching relationship is established between the attribute information and the associated scenes, wherein the associated scenes include scenes corresponding to the attribute information under different state information.

[0049] In a third aspect, an embodiment of the present application provides a vehicle cabin control system, including: an OMS camera, a cabin domain controller, a regional controller, a sensor, and an actuator; wherein the cabin domain controller includes an OMS identification module, an association module, and a scene engine module;

[0050] The OMS camera is connected to the input end of the cabin domain controller for transmitting the target video of the target time period to the cabin domain controller;

[0051] The output end of the cabin domain controller is connected to the actuator through the regional controller; the output end of the cabin domain controller is connected to the sensor;

[0052] The berthing domain controller is used to implement the above first aspect and / or various possible implementations of the first aspect.

[0053] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0054] Memory stores computer-executable instructions;

[0055] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0056] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0057] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0058] The control method, electronic device and storage medium of the car cabin provided in the embodiment of the present application obtains the target video of the cabin user image data within the target time period through the docking domain controller, and performs face recognition and feature extraction on the image information in the target video to obtain the attribute information corresponding to the user and the status information of the corresponding user within the target time period, thereby selecting the target scene matching the user from the pre-set multiple scenes according to the attribute information and status information of the user, determining the corresponding target scene engine based on the target scene, and controlling the actuator operation of the cabin to realize the target scene. The accuracy and convenience of the cabin scene control are improved, the reliability of the cabin scene control is enhanced, and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] Figure 1 Schematic diagram of the process of the vehicle cockpit control method provided in this application Figure 1 ;

[0061] Figure 2 Schematic diagram of the process of the vehicle cockpit control method provided in this application Figure 2 ;

[0062] Figure 3 A schematic diagram of the structure of the vehicle cockpit control system provided for this application;

[0063] Figure 4 A schematic diagram of the structure of the control device for the car cabin provided in this application;

[0064] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0065] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0067] First, the terms involved in this application are explained:

[0068] OMS camera: The camera in the passenger monitoring system obtains the perception data of passengers in the cabin through the OMS camera to further improve the safety performance of the car based on the perception data. For example, based on the perception data obtained by the OMS camera, it can determine whether the passengers have fastened their seat belts, whether there are children or pets left alone in the car, and then remind the driver through the mobile phone that there are objects left in the car.

[0069] Scene engine: It is an entrance that opens all sensors and actuators in the car to users. Users can select and trigger the target scene on the central control screen interface in the car cabin, so that the relevant actuators of the vehicle can be executed in the order of selection. For example, the user triggers the sleep scene through the buttons or voice of the central control screen interface, so that the relevant actuators of the vehicle recline the seats, open the sunshade of the skylight, turn on the light music to help sleep and turn on the fragrance to enter a comfortable sleep state; in addition, when the user leaves the vehicle, the dog mode scene is triggered by buttons or voice, so that the vehicle keeps the air conditioning running to ensure the temperature in the car is suitable, and the current temperature in the car is displayed in real time through the central control display of the vehicle, so that passers-by can know in time that the pets in the car are in a safe environment.

[0070] During the operation of the car, users in the car cabin usually have different riding scene requirements. In order to meet the needs of users in the car cabin, the existing technology provides users with scene entrances through the central control screen in the car cabin, allowing users to adjust the cabin scene by selecting various scene entrances on the central control screen to improve the riding experience.

[0071] However, in actual situations, the needs of each user in the cockpit are different. Therefore, in the existing technology, it is difficult for the driver to design a target scene that suits each user's needs, which reduces the accuracy and convenience of the cockpit scene control and further reduces the user experience. When the user is unable to move in the cockpit, it is difficult to adjust the scene in time according to the current actual scene, which reduces the reliability of the cockpit scene control.

[0072] In addition, in order to improve the convenience of scene control, the existing technology obtains real-time image information of the co-pilot user in the car cockpit through the camera in the cockpit, so that the real-time mental state and / or sleep state of the co-pilot user can be identified based on the real-time image information through the scene control system, and the entertainment system and air conditioning system are controlled according to the real-time mental state. However, although the same mental state and / or sleep state are reflected in different user groups, the target scene actually required by each user is different. The existing technology adjusts the scene according to the user's mental state and / or sleep state, which reduces the accuracy of scene control, makes it difficult to meet the needs of different user groups, and reduces the user experience.

[0073] The control method of the car cabin provided by the present application configures the scene corresponding to the demand for the attribute information of the special group of people, and during the start-up of the car, obtains the target video of the target time period through the camera in the car cabin, and processes the image data of the user in the obtained target video to determine the attribute information of the user in the current cabin and the status information of the user in the target time period, so as to determine the associated scene matching the attribute information in the multiple scenes previously pre-set according to the attribute information of the user, and determine the target scene corresponding to the status information in the associated scene. After determining the target scene, the target scene engine corresponding to the target scene generates and outputs a control signal to drive the relevant actuator to perform the corresponding operation according to the control signal to meet the scene requirements of users with different attribute information in the current cabin, improve the accuracy and reliability of the cabin scene control, improve the convenience of scene control, and enhance the user experience.

[0074] The control method of the car cockpit of the present application is a method of triggering a related scene engine according to user needs to control a corresponding actuator to perform a corresponding operation. The executor of the control method of the car cockpit is a mooring domain controller in the car cockpit control system, and the mooring domain controller can be deployed in an electronic device.

[0075] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0076] Figure 1 Schematic diagram of the process of the vehicle cockpit control method provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0077] S101: Acquire a target video in a target time period.

[0078] More specifically, the ship berthing domain controller obtains a target video in a target time period, wherein the target video includes image data of a user in the cockpit.

[0079] In a possible embodiment, a target video of the user sitting in the main driver's seat and the co-driver's seat during a target time period is captured by a camera in the car cabin, thereby obtaining face image data and body posture image data corresponding to the user in the car cabin, respectively. The camera includes but is not limited to an OMS camera and a DMS camera.

[0080] Optionally, within the target time period, image data of the driver in the main driving seat in the cockpit is captured by the OMS camera, and image data of the passenger in the co-pilot seat in the cockpit is captured by the DMS camera.

[0081] S102: Determine attribute information of the user according to the target video.

[0082] More specifically, face detection is performed based on the image data of the user in the target video to locate and identify the face corresponding to the user in the image data, and based on the user's face, feature extraction is performed on the user's face through a local information method to extract the user's facial features, and the user's facial features are processed through a deep learning model to obtain the user's corresponding attribute information, wherein the user's facial features include but are not limited to the user's corresponding facial shape, color, eye size and eye shape, and the attribute information includes gender information and age information.

[0083] Optionally, based on the facial features of each user, the deep learning model is used to perform gender prediction and age prediction on the user's facial features, thereby determining and outputting the gender and age information corresponding to the user.

[0084] Optionally, before the deep learning model predicts the gender and age of the user's facial features respectively, the deep learning model is pre-trained, and the pre-training process includes: obtaining different facial images, and marking the gender corresponding to each facial image in the corresponding facial image, inputting the facial images with gender marked into the deep learning model, predicting each facial image through the deep learning model, and comparing the obtained prediction results with the actual marked gender, adjusting the model parameters of the neural network according to the comparison results until the prediction results output by the deep learning model are consistent with the actual marked gender in the corresponding facial image, outputting the corresponding model parameters at this time, and performing gender prediction and age prediction in actual application based on the model parameters.

[0085] In a possible embodiment, the target video is detected by the DNN face detector of OpenCV to obtain the face information of the driver in the main driving seat in the target video. Based on the detected face information of the driver, the detected face information is extracted by the HOG (Histogram of oriented gradient) feature descriptor, and each local feature vector in the face information is connected in series to obtain the face features corresponding to the driver, and the face features corresponding to the driver are input into the pre-trained convolutional neural network model, and the convolutional neural network model is used to predict and output the male and female information corresponding to the driver. Among them, the output male and female information includes male and female. OpenCV is an open source cross-platform computer vision and machine learning software library that can run on Linux, Windows, Android and Mac OS operating systems. HOG is a feature descriptor used for target detection in the field of computer vision and image processing.

[0086] In a possible embodiment, based on the detected facial information of the driver, the target video is processed by the NPU to extract the facial area corresponding to the driver, and the image of the facial area is preprocessed, and the preprocessing method includes but is not limited to adjusting the image size, image normalization and image grayscale. The preprocessed facial image is input into the pre-trained convolutional neural network model, and the age information corresponding to the driver is predicted and output by the convolutional neural network model.

[0087] Exemplarily, the preprocessed facial image is processed by a convolutional neural network model to obtain a predicted age, and the corresponding age information is determined and output based on the predicted age.

[0088] For example, if the predicted age is in the age group of 0-5 years old, the age information is determined and output as a child; if the predicted age is in the age group of 5-14 years old, the age information is determined and output as a child; if the predicted age is in the age group of 14-35 years old, the age information is determined and output as a young person; if the predicted age is in the age group of 35-65 years old, the age information is determined and output as a middle-aged person; if the predicted age is over 65 years old, the age information is determined and output as an elderly person.

[0089] Optionally, feature extraction processing is performed on the detected face through LBP (Local binary patterns) algorithm, SIFT (Scale-invariant feature transform) algorithm, SURF (Speeded up robust features) algorithm.

[0090] S103: Determine the user's status information within the target time period according to the target video.

[0091] More specifically, the target video within the target time period captured by the camera is processed through a deep learning model to obtain the status information of the user in the cabin within the target time period.

[0092] In a possible embodiment, a camera in the cockpit captures facial image data and body posture image data of the driver within a target time period, wherein the facial image data includes but is not limited to facial expressions, and the body posture image data includes but is not limited to eye movements, head positions, and other behaviors indicating fatigue. Feature extraction is performed based on the facial image data and body posture image data of the driver to obtain features indicating fatigue, and eye movement information and head position information of the driver are determined based on the features indicating fatigue, thereby determining the state information indicating fatigue of the user within the target time period based on the obtained eye movement information and head position information.

[0093] For example, how long your eyes are closed, how often you blink, and the angle at which your head is tilted.

[0094] Optionally, the eye movement information of the corresponding user is determined based on the eye closure time and the frequency of blinking, and the head tilt angle of the corresponding user is determined based on the head tilt angle.

[0095] In one possible embodiment, Figure 2 Schematic diagram of the process of the vehicle cockpit control method provided in this application Figure 2 ,like Figure 2 As shown, in this embodiment Figure 1 Based on the embodiment, determining the state information indicating fatigue of the user within the target time period is described in detail, the method includes:

[0096] S201: Determine eye movement information and head position information of the user at each time stamp.

[0097] More specifically, the eye movement information and head position information corresponding to each time stamp of the user in the target time period are determined according to the features indicating user fatigue, wherein the eye movement information is used to indicate the closing information of the eyes at each time stamp in the target time period, and the head position information is used to indicate the position information of the head at each time stamp in the target time period.

[0098] S202: Determine whether the eye closure angle at each time stamp is greater than a closure angle threshold according to the eye movement information.

[0099] More specifically, the eye closure angle at each timestamp is determined based on the eye movement information, and it is determined whether each eye closure angle is greater than the closure angle threshold. If the eye closure angle is greater than the closure angle threshold, it is determined that the user's eyes are closed at the corresponding timestamp; otherwise, it is determined that the user's eyes are open at the corresponding timestamp.

[0100] In a possible embodiment, the closed angle of the user's eyes at each time stamp is determined by detecting the user's eye movement information at each time stamp, and the closed angle of the user's eyes at each time stamp is compared with the closed angle threshold. When the closed angle threshold is 80%, it is determined whether each closed angle is greater than 80% of the eye closed angle, and when the closed angle at any time stamp is 70%, it is determined that the user's eyes at the time stamp are in an open state, and if the closed angle at the time stamp is 90%, it is determined that the user's eyes at the time stamp are in a closed state.

[0101] S203: Determine whether the posture angle at each time stamp is greater than a posture angle threshold according to the head position information.

[0102] More specifically, the posture angle at each timestamp is determined based on the head position information, and it is determined whether each posture angle is greater than the posture angle threshold. If the posture angle is greater than the posture angle threshold, the user posture at the corresponding timestamp is determined to be in a normal orientation; otherwise, the user posture at the corresponding timestamp is determined to be in an abnormal orientation.

[0103] In a possible embodiment, the normal posture angle of the driver in the main driving seat and the passenger in the co-pilot seat in the cockpit is calibrated by an infrared camera, the normal posture angle is used as the reference angle, and the posture angle threshold is set to exceed the reference angle by 20 degrees upward or downward. For example, when the posture angle corresponding to the real-time head position information exceeds the reference angle by 25 degrees upward, it is determined that the user posture is abnormal at this time.

[0104] Exemplarily, when the posture angle corresponding to the real-time head position information exceeds the reference angle by 30 degrees downward, it is determined that the user posture is in an abnormal orientation.

[0105] Exemplarily, when the posture angle corresponding to the real-time head position information exceeds the reference angle by 10 degrees upward or downward, it is determined that the user posture is in a normal orientation at this time.

[0106] S204: Determine the state information of the user in the target time period according to the information of the user's eyes and the user's posture at each time stamp.

[0107] More specifically, based on the information of the user's eyes at each timestamp, the time interval from each time the user closes his eyes to the next time he opens his eyes is determined, and based on the information of the user's posture at each timestamp, the time interval from each time the user's abnormal orientation state to the next normal orientation state is determined; based on the time interval corresponding to each eye closure and the time interval corresponding to each abnormal orientation state, the user's state information within the target time period is determined.

[0108] In a possible embodiment, within the target time period, for any eye closure, a corresponding timestamp is determined, and after the current eye closure, a timestamp corresponding to the next eye opening is determined, so as to calculate the time interval corresponding to the current eye closure. If the time interval exceeds 50 seconds, it is determined that the user's status information is severe fatigue within the current eye closure time interval.

[0109] In a possible embodiment, if the current eye closure time interval does not exceed 50 seconds, it is further determined whether there is an abnormal orientation state of the user within the time interval, and the time interval between the abnormal orientation state and the next normal orientation state exceeds the first preset time threshold. If there is an abnormal orientation state of the user, and the time interval between the abnormal orientation state and the next normal orientation state exceeds the first preset time threshold, it is determined that within the current first preset time threshold, the user's state information is severe fatigue.

[0110] In a possible embodiment, if the current eye closure time interval does not exceed 50 seconds and there is no abnormal orientation state of the user, it is further determined whether the current eye closure time interval exceeds 15 seconds, and if it exceeds 15 seconds, it is determined that the current eye closure indicates that the user's status information is moderate fatigue.

[0111] In a possible embodiment, if the current eye closure time interval does not exceed 15 seconds, it is further determined whether there is an abnormal orientation state of the user within the time interval, and the time interval between the abnormal orientation state and the next normal orientation state exceeds the second preset time threshold. If there is an abnormal orientation state of the user, and the time interval between the abnormal orientation state and the next normal orientation state exceeds the second preset time threshold, it is determined that within the current second preset time threshold, the user's state information is moderate fatigue.

[0112] In a possible embodiment, if the current eye closure time interval does not exceed 15 seconds and there is no abnormal orientation state of the user, it is further determined whether the current eye closure time interval exceeds 10 seconds, and if it exceeds 10 seconds, it is determined that the current eye closure indicates that the user's state information is mild fatigue.

[0113] In a possible embodiment, if the current eye closure time interval does not exceed 10 seconds, it is further determined whether there is an abnormal orientation state of the user within the time interval, and the time interval between the abnormal orientation state and the next normal orientation state exceeds a third preset time threshold. If there is an abnormal orientation state of the user, and the time interval between the abnormal orientation state and the next normal orientation state exceeds the third preset time threshold, it is determined that within the current third preset time threshold, the user's state information is mild fatigue.

[0114] The embodiment of the present application improves the accuracy of user status information by determining the time interval from each time the user closes their eyes to the next time they open their eyes, and the time interval from each abnormal orientation state to the next normal orientation state, to determine detailed status information of the degree of fatigue within a target time period, thereby further improving the accuracy and reliability of the target scenarios provided for different users.

[0115] Optionally, according to each time stamp of the user's eyes in the target time period, the closing speed and closing frequency of the eyes in the target time period are determined, and the state information of the user in the target time period is determined according to the closing speed and closing frequency. The closing speed is the speed of the user's eyes switching from open to closed, and the closing frequency is the number of times the eyes switch from open to closed in the target time period.

[0116] In a possible embodiment, the user's eyes change from open to closed once within the target time period of 10 seconds, so the closing frequency of the user's eyes within the target time period of 10 seconds is 1, which is less than the normal closing frequency of 2, and the movement speed of the user's eyes changing from open to closed is 0.8. After being closed for 0.1 seconds, the eyes are opened and maintained, so the closing speed of the user's eyes within the target time period of 10 seconds is greater than the normal closing speed of 0.5, that is, it is determined that the user occasionally blinks within the target time period of 10 seconds, and the user is in a non-fatigued state within the current target time period.

[0117] In a possible embodiment, the user's eyes change from open to closed three times within the target time period of 10 seconds, so the frequency of eye closure of the user within the target time period of 10 seconds is 3, which is greater than the normal closure frequency of 2. Each time the user's eyes change from open to closed, the speed of the movement is 0.1, and the eyes are closed for 2 seconds before opening. Therefore, the closing speed of the user's eyes within the target time period of 10 seconds is less than the normal closing speed of 0.5, which means that the user frequently closes his eyes for a long time within the target time period of 10 seconds, and the user is in a fatigued state within the current target time period.

[0118] The control method of the car cabin provided in the embodiment of the present application processes the target video and determines the user status information based on the processed eye movement information and head position information, thereby providing a guarantee for the subsequent accurate determination of the fatigue status information of the cabin user, further improving the accuracy of the cabin scene control, and enhancing the user's riding experience.

[0119] S104: Based on the attribute information and state information of the user, determine a target scene matching the user from a plurality of pre-set scenes.

[0120] More specifically, based on the attribute information of the user, an associated scene matching the attribute information of the user is determined from a plurality of preset scenes; and according to the status information of the user, a target scene matching the status information is determined from the associated scenes.

[0121] For example, if the co-pilot user is a child user, a child care scene matching the child user is determined from multiple pre-set scenes; if the main driver is a young woman and the young woman starts the vehicle, a young woman care scene matching the young woman user is determined from multiple pre-set scenes; if the co-pilot user is an elderly user, an elderly care scene matching the elderly user is determined from multiple pre-set scenes. Based on the above associated scenes, the target scene based on the corresponding associated scene is further determined according to the user's status information.

[0122] The embodiment of the present application matches different associated scenes according to the different attribute information of each user in the cockpit, and thus determines the target scene that matches the user status information from the corresponding associated scenes based on the user's status information, thereby improving the matching accuracy of the target scene, reducing the probability that the user needs to manually adjust the scene again according to the actual attributes after matching the scene, and improving convenience.

[0123] S105: Control at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

[0124] More specifically, through the target scene engine corresponding to the target scene, at least one actuator in the cockpit is controlled to perform an operation matching the target scene, so as to provide accurate services for users.

[0125] Optionally, for the young female caring scenario, the engine corresponding to the young female user in the main driving position is determined, so that when the vehicle is started, the headrest speakers in the cockpit are controlled to play pre-set female music, and the air-conditioning system is controlled to automatically adjust the temperature of the main driving temperature zone to a pre-set female temperature range to create a comfortable driving environment.

[0126] Optionally, for the young female caring scenario, the engine corresponding to the young female user who is the main driver is determined to start the fragrance in the cabin, and the sunroof is controlled to close automatically to improve the air quality in the vehicle cabin.

[0127] In a possible embodiment, when a young woman enters the main driving seat, the young women care engine in the young women care scene drives the air-conditioning system to adjust the temperature of the main driving temperature zone to a preset female temperature value, and when the road section ahead of the vehicle is affected by the air, the above-mentioned female care engine drives the windows in the car to close automatically, and controls the opening of the fragrance to improve the air quality in the car.

[0128] Exemplarily, during the driving of a young female, if it is determined that the young female is in a state of severe fatigue, the young female severe fatigue engine in the young female care scene drives the in-car speaker to play a parking voice to remind the user to pull over in time to relieve fatigue. Based on the above-mentioned car-picking voice, after a preset time, it is determined whether the severe fatigue state of the female user has changed. If it has not changed, the window on the main driver's side is opened by the above-mentioned young female severe fatigue engine, so that the main driver can breathe fresh air to relieve fatigue; if the main driver's young female user changes from severe fatigue to moderate fatigue, the young female moderate fatigue engine in the young female care scene drives the in-car window to close, and turns on the fragrance to release a refreshing smell (such as the smell of wind oil); if the main driver's young female user changes from severe fatigue to mild fatigue, the female mild fatigue engine in the young female care scene drives the main driver's headrest speaker in the cabin to play soothing music.

[0129] In a possible embodiment, during driving, when it is determined that the young female user in the co-pilot seat is in a state of mild fatigue, the co-pilot mild fatigue engine in the young female care scenario drives the screen actuator in the cockpit to play female films and TV shows; when the young female user in the co-pilot seat is in a state of moderate fatigue, the co-pilot moderate fatigue engine in the young female care scenario drives the main driver's headrest speaker in the cockpit to play soothing music, and drives the cabin massage system to turn on the mild massage function of the co-pilot seat; when it is determined that the young female user in the co-pilot seat is in a state of severe fatigue, the co-pilot severe fatigue engine in the young female care scenario drives the co-pilot seat to automatically adjust to a preset female tilt angle, so that the young female user in the co-pilot seat can rest in the co-pilot seat.

[0130] Optionally, for the child care scenario, the engine corresponding to the child user in the front passenger seat is determined to control the air-conditioning system to automatically adjust the temperature of the front passenger seat temperature zone to 21°C-24°C, and drive the seat for the child to rise to a preset height.

[0131] In a possible embodiment, when a child user enters the front passenger seat, the child care engine in the child care scenario drives the air conditioning system to adjust the temperature of the front passenger temperature zone to a preset child temperature value, and drives the front passenger seat to rise to a preset child height value.

[0132] For example, during the driving process of the car, if it is determined that the child user in the front passenger seat is in a state of moderate fatigue at the moment, the child moderate fatigue engine in the child care scenario drives the screen actuator in the cockpit to play the animation video, and drives the speakers in the car to play the preset children's music; when it is determined that the child changes from a moderate fatigue state to a severe fatigue state, the child severe fatigue engine in the child care scenario drives the front passenger seat to automatically adjust to the preset child tilt angle, and controls the speakers in the car to lower the volume to play children's stories. During the period of severe fatigue of the child user, if the front of the vehicle is a preset special road for children (such as a bumpy road), the above-mentioned child severe fatigue engine drives the main driver's headrest speaker to play the child safety advice voice, so that the driver drives slowly during the period of child fatigue.

[0133] Optionally, for the elderly care scenario, the engine corresponding to the elderly user in the front passenger seat is determined to control the headrest speakers in the cabin to play music, and to control the air-conditioning system to automatically adjust the temperature of the front passenger temperature zone to a pre-set temperature range for the elderly.

[0134] In a possible embodiment, when an elderly user enters the co-pilot seat, the elderly care engine in the elderly care scenario drives the co-pilot headrest speaker to play preset elderly music, and drives the air-conditioning system to adjust the temperature of the co-pilot temperature zone to a preset elderly temperature value.

[0135] For example, during the driving of the car, if it is determined that the elderly user in the front passenger seat is moderately fatigued, the elderly moderate fatigue engine in the elderly care scenario drives the actuator of the front passenger seat where the elderly user sits to open the footrest, and drives the speakers in the car to play preset soothing music for the elderly; if it is determined that the elderly user in the front passenger seat changes from moderate fatigue to severe fatigue, the elderly severe fatigue engine in the elderly care scenario drives the cabin massage system to turn on the light massage function of the front passenger seat. During the period of severe fatigue of the elderly user, if there is a preset special road for the elderly (such as a sharp turn) in front of the vehicle, the headrest speaker of the main driver is driven by the above-mentioned severe fatigue engine to play the elderly safety advice voice, so that the driver can reduce the driving speed during the period of fatigue of the elderly user and pay attention to the surrounding vehicles.

[0136] Optionally, after acquiring the target video in the car cabin, face recognition and feature extraction are performed on the target video. When it is determined that there is a pet on the co-pilot seat, the target scene engine is determined to be a pet care scene engine, and the pet entertainment system is started through the pet care scene engine, for example, controlling the windows to close automatically and driving the speakers in the car to play pet videos.

[0137] In a possible embodiment, if the user is in a state of mild fatigue, the mild fatigue engine corresponding to the mild fatigue scene drives the in-car speakers to play soothing music. If the user is in a state of moderate fatigue, the moderate fatigue engine corresponding to the moderate fatigue scene drives the in-car speakers to play soothing music, and drives the actuator of the seat where the user is sitting to turn on the footrest. If the user is in a state of severe fatigue, the severe fatigue engine corresponding to the severe fatigue scene drives the cabin massage system to turn on the mild massage function and footrest of the seat where the user is sitting, and controls the in-car speakers to play soothing music.

[0138] The embodiment of the present application targets users with different attribute information, different fatigue states and different sitting positions. By using a corresponding target scene engine, the corresponding actuators in the cockpit are driven to execute corresponding solutions, thereby improving the accuracy of the cockpit scene control and enhancing the user's riding experience.

[0139] The control method of the automobile cabin provided in the embodiment of the present application processes the target video taken by the camera, obtains the attribute information of the users in the cabin, and extracts the fatigue characteristics based on the processed target video to obtain the fatigue status information of the users in the cabin, thereby specifically determining the target scene matching each user based on the attribute information and status information corresponding to each user in the cabin, and controlling the execution of the actuator through the corresponding target scene engine to accurately match the appropriate scene to each user, thereby improving the accuracy and reliability of scene control and further enhancing the user experience.

[0140] Optionally, before determining a target scene that matches the user from a plurality of pre-set scenes, the method for controlling a car cabin includes: obtaining configuration instructions, the configuration instructions including indicated attribute information and corresponding associated scenes; based on the configuration instructions, establishing a matching relationship between the attribute information and the associated scenes, wherein the associated scenes include scenes corresponding to the attribute information under different state information.

[0141] In a possible embodiment, the user generates a configuration instruction for a preset associated scene corresponding to a young male through a scene engine module in the cockpit, and sends the configuration instruction to the ship berthing domain controller. When the ship berthing domain controller obtains the target video and determines that the attribute information is a young male after processing the target video, the associated scene corresponding to the young male is determined based on the obtained configuration instruction, and according to the indication of the associated scene, the corresponding scene engine is determined to control the internal lighting system to turn on and adjust the internal lighting to the red light indicated by the configuration instruction. The embodiment of the present application obtains the configuration instruction before matching the target scene, and presets the corresponding associated scene for any attribute information in advance according to the configuration instruction, and presets the target scene with different status information under the associated scene, so as to meet the user's needs of matching personalized scenes for different attribute information and status information, and improve the user experience.

[0142] Figure 3 The structural diagram of the vehicle cockpit control system provided in this application is as follows: Figure 3 As shown, the car cabin control system includes an OMS camera, a cabin domain controller, a regional controller, a sensor and an actuator; wherein the cabin domain controller includes an OMS identification module, an association module and a scene engine module; the OMS camera is communicatively connected to the input end of the cabin domain controller, and is used to transmit the target video of the target time period to the cabin domain controller; the output end of the cabin domain controller is communicatively connected to the actuator through the regional controller; the output end of the cabin domain controller is communicatively connected to the sensor; the cabin domain controller is used to implement any one of the above embodiments.

[0143] In a possible embodiment, the OMS camera captures the image information of the users sitting in the main driving and co-pilot seats in the cockpit, obtains the target video after processing by the deserializer and sends it to the OMS recognition module. After the OMS recognition module obtains the target video, the OMS recognition module performs face recognition and feature extraction based on the image information in the target video, determines the user's attribute information and status information based on the extracted features, sends the attribute information and status information to the association module, and enables the association module to determine the target scene based on the attribute information and status information, and transmits the target scene to the scene engine module, so that the scene engine module determines the corresponding target scene engine according to the target scene and generates the corresponding control instruction, and sends the control instruction to the regional controller through the ETH switch or CAN / LIN transceiver, so as to control the actuator to perform the corresponding operation through the regional controller.

[0144] In one possible embodiment, the actuators include aromatherapy, door / window motors, air conditioning compressors, interior lights, sunroofs, seats, exterior lamps, wheel motors, and other actuators.

[0145] In one possible embodiment, the sensor includes a screen, a power amplifier, a speaker, and a microphone.

[0146] Optionally, the scene engine module generates a corresponding byte stream through a serializer for the corresponding control instruction to directly drive the screen to play the corresponding video.

[0147] Optionally, the scene engine module sends the corresponding control instruction to the power amplifier through the audio bus to directly drive the power amplifier to start the speaker and / or the microphone.

[0148] Optionally, the user inputs configuration instructions through the central control screen interface of the scene engine module to implement a scene engine corresponding to a user-defined preset for different attribute information.

[0149] Optionally, the user connects the user terminal to the car Bluetooth and / or wireless, and after the connection is successful, the configuration instruction is sent to the scene engine module through the user terminal. The technical effect of the configuration instruction provided in this embodiment is similar to the method provided in the above method embodiment, and this embodiment is not repeated here.

[0150] The car cockpit control system of the embodiment of the present application processes the target video taken by the OMS camera through the ship domain controller, and matches the target scene based on the user attribute information and status information obtained after processing, so that the corresponding target scene triggers the alarm to control the corresponding actuator to perform the corresponding operation, thereby improving the accuracy of the target scene and further improving the user experience.

[0151] This embodiment provides a control device for a car cabin, such as Figure 4 FIG. 4 is a schematic diagram of the structure of the control device for the vehicle cockpit provided by the present application. The control device 40 for the vehicle cockpit provided by the present embodiment includes:

[0152] An acquisition module 401 is used to acquire a target video in a target time period, where the target video includes image data of a user in the cockpit;

[0153] Processing module 402, used to determine attribute information of the user according to the target video;

[0154] The processing module 402 is used to determine the status information of the user in the target time period according to the target video;

[0155] The processing module 402 is used to determine a target scene matching the user from a plurality of pre-set scenes based on the attribute information and state information of the user;

[0156] The control module 403 is used to control at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

[0157] In a possible implementation, the processing module 402 is used to determine the eye movement information and the head position information of the user at each time stamp according to the target video;

[0158] Determine whether the eye closure angle at each timestamp is greater than the closure angle threshold according to the eye movement information; if the eye closure angle is greater than the closure angle threshold, determine that the user's eyes are closed at the corresponding timestamp; otherwise, determine that the user's eyes are open at the corresponding timestamp;

[0159] Determine whether the posture angle at each timestamp is greater than the posture angle threshold according to the head position information; if the posture angle is greater than the posture angle threshold, determine that the user posture at the corresponding timestamp is a normal orientation; otherwise, determine that the user posture at the corresponding timestamp is an abnormal orientation;

[0160] The user's state information in the target time period is determined based on the information of the user's eyes and user posture at each time stamp.

[0161] In a possible implementation, the processing module 402 is configured to determine, based on information about the user's eyes at each time stamp, a time interval from each time the user's eyes are closed to the next time the eyes are opened, and based on information about the user's posture at each time stamp, determine a time interval from each time the user's abnormal orientation state to the next time the user's normal orientation state;

[0162] The state information of the user in the target time period is determined according to the time intervals corresponding to the closure of each eye and the time intervals corresponding to the abnormal orientation states.

[0163] In a possible implementation, the processing module 402 is configured to determine, based on the attribute information of the user, an associated scene that matches the attribute information of the user from a plurality of preset scenes;

[0164] According to the user's state information, a target scene matching the state information is determined in the associated scenes.

[0165] In a possible implementation, the control module 403 is used to determine a target scenario engine corresponding to moderate fatigue of the young female user who is the main driver, based on the target scenario of moderate fatigue of the young female user who is the main driver, so as to drive the headrest speaker in the cabin to play preset music and turn on the fragrance to release a preset smell;

[0166] Based on the target scenario of moderate fatigue of the child user on the front passenger seat, a target scenario engine corresponding to moderate fatigue of the child user on the front passenger seat is determined to drive the screen actuator in the cockpit to play the video and drive the speakers in the car to play the pre-set music;

[0167] Based on the target scenario of moderate fatigue of an elderly user on the front passenger seat, a target scenario engine corresponding to the moderate fatigue of an elderly user on the front passenger seat is determined to control the seat actuator to open the footrest, and to control the air-conditioning actuator to adjust the cabin temperature to a preset temperature value.

[0168] In a possible implementation, the processing module 402 is configured to obtain a configuration instruction, where the configuration instruction includes indicated attribute information and a corresponding associated scenario;

[0169] Based on the configuration instruction, a matching relationship is established between the attribute information and the associated scene, wherein the associated scene includes scenes corresponding to the attribute information under different state information.

[0170] The control device for the car cabin provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0171] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0172] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0173] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0174] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0175] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0176] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0177] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0178] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0179] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0180] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0181] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0184] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0185] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0186] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for controlling a car cabin, characterized in that: include: Acquire a target video in a target time period, wherein the target video includes image data of a user in the cockpit; Determining attribute information of the user according to the target video; Determining, according to the target video, status information of the user within the target time period; Based on the attribute information and state information of the user, determining a target scene matching the user from a plurality of pre-set scenes; At least one actuator in the cockpit is controlled to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

2. The method according to claim 1, characterized in that: Determining the state information of the user within the target time period according to the target video includes: Determine eye movement information and head position information of the user at each time stamp according to the target video; determining whether the eye closure angle at each time stamp is greater than a closure angle threshold according to the eye movement information; if the eye closure angle is greater than the closure angle threshold, determining that the user's eyes are closed at the corresponding time stamp, otherwise, determining that the user's eyes are open at the corresponding time stamp; Determine whether the posture angle at each timestamp is greater than a posture angle threshold according to the head position information; if the posture angle is greater than the posture angle threshold, determine that the user posture at the corresponding timestamp is a normal orientation; otherwise, determine that the user posture at the corresponding timestamp is an abnormal orientation; The state information of the user in the target time period is determined according to the information of the user's eyes and the user's posture at each time stamp.

3. The method according to claim 2, characterized in that Determining the state information of the user in the target time period according to the information of the user's eyes and the user's posture at each time stamp includes: Based on the information of the user's eyes at each time stamp, determine the time interval from each time the user's eyes are closed to the next time the eyes are opened, and based on the information of the user's posture at each time stamp, determine the time interval from each time the user's abnormal orientation state to the next normal orientation state; The state information of the user within the target time period is determined according to the time intervals corresponding to the closure of each eye and the time intervals corresponding to the abnormal orientation states.

4. The method according to claim 1, characterized in that: Based on the attribute information and state information of the user, determining a target scene matching the user from a plurality of preset scenes includes: Based on the attribute information of the user, determining an associated scene matching the attribute information of the user from a plurality of preset scenes; According to the state information of the user, a target scene matching the state information is determined in the associated scenes.

5. The method according to claim 4, characterized in that Controlling at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene includes: Based on the target scenario of moderate fatigue of a young female user as the main driver, a target scenario engine corresponding to moderate fatigue of a young female user as the main driver is determined to drive the headrest speakers in the cabin to play pre-set music and turn on the fragrance to release a pre-set scent; Based on the target scenario of moderate fatigue of the child user on the front passenger seat, determining the target scenario engine corresponding to the moderate fatigue of the child user on the front passenger seat, so as to drive the screen actuator in the cockpit to play the video and drive the speakers in the car to play the preset music; Based on a target scenario of moderate fatigue of an elderly user on the co-pilot seat, a target scenario engine corresponding to the moderate fatigue of the elderly user on the co-pilot seat is determined to control the seat actuator to open the footrest, and to control the air-conditioning actuator to adjust the cabin temperature to a preset temperature value.

6. The method according to claim 4, characterized in that Before determining a target scene matching the user from a plurality of preset scenes, the method includes: Obtaining a configuration instruction, wherein the configuration instruction includes indicated attribute information and a corresponding associated scenario; Based on the configuration instruction, a matching relationship is established between the attribute information and the associated scene, wherein the associated scene includes scenes corresponding to the attribute information under different state information.

7. A control device for a car cabin, characterized in that: include: An acquisition module, used to acquire a target video in a target time period, wherein the target video includes image data of a user in the cockpit; A processing module, used to determine attribute information of the user according to the image data of the user; The processing module is used to determine the state information of the user within the target time period according to the target video; The processing module is used to determine a target scene matching the user from a plurality of pre-set scenes based on the attribute information and state information of the user; The control module is used to control at least one actuator in the cockpit to perform an operation matching the target scene through a target scene engine corresponding to the target scene.

8. An automobile cockpit control system, characterized in that: include: OMS camera, cabin domain controller, zone controller, sensor and actuator; wherein the cabin domain controller includes OMS identification module, association module and scene engine module; The OMS camera is communicatively connected to the input end of the cabin domain controller, and is used to transmit the target video of the target time period to the cabin domain controller; The output end of the cabin and mooring domain controller is connected to the actuator through the regional controller; the output end of the cabin and mooring domain controller is connected to the sensor; The berthing domain controller is used to execute the method as described in any one of claims 1-6.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

Images