Vehicle control method and device

By comprehensively analyzing the driver's facial images and physiological data, identifying his emotional state and performing appropriate comfort operations, the problem of low accuracy in the prior art emotional judgment is solved, and driving safety is improved.

CN118953372APending Publication Date: 2024-11-15CHERY NEW ENERGY AUTOMOBILE TECH CO LTD

Patent Information

Application Number
CN202411312610.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-20
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, the accuracy of emotional judgment by analyzing the driver's face image is low, which is prone to misjudgment, which affects driving safety.

Method used

By obtaining multiple facial images and physiological data of the driver during the historical time period, facial feature representations and physiological data are extracted, comprehensive analysis is performed to identify the driver's emotional state, and matching comfort operations are performed when the driver is in negative emotions.

Benefits of technology

It improves the accurate identification of driver's emotions and formulates more accurate comfort measures to help drivers adjust their emotions quickly, thereby improving driving safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118953372A_ABST
    Figure CN118953372A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle control method and device, and relates to the technical field of vehicles. The method comprises the following steps: in a driving process of a first vehicle, acquiring a plurality of face images of a driver of the first vehicle in a historical time period and acquiring physiological data of the driver of the first vehicle; extracting facial feature representations respectively corresponding to the plurality of facial images; recognizing the emotion of the driver based on the plurality of facial feature representations and the physiological data to obtain an emotion recognition result; and under the condition that the emotion recognition result indicates that the driver is in the negative emotion state, the first vehicle is controlled to execute pacifying operation. Based on comprehensive analysis of facial feature representation and physiological data, the emotion state of the driver can be recognized more accurately, more accurate and reasonable pacifying measures can be formulated, and therefore the vehicle is controlled to execute pacifying operation matched with the current negative emotion of the driver, the driver is helped to rapidly adjust the emotion, and the driving safety is improved. Therefore, the driving safety is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of vehicle technology, and in particular to a vehicle control method and device. Background Art

[0002] With the rapid development of science and technology and the improvement of people's living standards, cars have become one of the indispensable means of transportation in modern society. Whether the driver's driving mentality is in a good state when driving the vehicle is crucial to safe driving.

[0003] In the related technology, the driver's facial image can be recognized by a camera, and then the driver's emotions can be judged by analyzing the facial image, so as to promptly remind the driver to keep his emotions stable.

[0004] However, the accuracy of emotion judgment by analyzing the driver's facial image is low, and misjudgment is prone to occur, which in turn affects driving safety. Summary of the invention

[0005] The embodiment of the present application provides a vehicle control method and device, which can improve the accuracy of judging the driver's emotions and formulate a vehicle control scheme adapted thereto, thereby ensuring the safety of vehicle driving. The technical scheme is as follows:

[0006] In one aspect, a vehicle control method is provided, the method comprising:

[0007] During the driving of the first vehicle, acquiring multiple facial images of the driver of the first vehicle within a historical time period and acquiring physiological data of the driver of the first vehicle, wherein the historical time period is used to indicate a time period from a historical moment to a current moment, and the physiological data is used to indicate data related to the physical state of the driver;

[0008] Extracting facial feature representations corresponding to the plurality of facial images respectively;

[0009] Recognizing the driver's emotions based on multiple facial feature representations and the physiological data to obtain an emotion recognition result;

[0010] In a case where the emotion recognition result indicates that the driver is in a negative emotional state, the first vehicle is controlled to perform a soothing operation, where the soothing operation matches the emotion recognition result.

[0011] In another aspect, a vehicle control device is provided, the device comprising:

[0012] an acquisition module, used for acquiring, during the driving process of the first vehicle, a plurality of facial images of the driver of the first vehicle within a historical time period and acquiring physiological data of the driver of the first vehicle, wherein the historical time period is used to indicate a time period between a historical moment and a current moment, and the physiological data is used to indicate data related to the physical state of the driver;

[0013] A recognition module, used to extract facial feature representations corresponding to the plurality of facial images respectively;

[0014] The recognition module is used to recognize the driver's emotions based on multiple facial feature representations and the physiological data to obtain an emotion recognition result;

[0015] A control module is used to control the first vehicle to perform a soothing operation when the emotion recognition result indicates that the driver is in a negative emotional state, and the soothing operation matches the emotion recognition result.

[0016] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement any of the above-mentioned vehicle control methods.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement any of the above-mentioned vehicle control methods.

[0018] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned vehicle control methods.

[0019] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0020] During the driving process of the vehicle, by capturing multiple facial images of the driver within a historical time period and combining them with real-time physiological data, the driver's emotional changes can be continuously and dynamically monitored. The comprehensive analysis based on facial feature representation and physiological data can more accurately identify the driver's emotional state, which helps to formulate more accurate and reasonable soothing measures, thereby controlling the vehicle to perform soothing operations that match the driver's current negative emotions, helping the driver to quickly adjust his emotions and thus improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a structural block diagram of a computer system provided by an exemplary embodiment of the present application;

[0023] Figure 2 is a system schematic diagram provided by an exemplary embodiment of the present application;

[0024] Figure 3 is a flow chart of a vehicle control method provided by an exemplary embodiment of the present application;

[0025] Figure 4 is a flow chart of a vehicle control method provided by another exemplary embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a vehicle control method provided by an exemplary embodiment of the present application;

[0027] Figure 6 is a structural block diagram of a vehicle control device provided by an exemplary embodiment of the present application;

[0028] Figure 7 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the implementation mode of the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0030] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on quantity and execution order.

[0031] It should be noted that the information, data (including but not limited to data for analysis, storage, display, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the framework data involved in this application are all obtained with full authorization.

[0032] With the rapid development of science and technology and the improvement of people's living standards, cars have become one of the indispensable means of transportation in modern society. Whether the driver is in a good state of driving psychology when driving a vehicle is crucial to safe driving. In related technologies, the driver's facial image can be recognized by a camera, and then the driver's emotions can be judged by analyzing the facial image, so as to remind the driver to keep his emotions stable in time. However, the accuracy of emotion judgment by analyzing the driver's facial image is low, and it is easy to make misjudgments, which in turn affects driving safety.

[0033] Based on this, the present application provides a vehicle control method that can improve the accuracy of driver emotion judgment and formulate a vehicle control plan adapted thereto, thereby ensuring the safety of vehicle driving.

[0034] First, the computer system for executing the vehicle control method provided in the present application is introduced.

[0035] Figure 1 The structure block diagram of a computer system 100 provided by an exemplary embodiment of the present application is shown. The computer system 100 can be implemented as a system architecture of a vehicle control method. The computer system 100 includes: a vehicle 110.

[0036] The vehicle 110 includes at least one of a fuel vehicle, an electric vehicle, a hybrid vehicle (a combination of a fuel vehicle and an electric vehicle), a fuel cell vehicle, a solar vehicle, etc., which is not limited in the embodiments of the present application.

[0037] In some embodiments, the computer system 100 further includes a terminal 120 .

[0038] The terminal 120 includes but is not limited to a vehicle-mounted terminal, a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, etc. In some embodiments, the terminal 120 is implemented as a mobile control terminal of the vehicle 110, and the terminal 120 can control the vehicle 110 to perform a target operation (for example, light control, parking, closing windows, starting, etc.). Schematically, a client with a target application is installed and running in the terminal 120, and the target application includes at least one of the applications with vehicle control functions such as vehicle control applications, instant messaging applications, and navigation applications (for example, map applications). This embodiment of the application does not limit this. In addition, the application does not limit the form of the target application, including but not limited to App (Application, application) installed in the terminal 120, applets, etc., and can also be in the form of a web page.

[0039] Optionally, the vehicle 110 and the terminal 120 communicate via a wireless network (such as 4G / 5G, Wi-Fi, etc.) or a wired connection (such as USB, Bluetooth, etc.).

[0040] In some embodiments, the computer system 100 further includes a server 130, which can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. Optionally, the server 130 can also be implemented as a node in a blockchain system.

[0041] Illustratively, if the server 130 is implemented as a cloud server, the vehicle 110 and the server 130 communicate via a wireless network, and the terminal 120 and the server 130 communicate via a wireless network.

[0042] Optionally, the vehicle control method provided in the embodiment of the present application can be executed by the vehicle 110, or by the terminal 120, or by the server 130; or, it can be interactively executed by any two devices among the vehicle 110, the terminal 120, and the server 130; or, it can be interactively executed by the vehicle 110, the terminal 120, and the server 130, which is not limited by the embodiment of the present application.

[0043] The vehicle 110 is used as an example to illustrate the vehicle control method. Figure 2 , which shows a schematic diagram of a vehicle system for executing a vehicle control method, such as Figure 2 As shown, the vehicle includes a vehicle controller 210, a display host 220, a body domain controller 230 and an audio system 240.

[0044] Among them, the vehicle controller 210 is used to receive the facial image and physiological data of the driver of the first vehicle, and is also used to analyze the facial image and physiological data; the display host 220 is used to receive the control signal from the vehicle controller 210 and activate the corresponding driving mode, such as: sad driving mode; the body domain controller 230 is used to receive the control signal from the display host 220, so as to control the device (for example: audio 240) to perform the corresponding operation. The domain controller integrates distributed electronic control units with similar or related functions to form a more powerful control unit, thereby realizing centralized control of a certain area or a certain functional domain of the vehicle.

[0045] Schematically, during the driving process of the vehicle, multiple facial images of the driver of the first vehicle are obtained through an image acquisition device installed in the vehicle (for example, a camera), and the physiological data of the driver of the first vehicle is collected through a sensor on the steering wheel of the vehicle. After the facial images and physiological data are obtained, they are distributed to the vehicle controller 210. The vehicle controller 210 first extracts the facial feature representations corresponding to the multiple facial images, and then identifies the driver's emotions based on the multiple facial feature representations and physiological data to obtain an emotion recognition result. When the emotion recognition result belongs to a negative emotion, for example, a sad emotion, a "sad driving signal" is generated. The vehicle controller 210 sends the "sad driving signal" to the display host 220. After receiving the signal, the display host 220 determines a control signal. The control signal is used to control the vehicle to perform a soothing operation matching the driver's sad emotion. Taking the soothing operation as voice soothing as an example, the display host 220 sends the control signal to the body domain controller 230. After receiving the control signal, the body domain controller 230 forwards it to the audio 240. After receiving the control signal, the audio 240 plays a soothing voice according to the control signal to stabilize the driver's emotion.

[0046] In summary, the vehicle control method provided in the embodiment of the present application can continuously and dynamically monitor the driver's emotional changes during the driving process of the vehicle by capturing multiple facial images of the driver within a historical time period and combining them with real-time physiological data. The comprehensive analysis based on facial feature representation and physiological data can more accurately identify the driver's emotional state, which helps to formulate more accurate and reasonable soothing measures, thereby controlling the vehicle to perform soothing operations that match the driver's current negative emotions, helping the driver to quickly adjust his emotions, thereby improving driving safety.

[0047] Secondly, the process of the vehicle control method provided in the embodiment of the present application is explained.

[0048] Combined with the above introduction, Figure 3 is a flow chart of a vehicle control method provided in an embodiment of the present application, and the method is applied to Figure 1Taking the vehicle 110 as an example, the method is described as follows: steps 310 to 340.

[0049] Step 310 , during the driving process of the first vehicle, a plurality of facial images of the driver of the first vehicle within a historical time period and physiological data of the driver of the first vehicle are obtained.

[0050] The historical time period is used to indicate the time period between the historical moment and the current moment. In an illustrative example, if the historical time period is 2 minutes long, multiple facial images of the driver of the first vehicle within the last 2 minutes are acquired.

[0051] Optionally, during the historical time period, multiple facial images of the driver of the first vehicle are collected by an image acquisition device installed in the first vehicle. The image acquisition device includes at least one of a vehicle-mounted camera, an infrared camera, or a depth camera, etc., which are not limited here.

[0052] Illustratively, during vehicle driving, the image acquisition device will continuously or intermittently capture the driver's facial image, and then send the facial image to the vehicle controller of the first vehicle. After obtaining the facial image, the vehicle controller can pre-process it, where the pre-processing includes at least one of image denoising, grayscale, normalization, face detection and alignment, etc., which is not limited here.

[0053] Physiological data is used to indicate data related to the driver's physical state. Optionally, the physiological data includes at least one of the driver's heart rate, body temperature, and skin conductance, wherein the heart rate refers to the number of heart beats per minute, the body temperature refers to the temperature inside the human body, and the skin conductance is a physiological parameter that measures the process of electrical signal transmission between two points on the skin surface (skin conductance is an indicator of the amount of skin sweating).

[0054] In some embodiments, physiological data of the driver of the first vehicle may be collected via sensors installed on the first vehicle.

[0055] Optionally, physiological data of the driver of the first vehicle collected by a first sensor installed in a steering wheel of the first vehicle is obtained.

[0056] Schematically, at least one first sensor may be installed on the steering wheel of the first vehicle. The types of the first sensor include an electrocardiogram sensor, a body temperature sensor, and a skin conductance sensor, among which the electrocardiogram sensor is used to collect the driver's heart rate data, the infrared body temperature sensor is used to collect the driver's body temperature, and the skin conductance sensor is used to collect the driver's skin conductance. After the first sensor collects the driver's physiological data, it sends the collected physiological data to the vehicle controller of the first vehicle.

[0057] It should be noted that the above examples of the installation position of the first sensor for collecting physiological data on the first vehicle are only illustrative. For example, an infrared temperature sensor can be installed on the seat back of the main driver to collect the driver's body temperature.

[0058] In other embodiments, the physiological data of the driver of the first vehicle may be collected through sensors on a terminal device connected to the first vehicle.

[0059] Optionally, a communication connection is established between the first vehicle and a wearable device worn by a driver to obtain physiological data of the driver of the first vehicle collected by a second sensor installed in the wearable device.

[0060] Among them, wearable devices include smart watches, health bracelets, smart headphones, etc. At least one second sensor is installed in the wearable device for collecting the driver's physiological data. The types of second sensors include electrocardiogram sensors, body temperature sensors, and skin conductance sensors, etc. The wearable device continuously monitors the driver's physiological state and transmits the collected physiological data to the vehicle controller of the first vehicle in real time through a communication connection.

[0061] Step 320: extract facial feature representations corresponding to the plurality of facial images.

[0062] Optionally, facial features are represented to reflect the driver's emotional state.

[0063] In some embodiments, the driver's emotions can be identified by an emotion recognition model to obtain an emotion recognition result. Optionally, multiple facial images are input into the emotion recognition model, and facial feature representations corresponding to the multiple facial images are extracted by the emotion recognition model.

[0064] Step 330 , identifying the driver's emotions based on multiple facial feature representations and physiological data to obtain an emotion recognition result.

[0065] Optionally, the facial feature representations and physiological data corresponding to the plurality of facial images are analyzed by an emotion recognition model to predict the driver's emotions and obtain an emotion recognition result.

[0066] Schematically, the physiological data feature representation corresponding to the physiological data is extracted through the emotion recognition model, and the physiological data feature representation and the image feature representation are spliced ​​to obtain a fused feature representation, and the fused feature representation is analyzed by the emotion recognition model to obtain an emotion recognition result. Alternatively, the physiological data feature representation is analyzed by the emotion recognition model to obtain a first recognition result, and the image feature representation is analyzed by the emotion recognition model to obtain a second recognition result, and finally the emotion recognition result is determined based on the first recognition result and the second recognition result. For example, the first recognition result and the second recognition result can be scoring results, and the scoring result indicates the possibility of negative emotions. The scoring result can be between 0 and 1, where 0 indicates no negative emotions at all and 1 indicates extremely high negative emotions. After obtaining the first recognition result and the second recognition result, the first recognition result and the second recognition result are weighted and summed to obtain the final emotion recognition result.

[0067] In some embodiments, the emotion recognition result generally refers to negative emotions.

[0068] Illustratively, the emotion recognition result does not indicate which negative emotion it is. For example, the emotion recognition result output by the emotion recognition model may be "negative emotion, 0.9", where 0.9 is a score indicating that the driver's current emotion is likely to be negative.

[0069] In some embodiments, the emotion recognition result specifically refers to a certain negative emotion.

[0070] In schematic form, the emotion recognition result refers to a specific message emotion, such as sadness, anger, frustration, fear, etc. Taking sadness as an example, the emotion recognition result output by the emotion recognition model may be "sadness, 0.9", where 0.9 is a score indicating that the driver's current emotion is likely to be sad.

[0071] Optionally, the above-mentioned emotion recognition model is a pre-trained machine learning model, which schematically obtains multiple facial images and physiological data of the driver under a reference emotion (the reference emotion is a negative emotion), inputs these multiple facial images and physiological data into a sample emotion recognition model, extracts image feature representations corresponding to the multiple facial images respectively through the sample emotion recognition model, and extracts physiological data feature representations of the physiological data through the sample emotion recognition model.

[0072] The image feature representation and the physiological data feature representation are spliced ​​to obtain a fused feature representation, and the fused feature representation is analyzed by a sample emotion recognition model to obtain a predicted emotion recognition result; or, the physiological data feature representation is analyzed by a sample emotion recognition model to obtain a first recognition result, and the image feature representation is analyzed by a sample emotion recognition model to obtain a second recognition result, and a predicted emotion recognition result is obtained based on the first recognition result and the second recognition result.

[0073] After obtaining the predicted emotion recognition result, the target loss is determined based on the difference between the predicted emotion recognition result and the reference emotion, and then the model parameters of the sample emotion recognition model are updated according to the target loss. The training is iterated multiple times until the target loss is less than the preset loss or the number of training times reaches the preset number, and the training is stopped. The sample emotion recognition model obtained at this time is the trained emotion recognition model.

[0074] In some embodiments, taking into account the possible differences in facial expressions and emotional expressions of different drivers, a personalized emotion recognition model can be established. By collecting and analyzing historical data of a specific driver (including facial images and physiological data), a personalized emotion recognition model suitable for the driver can be trained, thereby improving the accuracy and specificity of emotion recognition.

[0075] Step 340: When the emotion recognition result indicates that the driver is in a negative emotion, control the first vehicle to perform a soothing operation.

[0076] The soothing operation is matched with the emotion recognition result. Optionally, different negative emotions correspond to different soothing operations, for example, the soothing operation corresponding to the sad emotion is different from the soothing operation corresponding to the angry emotion.

[0077] Illustratively, when the vehicle controller of the first vehicle determines that the driver is in a negative emotion, the vehicle controller sends a negative emotion signal to the display screen host of the first vehicle. After receiving the negative emotion signal, the display screen host activates the negative driving mode. In the negative driving mode, the display screen host obtains a soothing strategy that matches the emotion recognition result, and generates a corresponding control signal based on the soothing strategy. The display screen host sends the control signal to the body domain controller of the first vehicle. After receiving the control signal, the body domain controller forwards it to the corresponding execution device on the first vehicle, such as audio, indicator light, etc., according to the operation type of the soothing operation indicated by the control signal.

[0078] The soothing operation includes at least one of voice soothing, speed control, assisted driving, emergency contact, in-car atmosphere adjustment, etc. Voice soothing refers to controlling the first vehicle to play soothing voice, speed control refers to controlling the speed of the first vehicle to be within a preset range, assisted driving includes automatic lane keeping, adaptive cruise control, etc., emergency contact refers to automatically contacting an emergency contact or relevant agency, and in-car atmosphere adjustment refers to automatically adjusting environmental factors such as light, temperature and smell in the car.

[0079] In some embodiments, taking the soothing operation as voice soothing as an example, when the emotion recognition result indicates that the driver is in a negative emotion, a voice control signal is generated; and the voice control signal is used to control the audio system of the first vehicle to play the soothing voice that matches the emotion recognition result.

[0080] In principle, if it is confirmed that the driver is in a negative emotion, a corresponding voice control signal is generated based on the specific results of emotion recognition (such as negativity, sadness, anger, anxiety, etc.). The voice control signal contains detailed information on soothing voice content, timbre, volume and other parameters. After the first vehicle's audio system receives the voice control signal, it will parse the soothing voice content to be played based on the voice control signal, and adjust the volume, timbre and other parameters of the audio system to meet the playback requirements of the soothing voice.

[0081] Optionally, when the emotion recognition result indicates that the driver is in a negative emotion, the driver's negative emotion level is determined; a soothing text and a target timbre are determined based on the negative emotion level; a voice control signal is generated according to the soothing text and the target timbre, and the voice control signal is used to control the audio of the first vehicle to play a soothing voice matching the soothing text with the target timbre.

[0082] Among them, the negative emotion level is used to indicate the degree of negativity in the driver's emotional state. Schematically, when it is determined that the driver is in a negative emotion, multiple facial images and physiological data of the driver can be input into the emotion grading model, wherein the emotion grading model is a pre-trained machine learning model. The emotion grading model can identify the degree of negativity in the driver's emotional state and output the negative emotion level, wherein the higher the negative emotion level, the higher the degree of negativity in the driver's emotional state, and the more it affects the driver's driving state.

[0083] After determining the negative emotion level, the corresponding soothing text and target timbre can be determined according to the negative emotion level, and the soothing text and target timbre corresponding to different negative emotion levels are different. Schematically, it is assumed that the negative emotion level includes three levels: slightly negative, moderately negative, and highly negative, where different negative emotion levels correspond to different soothing texts and target timbres. Optionally, the soothing text and target timbre are pre-configured, or corresponding prompt information (promot) is generated according to the negative emotion level, and the prompt information is input into the large language model. The large language model can automatically generate soothing text matching the negative emotion level under the guidance of the prompt information, wherein the large language model is a pre-trained generative model.

[0084] After determining the soothing text and target timbre, the display screen host of the first vehicle encodes the soothing text and target timbre to obtain a voice control signal. The display screen host sends the voice control signal to the body domain controller of the first vehicle. After receiving the voice control signal, the body domain controller sends the voice control signal to the audio. After receiving the voice control signal, the audio parses the soothing text and target timbre therein, and plays the soothing voice.

[0085] In other embodiments, taking the soothing operation as vehicle speed control as an example, when the emotion recognition result indicates that the driver is in a negative emotion, determining the negative emotion level of the driver; when the negative emotion level meets the preset speed limit requirement, obtaining the driving speed of the first vehicle; when the driving speed is greater than the preset speed indicated by the preset speed limit requirement, generating a speed control signal; and controlling the first vehicle to decelerate through the speed control signal until the driving speed of the first vehicle is less than or equal to the preset speed.

[0086] The negative emotion level is used to indicate the degree of negativity in the driver's emotional state. The specific method for determining the negative emotion level can be found above and will not be described in detail here.

[0087] After determining the negative emotion level, if the negative emotion level reaches the preset limit level, such as the negative emotion level is moderately negative, it is determined that the negative emotion level meets the preset speed limit requirement. At this time, the driving speed of the first vehicle is obtained and compared with the preset speed indicated in the preset speed limit requirement, where different negative emotion levels correspond to different preset speeds, such as moderate negativity corresponds to the first preset speed, and high negativity corresponds to the second preset speed, and the first preset speed is greater than the second preset speed.

[0088] If the driving speed of the first vehicle is greater than the preset speed, it is considered that the current driver's driving behavior poses a safety hazard and requires immediate intervention. At this time, the display host can generate a speed control signal and send it to the brake controller of the first vehicle. After receiving the speed control signal, the brake controller controls the brakes of the first vehicle to achieve deceleration control of the first vehicle. The deceleration process will continue until the driving speed of the first vehicle is reduced to less than or equal to the preset speed. Therefore, when the driver is in a negative emotion, driving safety can be ensured by automatic deceleration.

[0089] Optionally, the preset speed can also be determined according to the type of road on which the first vehicle is currently traveling. Schematically, if the current road type is a highway, the preset speed can be set to a higher speed limit, such as the common 100 km / h or higher (the specific value is subject to local laws and regulations); if the vehicle is currently traveling on a city trunk road or expressway, the preset speed will be slightly lower, such as 80 km / h or 60 km / h; if the vehicle is traveling on an ordinary street or residential area, the preset speed needs to be further reduced to ensure the safety of pedestrians and non-motor vehicles, and can be set to 40 km / h or lower.

[0090] In other embodiments, when the emotion recognition result indicates that the driver is in a negative emotional state, the driver's negative emotion level is determined; when the negative emotion level meets the preset restriction requirements, the current driving environment information of the first vehicle is obtained, and the driving environment information includes the driving section and the driving road condition; when the driving environment information meets the first environmental requirements, the driving speed of the first vehicle is obtained; when the driving speed is greater than the preset speed indicated by the preset speed limit requirement, the first vehicle is controlled to decelerate until the driving speed of the first vehicle is less than or equal to the preset speed; when the driving environment information meets the second environmental requirements, the first vehicle is controlled to start the automatic driving mode; when the driving environment information meets the third environmental requirements, the display screen of the first vehicle is controlled to display a recommended driving route, and the recommended driving route is used to recommend the first vehicle to drive to the target parking area.

[0091] The negative emotion level is used to indicate the degree of negativity in the driver's emotional state. The specific method for determining the negative emotion level can be found above and will not be described in detail here.

[0092] After determining the negative emotion level, if the negative emotion level reaches the preset limit level, such as the negative emotion level is moderately negative, it is determined that the negative emotion level meets the preset limit requirements. At this time, the driving environment information of the first vehicle is obtained, such as the current driving section and driving conditions of the first vehicle, where the driving section is used to indicate the road type of the road on which the first vehicle is currently driving, and the driving sections include highways, urban roads, school sections, etc.; the road condition information includes the traffic conditions of the road on which the first vehicle is currently driving, such as the degree of traffic congestion, traffic accident information, construction obstacle information, weather information, etc.

[0093] After obtaining the current driving section and driving road conditions of the first vehicle, if the current driving section of the first vehicle is a non-highway, such as a city road, a school section, etc., it is determined that the driving environment information meets the first environment requirements, and then the driving speed of the first vehicle is obtained and compared with the preset speed, wherein different negative emotion levels correspond to different preset speeds, such as moderate negativity corresponds to the first preset speed, and high negativity corresponds to the second preset speed, and the first preset speed is greater than the second preset speed. If the driving speed of the first vehicle is greater than the preset speed, the brake of the first vehicle is controlled to work so as to achieve deceleration control of the first vehicle, and the deceleration process will continue until the driving speed of the first vehicle is reduced to less than or equal to the preset speed. Optionally, the preset speed can also be determined according to the current driving section of the first vehicle. Schematically, if the current driving section is on a city trunk road or expressway, the preset speed will be slightly lower, such as 80 kilometers per hour or 60 kilometers per hour; if it is driving on an ordinary street or residential area, the preset speed needs to be further reduced to ensure the safety of pedestrians and non-motor vehicles, and can be set at 40 kilometers per hour or lower.

[0094] If the current driving section of the first vehicle is a highway, and the driving road condition of the first vehicle meets the first road condition requirement corresponding to the automatic driving (for example: the current traffic congestion on the highway is low, there are no traffic accidents, no construction, etc.), it is determined that the driving environment information meets the second environment requirement, which is used to control the first vehicle to enter the automatic driving mode. The automatic driving mode includes adaptive cruise control, lane keeping assist, automatic lane change, etc., which are not limited here. Optionally, before the first vehicle enters the automatic driving mode, it can request the driver to start the automatic driving mode through a display screen, sound prompt or other means. After the driver confirms, the first vehicle enters the automatic driving mode.

[0095] If the current driving road condition of the first vehicle meets the second road condition requirement, for example: the traffic congestion level of the current driving section of the first vehicle is high or the weather conditions of the current driving section of the first vehicle are poor (such as rain, snow, haze, strong wind, etc.) or the first vehicle is currently driving at night, it is determined that the driving environment information meets the third environment requirement. At this time, the first vehicle is controlled to display recommended route information. The recommended route information is used to guide the driver to drive the first vehicle to the nearest service area or other parking area, that is, the target parking area.

[0096] In some embodiments, a display component is installed on the exterior of the first vehicle. Schematically, the display component installed on the exterior of the first vehicle can be implemented as an LED (Light Emitting Diode) screen, a color-changing lighting system or other form of visualization device, which is not limited here.

[0097] Optionally, when the emotion recognition result indicates that the driver is in a negative emotion, a display control signal is generated; the display control signal is used to control a display component installed on the outside of the first vehicle to display a warning message, and the warning message is used to warn the driver of the first vehicle that he is in a negative emotion.

[0098] Schematically, when it is detected that the driver is in a negative emotion, the display host will automatically generate a display control signal and send it to the body domain controller. The body domain controller sends the display control signal to a display component installed on the outside of the first vehicle, and a warning message is displayed through the display component. When the display component is implemented as an LED screen, the warning message can be implemented as "The driver is emotionally unstable, please avoid carefully" to alert surrounding vehicles and pedestrians and reduce the risk of traffic accidents.

[0099] It should be noted that before obtaining the facial image and physiological data of the user (such as the driver), the present application will display an information authorization interface, and an information authorization agreement will be displayed in the information authorization interface; the information authorization agreement mainly explains the facial images and physiological data that need to be obtained and the purpose of the facial images and physiological data; only when the user agrees to the agreement can the acquisition operation be successfully executed, thereby executing the vehicle control method.

[0100] In summary, the embodiments of the present application provide a vehicle control method. During the driving process of the vehicle, by capturing multiple facial images of the driver within a historical time period and combining them with real-time physiological data, the driver's emotional changes can be continuously and dynamically monitored. Among them, based on the comprehensive analysis of facial feature representation and physiological data, the driver's emotional state can be more accurately identified, which helps to formulate more accurate and reasonable soothing measures, thereby controlling the vehicle to perform soothing operations that match the driver's current negative emotions, helping the driver to quickly adjust his emotions, thereby improving driving safety.

[0101] In some embodiments, the negative emotion includes sadness. Figure 4 , the above Figure 3 The illustrated embodiment may also be implemented as steps 410 to 440 described below.

[0102] Step 410 , during the driving process of the first vehicle, a plurality of facial images of the driver of the first vehicle within a historical time period and physiological data of the driver of the first vehicle are obtained.

[0103] The historical time period is used to indicate the time period between the historical moment and the current moment, and the physiological data is used to indicate data related to the driver's physical state.

[0104] Illustratively, during a historical time period, a plurality of facial images of a driver of a first vehicle are acquired based on a preset acquisition time interval, and physiological data of the driver of the first vehicle is acquired based on a preset acquisition time interval.

[0105] The physiological data is used to indicate data related to the driver's physical state. Optionally, the physiological data includes at least one of the driver's heart rate, body temperature, and skin conductance.

[0106] Step 420: extract facial feature representations corresponding to the plurality of facial images.

[0107] Optionally, the above-mentioned emotion recognition model can be implemented as a sadness emotion recognition model, and multiple facial images are input into the sadness emotion recognition model, and facial feature representations corresponding to the multiple facial images are extracted through the sadness emotion recognition model.

[0108] Among them, the facial feature representation is a feature associated with sad emotions, for example, the facial feature representation includes the shape of the eyes, the position of the eyebrows, the degree of opening and closing of the mouth, etc.

[0109] Step 431 , based on the multiple facial feature representations, determine the frequency of occurrence of the target facial action of the driver in the historical time period.

[0110] The target facial action is an action associated with a sad emotion, and the target facial action includes at least one of a squinting action, a mouth opening action, and a head turning action.

[0111] Illustratively, multiple facial feature representations are analyzed through the sadness emotion recognition model to identify target facial actions in multiple facial images. After determining the target facial actions in the multiple facial images, the frequency of occurrence of the same type of target facial actions in a historical time period is counted, such as squinting frequency, mouth opening frequency, head turning frequency, etc.

[0112] Step 432, obtaining a data threshold corresponding to the physiological data, and comparing the physiological data with the data threshold to obtain a comparison result.

[0113] Optionally, different types of physiological data correspond to different data thresholds, and the difference between the physiological data and the data threshold is determined as the comparison result. The data threshold represents the physiological data of the driver in a calm state, and the difference reflects the degree of deviation of the driver's current sad emotional state from the calm state.

[0114] In schematic form, the heart rate threshold, body temperature threshold and skin conductance threshold are obtained, the driver's heart rate is compared with the heart rate threshold to obtain the heart rate difference, the driver's body temperature is compared with the body temperature threshold to obtain the body temperature difference, the driver's skin conductance is compared with the skin point conductance threshold to obtain the skin conductance difference, and the heart rate difference, body temperature difference and skin conductance difference are used as comparison results.

[0115] Step 433: Analyze the occurrence frequency and comparison results to obtain emotion recognition results.

[0116] The emotion recognition result is used to indicate whether the driver is in a sad mood.

[0117] Optionally, the occurrence frequency and comparison results are analyzed by a sadness emotion recognition model to obtain an emotion recognition result.

[0118] In some embodiments, a first weight corresponding to the target facial action and a second weight corresponding to the physiological data are obtained, wherein the first weight is used to indicate the degree of association between the target facial action and the sad emotion, and the second weight is used to indicate the degree of reflection of the sad emotion by the physiological data; a first sadness score is obtained based on the first weight and the frequency of occurrence, and a second sadness score is obtained based on the second weight and the comparison result; when the first sadness score and the second sadness score meet the scoring requirements for the sad emotion, a first emotion recognition result is obtained, and the first emotion recognition result indicates that the driver is in a sad emotion.

[0119] The sadness score is used to indicate the severity of the driver's sadness. Schematically, a sadness score of 0-1 may be used to represent the sadness score, wherein a sadness score of 0 represents no sadness, and a sadness score of 1 represents extreme sadness.

[0120] Schematically, the frequency of occurrence is analyzed by the sadness emotion recognition model to obtain a first predicted sadness score, the first predicted sadness score is weighted by a first weight to obtain a first sadness score, the comparison results are analyzed by the sadness emotion recognition model to obtain a second predicted sadness score, and the second predicted sadness score is weighted by a second weight to obtain a second sadness score.

[0121] After obtaining the first sadness score and the second sadness score, the average value / maximum value / minimum value of the first sadness score and the second sadness score are calculated. When the value is greater than the preset sadness value, it is determined that the first sadness score and the second sadness score meet the scoring requirements of the sad emotion, and the first emotion recognition result is obtained; or, a first threshold and a second threshold are determined. When the first sadness score is greater than or equal to the first threshold and the second sadness score is greater than or equal to the second threshold, it is determined that the first sadness score and the second sadness score meet the scoring requirements of the sad emotion, and the first emotion recognition result is obtained.

[0122] Step 440 , when the emotion recognition result indicates that the driver is in a sad mood, control the first vehicle to perform a soothing operation.

[0123] Optionally, when the emotion recognition result is the first emotion recognition result, the emotion recognition result indicates that the driver is in a sad mood, and the first vehicle is controlled to perform a soothing operation.

[0124] Among them, the soothing operation is matched with the sad emotion.

[0125] In an illustrative manner, when it is determined that the driver is in a sad mood, the vehicle audio system is controlled to automatically select and play some soft and soothing music tracks, and the ambient light in the car is adjusted to a soft and warm tone, such as light yellow or light blue. The driver's seat can also be adjusted to a more comfortable position, and the temperature in the car can be adjusted to the temperature range preferred by the driver. Among them, a personalized comfort plan can be formulated based on the driver's historical behavior and preferences. For example, if the driver has a preference for a specific type of music, the specific type of music can be played first.

[0126] In some embodiments, the corresponding soothing operation may also be determined in combination with driving behavior data such as the speed change of the first vehicle, the vehicle operation data of the driver on the first vehicle, and the driving time of the driver.

[0127] In schematic form, vehicle speed changes include acceleration, deceleration, and constant speed driving; vehicle operation data include steering wheel angle, brake and accelerator pedal pressure and frequency, gear shifting operations, etc.; driving time refers to the time the driver drives continuously.

[0128] Optionally, the driving behavior data is input into a driving behavior recognition model, and a driving abnormality recognition result is output, wherein the driving behavior recognition model is a pre-trained machine learning model, and the driving behavior recognition model is used to recognize whether the current driver's driving behavior is abnormal, such as whether the driver frequently changes lanes, whether the driver accelerates / decelerates suddenly, whether the driver drives continuously for a long time, etc. Optionally, when the driving abnormality recognition result is the first driving abnormality recognition result, it is determined that the driver's driving behavior is abnormal.

[0129] Illustratively, when it is determined that the driver is in a sad mood and the driver's driving behavior is abnormal, in addition to the above-mentioned soothing operations (that is, controlling the vehicle's audio to automatically select and play some soft, soothing music tracks, and adjusting the interior ambient light to soft, warm tones, such as light yellow or light blue, the driver's seat can also be adjusted to a more comfortable position, and the interior temperature can be adjusted to the driver's preferred temperature range), suggestion information can be displayed on the central control display screen of the first vehicle, such as suggesting that the driver stop and rest, or contact an emergency contact or rescue service.

[0130] In summary, the embodiment of the present application provides a vehicle control method, which can continuously and dynamically monitor the driver's emotional changes by capturing multiple facial images of the driver in a historical time period and combining them with real-time physiological data during the driving process of the vehicle. The comprehensive analysis based on facial feature representation and physiological data can more accurately identify the driver's emotional state, which helps to formulate more accurate and reasonable soothing measures, thereby controlling the vehicle to perform soothing operations that match the driver's current sad emotions, helping the driver to quickly adjust his emotions, thereby improving driving safety. In particular, when identifying the driver's emotional state, the frequency of facial movements and the comparison results of physiological data are comprehensively considered, and the driver's emotional recognition results are obtained through comprehensive judgment, thereby improving the accuracy and reliability of emotion recognition.

[0131] Schematically, the process of the vehicle control method provided in the embodiment of the present application is described by taking the recognition of sad emotions as an example. Please refer to Figure 5 , which shows a schematic diagram of a vehicle control method, the method comprising the following steps:

[0132] Step 501, obtaining a facial image of a driver and extracting facial feature representations in the facial image.

[0133] Schematically, the driver's facial expression is captured in real time by the camera of the first vehicle to obtain a facial image (i.e., facial image), and multiple facial images acquired within a historical time period are sent to the vehicle controller, and the facial feature representations corresponding to the multiple facial images are extracted by the vehicle controller.

[0134] Step 502, calculating the number of times the driver squints, opens his mouth, and turns his head.

[0135] Schematically, after the vehicle controller extracts the facial feature representation, it recognizes the driver's squinting movements, mouth opening movements and head turning movements in multiple face images, and then counts the number of times the driver squinted, opened his mouth and turned his head in a historical time period.

[0136] Step 503: collecting the driver's heart rate and body temperature.

[0137] Schematically, a sensor is installed on the steering wheel of the first vehicle. When the driver puts his hands on the steering wheel while driving, the sensor on the steering wheel can collect the driver's heart rate and body temperature, and then send the collected heart rate and body temperature to the vehicle controller.

[0138] Step 504 : The vehicle controller compares the collected data with the system threshold to obtain a comparison result.

[0139] Illustratively, the driver's heart rate is compared with a heart rate threshold to obtain a heart rate difference, and the driver's body temperature is compared with a body temperature difference to obtain a body temperature difference, and the heart rate difference and the body temperature difference are used as comparison results.

[0140] Step 505, determining whether the driver is sad.

[0141] Illustratively, after the vehicle controller determines the number of times the driver squints, opens his mouth, and turns his head, and determines the comparison result, it analyzes it to determine whether the driver is sad. The specific analysis process can be referred to step 440 or step 340, which will not be repeated here.

[0142] Step 506: If it is determined that the driver is sad, a buzzer sounds for 5 seconds.

[0143] In schematic form, if the vehicle controller determines that the driver is in a sad mood, it generates a "sad driving signal". The vehicle controller sends the "sad driving signal" to the display host. After receiving the signal, the display host determines the control signal. The control signal is used to control the vehicle to perform a soothing operation that matches the driver's sad mood. Taking the soothing operation as voice soothing as an example, the display host sends the control signal to the body domain controller. After receiving the control signal, the body domain controller forwards it to the audio system. After receiving the control signal, the audio system plays a soothing voice according to the control signal to stabilize the driver's emotions. For example, a 5-second beep allows the driver to instantly concentrate and drive well to achieve the purpose of safe travel.

[0144] Figure 6 is a structural block diagram of a vehicle control device provided by an exemplary embodiment of the present application, such as Figure 6 As shown, the device includes the following parts:

[0145] an acquisition module 610, configured to acquire, during the driving process of the first vehicle, a plurality of facial images of the driver of the first vehicle within a historical time period and to acquire physiological data of the driver of the first vehicle, wherein the historical time period is used to indicate a time period between a historical moment and a current moment, and the physiological data is used to indicate data related to the physical state of the driver;

[0146] A recognition module 620, configured to extract facial feature representations corresponding to the plurality of facial images;

[0147] The recognition module 620 is used to recognize the driver's emotions based on multiple facial feature representations and the physiological data to obtain an emotion recognition result;

[0148] The control module 630 is used to control the first vehicle to perform a soothing operation when the emotion recognition result indicates that the driver is in a negative emotion, and the soothing operation matches the emotion recognition result.

[0149] In some embodiments, the negative emotional state includes sadness; the recognition module 620 is used to determine the frequency of occurrence of the driver's target facial action within the historical time period based on the multiple facial feature representations, the target facial action is an action associated with the sadness, and the target facial action includes at least one of squinting, opening the mouth, and turning the head; obtain a data threshold corresponding to the physiological data, and compare the physiological data with the data threshold to obtain a comparison result; analyze the occurrence frequency and the comparison result to obtain the emotion recognition result, and the emotion recognition result is used to indicate whether the driver is in a sad emotion.

[0150] In some embodiments, the recognition module 620 is used to obtain a first weight corresponding to the target facial action and a second weight corresponding to the physiological data, wherein the first weight is used to indicate the degree of association between the target facial action and the sad emotion, and the second weight is used to indicate the degree to which the physiological data reflects the sad emotion; a first sadness score is obtained based on the first weight and the occurrence frequency, and a second sadness score is obtained based on the second weight and the comparison result; when the first sadness score and the second sadness score meet the scoring requirements of the sad emotion, a first emotion recognition result is obtained, and the first emotion recognition result indicates that the driver is in a sad emotion.

[0151] In some embodiments, the physiological data includes at least one of the driver's heart rate, body temperature, and skin conductance.

[0152] In some embodiments, the acquisition module 610 is used to obtain physiological data of the driver of the first vehicle collected by a first sensor installed in the steering wheel of the first vehicle; or, the first vehicle establishes a communication connection with a wearable device worn by the driver to obtain physiological data of the driver of the first vehicle collected by a second sensor installed in the wearable device.

[0153] In some embodiments, the control module 630 is used to generate a voice control signal when the emotion recognition result indicates that the driver is in a negative emotional state; and control the audio system of the first vehicle to play a soothing voice that matches the emotion recognition result through the voice control signal.

[0154] In some embodiments, the control module 630 is used to determine the negative emotion level of the driver when the emotion recognition result indicates that the driver is in a negative emotional state, and the negative emotion level is used to indicate the degree of negativity in the driver's emotional state; determine a soothing text and a target timbre based on the negative emotion level; generate the voice control signal according to the soothing text and the target timbre, and the voice control signal is used to control the audio of the first vehicle to play a soothing voice matching the soothing text with the target timbre.

[0155] In some embodiments, the control module 630 is used to determine the negative emotion level of the driver when the emotion recognition result indicates that the driver is in a negative emotional state; when the negative emotion level meets the preset restriction requirements, obtain the current driving environment information of the first vehicle, and the driving environment information includes the driving section and the driving road condition; when the driving environment information meets the first environmental requirements, obtain the driving speed of the first vehicle; when the driving speed is greater than the preset speed indicated by the preset speed limit requirement, control the first vehicle to decelerate until the driving speed of the first vehicle is less than or equal to the preset speed; when the driving environment information meets the second environmental requirements, control the first vehicle to start the automatic driving mode; when the driving environment information meets the third environmental requirements, control the display screen of the first vehicle to display the recommended driving route, and the recommended driving route is used to recommend the first vehicle to drive to the target parking area.

[0156] In some embodiments, a display component is installed on the outside of the first vehicle; the control module 630 is used to generate a display control signal when the emotion recognition result indicates that the driver is in a negative emotional state; the display component installed on the outside of the first vehicle is controlled by the display control signal to display warning information, and the warning information is used to warn the driver of the first vehicle that he is in a negative emotional state.

[0157] In summary, the embodiments of the present application provide a vehicle control device, which, during the driving process of the vehicle, can continuously and dynamically monitor the driver's emotional changes by capturing multiple facial images of the driver within a historical time period and combining them with real-time physiological data. The comprehensive analysis based on facial feature representation and physiological data can more accurately identify the driver's emotional state, which helps to formulate more accurate and reasonable soothing measures, thereby controlling the vehicle to perform soothing operations that match the driver's current negative emotions, helping the driver to quickly adjust his emotions, thereby improving driving safety.

[0158] It should be noted that the vehicle control device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control device provided in the above embodiment and the vehicle control method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0159] Figure 7 The block diagram of the structure of a computer device 700 provided by an exemplary embodiment of the present application is shown. The computer device 700 may be a smart phone, a vehicle-mounted terminal, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4), a laptop computer or a desktop computer. The computer device 700 may also be a vehicle with a vehicle-mounted terminal, which is not limited in the present application.

[0160] Typically, the computer device 700 includes a processor 701 and a memory 702 .

[0161] The processor 701 may include one or more processing cores, such as a 4-core processor, a 7-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0162] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is used to be executed by the processor 701 to implement the vehicle control method provided in the method embodiment of the present application.

[0163] In some embodiments, the computer device 700 further includes one or more sensors, including but not limited to: a proximity sensor, a gyroscope sensor, and a pressure sensor.

[0164] The proximity sensor, also called a distance sensor, is usually disposed on the front panel of the computer device 700. The proximity sensor is used to collect the distance between the user and the front of the computer device 700.

[0165] The gyroscope sensor can detect the body direction and rotation angle of the computer device 700, and the gyroscope sensor can cooperate with the acceleration sensor to collect the user's 3D actions on the computer device 700. The processor 701 can implement the following functions based on the data collected by the gyroscope sensor: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0166] The pressure sensor can be set on the side frame of the computer device 700 and / or the lower layer of the display screen. When the pressure sensor is set on the side frame of the computer device 700, it can detect the user's grip signal of the computer device 700, and the processor 701 performs left and right hand recognition or shortcut operations according to the grip signal collected by the pressure sensor. When the pressure sensor is set on the lower layer of the display screen, the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the display screen. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0167] In some embodiments, the computer device 700 also includes other components, which can be understood by those skilled in the art. Figure 7 The structure shown in the figure does not constitute a limitation on the computer device 700, and the computer device 700 may include more or less components than shown in the figure, or combine some components, or adopt a different arrangement of components.

[0168] An embodiment of the present application also provides a computer device, which includes a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the vehicle control method provided by the above-mentioned method embodiments.

[0169] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the vehicle control method provided by the above-mentioned method embodiments.

[0170] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the vehicle control method described in any of the above embodiments.

[0171] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0172] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0173] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: The method comprises: During the driving of the first vehicle, acquiring multiple facial images of the driver of the first vehicle within a historical time period and acquiring physiological data of the driver of the first vehicle, wherein the historical time period is used to indicate a time period from a historical moment to a current moment, and the physiological data is used to indicate data related to the physical state of the driver; Extracting facial feature representations corresponding to the plurality of facial images respectively; Recognizing the driver's emotions based on multiple facial feature representations and the physiological data to obtain an emotion recognition result; In a case where the emotion recognition result indicates that the driver is in a negative emotional state, the first vehicle is controlled to perform a soothing operation, where the soothing operation matches the emotion recognition result.

2. The method according to claim 1, characterized in that The negative emotional state includes sadness; The step of identifying the driver's emotion based on a plurality of facial feature representations and the physiological data to obtain the emotion recognition result comprises: Based on the multiple facial feature representations, determining the frequency of occurrence of a target facial action of the driver in the historical time period, the target facial action being an action associated with the sad emotion, and the target facial action comprising at least one of a squinting action, a mouth opening action, and a head turning action; Acquiring a data threshold corresponding to the physiological data, and comparing the physiological data with the data threshold to obtain a comparison result; The occurrence frequency and the comparison result are analyzed to obtain the emotion recognition result, where the emotion recognition result is used to indicate whether the driver is in a sad mood.

3. The method according to claim 2, characterized in that The analyzing the occurrence frequency and the comparison result to obtain the emotion recognition result, wherein the emotion recognition result is used to indicate whether the driver is in a sad mood, includes: Acquire a first weight corresponding to the target facial action, and acquire a second weight corresponding to the physiological data, wherein the first weight is used to indicate the degree of association between the target facial action and the sad emotion, and the second weight is used to indicate the degree to which the physiological data reflects the sad emotion; Obtain a first sadness score based on the first weight and the occurrence frequency, and obtain a second sadness score based on the second weight and the comparison result; When the first sadness score and the second sadness score meet the score requirement of the sadness emotion, a first emotion recognition result is obtained, and the first emotion recognition result indicates that the driver is in a sad emotion.

4. The method according to any one of claims 1 to 3, characterized in that: The physiological data includes at least one of the driver's heart rate, body temperature, and skin conductance.

5. The method according to any one of claims 1 to 3, characterized in that: The acquiring the physiological data of the driver of the first vehicle includes: Acquiring physiological data of a driver of the first vehicle collected by a first sensor installed in a steering wheel of the first vehicle; or, The first vehicle establishes a communication connection with the wearable device worn by the driver to obtain physiological data of the driver of the first vehicle collected by a second sensor installed in the wearable device.

6. The method according to any one of claims 1 to 3, characterized in that: When the emotion recognition result indicates that the driver is in a negative emotional state, controlling the first vehicle to perform a soothing operation includes: generating a voice control signal if the emotion recognition result indicates that the driver is in a negative emotional state; The voice control signal is used to control the audio system of the first vehicle to play a soothing voice that matches the emotion recognition result.

7. The method according to claim 6, characterized in that The generating of the voice control signal when the emotion recognition result indicates that the driver is in a negative emotional state comprises: In the case where the emotion recognition result indicates that the driver is in a negative emotional state, determining a negative emotion level of the driver, the negative emotion level being used to indicate a degree of negativity in the driver's emotional state; determining a soothing text and a target timbre based on the negative emotion level; The voice control signal is generated according to the soothing text and the target timbre, and the voice control signal is used to control the audio system of the first vehicle to play a soothing voice matching the soothing text with the target timbre.

8. The method according to any one of claims 1 to 3, characterized in that: When the emotion recognition result indicates that the driver is in a negative emotional state, controlling the first vehicle to perform a soothing operation includes: If the emotion recognition result indicates that the driver is in a negative emotional state, determining the negative emotion level of the driver; if the negative emotion level meets the preset restriction requirements, obtaining the current driving environment information of the first vehicle, the driving environment information including the driving section and the driving road condition; When the driving environment information meets the first environment requirement, obtaining the driving speed of the first vehicle; when the driving speed is greater than the preset speed indicated by the preset speed limit requirement, controlling the first vehicle to decelerate until the driving speed of the first vehicle is less than or equal to the preset speed; When the driving environment information meets the second environment requirement, controlling the first vehicle to start the automatic driving mode; When the driving environment information meets the third environment requirement, the display screen of the first vehicle is controlled to display a recommended driving route, where the recommended driving route is used to recommend the first vehicle to drive to a target parking area.

9. The method according to any one of claims 1 to 3, characterized in that: A display component is installed on the vehicle exterior of the first vehicle; The method further comprises: generating a display control signal when the emotion recognition result indicates that the driver is in a negative emotional state; The display control signal is used to control a display component installed outside the first vehicle to display warning information, where the warning information is used to warn the driver of the first vehicle that he is in a negative emotional state.

10. A vehicle control device, characterized in that: The device comprises: an acquisition module, used for acquiring, during the driving process of the first vehicle, a plurality of facial images of the driver of the first vehicle within a historical time period and acquiring physiological data of the driver of the first vehicle, wherein the historical time period is used to indicate a time period between a historical moment and a current moment, and the physiological data is used to indicate data related to the physical state of the driver; A recognition module, used to extract facial feature representations corresponding to the plurality of facial images respectively; The recognition module is used to recognize the driver's emotions based on multiple facial feature representations and the physiological data to obtain an emotion recognition result; A control module is used to control the first vehicle to perform a soothing operation when the emotion recognition result indicates that the driver is in a negative emotional state, and the soothing operation matches the emotion recognition result.

Citation Information

Patent Citations

  • Road rage warning method, device and system, vehicle and storage medium

    CN114023039A

  • Driver emotion recognition method and device, electronic equipment and storage medium

    CN116563829A

  • Vehicle assistance device and vehicle assistance method

    CN116569236A

  • Music intervention method and system for driver emotion adjustment

    CN117442843A

  • Vehicle driving mode adjusting method, device and system and storage medium

    CN117533339A

Cited By

  • Vehicle control method and device, computer equipment and storage medium

    CN119459733A

  • Control method and apparatus, and vehicle

    WO2026207649A1