A method, device, storage medium and electronic device for identifying emotional state

Abstract

The present application relates to the field of video processing technology, and specifically provides a method, device, storage medium and electronic device for identifying emotional states, the method may include: during the process of a passenger fastening a seat belt, detecting key points in a key video frame of a seat belt fastening acquired in real time, obtaining the instantaneous force of the seat belt fastening, wherein the key points include: the passenger's wrist key points and arm key points; obtaining the average speed and total time of the passenger during the process of fastening the seat belt; analyzing the instantaneous force, the average speed and the total time, and identifying the emotional state of the passenger. Some embodiments of the present application can realize accurate recognition of passenger emotions.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, storage medium and electronic device for identifying emotional states. Background Art

[0002] With the widespread popularity of online ride-hailing services, the passenger's riding experience has gradually become an important measure of service quality. In order to improve the passenger's riding experience, it is necessary to judge the passenger's emotional state to provide personalized services.

[0003] At present, when identifying the emotional state of a passenger, it usually relies on the passenger's facial features. The passenger's emotional state is determined by analyzing the passenger's facial features. However, facial features belong to the passenger's privacy, and emotion recognition by collecting facial features cannot guarantee the privacy security of the passenger.

[0004] Therefore, how to provide a technical solution for a method of identifying emotional states that is both accurate and safe has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of some embodiments of the present application is to provide a method, device, storage medium and electronic device for identifying emotional state. The technical solutions of the embodiments of the present application can avoid the use of facial recognition technology, achieve accurate identification of the emotional state of passengers, and have high privacy security.

[0006] In a first aspect, some embodiments of the present application provide a method for identifying emotional states, comprising: detecting key points in key video frames of seat belt fastening acquired in real time during the process of a passenger fastening a seat belt, and obtaining instantaneous force of fastening the seat belt, wherein the key points include: wrist key points and arm key points of the passenger; obtaining the average speed and total time of the passenger during the process of fastening the seat belt; analyzing the instantaneous force, the average speed and the total time to identify the emotional state of the passenger.

[0007] Some embodiments of the present application obtain key video frames by capturing the passenger's seat belt fastening operation during the process, and then detect the key points in the key video frames to obtain the instantaneous force of the seat belt fastening, and comprehensively analyze and identify the passenger's emotional state in combination with the average speed and total time of the seat belt fastening. Some embodiments of the present application avoid the use of facial recognition technology, achieve accurate recognition of the passenger's emotional state, have high privacy security, and enhance the passenger's riding experience.

[0008] In some embodiments, the key points in the key video frame of seat belt fastening acquired in real time are detected to obtain the instantaneous force of seat belt fastening, including: identifying the wrist key point and the arm key point in the key video frame; obtaining the maximum acceleration of the wrist key point in the key video frame; calculating the arm mass of the arm key point and the maximum acceleration to obtain the instantaneous force.

[0009] Some embodiments of the present application obtain instantaneous force by identifying and calculating wrist key points and arm key points in key video frames, thereby providing effective data support for subsequent emotion recognition.

[0010] In some embodiments, obtaining the maximum acceleration of the wrist key point in the key video frame includes: obtaining the displacement and wrist velocity of the arm key point in each frame of the key video frame; calculating the displacement and wrist velocity in each frame to obtain the wrist acceleration of each frame; and finding the maximum acceleration from the wrist acceleration of each frame.

[0011] Some embodiments of the present application calculate the wrist acceleration of each frame in the key video frame to obtain the maximum acceleration, thereby providing effective data for the subsequent accurate calculation of the instantaneous force.

[0012] In some embodiments, the arm mass is obtained by: estimating the arm volume based on the arm length of the arm key point in the key video frame; and determining the arm mass using the arm volume and human muscle density.

[0013] Some embodiments of the present application estimate the arm volume by the arm length and then obtain the arm mass, providing effective data for the subsequent accurate calculation of instantaneous force.

[0014] In some embodiments, the instantaneous force, the average speed and the total time are analyzed to identify the emotional state of the passenger, including: respectively comparing the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold to determine the emotional state.

[0015] Some embodiments of the present application can achieve accurate identification of the emotional state of the passenger by comparing the instantaneous force, average speed and total time with thresholds to determine the emotional state.

[0016] In some embodiments, the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold are compared respectively to determine the emotional state, including: when the instantaneous force is greater than the first force threshold, the average speed is greater than the first speed threshold and the total time is less than the first time threshold, the emotional state is an angry state; when the instantaneous force is greater than the second force threshold and not greater than the first force threshold, the average speed is greater than the second speed threshold and not greater than the first speed threshold, and the total time is less than the first time threshold, the emotional state is an anxious state; when the instantaneous force is greater than the third force threshold and not greater than the second force threshold, the average speed is greater than the third speed threshold and not greater than the second speed threshold, and the total time is greater than the second time threshold, the emotional state is a confused state; when the instantaneous force is not greater than the third force threshold, the average speed is not greater than the third speed threshold and the total time is greater than the second time threshold, the emotional state is a relaxed state.

[0017] Some embodiments of the present application can accurately identify the emotion of the passenger by comparing the instantaneous force, average speed and total time with different thresholds.

[0018] In some embodiments, after identifying the emotional state of the passenger, the method further includes: generating personalized service suggestions corresponding to the emotional state, wherein the personalized service suggestions are used to adjust the driving mode and the in-vehicle environment.

[0019] Some embodiments of the present application can improve service quality and enhance passengers' riding experience by generating personalized service suggestions.

[0020] In a second aspect, some embodiments of the present application provide a device for identifying emotional states, comprising: a detection module, for detecting key points in key video frames of seat belt fastening acquired in real time during the process of a passenger fastening a seat belt, to obtain instantaneous force of fastening the seat belt, wherein the key points include: wrist key points and arm key points of the passenger; an acquisition module, for obtaining the average speed and total time of the passenger during the process of fastening the seat belt; and an identification module, for analyzing the instantaneous force, the average speed and the total time, to identify the emotional state of the passenger.

[0021] In some embodiments, the detection module is used to: identify the wrist key point and the arm key point in the key video frame; obtain the maximum acceleration of the wrist key point in the key video frame; calculate the arm mass and the maximum acceleration of the arm key point to obtain the instantaneous force.

[0022] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0023] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0024] In a fifth aspect, some embodiments of the present application provide a computer program product, wherein the computer program product comprises a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A system diagram for identifying emotional states provided for some embodiments of the present application;

[0027] Figure 2 One of the flow charts of the method for identifying emotional state provided in some embodiments of the present application;

[0028] Figure 3 A second flow chart of a method for identifying an emotional state provided in some embodiments of the present application;

[0029] Figure 4 A block diagram of an apparatus for identifying an emotional state provided in some embodiments of the present application;

[0030] Figure 5 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0032] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0033] In the related technologies, in order to improve the riding comfort of passengers and provide personalized services, online car-hailing companies urgently need a technology that can quickly and accurately judge the emotional state of passengers. However, existing emotion recognition technologies mostly rely on facial expression analysis, which has privacy protection issues. For example, the existing technology achieves the accuracy of facial expression recognition by converting the passenger's face image into a frontal face image; because facial expressions are very rich and complex, the analysis of facial images is not limited to changes in facial muscles, but also includes subtle changes such as the opening and closing of eyes and the up and down movement of eyebrows. Obviously, this existing technology involves a large number of passengers' facial privacy features, and privacy security cannot be guaranteed.

[0034] In view of this, some embodiments of the present application provide a method for identifying emotional states, which collects and analyzes the relevant features of the wrists and arms of passengers in the process of fastening their seat belts, and then realizes the recognition of passengers' emotions in combination with the average speed and total time of passengers fastening their seat belts. Some embodiments of the present application can avoid the use of facial recognition technology to protect the privacy of passengers. By analyzing the details of the passenger's seat belt fastening movements, the passenger's emotional state can be accurately judged. Moreover, through the real-time emotional state analysis of the passenger, the driver can be helped to adjust the driving style and the in-car environment according to the passenger's emotions, thereby improving the passenger's comfort and riding experience.

[0035] The following is combined with Figure 1 The overall structure of the system for identifying emotional states provided by some embodiments of the present application is exemplified.

[0036] like Figure 1 As shown, some embodiments of the present application provide a system for identifying emotional states, which may include a vehicle 100 and a processing end 200. During the process of a passenger fastening a seat belt after getting on the vehicle, the shooting device in the vehicle 100 may shoot the passenger's seat belt fastening operation process in real time to obtain a seat belt fastening video, and send it to the processing end 200. The processing end 200 detects, calculates and analyzes key video frames in the seat belt fastening video to identify the passenger's emotional state. The processing end 200 may feed back the emotional state to the driver, and the driver may adjust the driving style and the in-car environment according to the emotional state. Alternatively, the processing end 200 may also automatically play corresponding music or recommend related playback content for automatic playback according to the emotional state, so as to achieve the purpose of adjusting the in-car environment.

[0037] In some embodiments of the present application, the processing end 200 may be a cloud server remotely connected to the vehicle 100; the processing end 200 may also be deployed inside the vehicle 100. Specifically, it may be determined according to actual conditions, and the embodiments of the present application are not specifically limited here.

[0038] The following is combined with Figure 2 The implementation process of identifying emotional states performed by the processing end 200 provided in some embodiments of the present application is exemplified.

[0039] Please see attached Figure 2 , Figure 2 A flow chart of a method for identifying an emotional state is provided for some embodiments of the present application. The method for identifying an emotional state may include:

[0040] S210, during the process of the passenger fastening the seat belt, detecting key points in the key video frames of the seat belt fastening acquired in real time to obtain the instantaneous force of the seat belt fastening, wherein the key points include: wrist key points and arm key points of the passenger.

[0041] For example, in some embodiments of the present application, the entire process of the passenger fastening the seat belt is captured in real time by a camera (that is, a shooting device) installed in the vehicle 100. Among them, the start frame is the first frame image of the passenger starting to perform the seat belt fastening action. This frame is usually the moment when the passenger first touches or just starts to reach for the seat belt. The end frame is the last frame image of the passenger completing the seat belt fastening. This frame is usually the moment when the seat belt is buckled or the passenger stops the action. The video of the passenger moving 1 / 4 of the seat belt distance from the beginning of the seat belt fastening to the wrist key point is extracted as the key video frame. Use a posture estimation model (such as OpenPose, MediaPipe) to detect the passenger's wrist key points, arm key points (or shoulders, elbows, etc.), and calculate the instantaneous force. The instantaneous force represents the force when the passenger just starts to fasten the seat belt, which often reflects the emotional state of the passenger at this time.

[0042] In some embodiments of the present application, S210 may include:

[0043] S211, identifying the wrist key point and the arm key point in the key video frame.

[0044] For example, in some embodiments of the present application, based on deep learning models such as HRNet, the wrist key points (referred to as wrists) and arm key points (referred to as arms) of passengers wearing seat belts in key video frames are identified.

[0045] S212: Obtain the maximum acceleration of the wrist key point in the key video frame.

[0046] For example, in some embodiments of the present application, a frame with the largest acceleration is found in the key video frames, that is, the frame with the largest acceleration when the wrist is pulled out of the seat belt a max moment.

[0047] In some embodiments of the present application, S212 may include: obtaining the displacement and wrist velocity of the arm key point in each frame of the key video frame; calculating the displacement and wrist velocity in each frame to obtain the wrist acceleration of each frame; and finding the maximum acceleration from the wrist acceleration of each frame.

[0048] For example, in some embodiments of the present application, the displacement of the wrist key point in each frame of the key video frame is calculated, and then the speed of the wrist in each frame is calculated. The speed of the wrist in each frame is calculated as follows: , where v t It is t Frame wrist speed, d t and d t-1 Respectively t Frame and t -1 frame wrist displacement, is the time interval (the inverse of the frame rate).

[0049] After that, the wrist acceleration of the wrist in each frame is calculated, that is: , where a t It is t Wrist acceleration per frame, v t and v t-1 Respectively t Frame and t -1 frame of wrist speed.

[0050] The maximum value is selected from the wrist acceleration of all frames calculated above to obtain the instantaneous acceleration of the seat belt. a max (That is, the maximum acceleration).

[0051] In some embodiments of the present application, the arm mass is obtained by the following method: estimating the arm volume based on the arm length of the arm key point in the key video frame; and determining the arm mass using the arm volume and human muscle density.

[0052] For example, in some embodiments of the present application, the arm length is calculated based on the key point coordinates. The arm length can be approximately regarded as the length from the shoulder to the elbow plus the length from the elbow to the wrist. Specifically, the arm length Larm The calculation method is as follows:

[0053]

[0054] In the formula, L upper_arm The length from shoulder to elbow. L lower_arm is the length from elbow to wrist, ( x elbow , y elbow ) is the coordinate of the elbow; ( x shoulder , y shoulder ) is the coordinate of the shoulder; ( x wrist , y wrist ) are the coordinates of the elbow.

[0055] Assuming the arm is a cylinder, use the arm length and average diameter to estimate the arm volume, that is: V arm =π·( r arm ) 2 · L arm ,in, r arm is the radius of the arm, which can be estimated based on standard human proportions or the arm width measured by a vision sensor.

[0056] Since different parts of the human body have different densities, the average density value in human anatomy can be used to estimate the arm mass, for example, the average density of human muscle is ρ muscle About 1.06g / cm 3 (As a specific example of human muscle density). Arm mass m arm =ρ muscle · V arm .

[0057] S213, calculating the arm mass of the key points of the arm and the maximum acceleration to obtain the instantaneous force.

[0058] For example, in some embodiments of the present application, at the moment when the wrist pulls out the safety belt and generates the maximum acceleration, the instantaneous force F is calculated by the arm mass and the maximum acceleration. instant , that is, F instant =m arm · V arm .

[0059] S220, obtaining the average speed and total time of the passenger in the process of fastening the seat belt.

[0060] For example, in some embodiments of the present application, the total length L of the motion path of the wrist key point from reaching for the safety belt to inserting the buckle is calculated. trajectory , , calculate the total time T of the passenger from the beginning to the completion of the seat belt fastening total =t end -t start Calculate the average speed based on the total path length and total time v avg ,Right now v avg =L trajectory / T total .in,( x i , y i ) is the position coordinate of the wrist at the i-th moment.

[0061] S230, analyzing the instantaneous force, the average speed and the total time to identify the emotional state of the passenger, so as to provide riding services for the passenger according to the emotional state.

[0062] For example, in some embodiments of the present application, the instantaneous force, average speed and total time are comprehensively analyzed to identify the emotional state of the passenger. This is because emotions have different effects on physiological reactions and cause different physiological reactions in different emotional states. For example, anxiety and anger are usually accompanied by tense and fast movements, while movements in a relaxed state are slower and smoother. Therefore, the emotional state of the passenger can be inferred by analyzing the force and speed of the movement. When a person is in a relaxed state, his movements are usually more stable and coherent, with a slower speed and less force. When in an anxious or angry state, the movements are often more hasty and forceful.

[0063] In some embodiments of the present application, S230 may include: comparing the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold, respectively, to determine the emotional state.

[0064] For example, in some embodiments of the present application, the instantaneous force, average speed and total time are respectively compared with their corresponding thresholds to determine the emotional state of the passenger. In some embodiments, the force threshold may include a first force threshold, a second force threshold and a third force threshold. The speed threshold may include: a first speed threshold, a second speed threshold and a third speed threshold. The time threshold may include: a first time threshold and a second time threshold. It is understandable that the force threshold, the speed threshold and the time threshold can be flexibly set according to the actual application scenario, and the embodiments of the present application are not specifically limited here.

[0065] For example, in some embodiments of the present application, the first force threshold is 3.0 N, the second force threshold is 2.0 N, and the third force threshold is 1.5 N. The first speed threshold is 2.0 m / s, the second speed threshold is 1.5 m / s, and the third speed threshold is 1.0 m / s. The first time threshold is 2 s, and the second time threshold is 3 s.

[0066] In some embodiments of the present application, S230 may include:

[0067] When the instantaneous force is greater than a first force threshold, the average speed is greater than a first speed threshold, and the total time is less than a first time threshold, the emotional state is an angry state.

[0068] For example, when F instant >3.0N, v avg >2.0m / s and T total When the time is less than 2s, the passenger’s emotional state is angry.

[0069] When the instantaneous force is greater than a second force threshold and not greater than the first force threshold, the average speed is greater than a second speed threshold and not greater than the first speed threshold, and the total time is less than the first time threshold, the emotional state is an anxious state.

[0070] For example, when 2.0N<F instant ≤3.0N, 1.5m / s< v avg ≤2.0m / s and T total When the time is less than 2s, the passenger’s emotional state is anxious.

[0071] When the instantaneous force is greater than a third force threshold and not greater than the second force threshold, the average speed is greater than a third speed threshold and not greater than the second speed threshold, and the total time is greater than a second time threshold, the emotional state is a confused state.

[0072] For example, when 1.5N<F instant ≤2.0N, 1.0m / s<v avg ≤1.5m / s and T total When it is greater than 3s, it indicates that the passenger's force changes are unstable and the passenger's emotional state is confused.

[0073] When the instantaneous force is not greater than a third force threshold, the average speed is not greater than a third speed threshold, and the total time is greater than a second time threshold, the emotional state is a relaxed state.

[0074] For example, when F instant ≤1.5N, v avg ≤1.0m / s and T total When the time is greater than 3s, the passenger’s emotional state is relaxed.

[0075] In some embodiments of the present application, after executing S230, the method for identifying an emotional state may further include: generating a personalized service suggestion corresponding to the emotional state, wherein the personalized service suggestion is used to adjust the driving mode and the in-vehicle environment.

[0076] For example, in some embodiments of the present application, corresponding personalized service suggestions can be generated according to the above-mentioned emotional state determination results. The personalized service suggestions can be directly notified to the driver, so that the driver can take relevant measures (such as adjusting the driving style, adjusting the in-car environment, etc.) according to the personalized service suggestions; or, the processing end 200 can automatically control the relevant playback devices in the car based on the personalized service suggestions, adjust the in-car environment, etc. It should be understood that the embodiments of the present application are not limited to this.

[0077] The following is combined with Figure 3 The specific process of identifying emotional states provided by some embodiments of the present application is exemplified.

[0078] Please see attached Figure 3 , Figure 3 A flow chart of a method for identifying an emotional state is provided for some embodiments of the present application.

[0079] The above process is explained below as an example.

[0080] S310, obtaining key video frames of the process of passengers fastening their seat belts.

[0081] S320, identifying wrist key points and arm key points in the key video frame.

[0082] S330, obtaining the maximum acceleration and arm mass of the wrist key point in the key video frame.

[0083] S340, calculating the arm mass and the maximum acceleration to obtain the instantaneous force.

[0084] S350, obtaining an average speed and a total time of the passenger in the process of fastening the seat belt.

[0085] S360, respectively comparing the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold to determine the emotional state of the passenger.

[0086] S370, generating personalized service suggestions corresponding to the emotional state.

[0087] It can be understood that the specific implementation process of S310~S370 can refer to the method embodiment provided above. To avoid repetition, the detailed description is appropriately omitted here.

[0088] Through some of the above embodiments of the present application, it can be seen that the present application can protect the privacy of passengers by analyzing the details of the movements during the seat belt fastening process instead of facial expressions. By accurately judging the emotional state of the passenger, the driver can adjust the driving behavior and the in-car environment to improve the comfort of the passenger. The present application can analyze the emotional state of the passenger in real time and provide instant feedback to help the driver make corresponding adjustments.

[0089] Please refer to Figure 4 , Figure 4 The block diagram of the composition of the device for identifying emotional state provided by some embodiments of the present application is shown. It should be understood that the device for identifying emotional state corresponds to the above method embodiment and can perform each step involved in the above method embodiment. The specific functions of the device for identifying emotional state can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0090] Figure 4 The device for identifying emotional states includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the device for identifying emotional states, and the device for identifying emotional states includes: a detection module 410, which is used to detect key points in a key video frame of seat belt fastening acquired in real time during the process of a passenger fastening a seat belt, and obtain the instantaneous force of fastening the seat belt, wherein the key points include: wrist key points and arm key points of the passenger; an acquisition module 420, which is used to obtain the average speed and total time of the passenger during the process of fastening the seat belt; an identification module 430, which is used to analyze the instantaneous force, the average speed and the total time, and identify the emotional state of the passenger, so as to provide the passenger with a ride service according to the emotional state.

[0091] In some embodiments of the present application, the detection module 410 is used to identify the wrist key point and the arm key point in the key video frame; obtain the maximum acceleration of the wrist key point in the key video frame; calculate the arm mass and the maximum acceleration of the arm key point to obtain the instantaneous force.

[0092] In some embodiments of the present application, the detection module 410 is used to obtain the displacement and wrist velocity of the arm key point in each frame of the key video frame; calculate the displacement and wrist velocity in each frame to obtain the wrist acceleration of each frame; and find the maximum acceleration from the wrist acceleration of each frame.

[0093] In some embodiments of the present application, the detection module 410 is used to estimate the arm volume based on the arm length of the arm key point in the key video frame; and determine the arm mass using the arm volume and human muscle density.

[0094] In some embodiments of the present application, the identification module 430 is used to compare the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold, respectively, to determine the emotional state.

[0095] In some embodiments of the present application, the identification module 430 is used to identify that the emotional state is an angry state when the instantaneous force is greater than a first force threshold, the average speed is greater than a first speed threshold, and the total time is less than a first time threshold; the emotional state is an anxious state when the instantaneous force is greater than a second force threshold and not greater than the first force threshold, the average speed is greater than the second speed threshold and not greater than the first speed threshold, and the total time is less than the first time threshold; the emotional state is a confused state when the instantaneous force is greater than a third force threshold and not greater than the second force threshold, the average speed is greater than the third speed threshold and not greater than the second speed threshold, and the total time is greater than the second time threshold; the emotional state is a relaxed state when the instantaneous force is not greater than the third force threshold, the average speed is not greater than the third speed threshold, and the total time is greater than the second time threshold.

[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0097] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations of the method corresponding to any of the above methods provided in the above embodiments.

[0098] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0099] like Figure 5 As shown, some embodiments of the present application provide an electronic device 500, which includes: a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520, wherein the processor 520 can implement a method as described in any of the above embodiments when reading the program from the memory 510 through a bus 530 and executing the program.

[0100] Processor 520 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 520 can be a microprocessor.

[0101] The memory 510 may be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 520 of the disclosed embodiment may be used to execute instructions in the memory 510 to implement the method shown above. The memory 510 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0102] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0103] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0104] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for identifying an emotional state, characterized in that: include: During the process of the passenger fastening the seat belt, the key points in the key video frame of the seat belt fastening acquired in real time are detected to obtain the instantaneous force of the seat belt fastening, wherein the key points include: the key points of the wrist and the key points of the arm of the passenger; Obtaining the average speed and total time of the passenger in the process of fastening the seat belt; Analyzing the instantaneous force, the average speed and the total time to identify the emotional state of the passenger; The step of detecting key points in a key video frame of seat belt fastening acquired in real time to obtain the instantaneous force of seat belt fastening includes: identifying the wrist key point and the arm key point in the key video frame; Obtaining the displacement and wrist velocity of the wrist key point in each frame of the key video frame; calculating the displacement and wrist velocity in each frame to obtain the wrist acceleration of each frame; and finding the maximum acceleration of the wrist key point from the wrist acceleration of each frame; The arm mass of the key points of the arm and the maximum acceleration are calculated to obtain the instantaneous force.

2. The method according to claim 1, characterized in that The arm mass is obtained by the following method: Estimating the arm volume based on the arm length of the arm key points in the key video frame; The arm mass is determined using the arm volume and human muscle density.

3. The method according to claim 1, characterized in that The analyzing the instantaneous force, the average speed and the total time to identify the emotional state of the passenger includes: The instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold are compared respectively to determine the emotional state.

4. The method according to claim 3, characterized in that The step of comparing the instantaneous force and the force threshold, the average speed and the speed threshold, and the total time and the time threshold to determine the emotional state comprises: When the instantaneous force is greater than a first force threshold, the average speed is greater than a first speed threshold, and the total time is less than a first time threshold, the emotional state is an angry state; When the instantaneous force is greater than a second force threshold and not greater than the first force threshold, the average speed is greater than a second speed threshold and not greater than the first speed threshold, and the total time is less than the first time threshold, the emotional state is an anxious state; When the instantaneous force is greater than a third force threshold and not greater than the second force threshold, the average speed is greater than a third speed threshold and not greater than the second speed threshold, and the total time is greater than a second time threshold, the emotional state is a confused state; When the instantaneous force is not greater than a third force threshold, the average speed is not greater than a third speed threshold, and the total time is greater than a second time threshold, the emotional state is a relaxed state.

5. The method according to claim 1, characterized in that After identifying the emotional state of the passenger, the method further comprises: Generate a personalized service suggestion corresponding to the emotional state, wherein the personalized service suggestion is used to adjust the driving mode and the in-vehicle environment.

6. A device for identifying an emotional state, characterized in that: The device is used to perform the method according to claim 1, comprising: A detection module is used to detect key points in a key video frame of seat belt fastening obtained in real time during the process of the passenger fastening the seat belt, and obtain the instantaneous force of the seat belt fastening, wherein the key points include: the passenger's wrist key points and arm key points; An acquisition module, used for acquiring the average speed and total time of the passenger in the process of fastening the seat belt; The identification module is used to analyze the instantaneous force, the average speed and the total time to identify the emotional state of the passenger.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program executes the method according to any one of claims 1 to 5 when executed by a processor.

8. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 5 when being run by the processor.

9. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program executes the method according to any one of claims 1 to 5 when executed by a processor.

Images