A Personalized Emotion Monitoring Method for Freight Drivers

Abstract

This invention discloses a personalized method for monitoring the emotions of freight drivers, comprising: 1. real-time acquisition of driver facial video streams, extraction of facial features, and construction of facial feature reference vectors; 2. establishment of a feature mapping equation to map facial information to spatial coordinates; 3. division of a three-dimensional coordinate space and labeling of each region; 4. establishment of a driver emotion risk function and calculation of the emotion risk value; 5. comparison of the emotion risk value with a preset threshold to determine whether to issue a risk warning signal to the driver. This invention can monitor and assess the driver's emotional state, thereby preventing traffic accidents and improving road traffic safety.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and driving safety, specifically a personalized method for monitoring the emotions of freight drivers. Background Technology

[0002] In recent years, with the continuous development of road transportation systems and the increasing emphasis on road traffic safety, drivers experience different emotional changes when facing different scenarios. During long-distance driving, these emotional fluctuations can severely affect a driver's judgment and threaten road traffic safety. However, most existing studies are based on fixed judgment rules and lack monitoring of drivers' individual emotional characteristics.

[0003] Furthermore, traditional driving risk monitoring systems lack the ability to comprehensively monitor and dynamically assess multiple emotional risks when processing multidimensional emotional characteristics. Additionally, drivers exhibit varying behavioral patterns across different emotional stages, and fixed assessment models cannot capture the most significant risk characteristics in real time, often neglecting the impact of the duration of accumulated abnormal emotions. Therefore, designing a driver emotion-based driving risk monitoring and warning system that integrates multidimensional emotion models and dynamically adjusts evaluation weights based on real-time emotional characteristic data fluctuations, supported by computer vision technology, is a pressing technical challenge. Summary of the Invention

[0004] This invention aims to overcome the shortcomings of existing technologies by proposing a personalized method for monitoring the emotions of freight drivers. The goal is to monitor and assess the emotional state of drivers, thereby improving driving safety, preventing traffic accidents, and ultimately enhancing road traffic safety.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides a personalized method for monitoring the emotions of freight drivers, characterized by the following steps: Step 1: Real-time acquisition and preprocessing of the driver's facial video stream inside the vehicle to obtain the... t Set of facial feature key points in a frame image And calculate the first t Instantaneous facial feature vector of a frame image ;in, Indicates the first t The aspect ratio of the eyes in the frame image. Indicates the first t The aspect ratio of the mouth in the frame image. Indicates the first t Mouth curvature in frame image Indicates the first t The distance between eyebrows in the frame image. Indicates the first t The slope of the eyebrows in the frame image. Indicates the first t Head pitch angle in frame image; Step 2, based on Obtain the driver's personalized baseline feature vector ; Step 3, based on and Establish a three-dimensional emotion space mapping model This is used to output the driver's emotional state label within the current sliding window; where, An index representing the driver's psychological state of pleasure. This indicates the driver's level of arousal. An indicator representing the driver's psychological dominance; Step 4: Construct any one of the following in the driver's psychological state: pleasure index, arousal index, and dominance index, using the AHP risk weight matrix. i Risk weight of each indicator and with Construct a driver's emotional driving risk function ; Step 5, if This indicates that the driver is in a safe driving state within the current sliding window. This indicates that the driver is in a Level 1 warning state within the current sliding window. This indicates that the driver is in a level 2 warning state within the current sliding window. , This indicates two thresholds.

[0006] The personalized emotion monitoring method for freight drivers described in this invention is also characterized in that step 2 includes the following steps: Step 2.1: Establish a preset length of... N A sliding time window is used to store continuously acquired data. N Independent variables of instantaneous facial features in a frame; Step 2.2: For the current sliding time window... N After normalizing the instantaneous facial feature vector of a frame, the normalized instantaneous facial feature vector under the current sliding time window is obtained. ; Step 2.3, Calculation The variance of the normalized instantaneous facial feature vectors in each frame is summed to obtain the total variance under the current sliding time window. ; Step 2.4, if If yes, return to step 2.2 to process the instantaneous facial feature vectors of the N frames in the next sliding time window; otherwise, it indicates that the driver is in a stable emotional state in the current sliding time window, and proceed to step 2.5. Step 2.5: Calculate and normalize the average value of each instantaneous facial feature in the normalized instantaneous facial feature vector of N frames within the current sliding window, and use this value as the driver's personalized baseline feature vector. ;in, This represents the stability threshold.

[0007] Furthermore, step 3 includes the following steps: Step 3.1: According to equations (1) to (3), we obtain... :

[0008]

[0009]

[0010] Step 3.2: Calculate according to formula (4) to Euclidean distance ;

[0011] In equation (4), This represents a normalized index of driver psychological well-being. This represents the normalized driver arousal index. This represents the normalized driver's psychological dominance index. Step 3.3, if Then for the first t The frame image is labeled as a stable emotional state and output. According to , , The combination of positive and negative signs determines the first... t The driver's emotional state is captured in the frame image, and the cumulative frequency of each emotional state within the current sliding time window is counted. The emotional state label with the highest frequency is then selected as the judgment result within the current sliding window and output.

[0012] Furthermore, step 4 includes the following steps: Step 4.1, according to Construct a third-order matrix for risk assessment ,in, Indicates the first i The first indicator is relative to the first j The importance scale of the first indicator is calculated using equation (5). i Risk weight of each indicator ;

[0013] In equation (6), Indicates the first k The first indicator is relative to the first j The importance scale of each indicator; Step 4.2: Construct the instantaneous emotional risk function according to equation (6). ;

[0014] In equation (7), The risk weighting represents the driver's psychological state of pleasure. This indicates the risk weight of the driver's arousal index. The risk weight represents the driver's psychological advantage index.

[0015] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor in performing the method described therein, and the processor is configured to execute the program stored in the memory.

[0016] The present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program is executed by a processor to perform the steps of the method described thereon.

[0017] Compared with existing technologies, the beneficial technical effects of this invention are reflected in: 1. This invention takes into account the physiological and behavioral differences among drivers, introduces the concept of personalized baseline feature vectors, and assigns corresponding weights based on the relative importance of emotions, thereby improving the prediction accuracy of emotional driving. This provides a certain reference for accident prevention and safety management of drivers in the future and is beneficial for relevant personnel to manage vehicle operation status.

[0018] 2. This invention addresses the misjudgment problem in emotion retrieval by introducing a cumulative frequency analysis mechanism and adjusting weights based on emotion characteristics. This effectively filters out false detections and invalid alarms caused by driver emotional fluctuations, enhancing the robustness of the method and contributing to driver safety.

[0019] 3. This invention takes into account the cumulative effect of abnormal driver emotions and introduces a risk discrimination mechanism based on weighted Euclidean distance to accurately locate the driver's emotional driving risk level, which is conducive to enabling safety management personnel to accurately quantify and manage the driver's emotional state. Attached Figure Description

[0020] Figure 1 This is the overall flowchart of the present invention; Figure 2 This is a flowchart of the driver's emotional state determination decision-making process of the present invention. Detailed Implementation

[0021] In this embodiment, a personalized emotion monitoring method for freight drivers constructs personalized baseline characteristics of the driver based on real-time acquired driver information during driving on the road, monitors the driver's comprehensive risk value, and determines whether to issue a risk warning signal, thereby improving driving safety. Specifically, as... Figure 1 As shown, the method includes the following steps: Step 1: Based on the in-vehicle camera and face detection algorithm, real-time acquisition of the driver's facial video stream inside the vehicle is performed and preprocessed to obtain the... t Set of facial feature key points in a frame image And calculate the first t Instantaneous facial feature vector of a frame image ;in, Indicates the first t The aspect ratio of the eyes in the frame image. Indicates the first t The aspect ratio of the mouth in the frame image. Indicates the first t Mouth curvature in frame image Indicates the first t The distance between eyebrows in the frame image. Indicates the first t The slope of the eyebrows in the frame image. Indicates the first t Head pitch angle in the frame image.

[0022] Step 2: In actual driving scenarios, to eliminate the innate differences in facial physiological structure among different drivers, based on... Obtain the driver's personalized baseline feature vector ; Step 2.1: Establish a preset length of... N A sliding time window is used to store continuously acquired data. N Instantaneous facial feature variables within a frame; in this example, a preset time window. .

[0023] Step 2.2: For the current sliding time window... N After normalizing the instantaneous facial feature vector of a frame, the normalized instantaneous facial feature vector under the current sliding time window is obtained. ; Step 2.3, Calculation The variance of the normalized instantaneous facial feature vectors in each frame is summed to obtain the total variance under the current sliding time window. .

[0024] Step 2.4, if If yes, return to step 2.2 to process the instantaneous facial feature vectors of the N frames in the next sliding time window; otherwise, it indicates that the driver is in a stable emotional state in the current sliding time window, and proceed to step 2.5. Step 2.5: Calculate and normalize the average value of each instantaneous facial feature in the normalized instantaneous facial feature vector of N frames within the current sliding window, and use this value as the driver's personalized baseline feature vector. ;in, This represents the stability threshold; in this example, the stability threshold is... .

[0025] Step 3, based on and Establish a three-dimensional emotion space mapping model This is used to output the driver's emotional state label within the current sliding window; where, An index representing the driver's psychological state of pleasure. This indicates the driver's level of arousal. This is an indicator of the driver's psychological dominance.

[0026] Step 3.1: According to equations (1) to (3), we obtain... :

[0027]

[0028]

[0029] Step 3.2: Calculate according to formula (4) to Euclidean distance ;

[0030] In equation (4), This represents a normalized index of driver psychological well-being. This represents the normalized driver arousal index. This represents the normalized driver's psychological dominance index.

[0031] Step 3.3, if Then for the first t The frame image is labeled as a stable emotional state and output. According to , , The combination of positive and negative signs determines the first... t The driver's emotional state is captured in the frame image, and the cumulative frequency of each emotional state within the current sliding time window is counted. The emotional state label with the highest frequency is then selected as the judgment result for the current sliding window and output. In this example, .

[0032] Step 4: Construct any one of the following in the driver's psychological state: pleasure index, arousal index, and dominance index, using the AHP risk weight matrix. i Risk weight of each indicator and with Construct a driver's emotional driving risk function ; Step 4.1, according to Construct a third-order matrix for risk assessment ,in, Indicates the first i The first indicator is relative to the first j The importance scale of the first indicator is calculated using equation (5). i Risk weight of each indicator ;

[0033] In equation (6), Indicates the first k The first indicator is relative to the first j The importance scale of each indicator; in this example, based on the actual impact of the driver's abnormal emotions on driving behavior, a third-order risk assessment matrix is ​​constructed as follows: The risk weights of the driver's psychological state pleasure index are obtained by using equation (5). Risk weighting of driver arousal index Risk weighting of driver psychological dominance index .

[0034] Step 4.2: Construct the instantaneous emotional risk function according to equation (6). ;

[0035] In equation (7), The risk weighting represents the driver's psychological state of pleasure. This indicates the risk weight of the driver's arousal index. The risk weight represents the driver's psychological advantage index.

[0036] Step 5, as follows Figure 2 As shown, a tiered early warning mechanism is constructed based on... Driving conditions are divided into three states: safe state, first-level warning state, and second-level warning state. This indicates that the driver is in a safe driving state within the current sliding window. This indicates that the driver is in a Level 1 warning state within the current sliding window. The vehicle's indicator light will flash yellow and display a yellow warning icon until the driver returns to a safe state. This indicates that the driver is in a level 2 warning state within the current sliding window. The vehicle's indicator light will flash yellow and display a yellow warning icon, while a buzzer will vibrate until the driver returns to a safe state. , This indicates two thresholds.

[0037] In this example, using a network dataset as the research object, after constructing a three-dimensional emotion space mapping model, the emotion label dictionary constructed based on positive and negative combinations of pleasure, arousal, and dominance is shown in Table 1: Table 1

[0038] In this example, the instantaneous frame emotion space mapping coordinates and window emotion labels calculated according to equations (1) to (6) are shown in Table 2: Table 2

[0039] Emotional risk values ​​for each driver are calculated based on a sliding time window. ,like If so, the driver is deemed to be in a safe driving state. If so, the driver is determined to be in a Level 1 warning state. If the driver is in a Level 2 warning state, taking window 052 as an example, the driver's emotional risk value is calculated. This is a Level 1 warning status.

[0040] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0041] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A method of emotion monitoring of individual freight haulers, characterized by, Includes the following steps: Step 1: Real-time acquisition and preprocessing of the driver's facial video stream inside the vehicle to obtain the... t Set of facial feature key points in a frame image And calculate the first t Instantaneous facial feature vector of a frame image ;in, Indicates the first t The aspect ratio of the eyes in the frame image. Indicates the first t The aspect ratio of the mouth in the frame image. Indicates the first t Mouth curvature in frame image Indicates the first t The distance between eyebrows in the frame image. Indicates the first t The slope of the eyebrows in the frame image. Indicates the first t Head pitch angle in frame image; Step 2, based on Obtain the driver's personalized baseline feature vector ; Step 3, based on and Establish a three-dimensional emotion space mapping model This is used to output the driver's emotional state label within the current sliding window; where, An index representing the driver's psychological state of pleasure. This indicates the driver's level of arousal. An indicator representing the driver's psychological dominance; Step 4: Construct any one of the following in the driver's psychological state: pleasure index, arousal index, and dominance index, using the AHP risk weight matrix. i Risk weight of each indicator and with Construct a driver's emotional driving risk function ; Step 5, if This indicates that the driver is in a safe driving state within the current sliding window. This indicates that the driver is in a Level 1 warning state within the current sliding window. This indicates that the driver is in a level 2 warning state within the current sliding window. , This indicates two thresholds.

2. The personalized emotion monitoring method for freight drivers according to claim 1, characterized in that, Step 2 includes the following steps: Step 2.1: Establish a preset length of... N A sliding time window is used to store continuously acquired data. N Independent variables of instantaneous facial features in a frame; Step 2.2: For the current sliding time window... N After normalizing the instantaneous facial feature vector of a frame, the normalized instantaneous facial feature vector under the current sliding time window is obtained. ; Step 2.3, Calculation The variance of the normalized instantaneous facial feature vectors in each frame is summed to obtain the total variance under the current sliding time window. ; Step 2.4, if If yes, return to step 2.2 to process the instantaneous facial feature vectors of the N frames in the next sliding time window; otherwise, it indicates that the driver is in a stable emotional state in the current sliding time window, and proceed to step 2.

5. Step 2.5: Calculate and normalize the average value of each instantaneous facial feature in the normalized instantaneous facial feature vector of N frames within the current sliding window, and use this value as the driver's personalized baseline feature vector. ;in, This represents the stability threshold.

3. The personalized emotion monitoring method for freight drivers according to claim 2, characterized in that, Step 3 includes the following steps: Step 3.1: According to equations (1) to (3), we obtain... : Step 3.2: Calculate according to formula (4) to Euclidean distance ; In equation (4), This represents a normalized index of driver psychological well-being. This represents the normalized driver arousal index. This represents the normalized driver's psychological dominance index. Step 3.3, if Then for the first t The frame image is labeled as a stable emotional state and output. According to , , The combination of positive and negative signs determines the first... t The driver's emotional state is captured in the frame image, and the cumulative frequency of each emotional state within the current sliding time window is counted. The emotional state label with the highest frequency is then selected as the judgment result within the current sliding window and output.

4. The personalized emotion monitoring method for freight drivers according to claim 3, characterized in that, Step 4 includes the following steps: Step 4.1, according to Construct a third-order matrix for risk assessment ,in, Indicates the first i The first indicator is relative to the first j The importance scale of the first indicator is calculated using equation (5). i Risk weight of each indicator ; In equation (6), Indicates the first k The first indicator is relative to the first j The importance scale of each indicator; Step 4.2: Construct the instantaneous emotional risk function according to equation (6). ; In equation (7), The risk weighting represents the driver's psychological state of pleasure. This indicates the risk weight of the driver's arousal index. The risk weight represents the driver's psychological advantage index.

5. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the method of any one of claims 1-4, the processor being configured to execute the program stored in the memory.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the method according to any one of claims 1-4.

Images