User fatigue state intervention method, device, equipment and medium

Abstract

The present application relates to a kind of user fatigue state intervention method, device, equipment and medium, the method comprises: obtaining user physiological data, user behavior data and user environment data, user physiological data and user behavior data are time series registration, and based on the time series registration after user physiological data and the time series registration after user behavior data constructs multi-source driving state monitoring time series data;Multi-source driving state monitoring time series data is input into user fatigue state assessment model, and the user fatigue present situation grade and user fatigue development grade are obtained;Call user fatigue state intervention measure set corresponding to user fatigue present situation grade;Based on user environment data and user fatigue development grade, user fatigue state intervention measure is filtered from user fatigue state intervention measure set, and user fatigue state intervention measure is executed.The present application can improve the accuracy of state recognition and the effectiveness of intervention by multidimensional data processing, fatigue state assessment and hierarchical intervention measures.

Description

Technical Field

[0001] This invention belongs to the field of driver assistance, and in particular relates to a method, device, equipment and medium for intervening in user fatigue. Background Technology

[0002] With the development of technology in the field of intelligent driving, fatigue driving intervention technology has emerged. This technology can perceive and intervene in the user's state during driving. Traditional fatigue driving intervention methods mostly rely on the driver's self-discipline and basic reminders from onboard warning devices, lacking the ability to accurately identify and target the user's fatigue state.

[0003] Traditional technologies often focus on collecting and analyzing single types of data, failing to comprehensively reflect a user's true fatigue state and leading to potential biases in fatigue assessment. Furthermore, existing intervention methods are relatively limited, primarily relying on audio and visual alerts, which are insufficient to effectively manage fatigue at different stages, resulting in weak risk control capabilities for fatigued driving. Moreover, warnings are often issued only after a fatigued state has already posed a risk, hindering early prediction and effective intervention. Summary of the Invention

[0004] Therefore, it is necessary to provide a user fatigue state intervention method, device, equipment, and medium that can accurately intervene in user fatigue state based on multi-dimensional data fusion to address the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a method for intervening in user fatigue, comprising:

[0006] Acquire user physiological data, user behavior data, and user environment data; perform time-series registration on the user physiological data and user behavior data; and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and user behavior data.

[0007] Multi-source driving status monitoring time series data are input into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level;

[0008] Call upon the set of user fatigue status intervention measures corresponding to the user's current fatigue level;

[0009] Based on user environment data and user fatigue development levels, we select user fatigue intervention measures from a set of user fatigue intervention measures and implement user fatigue intervention measures.

[0010] Furthermore, based on user environment data and user fatigue development levels, intervention measures for user fatigue status are screened from the set of user fatigue status intervention measures, including:

[0011] Based on the power set method, from the set of intervention measures corresponding to the user fatigue status level, a set of non-empty measure combinations is enumerated. Based on user environment data, non-empty measure combinations that violate environmental constraints are eliminated from the set of non-empty measure combinations to obtain a candidate combination set.

[0012] The multi-objective function values ​​of each candidate combination in the candidate combination set are calculated based on the multi-objective function, and a multi-objective function value matrix is ​​constructed. Based on the multi-objective function value matrix, the pairwise dominance relationship of the candidate combinations is judged, and the dominated candidate combinations are eliminated one by one to obtain the Pareto optimal solution set and the Pareto front. Among them, the Pareto optimal solution in the Pareto optimal solution set is the candidate combination that is not dominated by any other candidate combination in the candidate combination set, and the Pareto front is the set of multi-objective function values ​​of the Pareto optimal solution.

[0013] Multi-objective weights are assigned to a multi-objective function based on user environment data and user fatigue development levels.

[0014] The ideal point is obtained by solving the multi-objective function and multi-objective weights. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal intervention combination. The optimal intervention combination is used to characterize the user fatigue state intervention measures selected from the user fatigue state intervention measure set.

[0015] Furthermore, the multi-objective function includes an environmental adaptability objective function and a fatigue development level adaptability objective function. The multi-objective function values ​​include the environmental adaptability objective function value corresponding to the environmental adaptability objective function and the fatigue development level adaptability objective function value corresponding to the fatigue development level adaptability objective function. The multi-objective weights include the environmental adaptability weight corresponding to the environmental adaptability objective function and the fatigue development level adaptability weight corresponding to the fatigue development level adaptability objective function. The expressions for the multi-objective function and the multi-objective weights are as follows:

[0016]

[0017]

[0018]

[0019]

[0020] In the formula, For candidate combinations, and These are the objective function values ​​for the environmental adaptability and the fatigue development level adaptability of the candidate combinations, respectively. For environmental parameter weight vectors, For element-wise multiplication, This is the environmental state vector corresponding to the user's environmental data. The environmental requirement vector for candidate combinations. For the Euclidean norm, For normalization function, For the fatigue development level adaptation vector of the candidate combination, This represents the fatigue development level vector corresponding to the user's fatigue development level. and These are the weights for environmental adaptability and fatigue development level adaptability, respectively. The concentration parameter of the weight distribution. and These are the scenario importance scores for the environmental adaptability target and the fatigue development level adaptability target, respectively. It is a natural constant.

[0021] Preferably, the user fatigue status level includes mild fatigue, moderate fatigue, and severe fatigue. The user fatigue status intervention measure set includes an audio reminder intervention measure set, a physical stimulus intervention measure set, and an assisted driving intervention measure set. Calling the user fatigue status intervention measure set corresponding to the user fatigue status level includes:

[0022] If the user's current fatigue level is mild fatigue, invoke the set of audio reminder intervention measures;

[0023] If the user's current fatigue level is moderate fatigue, invoke the set of physical stimulation intervention measures;

[0024] If the user's current fatigue level is severe fatigue, invoke the set of driver assistance intervention measures.

[0025] In one embodiment, the set of audio alert interventions includes rhythmic audio interventions and voice alert interventions;

[0026] The set of physical stimulation interventions includes seat vibration interventions and air conditioning interventions;

[0027] The set of driver assistance interventions includes warning light control interventions and rest area navigation interventions.

[0028] In one embodiment, user physiological data includes eyelid closure degree, gaze deviation angle, and head posture angle;

[0029] User behavior data includes steering wheel grip force, steering angle, and steering frequency;

[0030] User environment data includes traffic flow data, ambient light data, and road type data.

[0031] Furthermore, the multi-source driving state monitoring time-series data is input into the user fatigue state assessment model to obtain the user's current fatigue level and user fatigue development level, including:

[0032] The time-series data of multi-source driving state monitoring are input into the temporal convolution component in the user fatigue state assessment model to obtain the local enhanced temporal features of multi-source driving state.

[0033] After adding temporal location encoding to the local temporal features of multi-source driving state, the input is fed into the encoder of the converter model in the user fatigue state assessment model to obtain the global enhanced temporal features of multi-source driving state.

[0034] The global enhanced temporal features of multi-source driving status are input into the fatigue status level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue status level.

[0035] The global enhanced temporal features of multi-source driving status are input into the fatigue development level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue development level; whereby the user fatigue development level is used to characterize the predicted future user fatigue status level.

[0036] Secondly, the present invention also provides a user fatigue state intervention device, comprising:

[0037] The multi-source data preprocessing module is used to acquire user physiological data, user behavior data and user environment data, perform time-series registration on user physiological data and user behavior data, and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data.

[0038] The user fatigue status assessment module is used to input multi-source driving status monitoring time series data into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level.

[0039] The intervention measure set invocation module is used to invoke the user fatigue status intervention measure set corresponding to the user fatigue status level;

[0040] The intervention execution module is used to select user fatigue state intervention measures from the user fatigue state intervention measure set based on user environment data and user fatigue development level, and then execute the user fatigue state intervention measures.

[0041] Thirdly, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method as described in any of the first aspects of the present invention.

[0042] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the first aspects of the present invention.

[0043] The beneficial effects of this invention are as follows: This invention discloses a method, device, equipment, and medium for intervening in user fatigue. By synchronously collecting user physiological data, user behavioral data, and user environmental data, and aligning them with timestamps, it can avoid feature misalignment caused by differences in data collection delays and ensure the alignment of data in each dimension over time. By using user environmental data and fatigue development levels as the basis for selecting intervention measures, the intervention strategy can be dynamically adjusted to eliminate measures that conflict with the current scenario and select a combination of solutions that can effectively alleviate fatigue and comply with driving regulations. This reduces the intervention risk caused by mismatch between measures and the environment, while also improving the user experience. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating the application environment of a user fatigue state intervention method according to an embodiment of the present invention;

[0046] Figure 2 A flowchart illustrating a user fatigue state intervention method provided in one embodiment of the present invention. Figure 1 ;

[0047] Figure 3 A flowchart illustrating a user fatigue state intervention method provided in one embodiment of the present invention. Figure 2 ;

[0048] Figure 4 This is a schematic diagram of a user fatigue state intervention device provided in one embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] The user fatigue state intervention method provided in this embodiment of the invention can be applied to, for example... Figure 1In the application environment shown, the data computing terminal 101 can communicate with the data acquisition terminal 102, the data response terminal 103, and the database 104 via a communication channel. The database 104 can store the data that the data computing terminal 101 needs to process. The database 104 can be integrated into the data computing terminal 101 or placed in the cloud or on a network server. The data computing terminal 101 can collect basic user status data based on the data acquisition terminal 102, identify the user's fatigue state based on the collected user status data, and generate fatigue intervention instructions based on the user's fatigue state. The data computing terminal 101 can send the generated fatigue intervention instructions to the data response terminal 103.

[0051] In one exemplary embodiment, such as Figure 2 As shown, a user fatigue state intervention method is provided, which can be applied to... Figure 1 Taking the data computing terminal 101 as an example, the explanation includes the following steps S201 to S204. Wherein:

[0052] Step S201: Obtain user physiological data, user behavior data, and user environment data; perform time-series registration on the user physiological data and user behavior data; and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data.

[0053] Specifically, the data computing terminal 101 can acquire user physiological data, user behavior data, and user environmental data collected by the data acquisition terminal 102 through a communication channel. The data computing terminal 101 can perform time-series registration on the user physiological data and user behavior data, or on the user physiological data, user behavior data, and user environmental data; this is not limited to any particular type. Based on the time-series registered user physiological data and time-series registered user behavior data, the data computing terminal 101 can construct multi-source driving state monitoring time-series data.

[0054] Optionally, user physiological data may include, but is not limited to, eyelid closure degree, gaze deviation angle, and head posture angle. User behavioral data may include, but is not limited to, steering wheel grip force, steering angle, and steering frequency. User environmental data may include, but is not limited to, traffic flow data, ambient brightness data, and road type data.

[0055] For example, the expression for eyelid closure degree can be:

[0056]

[0057]

[0058] In the formula, The degree of eyelid closure is quantified based on the proportion of eyelid closure time per unit time. The sampling period for eyelid closure. The first time within the eyelid closure sampling period The eyelid opening / closing ratio in time frames. and These are the sampling periods for eyelid closure. The coordinates of the highest point of the upper eyelid and the lowest point of the lower eyelid in the time frame. The first time within the eyelid closure sampling period Iris diameter in time frame The indicator function is conditionally true. This is the eyelid closure threshold.

[0059] Optionally, the head attitude angles may include, but are not limited to, the head pitch angle, the head yaw angle, and the head roll angle. Among them, the head roll angle is used to characterize the degree of left and right tilt of the head, and the head yaw angle is used to characterize the range of left and right rotation of the head.

[0060] Furthermore, user physiological data may also include respiratory rate, blood oxygen saturation, and heart rate, without limitation.

[0061] Furthermore, user behavior data may also include driving time, facial expressions, and user posture, without limitation.

[0062] Furthermore, user environment data may also include light intensity, in-vehicle temperature, in-vehicle humidity, ambient temperature, ambient humidity, and noise levels, without limitation.

[0063] Indicatively, in-vehicle temperature, in-vehicle humidity, ambient temperature, and ambient humidity can be used to screen for measures to reduce window interventions.

[0064] Step S202: Input the multi-source driving status monitoring time series data into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level.

[0065] Specifically, the data computing terminal 101 can input multi-source driving status monitoring time series data into the user fatigue status assessment model to perform feature extraction, feature enhancement and feature decoding to obtain the user fatigue status level and the user fatigue development level.

[0066] Optionally, the data computing terminal 101 can input multi-source driving state monitoring time-series data into the temporal convolutional component of the user fatigue state assessment model to obtain local enhanced temporal features of the multi-source driving state. The data computing terminal 101 can add temporal position encoding to the local temporal features of the multi-source driving state and then input them into the converter model encoder of the user fatigue state assessment model to obtain global enhanced temporal features of the multi-source driving state. The data computing terminal 101 can input the global enhanced temporal features of the multi-source driving state into the fatigue status level inference converter model decoder of the user fatigue state assessment model to obtain the user fatigue status level. The data computing terminal 101 can input the global enhanced temporal features of the multi-source driving state into the fatigue development level inference converter model decoder of the user fatigue state assessment model to obtain the user fatigue development level.

[0067] Optionally, the user fatigue development level can be used to characterize the predicted level of user fatigue in the future.

[0068] Step S203: Invoke the set of user fatigue status intervention measures corresponding to the user's current fatigue status level.

[0069] Optionally, the user fatigue status level may include, but is not limited to, a mild fatigue level, a moderate fatigue level, and a severe fatigue level. The user fatigue status intervention set may include, but is not limited to, an audio reminder intervention set corresponding to the mild fatigue level, a physical stimulation intervention set corresponding to the moderate fatigue level, and a driving assistance intervention set corresponding to the severe fatigue level.

[0070] Step S204: Based on user environment data and user fatigue development level, select user fatigue state intervention measures from the set of user fatigue state intervention measures, and implement user fatigue state intervention measures.

[0071] Specifically, the data computing terminal 101 can filter user fatigue state intervention measures from a set of user fatigue state intervention measures based on user environment data and user fatigue development level. The data computing terminal 101 can send the filtered user fatigue state intervention measures to the data response terminal 103 and control the data response terminal 103 to execute the user fatigue state intervention measures.

[0072] The aforementioned user fatigue intervention methods, through multi-source data fusion and time-series registration, can comprehensively capture user states, ensure the accuracy of the correlation between multi-source data, and improve the accuracy of fatigue state identification; through a multi-dimensional fatigue state assessment model, it can dynamically perceive user fatigue states, accurately quantify the current fatigue level, predict the development trend of fatigue states, and reduce the risk of fatigue state deterioration; through the invocation of a graded set of intervention measures and scenario-based screening, it can enhance the practicality and adaptability of intervention measures and improve the intervention effect.

[0073] In an optional embodiment of the present invention, please refer to Figure 3 Based on user environment data and user fatigue development levels, intervention measures for user fatigue status are screened from a set of user fatigue status intervention measures, which may include:

[0074] Step S304: Based on the power set method, enumerate the non-empty combination set of intervention measures corresponding to the user fatigue status level, and based on the user environment data, eliminate the non-empty combination of measures that violate the environmental constraints in the non-empty combination set to obtain the candidate combination set.

[0075] Step S305: Calculate the multi-objective function value of each candidate combination in the candidate combination set based on the multi-objective function, construct the multi-objective function value matrix, and judge the pairwise dominance relationship of the candidate combinations based on the multi-objective function value matrix, and eliminate the dominated candidate combinations one by one to obtain the Pareto optimal solution set and the Pareto front.

[0076] Optionally, the Pareto optimal solution in the Pareto optimal solution set is a candidate combination that is not dominated by any other candidate combination in the candidate combination set, and the Pareto front is the set of multi-objective function values ​​of the Pareto optimal solution.

[0077] Step S306: Assign multi-objective weights to the multi-objective function based on user environment data and user fatigue development level.

[0078] Step S307: Based on the multi-objective function and multi-objective weights, the ideal point is obtained. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal combination of intervention measures.

[0079] Optionally, the optimal combination of interventions is used to characterize the user fatigue state interventions selected from the set of user fatigue state interventions.

[0080] Schematic, the data computing terminal 101 can set the Pareto optimal solution corresponding to the optimal value of each objective function in the multi-objective function of each Pareto optimal solution as the ideal point corresponding to each objective function. The data computing terminal 101 can also set the multi-objective weights corresponding to each objective function as the weight attributes of the ideal points corresponding to each objective function. Furthermore, the data computing terminal 101 can set the Pareto optimal solution corresponding to the worst value of each objective function in the multi-objective function of each Pareto optimal solution as the anti-ideal point corresponding to each objective function.

[0081] The data computing terminal 101 can calculate the difference between the objective function value of each Pareto optimal solution and the ideal point of the objective function. The data computing terminal 101 can calculate the maximum difference between the anti-ideal point and the ideal point of each objective function. The data computing terminal 101 can divide the difference between the ideal objective function of each objective function of each Pareto optimal solution by the maximum difference between the objective functions of each objective function to calculate the standardized objective value of each objective function of each Pareto optimal solution.

[0082] The data computing terminal 101 can weight the standardized objective values ​​of each objective function of each Pareto optimal solution based on the multi-objective weights corresponding to each objective function, obtaining weighted standardized objective values. The data computing terminal 101 can set the maximum value among the weighted standardized objective values ​​of each Pareto optimal solution as the weighted Chebyshev distance of each Pareto optimal solution. The data computing terminal 101 can select the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set as the optimal combination of intervention measures.

[0083] For example, taking a multi-objective function including an environmental adaptability objective function and a fatigue development level adaptability objective function as an example, the data computing terminal 101 can set the Pareto optimal solution corresponding to the optimal value of the environmental adaptability objective function of each Pareto optimal solution as the ideal point of the environmental adaptability objective function corresponding to the environmental adaptability objective function. The data computing terminal 101 can also set the Pareto optimal solution corresponding to the optimal value of the fatigue development level adaptability objective function of each Pareto optimal solution as the ideal point of the fatigue development level adaptability objective function corresponding to the fatigue development level adaptability objective function.

[0084] The data computing terminal 101 can set the environmental adaptability weight as the weight attribute corresponding to the ideal point of the environmental adaptability objective function, and the data computing terminal 101 can set the fatigue development level adaptability weight as the weight attribute corresponding to the ideal point of the fatigue development level adaptability objective function. The data computing terminal 101 can set the Pareto optimal solution corresponding to the worst value of the environmental adaptability objective function of each Pareto optimal solution as the inverse ideal point of the environmental adaptability objective function, and the data computing terminal 101 can set the Pareto optimal solution corresponding to the worst value of the fatigue development level adaptability objective function of each Pareto optimal solution as the inverse ideal point of the fatigue development level adaptability objective function.

[0085] The data computing terminal 101 can calculate the difference between the environment fitness objective function value of each Pareto optimal solution and the ideal point of the environment fitness objective function. The data computing terminal 101 can also calculate the maximum difference between the anti-ideal point of the environment fitness objective function and the ideal point of the environment fitness objective function. The data computing terminal 101 can divide the difference between the ideal and ideal points of each Pareto optimal solution by the maximum difference in the environment fitness objective function to obtain the standardized environment fitness objective value of each Pareto optimal solution.

[0086] The data computing terminal 101 can calculate the difference between the fatigue development level fit objective function value of each Pareto optimal solution and the ideal point of the fatigue development level fit objective function. The data computing terminal 101 can also calculate the maximum difference between the anti-ideal point of the fatigue development level fit objective function and the ideal point of the fatigue development level fit objective function. The data computing terminal 101 can divide the difference between the ideal fatigue development level fit objective function of each Pareto optimal solution by the maximum difference between the ideal and ideal points of the fatigue development level fit objective function to obtain the standardized fatigue development level fit objective value of each Pareto optimal solution.

[0087] The data computing terminal 101 can weight the standardized environmental adaptability target value based on the environmental adaptability weight to obtain the weighted standardized environmental adaptability target value. The data computing terminal 101 can also weight the standardized fatigue development level adaptability target value based on the fatigue development level adaptability weight to obtain the weighted standardized fatigue development level adaptability target value.

[0088] The data computing terminal 101 can set the maximum value of the weighted normalized environmental fitness target value and the weighted normalized fatigue development level fitness target value of each Pareto optimal solution as the weighted Chebyshev distance of each Pareto optimal solution. The data computing terminal 101 can select the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set as the optimal combination of intervention measures.

[0089] In an optional embodiment of the present invention, the multi-objective function may include an environmental adaptability objective function and a fatigue development level adaptability objective function. The multi-objective function value may include the environmental adaptability objective function value corresponding to the environmental adaptability objective function and the fatigue development level adaptability objective function value corresponding to the fatigue development level adaptability objective function. The multi-objective weight may include the environmental adaptability weight corresponding to the environmental adaptability objective function and the fatigue development level adaptability weight corresponding to the fatigue development level adaptability objective function. The expressions for the multi-objective function and the multi-objective weight can be:

[0090]

[0091]

[0092]

[0093]

[0094] In the formula, For candidate combinations, and These are the objective function values ​​for the environmental adaptability and the fatigue development level adaptability of the candidate combinations, respectively. For environmental parameter weight vectors, For element-wise multiplication, This is the environmental state vector corresponding to the user's environmental data. The environmental requirement vector for candidate combinations. For the Euclidean norm, For normalization function, For the fatigue development level adaptation vector of the candidate combination, This represents the fatigue development level vector corresponding to the user's fatigue development level. and These are the weights for environmental adaptability and fatigue development level adaptability, respectively. The concentration parameter of the weight distribution. and These are the scenario importance scores for the environmental adaptability target and the fatigue development level adaptability target, respectively. It is a natural constant.

[0095] In an optional embodiment of the present invention, the user fatigue status level may include a mild fatigue level, a moderate fatigue level, and a severe fatigue level. The user fatigue status intervention measure set may include an audio reminder intervention measure set, a physical stimulation intervention measure set, and a driver assistance intervention measure set. Calling the user fatigue status intervention measure set corresponding to the user fatigue status level may include:

[0096] Specifically, if the user's current fatigue level is mild fatigue, the data computing terminal 101 can call up the set of audio reminder intervention measures.

[0097] Optionally, audio alert interventions may include, but are not limited to, dynamic music interventions and voice alert interventions.

[0098] Specifically, if the user's current fatigue level is moderate, the data calculation terminal 101 can call up the set of physical stimulation intervention measures.

[0099] Optional, physical stimulation interventions may include, but are not limited to, seat vibration interventions and cool air interventions.

[0100] Specifically, if the user's current fatigue level is severe fatigue, the data computing terminal 101 can call up the set of assisted driving intervention measures.

[0101] Optionally, driver assistance interventions may include, but are not limited to, warning light flashing interventions, window lowering interventions, and rest area navigation interventions. The rest area navigation intervention is used to provide the user with navigation to the nearest safe rest area.

[0102] In an optional embodiment of the invention, the set of audio alert interventions may include rhythmic audio interventions and voice alert interventions. The set of physical stimulus interventions may include seat vibration interventions and air conditioning interventions. The set of driver assistance interventions may include hazard light control interventions and rest area navigation interventions.

[0103] In an optional embodiment of the invention, user physiological data may include eyelid closure degree, gaze deviation angle, and head posture angle. User behavioral data may include steering wheel grip force, steering angle, and steering frequency. User environmental data may include traffic flow data, ambient brightness data, and road type data.

[0104] In an optional embodiment of the present invention, inputting multi-source driving state monitoring time-series data into a user fatigue state assessment model to obtain the user's current fatigue level and user fatigue development level may include:

[0105] Specifically, the data computing terminal 101 can input multi-source driving state monitoring time-series data into the temporal convolution component in the user fatigue state assessment model to obtain multi-source driving state local enhanced temporal features.

[0106] Specifically, the data computing terminal 101 can add temporal position encoding to the local temporal features of the multi-source driving state and input them into the converter model encoder in the user fatigue state assessment model to obtain the global enhanced temporal features of the multi-source driving state.

[0107] Specifically, the data computing terminal 101 can input the global enhanced temporal features of multi-source driving status into the fatigue status level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue status level.

[0108] Specifically, the data computing terminal 101 can input the global enhanced temporal features of multi-source driving status into the fatigue development level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue development level.

[0109] Optionally, the user fatigue development level can be used to characterize the predicted level of user fatigue in the future.

[0110] In one exemplary embodiment of the present invention, such as Figure 3 As shown, a method for intervening in user fatigue state is provided, including:

[0111] Step S301: Obtain user physiological data, user behavior data, and user environment data; perform time-series registration on the user physiological data and user behavior data; and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data.

[0112] Step S302: Input the multi-source driving status monitoring time series data into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level.

[0113] Step S303: Call the set of user fatigue status intervention measures corresponding to the user fatigue status level.

[0114] Step S304: Based on the power set method, enumerate the non-empty combination set of intervention measures corresponding to the user fatigue status level, and based on the user environment data, eliminate the non-empty combination of measures that violate the environmental constraints in the non-empty combination set to obtain the candidate combination set.

[0115] Optionally, environmental constraints may include window intervention environmental constraints based on in-vehicle temperature, in-vehicle humidity, ambient temperature, and ambient humidity that can filter for reducing window intervention measures. Environmental constraints may also include traffic flow environmental constraints, ambient brightness environmental constraints, and road type environmental constraints that respectively filter for combinations of non-empty measures based on traffic flow data, ambient brightness data, and road type data. Environmental constraints may also include comprehensive environmental constraints that combine at least two of the traffic flow data, ambient brightness data, and road type data environments to filter for combinations of non-empty measures.

[0116] Step S305: Calculate the multi-objective function value of each candidate combination in the candidate combination set based on the multi-objective function, construct the multi-objective function value matrix, and judge the pairwise dominance relationship of the candidate combinations based on the multi-objective function value matrix, and eliminate the dominated candidate combinations one by one to obtain the Pareto optimal solution set and the Pareto front.

[0117] Step S306: Assign multi-objective weights to the multi-objective function based on user environment data and user fatigue development level.

[0118] Step S307: Based on the multi-objective function and multi-objective weights, the ideal point is obtained. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal combination of intervention measures.

[0119] Step S308: Implement user fatigue intervention measures.

[0120] The aforementioned user fatigue intervention methods, through multi-source data collection, dual-dimensional fatigue state assessment, environmental constraint screening, and multi-objective function analysis, can improve the accuracy of fatigue assessment, ensure the scientific nature and pertinence of the intervention measures combination, and enhance the adaptability of the intervention plan to the environment and dynamic changes in fatigue. This can improve the precision and effectiveness of fatigue intervention and provide reliable protection for user safety.

[0121] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0122] Based on the same inventive concept, embodiments of the present invention also provide a user fatigue state intervention device for implementing the user fatigue state intervention method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more user fatigue state intervention device embodiments provided below can be found in the limitations of the user fatigue state intervention method described above, and will not be repeated here.

[0123] In one exemplary embodiment, such as Figure 4 As shown, a user fatigue state intervention device 400 is provided, comprising:

[0124] The multi-source data preprocessing module 401 can be used to acquire user physiological data, user behavior data and user environmental data, perform time-series registration on user physiological data and user behavior data, and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data.

[0125] The user fatigue status assessment module 402 can be used to input multi-source driving status monitoring time series data into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level.

[0126] The intervention measure set invocation module 403 can be used to invoke the user fatigue status intervention measure set corresponding to the user fatigue status level.

[0127] The intervention execution module 404 can be used to select user fatigue state intervention measures from the user fatigue state intervention measure set based on user environment data and user fatigue development level, and then execute the user fatigue state intervention measures.

[0128] In an optional embodiment of the present invention, the intervention execution module 404 may also be used for:

[0129] Based on the power set method, a set of non-empty measure combinations is enumerated from the set of user fatigue status intervention measures corresponding to the current user fatigue level. Based on user environment data, non-empty measure combinations that violate environmental constraints are eliminated from the set of non-empty measure combinations to obtain a candidate combination set.

[0130] The multi-objective function values ​​of each candidate combination in the candidate combination set are calculated based on the multi-objective function, and a multi-objective function value matrix is ​​constructed. Based on this matrix, pairwise dominance relationships are determined among the candidate combinations, and dominated candidate combinations are eliminated one by one, resulting in the Pareto optimal solution set and the Pareto front. Specifically, the Pareto optimal solutions in the Pareto optimal solution set are candidate combinations that are not dominated by any other candidate combinations in the candidate combination set, and the Pareto front is the set of multi-objective function values ​​of the Pareto optimal solutions.

[0131] Multi-objective weights are assigned to a multi-objective function based on user environment data and user fatigue development level.

[0132] The ideal point is obtained by solving the multi-objective function and multi-objective weights. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal intervention combination. The optimal intervention combination is used to characterize the user fatigue state intervention measures selected from the set of user fatigue state intervention measures.

[0133] In an optional embodiment of the present invention, the intervention set invocation module 403 may also be used for:

[0134] If the user's current fatigue level is mild, invoke the set of audio reminder intervention measures.

[0135] If the user's current fatigue level is moderate, then invoke the set of physical stimulation intervention measures.

[0136] If the user's current fatigue level is severe fatigue, invoke the set of driver assistance intervention measures.

[0137] In an optional embodiment of the present invention, the user fatigue status assessment module 402 can also be used for:

[0138] The time-series data of multi-source driving state monitoring are input into the temporal convolution component in the user fatigue state assessment model to obtain the local enhanced temporal features of multi-source driving state.

[0139] After adding temporal position encoding to the local temporal features of multi-source driving states, the data is input into the encoder of the converter model in the user fatigue state assessment model to obtain the global enhanced temporal features of multi-source driving states.

[0140] The global enhanced temporal features of multi-source driving status are input into the fatigue status level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue status level.

[0141] The global enhanced temporal features of multi-source driving states are input into the fatigue development level inference converter model decoder in the user fatigue state assessment model to obtain the user fatigue development level. The user fatigue development level is used to characterize the predicted future user fatigue status level.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a user fatigue state intervention method as described above.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0144] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0145] The above-described embodiments are merely illustrative of several implementation methods of the present invention, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A method for intervening in user fatigue, characterized in that, The method includes: Acquire user physiological data, user behavior data, and user environment data; perform time-series registration on the user physiological data and user behavior data; and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data. The multi-source driving state monitoring time series data is input into the user fatigue state assessment model to obtain the user fatigue status level and the user fatigue development level. The user fatigue development level is used to characterize the predicted future user fatigue status level. Call upon the set of user fatigue status intervention measures corresponding to the user's current fatigue level; Based on the user environment data and user fatigue development level, user fatigue intervention measures are selected from the set of user fatigue intervention measures, and the user fatigue intervention measures are executed. The step of selecting user fatigue state intervention measures from the set of user fatigue state intervention measures based on the user environment data and user fatigue development level includes: Based on the power set method, from the set of intervention measures for user fatigue status corresponding to the user fatigue status level, a set of non-empty measure combinations is enumerated. Based on the user environment data, non-empty measure combinations that violate environmental constraints are removed from the set of non-empty measure combinations to obtain a candidate combination set. The multi-objective function values ​​of each candidate combination in the candidate combination set are calculated based on the multi-objective function, a multi-objective function value matrix is ​​constructed, and the candidate combinations are judged pairwise based on the multi-objective function value matrix. The dominated candidate combinations are eliminated one by one to obtain the Pareto optimal solution set and the Pareto front. The Pareto optimal solution in the Pareto optimal solution set is the candidate combination that is not dominated by any other candidate combination in the candidate combination set, and the Pareto front is the set of multi-objective function values ​​of the Pareto optimal solution. The multi-objective weights of the multi-objective function are assigned based on the user environment data and the user fatigue development level. The ideal point is obtained by solving the multi-objective function and the multi-objective weights. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal intervention combination. The optimal intervention combination is used to characterize the user fatigue state intervention measures selected from the user fatigue state intervention measure set. The multi-objective function includes an environmental adaptability objective function and a fatigue development level adaptability objective function. The multi-objective function values ​​include the environmental adaptability objective function value corresponding to the environmental adaptability objective function and the fatigue development level adaptability objective function value corresponding to the fatigue development level adaptability objective function. The multi-objective weights include the environmental adaptability weight corresponding to the environmental adaptability objective function and the fatigue development level adaptability weight corresponding to the fatigue development level adaptability objective function. The expressions for the multi-objective function and the multi-objective weights are as follows: In the formula, For candidate combinations, and These are the objective function values ​​for the environmental adaptability and the fatigue development level adaptability of the candidate combinations, respectively. For environmental parameter weight vectors, For element-wise multiplication, This is the environmental state vector corresponding to the user's environmental data. The environmental requirement vector for candidate combinations. For the Euclidean norm, For normalization function, For the fatigue development level adaptation vector of the candidate combination, This represents the fatigue development level vector corresponding to the user's fatigue development level. and These are the weights for environmental adaptability and fatigue development level adaptability, respectively. The concentration parameter of the weight distribution. and These are the scenario importance scores for the environmental adaptability target and the fatigue development level adaptability target, respectively. It is a natural constant.

2. The method according to claim 1, characterized in that, The user fatigue status level includes mild fatigue, moderate fatigue, and severe fatigue. The user fatigue status intervention measure set includes an audio reminder intervention measure set, a physical stimulus intervention measure set, and a driver assistance intervention measure set. Calling the user fatigue status intervention measure set corresponding to the user fatigue status level includes: If the user's current fatigue level is the user's mild fatigue level, then invoke the set of audio reminder intervention measures; If the user's current fatigue level is moderate fatigue, then the set of physical stimulation intervention measures is invoked. If the user's current fatigue level is the severe fatigue level, the set of assisted driving intervention measures will be invoked.

3. The method according to claim 2, characterized in that: The set of audio alert intervention measures includes rhythmic audio intervention measures and voice alert intervention measures; The set of physical stimulation interventions includes seat vibration interventions and air conditioning interventions; The set of driver assistance interventions includes warning light control interventions and rest area navigation interventions.

4. The method according to claim 1, characterized in that: The user's physiological data includes eyelid closure degree, gaze deviation angle, and head posture angle; The user behavior data includes steering wheel grip force, steering angle, and steering frequency; The user environment data includes traffic flow data, ambient brightness data, and road type data.

5. The method according to claim 1, characterized in that, The step of inputting the multi-source driving state monitoring time-series data into the user fatigue state assessment model to obtain the user's current fatigue level and user fatigue development level includes: The multi-source driving state monitoring time series data is input into the temporal convolution component in the user fatigue state assessment model to obtain multi-source driving state local enhanced temporal features. After adding temporal position encoding to the local temporal features of the multi-source driving state, the input is fed into the encoder of the converter model in the user fatigue state assessment model to obtain the global enhanced temporal features of the multi-source driving state. The multi-source driving state global enhanced temporal features are input into the fatigue status level inference converter model decoder in the user fatigue status assessment model to obtain the user fatigue status level. The multi-source driving state global enhanced temporal features are input into the fatigue development level inference converter model decoder in the user fatigue state assessment model to obtain the user fatigue development level.

6. A user fatigue state intervention device, characterized in that, include: A multi-source data preprocessing module is used to acquire user physiological data, user behavior data, and user environment data, perform time-series registration on the user physiological data and user behavior data, and construct multi-source driving state monitoring time-series data based on the time-series registered user physiological data and time-series registered user behavior data. The user fatigue status assessment module is used to input the multi-source driving status monitoring time series data into the user fatigue status assessment model to obtain the user fatigue status level and the user fatigue development level. The user fatigue development level is used to characterize the predicted future user fatigue status level. The intervention measure set invocation module is used to invoke the user fatigue status intervention measure set corresponding to the user fatigue status level; The intervention execution module is used to select user fatigue state intervention measures from the set of user fatigue state intervention measures based on the user environment data and the user fatigue development level, and to execute the user fatigue state intervention measures. The step of selecting user fatigue state intervention measures from the set of user fatigue state intervention measures based on the user environment data and user fatigue development level includes: Based on the power set method, from the set of intervention measures for user fatigue status corresponding to the user fatigue status level, a set of non-empty measure combinations is enumerated. Based on the user environment data, non-empty measure combinations that violate environmental constraints are removed from the set of non-empty measure combinations to obtain a candidate combination set. The multi-objective function values ​​of each candidate combination in the candidate combination set are calculated based on the multi-objective function, a multi-objective function value matrix is ​​constructed, and the candidate combinations are judged pairwise based on the multi-objective function value matrix. The dominated candidate combinations are eliminated one by one to obtain the Pareto optimal solution set and the Pareto front. The Pareto optimal solution in the Pareto optimal solution set is the candidate combination that is not dominated by any other candidate combination in the candidate combination set, and the Pareto front is the set of multi-objective function values ​​of the Pareto optimal solution. The multi-objective weights of the multi-objective function are assigned based on the user environment data and the user fatigue development level. The ideal point is obtained by solving the multi-objective function and the multi-objective weights. Combining the multi-objective weights and the Pareto front, the Pareto optimal solution with the smallest weighted Chebyshev distance to the ideal point in the Pareto optimal solution set is selected as the optimal intervention combination. The optimal intervention combination is used to characterize the user fatigue state intervention measures selected from the user fatigue state intervention measure set. The multi-objective function includes an environmental adaptability objective function and a fatigue development level adaptability objective function. The multi-objective function values ​​include the environmental adaptability objective function value corresponding to the environmental adaptability objective function and the fatigue development level adaptability objective function value corresponding to the fatigue development level adaptability objective function. The multi-objective weights include the environmental adaptability weight corresponding to the environmental adaptability objective function and the fatigue development level adaptability weight corresponding to the fatigue development level adaptability objective function. The expressions for the multi-objective function and the multi-objective weights are as follows: In the formula, For candidate combinations, and These are the objective function values ​​for the environmental adaptability and the fatigue development level adaptability of the candidate combinations, respectively. For environmental parameter weight vectors, For element-wise multiplication, This is the environmental state vector corresponding to the user's environmental data. The environmental requirement vector for candidate combinations. For the Euclidean norm, For normalization function, For the fatigue development level adaptation vector of the candidate combination, This represents the fatigue development level vector corresponding to the user's fatigue development level. and These are the weights for environmental adaptability and fatigue development level adaptability, respectively. The concentration parameter of the weight distribution. and These are the scenario importance scores for the environmental adaptability target and the fatigue development level adaptability target, respectively. It is a natural constant.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

Images