A fatigue monitoring method based on multi-modal data fusion

Abstract

The application discloses a fatigue monitoring method based on multi-modal data fusion, and belongs to the technical field of fatigue monitoring, and the specific steps comprise: synchronously collecting multi-source data, wherein the multi-source data comprises eye movement data, seat pressure data and environmental data; pre-processing the multi-source data, including IPD dynamic calibration on the eye movement data, COP offset calculation on the seat pressure data, and illumination compensation on the environmental data; dynamic interactive calibration, including mutual inspection of the eye movement data and the seat pressure data, environmental data detection and output of a dynamic weight vector; cross-modal feature fusion, fatigue degree grade determination; real-time dynamic updating, and parameter back propagation to the cross-modal feature fusion, weight correction dynamic interactive calibration, and if abnormal data occurs, the abnormal data is marked to data preprocessing. The application realizes comprehensive monitoring of the fatigue state of a user by synchronously collecting multi-source data such as eye movement data, seat pressure data and environmental data.

Description

Technical Field

[0001] This invention belongs to the field of fatigue detection technology, specifically relating to a fatigue monitoring method based on multimodal data fusion. Background Technology

[0002] With the increasing length of office hours in modern society, the problem of prolonged sitting has become increasingly prominent. Prolonged sitting not only leads to physical fatigue but can also trigger a series of health problems. While existing office chairs possess some ergonomic design features, they lack functions to actively alleviate fatigue from prolonged sitting. Most office workers need to manually adjust their chair posture or activate the massage function when they feel fatigued, which is cumbersome and has limited effectiveness.

[0003] While some existing technologies for chair adjustment based on human posture recognition and eye tracking exist, they primarily address general poor posture and lack stress-relieving functions for people who sit for long periods. Prolonged sitting can lead to eye fatigue and difficulty concentrating for office workers, impacting work efficiency. Existing technologies cannot effectively alleviate this physical fatigue.

[0004] Several invention patents have been issued to address the issues of improving office efficiency and health management. For example, patent number CN118749800A discloses a method, system, and smart chair for relieving prolonged sitting pressure. The method includes: detecting pressure signals on the chair surface in response to the activation of a pressure relief mode; after the pressure signal is greater than or equal to a preset pressure threshold for a period of time exceeding a first preset time, driving a massage head to massage the body while simultaneously detecting and recording the body's fatigue level as a first fatigue level; when the massage head massages the body for a period of time exceeding a second preset time, detecting and recording the body's fatigue level as a second fatigue level; if the ratio of the second fatigue level to the first fatigue level is less than or equal to a preset ratio, stopping the massage head massages the body; otherwise, repeating the above steps until the ratio of the second fatigue level to the first fatigue level is less than or equal to the preset ratio. However, this patent still has some issues, such as the need to further optimize the millimeter-wave radar detection algorithm to improve the accuracy and stability of fatigue detection. Furthermore, another patent, CN118340526A, discloses a driving safety monitoring method based on virtual reality. This method collects the driver's eye movement data using an eye-tracking device in a head-mounted display. By using this eye movement data and combining it with a driving scenario model, the driver's gaze point position information is calculated in a virtual reality environment. The head-mounted display uses an infrared light sensor or thermal imaging camera to monitor the driver's head temperature changes and blood flow velocity in real time. The driver's initial fatigue level is calculated using the gaze point position information, head temperature information, and blood flow velocity. Regarding the eye-tracking device, this patent needs further improvement in accuracy and stability to ensure accurate collection of the driver's eye movement data. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the present invention aims to provide a fatigue monitoring method based on multimodal data fusion. This method efficiently acquires and accurately analyzes user fatigue information and triggers a stress relief mechanism based on the fatigue level determination result, thereby overcoming the limitations of existing technologies in accurately monitoring and assessing user fatigue in office scenarios.

[0006] The problem solved by this invention can be achieved through the following specific technical solutions:

[0007] The fatigue monitoring method based on multimodal data fusion includes the following specific steps:

[0008] Step 1: Simultaneously collect multi-source data, including eye-tracking data, seat pressure data, and environmental data;

[0009] Step 2: Preprocess the multi-source data, including dynamic IPD (interpupillary distance) calibration of eye-tracking data, COP (center of pressure) offset calculation of seat pressure data, and illumination compensation of environmental data;

[0010] Step 3: Dynamic interactive calibration, including cross-checking of eye-tracking data and seat pressure data, environmental data detection, and output of dynamic weight vector;

[0011] Step 4: Cross-modal feature fusion for fatigue level determination;

[0012] Step 5: Update dynamically in real time and backfeed the parameters (i.e., some adjustable values ​​or variables involved in the fatigue monitoring method) to cross-modal feature fusion, weight correction and dynamic interactive calibration. If data anomalies occur, mark the abnormal data to data preprocessing.

[0013] Furthermore, in step 1,

[0014] The eye movement data refers to the user's eye image data collected by an eye-tracking camera. The eye image data is processed to calculate the user's eye movement parameters.

[0015] The seat pressure data is obtained by using pressure sensors and gyroscopes integrated into the office chair to capture real-time pressure changes on the seat surface and the seat's movements. The user's posture adjustment is monitored by fusing static pressure and dynamic posture change data.

[0016] The environmental data is obtained by using photosensitive sensors and noise sensors to monitor light intensity and noise levels, in order to identify the impact of external interference on fatigue and assist in the accurate determination of fatigue level.

[0017] Furthermore, in step 2, the specific operation of dynamically calibrating the eye-tracking data using IPD (interpupillary distance) is as follows:

[0018] First, the pupil position is captured by an infrared camera, displaying three calibration points: the center, upper left, and lower right. The formula is as follows:

[0019]

[0020] Among them, IPD base Represents the basic interpupillary distance, (x left ,y left ) and (x right ,y right The coordinates of the left and right pupil centers are shown below.

[0021] Subsequently, the positions of the nose tip and corner of the eyes are obtained in real time through facial key point detection, and the head offset (Δx,Δy) is calculated.

[0022] Finally, IPD is calculated using the supplementary IPD formula. current :

[0023]

[0024] Among them, IPD current This represents the current interpupillary distance after dynamic adjustment. W and H are the screen width and height, and α = 0.1 and β = 0.05 are empirical coefficients. IPD is updated every 60ms to compensate for head micro-movements.

[0025] Furthermore, in step 2, the position of the pressure center and its change over time are calculated by analyzing the data collected by the pressure sensor. The specific operation of COP (pressure center) offset calculation is as follows:

[0026] First, an array of pressure sensors mounted on the seat is used to collect the pressure value Pi from each sensor in real time, i = 1, 2, ..., n, where n is the number of sensors. Simultaneously, the fixed coordinates (x, y) of each sensor are recorded. i ,y i );

[0027] Then, according to the formula The collected pressure values ​​are normalized to eliminate differences in the measurement ranges of different sensors. max P represents the maximum pressure value. min Indicates the minimum pressure value;

[0028] Finally, the coordinates of the pressure center were calculated using the weighted average method. By comparing the COP coordinates at different times, the offset and direction of the COP are obtained, which can be used to analyze changes in human posture and the impact of fatigue on postural stability.

[0029] Furthermore, in step 2, the ambient light compensation is performed by collecting ambient light intensity using a photosensor and monitoring display brightness using a screen brightness sensor. The compensation weight is calculated using the following formula: Where w represents the compensation weight, with a value range of [0,1], I is the real-time collected light intensity, I0 is the reference light intensity, and k is the adjustment system. When k>0, the larger k is, the more drastic the weight changes with the light intensity. When the light intensity I deviates from the reference value I0, the weight w is dynamically adjusted.

[0030] Furthermore, under strong light (I > I0), w approaches 1, increasing the compensation strength; under weak light (I < I0), w approaches 0, reducing overcompensation; the light intensity is divided into multiple intervals, each assigned a fixed weight:

[0031]

[0032] Furthermore, in step 3, the specific operation of dynamic interactive calibration is as follows: performing body motion frequency detection and calculating body motion frequency. If f motion If the frequency is greater than 5 times per minute, it is considered high-frequency body movement. At this point, the weight of eye movement data is reduced. The eye movement weight w eye The value decreased from 0.6 to 0.3. If eye-tracking data showed fatigue but the seat pressure distribution remained stable, the ambient light intensity (Lenv) was checked. If Lenv > 800 lux, it was determined that strong light was interfering with the eye-tracking data, and the seat data was trusted. The posture weight was adjusted to w. seat The weighting increases from 0.3 to 0.6; if Lenv < 300 lux, it is determined that the pupil measurement error is caused by low light, triggering infrared supplementary lighting enhancement; when the collected noise > 80 dB, the reliability of the seat pressure data is improved, and the weighting increases from 0.4 to 0.6.

[0033] Furthermore, in step 4, based on the real-time calibration results of eye-tracking data, seat pressure data, and environmental data, a fusion weight vector of each modal data is generated for fatigue level determination.

[0034] The formula for calculating the weight vector W is: W = [w eye ,w seat ,w env ], w eye w seat w env These are eye-tracking weights, posture weights, and environmental weights, respectively.

[0035] The fatigue comprehensive index (FCI) is calculated using the formula: FCI = w eye ·F eye +w seat ·F seat +w env ·F env ,

[0036] The final fatigue level is determined based on the Fatigue Comprehensive Index (FCI) threshold. FCI < 0.4 indicates mild fatigue, triggering lumbar airbag inflation and screen dimming; FCI ≤ 0.4 < FCI < 0.7 indicates moderate fatigue, triggering a sedentary reminder and seat cushion heating. If the user gets up, the sedentary reminder and seat cushion heating functions will be triggered sequentially; FCI ≥ 0.7 indicates severe fatigue, triggering seat reclining and back massage.

[0037] Furthermore, in step 5, during the user feedback process that triggers fatigue relief, users are allowed to directly and manually intervene in the system actions, which are recorded as negative feedback. At the same time, based on the user's historical behavior data update threshold, the data is dynamically updated in real time, and the parameters are back-transmitted to cross-modal feature fusion and weight correction dynamic interactive calibration. If data anomalies occur, they are marked for data preprocessing.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) This invention achieves comprehensive monitoring of user fatigue by simultaneously collecting multi-source data such as eye movement data, seat pressure data and environmental data, overcoming the shortcomings of existing technologies that rely solely on single modal data, resulting in incomplete and inaccurate monitoring results; it adopts a dynamic interactive calibration mechanism to dynamically adjust the weight of each modal data according to environmental changes and real-time user feedback, thereby improving the robustness of monitoring results and avoiding the problem of unstable monitoring results caused by the lack of dynamic adjustment strategies in existing technologies.

[0040] (2) The present invention automatically triggers personalized stress relief mechanisms based on fatigue level, such as lumbar airbag inflation, seat cushion heat therapy, and seat reclining, which effectively relieves the user's fatigue and improves the office experience; through the user feedback mechanism, users are allowed to manually intervene in the system actions, which improves the user's participation and sense of control and enhances the friendliness of human-computer interaction. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the operation of the fatigue monitoring method of the present invention.

[0042] Figure 2 This is a schematic diagram illustrating the fatigue level determination method of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] like Figure 1As shown, a fatigue monitoring method based on multimodal data fusion includes synchronous acquisition of multi-source data, preprocessing of multi-source data, dynamic interactive calibration, cross-modal feature fusion, and real-time dynamic updating, which are detailed below:

[0045] (I) Step 1: Synchronously collect multi-source data, including eye-tracking data, seat pressure data and environmental data.

[0046] Eye-tracking data involves collecting images of the user's eyes using eye-tracking cameras. This data is then processed to calculate eye movement parameters such as blink frequency and pupil diameter. Seat pressure data utilizes pressure sensors and gyroscopes integrated into office chairs to capture real-time pressure changes on the seat surface and seat movements (such as tilting and rotation). By fusing static pressure (seat surface pressure) and dynamic posture changes (seat movements), the system provides a more comprehensive monitoring of the user's posture adjustments. Environmental data uses light and noise sensors to monitor light intensity and noise levels. This is primarily used to identify the impact of external disturbances on fatigue and to assist in the accurate determination of fatigue levels. For example, in a high-noise environment, if a user frequently adjusts their posture (attempting to alleviate irritability), noise data can help determine "environmental disturbance-related fatigue."

[0047] (ii) Step 2: Preprocess the multi-source data, including IPD dynamic calibration of eye-tracking data, COP offset calculation of seat pressure data, and illumination compensation of environmental data.

[0048] (1) Dynamic calibration of IPD (interpupillary distance) using eye-tracking data

[0049] Dynamic calibration using IPD (Interpupillary Distance) adapts to user head movements, improving eye-tracking accuracy. Specifically, this includes:

[0050] An infrared camera is used to capture the pupil position, displaying three calibration points: center, upper left, and lower right. The center point is used for basic interpupillary distance calculation, while the upper left and lower right points are used for user coordinate alignment and head posture compensation. The formula is as follows:

[0051]

[0052] Among them, IPD base Represents the basic interpupillary distance, (x left ,y left ) and (x right ,y right The coordinates of the left and right pupil centers are shown below.

[0053] Subsequently, the positions of the nose tip and corner of the eyes are obtained in real time through facial key point detection, and the head offset (Δx,Δy) is calculated.

[0054] Finally, IPD is calculated using the supplementary IPD formula. current :

[0055]

[0056] Among them, IPD current This represents the current interpupillary distance after dynamic adjustment. W and H are the screen width and height, and α = 0.1 and β = 0.05 are empirical coefficients. IPD is updated every 60ms to compensate for head micro-movements.

[0057] In one specific embodiment, if the user is wearing glasses and their head moves 5cm to the right, the system detects Δx = +5cm. The error is reduced from ±2mm in the traditional method to ±0.5mm.

[0058] (2) Calculate the COP (center of pressure) offset from the seat pressure data.

[0059] COP, or center of pressure, reflects the distribution of human weight on the support surface in data acquisition scenarios involving human posture balance and seat pressure distribution. As human movements and postures constantly change, the center of pressure shifts accordingly. COP offset calculation involves analyzing data collected by pressure sensors to determine the position of the center of pressure and how it changes over time.

[0060] The specific operation is as follows: A pressure sensor array installed on the seat collects the pressure value Pi (i = 1, 2, ..., n, where n is the number of sensors) from each sensor in real time, and simultaneously records the fixed coordinates (x, y) of each sensor. i ,y i According to the formula The collected pressure values ​​are normalized to eliminate differences in the measurement ranges of different sensors. max P represents the maximum pressure value. min The minimum pressure value is represented; finally, the coordinates of the pressure center are calculated using the weighted average method. By comparing the COP coordinates at different times, the offset and direction of the COP can be obtained, which can be used to analyze changes in human posture and the impact of fatigue on postural stability.

[0061] (3) Environmental data for illumination compensation

[0062] Illumination compensation is used to suppress ambient light interference and ensure the stability of eye-tracking data under strong / weak light conditions. Ambient light intensity (lux) is collected by a photosensor, and screen brightness (nit) is monitored by a screen brightness sensor (I2C interface). The formula is: Where w represents the compensation weight, with a value range of [0,1], I is the real-time collected light intensity, I0 is the reference light intensity, and k is the system adjustment. When k > 0, the larger k is, the more drastic the weight changes with light intensity. When the light intensity I deviates from the reference value I0, the weight w is dynamically adjusted. For example, under strong light (I > I0), w approaches 1, increasing the compensation strength; under weak light (I < I0), w approaches 0, reducing overcompensation. The light intensity is divided into multiple intervals, and each interval is assigned a fixed weight.

[0063]

[0064] (III) Step 3: Dynamic interactive calibration, including mutual inspection of eye movement data and seat pressure data, environmental data detection and output of dynamic weight vector.

[0065] Cross-validation of eye-tracking and seat pressure data improves the robustness of fatigue assessment and avoids false alarms caused by the failure of a single sensor. First, body movement frequency is detected and calculated. Angular velocity values ​​are collected at different times using a gyroscope, and the peak angular velocity is determined by finding the peak value in the data; the time threshold is based on the time range within which the peak angular velocity is maximized. If f motion If the frequency is greater than 5 times per minute, it is considered high-frequency body movement. At this point, the weight of eye movement data is reduced. The eye movement weight w eye The value was reduced from 0.6 to 0.3. If eye movement data showed fatigue (e.g., pupil constriction of 15%), but the seat pressure distribution remained stable (COP shift <5%), the ambient light intensity (Lenv) was checked. If Lenv > 800 lux, it was determined that strong light was interfering with the eye movement data, and the seat data was trusted. The posture weight was adjusted to w. seat The value increases from 0.3 to 0.6; if Lenv < 300 lux, it is determined that the pupil measurement error is caused by low light, and infrared supplementary lighting enhancement is triggered.

[0066] In addition, when the collected noise is >80dB, the reliability of the seat pressure data is improved (the weight increases from 0.4 to 0.6), because noise may distract the user's attention and cause distortion of eye movement data.

[0067] (iv) Step 4: Cross-modal feature fusion for fatigue level determination.

[0068] like Figure 2 As shown, based on the real-time calibration results of eye-tracking data, seat pressure data, and environmental data, a fusion weight vector for each modality is generated for fatigue level determination. Weight vector formula: W =

[0069] [w eye ,w seat ,w env ], w eyew seat w env These are eye movement weights, posture weights, and environmental weights; the Fatigue Composite Index (FCI) is calculated using the formula FCI = w eye ·F eye +w seat ·F seat +w env ·F env The final fatigue level is determined based on the Fatigue Comprehensive Index (FCI) threshold. FCI < 0.4 indicates mild fatigue, triggering lumbar airbag inflation and screen dimming; FCI 0.4 ≤ FCI < 0.7 indicates moderate fatigue, triggering a sedentary reminder and seat cushion heating, which are triggered sequentially if the user leaves the seat (i.e., after triggering moderate fatigue, if the user leaves the seat, the sedentary reminder and seat cushion heating will be triggered sequentially; if the user does not leave the seat, both functions will be triggered simultaneously); FCI ≥ 0.7 indicates severe fatigue, triggering seat reclining and back massage.

[0070] In one specific embodiment, if W = [0.6, 0.3, 0.1], F eye =0.7, F seat =0.5, F env =0.3, then FCI = 0.6×0.7 + 0.3×0.5 + 0.1×0.3 = 0.6, which is consistent with moderate fatigue. Continue monitoring for 15 minutes, then reassess FCI until fatigue is eliminated.

[0071] (V) Step 5: Real-time dynamic update, and back-transmit the parameters (i.e., some adjustable values ​​or variables involved in the fatigue monitoring method) to cross-modal feature fusion, weight correction dynamic interactive calibration, and if data abnormality occurs, mark the abnormal data to data preprocessing.

[0072] During the fatigue relief process, users can directly intervene manually in the system, such as pausing the massage or manually adjusting the seat angle; this is recorded as negative feedback. Simultaneously, thresholds are updated based on historical customer behavior data; for example, if a user frequently fails to rest despite mild fatigue alerts, the mild fatigue threshold will be lowered. Data is dynamically updated in real time, and parameters are backpropagated to cross-modal feature fusion, weight correction, and dynamic interactive calibration. Any data anomalies are flagged for inclusion in data preprocessing.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fatigue monitoring method based on multi-modal data fusion, characterized in that, The specific steps are as follows: Step 1, synchronously collecting multi-source data, including eye movement data, seat pressure data and environmental data; Step 2, preprocessing multi-source data, including IPD dynamic calibration of eye movement data, COP offset calculation of seat pressure data, and light compensation of environmental data; Step 3, dynamic interactive calibration, including eye movement data and seat pressure data mutual inspection, environmental data detection and output dynamic weight vector; the specific operation of dynamic interactive calibration is: body motion frequency detection, calculating body motion frequency If f motion > 5 times per minute, it is determined that high frequency body motion, at this time the eye movement data weight is reduced, and the eye movement weight w eye From 0.6 to 0.3; if the eye movement data shows fatigue, but the seat pressure distribution is stable, at this time the environmental light intensity Lenv is checked, if Lenv>800lux, it is determined that strong light interferes with eye movement data, and the seat data is trusted, and the sitting posture weight w seat From 0.3 to 0.6; if Lenv<300lux, it is determined that low light causes pupil measurement error, and infrared light supplement enhancement is triggered; when the noise collected is >80dB, the reliability of the seat pressure data is improved, and the weight is increased from 0.4 to 0.6; Step 4, cross-modal feature fusion for fatigue level determination; Step 5, real-time dynamic updating, and parameter back propagation to cross-modal feature fusion for weight correction dynamic interactive calibration, and if data anomaly occurs, the abnormal data is labeled to data preprocessing.

2. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 1, the eye movement data is the user's eye image data collected by the eye tracking camera, and the eye image data is processed to calculate the user's eye movement parameters; The seat pressure data is obtained by using the pressure sensor and gyroscope integrated in the office chair to capture the pressure change on the surface of the seat and the action of the seat in real time, and the static pressure and dynamic posture change data are fused to monitor the user's sitting posture adjustment; the environmental data is obtained by using the light sensor and noise sensor to monitor the light intensity and noise level, to identify the influence of external interference on fatigue, and assist in accurate determination of fatigue level.

3. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 2, the specific operation of IPD dynamic calibration of eye movement data is as follows: First, capture the pupil position by infrared camera, display 3 calibration points, center, upper left and lower right, the formula is as follows: where IPD base represents the interpupillary distance, (x left ,y left ) and (x right ,y right ) are the left and right pupil center coordinates, respectively. Then, real-time nose tip and eye corner position is obtained by face key point detection, and head offset (Δx, Δy) is calculated; Finally, IPD is calculated by IPD supplementary formula current : where IPD current represents the current interpupillary distance after dynamic adjustment, W and H are the screen width and height, and a = 0.1 and β = 0.05 are empirical coefficients; IPD is updated every 60 ms to compensate for head micro-motion.

4. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 2, the position of the pressure center and its change over time are calculated by analyzing the data collected by the pressure sensor, and the specific operation of COP offset calculation is as follows: Firstly, the pressure sensor array installed on the seat collects the pressure value P of each sensor in real time i , i = 1, 2, …, n, n is the number of sensors, and the fixed coordinates (x i , y i ) of each sensor are recorded simultaneously; Then, according to the formula The collected pressure values are normalized to eliminate the differences in the ranges of different sensors, P max represents the maximum pressure value, P min represents the minimum pressure value; Finally, the coordinates of the pressure center are calculated by using the weighted average method The COP coordinates at different times are compared to obtain the offset amount and offset direction of COP, which are used to analyze the changes of human posture and the influence of fatigue on the stability of posture.

5. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 2, the illumination compensation of the environmental data is performed by collecting the environmental light intensity through a photosensitive sensor, monitoring the display screen brightness through a screen brightness sensor, and using the formula: wherein w represents the compensation weight, the value range is [0, 1], I is the real-time collected light intensity, I0 is the reference light intensity, k is a system adjustment parameter, when k > 0, the greater k is, the more intense the weight changes with the light; when the light intensity I deviates from the reference value I0, the weight w is dynamically adjusted.

6. The fatigue monitoring method based on multi-modal data fusion according to claim 5, characterized in that, Under strong light, I>I0, w tends to 1, and the compensation strength is enhanced; under weak light, I Divide the light intensity into multiple intervals, and assign a fixed weight to each interval:

7. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 4, based on the real-time calibration results of eye movement data, seat pressure data and environmental data, a fusion weight vector of each modal data is generated for fatigue level determination; The weight vector W is calculated by the formula: W=[w eye ,w seat ,w env ],w eye 、w seat 、w env are respectively eye movement weight, sitting posture weight, and environment weight. A fatigue comprehensive index FCI is calculated, and its formula is: FCI = w eye ·F eye +w seat ·F seat +w env ·F env , According to the fatigue comprehensive index FCI threshold, the final fatigue level is divided, wherein FCI<0.4 is mild fatigue, triggering the inflation of the waist air bag and the dimming of the screen brightness; 0.4≤FCI<0.7 is moderate fatigue, triggering the sitting reminder and the seat cushion heating, and if getting up, the sitting reminder and the seat cushion heating will be triggered in sequence; FCI≥0.7 is severe fatigue, triggering the seat reclining and back massage.

8. The fatigue monitoring method based on multi-modal data fusion according to claim 1, characterized in that, In step 5, during the fatigue relief process triggered by user feedback, the user is allowed to directly intervene the system action, which is recorded as negative feedback; at the same time, the threshold is updated based on the user's historical behavior data, the data is dynamically updated in real time, and the parameters are back propagated to the cross-modal feature fusion for weight correction dynamic interactive calibration, and if data anomaly occurs, it is labeled to data preprocessing.

Images