Fatigue monitoring method based on multi-modal data fusion

Abstract

The invention discloses a fatigue monitoring method based on multi-modal data fusion, and belongs to the technical field of fatigue monitoring, and the method specifically comprises the steps: synchronously collecting multi-source data, the multi-source data comprising eye movement data, seat pressure data and environment data; preprocessing the multi-source data, including performing IPD dynamic calibration on eye movement data, performing COP offset calculation on seat pressure data, and performing illumination compensation on environment data; dynamic interactive calibration, including eye movement data and seat pressure data mutual detection, environment data detection and dynamic weight vector output; carrying out cross-modal feature fusion, and carrying out fatigue grade judgment; and performing real-time dynamic updating, back-transmitting parameters to cross-modal feature fusion and weight correction dynamic interactive calibration, and if data exception occurs, labeling the exceptional data to data preprocessing. According to the invention, by synchronously collecting the eye movement data, the seat pressure data, the environment data and other multi-source data, comprehensive monitoring of the fatigue state of the user is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of fatigue detection, and particularly relates to a fatigue monitoring method based on multi-modal data fusion. Background Art

[0002] With the extension of office hours in modern society, the problem of sedentary behavior has become increasingly prominent. Prolonged sitting not only causes physical fatigue but may also lead to a series of health problems. Although existing office chairs have certain ergonomic designs, they lack the function of actively alleviating sedentary fatigue. Most office workers need to manually adjust the seat posture or activate the massage function when they feel physically fatigued, which is cumbersome and has limited effects.

[0003] Although there are some seat adjustment methods based on technologies such as human sitting posture recognition and eye movement tracking in the prior art, they mainly target general cases of poor sitting postures and lack the function of relieving pressure for sedentary people. Prolonged sitting can cause symptoms such as eye fatigue and inattention in office workers, affecting work efficiency, and the prior art cannot effectively relieve the physical fatigue of sedentary people.

[0004] Regarding the problems of improving office efficiency and health management, there have already been some invention patents. For example: Patent No. CN118749800A discloses a sedentary pressure relief method, system, and intelligent seat. The method includes: in response to the activation of the pressure relief mode, detecting the pressure signal on the seat surface; after the duration when the pressure signal is greater than or equal to the preset pressure threshold accumulates to the first preset time, driving the massage head to massage the human body, and at the same time detecting the fatigue degree of the human body, denoted as the first fatigue degree; when the duration of driving the massage head to massage the human body reaches the second preset time, detecting the fatigue degree of the human body, denoted as the second fatigue degree; if the ratio of the second fatigue degree to the first fatigue degree is less than or equal to the preset ratio, stop the massage head from massaging the human body, otherwise, execute the above steps until the ratio of the second fatigue degree to the first fatigue degree is less than or equal to the preset ratio. However, this patent still has some problems, such as the need to further optimize the detection algorithm of the millimeter-wave radar to improve the accuracy and stability of fatigue degree detection. In addition, another Patent No. CN118340526A discloses a driving safety monitoring method based on virtual reality. The eye movement data of the driver is collected through the eye movement tracking device in the head-mounted display device. By using the eye movement data and combining the driving scene model, the gaze point position information of the driver is calculated in the virtual reality environment. Through the infrared light sensor or thermal imaging camera in the head-mounted display device, the temperature change and blood flow velocity of the driver's head are monitored in real time; the first fatigue degree of the driver is calculated through the gaze point position information, the head temperature information, and the blood flow velocity. In terms of the eye movement device, this patent needs to further improve its accuracy and stability to ensure that the eye movement data of the driver can be accurately collected. Summary of the Invention

[0005] To overcome the deficiencies of the prior art, the object of the present invention is to provide a fatigue monitoring method based on multi-modal data fusion, which can efficiently obtain and accurately analyze user fatigue information, and trigger a stress relief mechanism based on the fatigue level determination result, so as to solve the limitations of the prior art in the accurate monitoring and evaluation of the fatigue state of users in the office scenario.

[0006] The problems solved by the present invention can be realized by the following specific technical solutions:

[0007] The described fatigue monitoring method based on multi-modal data fusion specifically includes the following steps:

[0008] Step 1, synchronously collect multi-source data, where the multi-source data includes eye movement data, seat pressure data, and environmental data;

[0009] Step 2, preprocess the multi-source data, including dynamically calibrating the IPD (interpupillary distance) of the eye movement data, calculating the COP (center of pressure) offset of the seat pressure data, and performing light compensation on the environmental data;

[0010] Step 3, perform dynamic interactive calibration, including mutual inspection of eye movement data and seat pressure data, environmental data detection, and output of a dynamic weight vector;

[0011] Step 4, perform cross-modal feature fusion for fatigue level determination;

[0012] Step 5, perform real-time dynamic update, and backpropagate the parameters (i.e., some adjustable numerical values or variables involved in the fatigue monitoring method) to the cross-modal feature fusion, and correct the weights for dynamic interactive calibration. If data anomalies occur, mark the abnormal data for data preprocessing.

[0013] Further, in the above step 1,

[0014] The eye movement data is the user's eye image data collected by an eye tracking camera. Process the eye image data to calculate the user's eye movement parameters;

[0015] The seat pressure data is obtained by using pressure sensors and gyroscopes integrated in the office chair to capture the pressure changes on the seat surface and the movements of the seat in real time, and monitor the user's sitting posture adjustment through the fusion of static pressure and dynamic attitude change data;

[0016] The environmental data is obtained by using a photosensitive sensor and a noise sensor to monitor the light intensity and noise level, so as to identify the impact of external interference on fatigue and assist in the accurate determination of the fatigue level.

[0017] Further, in the above step 2, the specific operation of dynamically calibrating the IPD (interpupillary distance) of the eye movement data is:

[0018] First, capture the pupil position through an infrared camera and display three calibration points, namely the center, upper left, and lower right. The formula is as follows:

[0019]

[0020] Among them, IPD base represents the basic interpupillary distance, (x left , y left ) and (x right , y right ) are the coordinates of the centers of the left and right pupils respectively;

[0021] Subsequently, obtain the positions of the tip of the nose and the corners of the eyes in real time through facial key point detection, and calculate the head offset (Δx, Δy);

[0022] Finally, perform the IPD supplementary formula to calculate IPD current :

[0023]

[0024] Among them, IPD current represents the current interpupillary distance after dynamic adjustment, W and H are the width and height of the screen, α = 0.1, β = 0.05 are empirical coefficients; update IPD every 60 ms to compensate for the slight movement of the head.

[0025] Furthermore, in step 2, calculate the position of the center of pressure and its change over time by analyzing the data collected by the pressure sensor. The specific operation of COP (center of pressure) offset calculation is as follows:

[0026] First, the pressure sensor array installed on the seat collects the pressure value Pi of each sensor in real time, i = 1, 2,..., n, where n is the number of sensors, and at the same time records the fixed coordinates (x i , y i ) of each sensor;

[0027] Then, according to the formula normalize the collected pressure values to eliminate the range differences of different sensors. P max represents the maximum pressure value, and P min represents the minimum pressure value;

[0028] Finally, use the weighted average method to calculate the center of pressure coordinates Compare the COP coordinates at different times to obtain the offset and offset direction of COP, which are used to analyze the changes in human body postures and the impact of fatigue on postural stability.

[0029] Further, in step 2, the illumination compensation for environmental data is achieved by collecting the ambient light intensity through a photosensitive sensor and monitoring the display screen brightness through a screen brightness sensor. The compensation weight adopts the formula: where w represents the compensation weight, with a value range of [0, 1], I is the real-time collected illumination intensity, I0 is the reference illumination intensity, and k is the adjustment system. When k > 0, the larger k is, the more drastic the change of the weight with illumination; when the illumination intensity I deviates from the reference value I0, the weight w is dynamically adjusted.

[0030] Further, under strong light, i.e., I > I0, w approaches 1 to enhance the compensation intensity; under weak light, i.e., I < I0, w approaches 0 to reduce overcompensation; the illumination intensity is divided into multiple intervals, and each interval is assigned a fixed weight:

[0031]

[0032] Further, in step 3, the specific operation of dynamic interaction calibration is as follows: conduct body movement frequency detection and calculate the body movement frequency If f motion > 5 times / minute, it is determined as high-frequency body movement. At this time, the weight of eye movement data is triggered to decrease, and the eye movement weight w eye decreases from 0.6 to 0.3; if the eye movement data shows fatigue, but the seat pressure distribution is stable, at this time, check the ambient illumination intensity Lenv. If Lenv > 800 lux, it is determined that strong light interferes with the eye movement data, and the seat data is trusted, and the sitting posture weight w seat increases from 0.3 to 0.6; if Lenv < 300 lux, it is determined that low light causes pupil measurement error, and infrared supplementary light enhancement is triggered; when the collected noise > 80 dB, the feasibility of the seat pressure data is improved, and the weight increases from 0.4 to 0.6.

[0033] Further, in step 4, based on the real-time calibration results of eye movement data, seat pressure data, and environmental data, a fusion weight vector of each modal data is generated for fatigue level determination;

[0034] The calculation formula of the weight vector W: W = [w eye , w seat , w env , where w eye , w seat , w env are the eye movement weight, sitting posture weight, and environmental weight respectively;

[0035] Calculate the comprehensive fatigue index FCI, and its formula is: FCI = w eye ·F eye + w seat ·F seat + w env ·F env ,

[0036] The final fatigue level is classified according to the fatigue comprehensive index (FCI) threshold. Among them, when FCI < 0.4, it is mild fatigue, triggering the inflation of the lumbar airbag and dimming the screen brightness; when 0.4 ≤ FCI < 0.7, it is moderate fatigue, triggering a sedentary reminder and hot compress on the seat cushion. If getting up from the seat, the two functions of sedentary reminder and hot compress on the seat cushion will be triggered in sequence; when FCI ≥ 0.7, it is severe fatigue, triggering the seat to recline and back massage.

[0037] Further, in step 5, during the fatigue relief process triggered by user feedback, the user is allowed to directly manually intervene in the system operation, which is recorded as negative feedback. At the same time, the threshold is updated based on the user's historical behavior data, and the data is dynamically updated in real time. The parameters are then transmitted back to cross-modal feature fusion for weight correction and dynamic interaction calibration. If data anomalies occur, they are marked for data preprocessing.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) By synchronously collecting multi-source data such as eye movement data, seat pressure data, and environmental data, the present invention realizes the comprehensive monitoring of the user's fatigue state, overcoming the defect that the prior art only relies on single-modal data, resulting in incomplete and inaccurate monitoring results. By adopting a dynamic interaction calibration mechanism, the weights of each modal data are dynamically adjusted according to environmental changes and real-time user feedback, improving the robustness of the monitoring results and avoiding the problem of unstable monitoring results caused by the lack of a dynamic adjustment strategy in the prior art.

[0040] (2) The present invention automatically triggers a personalized pressure relief mechanism according to the fatigue level, such as inflation of the lumbar airbag, hot compress on the seat cushion, seat recline, etc., effectively relieving the user's fatigue state and improving the office experience. Through the user feedback mechanism, the user is allowed to manually intervene in the system operation, improving the user's participation and sense of control, and enhancing the friendliness of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is the operation flow chart of the fatigue monitoring method of the present invention;

[0042] Figure 2 is the schematic diagram for determining the fatigue level of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] As Figure 1As shown, a fatigue monitoring method based on multi-modal data fusion includes synchronously collecting multi-source data, preprocessing the multi-source data, dynamically interactive calibration, cross-modal feature fusion, and real-time dynamic update, which are described in detail as follows:

[0045] (1) Step 1: Synchronously collect multi-source data, where the multi-source data includes eye movement data, seat pressure data, and environmental data.

[0046] The eye movement data is to collect the user's eye image data through an eye movement tracking camera, process the eye image data, and calculate the user's eye movement parameters, such as blink frequency, pupil diameter, etc. The seat pressure data is to use the pressure sensors and gyroscopes integrated in the office seat to capture the pressure changes on the seat surface and the movements of the seat (such as tilting, rotating, etc.) in real time. Through the data fusion of static pressure (i.e., the pressure on the seat surface) and dynamic posture changes (i.e., the movements of the seat), the user's sitting posture adjustment can be monitored more comprehensively. The environmental data is to use photosensitive sensors and noise sensors to monitor the light intensity and noise level, which is mainly used to identify the impact of external interference on fatigue and assist in the accurate determination of fatigue levels. For example, in a high-noise environment, if the user frequently adjusts their sitting posture (trying to relieve irritability), the noise data can assist in judging "environment interference-induced fatigue".

[0047] (2) Step 2: Preprocess the multi-source data, including performing IPD dynamic calibration on the eye movement data, calculating the COP offset for the seat pressure data, and performing light compensation on the environmental data.

[0048] (1) Perform IPD (interpupillary distance) dynamic calibration on the eye movement data

[0049] Perform IPD (interpupillary distance) dynamic calibration to cope with the user's head movement and improve the accuracy of eye movement tracking. Specifically, it includes:

[0050] Use an infrared camera to capture the pupil positions and display 3 calibration points, namely the center, upper left, and lower right. The center point is used for basic pupil distance calculation, and the upper left point and lower right point are used for user coordinate alignment and head posture compensation. The formula is as follows:

[0051]

[0052] Among them, IPD base represents the basic pupil distance, (x left , y left ) and (x right , y right ) are the center coordinates of the left and right pupils respectively;

[0053] Subsequently, obtain the positions of the tip of the nose and the corners of the eyes in real time through facial key point detection, and calculate the head offset (Δx, Δy);

[0054] Finally, perform IPD supplementary formula calculation for IPD current :

[0055]

[0056] Among them, IPD current represents the current pupil distance after dynamic adjustment, W and H are the width and height of the screen, and α = 0.1, β = 0.05 are empirical coefficients; IPD is updated every 60 ms to compensate for the slight head movement.

[0057] In a specific embodiment, if the user wears glasses and the head moves 5 cm to the right, the system detects Δx = +5 cm, then the error is reduced from ±2 mm of the traditional method to ±0.5 mm.

[0058] (2) Perform COP (center of pressure) offset calculation on the seat pressure data

[0059] COP is the center of pressure. In data acquisition scenarios involving human body posture balance, seat pressure distribution, etc., the position of the center of pressure reflects the distribution of the human body weight on the support surface. Due to the continuous changes in the actions and postures of the human body, the center of pressure will also shift accordingly. The COP offset calculation is to calculate the position of the center of pressure and its change over time by analyzing the data collected by the pressure sensors.

[0060] The specific operation is as follows: The pressure sensor array installed on the seat collects the pressure value Pi of each sensor in real time (i = 1, 2,..., n, n is the number of sensors), and at the same time records the fixed coordinates (x i , y i ) of each sensor; According to the formula normalize the collected pressure values to eliminate the range differences of different sensors, P max represents the maximum pressure value, and P min represents the minimum pressure value; Finally, use the weighted average method to calculate the center of pressure coordinates Compare the COP coordinates at different times to obtain the offset amount and offset direction of COP, which are used to analyze the changes in human body postures and the impact of fatigue on posture stability, etc.

[0061] (3) Perform light compensation on the environmental data

[0062] Light compensation is to suppress the interference of ambient light and ensure the stability of eye movement data under strong / weak light. The ambient light intensity (lux) is collected by a photosensitive sensor, and the display screen brightness (nit) is monitored by a screen brightness sensor (I2C interface). The formula: Among them, w represents the compensation weight, with a value range of [0,1], I is the light intensity collected in real time, I0 is the reference light intensity, and k is the adjustment system. When k>0, the larger k is, the more drastic the weight changes with the light; when the light intensity I deviates from the reference value I0, the weight w is dynamically adjusted. For example, under strong light (I>I0), w approaches 1, which increases the compensation strength; under weak light (I<I0), w approaches 0, which reduces over-compensation. Divide the light intensity into multiple intervals, and assign a fixed weight to each interval:

[0063]

[0064] (III) Step 3: Dynamic interactive calibration, including mutual checking of eye movement data and seat pressure data, environmental data detection and output of dynamic weight vector.

[0065] Through cross-validation of eye movement data and seat pressure data, the robustness of fatigue judgment is improved to avoid misjudgment caused by failure of a single sensor. First, body movement frequency detection is performed to calculate the body movement frequency. The gyroscope collects angular velocity values ​​at different times, and then determines the angular velocity peak by finding the peak value in the data; the time threshold is based on the time range with the largest angular velocity peak value. motion >5 times / minute, it is considered high-frequency body movement, which triggers the reduction of eye movement data weight. The eye movement weight w eye From 0.6 to 0.3; if the eye movement data shows fatigue (such as pupil constriction of 15%), but the seat pressure distribution is stable (COP deviation <5%), then check the ambient light intensity Lenv. If Lenv>800lux, it is determined that strong light interferes with the eye movement data, trust the seat data, and the sitting posture weight w seat From 0.3 to 0.6; if Lenv < 300lux, it is judged as low light causing pupil measurement error, triggering infrared fill light enhancement.

[0066] In addition, when the collected noise is greater than 80dB, the feasibility of the seat pressure data is improved (the weight is increased from 0.4 to 0.6), because noise may distract the user's attention and cause distortion of the eye movement data.

[0067] (IV) Step 4: Cross-modal feature fusion for fatigue level determination.

[0068] like Figure 2 As shown in FIG. 1 , based on the real-time calibration results of eye movement data, seat pressure data and environmental data, a fusion weight vector of each modal data is generated for fatigue level determination. Weight vector formula: W =

[0069] [w eye ,w seat ,w env ],w eye, w seat , w env are the eye movement weight, sitting posture weight, and environment weight respectively; calculate the comprehensive fatigue index FCI, and its formula is FCI = w eye ·F eye +w seat ·F seat +w env ·F env . According to the threshold of the comprehensive fatigue index FCI, divide the final fatigue level. When FCI < 0.4, it is mild fatigue, trigger the inflation of the lumbar airbag, and dim the screen brightness; when 0.4 ≤ FCI < 0.7, it is moderate fatigue, trigger the sedentary reminder, heat the seat cushion. If the user leaves the seat, the sedentary reminder and seat cushion heating will be triggered successively (that is, after triggering moderate fatigue, if the user leaves the seat, then the sedentary reminder and seat cushion heating will be triggered successively in order; if the user does not leave the seat, the sedentary reminder and seat cushion heating will be triggered simultaneously); when FCI ≥ 0.7, it is severe fatigue, trigger the seat to recline and back massage.

[0070] In a specific embodiment, if W = [0.6, 0.3, 0.1], F eye = 0.7, F seat = 0.5, F env = 0.3, then FCI = 0.6×0.7 + 0.3×0.5 + 0.1×0.3 = 0.6, which meets the moderate fatigue. Continuously monitor for 15 minutes, and judge FCI again until there is no fatigue.

[0071] (5) Step 5: Update in real time and dynamically, and backpropagate the parameters (that is, some adjustable numerical values or variables involved in the fatigue monitoring method) to the cross-modal feature fusion, weight correction dynamic interaction calibration. If data anomalies occur, mark the abnormal data for data preprocessing.

[0072] During the trigger of the fatigue relief process, the user is allowed to directly manually intervene in the system actions, such as pausing the massage and manually adjusting the seat angle, which is recorded as negative feedback; at the same time, update the threshold based on the customer's historical behavior data. For example, if the user often does not rest under the mild fatigue reminder, the threshold of mild fatigue will be reduced. The data is updated dynamically in real time, and the parameters are backpropagated to the cross-modal feature fusion, weight correction dynamic interaction calibration. If data anomalies occur, mark them for data preprocessing.

[0073] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fatigue monitoring method based on multimodal data fusion, characterized in that: The specific steps are as follows: Step 1: synchronously collect multi-source data, which includes eye movement data, seat pressure data and environmental data; Step 2: Preprocess the multi-source data, including IPD dynamic calibration of eye movement data, COP offset calculation of seat pressure data, and illumination compensation of environmental data; Step 3: Dynamic interactive calibration, including mutual inspection of eye movement data and seat pressure data, environmental data detection and output of dynamic weight vector; Step 4: Cross-modal feature fusion for fatigue level determination; Step 5: Real-time dynamic update, and the parameters are fed back to the cross-modal feature fusion, and the weight correction is dynamically interactively calibrated. If data anomalies occur, the abnormal data will be marked as data preprocessing.

2. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 1, the eye movement data is the user's eye image data collected by an eye tracking camera, and the eye image data is processed to calculate the user's eye movement parameters; Seat pressure data uses pressure sensors and gyroscopes integrated in office chairs to capture real-time changes in seat surface pressure and seat movements, and monitor the user's sitting posture adjustment through the fusion of static pressure and dynamic posture change data. Environmental data uses photosensitive sensors and noise sensors to monitor light intensity and noise levels, which is used to identify the impact of external interference on fatigue and assist in the accurate determination of fatigue levels.

3. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, the specific operation of performing IPD dynamic calibration on the eye movement data is: First, the pupil position is captured by an infrared camera, and three calibration points are displayed, namely the center, upper left, and lower right. The formula is as follows: Among them, IPD base represents the basic pupil distance, (x left ,y left ) and (x right ,y right ) are the coordinates of the left and right pupil centers respectively; Then, the positions of the nose tip and eye corners are obtained in real time through facial key point detection, and the head offset (Δx, Δy) is calculated; Finally, the IPD supplementary formula is used to calculate the IPD current : Among them, IPD current Represents the current pupil distance after dynamic adjustment, W and H are the screen width and height, α=0.1 and β=0.05 are empirical coefficients; IPD is updated every 60ms to compensate for slight head movements.

4. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, the position of the pressure center and its change over time are calculated by analyzing the data collected by the pressure sensor. The specific operation of COP offset calculation is: First, the pressure sensor array installed on the seat collects the pressure value P of each sensor in real time. i , i = 1, 2, ..., n, n is the number of sensors, and the fixed coordinates (x i ,y i ); Then, according to the formula Normalize the collected pressure values ​​to eliminate the range differences of different sensors, P max Indicates the maximum pressure value, P min Indicates the minimum pressure value; Finally, the pressure center coordinates are calculated using the weighted average method By comparing the COP coordinates at different times, the offset and direction of the COP are obtained, which are used to analyze the changes in human posture and the impact of fatigue on posture stability.

5. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, the illumination compensation of the environmental data is performed by collecting the ambient light intensity through the photosensitive sensor, and the screen brightness sensor monitors the brightness of the display screen. The compensation weight adopts the formula: Wherein, w represents the compensation weight, and its value range is [0,1]. I is the light intensity collected in real time, I0 is the reference light intensity, and k is the adjustment system. When k>0, the larger the k, the more drastic the weight changes with the light. When the light intensity I deviates from the reference value I0, the weight w is adjusted dynamically.

6. The fatigue monitoring method based on multimodal data fusion according to claim 5 is characterized in that: In strong light, that is, I>I0, w approaches 1, which increases the compensation strength; in weak light, that is, I<I0, w approaches 0, which reduces overcompensation; Divide the light intensity into multiple intervals, and assign a fixed weight to each interval:

7. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 3, the specific operation of dynamic interactive calibration is: performing body motion frequency detection and calculating body motion frequency. If f motion >5 times / minute, it is considered high-frequency body movement, which triggers the reduction of eye movement data weight. The eye movement weight w eye From 0.6 to 0.3; if the eye movement data shows fatigue, but the seat pressure distribution is stable, check the ambient light intensity Lenv. If Lenv>800lux, it is determined that strong light interferes with the eye movement data, trust the seat data, and the sitting posture weight w seat From 0.3 to 0.6; if Lenv < 300lux, it is judged that low light causes pupil measurement errors, triggering infrared fill light enhancement; when the collected noise is > 80dB, the feasibility of seat pressure data is improved, and the weight is increased from 0.4 to 0.

6.

8. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In the step 4, based on the real-time calibration results of the eye movement data, the seat pressure data and the environmental data, a fusion weight vector of each modal data is generated for fatigue level determination; The weight vector W is calculated by: W = [w eye ,w seat ,w env ],w eye 、w seat 、w env They are eye movement weight, sitting weight, and environment weight; Calculate the fatigue comprehensive index FCI, the formula is: FCI = w eye ·F eye +w seat ·F seat +w env ·F env , The final fatigue level is divided according to the fatigue comprehensive index FCI threshold, where FCI < 0.4 is mild fatigue, triggering the inflation of the waist airbag and dimming the screen brightness; 0.4≤FCI<0.7 indicates moderate fatigue, which triggers a sedentary reminder and seat heating. If you leave your seat, the sedentary reminder and seat heating will be triggered in that order. FCI≥0.7 indicates severe fatigue, which triggers seat reclining and back massage.

9. The fatigue monitoring method based on multimodal data fusion according to claim 1 is characterized in that: In step 5, user feedback triggers the fatigue relief process, allowing the user to directly manually intervene in the system action and record it as negative feedback; at the same time, based on the user's historical behavior data, the threshold is updated, the data is dynamically updated in real time, and the parameters are fed back to the cross-modal feature fusion, the weight correction dynamic interaction calibration, and if any data anomalies occur, they are marked as data preprocessing.

Images