A method for detecting fatigue of on-duty crew on the ship's bridge and periodic fatigue assessment

Abstract

The present invention discloses a fatigue detection and periodic fatigue assessment method for the watchkeeping crew on the ship's bridge, which relates to the field of ship safety technology. The facial information and human body posture information of the crew are collected through an on-board camera, and data preprocessing is carried out, including denoising, light compensation and video frame extraction. In the feature extraction stage, facial key point features (such as blinking frequency, eye closure ratio, mouth opening degree) and human body posture features (such as head movement, center of gravity offset, leg movement) are extracted respectively. After feature standardization and splicing, a random forest model is used to calculate the short-term fatigue index. Combining the periodic work and rest data, the cumulative fatigue index is improved through a non-linear cumulative task load model, a Fourier series periodic load modeling and a dynamic fatigue recovery model, so as to realize the dynamic assessment and early warning of the long-term fatigue of the crew. The present invention can accurately monitor the fatigue state of the crew in a complex marine environment, and ensure the safety of ship operation and the health of the crew.

Description

Technical Field

[0001] The present invention relates to the technical field of ship safety, and more particularly, to a method for detecting fatigue of duty officers on the ship's bridge and periodic fatigue assessment. Background Art

[0002] During long-term maritime work, crew members are prone to fatigue due to the continuous high-intensity working environment. Once this fatigue accumulates, it will not only affect the accuracy and judgment of their tasks, but may also directly threaten the safety of the ship and crew. According to statistics, fatigue driving of duty officers is the most important factor leading to maritime traffic accidents. Therefore, accurately detecting the fatigue state of crew members, especially in an environment where high-intensity work alternates with rest, and analyzing their periodic fatigue accumulation and recovery status are of great significance for ensuring the safe operation of ships and the health of crew members.

[0003] At present, the methods for detecting crew fatigue mostly focus on facial feature detection, physiological signal monitoring and behavior recognition. However, there are some deficiencies in existing detection methods. For example, the acquisition of physiological signals usually requires wearing additional sensing devices, which will affect the operation comfort of crew members; a single facial or physiological signal cannot comprehensively reflect the fatigue state of crew members; existing fatigue detection methods only detect the fatigue of crew members in the short term, without considering the analysis of their periodic fatigue accumulation and recovery status in an environment where high-intensity work alternates with rest. Summary of the Invention

[0004] In view of the above technical problems, a method for detecting fatigue of duty officers on the ship's bridge and periodic fatigue assessment is provided. The present invention realizes precise monitoring of the fatigue state of crew members through video data acquisition and processing, facial feature extraction, human body posture feature extraction, feature fusion, fatigue index detection and long-term fatigue assessment.

[0005] The technical means adopted by the present invention are as follows:

[0006] A method for detecting fatigue of duty officers on the ship's bridge and periodic fatigue assessment includes the following steps:

[0007] S1. Video data acquisition and processing: Collect the facial information and human body posture information of crew members through an on-board camera, and perform denoising processing, light compensation, video frame extraction and region of interest (ROI) selection;

[0008] S2. Facial feature extraction: Extract the facial feature points of crew members from the video frames, calculate the blink frequency, eye closure ratio and mouth opening degree, and at the same time input the facial image region into a pre-trained convolutional neural network model (CNN model) to extract deep features;

[0009] S3. Extract human body posture features. Select a time window of a fixed length, and extract head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position, and leg tilt angle;

[0010] S4. Feature fusion. Perform standardization processing on each type of feature, splice the facial features and human body posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing;

[0011] S5. Detect short-term fatigue index. Construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after fusing features;

[0012] S6. Long-term fatigue detection. Capture periodic work and rest data, smooth the short-term fatigue index, establish a relationship model between complex task load and short-term fatigue, introduce a non-linear and time-dependent cumulative fatigue index, model the periodic task load, introduce an interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and predict the fatigue trend and give an alarm according to the change trend of the cumulative fatigue index.

[0013] Further, step S1 specifically includes:

[0014] S11. Use a filter to remove noise in the image;

[0015] S12. Use adaptive histogram equalization to enhance areas with uneven illumination;

[0016] S13. Extract frames from the video at a fixed frame rate as the analysis input, and select the region of interest (ROI) according to the specific regions of the face and actions to reduce the calculation amount. In this embodiment, the ROI can frame the regions of the face and the lower body with a rectangle or other shapes.

[0017] Further, step S2 specifically includes:

[0018] S21. Extract the facial feature points of the crew from the video frames and detect the facial key points;

[0019] S22. Calculate the number of blinks per minute by detecting the change in the distance between the upper and lower eyelids, and extract the facial movement features;

[0020] S23. Calculate the eye closure ratio through specific key points and extract the facial movement features;

[0021] S24. Detect the opening degree of the mouth through the key points on both sides of the upper and lower lips and extract the facial movement features;

[0022] S25. Input the facial image region into a pre-trained convolutional neural network model (CNN model) to extract deep features.

[0023] Further, step S3 specifically includes:

[0024] S31. Define a time window;

[0025] S32. Within the defined time window, calculate the coordinate change of the top of the head to obtain the head displacement feature;

[0026] S33. Within the defined time window, calculate the angle between the top of the head and the shoulders to obtain the head tilt angle;

[0027] S34. Within the defined time window, determine the head rotation by the relative positions between the ears and calculate the head rotation angle;

[0028] S35. Within the defined time window, calculate the displacement of the knees to obtain the leg position change feature;

[0029] S36. Within the defined time window, calculate the center of gravity position through the weighted average of multiple key points:

[0030] S37. Within the defined time window, calculate the leg tilt angle.

[0031] Further, step S4 specifically includes:

[0032] S41. Standardize each type of feature;

[0033] S42. Concatenate the facial features and human pose features to fuse them into a high-dimensional feature vector, and perform dimensionality reduction on the high-dimensional feature vector.

[0034] Further, step S5 specifically includes:

[0035] Step S51. Construct a training data set consisting of the fused features and the corresponding short-term fatigue index;

[0036] Step S52. The random forest model trains the data by constructing multiple decision trees;

[0037] Step S53. After fusing the facial and human pose features, the possibility of the individual's fatigue state is represented by the score obtained from the random forest model, that is, the instantaneous fatigue state score is the predicted value output by the random forest model.

[0038] Further, step S6 specifically includes:

[0039] S61. Capture periodic work data through the work schedule and task log;

[0040] S62. Smooth the short-term fatigue index for short-term fatigue detection using exponentially weighted moving average;

[0041] S63. Establish a relationship model between complex task load and short-term fatigue using a multi-dimensional non-linear function;

[0042] S64. Improve the cumulative fatigue index by introducing non-linear and time-dependent patterns;

[0043] S65. Use Fourier series expansion to model the periodic components of task load, introduce the periodic interaction term between task load and fatigue index, combine task intensity with periodic fluctuations, and enhance the sensitivity to fatigue under periodic fluctuations;

[0044] S66. To simulate the fatigue recovery effect, introduce a dynamic recovery equation based on negative exponential decay, and update the cumulative fatigue index after adding the recovery term;

[0045] S67. Detect the fatigue accumulation risk according to the change trend of the cumulative fatigue index, and add acceleration and curvature terms to capture the non-linear change trend of fatigue accumulation.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. A method for fatigue detection and periodic fatigue assessment of the duty crew on the ship's bridge provided by the present invention does not require additional physiological signal acquisition equipment, reducing the interference to the crew's operation.

[0048] 2. A method for fatigue detection and periodic fatigue assessment of the duty crew on the ship's bridge provided by the present invention combines facial features and body posture features, and can comprehensively reflect the fatigue state of the crew.

[0049] 3. A method for fatigue detection and periodic fatigue assessment of the duty crew on the ship's bridge provided by the present invention takes into account the periodic fatigue accumulation and recovery process of the crew's high-intensity work and rest alternation, improving the accuracy and reliability of fatigue detection.

[0050] 4. A method for fatigue detection and periodic fatigue assessment of the duty crew on the ship's bridge provided by the present invention can real-time evaluate the fatigue state of the crew and give early warnings through a dynamic fatigue recovery model and periodic task load modeling.

[0051] Based on the above reasons, the present invention can be widely promoted in the fields such as ship safety. Brief Description of the Drawings

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0053] Figure 1 This is the flowchart of the method of the present invention. Detailed implementation manners

[0054] To enable those skilled in the art of the present technology to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0055] It should be noted that the terms "including" and "having" in the specification and claims of the present invention and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0056] As Figure 1 shown, the present invention provides a method for fatigue detection and periodic fatigue assessment of the duty crew on the ship's bridge, including the following steps:

[0057] S1. Video data collection and processing: Collect the facial information and human body posture information of the crew through the on-board camera, and perform denoising processing, light compensation, video frame extraction, and ROI selection;

[0058] S2. Facial feature extraction: Extract the facial feature points of the crew from the video frames, calculate the blink frequency, eye closure ratio, and mouth opening degree, and at the same time input the facial image area into the pre-trained CNN model to extract deep features;

[0059] S3. Human body posture feature extraction: Select a time window with a fixed length, and extract the head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position, and leg tilt angle;

[0060] S4. Feature fusion: Standardize each type of feature, splice the facial features and human body posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing.

[0061] S5. Short-term fatigue index detection: Construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after fusing features.

[0062] S6. Long-term fatigue detection: Capture periodic work and rest data, smooth the short-term fatigue index, establish a relationship model between complex task load and short-term fatigue, introduce a non-linear and time-dependent cumulative fatigue index, model the periodic task load, introduce the interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and predict the fatigue trend and give an alarm according to the change trend of the cumulative fatigue index.

[0063] In specific implementation, as a preferred implementation manner of the present invention, step S1 specifically includes:

[0064] S11. Use a filter to remove noise in the image and reduce interference in subsequent analysis. The filtering formula is as follows:

[0065]

[0066] Among them, represents the pixel value, represents the filtering weight, represents the radius of the filter, which determines the size of the filter, and respectively represent the offsets of the filter in the vertical and horizontal directions.

[0067] S12. Perform light compensation, and use adaptive histogram equalization to enhance the unevenly illuminated areas and maintain the clarity of the video. The light compensation formula is as follows:

[0068]

[0069] Among them, and respectively represent the minimum and maximum gray values within the local window, represents the gray level.

[0070] S13. Extract frames from the video at a fixed frame rate as the analysis input, and select the region of interest (ROI) according to the specific regions of the face and actions to reduce the calculation amount. In this embodiment, the ROI can frame the regions of the face and the lower body with a rectangle or other shapes.

[0071] In specific implementation, as a preferred implementation manner of the present invention, step S2 specifically includes:

[0072] S21. Extract the facial feature points of the crew from the video frame and detect the facial key points. In this embodiment, the facial key points include eyes, nose, mouth, chin, etc. Facial key point coordinate extraction: Assume the coordinates of the facial key points are , where represents different key points (such as eyes, mouth corners, etc.). Then, at frame , the coordinates of the facial key points can be expressed as .

[0073] S22. Calculate the number of blinks per minute by detecting the change in the distance between the upper and lower eyelids and extract the facial action features. In this embodiment, assume the distance between the upper and lower eyelids of the left eye is:

[0074]

[0075] When the distance is below a certain threshold, it can be determined as a blink. Among them, represents the vertical coordinate of the upper eyelid at moment, represents the vertical coordinate of the lower eyelid at moment, represents the vertical distance between the upper and lower eyelids at moment.

[0076] S23. Calculate the eye closure ratio through specific key points and extract the facial action features. The calculation formula for the eye closure ratio is as follows:

[0077]

[0078] Among them, to respectively represent 6 key points of the left and right eyes.

[0079] S24. Detect the opening degree of the mouth through the key points on both the upper and lower sides of the mouth and extract the facial action features. The calculation formula for the opening degree of the mouth is as follows:

[0080]

[0081] Among them, represents the coordinate (ordinate) of the key point on the upper side of the mouth, represents the coordinate (ordinate) of the key point on the lower side of the mouth, represents the coordinate (abscissa) of the key point on the left side of the mouth, represents the coordinate (abscissa) of the key point on the right side of the mouth.

[0082] S25. Input the facial image region into a pre-trained CNN model to extract deep features. The CNN network architecture usually includes convolutional layers, pooling layers, and fully connected layers. Given an input image , the operation of the convolutional layer is as follows:

[0083]

[0084] Among them, represents the activation function, and represent the sizes of the convolutional kernel in the vertical and horizontal directions respectively. represents the convolutional kernel weight, represents the position of the image at the pixel value, represents the bias; the convolutional layer extracts local features. After the operations of multiple convolutional layers and pooling layers, a deep feature vector is finally obtained.

[0085] In specific implementation, as a preferred implementation manner of the present invention, step S3 specifically includes:

[0086] S31. Define a time window; during the detection process, select a time window with a fixed length , and the sampling rate is . Extract various features within the time window, then there are frames of data within the time window.

[0087] S32. Within the defined time window, calculate the coordinate change of the top of the head to obtain the head displacement feature; the calculation formula for the coordinate change of the top of the head is as follows:

[0088]

[0089] Among them, represents the coordinate of the top of the head at time , represents the coordinate of the top of the head at time . represents the start time, that is, the start moment of the time window. represents the length of the time window.

[0090] S33. Within the defined time window, calculate the angle between the top of the head and the shoulders to obtain the head tilt angle; that is, the head tilt angle can be expressed as:

[0091]

[0092] Among them, represents the start time, that is, the start moment of the time window; represents the length of the time window; () represents the inverse cosine function, which is used to calculate the included angle; represents time the coordinates of the left shoulder at represents time the coordinates of the right shoulder at represents time the coordinates of the top of the head at represents time the coordinates of the center point of the shoulders at

[0093] S34. Within the defined time window, judge the head rotation through the relative positions between the ears, and calculate the head rotation angle; that is, the head rotation angle can be calculated by the following formula:

[0094]

[0095] where and respectively represent the coordinates of the left ear and the right ear at the moment.

[0096] S35. Within the defined time window, calculate the displacement of the knees to obtain the characteristics of the leg position change; where:

[0097] The displacement of the left knee can be expressed as:

[0098]

[0099] where represents time the coordinates of the left knee at represents time the coordinates of the left knee at

[0100] The displacement of the right knee can be expressed as:

[0101]

[0102] where represents time the coordinates of the right knee at represents time the coordinates of the right knee at

[0103] S36. Within the defined time window, calculate the center of gravity position through the weighted average of multiple key points; the change of the crew's center of gravity reflects whether their body is in an unstable state. The center of gravity position is calculated through the weighted average of multiple key points, such as the left knee, right knee, left ankle, right ankle, left hip, right hip, etc. The formula for the center of gravity position is:

[0104]

[0105] Among them, represents the weight value of the key point, represents time at the position of the

[0106] S37. Calculate the leg tilt angle within the defined time window. The leg tilt angle is represented by and its calculation formula is as follows:

[0107]

[0108] Among them, represents the coordinates of the hip at time , represents the coordinates of the knee at time , represents the coordinates of the ankle at time .

[0109] In specific implementation, as a preferred implementation manner of the present invention, step S4 specifically includes:

[0110] S41. Perform standardization processing on each type of feature;

[0111] Since the value ranges of facial features and human pose features are different, in order to make the influence of each feature on the model consistent, in this embodiment, standardization processing is performed on each type of feature. Assume represents the value of feature , and the standardization formula is:

[0112]

[0113] Among them, and are respectively the mean and standard deviation of feature .

[0114] S42. Concatenate the facial features and human pose features and fuse them into a high-dimensional feature vector, as follows:

[0115]

[0116] And perform dimensionality reduction processing on the fused high-dimensional feature vector (such as PCA or linear discriminant analysis LDA) to reduce the influence of noise and improve the training efficiency of the model. Among them, represents the facial feature vector extracted from the video frame, represents the human pose feature vector extracted from the video.

[0117] In specific implementation, as a preferred implementation manner of the present invention, step S5 specifically includes:

[0118] Step S51: Construct a training data set consisting of fused features and corresponding short-term fatigue indices;

[0119] In this embodiment, the fused features and the corresponding labeled short-term fatigue indices are used to form a training data set as follows:

[0120]

[0121] where, represents the number of samples.

[0122] Step S52: The random forest model trains the data by constructing multiple decision trees;

[0123] In this embodiment, the prediction function of the random forest model is:

[0124]

[0125] where, represents the number of decision trees, represents the th decision tree.

[0126] Step S53: After fusing facial and human pose features, the possibility of an individual's fatigue state is represented by the score obtained from the random forest model, that is, the fatigue index is the predicted value output by the random forest model.

[0127] In this embodiment, the instantaneous fatigue state score is the score obtained from the model after fusing facial and human pose features, and the calculation formula is as follows:

[0128]

[0129] where, represents the predicted value output by the random forest model.

[0130] Specifically, as a preferred implementation manner of the present invention, step S6 specifically includes:

[0131] S61: Capture periodic work data through the work schedule and task log;

[0132] In this embodiment, the work schedule and task log are used to capture periodic work and rest data, record the task type and time length of each time period, and form a multi-dimensional workload sequence , representing information such as work intensity and task complexity at the moment.

[0133] S62. Smooth the short-term fatigue index of short-term fatigue detection using exponentially weighted moving average;

[0134] The short-term fatigue index of short-term fatigue detection represents the fatigue level at each time point. To eliminate noise and enhance detection stability, exponentially weighted moving average is applied to perform smoothing:

[0135]

[0136] where represents the smoothing coefficient, represents the instantaneous fatigue state score at time represents the previous time of the smoothed fatigue index.

[0137] S63. Establish a relationship model between complex task load and short-term fatigue using a multi-dimensional non-linear function;

[0138] In this embodiment, to improve the cumulative characterization of task load on long-term fatigue, the task load is modeled as a multi-dimensional non-linear function to capture the interaction effects between different loads. Define the non-linear task load at each time as:

[0139]

[0140] where is the dimension of the task load, representing different categories or factors of the task, represents the linear coefficient of the task load, represents at time the corresponding dimension of the task load value, represents the interaction coefficient between loads, controlling the cumulative effect of the synergistic action of different loads on fatigue; represents at time the corresponding dimension of the task load value.

[0141] S64. Introduce non-linear and time-dependent patterns to improve the Cumulative Fatigue Index (CFI);

[0142] In this embodiment, a non-linear and time-dependent cumulative fatigue index is introduced, and the complex accumulation of fatigue is represented using short-term fatigue and non-linear task load. The formula is:

[0143]

[0144] where is the decay factor of the cumulative fatigue index, representing the influence degree of the fatigue index at the previous moment on the current moment. represents the cumulative fatigue index at the previous moment. Converts the influence of task load on fatigue into a slow cumulative mode to avoid large fluctuations in a single task. is a parameter used to adjust the influence degree of task load on the cumulative fatigue index. is a non - linear mapping function of short - term fatigue, which controls the non - linear degree of the influence of short - term fatigue.

[0145] S65. Adopts Fourier series expansion to model the periodic components of task load, introduces the periodic interaction term between task load and fatigue index, combines task intensity with periodic fluctuations, and enhances the sensitivity of fatigue to periodic fluctuations.

[0146] In this embodiment, to capture the periodic fluctuations of task load, Fourier series expansion is used to model the periodic components of task load. Definition:

[0147]

[0148] Among them, represents the basic level of task load. and represent the amplitudes of task load at different frequencies. represents the main period of task load (such as 12 hours or 24 hours). This model can simulate the periodic changes of task load and more accurately predict the fatigue accumulation process in combination with the CFI cumulative fatigue index.

[0149] Introduces the periodic interaction term between task load and fatigue index , combines task intensity with periodic fluctuations, and enhances the sensitivity of fatigue to periodic fluctuations. Among them, the introduced periodic interaction term between task load and fatigue index is as follows:

[0150]

[0151] This interaction term can adjust the periodic effect of task load and make the fatigue prediction more in line with the actual periodic changes. Among them, is the task load, representing the task intensity and complexity at time , and represent the amplitudes of periodic task load fluctuations, that is, the periodic changes of task load over time.

[0152] S66. To simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced, and the cumulative fatigue index is updated after adding the recovery term.

[0153] In this embodiment, to simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced. When i.e., during the rest period, the dynamic recovery model is expressed as:

[0154]

[0155] where represents the recovery coefficient, which controls the natural recovery rate of fatigue; represents the decay coefficient of recovery; represents the cumulative rest time since entering the rest state.

[0156] The updated formula for CFI after adding the recovery term is:

[0157]

[0158] This formula dynamically adjusts the task load, short-term fatigue, and recovery effect in the cumulative fatigue index, making the fatigue accumulation more in line with the dynamic changes on a long time scale.

[0159] S67. Detect the fatigue accumulation risk according to the change trend of the cumulative fatigue index, and add the acceleration and curvature terms to capture the non-linear change trend of fatigue accumulation.

[0160] In this embodiment, the fatigue accumulation risk is detected according to the change trend of CFI, and the acceleration and curvature terms are added to capture the non-linear change trend of fatigue accumulation:

[0161]

[0162] If it indicates that the fatigue index grows too fast and an early warning is given; if then a fatigue alarm is triggered and it is recommended to rest immediately; where and are set thresholds, and the specific values can be adjusted according to the working modes and fatigue performances of different crew members. Generally, represents the growth rate threshold of the fatigue index. When the fatigue index exceeds this value, it indicates that the fatigue state of the crew member has reached a risk level that requires attention. is used to represent the critical value for triggering an alarm when the fatigue index accumulates too fast, usually set to trigger an alarm when the cumulative fatigue index increases to a certain amplitude. According to the experimental results, the reference value of can be given as 0.01 to 0.05. The reference value of

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting fatigue of duty crew on the ship's bridge and periodic fatigue assessment, characterized in that It includes the following steps: S1. Video data acquisition and processing: Collect the facial information and human body posture information of crew members through an on-board camera, and perform denoising processing, light compensation, video frame extraction, and region of interest selection; S2. Facial feature extraction: Extract the facial feature points of crew members from the video frames, calculate the blink frequency, eye closure ratio, and mouth opening degree, and at the same time input the facial image region into a pre-trained convolutional neural network model to extract deep features; S3. Human body posture feature extraction: Select a time window with a fixed length, and extract the head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position, and leg tilt angle; S4. Feature fusion: Perform standardization processing on each type of feature, splice the facial features and human body posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing; S5. Short-term fatigue index detection: Construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after feature fusion; S6. Long-term fatigue detection: Capture periodic work and rest data, smooth the short-term fatigue index, establish a relationship model between complex task load and short-term fatigue, introduce a non-linear and time-dependent cumulative fatigue index, model the periodic task load, introduce an interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and perform fatigue trend prediction and alarm according to the change trend of the cumulative fatigue index. Specifically, it includes: S61. Capture periodic work data through the work schedule and task log; S62. Use the exponentially weighted moving average to smooth the short-term fatigue index detected in short-term fatigue; S63. Use a multi-dimensional non-linear function to establish a relationship model between complex task load and short-term fatigue; S64. Introduce a non-linear and time-dependent pattern to improve the cumulative fatigue index; S65. The periodic component of the task load is modeled by Fourier series expansion, a periodic interaction term between the task load and the fatigue index is introduced, and the task intensity is combined with the periodic fluctuation to enhance the sensitivity of fatigue to the periodic fluctuation; among them, the introduced periodic interaction term between the task load and the fatigue index is as follows: This interaction term adjusts the periodic effect of task load, making the fatigue prediction conform to the actual periodic changes; among them, is the task load, indicating the task intensity and complexity at time , and represents the amplitude of the periodic task load fluctuation, that is, the periodic change of the task load over time; represents the main period of the task load. S66. To simulate the fatigue recovery effect, introduce a dynamic recovery equation based on negative exponential decay, and update the cumulative fatigue index after adding a recovery term; S67. Detect the fatigue accumulation risk according to the change trend of the cumulative fatigue index, and add acceleration and curvature terms to capture the non-linear change trend of fatigue accumulation.

2. A fatigue detection and periodic fatigue assessment method for the duty crew on the ship's bridge according to claim 1, characterized in that Step S1 specifically includes: S11. Use a filter to remove the noise in the image; S12. Use adaptive histogram equalization to enhance the regions with uneven illumination; S13. Extract frames from the video at a fixed frame rate as the analysis input, and select the region of interest according to the specific regions of the face and actions to reduce the calculation amount.

3. A fatigue detection and periodic fatigue assessment method for the duty crew on the ship's bridge according to claim 1, characterized in that, Step S2 specifically includes: S21. Extract the facial feature points of crew members from the video frames and detect the facial key points; S22. Calculate the number of blinks per minute by detecting the change in the distance between the upper and lower eyelids, and extract the facial action features; S23. Calculate the eye closure ratio through specific key points and extract the facial action features; S24. Detect the mouth opening degree through the key points on both sides of the upper and lower lips and extract the facial action features; S25. Input the facial image region into a pre-trained convolutional neural network model to extract deep features.

4. A method for fatigue detection and periodic fatigue assessment of ship bridge watch officers, according to claim 1, characterized in that Step S3 specifically includes: S31. Define a time window; S32. Calculate the coordinate change of the top of the head within the defined time window to obtain the head displacement feature; S33. Calculate the angle between the top of the head and the shoulders within the defined time window to obtain the head tilt angle; S34. Determine the head rotation by the relative positions between the ears within the defined time window, and calculate the head rotation angle; S35. Calculate the displacement of the knees within the defined time window to obtain the leg position change feature; S36. Calculate the center of gravity position by the weighted average of multiple key points within the defined time window; S37. Calculate the leg tilt angle within the defined time window.

5. A method for fatigue detection and periodic fatigue assessment of the on-duty crew on the ship's bridge according to claim 1, characterized in that, Step S4 specifically includes: S41. Standardize each type of feature; S42. Concatenate the facial features and human body pose features to fuse them into a high-dimensional feature vector, and perform dimensionality reduction on the high-dimensional feature vector.

6. A fatigue detection and periodic fatigue assessment method for on-duty crew members on a ship's bridge according to claim 1, characterized in that Step S5 specifically includes: Step S51. Construct a training data set consisting of the fused features and the corresponding short-term fatigue index; Step S52. The random forest model trains the data by constructing multiple decision trees; Step S53. After fusing the facial and human body pose features, the score obtained by the random forest model represents the possibility of the individual's fatigue state, that is, the instantaneous fatigue state score is the predicted value output by the random forest model.

Images