Fatigue detection and periodic fatigue evaluation method for ship console on-duty crew

Abstract

The invention discloses a ship bridge on-duty crew fatigue detection and periodic fatigue evaluation method, which relates to the technical field of ship safety, and comprises the following steps of: acquiring facial information and human body posture information of a crew through a shipborne camera, and performing data preprocessing including denoising, illumination compensation and video frame extraction; in the feature extraction stage, face key point features (such as blink frequency, eye closing proportion and mouth opening) and human body posture features (such as head actions, center-of-gravity shift and leg actions) are extracted respectively. And after feature standardization and splicing, a short-term fatigue index is calculated by using a random forest model. And in combination with periodic work and rest data, a cumulative fatigue index is improved through a nonlinear cumulative task load model, Fourier series periodic load modeling and a dynamic fatigue recovery model, and dynamic evaluation and early warning of the long-term fatigue of the sailors are realized. The system can accurately monitor the fatigue state of the sailors in a complex marine environment, and guarantees the operation safety of the ship and the health of the sailors.

Description

Technical Field

[0001] The present invention relates to the field of ship safety technology, and in particular to a method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge. Background Art

[0002] Crew members are prone to fatigue due to the continuous high-intensity working environment during long-term work at sea. Once this fatigue accumulates, it will not only affect the accuracy and judgment of their tasks, but may also directly threaten the safety of ships and crew members. According to statistics, fatigue driving by duty personnel is the most important factor causing maritime traffic accidents. Therefore, accurate detection of crew fatigue status, especially in an environment where high-intensity work and rest alternate, and analysis of their periodic fatigue accumulation and recovery status are of great significance to ensure the safety of ship operation and the health of crew members.

[0003] At present, the methods for crew fatigue detection are mostly focused on facial feature detection, physiological signal monitoring and behavior recognition. However, the existing detection methods have some shortcomings. For example, the collection of physiological signals usually requires wearing additional sensor equipment, which will affect the crew's operating comfort; a single facial or physiological signal cannot fully reflect the crew's fatigue state; the existing fatigue detection methods are all short-term fatigue detection of crew members, and do not consider the analysis of their periodic fatigue accumulation and recovery state in an environment where the crew members alternate between high-intensity work and rest. Summary of the invention

[0004] According to the technical problems raised above, a method for fatigue detection and periodic fatigue assessment of crew members on duty at the ship's bridge is provided. The present invention realizes accurate monitoring of crew fatigue status through video data acquisition and processing, facial feature extraction, human posture feature extraction, feature fusion, fatigue index detection and long-term fatigue assessment.

[0005] The technical means adopted by the present invention are as follows: A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge comprises the following steps: S1. Video data acquisition and processing: collect crew members’ facial information and human posture information through the ship-borne camera, and perform denoising, illumination compensation, video frame extraction and region of interest (ROI) selection; S2, facial feature extraction, extracting facial feature points of crew members from video frames, and calculating blinking frequency, eye closure ratio and mouth opening, while inputting the facial image area into a pre-trained convolutional neural network model (CNN model) to extract deep features; S3, human posture feature extraction, select a time window of fixed length, and extract head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position and leg tilt angle; S4, feature fusion, standardize each type of feature, splice facial features and human posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing; S5. Short-term fatigue index detection: construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after fusion features; S6. Long-term fatigue detection, capture periodic work and rest data, smooth the short-term fatigue index, establish a model of the relationship between complex task load and short-term fatigue, introduce a nonlinear and time-dependent cumulative fatigue index, model the periodic task load, introduce the interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and predict and alarm the fatigue trend based on the changing trend of the cumulative fatigue index.

[0006] Furthermore, step S1 specifically includes: S11, using a filter to remove noise in the image; S12, using adaptive histogram equalization to enhance areas with uneven lighting; S13, extracting frames from the video at a fixed frame rate as analysis input, and selecting a region of interest (ROI) according to the specific area of ​​the face and action to reduce the amount of calculation. In this embodiment, the ROI can be a rectangle or other shape to frame the area of ​​the face and lower body.

[0007] Furthermore, step S2 specifically includes: S21, extracting facial feature points of the crew member from the video frame and detecting facial key points; S22, calculating the number of blinks per minute by detecting the change in the distance between the upper and lower eyelids, and extracting facial movement features; S23, calculating the eye closure ratio through specific key points and extracting facial action features; S24, detecting the opening degree of the mouth through key points on the upper and lower sides of the mouth, and extracting facial action features; S25. Input the facial image area into a pre-trained convolutional neural network model (CNN model) to extract deep features.

[0008] Furthermore, step S3 specifically includes: S31, defining a time window; S32, calculating the coordinate change of the top of the head within the defined time window to obtain the head displacement feature; S33, within the defined time window, calculating the angle between the top of the head and the shoulder to obtain the head tilt angle; S34, within the defined time window, judging the head rotation according to the relative positions between the ears, and calculating the head rotation angle; S35, calculating the displacement of the knee within the defined time window to obtain the leg position change characteristics; S36. Calculate the center of gravity position by weighted average of multiple key points within the defined time window: S37. Calculate the leg tilt angle within the defined time window.

[0009] Further, step S4 specifically includes: S41, standardize each type of feature; S42, concatenating facial features and human body posture features to fuse them into a high-dimensional feature vector, and performing dimensionality reduction processing on the high-dimensional feature vector.

[0010] Furthermore, step S5 specifically includes: Step S51, constructing a training data set consisting of fusion features and corresponding short-term fatigue index; Step S52: The random forest model trains the data by constructing multiple decision trees; Step S53: After fusing the facial and human posture features, the score obtained by the random forest model is used to represent the possibility of the individual's fatigue state, that is, the instantaneous fatigue state score is the predicted value output by the random forest model.

[0011] Further, step S6 specifically includes: S61. Periodic work data capture through shift schedules and task logs; S62, using an exponentially weighted moving average to smooth the short-term fatigue index of the short-term fatigue detection; S63. Use multidimensional nonlinear functions to establish a model of the relationship between complex task load and short-term fatigue; S64, introduce nonlinear and time-dependent models to improve the cumulative fatigue index; S65. Use Fourier series expansion to model the periodic component of task load, introduce the periodic interaction term between task load and fatigue index, combine task intensity with periodic fluctuations, and enhance the sensitivity of fatigue under periodic fluctuations; S66. To simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced, and the cumulative fatigue index is updated after adding the recovery term; S67. Detect fatigue accumulation risk based on the changing trend of the cumulative fatigue index, and add acceleration and curvature terms to capture the nonlinear changing trend of fatigue accumulation.

[0012] Compared with the prior art, the present invention has the following advantages: 1. The present invention provides a method for fatigue detection and periodic fatigue assessment of crew members on duty at the ship's bridge, which does not require additional physiological signal acquisition equipment and reduces interference with crew operations.

[0013] 2. The present invention provides a method for fatigue detection and periodic fatigue assessment of crew members on duty at the ship's bridge, which can comprehensively reflect the fatigue status of the crew members by combining facial features and body posture features.

[0014] 3. The present invention provides a method for fatigue detection and periodic fatigue assessment for crew members on duty at the ship's bridge, which takes into account the periodic fatigue accumulation and recovery process of the crew members' alternating high-intensity work and rest, and improves the accuracy and reliability of fatigue detection.

[0015] 4. The present invention provides a method for fatigue detection and periodic fatigue assessment of crew members on duty at the ship's bridge. Through a dynamic fatigue recovery model and periodic task load modeling, the fatigue status of the crew members can be assessed in real time and early warning can be given.

[0016] Based on the above reasons, the present invention can be widely promoted in the fields of ship safety and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0021] like Figure 1 As shown, the present invention provides a method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge, comprising the following steps: S1. Video data acquisition and processing: collect crew members’ facial information and human posture information through ship-borne cameras, and perform denoising, illumination compensation, video frame extraction and ROI selection; S2, facial feature extraction, extracting facial feature points of crew members from video frames, and calculating blinking frequency, eye closure ratio and mouth opening, while inputting facial image area into the pre-trained CNN model to extract deep features; S3, human posture feature extraction, select a time window of fixed length, and extract head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position and leg tilt angle; S4, feature fusion, standardize each type of feature, splice facial features and human posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing; S5. Short-term fatigue index detection: construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after fusion features; S6. Long-term fatigue detection, capture periodic work and rest data, smooth the short-term fatigue index, establish a model of the relationship between complex task load and short-term fatigue, introduce a nonlinear and time-dependent cumulative fatigue index, model the periodic task load, introduce the interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and predict and alarm the fatigue trend based on the changing trend of the cumulative fatigue index.

[0022] In specific implementation, as a preferred embodiment of the present invention, step S1 specifically includes: S11. Use a filter to remove noise from the image to reduce interference in subsequent analysis. The filtering formula is as follows:

[0023] in, represents the pixel value, represents the filter weight, Represents the radius of the filter, which determines the size of the filter. and Respectively represent the offset of the filter in the vertical and horizontal directions.

[0024] S12, perform illumination compensation, use adaptive histogram equalization to enhance the area with uneven illumination and maintain the clarity of the video. The illumination compensation formula is as follows:

[0025] in, and Respectively represent the minimum and maximum grayscale values ​​in the local window, Indicates grayscale levels.

[0026] S13, extracting frames from the video at a fixed frame rate as analysis input, and selecting a region of interest (ROI) according to the specific area of ​​the face and action to reduce the amount of calculation. In this embodiment, the ROI can be a rectangle or other shape to frame the area of ​​the face and lower body.

[0027] In specific implementation, as a preferred embodiment of the present invention, step S2 specifically includes: S21, extract facial feature points of the crew member from the video frame and detect facial key points; in this embodiment, facial key points include eyes, nose, mouth, chin, etc. Facial key point coordinate extraction: Assume that the coordinates of facial key points are ,in Indicates different key points (such as eyes, mouth corners, etc.), then in the frame When , the coordinates of the facial key points can be expressed as .

[0028] S22, by detecting the change in the distance between the upper and lower eyelids, the number of blinks per minute is calculated to extract facial action features; in this embodiment, it is assumed that the distance between the upper and lower eyelids of the left eye is:

[0029] When the distance is below a certain threshold, it can be judged as blinking. Indicated in At this moment, the vertical coordinate of the upper eyelid, Indicated in At this moment, the vertical coordinate of the lower eyelid, Indicated in At this moment, the vertical distance between the upper and lower eyelids.

[0030] S23. Calculate the eye closure ratio through specific key points to extract facial action features; the calculation formula of the eye closure ratio is as follows:

[0031] in, arrive Represents the 6 key points of the left and right eyes respectively.

[0032] S24, detecting the mouth opening degree through the key points on the upper and lower sides of the mouth, and extracting facial action features. The calculation formula for the mouth opening degree is as follows:

[0033] in, Indicates the coordinates (vertical coordinates) of the key points on the upper side of the mouth, Represents the coordinates (vertical coordinates) of the key points on the lower side of the mouth, Indicates the coordinates of the key points on the left side of the mouth (horizontal coordinate), Represents the coordinates (horizontal coordinate) of the key points on the right side of the mouth.

[0034] S25. Input the facial image region into the pre-trained CNN model to extract deep features. The CNN network architecture usually includes convolutional layers, pooling layers, and fully connected layers. Given an input image , the convolutional layer operation is:

[0035] in, represents the activation function, and Respectively represent the size of the convolution kernel in the vertical and horizontal directions. represents the convolution kernel weight, Indicates that the image is at position The pixel value at Represents bias; the convolution layer extracts local features, and after multiple convolution layers and pooling layers, a deep feature vector is finally obtained.

[0036] In specific implementation, as a preferred embodiment of the present invention, step S3 specifically includes: S31, define the time window; during the detection process, select a time window of fixed length , the sampling rate is , extract various features within the time window, then the time window contains Frame data.

[0037] S32. Calculate the coordinate change of the top of the head within the defined time window to obtain the head displacement feature. The calculation formula of the coordinate change of the top of the head is as follows:

[0038] in, Indicates time The coordinates of the top of the head, Indicates time The coordinates of the top of the head. Indicates the start time, that is, the start time of the time window. Indicates the length of the time window.

[0039] S33. Calculate the angle between the top of the head and the shoulder within the defined time window to obtain the head tilt angle; i.e., the head tilt angle It can be expressed as:

[0040] in, Indicates the start time, that is, the start time of the time window; Indicates the length of the time window; () represents the arccosine function, which is used to calculate the angle; Indicates time The coordinates of the left shoulder at ; Indicates time The coordinates of the right shoulder at ; Indicates time The coordinates of the top of the head; Indicates time The coordinates of the center point of the shoulder; S34, within the defined time window, determine the head rotation by the relative position between the ears, and calculate the head rotation angle; that is, the head rotation angle It can be calculated by the following formula:

[0041] in, and Respectively The coordinates of the left and right ears at the moment.

[0042] S35. Calculate the displacement of the knee within the defined time window to obtain the leg position change characteristics; wherein: The displacement of the left knee can be expressed as:

[0043] in, Indicates time The coordinates of the left knee, Indicates time The coordinates of the left knee.

[0044] The displacement of the right knee can be expressed as:

[0045] in, Indicates time The coordinates of the right knee, Indicates time The coordinates of the right knee.

[0046] S36. In the defined time window, the center of gravity position is calculated by weighted average of multiple key points; the change of the crew's center of gravity reflects whether their body is in an unstable state. The center of gravity position is calculated by weighted average of multiple key points, such as left knee, right knee, left ankle, right ankle, left hip, right hip, etc. The formula for the center of gravity position is:

[0047] in, Represents the weight value of the key point, Indicates time Time The location of the key points.

[0048] S37, within the defined time window, calculating the leg tilt angle. The leg tilt angle is It is expressed as follows:

[0049] in, Indicates time The coordinates of the hip at Indicates time The coordinates of the knee, Indicates time The coordinates of the ankle at time .

[0050] In specific implementation, as a preferred embodiment of the present invention, step S4 specifically includes: S41, standardize each type of feature; Since the range of values ​​of facial features and human posture features is different, in order to make the influence of each feature on the model consistent, in this embodiment, each type of feature is standardized. Representation characteristics The value of is standardized as follows:

[0051] in, and Characteristics The mean and standard deviation of .

[0052] S42, concatenate facial features and human body posture features to form a high-dimensional feature vector, as follows:

[0053] The fused high-dimensional feature vector is then subjected to dimensionality reduction processing (such as PCA or linear discriminant analysis LDA) to reduce the impact of noise and improve the training efficiency of the model. represents the facial features extracted from the video frame, Represents the human posture feature vector extracted from the video.

[0054] In specific implementation, as a preferred embodiment of the present invention, step S5 specifically includes: Step S51, constructing a training data set consisting of fusion features and corresponding short-term fatigue index; In this embodiment, the fusion feature and the corresponding labeled short-term fatigue index The training data set is composed as follows: in, Indicates the sample size.

[0055] Step S52: The random forest model trains the data by constructing multiple decision trees; In this embodiment, the prediction function of the random forest model is:

[0056] in, represents the number of decision trees, Indicates A decision tree.

[0057] Step S53: After fusing the facial and human posture features, the score obtained by the random forest model is used to represent the possibility of the individual's fatigue state, that is, the fatigue index is the predicted value output by the random forest model.

[0058] In this embodiment, the instantaneous fatigue state score It is the score obtained by the model after fusing facial and human posture features. The calculation formula is as follows:

[0059] in, Represents the predicted value output by the random forest model.

[0060] In specific implementation, as a preferred embodiment of the present invention, step S6 specifically includes: S61. Periodic work data capture through shift schedules and task logs; In this embodiment, the shift schedule and task log are used to capture periodic work and rest data, record the task type and duration of each time period, and form a multi-dimensional workload sequence. , indicating that Information such as work intensity and task complexity at any given moment.

[0061] S62, using an exponentially weighted moving average to smooth the short-term fatigue index of the short-term fatigue detection; Short-term fatigue index for short-term fatigue detection Indicates the fatigue level at each time point. To eliminate noise and enhance detection stability, an exponentially weighted moving average is applied to To perform smoothing:

[0062] in, represents the smoothing coefficient, express The instantaneous fatigue state score at each moment, Indicates the last moment Smooth fatigue index.

[0063] S63. Use multidimensional nonlinear functions to establish a model of the relationship between complex task load and short-term fatigue; In this embodiment, in order to improve the cumulative characterization of task load on long-term fatigue, the task load is modeled as a multidimensional nonlinear function to capture the interactive effects between different loads. The nonlinear task load at each moment is defined as:

[0064] in, It is the dimension of task load, representing different categories or factors of tasks. represents the linear coefficient of task load, Indicates at time Corresponding dimensions The task load value, It represents the interaction coefficient between loads, controlling the cumulative effect of the synergy of different loads on fatigue; Indicates at time Corresponding dimensions task load value.

[0065] S64, introduce nonlinear and time-dependent models to improve the Cumulative Fatigue Index (CFI); In this embodiment, a nonlinear and time-dependent cumulative fatigue index is introduced. , using short-term fatigue and nonlinear task load to represent the complex accumulation of fatigue, the formula is:

[0066] in, It is the attenuation factor of the cumulative fatigue index, which indicates the influence of the fatigue index at the previous moment on the current moment. Indicates the cumulative fatigue index of the previous moment, Convert the effect of task load on fatigue into a slowly accumulating mode to avoid excessive fluctuations in a single task; is a parameter used to adjust the impact of task load on the cumulative fatigue index. is the nonlinear mapping function of short-term fatigue, Controls the degree of nonlinearity of short-term fatigue effects.

[0067] S65. Use Fourier series expansion to model the periodic component of task load, introduce the periodic interaction term between task load and fatigue index, combine task intensity with periodic fluctuations, and enhance the sensitivity of fatigue under periodic fluctuations; In this embodiment, in order to capture the periodic fluctuation of the task load, Fourier series expansion is used to model the periodic component of the task load. Definition:

[0068] in, represents the basic level of task load, and represents the amplitude of the task load at different frequencies, Represents the main cycle of task load (such as 12 hours or 24 hours). The model can simulate the periodic changes of task load and more accurately predict the fatigue accumulation process in combination with the CFI cumulative fatigue index.

[0069] Introducing the period interaction term between task load and fatigue index , combining task intensity with periodic fluctuations to enhance the sensitivity of fatigue under periodic fluctuations. The periodic interaction term between task load and fatigue index is introduced. as follows:

[0070] This interaction term can adjust the periodic effect of task load and make fatigue prediction more consistent with actual periodic changes. is the task load, which means at time The intensity and complexity of the tasks and It represents the amplitude of periodic task load fluctuation, that is, the periodic change of task load over time.

[0071] S66. To simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced, and the cumulative fatigue index is updated after adding the recovery term; In this embodiment, in order to simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced. When , that is, during the rest period, the dynamic recovery model is expressed as:

[0072] in, represents the coefficient of recovery, which controls the natural recovery rate from fatigue; represents the attenuation coefficient of recovery; Indicates the accumulated rest time since entering the rest state.

[0073] The CFI update formula after adding the recovery term is:

[0074] This formula dynamically adjusts task load, short-term fatigue and recovery effect in the cumulative fatigue index, making fatigue accumulation more consistent with dynamic changes over a long time scale.

[0075] S67. Detect fatigue accumulation risk based on the changing trend of the cumulative fatigue index, and add acceleration and curvature terms to capture the nonlinear changing trend of fatigue accumulation.

[0076] In this embodiment, fatigue accumulation risk is detected according to the change trend of CFI, and acceleration and curvature terms are added to capture the nonlinear change trend of fatigue accumulation:

[0077] like Indicates that the fatigue index is increasing too fast, early warning; if A fatigue alarm is triggered and it is recommended to take a rest immediately. and The specific value can be adjusted according to the working mode and fatigue performance of different crew members. It represents the growth rate threshold of the fatigue index. When the fatigue index exceeds this value, it means that the crew's fatigue status has reached a risk level that requires attention. It is used to indicate the critical value of triggering an alarm when the fatigue index accumulates too quickly. It is usually set to trigger an alarm when the accumulated fatigue index increases to a certain extent. According to the experimental results, it can be given The reference value is 0.01 to 0.05. The reference value is 10% to 20%.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge, characterized in that: The following steps are involved: S1. Video data acquisition and processing: collect crew members’ facial information and human posture information through ship-borne cameras, and perform denoising, illumination compensation, video frame extraction and region of interest selection; S2, facial feature extraction, extracting facial feature points of crew members from video frames, and calculating blinking frequency, eye closure ratio and mouth opening, while inputting facial image area into the pre-trained convolutional neural network model to extract deep features; S3, human posture feature extraction, select a time window of fixed length, and extract head movement features and leg movement features within the time window, including head displacement, head tilt angle, head rotation angle, leg position change, center of gravity position and leg tilt angle; S4, feature fusion, standardize each type of feature, splice facial features and human posture features to form a high-dimensional feature vector, and perform dimensionality reduction processing; S5. Short-term fatigue index detection: construct a training data set, train the data through a random forest model, and calculate the short-term fatigue index after fusion features; S6. Long-term fatigue detection, capture periodic work and rest data, smooth the short-term fatigue index, establish a model of the relationship between complex task load and short-term fatigue, introduce a nonlinear and time-dependent cumulative fatigue index, model the periodic task load, introduce the interaction term of periodic load on fatigue, establish a dynamic fatigue recovery model, update the cumulative fatigue index, and predict and alarm the fatigue trend based on the changing trend of the cumulative fatigue index.

2. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge according to claim 1, characterized in that: Step S1 specifically includes: S11, using a filter to remove noise in the image; S12, using adaptive histogram equalization to enhance areas with uneven lighting; S13. Extract frames from the video at a fixed frame rate as analysis input, and select regions of interest based on specific areas of the face and action to reduce the amount of calculation.

3. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship's bridge according to claim 1, characterized in that: Step S2 specifically includes: S21, extracting facial feature points of the crew member from the video frame and detecting facial key points; S22, calculating the number of blinks per minute by detecting the change in the distance between the upper and lower eyelids, and extracting facial movement features; S23, calculating the eye closure ratio through specific key points and extracting facial action features; S24, detecting the opening degree of the mouth through key points on the upper and lower sides of the mouth, and extracting facial action features; S25. Input the facial image region into the pre-trained convolutional neural network model to extract deep features.

4. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship bridge according to claim 1, characterized in that: Step S3 specifically includes: S31, defining a time window; S32, calculating the coordinate change of the top of the head within the defined time window to obtain the head displacement feature; S33, within the defined time window, calculating the angle between the top of the head and the shoulder to obtain the head tilt angle; S34, within the defined time window, judging the head rotation according to the relative positions between the ears, and calculating the head rotation angle; S35, calculating the displacement of the knee within the defined time window to obtain the leg position change characteristics; S36, calculating the center of gravity position by weighted average of multiple key points within the defined time window; S37. Calculate the leg tilt angle within the defined time window.

5. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship bridge according to claim 1, characterized in that: Step S4 specifically includes: S41, standardize each type of feature; S42, concatenating facial features and human body posture features to fuse them into a high-dimensional feature vector, and performing dimensionality reduction processing on the high-dimensional feature vector.

6. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship bridge according to claim 1, characterized in that: Step S5 specifically includes: Step S51, constructing a training data set consisting of fusion features and corresponding short-term fatigue index; Step S52: The random forest model trains the data by constructing multiple decision trees; Step S53: After fusing the facial and human posture features, the score obtained by the random forest model is used to represent the possibility of the individual's fatigue state, that is, the instantaneous fatigue state score is the predicted value output by the random forest model.

7. A method for fatigue detection and periodic fatigue assessment of crew members on duty at a ship bridge according to claim 1, characterized in that: Step S6 specifically includes: S61. Periodic work data capture through shift schedules and task logs; S62, using an exponentially weighted moving average to smooth the short-term fatigue index of the short-term fatigue detection; S63. Use multidimensional nonlinear functions to establish a model of the relationship between complex task load and short-term fatigue; S64, introduce nonlinear and time-dependent models to improve the cumulative fatigue index; S65. Use Fourier series expansion to model the periodic component of task load, introduce the periodic interaction term between task load and fatigue index, combine task intensity with periodic fluctuations, and enhance the sensitivity of fatigue under periodic fluctuations; S66. To simulate the recovery effect of fatigue, a dynamic recovery equation based on negative exponential decay is introduced, and the cumulative fatigue index is updated after adding the recovery term; S67. Detect fatigue accumulation risk based on the changing trend of the cumulative fatigue index, and add acceleration and curvature terms to capture the nonlinear changing trend of fatigue accumulation.

Images