Ship crew working state monitoring method and device and electronic equipment

Abstract

The invention discloses a ship crew working state monitoring method, and belongs to the technical field of ship control. The method comprises the following steps: acquiring environmental data of a ship and physiological data and behavior data of a target crew in the ship; performing time-frequency analysis on the environment data, the physiological data and the behavior data to obtain state time-frequency characteristics of the target crew; performing fusion time sequence modeling based on the state time-frequency characteristics to obtain a working state model of the target crew; the working state model is called to adjust early warning rules, the working state parameters are evaluated through the adjusted early warning rules, the working state monitoring result of the target crew is obtained, the early warning rules comprise a threshold rule, a trend rule and an association rule, and the working state parameters are obtained based on the physiological data and the behavior data. According to the method, by fusing time sequence modeling and dynamically adjusting the early warning rule, the early warning accuracy and practicability are improved, and a powerful guarantee is provided for safe operation of the ship.

Description

Technical Field

[0001] This application belongs to the field of photovoltaic power generation technology, and in particular relates to a method, device and electronic equipment for monitoring the working status of ship crew. Background Technology

[0002] Seafarers typically work in a unique environment, often in a closed setting. Long voyages and high-intensity work can easily lead to physical and mental fatigue. In addition, seafarers may face psychological problems such as loneliness and anxiety, and may even develop depression. Therefore, it is necessary to monitor the working status of seafarers to understand changes in their health status and avoid affecting work efficiency and safety.

[0003] Traditional monitoring of crew working conditions typically involves evaluating physiological data and psychological reports obtained from physical examinations of crew members and setting fixed thresholds to trigger early warnings.

[0004] However, this method ignores the dynamic continuity of crew status, which may lead to inaccurate or excessive warnings, affecting crew work efficiency and mental health. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, device, and electronic equipment for monitoring the working status of ship crew members. By integrating time-series modeling and dynamically adjusting early warning rules, the accuracy and practicality of early warning are improved, providing strong support for the safe operation of ships.

[0006] Firstly, this application provides a method for monitoring the working status of a ship's crew, the method comprising:

[0007] Acquire environmental data of the ship, as well as physiological and behavioral data of the target crew members on the ship;

[0008] Time-frequency analysis is performed on the environmental data, physiological data, and behavioral data to obtain the time-frequency characteristics of the target crew member's state.

[0009] Based on the aforementioned time-frequency characteristics, a fusion time-series model is performed to obtain the working state model of the target crew member. The working state model is used to describe the temporal changes and dynamic characteristics of the working state of the target crew member. The dynamic characteristics include the changing trends and fluctuations of the physiological and behavioral data of the target crew member in different time periods.

[0010] The work status model is invoked to adjust the early warning rules. The work status parameters are evaluated using the adjusted early warning rules to obtain the work status monitoring results of the target crew member. The early warning rules include threshold rules, trend rules, and correlation rules. The work status parameters are obtained based on the physiological data and the behavioral data.

[0011] According to one embodiment of this application, the step of performing time-frequency analysis on the environmental data, the physiological data, and the behavioral data to obtain the time-frequency characteristics of the target crew member's state includes:

[0012] Based on the changing trends of the environmental data, physiological data, and behavioral data, the environmental data, physiological data, and behavioral data are respectively divided into multiple stage data, including stable stage data, fluctuating stage data, and abnormal stage data.

[0013] Time-frequency analysis is performed on the stage data to obtain the time-frequency domain representation of the stage data at multiple frequencies, so as to adjust the order of the numerical transformation of the time-frequency analysis according to the time-frequency domain representation;

[0014] Extract key local features and global time-frequency features from the time-frequency domain representation;

[0015] The key local features and global time-frequency features are fused to obtain the state time-frequency features.

[0016] According to one embodiment of this application, the extraction of key local features and global time-frequency features from the time-frequency domain representation includes:

[0017] Based on the target crew's work rhythm, multiple time windows are set, and the window length and transformation scale of each time window are determined.

[0018] Based on the window length and the transform scale, the time-frequency domain representation is divided into low-order representation and high-order representation;

[0019] The frequency components, energy distribution, and time-frequency variation trends of the higher-order representation are extracted to obtain the key local features;

[0020] The overall energy level, spectral distribution, and time-frequency consistency are extracted from the low-order representation to obtain the global time-frequency features.

[0021] According to one embodiment of this application, the key local features and global time-frequency features are fused to obtain the state time-frequency features, including:

[0022] Higher-order statistical analysis is performed on key local features within each time window to obtain higher-order statistics, which are used to describe the nonlinear relationship between the key local features.

[0023] The global time-frequency features are subjected to low-rank matrix decomposition to obtain a low-rank matrix and a sparse matrix;

[0024] Based on the low-rank matrix and the sparse matrix, the key components and noise components in the global time-frequency features are determined. The key components are used to characterize the differences in physiological and behavioral characteristics of the target crew members under multiple working conditions.

[0025] Based on the correlation and complementarity between the higher-order statistics, the key components, and the noise components, the key components of the global time-frequency features within each time window are weighted and fused to obtain the state time-frequency features.

[0026] According to one embodiment of this application, the step of performing fusion time-series modeling based on the state time-frequency features to obtain the working state model of the target crew member includes:

[0027] The state time-frequency features are subjected to univariate multi-step time series detection to generate multiple time series prediction sequences, which include a forward feature sequence and a reverse feature sequence.

[0028] Forward features are extracted from the forward feature sequence, and inverse features are extracted from the inverse feature sequence. The forward features and inverse features are then interactively integrated to obtain time-series features.

[0029] The working state model is constructed using the aforementioned temporal features.

[0030] According to one embodiment of this application, constructing the working state model using the time-series features includes:

[0031] The temporal features are input into a temporal prediction network to obtain multiple predicted sequences output by the temporal prediction network.

[0032] Calculate the similarity and difference between each predicted sequence and the historical working status of the target crew member to obtain a similarity sequence and a difference sequence;

[0033] Based on the similarity sequence and the difference sequence, a work state transition probability matrix is ​​constructed, which is used to describe the transition probability of the target crew member between different work states.

[0034] The working state model is constructed based on the working state transition probability matrix.

[0035] According to one embodiment of this application, the work status model is invoked to adjust the early warning rules, and the work status parameters are evaluated using the adjusted early warning rules to obtain the work status monitoring results of the target crew member, including:

[0036] Based on the correlation between the physiological data and the behavioral data and the work status, work status parameters are determined, including fatigue index, stress level and health status.

[0037] Random subspace identification is performed on the working state model to extract the dynamic features of the working state model;

[0038] Based on the aforementioned dynamic characteristics, the working state transition patterns of the target crew member are analyzed;

[0039] Based on the aforementioned working state transition pattern, adjust the threshold rule, trend rule, and correlation rule in the early warning rule;

[0040] The working status parameters are evaluated using the adjusted threshold rules, trend rules, and correlation rule warning rules to generate the working status monitoring results.

[0041] According to one embodiment of this application, after obtaining the working state model of the target crew member, the method further includes:

[0042] By using a risk situation awareness model, risk situation analysis is performed on the time-frequency characteristics of the state to identify multivariate risks, including fatigue trends, health trends, and behavioral trends.

[0043] A dynamic and continuous analysis is performed on the multivariate risk and the working status model to obtain the risk situation evolution trend of the target crew member;

[0044] Based on the evolving trend of the risk situation, the parameters of the working state model are adjusted.

[0045] Secondly, this application provides a device for monitoring the working status of ship crew members, the device comprising:

[0046] The acquisition module is used to acquire environmental data of the ship, as well as physiological and behavioral data of the target crew members on the ship;

[0047] The first processing module is used to perform time-frequency analysis on the environmental data, the physiological data, and the behavioral data to obtain the time-frequency characteristics of the target crew member's state.

[0048] The second processing module is used to perform fusion time-series modeling based on the state time-frequency features to obtain the working state model of the target crew member. The working state model is used to describe the temporal changes and dynamic features of the working state of the target crew member. The dynamic features include the changing trends and fluctuations of the physiological and behavioral data of the target crew member in different time periods.

[0049] The third processing module is used to call the work status model to adjust the early warning rules, evaluate the work status parameters through the adjusted early warning rules, and obtain the work status monitoring results of the target crew member. The early warning rules include threshold rules, trend rules and correlation rules, and the work status parameters are obtained based on the physiological data and the behavioral data.

[0050] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for monitoring the working status of ship crew as described in the first aspect above.

[0051] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for monitoring the working status of ship crew as described in the first aspect above.

[0052] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the ship crew working status monitoring method as described in the first aspect.

[0053] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method for monitoring the crew working status of a ship as described in the first aspect above.

[0054] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0055] The method, device, and electronic equipment for monitoring the working status of ship crew members provided in this application have the following advantages over the prior art:

[0056] (1) By collecting and analyzing ship environmental data, target crew physiological data and behavioral data in real time, and using time-frequency analysis and fusion time-series modeling technology, a model that can accurately reflect changes in crew working status was established. By adjusting the early warning rules, real-time monitoring and early warning of crew working status were realized, providing strong support for the safe operation of ships and crew health management. By integrating time-series modeling and dynamically adjusting the early warning rules, the accuracy and practicality of the early warning were improved, providing strong protection for the safe operation of ships.

[0057] (2) By dividing environmental data, physiological data and behavioral data into stages based on the changing trends, we can more accurately locate the different characteristic stages of the data, making the subsequent analysis of each stage of data more targeted and effective. This avoids the problem of information overload and lack of feature prominence that may result from uniform processing of the overall data. By extracting key local features and global time-frequency features, we can mine useful information from different levels of the data. This not only pays attention to the local details of the data but also grasps the overall characteristics of the data, providing rich feature information for comprehensively and accurately depicting the working status of crew members. Attached Figure Description

[0058] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0059] Figure 1 This is a flowchart illustrating the method for monitoring the working status of ship crew members provided in the embodiments of this application;

[0060] Figure 2 This is a schematic diagram of the structure of the ship's crew working status monitoring device provided in the embodiments of this application;

[0061] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0063] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0064] The following description, in conjunction with the accompanying drawings, details the method for monitoring the working status of ship crew members, the device for monitoring the working status of ship crew members, the electronic equipment, and the readable storage medium provided in this application, through specific embodiments and application scenarios.

[0065] Among them, the method for monitoring the working status of ship crew members can be applied to a terminal, which can be executed by the hardware or software in the terminal.

[0066] The method for monitoring the working status of ship crew members provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the method for monitoring the working status of ship crew members. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The following uses an electronic device as the execution subject to illustrate the method for monitoring the working status of ship crew members provided in this application embodiment.

[0067] like Figure 1 As shown, the method for monitoring the working status of the ship's crew includes:

[0068] Step 110: Obtain environmental data of the ship, as well as physiological and behavioral data of the target crew members on the ship;

[0069] Among them, environmental data, such as temperature, humidity, noise level, vibration, and light, reflect the working environment inside the ship.

[0070] The physiological data of the target crew members include heart rate, blood pressure, body temperature, electromyography, and electroencephalography; the behavioral data covers the crew members' facial expressions, eye status, head movements, movement trajectory, operation frequency, rest time, etc. Together, these data constitute a comprehensive monitoring of the crew members' working status.

[0071] In step 110, data is collected and recorded in real time using sensors and monitoring equipment deployed throughout the ship. For example, environmental data is acquired using devices such as temperature and humidity sensors, noise meters, vibration sensors, and light meters; physiological data is measured using wearable heart rate monitors, blood pressure monitors, thermometers, electromyography (EMG) sensors, and electroencephalography (EEG) sensors; and behavioral data is captured and analyzed using devices such as cameras, motion capture devices, and position tracking systems. The obtained data is then aggregated and transmitted to the ship's data processing center to provide a basis for subsequent analysis and evaluation.

[0072] Step 120: Perform time-frequency analysis on the environmental data, physiological data, and behavioral data to obtain the time-frequency characteristics of the target crew member's state.

[0073] Time-frequency analysis involves analyzing environmental, physiological, and behavioral data in the time and frequency domains to obtain the frequency components of the data at different time periods and how these frequency components change over time.

[0074] State-frequency characteristics include the frequency components of environmental, physiological, and behavioral data and their trends over time. They are used to characterize the distribution characteristics of crew members in different working states in terms of time and frequency. They can be used to distinguish the working states of crew members, such as normal, fatigued, and stressed, and thus determine the adaptability and stability of crew members in the ship's working environment, as well as possible health problems or excessive workload.

[0075] For example, the state-time frequency characteristics of environmental data include temperature fluctuations, humidity change rates, and periodic changes in noise levels; the state-time frequency characteristics of physiological data include heart rate variability, blood pressure fluctuations, the diurnal rhythm of body temperature, periodic changes in electromyographic activity, and changes in the intensity of different frequency bands of brain waves over time; the state-time frequency characteristics of behavioral data include the frequency of facial expression changes, the periodicity of eye opening and closing, the frequency and amplitude of head movements, the smoothness and speed changes of movement trajectories, the rhythmicity of operation frequency, and the distribution of rest time.

[0076] In step 120, the environmental data, physiological data, and behavioral data are decomposed into time-frequency features using Short Time Fourier Transform (STFT) to extract the time-frequency characteristics of the ship's working environment and the target crew's state in different time periods. These include the frequency domain features (such as periodicity) and time domain features (such as fluctuations) of each data point. For example, the characteristics of heart rate changes in different time periods, blood pressure fluctuations, body temperature fluctuation trends, frequency components of electromyography and electroencephalography signals, and their changes with time and environmental influences.

[0077] Step 130: Based on the time-frequency characteristics of the state, perform fusion time-series modeling to obtain the working state model of the target crew member. The working state model is used to describe the temporal changes and dynamic characteristics of the working state of the target crew member. The dynamic characteristics include the changing trends and fluctuations of the physiological and behavioral data of the target crew member in different time periods.

[0078] Among them, fusion time series modeling integrates and analyzes environmental, physiological and behavioral data on a time series basis using state time-frequency characteristics to establish a model that can accurately reflect changes in the working status of crew members.

[0079] The work status model can describe the physiological changes, behavioral pattern fluctuations, and their correlation with environmental factors of crew members at different times in a multi-dimensional, dynamic, and temporal manner. It comprehensively considers the impact of the ship's environment on crew members, as well as their own physiological and behavioral responses, and can fully and accurately reflect the work status of crew members in different time periods and working environments. It includes not only changes in crew members' physiological indicators, such as heart rate, blood pressure, and body temperature, but also their behavioral characteristics, such as facial expressions, eye positions, and head movements, thus enabling an understanding of the crew members' work status and its underlying influencing factors.

[0080] Work status is used to characterize the overall work efficiency and health status of crew members in different time periods and working environments.

[0081] Temporal changes are continuous or discrete changes in the working state of crew members over time, reflecting the dynamic evolution of crew members' adaptability and stability in the ship's working environment.

[0082] Dynamic features are used to describe the diversity and complexity of crew members’ working status in temporal changes, including the fluctuations, trends and interactions of physiological and behavioral data.

[0083] In step 130, the time-frequency features of environmental data, physiological data, and behavioral data are aligned and matched in time and frequency, and fused into a unified dataset;

[0084] By using a Hidden Markov Model (HMM) to perform time-series modeling on the dataset, different types of data features are fused to obtain a comprehensive working state model, thereby obtaining a working state model of the target crew members. This model can capture the physiological and behavioral changes of crew members in different time periods, as well as the correlation between these changes and environmental factors.

[0085] In the process of constructing the work state model, the hidden state is defined as the work state, which is used to represent the different physiological or psychological states of the crew members at work. For example, state 1 is the normal working state, state 2 is the fatigue state, state 3 is the high-pressure state, state 4 is the rest state, and state 5 is the abnormal state.

[0086] Using the time-frequency characteristics of aligned environmental, physiological, and behavioral data as observational data associated with each latent state, under normal working conditions, crew members' heart rate and blood oxygen saturation are usually within the normal range, and their movements are relatively regular. Under fatigue, crew members' heart rate will increase, and their behavioral patterns may be characterized by slow movements. Under high stress, crew members may exhibit frequent movement or tense behavioral patterns.

[0087] In a Hidden Markov Model, states are interconnected through transition probabilities. Initialization is performed based on prior knowledge, with the probability of a crew member being in each hidden state at the initial moment being used as the initial state probability distribution π. The state transition probability matrix A is constructed using the probability of a crew member transitioning from one working state to another, and the observation probability matrix B describes the probability distribution of observed data in each hidden state.

[0088] Suppose there are N hidden states, and the transition probability matrix A is an N×N matrix, where each element a pq This represents the probability of transitioning from state p to state q. Each hidden state has a corresponding probability distribution of observation data. Assuming that the observations for each state are drawn from other probability distributions, we obtain the observation probability matrix B.

[0089] During the training process based on HMM, the probability of the model's state sequence under a given observation sequence is calculated using the forward-backward algorithm. The state transition probability matrix A, the observation probability matrix B, and the initial state probability distribution π are estimated as model parameters by maximizing the likelihood function of the observed data.

[0090] By using the time-frequency features of the state as training data input to the model, and continuously iterating and optimizing the model's parameters, the model can make optimal state predictions based on the existing data.

[0091] After model training, the Viterbi algorithm is used for decoding to calculate the probability sequence of crew members in different working states given a series of observation data. The working states of crew members are inferred in real time based on the observed data. Given a new observation sequence based on new environmental, physiological, and behavioral data, the Hidden Markov Model (HMM) is used to infer the working states of crew members. Based on the predicted hidden state sequence, it can be determined whether the target crew member is in a normal, fatigued, stressed, or abnormal state. Precision, recall, and F1 score are used as evaluation metrics. The model's performance is evaluated by comparing the predicted results with the actual situation.

[0092] Step 140: Invoke the work status model to adjust the early warning rules, evaluate the work status parameters through the adjusted early warning rules, and obtain the work status monitoring results of the target crew member. The early warning rules include threshold rules, trend rules and correlation rules. The work status parameters are obtained based on the physiological data and the behavioral data.

[0093] Among them, the early warning rules are used to determine whether the crew's working status has reached or exceeded the preset threshold or trend, thereby triggering corresponding early warning measures. They can be flexibly adjusted and optimized according to the actual situation to adapt to the needs of different ships, different crew members and different working environments.

[0094] Threshold rules are based on specific numerical thresholds set from physiological and behavioral data to trigger warnings. When a data point exceeds or falls below a set threshold, it is considered that the crew member's working condition may be problematic, requiring intervention. For example, a persistently elevated heart rate may indicate that the crew member is fatigued or under high stress, in which case the crew member should be reminded to rest or manage stress.

[0095] Trend rules are based on the changing trends of data. When physiological or behavioral data show an abnormal trend, it is considered that the crew's working condition may be deteriorating, requiring early warning. For example, when a crew member's sleep time gradually decreases while heart rate variability gradually increases, it may indicate that the crew member is gradually entering a state of fatigue, and measures need to be taken to prevent the accumulation of fatigue.

[0096] Association rules are used to capture complex state patterns through joint data analysis, identifying the interrelationships between physiological, behavioral, and environmental data to trigger alerts when multiple data points change simultaneously. For example, when a crew member's facial expressions and eye movements simultaneously indicate tension or anxiety, it may indicate that the crew member is facing a high-stress environment, requiring measures to alleviate their stress. Alternatively, the association between increased heart rate and certain behaviors in high-temperature environments, combined with the complexity of the work task and environmental factors, can help determine if there are potential risks.

[0097] Work status parameters are quantitative indicators that reflect the work status of crew members. Physiological and behavioral data are fused through principal component analysis (PCA), and features are extracted from the fused data to obtain key indicators that reflect the work status of crew members. These indicators are then processed and analyzed to obtain fatigue parameters, stress parameters, and health status parameters, which are used as work status parameters.

[0098] The results of the work status monitoring are a comprehensive assessment of the current work status of the target crew members through early warning rules, including the work status and corresponding early warning information, which is used to guide subsequent management and intervention measures.

[0099] In step 140, the time-frequency characteristics of the target crew member's environmental, physiological, and behavioral data are input into the work status model. The model calculates the target crew member's current work status, such as normal, fatigued, stressed, or abnormal. Based on the work status, threshold rules, trend rules, and correlation rules are adjusted. For example, for crew members in a fatigued state, the threshold rule is appropriately lowered to provide early warning, and a more sensitive trend rule is set to detect the cumulative effect of fatigue in advance. The correlation rule focuses on the correlation between fatigue and heart rate and stress to ensure timely response when multiple physiological indicators are abnormal at the same time.

[0100] The work status parameters are evaluated according to the early warning rules to determine whether the crew's work status has reached or exceeded the early warning threshold or trend. If the early warning threshold or trend is reached or exceeded, corresponding early warning measures are triggered, such as reminding the crew to rest, implementing stress management, and adjusting the work environment. By setting and applying early warning rules, real-time monitoring and early warning of the crew's work status can be achieved, problems can be identified in a timely manner, and intervention measures can be taken to ensure the crew's work efficiency and health.

[0101] The method for monitoring the working status of ship crew members provided in this application collects and analyzes ship environmental data, target crew members' physiological data and behavioral data in real time. By utilizing time-frequency analysis and fusion time-series modeling technology, a model that can accurately reflect changes in the working status of crew members is established. By adjusting the early warning rules, real-time monitoring and early warning of the working status of crew members are realized, providing strong support for the safe operation of ships and crew health management. By integrating time-series modeling and dynamically adjusting the early warning rules, the accuracy and practicality of the early warning are improved, providing a strong guarantee for the safe operation of ships.

[0102] In some embodiments, performing time-frequency analysis on the environmental data, the physiological data, and the behavioral data to obtain the time-frequency characteristics of the target crew member's state includes:

[0103] Based on the changing trends of the environmental data, physiological data, and behavioral data, the environmental data, physiological data, and behavioral data are respectively divided into multiple stage data, including stable stage data, fluctuating stage data, and abnormal stage data.

[0104] Time-frequency analysis is performed on the stage data to obtain the time-frequency domain representation of the stage data at multiple frequencies, so as to adjust the order of the numerical transformation of the time-frequency analysis according to the time-frequency domain representation;

[0105] Extract key local features and global time-frequency features from the time-frequency domain representation;

[0106] The key local features and global time-frequency features are fused to obtain the state time-frequency features.

[0107] Phase data refers to data segmented into specific time periods based on the trends and characteristics of environmental, physiological, and behavioral data during continuous monitoring. Each phase data represents the crew's performance at different times or under different working environments, providing a more detailed reflection of their work status and its changes. For example, a workday can be divided into different phases, such as morning, forenoon, afternoon, and evening, or phases can be defined based on the urgency and complexity of the work tasks.

[0108] Stable phase data refers to data that shows relatively stable changes and small fluctuations, indicating that the crew is in a normal working or resting state. During this period, the crew's physiological and behavioral data are relatively stable, without significant abnormal fluctuations. Fluctuating phase data refers to data that shows more drastic changes and larger fluctuations, indicating that the crew is in a state of fatigue, tension, or high stress. During this period, the crew's physiological and behavioral data will show significant fluctuations or abnormalities. Abnormal phase data refers to data that changes abnormally, exceeding the normal range or expectations, indicating that the crew is in an abnormal state or may have health problems, requiring immediate intervention.

[0109] Time-frequency domain representation jointly represents environmental, physiological, and behavioral data in terms of time and frequency, revealing the frequency components and variation characteristics of the data over different time periods. By performing time-frequency analysis on phased data, we can obtain the energy distribution of the data at different frequencies and its changes over time, thereby gaining a more accurate understanding of the data's patterns and characteristics.

[0110] By performing time-frequency analysis on phased data, time-frequency domain representations of each phase can be obtained at multiple frequencies. These representations reflect the components of the data at different frequencies and their changes over time, contributing to a deeper understanding of the physiological and behavioral characteristics of crew members under different conditions. During time-frequency analysis, the order of the time-frequency transformation can be dynamically adjusted based on the characteristics of the time-frequency domain representation to better capture the detailed features and trends of the data. For example, a lower order of transformation can be selected for stationary phase data to reduce computational load; a higher order of transformation can be selected for fluctuating and abnormal phase data to more accurately extract data features.

[0111] The order of a time-frequency transform refers to the order in which data is transformed during time-frequency analysis. Different orders affect the transformation results and the interpretation of the data. By adjusting the order of the time-frequency transform, the effectiveness of time-frequency analysis can be optimized, making the analysis results more accurate and reliable. For example, using a higher-order transform can better capture the nonlinear characteristics of the data, while using a lower-order transform may better highlight the main frequency components of the data.

[0112] Key local features refer to the changes and anomalies of data within a specific time period in the time-frequency domain representation, such as data peaks, valleys, and abrupt changes. Global time-frequency features refer to the overall performance of data in the time-frequency domain over the entire time range, such as main frequency components and energy distribution, which can reflect the overall working status and changing trends of crew members in different time periods.

[0113] In practice, a sliding window approach is used to analyze environmental, physiological, and behavioral data segment by segment. The statistical characteristics of each segment are used to determine its trend. Based on these trends, the environmental, physiological, and behavioral data are divided into stable, fluctuating, and abnormal phases to identify the data characteristics of crew members under different conditions. In the stable phase, the data is stable and reflects the crew member's physiological and behavioral state during normal work or rest. In the fluctuating phase, data fluctuations may indicate that the crew member is fatigued, stressed, or under high pressure. In the abnormal phase, abnormal data changes may indicate that the crew member is in an abnormal state or has health problems, requiring timely intervention.

[0114] STFT is used to analyze stationary or quasi-stationary signals, while Continuous Wavelet Transform (CWT) is used to analyze non-stationary signals. Based on the characteristics of the data and the analysis requirements, parameters for time-frequency analysis, such as window length, step size, and wavelet basis functions, are determined. During the analysis, the appropriateness of the current transform order is evaluated based on the results of the time-frequency domain representation. For example, if the time-frequency domain representation is too sparse or too dense, it may be necessary to adjust the transform order to obtain clearer and more representative time-frequency characteristics.

[0115] By gradually increasing or decreasing the order of the digital transform, the time-frequency analysis is repeated until a satisfactory time-frequency domain representation is obtained. For example, when analyzing motion signals in behavioral data, if the initial digital transform order is too low, causing the time-frequency domain representation to fail to accurately reflect the detailed features of the motion, the digital transform order can be appropriately increased to better capture the frequency components and temporal variations of the motion.

[0116] From the time-frequency domain representation, key features that reflect local changes in the data are identified and extracted as key local features. For example, in the time-frequency plot of heart rate data, feature frequencies, amplitude changes, and the time points of occurrence during heart rate acceleration and deceleration are extracted; in behavioral data, the start and end times of actions, as well as the corresponding action frequency and amplitude, are extracted.

[0117] Analyze the time-frequency domain representation of data throughout the entire period to extract global features that describe the overall characteristics of the data. For example, calculate the energy distribution, spectral center, and spectral entropy of each frequency component to reflect the overall trend and distribution of the data in the frequency domain; it can also calculate the time-frequency correlation coefficient of the time-frequency domain representation to describe the correlation of frequency components in different time periods.

[0118] By weighted fusion of key local features and global time-frequency features, a more comprehensive and accurate state time-frequency feature can be obtained. This fusion can comprehensively consider the local and global information of the data and more accurately reflect the working status and changes of crew members in different time periods and environments.

[0119] In this embodiment, by dividing environmental, physiological, and behavioral data into stages based on their changing trends, different characteristic stages of the data can be more accurately identified. This makes the subsequent analysis of data at each stage more targeted and effective, avoiding the problems of information overload and lack of feature prominence that may result from uniform processing of the overall data. By extracting key local features and global time-frequency features, useful information in the data can be mined from different levels. This approach not only focuses on the local details of the data but also grasps the overall characteristics of the data, providing rich feature information for a comprehensive and accurate portrayal of the crew's working status.

[0120] In some embodiments, the extraction of key local features and global time-frequency features from the time-frequency domain representation includes:

[0121] Based on the target crew's work rhythm, multiple time windows are set, and the window length and transformation scale of each time window are determined.

[0122] Based on the window length and the transform scale, the time-frequency domain representation is divided into low-order representation and high-order representation;

[0123] The frequency components, energy distribution, and time-frequency variation trends of the higher-order representation are extracted to obtain the key local features;

[0124] The overall energy level, spectral distribution, and time-frequency consistency are extracted from the low-order representation to obtain the global time-frequency features.

[0125] Work rhythm is the periodic activity pattern exhibited by crew members when performing work tasks. It is influenced by factors such as the needs of the work tasks, the ship's operational plans, and the crew members' biological clock. Different work rhythms will cause the physiological and behavioral data of crew members to show different patterns and characteristics over time.

[0126] Setting multiple time windows is to better capture the physiological and behavioral characteristics of crew members at different times. The length of the time window can be set according to the crew's work rhythm and needs to ensure that the data within each window can fully reflect the crew's state during that time period.

[0127] The transformation scale is the scale at which the time and frequency domains are transformed during time-frequency analysis, determining the level of precision with which the data is transformed. By adjusting the transformation scale, the components and characteristics of the data at different scales can be better captured.

[0128] Frequency components are the dominant frequencies of data in different time periods, reflecting the activity of crew members' physiological and behavioral data at different frequencies; energy distribution refers to the energy distribution of data at different frequencies, reflecting the intensity of crew members' physiological and behavioral activities at different frequencies; time-frequency variation trend refers to the variation trend of data in time and frequency, reflecting the changes of crew members' physiological and behavioral data over time.

[0129] Low-order representation is a coarser-grained representation of data during time-frequency analysis, mainly reflecting the overall energy distribution and spectral characteristics of the data.

[0130] Overall energy level measures the total energy of data across the entire time-frequency domain, reflecting the overall intensity of crew members' physiological and behavioral activities over a period of time. Spectral distribution characterizes the energy distribution of data across different frequency components, reflecting the characteristics of crew members' physiological and behavioral activities at different frequencies. Time-frequency consistency assesses the consistency and stability of data in time and frequency, reflecting the stability and continuity of the frequency components of crew members' physiological and behavioral data across different time periods. Higher time-frequency consistency indicates that the frequency components of the data are relatively stable across different time periods, without significant fluctuations or changes, while lower time-frequency consistency indicates that the frequency components of the data exhibit significant fluctuations or changes across different time periods.

[0131] Higher-order representations are representations of data at a finer granularity, capable of capturing more detailed features and trends in time and frequency.

[0132] In practice, in-depth analysis of the target crew members' work logs, task assignments, and other information is conducted to identify representative rhythmic segments in their work process. For regular, stable, long-cycle work rhythms, the window length can be set to a larger value matching the work cycle, such as 1-2 hours, to comprehensively capture data change trends under this work rhythm. For short-term, high-intensity work rhythms, such as 15 minutes of emergency equipment operation, the window length is set to a shorter value, such as 5-10 minutes, to allow for more detailed analysis of rapid data changes. Simultaneously, considering data update frequency and real-time requirements, the window length ensures that it includes a sufficient number of data samples for effective analysis while also reflecting changes in work status in a timely manner.

[0133] Choose an appropriate transform scale based on the window length and data characteristics. Taking wavelet transform as an example, for a longer window length, a larger-scale wavelet function can be selected to better capture the low-frequency components and overall trends of the data; for a shorter window length, a smaller-scale wavelet function can be selected to highlight the high-frequency components and local detail changes of the data.

[0134] Normalization is performed on the time-frequency domain representations obtained from environmental, physiological, and behavioral data to eliminate dimensional and numerical range differences between different data types, making the data comparable and processable. For example, the time-frequency domain representations of heart rate and ambient temperature are mapped to the interval [0, 1].

[0135] Based on the window length of each time window and the selected transform scale, the time-frequency domain representation is decomposed into low-order and high-order representations. The low-order representation corresponds to features at lower frequencies and larger scales, reflecting the overall trend and slow changes of the data; the high-order representation corresponds to features at higher frequencies and smaller scales, reflecting the local details and rapid changes of the data.

[0136] Spectral analysis is performed on the higher-order representation to calculate its power spectral density, identify the main frequency components, and identify the local variations with significant periodicity in the data. The energy distribution of the higher-order representation in different frequency bands is calculated, i.e., the energy proportion of each frequency band. The frequency band with higher energy indicates that the local variation in that frequency band is more intense or lasts longer. The trend of the higher-order representation in the time-frequency plane is analyzed, including the appearance, disappearance and intensity changes of frequency components over time, as key local features.

[0137] Calculate the total energy of the low-order representation in the entire time-frequency domain to reflect the overall activity of the data in that time period. Analyze the spectral distribution characteristics of the low-order representation to determine the frequency range and spectral shape of the main energy concentration. Evaluate the consistency of the low-order representation in the time-frequency domain, i.e., the similarity and stability of the spectral characteristics in different time periods, as global time-frequency features.

[0138] In the embodiments, the length of the time window, the scale of change, and the feature extraction method can be flexibly adjusted according to actual application needs and data characteristics to adapt to the needs of monitoring crew working status in different scenarios and improve the accuracy and practicality of monitoring.

[0139] In some embodiments, the key local features and global time-frequency features are fused to obtain the state time-frequency features, including:

[0140] Higher-order statistical analysis is performed on key local features within each time window to obtain higher-order statistics, which are used to describe the nonlinear relationship between the key local features.

[0141] The global time-frequency features are subjected to low-rank matrix decomposition to obtain a low-rank matrix and a sparse matrix;

[0142] Based on the low-rank matrix and the sparse matrix, the key components and noise components in the global time-frequency features are determined. The key components are used to characterize the differences in physiological and behavioral characteristics of the target crew members under multiple working conditions.

[0143] Based on the correlation and complementarity between the higher-order statistics, the key components, and the noise components, the key components of the global time-frequency features within each time window are weighted and fused to obtain the state time-frequency features.

[0144] Higher-order statistical analysis is used to analyze the nonlinear characteristics of data. By performing higher-order statistical analysis on key local features, higher-order statistics can be obtained. These statistics can reveal complex relationships and nonlinear dependencies between features. For example, by calculating statistics such as covariance, correlation coefficient, or higher-order cumulants between features, the interactions and mutual influences between features can be quantified, which helps to gain a deeper understanding of the intrinsic connections between the physiological and behavioral characteristics of crew members under different conditions.

[0145] Low-rank matrix factorization is used for data dimensionality reduction and feature extraction. By performing low-rank matrix factorization on global time-frequency features, low-rank matrices and sparse matrices can be obtained, which can effectively separate useful information and interference information in global time-frequency features, providing a foundation for subsequent feature fusion and state recognition.

[0146] The low-rank matrix includes the main structure and key components of the data, which are used to describe the main structural information and correlation patterns between the global time-frequency features. The sparse matrix contains noise and anomaly information in the data, which are used to describe the anomaly or mutation information in the global time-frequency features.

[0147] The key component is a representative part of the global time-frequency features, which can reflect the significant differences in the physiological and behavioral characteristics of crew members under different working conditions. It is used to identify the working conditions of crew members, assess their health status, and predict potential risks.

[0148] Noise components are interference information in global time-frequency features, representing abnormal or abrupt changes in the data. They may originate from factors such as measurement errors, environmental changes, or abnormal states of individual crew members. Noise components can mask the true characteristics of the data and affect the accuracy of state identification. Before feature fusion, noise components need to be effectively suppressed or removed to improve the accuracy and reliability of the data.

[0149] Differences in physiological and behavioral characteristics refer to the changes in the characteristics of crew members under different working conditions due to differences in physiological responses and behavioral patterns. These differences may be caused by a variety of factors, such as the nature of the work task, changes in environmental conditions, individual differences among crew members, and changes in psychological state.

[0150] Correlation refers to the association between different features, reflecting the interaction and mutual influence between features; complementarity refers to the different focuses and advantages of different features in describing the working status of crew members. By integrating them, they can complement each other and improve the comprehensiveness and accuracy of status identification.

[0151] In practice, the n key local features within each time window are combined into a feature vector X = [x1, x2, ..., xn]. n ] Calculate the third-order cumulant C3(k1,k2) of the eigenvector X as a higher-order statistic:

[0152] C3(k1,k2)=E[(X(t+k1)-μ)(X(t+k2)-μ)(X(t)-μ)]

[0153] Wherein, the third-order cumulant C3(k1,k2) represents the nonlinear relationship between the eigenvalue X(t) and its delayed version under time delays k1 and k2, E represents the expectation, μ is the mean of the eigenvector X μ=E(X(t)), k1 and k2 are delay parameters, representing the time difference between different time points in the time series, X(t+k1) is the value of the eigenvector X at time t+k1, X(t+k2) is the value of the eigenvector X at time t+k2, and X(t) is the value of the eigenvector X at time t.

[0154] By calculating the third-order cumulants under different combinations of delay parameters, a third-order cumulant matrix is ​​obtained, which reflects the nonlinear relationship between key local features.

[0155] The global time-frequency features are arranged in time series within the time window to form matrix A, where each row represents the global time-frequency features at a point in time, and each column represents a specific feature dimension, such as the overall energy level or spectral distribution.

[0156] Performing singular value decomposition (SVD) on matrix A yields A = UΣV T U and V are orthogonal matrices, V T Σ is the transpose of V, and Σ is a diagonal matrix where the diagonal elements are the singular values ​​of matrix A, representing the matrix's scale and information content.

[0157] Based on the magnitude of the singular values, select the left and right singular vectors corresponding to the larger singular values ​​to form a low-rank matrix ALow, and the remaining part forms a sparse matrix ASparse.

[0158] Key components are extracted from the low-rank matrix ALow. These components correspond to large singular values, representing the main structure in the global time-frequency features and characterizing the differences in physiological and behavioral features of crew members under different working conditions. For example, cluster analysis or expert knowledge can be used to determine which singular vectors correspond to components that are closely related to the crew members' working conditions.

[0159] The components in the sparse matrix ASparse correspond to small singular values ​​and mainly contain noise and other interference factors.

[0160] Calculate the correlation between higher-order statistics and key components, such as using the Pearson correlation coefficient, while assessing the complementarity of key and noise components to determine the weighting strategy for fusion.

[0161] Based on the results of correlation and complementarity analysis, weights are assigned to each key component. For example, key components with high correlation and high signal-to-noise ratio are given larger weights, while those with low correlation and low signal-to-noise ratio are given smaller weights.

[0162] The weighted key components are summed to obtain the fused state-time frequency feature vector S:

[0163] S = w1f1 + w2f2 + ... + w n f n

[0164] Among them, w i It is the weight of the i-th key component, f i It is the i-th key component, i = 1, 2, ..., n.

[0165] In this embodiment, during the weighted fusion process, the correlation and complementarity among higher-order statistics, key components, and noise components are considered, and different weights are assigned according to their importance. This ensures that the fused state-time frequency features can comprehensively and accurately reflect the working status and changes of crew members in different time periods and environments. The resulting state-time frequency features include both local detailed features of the data and reflect the overall trend of data changes, thus providing a more comprehensive reflection of the working status of crew members in different time periods and environments. These features can provide strong data support for subsequent status monitoring, health management, and task scheduling, helping to improve the work efficiency of crew members, protect their physical and mental health, and provide strong guarantees for the safe operation of ships.

[0166] In some embodiments, the step of fusing time-series modeling based on the state time-frequency features to obtain the working state model of the target crew member includes:

[0167] The state time-frequency features are subjected to univariate multi-step time series detection to generate multiple time series prediction sequences, which include a forward feature sequence and a reverse feature sequence.

[0168] Forward features are extracted from the forward feature sequence, and inverse features are extracted from the inverse feature sequence. The forward features and inverse features are then interactively integrated to obtain time-series features.

[0169] The working state model is constructed using the aforementioned temporal features.

[0170] Univariate multi-step time series detection is a time series analysis method used to capture the changing trend of state time-frequency features over time. By performing multi-step forward and backward predictions on the state time-frequency features, multiple time series prediction sequences can be generated, reflecting the possible states of the data at different time points.

[0171] Forward feature sequences are feature sequences predicted from the current time point forward, used to reveal the future development trend of data time series; backward feature sequences are feature sequences predicted from future time points backward, used to provide information on the historical evolution of data time series.

[0172] Extracting positive and negative features is the key information obtained from time-series prediction sequences. Positive features include the growth trend and periodic changes of the data, while negative features include the decay trend and historical peaks or troughs of the data.

[0173] By interactively integrating positive and negative features, more comprehensive and accurate time-series features can be obtained, which can more deeply reflect the changing patterns of crew members' working status.

[0174] The construction of a work status model based on time-series features is to simulate and predict the work status of crew members.

[0175] In actual implementation, assuming that each state's time-frequency feature is a separate time series, the sliding window technique is used to segment each feature, obtaining subsequences with multiple time steps. Each subsequence represents the change pattern of that feature over a period of time.

[0176] For each time-frequency feature, multiple time steps (such as short-term, medium-term, and long-term steps) are set to generate multiple time series sequences at different time steps. Each sequence corresponds to a different time scale, which helps to capture dynamic changes at different time granularities.

[0177] Feature extraction is performed on the positive feature sequence according to the time order from morning to night, including trends, periods, and volatility. For example, statistical methods such as mean, variance, maximum and minimum values, skewness, and kurtosis can be used to extract the basic features of the time series sequence, and Long Short-Term Memory Networks (LSTM) can be used to further extract the time series patterns.

[0178] Following the time sequence from late to early, similar feature extraction is performed on the reverse feature sequence to capture the changing trend from the later stage back to the earlier stage, as the reverse dynamics of the state.

[0179] By calculating the correlation or interaction terms between positive and negative features, these cross features are input as new features into the working state model.

[0180] The LSTM row model parameters were optimized using a labeled historical dataset. During training, methods such as cross-validation were employed to evaluate the model's generalization ability and avoid overfitting. The trained Long Short-Term Memory network was then used as the working model.

[0181] In this embodiment, by combining forward and reverse feature sequences, dynamic changes in the working state can be effectively captured, providing accurate monitoring and early warning for crew members or other working objects.

[0182] In some embodiments, constructing the working state model using the temporal features includes:

[0183] The temporal features are input into a temporal prediction network to obtain multiple predicted sequences output by the temporal prediction network.

[0184] Calculate the similarity and difference between each predicted sequence and the historical working status of the target crew member to obtain a similarity sequence and a difference sequence;

[0185] Based on the similarity sequence and the difference sequence, a work state transition probability matrix is ​​constructed, which is used to describe the transition probability of the target crew member between different work states.

[0186] The working state model is constructed based on the working state transition probability matrix.

[0187] The temporal prediction network comprises an input layer, at least one LSTM layer, a fully connected layer, and an output layer. The input layer receives temporal features, the LSTM layer extracts long-term dependencies from these features, the fully connected layer maps the LSTM output to the target dimension, and the output layer generates a prediction sequence. The network is trained using historical job state data, enabling it to predict future states based on the input temporal features. The prediction sequence represents job state predictions for multiple future time steps, simulating possible future job states; each prediction sequence represents a possible state transition path.

[0188] Similarity measures how closely a predicted sequence approaches the historical working conditions of the target crew member, reflecting the degree of agreement between the predicted and actual conditions. By calculating the similarity between the predicted sequence and historical conditions, the accuracy and reliability of the prediction can be assessed. Higher similarity indicates that the prediction is closer to the actual condition, and the prediction is more accurate.

[0189] The dissimilarity measure is used to measure the differences between the predicted sequence and historical states, revealing the differences and changes between the predicted and actual states. Calculating the dissimilarity helps identify errors and biases in the prediction, providing a basis for further analysis and improvement. A higher dissimilarity indicates a more significant difference between the predicted result and the actual state, potentially requiring adjustments and optimization of the prediction model.

[0190] Similarity sequence measures the feature matching degree or distance between the predicted sequence and the historical working status of the target crew member. A higher similarity indicates that the predicted status and the historical status have a high degree of consistency, which may reflect the degree of similarity between them and the stability and continuity of the crew member's working status.

[0191] The difference sequence reflects the degree of difference between the predicted sequence and the historical state. It is used to capture abnormal or abrupt information in state changes. A large difference may mean that the crew's working status has changed significantly, or that the prediction model has some bias on historical data.

[0192] The work state transition probability matrix describes the transition probability of a target crew member between different work states, that is, the likelihood of transitioning from one state to another. The calculation of the transition probability takes into account the similarity and differences between historical states and predicted states, which can more accurately reflect the dynamic changes in the crew member's work state and provide strong support for state monitoring and prediction.

[0193] In actual execution, the temporal features are organized into an input sequence in chronological order. Each input sequence contains temporal features at multiple time steps. The input sequence is then fed into a trained temporal prediction network to obtain multiple prediction sequences.

[0194] For each predicted sequence, its similarity to the target crew's historical work status sequence is calculated to obtain a similarity sequence. Euclidean distance is used to calculate the straight-line distance between vectors, cosine similarity is used to calculate the cosine of the angle between vectors, and Dynamic Time Warping (DTW) distance is used to calculate the similarity between non-linearly aligned time series. Taking Euclidean distance as an example, the square root of the sum of squares of the differences between corresponding elements in the predicted sequence and the historical work status sequence is calculated to obtain the similarity value for each time step, thus forming a similarity sequence.

[0195] A difference sequence is obtained by calculating the difference between the corresponding elements in the predicted sequence and the historical working state sequence.

[0196] Based on the target crew members' work characteristics and historical data, different work states are classified, such as normal work state, fatigue state, high-pressure state, and rest state. The classified work states are coded. For example, using integer coding, the normal work state is coded as 0, the fatigue state is coded as 1, the high-pressure state is coded as 2, and the rest state is coded as 3.

[0197] Based on similarity and difference sequences, the number of transitions between different work states of the target crew member is counted. For example, if the similarity is high and the difference is low, the crew member's work state is considered to remain unchanged; if the similarity is low and the difference is high, the crew member's work state is considered to have transitioned.

[0198] Calculate the working state transition probability matrix based on the statistical number of transitions. The rows of the matrix represent the current working state, the columns represent the next working state, and the element values ​​represent the probability of transitioning from the current state to the next state.

[0199] The work state transition probability matrix can be used as the transition matrix of a Markov model to construct a Markov model of the crew's work state.

[0200] Determine the initial state probability distribution and transition matrix of the Markov model. The initial state probability distribution can be determined based on the frequency of each state in the historical working state data, and the transition matrix is ​​the working state transition probability matrix calculated earlier.

[0201] The Markov model is trained using historical work status data to estimate model parameters. By considering the influence of the previous n states on the current state, the higher-order nature of state transitions is increased. Environmental features are then incorporated to optimize the model and improve the accuracy of predicting crew work status.

[0202] In this embodiment, temporal features are taken as input and processed by a temporal prediction network to obtain multiple prediction sequences representing the possible working states of the target crew members at different points in the future.

[0203] In some embodiments, the work status model is invoked to adjust the early warning rules, and the work status parameters are evaluated using the adjusted early warning rules to obtain the work status monitoring results of the target crew member, including:

[0204] Based on the correlation between the physiological data and the behavioral data and the work status, work status parameters are determined, including fatigue index, stress level and health status.

[0205] Random subspace identification is performed on the working state model to extract the dynamic features of the working state model;

[0206] Based on the aforementioned dynamic characteristics, the working state transition patterns of the target crew member are analyzed;

[0207] Based on the aforementioned working state transition pattern, adjust the threshold rule, trend rule, and correlation rule in the early warning rule;

[0208] The working status parameters are evaluated using the adjusted threshold rules, trend rules, and correlation rule warning rules to generate the working status monitoring results.

[0209] The correlation between physiological data and behavioral data and work status is determined by methods such as statistical analysis, machine learning algorithms, or expert scoring. The higher the correlation, the greater the impact of the physiological or behavioral data on work status, and the higher its weight should be in subsequent analysis.

[0210] Fatigue index, stress level, and health status are key work status parameters calculated based on physiological and behavioral data. The fatigue index is used to quantify the degree of fatigue of crew members, the stress level reflects the psychological stress state of crew members, and the health status comprehensively considers the physiological functions and overall health status of crew members.

[0211] The working state model contains multiple interrelated states and dynamic processes.

[0212] Random subspace identification is used to extract low-dimensional dynamic features from high-dimensional data and to extract dynamic features of complex systems. By performing random subspace decomposition on the working state model, key information and changing trends in the working state model can be analyzed, and feature vectors in multiple subspaces can be obtained to describe the dynamic changes of the working state model in different dimensions. By analyzing these feature vectors, dynamic features closely related to the working state transition can be extracted, such as the frequency, amplitude and stability of the state transition.

[0213] The working state transition pattern describes the probability and trend of the target crew member transitioning between different working states.

[0214] Based on the extracted dynamic features, we observe how the working status of crew members changes in different time periods, different task stages, or different environmental conditions, and what physiological or behavioral data these changes are associated with, thus obtaining the working status transformation patterns of the target crew members.

[0215] Early warning rules are criteria used to assess the working status of target crew members and trigger early warnings. These rules include threshold rules, trend rules, and correlation rules. Threshold rules are used to set critical values ​​for working status parameters. When a parameter value exceeds the critical value, an early warning is triggered. Trend rules are used to determine the changing trend of working status parameters, such as continuous increase or decrease. When the trend is abnormal, an early warning is triggered. Correlation rules are used to analyze the correlation between different working status parameters. When the correlation between parameters is abnormal, an early warning is triggered.

[0216] Based on the analysis of the work state transition patterns, more reasonable threshold ranges can be set according to the physiological and behavioral characteristics of crew members in different work states; or the sensitivity of trend rules can be adjusted according to the frequency and magnitude of state transitions; and the triggering conditions of correlation rules can be optimized according to the correlation between parameters, so as to adjust the thresholds, trends and correlation conditions in the early warning rules to improve the accuracy and timeliness of early warnings.

[0217] The revised early warning rules are better adapted to changes in the actual working conditions of crew members, can more accurately assess their working status parameters, and issue timely warnings, thus improving the practicality and reliability of the early warning system. By applying these rules, crew members' working status parameters can be monitored in real time, abnormal conditions can be detected promptly, and warnings can be triggered, providing a basis for subsequent intervention and management.

[0218] In some embodiments, after obtaining the working state model of the target crew member, the method further includes:

[0219] By using a risk situation awareness model, risk situation analysis is performed on the time-frequency characteristics of the state to identify multivariate risks, including fatigue trends, health trends, and behavioral trends.

[0220] A dynamic and continuous analysis is performed on the multivariate risk and the working status model to obtain the risk situation evolution trend of the target crew member;

[0221] Based on the evolving trend of the risk situation, the parameters of the working state model are adjusted.

[0222] Among them, the risk situation awareness model can comprehensively consider multiple state characteristics and their interrelationships. By analyzing various factors in the time-frequency characteristics of the state, such as physiological indicators, behavioral data and external environmental factors, it can capture factors that may act alone or in combination to affect the working state of crew members, and conduct quantitative analysis to identify potential risk factors.

[0223] In risk situation analysis, fatigue trends reflect the cumulative and changing levels of fatigue among crew members under prolonged working hours or specific conditions; health trends focus on the crew members' physical health, including disease risk and recovery capacity; and behavioral trends involve the crew members' performance at work, such as concentration and adherence to operational procedures. Analyzing these trends helps to identify potential problems in a timely manner and provides a basis for subsequent interventions.

[0224] Dynamic continuity analysis of multivariate risk and work status models takes into account the continuity and changing trends of multivariate risk and work status models over time. By jointly analyzing risks and models, the evolution trend of the risk situation of target crew members can be obtained. It not only considers the current risk status, but also focuses on the dynamic changes of risks and the interaction between these changes and the crew work status model. This allows for a deeper understanding of the evolution process of the risk situation, prediction of possible future risk trends, and timely detection of risks that crew members may face, such as overwork, deteriorating health, or undesirable behavioral tendencies.

[0225] Based on the evolving risk situation and historical and real-time data, optimization methods are employed to adjust the parameters of the work status model. This allows for better adaptation to changes in the actual working conditions of crew members and the evolving risk situation. Through continuous iteration and optimization, a more accurate and reliable work status model can be obtained, improving the accuracy and timeliness of early warnings. Such adjustments may involve multiple aspects, such as changing the threshold of the early warning rules or adjusting the weights of the state transition probability matrix. The specific adjustment method needs to be determined based on the results of the risk situation analysis.

[0226] In this embodiment, risk situation analysis is performed on the time-frequency characteristics of the status using a risk situation awareness model, and dynamic continuity analysis is conducted in conjunction with a work status model. This provides more comprehensive and accurate support for the monitoring and early warning of crew work status. This integrated analysis method helps to identify potential problems in a timely manner, providing strong support for subsequent intervention and management.

[0227] The method for monitoring the working status of ship crew members provided in this application can be executed by a ship crew working status monitoring device. This application uses the ship crew working status monitoring device executing the method as an example to illustrate the ship crew working status monitoring device provided in this application.

[0228] This application also provides a device for monitoring the working status of ship crew members.

[0229] like Figure 2 As shown, the ship's crew working status monitoring device includes:

[0230] The acquisition module 210 is used to acquire environmental data of the ship, as well as physiological and behavioral data of the target crew members in the ship;

[0231] The first processing module 220 is used to perform time-frequency analysis on the environmental data, the physiological data and the behavioral data to obtain the time-frequency characteristics of the target crew member's state.

[0232] The second processing module 230 is used to perform fusion time-series modeling based on the state time-frequency features to obtain the working state model of the target crew member. The working state model is used to describe the temporal changes and dynamic features of the working state of the target crew member. The dynamic features include the changing trends and fluctuations of the physiological and behavioral data of the target crew member in different time periods.

[0233] The third processing module 240 is used to call the work status model to adjust the early warning rules, evaluate the work status parameters through the adjusted early warning rules, and obtain the work status monitoring results of the target crew member. The early warning rules include threshold rules, trend rules and correlation rules, and the work status parameters are obtained based on the physiological data and the behavioral data.

[0234] The ship crew working status monitoring device provided in this application collects and analyzes ship environmental data, target crew physiological data, and behavioral data in real time. By utilizing time-frequency analysis and fusion time-series modeling technology, a model that can accurately reflect changes in crew working status is established. By adjusting the early warning rules, real-time monitoring and early warning of crew working status are achieved, providing strong support for the safe operation of ships and crew health management. By integrating time-series modeling and dynamically adjusting early warning rules, the accuracy and practicality of early warning are improved, providing strong protection for the safe operation of ships.

[0235] The shipboard crew working status monitoring device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal.

[0236] The shipboard crew work status monitoring device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0237] The ship crew working status monitoring device provided in this application embodiment can realize the various processes implemented in the ship crew working status monitoring method embodiment as described above. To avoid repetition, it will not be described again here.

[0238] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-described method embodiment for monitoring the working status of ship crew members and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0239] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0240] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described method for monitoring the working status of ship crew members and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0241] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0242] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-mentioned method for monitoring the working status of crew members on a ship.

[0243] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0244] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described ship crew working status monitoring method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0245] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0246] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0247] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the ship crew working status monitoring method of various embodiments of this application.

[0248] In the description of this application, "first feature" and "second feature" may include one or more of the features.

[0249] In the description of this application, "multiple" means two or more.

[0250] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0251] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0252] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of monitoring the working state of a crew member of a ship, characterized by, The method comprises the following steps: acquiring environmental data of a ship, and physiological data and behavior data of a target crew member in the ship; performing time-frequency analysis on the environmental data, the physiological data and the behavior data to obtain state time-frequency features of the target crew member; performing fusion time sequence modeling based on the state time-frequency features to obtain a working state model of the target crew member, the working state model being used to describe time sequence changes and dynamic characteristics of the working state of the target crew member, the dynamic characteristics including change trends and fluctuation conditions of the physiological data and the behavior data of the target crew member in different time periods; calling the working state model to adjust a pre-warning rule, and evaluating a working state parameter based on the adjusted pre-warning rule to obtain a working state monitoring result of the target crew member, the pre-warning rule including threshold rules, trend rules and correlation rules, and the working state parameter being obtained based on the physiological data and the behavior data.

2. The method of claim 1, wherein The time-frequency analysis on the environmental data, the physiological data and the behavior data to obtain the state time-frequency features of the target crew member comprises the following steps: dividing the environmental data, the physiological data and the behavior data into stage data including stationary stage data, fluctuation stage data and abnormal stage data based on change trends of the environmental data, the physiological data and the behavior data; performing time-frequency analysis on the stage data to obtain time-frequency domain representations of the stage data at multiple frequencies, so as to adjust the order of transformation of the time-frequency analysis according to the time-frequency domain representations; extracting key local features and global time-frequency features from the time-frequency domain representations; fusing the key local features and the global time-frequency features to obtain the state time-frequency features.

3. The method of claim 2, wherein The extraction of the key local features and the global time-frequency features from the time-frequency domain representations comprises the following steps: setting multiple time windows based on a working rhythm of the target crew member, and determining window lengths and transformation scales of each time window; dividing the time-frequency domain representations into low-order representations and high-order representations based on the window lengths and the transformation scales; extracting frequency components, energy distributions and time-frequency change trends from the high-order representations to obtain the key local features; extracting overall energy levels, frequency spectrum distributions and time-frequency consistencies from the low-order representations to obtain the global time-frequency features.

4. The method of claim 3, wherein The fusion of the key local features and the global time-frequency features to obtain the state time-frequency features comprises the following steps: performing high-order statistical analysis on the key local features in each time window to obtain high-order statistics, the high-order statistics being used to describe nonlinear relationships between the key local features; performing low-rank matrix decomposition on the global time-frequency features to obtain a low-rank matrix and a sparse matrix; determining key components and noise components in the global time-frequency features according to the low-rank matrix and the sparse matrix, the key components being used to represent physiological feature differences and behavior feature differences of the target crew member in multiple working states. The key components of the global time-frequency features in each time window are fused by weighting based on the correlation and complementarity between the high-order statistics, the key components and the noise components, to obtain the state time-frequency features.

5. The method of claim 1, wherein The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor.

6. The method of claim 5, wherein The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor.

7. The method of claim 1-6, wherein, The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor.

8. The method of claim 1-6, wherein, The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor.

9. A device for monitoring the working status of ship crew members, characterized in that, The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency features are fused to obtain a working state model of the target sailor. The state time-frequency The second processing module is configured to perform fusion time sequence modeling based on the state time-frequency features, and obtain a work state model of the target sailor, the work state model being configured to describe time sequence changes and dynamic characteristics of the work state of the target sailor, the dynamic characteristics including change trends and fluctuation conditions of the physiological data and the behavior data of the target sailor in different time periods. The third processing module is configured to call the work state model to adjust a pre-warning rule, evaluate work state parameters based on the adjusted pre-warning rule, and obtain a work state monitoring result of the target sailor, the pre-warning rule including a threshold rule, a trend rule, and a correlation rule, and the work state parameters being obtained based on the physiological data and the behavior data.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method for monitoring the work state of the sailor of the ship according to any one of claims 1-8.

Images