A diving safety early warning method and system based on multi-modal data fusion

Abstract

This invention discloses a diving safety early warning method and system based on multimodal data fusion. The method is as follows: if the diver's movement state is descending or stationary, first environmental data, physiological data, and movement posture data are input into a first LSTM model for risk assessment to obtain a hypothermia risk index; if the diver's movement state is ascending, first environmental data, second environmental data, physiological data, and movement posture data are input into a second LSTM model for risk assessment to obtain a hypothermia risk index and a decompression risk index; when the hypothermia risk index exceeds a first threshold or the decompression risk index exceeds a second threshold, a safety early warning signal is issued, and a safe return path is generated through an algorithm and a flexible motion-evaluation algorithm. Therefore, by implementing this invention, differentiated risks can be dynamically assessed for different diving stages by combining different multimodal data, and a safe return path can be dynamically generated based on the risk assessment results.

Description

Technical Field

[0001] This invention relates to the field of diving safety monitoring technology, and in particular to a diving safety early warning method and system based on multimodal data fusion. Background Technology

[0002] Cave and shipwreck diving, as high-risk underwater activities, are characterized by frequent and irregular ascents and descents within complex, enclosed spaces (such as traversing vertical fissures in caves or the multiple decks and compartments of shipwrecks). This dynamic state of motion makes the coupling between the underwater environment and the diver's physiological state more complex, and existing safety warning technologies are difficult to adapt to the special needs of such scenarios. During frequent ascents and descents, risk factors exhibit significant state dependence: during descent, divers must traverse areas with distinct water temperature stratification (such as the confluence of groundwater at different depths in caves or the temperature difference interface between shipwreck compartments and the external waters). The water temperature may drop suddenly within a few meters, and the increased water resistance during descent reduces the range of motion, further accelerating heat loss, making hypothermia a major threat. During ascent, in addition to the continuous risk of hypothermia caused by water temperature fluctuations, frequent depth changes can lead to a non-linear drop in water pressure (such as the rapid change in water pressure caused by the vertical drop between shipwreck compartments). If the ascent speed is not properly controlled, it can easily trigger acute decompression sickness and other risks. At this point, it is necessary to address the double risk simultaneously.

[0003] From a technical perspective, current solutions suffer from the following main shortcomings: First, most traditional monitoring systems employ static risk assessment models, failing to distinguish the different risk characteristics of the ascent and descent phases. Second, traditional dive computers primarily monitor environmental data such as depth and time, but do not incorporate physiological indicators and movement posture for comprehensive judgment, leading to delayed risk warnings. Finally, traditional path planning algorithms are ill-suited for dynamic environments like caves and shipwrecks, lacking the ability to generate intelligent emergency paths when divers simultaneously face hypothermia and decompression risks. Summary of the Invention

[0004] This invention provides a diving safety early warning method and system based on multimodal data fusion, which can dynamically assess differentiated risks for different diving stages by combining different multimodal data, and dynamically generate a safe return path based on the risk assessment results.

[0005] This invention provides a diving safety early warning method based on multimodal data fusion, comprising:

[0006] The system collects first environmental data, second environmental data, and the diver's physiological and motion posture data in real time. The first environmental data includes water temperature and current velocity, and the second environmental data includes water pressure and water depth. The physiological data includes heart rate, blood pressure, and blood oxygen saturation. The motion posture data includes triaxial acceleration.

[0007] If the diver's movement state is descending or stationary, the first environmental data, physiological data, and movement posture data are input into the pre-trained first LSTM model for risk assessment to obtain the hypothermia risk index; wherein, the first LSTM model calculates the attention weight for each time step based on the water temperature distribution gradient and the diver's twitching frequency.

[0008] If the diver's motion state is ascent, the first environmental data, the second environmental data, physiological data, and motion posture data are input into the pre-trained second LSTM model for risk assessment to obtain the hypothermia risk index and decompression risk index. The second LSTM model calculates the attention weight for each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate, and the diver's ascent speed.

[0009] When the temperature loss risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, a safety warning signal is issued and transmitted via A. Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path.

[0010] This invention provides a data foundation for subsequent assessments of hypothermia and decompression risks by real-time acquisition of environmental data, as well as the diver's physiological and movement posture data. By employing different multimodal data and risk assessment models tailored to different diver movement states, it enables precise early warnings specific to each diving phase. By introducing an LSTM model and combining it with an attention mechanism, it can dynamically capture the time dependence of key features such as water temperature gradient, convulsion frequency, water pressure change rate, and ascent speed. Furthermore, by combining A... The algorithm and flexible motion evaluation algorithm generate a safe return path, which can improve the dynamic adaptability of the safe return path planning and avoid increasing the risk of hypothermia or decompression for divers during the return. Compared with the existing technology that relies on single-dimensional data for safety warning and single risk identification, this application can dynamically assess differentiated risks for different diving stages by combining different multimodal data, and dynamically generate a safe return path based on the risk assessment results.

[0011] Furthermore, the real-time acquisition of first environmental data, second environmental data, and the diver's physiological data and motion posture data includes:

[0012] Based on PPS pulses, the acquisition clocks for the first environmental data, the second environmental data, physiological data, and motion posture data are synchronized.

[0013] Z-axis analysis was performed on the first environmental data, the second environmental data, the physiological data, and the motion posture data. Score standardization.

[0014] This invention, through the use of PPS pulses to achieve multi-source data clock synchronization, can avoid risk assessment delays caused by time deviations; through Z... Score standardization eliminates the dimensional differences between different sensors, providing a reliable foundation for subsequent model fusion analysis.

[0015] Furthermore, after the real-time acquisition of the first environmental data, the second environmental data, and the diver's physiological data and motion posture data, the method further includes:

[0016] The current motion state of the diver is determined based on the water depth and triaxial acceleration at a preset number of sampling points, specifically:

[0017] If the water depth at the preset number of sampling points continues to decrease and the vertical acceleration of the triaxial acceleration remains positive, then the current motion state of the diver is determined to be ascending.

[0018] If the water depth at the preset number of sampling points continues to rise and the vertical acceleration of the triaxial acceleration remains negative, then the current motion state of the diver is determined to be descent.

[0019] If the water depth at the preset number of sampling points remains constant and the vertical acceleration of the triaxial acceleration remains zero, then the current motion state of the diver is determined to be stationary.

[0020] The embodiments of the present invention determine the motion state by the change in water depth and the vertical component of triaxial acceleration, which can provide a basis for the dynamic switching of the risk assessment model.

[0021] Further, the step of inputting the first environmental data, physiological data, and motion posture data into a pre-trained first LSTM model for risk assessment to obtain a hypothermia risk index includes:

[0022] The first environmental data, physiological data, and motion posture data are weighted and fused into a temporal feature; wherein, the weights of the first environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0023] The hidden state of each time step of the temporal feature is extracted by a multi-layer LSTM, and the weight of each hidden state is calculated by an attention mechanism layer.

[0024] The hidden states are weighted and summed based on the weights to obtain the context vector;

[0025] A fully connected layer is used to map the context vector into a temperature loss risk index.

[0026] The embodiments of the present invention can enhance the impact of different types of data on hypothermia risk by weighting and fusing first environmental data, physiological data and motion posture data based on prior knowledge; by extracting temporal features through multi-layer LSTM and combining them with an attention mechanism, the accumulation of hypothermia risk can be captured, and the hypothermia risk can be accurately quantified.

[0027] Furthermore, the calculation of the weights of each hidden state through the attention mechanism layer includes:

[0028] Calculate the water temperature distribution gradient at the current time step based on the water temperature at the current time step, multiple water temperatures before the current time step, and the corresponding time intervals.

[0029] The twitching frequency of the diver at the current time step is calculated based on the total number of twitching events within a preset time window and the length of the time window; wherein, when the variance and peak frequency of the triaxial acceleration exceed the corresponding preset threshold, a twitching event is determined to exist.

[0030] The weights of the hidden states at the current time step are calculated based on the water temperature distribution gradient and the twitching frequency.

[0031] The embodiments of the present invention identify time steps that have a significant impact on hypothermia risk by using water temperature distribution gradient and convulsion frequency, and assign them higher weights, which can improve the pertinence and accuracy of hypothermia risk assessment.

[0032] Further, the step of inputting the first environmental data, the second environmental data, physiological data, and motion posture data into a pre-trained second LSTM model for risk assessment to obtain a hypothermia risk index and a decompression risk index includes:

[0033] The first environmental data, physiological data, and motion posture data are weighted and fused into a first temporal feature, and the second environmental data, physiological data, and motion posture data are weighted and fused into a second temporal feature; wherein, the weights of the first environmental data, second environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0034] The hidden states of the first and second time-series features at each time step are extracted by multi-layer LSTM, and the first weight of the hidden state of the first time-series feature at each time step is calculated by the first attention mechanism layer, and the second weight of the hidden state of the second time-series feature at each time step is calculated by the second attention mechanism layer.

[0035] Based on the first weight, the hidden states of the first time series feature at each time step are summed in a weighted manner to obtain a first context vector, and based on the second weight, the hidden states of the second time series feature at each time step are summed in a weighted manner to obtain a second context vector.

[0036] A first fully connected layer is used to map the first context vector to a temperature loss risk index, and a second fully connected layer is used to map the second context vector to a decompression risk index.

[0037] The embodiments of the present invention employ a dual-temporal feature separation processing strategy, which enables the parallel assessment of two types of risks—hysteresis and decompression—during the rising phase.

[0038] Further, the calculation of the second weight of the hidden state of the second temporal feature at each time step through the second attention mechanism layer includes:

[0039] Calculate the rate of change of water pressure at the current time step based on the water pressure at the current time step, multiple water pressures prior to the current time step, and the corresponding time intervals.

[0040] Calculate the diver's ascent rate at the current time step based on the water depth at the current time step, multiple water depths before the current time step, and the corresponding time intervals.

[0041] The second weight of the hidden state of the second time series feature at the current time step is calculated based on the water pressure change rate and the rise speed.

[0042] The embodiments of the present invention identify time steps that have a significant impact on decompression risk by using the rate of change of water pressure and the rate of rise, and assign them higher weights, which can improve the pertinence and accuracy of decompression risk assessment.

[0043] Furthermore, the above is achieved through A Algorithms and Flexible Motion Evaluation: The algorithm generates a safe return path, including:

[0044] Establish a three-dimensional terrain model that includes the water flow direction vector field, depth gradient field, and obstacle distance field;

[0045] Based on the three-dimensional terrain model, node expansion is performed according to the preset step size and movement cost;

[0046] The safe return route is obtained based on the cost function and B-spline curve interpolation; wherein, the cost function includes the actual oxygen consumption cost and the heuristically estimated route cost.

[0047] The embodiments of the present invention are implemented through A The algorithm combines factors such as 3D terrain models and oxygen consumption to generate a global path for safe return in dynamic environments.

[0048] Furthermore, the above is achieved through A Algorithms and Flexible Motion Evaluation Algorithms generate safe return paths, and also include:

[0049] The system acquires the hypothermia risk index and decompression risk index of divers in real time when they return to base according to the safe return route, and calculates the increments of the hypothermia risk index and decompression risk index.

[0050] When the increase in the hypothermia risk index exceeds a preset third threshold or the increase in the decompression risk index exceeds a preset fourth threshold, path optimization is performed using a flexible action-evaluation algorithm, specifically:

[0051] The safe return path is optimized based on the diver's state space and motion space, as well as a preset reward function; wherein the reward function includes oxygen consumption efficiency reward, time cost reward, hypothermia risk penalty, and decompression risk penalty.

[0052] This invention provides data support for determining whether a safe return path needs optimization by calculating the increments of the hypothermia risk index and the decompression risk index. By considering the hypothermia and decompression risks of divers during the return journey and using a flexible action-evaluation algorithm for local path optimization, it can prevent divers from experiencing hypothermia and decompression during the return journey.

[0053] Another embodiment of the present invention provides a diving safety early warning system based on multimodal data fusion, including: a data acquisition module, a first risk assessment module, a second risk assessment module, and a return path module;

[0054] The data acquisition module is used to collect first environmental data, second environmental data, and the diver's physiological and motion posture data in real time; wherein, the first environmental data includes water temperature and water flow velocity, the second environmental data includes water pressure and water depth; the physiological data includes heart rate, blood pressure, and blood oxygen saturation; and the motion posture data includes triaxial acceleration.

[0055] The first risk assessment module is used to input the first environmental data, physiological data and motion posture data into the pre-trained first LSTM model to conduct risk assessment and obtain the hypothermia risk index if the diver's motion state is diving or stationary. The first LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient and the diver's convulsion frequency.

[0056] The second risk assessment module is used to input the first environmental data, the second environmental data, the physiological data, and the motion posture data into the pre-trained second LSTM model to perform risk assessment if the diver's motion state is ascending, and to obtain the hypothermia risk index and the decompression risk index; wherein, the second LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate, and the diver's ascent speed.

[0057] The return path module is used to issue a safety warning signal and, when the hypothermia risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, via A... Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating an embodiment of the diving safety early warning method based on multimodal data fusion provided by the present invention.

[0059] Figure 2 This is a schematic diagram of an embodiment of the diving safety early warning system based on multimodal data fusion provided by the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0062] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0065] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0066] See Figure 1 To address the shortcomings of existing technologies that rely on single-dimensional data for safety early warning and have limited risk identification capabilities, an embodiment of the present invention provides a diving safety early warning method based on multimodal data fusion, comprising steps S101 to S104:

[0067] Step S101: Real-time acquisition of first environmental data, second environmental data, and the diver's physiological data and motion posture data; wherein, the first environmental data includes water temperature and water flow velocity, the second environmental data includes water pressure and water depth; the physiological data includes heart rate, blood pressure, and blood oxygen saturation; and the motion posture data includes triaxial acceleration.

[0068] Specifically, water temperature is a core factor influencing hypothermia risk, especially in environments with temperature stratification such as caves and shipwrecks, where water temperature can drop suddenly by 5-10°C within a few meters, directly accelerating heat loss for divers. Water temperature can be collected in real time using thermocouple sensors, which can be encapsulated in waterproof probes and attached to the outside of the diving suit (close to the body surface), monitoring both ambient water temperature and body surface temperature differences. Water flow velocity is related to the diver's energy expenditure and movement trajectory stability, and is an indirect factor influencing hypothermia risk. Water flow velocity can be collected in real time using a miniature Doppler current meter, which can be installed at the front of the diving equipment to output the direction and velocity vector of the water flow in real time. Water pressure and water depth are directly related and are core indicators of decompression risk: a rapid ascent causes a sudden drop in water pressure, which can cause nitrogen in the blood to form bubbles, leading to decompression sickness. Water pressure can be collected in real time by integrating a pressure sensor into the diving equipment, and water depth data can be output simultaneously using a water pressure-depth conversion formula. Heart rate reflects a diver's stress state (heart rate may rise or fall sharply under the risk of hypothermia or decompression), blood pressure changes indicate circulatory system stability, and blood oxygen saturation reflects oxygen supply status; heart rate and blood oxygen can be collected in real time using a wrist-worn photoelectric sensor, while blood pressure can be collected in real time using a non-invasive wrist monitoring module. Triaxial acceleration is used to capture the diver's motion characteristics and can be collected in real time using an inertial sensor integrated into the dive computer.

[0069] In the real-time acquisition of multimodal data through various sensors, the acquisition clocks of each sensor can be synchronized based on the PPS (Pulse Per Second) pulse. That is, when each sensor detects the rising edge of the PPS pulse, it immediately adds a unified timestamp to the currently acquired data. At the same time, the multimodal data is subjected to Z-score normalization to eliminate the dimensional differences between different sensors.

[0070] The process, following the real-time acquisition of first and second environmental data, as well as the diver's physiological and motion posture data, further includes determining the diver's current motion state based on the water depth and triaxial acceleration at a preset number of sampling points. If the water depth continuously decreases for N consecutive sampling points, and the vertical acceleration in the triaxial acceleration remains positive, the diver is determined to be in an ascending state; if the water depth continuously increases for N consecutive sampling points, and the vertical acceleration in the triaxial acceleration remains negative, the diver is determined to be in a descending state; if the water depth remains unchanged for N consecutive sampling points, and the vertical acceleration in the triaxial acceleration remains zero, the diver is determined to be in a stationary state. When discrepancies arise in the sampling point data (e.g., water depth decreases but vertical acceleration is negative), the sampling window can be extended until N consecutive points show a consistent trend, avoiding frequent switching of state labels in complex water flow environments and ensuring the stability of the subsequent LSTM (Long Short-Term Memory Network) model input.

[0071] Step S102: If the diver's motion state is diving or stationary, the first environmental data, physiological data and motion posture data are input into the pre-trained first LSTM model for risk assessment to obtain the hypothermia risk index; wherein, the first LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient and the diver's twitching frequency.

[0072] Specifically, when a diver is submerged or stationary, hypothermia risk is the core safety threat at this stage. A pre-trained first LSTM model dynamically quantifies hypothermia risk by fusing multimodal data, extracting temporal features, and incorporating an attention mechanism. The input data for the first LSTM model includes environmental data, physiological data, and motion posture data. These three types of data need to be weighted and fused based on prior knowledge of hypothermia (e.g., water temperature has the greatest direct impact on hypothermia and is given the highest weight) to form temporal features. Each time step of these temporal features corresponds to a set of fused multimodal data. These temporal features are then input into a multi-layer LSTM to capture long-term dependencies in the multimodal data and extract the hidden states at each time step. These hidden states are a comprehensive representation of the current time step data and historical data, reflecting the cumulative trend of hypothermia risk. To accurately identify the time step that contributes the most to hypothermia risk, the first LSTM model calculates the weights of each hidden state through an attention mechanism layer. The core calculation basis for these weights is the water temperature distribution gradient and the diver's convulsion frequency. The water temperature distribution gradient reflects the magnitude of water temperature change per unit time and is a key indicator for determining the rate of increase in hypothermia risk. It can be calculated based on the water temperature at the current time step, the water temperature at multiple time steps prior, and the corresponding time intervals. Twitching is a typical physiological response to hypothermia, and the twitching frequency reflects the severity of hypothermia. When the variance and peak frequency of the triaxial acceleration exceed a preset threshold, a twitching event is determined to exist at that time step. The twitching frequency can be calculated based on the total number of twitching events within a preset time window and the length of the time window. The attention mechanism layer assigns weights to the hidden states at each time step based on the water temperature distribution gradient and twitching frequency. For example, the softmax function is used to transform the joint influence of the water temperature distribution gradient and twitching frequency into weight values ​​between 0 and 1. The hidden states at all time steps are weighted and summed according to the attention weights to obtain a context vector. This context vector is then input into a fully connected layer and transformed into a hypothermia risk index through nonlinear mapping.

[0073] Step S103: If the diver's motion state is ascending, the first environmental data, the second environmental data, the physiological data, and the motion posture data are input into the pre-trained second LSTM model for risk assessment to obtain the hypothermia risk index and the decompression risk index; wherein, the second LSTM model calculates the attention weight for each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate, and the diver's ascent speed.

[0074] Specifically, when divers are ascending, they need to simultaneously address the risks of hypothermia and decompression. A pre-trained second LSTM model uses a dual-branch temporal feature processing and a multi-dimensional attention mechanism to achieve parallel and accurate assessment of both types of risks. The input data for the second LSTM model includes: first environmental data, second environmental data, physiological data, and motion posture data. The first environmental data, physiological data, and motion posture data are weighted and fused based on hypothermia prior knowledge to form a first temporal feature used to assess hypothermia risk. The second environmental data, physiological data, and motion posture data are weighted and fused based on decompression prior knowledge to form a second temporal feature used to assess decompression risk. The first and second temporal features are input into independent multi-layer LSTMs to extract their respective time-step hidden states. To accurately identify the key time steps that contribute most to the two types of risks, the second LSTM model includes two attention mechanism layers, each calculating weights based on different indicators. The first attention mechanism layer is used for hypothermia risk weighting, and its logic is consistent with the attention mechanism layer of the first LSTM model, assigning weights to the hidden states of each time step of the first temporal feature based on the water temperature distribution gradient and twitching frequency. The second attention mechanism layer is used for decompression risk weighting. The core calculation basis for the decompression risk weighting is the rate of change of water pressure and the diver's ascent speed. The rate of change of water pressure reflects the severity of the water pressure drop and can be calculated based on the water pressure at the current time step, the water pressure at multiple time steps ago, and the corresponding time intervals. The ascent speed can be calculated based on the water depth at the current time step, the water depth at multiple time steps ago, and the corresponding time intervals. The hidden states of the first and second temporal features are weighted and summed according to their respective attention weights to obtain the first and second context vectors. The first and second context vectors are then input into the first and second fully connected layers for mapping to obtain the hypothermia risk index and the decompression risk index.

[0075] Step S104: When the temperature loss risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, a safety warning signal is issued and transmitted via A. Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path.

[0076] Specifically, A The algorithm is a heuristic pathfinding algorithm that can be used to construct a global path from the current location to the surface safety exit. To accurately simulate the constraints of the underwater environment on the path, A... The algorithm needs to construct a three-dimensional terrain model including a water flow direction vector field, a depth gradient field, and an obstacle distance field. Specifically, water flow velocity and direction at different underwater spatial locations can be collected in real time using water flow sensors to form discrete water flow observation point data. A spatial interpolation algorithm is then used to fit these discrete observation points, generating a continuous water flow direction vector field. Based on depth data, underwater depth distribution data can be generated through spatial gridding, and the depth difference between each grid cell and its adjacent cells can be calculated to obtain the depth change rate, forming a depth gradient field. The location and contour data of obstacles within the diving area can be obtained using underwater sonar, imaging sensors, or a pre-set underwater terrain database. The straight-line distance from any underwater location to the nearest obstacle is calculated, centered on the obstacle, forming an obstacle distance field. Using the diver's current position as the initial node and the return target position as the target node, the search space is divided based on the spatial resolution of the three-dimensional terrain model. A fixed spatial expansion step size is set, i.e., each time the candidate node is expanded from the current node to the adjacent spatial direction by this step size distance. The generation of candidate nodes requires calculation of the movement cost using the three-dimensional terrain model; only nodes with costs below a threshold are retained for the next round of expansion. The algorithm's cost function includes the actual oxygen consumption cost and the heuristically estimated path cost. The actual oxygen consumption cost can be calculated based on water flow resistance and depth change intensity, while the heuristically estimated path cost can be obtained by multiplying the straight-line distance from the current node to the target node by the average oxygen consumption per unit distance. (After A...) The optimal path obtained by the algorithm consists of discrete nodes, which can lead to problems such as polylines and numerous corners, potentially increasing the diver's range of motion and oxygen consumption. To address this, B-spline curve interpolation can be used to smooth the discrete nodes. For example, a continuous three-dimensional curve can be fitted based on the node coordinates to keep the path curvature within a safe range.

[0077] Since the underwater environment and diver's condition are dynamic, the initial path may no longer be optimal as risks change. A flexible action-evaluation algorithm can be used to adjust the path in real time to ensure it always adapts to the current risk state. The hypothermia risk index and decompression risk index are collected in real time during the diver's return journey, and their increments are calculated. If the hypothermia risk index increment exceeds a preset third threshold or the decompression risk index increment exceeds a preset fourth threshold, path optimization is triggered. First, the diver's state space and action space are determined. The state space may include spatial location information, physiological and risk state, and motion and environmental state; the action space may include movement direction adjustment, movement speed adjustment, and attitude correction actions. The reward function value of the current path is evaluated based on the state space. The reward function consists of oxygen consumption efficiency reward, time cost reward, hypothermia risk penalty, and decompression risk penalty. Available actions are traversed in the action space to generate candidate path segments, and the reward function value of each segment is calculated. The path segment corresponding to the action with the highest reward value is selected, the current state space is updated, and this iteration is repeated until the target point is reached.

[0078] This invention provides a data foundation for subsequent assessments of hypothermia and decompression risks by real-time acquisition of environmental data, as well as the diver's physiological and movement posture data. By employing different multimodal data and risk assessment models tailored to different diver movement states, it enables precise early warnings specific to each diving phase. By introducing an LSTM model and combining it with an attention mechanism, it can dynamically capture the time dependence of key features such as water temperature gradient, convulsion frequency, water pressure change rate, and ascent speed. Furthermore, by combining A... Algorithms and Flexible Motion Evaluation Algorithms generate safe return paths, which can improve the dynamic adaptability of safe return path planning and avoid increasing the risk of hypothermia or decompression for divers during the return journey.

[0079] Optionally, in this embodiment of the invention, the real-time acquisition of first environmental data, second environmental data, and the diver's physiological data and motion posture data includes:

[0080] Based on PPS pulses, the acquisition clocks for the first environmental data, the second environmental data, physiological data, and motion posture data are synchronized.

[0081] Z-axis analysis was performed on the first environmental data, the second environmental data, the physiological data, and the motion posture data. Score standardization.

[0082] This invention, through the use of PPS pulses to achieve multi-source data clock synchronization, can avoid risk assessment delays caused by time deviations; through Z... Score standardization eliminates the dimensional differences between different sensors, providing a reliable foundation for subsequent model fusion analysis.

[0083] Optionally, in this embodiment of the invention, after the real-time acquisition of the first environmental data, the second environmental data, and the diver's physiological data and motion posture data, the method further includes:

[0084] The current motion state of the diver is determined based on the water depth and triaxial acceleration at a preset number of sampling points, specifically:

[0085] If the water depth at the preset number of sampling points continues to decrease and the vertical acceleration of the triaxial acceleration remains positive, then the current motion state of the diver is determined to be ascending.

[0086] If the water depth at the preset number of sampling points continues to rise and the vertical acceleration of the triaxial acceleration remains negative, then the current motion state of the diver is determined to be descent.

[0087] If the water depth at the preset number of sampling points remains constant and the vertical acceleration of the triaxial acceleration remains zero, then the current motion state of the diver is determined to be stationary.

[0088] The embodiments of the present invention determine the motion state by the change in water depth and the vertical component of triaxial acceleration, which can provide a basis for the dynamic switching of the risk assessment model.

[0089] Optionally, in this embodiment of the invention, the step of inputting the first environmental data, physiological data, and motion posture data into a pre-trained first LSTM model for risk assessment to obtain a hypothermia risk index includes:

[0090] The first environmental data, physiological data, and motion posture data are weighted and fused into a temporal feature; wherein, the weights of the first environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0091] The hidden state of each time step of the temporal feature is extracted by a multi-layer LSTM, and the weight of each hidden state is calculated by an attention mechanism layer.

[0092] The hidden states are weighted and summed based on the weights to obtain the context vector;

[0093] A fully connected layer is used to map the context vector into a temperature loss risk index.

[0094] The embodiments of the present invention can enhance the impact of different types of data on hypothermia risk by weighting and fusing first environmental data, physiological data and motion posture data based on prior knowledge; by extracting temporal features through multi-layer LSTM and combining them with an attention mechanism, the accumulation of hypothermia risk can be captured, and the hypothermia risk can be accurately quantified.

[0095] Optionally, in this embodiment of the invention, calculating the weights of each hidden state through the attention mechanism layer includes:

[0096] Calculate the water temperature distribution gradient at the current time step based on the water temperature at the current time step, the water temperature at multiple time steps ago, and the corresponding time intervals.

[0097] The twitching frequency of the diver at the current time step is calculated based on the total number of twitching events within a preset time window and the length of the time window; wherein, when the variance and peak frequency of the triaxial acceleration exceed the corresponding preset threshold, a twitching event is determined to exist.

[0098] The weights of the hidden states at the current time step are calculated based on the water temperature distribution gradient and the twitching frequency.

[0099] The embodiments of the present invention identify time steps that have a significant impact on hypothermia risk by using water temperature distribution gradient and convulsion frequency, and assign them higher weights, which can improve the pertinence and accuracy of hypothermia risk assessment.

[0100] Optionally, in this embodiment of the invention, the step of inputting the first environmental data, the second environmental data, physiological data, and motion posture data into a pre-trained second LSTM model for risk assessment to obtain a hypothermia risk index and a decompression risk index includes:

[0101] The first environmental data, physiological data, and motion posture data are weighted and fused into a first temporal feature, and the second environmental data, physiological data, and motion posture data are weighted and fused into a second temporal feature; wherein, the weights of the first environmental data, second environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0102] The hidden states of the first and second time-series features at each time step are extracted by multi-layer LSTM, and the first weight of the hidden state of the first time-series feature at each time step is calculated by the first attention mechanism layer, and the second weight of the hidden state of the second time-series feature at each time step is calculated by the second attention mechanism layer.

[0103] Based on the first weight, the hidden states of the first time series feature at each time step are summed in a weighted manner to obtain a first context vector, and based on the second weight, the hidden states of the second time series feature at each time step are summed in a weighted manner to obtain a second context vector.

[0104] A first fully connected layer is used to map the first context vector to a temperature loss risk index, and a second fully connected layer is used to map the second context vector to a decompression risk index.

[0105] The embodiments of the present invention employ a dual-temporal feature separation processing strategy, which enables the parallel assessment of two types of risks—hysteresis and decompression—during the rising phase.

[0106] Optionally, in this embodiment of the invention, the step of calculating the second weight of the hidden state of the second temporal feature at each time step through the second attention mechanism layer includes:

[0107] Calculate the rate of change of water pressure at the current time step based on the water pressure at the current time step, the water pressure at multiple time steps ago, and the corresponding time intervals.

[0108] Calculate the diver's ascent rate at the current time step based on the water depth at the current time step, the water depth at multiple time steps ago, and the corresponding time intervals.

[0109] The second weight of the hidden state of the second time series feature at the current time step is calculated based on the water pressure change rate and the rise speed.

[0110] The embodiments of the present invention identify time steps that have a significant impact on decompression risk by using the rate of change of water pressure and the rate of rise, and assign them higher weights, which can improve the pertinence and accuracy of decompression risk assessment.

[0111] Optionally, in an embodiment of the present invention, the step of using A... Algorithms and Flexible Motion Evaluation: The algorithm generates a safe return path, including:

[0112] Establish a three-dimensional terrain model that includes the water flow direction vector field, depth gradient field, and obstacle distance field;

[0113] Based on the three-dimensional terrain model, node expansion is performed according to the preset step size and movement cost;

[0114] The safe return route is obtained based on the cost function and B-spline curve interpolation; wherein, the cost function includes the actual oxygen consumption cost and the heuristically estimated route cost.

[0115] The embodiments of the present invention are implemented through A The algorithm combines factors such as 3D terrain models and oxygen consumption to generate a global path for safe return in dynamic environments.

[0116] Optionally, in an embodiment of the present invention, the step of using A... Algorithms and Flexible Motion Evaluation Algorithms generate safe return paths, and also include:

[0117] The system acquires the hypothermia risk index and decompression risk index of divers in real time when they return to base according to the safe return route, and calculates the increments of the hypothermia risk index and decompression risk index.

[0118] When the increase in the hypothermia risk index exceeds a preset third threshold or the increase in the decompression risk index exceeds a preset fourth threshold, path optimization is performed using a flexible action-evaluation algorithm, specifically:

[0119] The safe return path is optimized based on the diver's state space and motion space, as well as a preset reward function; wherein the reward function includes oxygen consumption efficiency reward, time cost reward, hypothermia risk penalty, and decompression risk penalty.

[0120] This invention provides data support for determining whether a safe return path needs optimization by calculating the increments of the hypothermia risk index and the decompression risk index. By considering the hypothermia and decompression risks of divers during the return journey and using a flexible action-evaluation algorithm for local path optimization, it can prevent divers from experiencing hypothermia and decompression during the return journey.

[0121] like Figure 2 As shown, based on the above method embodiments, corresponding system embodiments are provided;

[0122] An embodiment of the present invention provides a diving safety early warning system based on multimodal data fusion, comprising: a data acquisition module 201, a first risk assessment module 202, a second risk assessment module 203, and a return path module 204;

[0123] The data acquisition module 201 is used to collect first environmental data, second environmental data, and the diver's physiological data and motion posture data in real time; wherein, the first environmental data includes water temperature and water flow velocity, the second environmental data includes water pressure and water depth; the physiological data includes heart rate, blood pressure, and blood oxygen saturation; and the motion posture data includes triaxial acceleration.

[0124] The first risk assessment module 202 is used to input the first environmental data, physiological data and motion posture data into the pre-trained first LSTM model to conduct risk assessment if the diver's motion state is diving or stationary, and obtain the hypothermia risk index; wherein, the first LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient and the diver's twitching frequency.

[0125] The second risk assessment module 203 is used to input the first environmental data, the second environmental data, the physiological data and the motion posture data into the pre-trained second LSTM model to conduct risk assessment if the diver's motion state is ascending, and obtain the hypothermia risk index and the decompression risk index; wherein, the second LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate and the diver's ascent speed.

[0126] The return path module 204 is used to issue a safety warning signal and, when the hypothermia risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, and via A... Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path.

[0127] Optionally, in this embodiment of the invention, the data acquisition module 201 includes: a clock synchronization submodule and a standardization submodule;

[0128] The clock synchronization submodule is used to synchronize the acquisition clocks of the first environmental data, the second environmental data, the physiological data, and the motion posture data based on the PPS pulse.

[0129] The standardization submodule is used to perform Z-factor analysis on the first environmental data, the second environmental data, the physiological data, and the motion posture data. Score standardization.

[0130] This invention, through the use of PPS pulses to achieve multi-source data clock synchronization, can avoid risk assessment delays caused by time deviations; through Z... Score standardization eliminates the dimensional differences between different sensors, providing a reliable foundation for subsequent model fusion analysis.

[0131] Optionally, in this embodiment of the invention, a status determination submodule is further included after the data acquisition module 201;

[0132] The state determination submodule is used to determine the current motion state of the diver based on the water depth and triaxial acceleration of a preset number of sampling points, specifically:

[0133] If the water depth at the preset number of sampling points continues to decrease and the vertical acceleration of the triaxial acceleration remains positive, then the current motion state of the diver is determined to be ascending.

[0134] If the water depth at the preset number of sampling points continues to rise and the vertical acceleration of the triaxial acceleration remains negative, then the current motion state of the diver is determined to be descent.

[0135] If the water depth at the preset number of sampling points remains constant and the vertical acceleration of the triaxial acceleration remains zero, then the current motion state of the diver is determined to be stationary.

[0136] The embodiments of the present invention determine the motion state by the change in water depth and the vertical component of triaxial acceleration, which can provide a basis for the dynamic switching of the risk assessment model.

[0137] Optionally, in this embodiment of the invention, the first risk assessment module 202 includes: a first data fusion submodule, a first hidden state submodule, a first feature weighting submodule, and a first feature mapping submodule;

[0138] The first data fusion submodule is used to weightedly fuse the first environmental data, physiological data, and motion posture data into temporal features; wherein, the weights of the first environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0139] The first hidden state submodule is used to extract the hidden state of the temporal feature at each time step through a multi-layer LSTM, and to calculate the weight of each hidden state through an attention mechanism layer.

[0140] The first feature weighting submodule is used to perform a weighted summation of each of the hidden states based on the weights to obtain a context vector;

[0141] The first feature mapping submodule is used to map the context vector into a temperature loss risk index using a fully connected layer.

[0142] The embodiments of the present invention can enhance the impact of different types of data on hypothermia risk by weighting and fusing first environmental data, physiological data and motion posture data based on prior knowledge; by extracting temporal features through multi-layer LSTM and combining them with an attention mechanism, the accumulation of hypothermia risk can be captured, and the hypothermia risk can be accurately quantified.

[0143] Optionally, in this embodiment of the invention, the first hidden state submodule includes: a water temperature distribution gradient unit, a twitching frequency unit, and a first weight calculation unit;

[0144] The water temperature distribution gradient unit is used to calculate the water temperature distribution gradient at the current time step based on the water temperature at the current time step, the water temperature at multiple time steps ago, and the corresponding time interval.

[0145] The twitching frequency unit is used to calculate the twitching frequency of the diver at the current time step based on the total number of twitching events within a preset time window and the length of the time window; wherein, when the variance and peak frequency of the triaxial acceleration exceed the corresponding preset threshold, it is determined that a twitching event exists.

[0146] The first weight calculation unit is used to calculate the weight of the hidden state at the current time step based on the water temperature distribution gradient and the twitching frequency.

[0147] The embodiments of the present invention identify time steps that have a significant impact on hypothermia risk by using water temperature distribution gradient and convulsion frequency, and assign them higher weights, which can improve the pertinence and accuracy of hypothermia risk assessment.

[0148] Optionally, in this embodiment of the invention, the second risk assessment module 203 includes: a second data fusion submodule, a second hidden state submodule, a second feature weighting submodule, and a second feature mapping submodule;

[0149] The second data fusion submodule is used to weightedly fuse the first environmental data, physiological data, and motion posture data into a first temporal feature, and to weightedly fuse the second environmental data, physiological data, and motion posture data into a second temporal feature; wherein, the weights of the first environmental data, second environmental data, physiological data, and motion posture data are obtained based on prior knowledge;

[0150] The second hidden state submodule is used to extract the hidden state of the first temporal feature and the second temporal feature at each time step through a multi-layer LSTM, calculate the first weight of the hidden state of the first temporal feature at each time step through a first attention mechanism layer, and calculate the second weight of the hidden state of the second temporal feature at each time step through a second attention mechanism layer.

[0151] The second feature weighting submodule is used to perform weighted summation of the hidden states of the first time series feature at each time step based on the first weight to obtain a first context vector, and to perform weighted summation of the hidden states of the second time series feature at each time step based on the second weight to obtain a second context vector;

[0152] The second feature mapping submodule is used to map the first context vector to a heat loss risk index using a first fully connected layer, and to map the second context vector to a decompression risk index using a second fully connected layer.

[0153] The embodiments of the present invention employ a dual-temporal feature separation processing strategy, which enables the parallel assessment of two types of risks—hysteresis and decompression—during the rising phase.

[0154] Optionally, in this embodiment of the invention, the second hidden state submodule includes: a water pressure change rate unit, an ascent speed unit, and a second weight calculation unit;

[0155] The water pressure change rate unit is used to calculate the water pressure change rate at the current time step based on the water pressure at the current time step, the water pressure at multiple time steps ago, and the corresponding time interval.

[0156] The ascent rate unit is used to calculate the ascent rate of the diver at the current time step based on the water depth at the current time step, the water depth at multiple time steps ago, and the corresponding time interval.

[0157] The second weight calculation unit is used to calculate the second weight of the hidden state of the second time series feature at the current time step based on the water pressure change rate and the rise speed.

[0158] The embodiments of the present invention identify time steps that have a significant impact on decompression risk by using the rate of change of water pressure and the rate of rise, and assign them higher weights, which can improve the pertinence and accuracy of decompression risk assessment.

[0159] Optionally, in this embodiment of the invention, the return path module 204 includes: a terrain modeling submodule, a node expansion submodule, and a path planning submodule;

[0160] The terrain modeling submodule is used to establish a three-dimensional terrain model that includes a water flow direction vector field, a depth gradient field, and an obstacle distance field.

[0161] The node expansion submodule is used to expand nodes based on the three-dimensional terrain model according to a preset step size and movement cost.

[0162] The path planning submodule is used to obtain a safe return path based on the cost function and B-spline curve interpolation; wherein, the cost function includes the actual oxygen consumption cost and the heuristically estimated path cost.

[0163] The embodiments of the present invention are implemented through A The algorithm combines factors such as 3D terrain models and oxygen consumption to generate a global path for safe return in dynamic environments.

[0164] Optionally, in this embodiment of the invention, the return path module 204 further includes: a risk increment submodule and a path optimization submodule;

[0165] The risk increment submodule is used to acquire the hypothermia risk index and decompression risk index of divers in real time when they return to shore according to the safe return route, and to calculate the increment of the hypothermia risk index and the increment of the decompression risk index.

[0166] The path optimization submodule is used to optimize the path using a flexible action-evaluation algorithm when the increase in the hypothermia risk index exceeds a preset third threshold or the increase in the decompression risk index exceeds a preset fourth threshold. Specifically:

[0167] The safe return path is optimized based on the diver's state space and motion space, as well as a preset reward function; wherein the reward function includes oxygen consumption efficiency reward, time cost reward, hypothermia risk penalty, and decompression risk penalty.

[0168] This invention provides data support for determining whether a safe return path needs optimization by calculating the increments of the hypothermia risk index and the decompression risk index. By considering the hypothermia and decompression risks of divers during the return journey and using a flexible action-evaluation algorithm for local path optimization, it can prevent divers from experiencing hypothermia and decompression during the return journey.

[0169] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can implement the diving safety early warning method based on multimodal data fusion provided by any of the above method embodiments of the present invention.

[0170] This invention, through the data acquisition module 201, collects environmental data, as well as the diver's physiological and movement posture data in real time, providing a data foundation for subsequent assessments of hypothermia and decompression risks. By employing the first risk assessment module 202 and the second risk assessment module 203 to process corresponding multimodal data according to the diver's different movement states, it can achieve precise early warnings specific to the diving phase. By introducing the first and second risk assessment modules 202 and combining them with an attention mechanism, it can dynamically capture the time dependence of key features such as water temperature distribution gradient, convulsion frequency, water pressure change rate, and ascent speed. The return path module 204, combined with A... Algorithms and Flexible Motion Evaluation Algorithms generate safe return paths, which can improve the dynamic adaptability of safe return path planning and avoid increasing the risk of hypothermia or decompression for divers during the return journey.

[0171] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0172] Based on the above embodiment of a diving safety early warning method based on multimodal data fusion, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a diving safety early warning method based on multimodal data fusion according to any embodiment of the present invention.

[0173] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0174] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0175] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0176] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a diving safety early warning method based on multimodal data fusion as described in any of the above-described method embodiments of the present invention.

[0177] The modules / units integrated into the system / terminal device, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0178] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A diving safety early warning method based on multimodal data fusion, characterized in that, include: The system collects first environmental data, second environmental data, and the diver's physiological and motion posture data in real time. The first environmental data includes water temperature and current velocity, and the second environmental data includes water pressure and water depth. The physiological data includes heart rate, blood pressure, and blood oxygen saturation. The motion posture data includes triaxial acceleration. If the diver's movement state is either descending or stationary, the first environmental data, physiological data, and movement posture data are input into the pre-trained first LSTM model for risk assessment to obtain the hypothermia risk index; wherein, the first LSTM model calculates the attention weight for each time step based on the water temperature distribution gradient and the diver's twitching frequency. If the diver's motion state is ascent, the first environmental data, the second environmental data, physiological data, and motion posture data are input into the pre-trained second LSTM model for risk assessment to obtain the hypothermia risk index and decompression risk index. The second LSTM model calculates the attention weight for each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate, and the diver's ascent speed. When the temperature loss risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, a safety warning signal is issued and transmitted via A. Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path.

2. The diving safety early warning method based on multimodal data fusion as described in claim 1, characterized in that, The real-time acquisition of first and second environmental data, as well as the diver's physiological and movement posture data, includes: Based on PPS pulses, the acquisition clocks for the first environmental data, the second environmental data, physiological data, and motion posture data are synchronized. Z-score standardization was performed on the first environmental data, the second environmental data, the physiological data, and the motion posture data.

3. The diving safety early warning method based on multimodal data fusion as described in claim 1, characterized in that, After the real-time acquisition of the first environmental data, the second environmental data, and the diver's physiological data and motion posture data, the method further includes: The current motion state of the diver is determined based on the water depth and triaxial acceleration at a preset number of sampling points, specifically: If the water depth at the preset number of sampling points continues to decrease and the vertical acceleration of the triaxial acceleration remains positive, then the current motion state of the diver is determined to be ascending. If the water depth at the preset number of sampling points continues to rise and the vertical acceleration of the triaxial acceleration remains negative, then the current motion state of the diver is determined to be descent. If the water depth at the preset number of sampling points remains constant and the vertical acceleration of the triaxial acceleration remains zero, then the current motion state of the diver is determined to be stationary.

4. The diving safety early warning method based on multimodal data fusion as described in claim 1, characterized in that, The step of inputting the first environmental data, physiological data, and motion posture data into a pre-trained first LSTM model for risk assessment to obtain a hypothermia risk index includes: The first environmental data, physiological data, and motion posture data are weighted and fused into a temporal feature; wherein, the weights of the first environmental data, physiological data, and motion posture data are obtained based on prior knowledge; The hidden state of each time step of the temporal feature is extracted by a multi-layer LSTM, and the weight of each hidden state is calculated by an attention mechanism layer. The hidden states are weighted and summed based on the weights to obtain the context vector; A fully connected layer is used to map the context vector into a temperature loss risk index.

5. The diving safety early warning method based on multimodal data fusion as described in claim 4, characterized in that, The calculation of the weights of each hidden state through the attention mechanism layer includes: Calculate the water temperature distribution gradient at the current time step based on the water temperature at the current time step, multiple water temperatures before the current time step, and the corresponding time intervals. The twitching frequency of the diver at the current time step is calculated based on the total number of twitching events within a preset time window and the length of the time window; wherein, when the variance and peak frequency of the triaxial acceleration exceed the corresponding preset threshold, a twitching event is determined to exist. The weights of the hidden states at the current time step are calculated based on the water temperature distribution gradient and the twitching frequency.

6. The diving safety early warning method based on multimodal data fusion as described in claim 1, characterized in that, The process involves inputting the first environmental data, the second environmental data, physiological data, and motion posture data into a pre-trained second LSTM model for risk assessment, resulting in hypothermia risk index and decompression risk index, including: The first environmental data, physiological data, and motion posture data are weighted and fused into a first temporal feature, and the second environmental data, physiological data, and motion posture data are weighted and fused into a second temporal feature; wherein, the weights of the first environmental data, second environmental data, physiological data, and motion posture data are obtained based on prior knowledge; The hidden states of the first and second time-series features at each time step are extracted by multi-layer LSTM, and the first weight of the hidden state of the first time-series feature at each time step is calculated by the first attention mechanism layer, and the second weight of the hidden state of the second time-series feature at each time step is calculated by the second attention mechanism layer. Based on the first weight, the hidden states of the first time series feature at each time step are summed in a weighted manner to obtain a first context vector, and based on the second weight, the hidden states of the second time series feature at each time step are summed in a weighted manner to obtain a second context vector. A first fully connected layer is used to map the first context vector to a temperature loss risk index, and a second fully connected layer is used to map the second context vector to a decompression risk index.

7. The diving safety early warning method based on multimodal data fusion as described in claim 6, characterized in that, The calculation of the second weight of the hidden state of the second temporal feature at each time step through the second attention mechanism layer includes: Calculate the rate of change of water pressure at the current time step based on the water pressure at the current time step, multiple water pressures prior to the current time step, and the corresponding time intervals. Calculate the diver's ascent rate at the current time step based on the water depth at the current time step, multiple water depths before the current time step, and the corresponding time intervals. The second weight of the hidden state of the second time series feature at the current time step is calculated based on the water pressure change rate and the rise speed.

8. The diving safety early warning method based on multimodal data fusion as described in claim 1, characterized in that, The passage through A Algorithms and Flexible Motion Evaluation: The algorithm generates a safe return path, including: Establish a three-dimensional terrain model that includes the water flow direction vector field, depth gradient field, and obstacle distance field; Based on the three-dimensional terrain model, node expansion is performed according to the preset step size and movement cost; A safe return route is obtained based on the cost function and B-spline curve interpolation; wherein, the cost function includes the actual oxygen consumption cost and the heuristically estimated route cost.

9. A diving safety early warning method based on multimodal data fusion as described in claim 8, characterized in that, The passage through A The algorithm and flexible motion evaluation algorithm generate a safe return path, and also includes: The system acquires the hypothermia risk index and decompression risk index of divers in real time when they return to base according to the safe return route, and calculates the increments of the hypothermia risk index and decompression risk index. When the increase in the hypothermia risk index exceeds a preset third threshold or the increase in the decompression risk index exceeds a preset fourth threshold, path optimization is performed using a flexible action-evaluation algorithm, specifically: The safe return path is optimized based on the diver's state space and motion space, as well as a preset reward function; wherein the reward function includes oxygen consumption efficiency reward, time cost reward, hypothermia risk penalty, and decompression risk penalty.

10. A diving safety early warning system based on multimodal data fusion, characterized in that, include: The system includes a data acquisition module, a first risk assessment module, a second risk assessment module, and a return route module. The data acquisition module is used to collect first environmental data, second environmental data, and the diver's physiological and motion posture data in real time; wherein, the first environmental data includes water temperature and water flow velocity, the second environmental data includes water pressure and water depth; the physiological data includes heart rate, blood pressure, and blood oxygen saturation; and the motion posture data includes triaxial acceleration. The first risk assessment module is used to input the first environmental data, physiological data and motion posture data into the pre-trained first LSTM model to conduct risk assessment and obtain the hypothermia risk index if the diver's motion state is diving or stationary. The first LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient and the diver's convulsion frequency. The second risk assessment module is used to input the first environmental data, the second environmental data, the physiological data, and the motion posture data into the pre-trained second LSTM model to perform risk assessment if the diver's motion state is ascending, and to obtain the hypothermia risk index and the decompression risk index; wherein, the second LSTM model calculates the attention weight of each time step based on the water temperature distribution gradient, the diver's convulsion frequency, the water pressure change rate, and the diver's ascent speed. The return path module is used to issue a safety warning signal and, when the hypothermia risk index exceeds a preset first threshold or the decompression risk index exceeds a preset second threshold, via A... Algorithms and Flexible Motion - Evaluation Algorithm Generates Safe Return Path.

Images