AR intelligent safety helmet auxiliary system for high-risk operation environment

Abstract

The invention discloses an AR intelligent safety helmet auxiliary system oriented to a high-risk operation environment, and relates to the technical field of construction safety, the system combines a Bayesian network with historical data to carry out probabilistic reasoning and LSTM network mining time sequence characteristics, and through D-S evidence theory fusion, the falling trend of an object can be identified more accurately, early warning can be given in advance, and the safety of construction is improved. And precious danger avoiding time is won for operators. Meanwhile, a dynamic decision-making mechanism of the system enables triggering of a buffering mechanism to be more reasonable, unnecessary resource waste is reduced while the occurrence rate of safety accidents is reduced, and a scientific and efficient solution is provided for high-risk operation safety protection.

Description

Technical Field

[0001] The present invention relates to the technical field of construction safety, and specifically to an AR intelligent safety helmet assistance system for high-risk working environments. Background Technique

[0002] As the most basic and crucial safety protection equipment in high-risk working scenarios such as industrial production and construction, the functions of safety helmets are evolving from traditional passive protection to intelligent and proactive warning. As a cutting-edge technological product in this field, the AR intelligent safety helmet assistance system aims to provide more intelligent and precise safety protection and operation assistance for workers in high-risk fields such as construction, mining, and high-altitude operations, becoming an important direction to promote the intelligent upgrade of safety protection equipment.

[0003] However, existing safety helmets have many deficiencies when dealing with potential risks in high-risk working environments. For example, traditional safety helmets only rely on physical structures to resist impacts. However, in construction sites, safety hazards such as falling objects from heights and equipment failures occur frequently. Traditional safety helmets can only provide passive physical protection during accidents and are difficult to perceive dangers in advance and issue warnings. For sudden object falls, workers often lack timely and effective prompts and are difficult to quickly make correct evasive responses, thus increasing the risk of injury. Summary of the Invention

[0004] In view of the above problems existing in the prior art, the present application provides an AR intelligent safety helmet assistance system for high-risk working environments.

[0005] The embodiments of the present disclosure provide an AR intelligent safety helmet assistance system for high-risk working environments, including:

[0006] The acquisition module is used to deploy monitoring devices in the safety helmet to obtain real-time environmental change data, and after preprocessing, determine abnormal feature points;

[0007] The analysis module is used to perform probability analysis on the abnormal feature points by using historical data and real-time environmental change data to generate command signals;

[0008] The feedback module is used to generate an optimal path when the command signal is a warning command, superimpose the optimal path on the worker's field of vision in the form of an AR virtual arrow through a head-mounted display for evasive operations, and monitor the actual impact during the evasive process to control the integrity of the buffer structure.

[0009] Optionally, the acquisition module includes:

[0010] The acquisition unit is used to deploy sensor groups, cameras, and head-mounted displays on the top, side, and rear of the safety helmet in advance to obtain real-time environmental change data and perform spatio-temporal alignment on the real-time environmental change data. Specifically, by connecting each sensor device to a GPS receiver, a unified time reference is obtained to complete the time synchronization operation. And by using a laser scanner to scan the high-risk operation environment site, three-dimensional point cloud data is obtained. Through matching and aligning the point cloud data collected by different sensor devices at the same spatial position, the spatial registration operation is completed.

[0011] Optionally, the acquisition module further includes:

[0012] The processing unit is used to remove outliers from the real-time environmental change data by using the median filtering algorithm, and extract the relative speed change rate, distance change acceleration, curvature of the object movement trajectory, sudden change angle of the trajectory direction, pressure fluctuation amplitude, and pressure fluctuation frequency of the object on the top of the safety helmet from the processed real-time environmental change data to generate a real-time feature set.

[0013] The recognition unit is used to draw a histogram of the historical data based on the statistical analysis of the historical data, determine the distribution shape of the historical data, determine the model category that the historical data follows according to the distribution shape, calculate the normal distribution interval of each feature parameter extracted in the processing unit according to the model category, and compare the real-time feature set with the normal distribution interval. If the feature points in the real-time feature set are outside the normal distribution interval, they are marked as abnormal feature points; otherwise, no marking process is performed.

[0014] Optionally, the analysis module includes:

[0015] The conditional analysis unit is used to use each feature parameter in the historical data as a node based on the historical data, and use the conditional probability table trained by the historical data as the weight of the edge between each node to construct a Bayesian network. And according to the historical data, calculate the probability of the abnormal feature points observed when an object falls and the frequency of the abnormal feature points appearing in the historical data, and count the number of object fall events when no abnormal feature points appear. When an abnormal feature point appears in the recognition unit, the corresponding abnormal feature point is input into the Bayesian network, and the first probability value of the object falling on the top of the safety helmet is calculated based on probability reasoning. The acquisition method of the first probability value is: divide the number of object fall events when no abnormal feature points appear by the total time involved in the historical data to obtain the probability of the object fall event when no abnormal feature points appear, and divide the product of the probability of the abnormal feature points observed when an object falls and the probability of the object fall event when no abnormal feature points appear by the frequency of the abnormal feature points appearing in the historical data to obtain the first probability value.

[0016] Optionally, the analysis module further includes:

[0017] The timing analysis unit is configured to input the feature parameters within consecutive time windows in the historical period into the long short-term memory network. The long short-term memory network captures the timing change pattern of the motion state of the object on the top of the safety helmet through the forget gate, input gate, and output gate, and inputs the abnormal feature points into the long short-term memory network to output a second probability value from the output gate.

[0018] Optionally, the analysis module further includes:

[0019] The fusion unit is configured to fuse the results of the Bayesian network and the long short-term memory network using the D-S evidence theory to obtain a fall probability evaluation value. Specifically, the first probability value calculated by the Bayesian network and the second probability value of the long short-term memory network are respectively regarded as evidence sources to obtain the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network, and in combination with the set of all possible results of the evidence sources, the set of all possible results of the evidence sources includes fall and non-fall. The D-S evidence synthesis rule is used to fuse the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network to obtain the fall probability evaluation value.

[0020] Optionally, the analysis module further includes:

[0021] The determination unit is configured to compare the fall probability evaluation value with a preset evaluation threshold to generate an instruction signal; if the fall probability evaluation value exceeds the evaluation threshold, it is determined that there is a falling trend, triggering a warning instruction and a buffer mechanism, otherwise it is determined that there is no falling trend.

[0022] Optionally, the feedback module includes:

[0023] The prompt unit is configured to receive the warning instruction, determine the passable paths around the staff wearing the safety helmet through the camera installed on the safety helmet. If there are multiple groups of passable paths, the optimal path is selected according to the path undulation degree in each passable path. The optimal path is the passable path corresponding to the minimum path undulation degree, and the optimal path is superimposed on the staff's field of vision in the form of an AR virtual arrow through the head-mounted display, and at the same time, the staff is guided to avoid through voice prompts.

[0024] Optionally, the feedback module further includes:

[0025] The impact and buffer unit is used to monitor the collapse during the dodging process through the optical fiber strain sensor used inside the helmet to obtain the deformation amount, and determine whether to trigger the buffer mechanism again based on the deformation amount. If the deformation exceeds the preset deformation threshold, the buffer layer is triggered. The buffer layer adopts a multi-stage trigger mechanism, including a pre-inflated layer and a rapid expansion layer. The pre-inflated layer is partially inflated when it is determined that there is a falling trend. When the buffer mechanism is triggered again, the rapid expansion layer completes the inflation within 50 milliseconds to form a complete buffer structure to absorb impact energy.

[0026] Beneficial effects of the present invention: The system uses Bayesian network combined with historical data for probabilistic reasoning, LSTM network to mine time series features, and then integrates through DS evidence theory, which can more accurately identify the falling trend of objects, issue early warnings, and strive for precious time for operators to avoid danger. At the same time, the dynamic decision-making mechanism of the system makes the triggering of the buffer mechanism more reasonable, while reducing the incidence of safety accidents, reducing unnecessary waste of resources, and providing a scientific and efficient solution for the safety protection of high-risk operations. The feedback module improves the emergency response ability and safety protection level of high-risk operators in the face of falling risks by optimizing path planning and dynamic buffer control. In terms of path planning, the prompt unit identifies the surrounding passable paths based on the camera, and calculates the undulation of each path using the distance data obtained by the laser altimeter, accurately selects the optimal path with the minimum undulation, and guides the operators to evacuate quickly and safely with AR virtual arrows and voice prompts. For example, at a construction site, when the warning instruction is triggered, the operator can quickly obtain clear escape instructions in a complex environment, avoid delaying the escape opportunity due to panic selection of rugged and obstacle-filled paths, and reduce the risk of being hit by falling objects. In terms of impact protection, the impact and buffer unit uses optical fiber strain sensors to monitor the collapse and deformation of the helmet in real time, combined with a multi-level trigger buffer mechanism to achieve efficient resolution of the impact force. When the deformation exceeds the preset threshold, the pre-inflated layer and the rapid expansion layer work together to quickly form a complete buffer structure, effectively absorb the impact energy, and reduce the impact force on the head. Compared with the passive protection mode of traditional helmets, it enhances the protection effect on workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application.

[0028] Figure 1 It is a module diagram of the present invention;

[0029] Figure 2 It is a cross-sectional structural diagram of a safety helmet according to the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following will clearly and completely describe the technical solutions in this application in conjunction with the accompanying drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0031] As Figure 1 shown, the implementation of the present invention proposes an AR intelligent safety helmet assistance system for high-risk working environments, including:

[0032] The acquisition module is used to deploy monitoring devices in the safety helmet to obtain real-time environmental change data. After preprocessing, abnormal feature points are determined.

[0033] The acquisition unit is used to pre-deploy sensor groups, cameras, and head-mounted displays on the top, side, and rear of the safety helmet to obtain real-time environmental change data and perform spatio-temporal alignment on the real-time environmental change data. Specifically: by connecting each sensor device to a GPS receiver to obtain a unified time reference, time synchronization operations are completed, and by using a laser scanner to scan the high-risk working environment site, three-dimensional point cloud data is obtained. By matching and aligning the point cloud data collected by different sensor devices at the same spatial position, spatial registration operations are completed.

[0034] Six-axis acceleration sensors, pressure sensor matrices, laser range sensors, and high-definition cameras are evenly deployed on the top, side, and rear of the safety helmet. The six-axis acceleration sensor collects acceleration and angular velocity data in three directions. The pressure sensor matrix consists of multiple micro-pressure sensors and can capture pressure changes in different areas; the laser range sensor scans the area above the working space in real time to detect the distance to objects; the high-definition camera collects visual information of the working environment.

[0035] Utilize the high-precision timekeeping function of the global positioning system to obtain accurate time information by receiving GPS satellite signals. The GPS receiver can provide time accuracy at the nanosecond level. Each sensor device is connected to the GPS receiver to obtain a unified time reference and achieve time synchronization.

[0036] The processing unit is used to remove outliers from the real-time environmental change data using the median filtering algorithm, and extract the relative speed change rate of the object on the top of the safety helmet, distance change acceleration, curvature of the object movement trajectory, sudden change angle of the trajectory direction, pressure fluctuation amplitude, and pressure fluctuation frequency from the processed real-time environmental change data to generate a real-time feature set.

[0037] The recognition unit is used to draw a histogram of historical data based on the statistical analysis of historical data, determine the distribution shape of historical data, and determine the model category that the historical data follows according to the distribution shape. For example, if the data presents a bell-shaped curve similar to a Gaussian distribution, it may follow a Gaussian mixture model or a Gaussian distribution; or some statistical test methods can be used to judge whether the data follows a specific distribution. For example, for a Gaussian distribution, the Shapiro-Wilk test can be used. The null hypothesis of this test is that the data follows a normal distribution. If the p-value of the test result is greater than the given significance level, the null hypothesis is accepted, and it is considered that the data follows a normal distribution. For a Gaussian mixture model, a likelihood ratio-based test method can be used.

[0038] Calculate the normal distribution interval of each feature parameter extracted in the processing unit according to the model category, and compare the real-time feature set with the normal distribution interval. If the feature points in the real-time feature set are outside the normal distribution interval, mark them as abnormal feature points; otherwise, no marking process is performed.

[0039] Historical data refers to the feature set collected during a historical period.

[0040] In the embodiments of the present invention, the acquisition module further improves the comprehensiveness and accuracy of safety hazard monitoring in high-risk working environments through multi-dimensional data acquisition, precise preprocessing, and intelligent anomaly recognition. Compared with traditional single monitoring methods, it can capture potential risks more timely and accurately.

[0041] In the data acquisition link, the acquisition unit deploys sensor groups, cameras and other devices at multiple parts of the safety helmet, and combines GPS and laser scanners to achieve spatio-temporal alignment. Taking mine exploitation as an example, the top pressure sensor, side displacement sensor, and rear temperature and humidity sensor work together, cooperate with the camera to capture the on-site picture, while GPS ensures the time synchronization of each data, and the laser scanner completes the spatial registration, enabling the system to build a multi-dimensional perception network for the working environment, avoiding monitoring deviations caused by data asynchronization and spatial position disorder, and ensuring that the acquired environmental data truly reflects the on-site situation.

[0042] The processing unit uses the median filtering algorithm to remove outliers, effectively filtering out error data generated by equipment noise, environmental interference, etc. For example, in a construction site, the electromagnetic interference generated during the operation of a crane may cause abnormal fluctuations in sensor data. The median filtering algorithm can quickly eliminate this interference data, and then extract key features such as the relative speed change rate of an object and the pressure fluctuation frequency, accurately focusing on information related to safety hazards, and providing reliable data support for subsequent analysis.

[0043] The recognition unit determines the normal distribution interval based on the statistical analysis of historical data to judge whether the real-time data is abnormal. For example, in the scenario of working at height, the historical data shows that the curvature of the object's movement trajectory fluctuates within a certain range during normal operation. When the curvature of the trajectory monitored in real time exceeds this range, the system immediately marks it as an abnormal feature point. Compared with manual judgment or simple threshold setting, this judgment method based on the data distribution law can detect subtle abnormalities more sensitively, give early warnings of potential dangers, gain more time for risk avoidance for the operators, and greatly reduce the probability of safety accidents.

[0044] Furthermore, the analysis module is used to perform probability analysis on the abnormal feature points by using historical data and real-time environmental change data to generate command signals; the analysis module includes: a condition analysis unit for using the historical data, taking each feature parameter in the historical data as a node, and using the conditional probability table obtained by training with the historical data as the weight of the edge between each node to construct a Bayesian network, and according to the historical data, calculating the probability of the abnormal feature points observed when the object falls and the frequency of the abnormal feature points appearing in the historical data, and counting the number of times of the object falling events when no abnormal feature points appear;

[0045] Among them, the probability of the abnormal feature points observed when the object falls can be estimated through historical data. Collect a large amount of characteristic parameter data corresponding to the object falling, and count the frequencies of various value combinations of these characteristic parameters when the object falls, and approximate this as the likelihood probability. For example, in multiple recorded object falling events, count the value situations of characteristic parameters such as acceleration and displacement collected by the sensor, and calculate the occurrence frequencies of different values.

[0046] The frequency of the abnormal feature points appearing in the historical data can be estimated by statistically analyzing a large amount of historical data (including the characteristic data when the object falls and does not fall), and calculating the frequencies of these characteristic parameters appearing in all the data to estimate the probability of the feature occurrence.

[0047] Dividing the number of times of the object falling events when no abnormal feature points appear by the total time involved in the historical data, the probability of the object falling events when no abnormal feature points appear can be obtained based on historical statistical data. For example, count the number of times of the object falling events within a certain period of time, and divide it by the total number of events (or total time) to obtain the probability of the object falling, that is, the prior probability;

[0048] When an abnormal feature point appears in the recognition unit, the corresponding abnormal feature point is input into the Bayesian network, and the first probability value of the object falling on the top of the safety helmet is calculated based on probabilistic reasoning. The way to obtain the first probability value is as follows: Divide the number of object falling events that occur when no abnormal feature points appear by the total time involved in the historical data to obtain the probability of the object falling event when no abnormal feature points appear. Divide the product of the probability of the abnormal feature point appearing when the object falls and the probability of the object falling event when no abnormal feature points appear by the frequency of the abnormal feature point appearing in the historical data to obtain the first probability value. The first probability value is the conditional probability, which reflects the actual possibility of the object falling under the given observed features, and is a probability estimate obtained by correcting the prior knowledge (prior probability) with new evidence (feature parameters).

[0049] The time series analysis unit is used to input the feature parameters within consecutive time windows in the historical period into the long short-term memory network (LSTM). The long short-term memory network captures the time series change law of the object movement state on the top of the safety helmet through the forget gate, input gate, and output gate, and inputs the abnormal feature points into the long short-term memory network to output the second probability value from the output gate.

[0050] In the LSTM network, the forget gate, input gate, and output gate are three core components. They solve the gradient vanishing problem in the traditional recurrent neural network (RNN) by selectively retaining, updating, and outputting information, enabling the network to learn the time-dependent relationships in long sequence data. The following explains their working principles in plain language:

[0051] The forget gate determines which information to discard from the cell state. The forget gate calculates the previous hidden state and the current input through a Sigmoid function (output range 0 - 1) to obtain a vector between 0 and 1. Each value in this vector represents the degree to which the corresponding information in the cell state needs to be forgotten (0 means completely forgotten, 1 means completely retained). For example, assume the LSTM is analyzing the object movement in a video. When the object leaves the frame, the forget gate will forget the previously stored object position information to make room for new object information.

[0052] The input gate determines which new information will be added to the cell state. Among them, the input gate is divided into two parts: the Sigmoid function: determines which values need to be updated (outputs a vector between 0 and 1); the Tanh function: creates a new candidate vector that may be added to the cell state. Finally, the input gate combines the results of these two parts to selectively update the cell state. For example, continuing the above video analysis example, when a new object enters the frame, the input gate will decide to add the initial position and features of the object as new information to the cell state.

[0053] The output gate determines which information to output from the cell state. First, the output gate processes the previous hidden state and the current input through the Sigmoid function to obtain a vector between 0 and 1, which controls which parts of the cell state will be output. Then, after processing the cell state through the Tanh function (scaling the values between -1 and 1), it is multiplied by the Sigmoid output vector to obtain the final output. For example, in video analysis, the output gate decides which information (such as the position and speed of a specific object) should be used to predict the content of the next frame based on all the object information stored in the current cell state. The forget gate, input gate, and output gate work together on the cell state to achieve selective information flow.

[0054] In a complex environment monitoring and safety warning system, a single model often has limitations and is difficult to comprehensively and accurately capture the characteristics of target events. For example, in the scenario of judging the falling trend of an AR intelligent safety helmet, although the Bayesian network can perform probability reasoning based on prior knowledge, its ability to capture temporal features is weak; while the LSTM network is good at dealing with the time-dependent relationship of sequence data, but lacks the effective utilization of domain knowledge. Therefore, it is necessary to effectively fuse the outputs of multiple complementary models to improve the accuracy and reliability of decision-making.

[0055] The analysis module also includes: a fusion unit for fusing the results of the Bayesian network and the long short-term memory network using the D-S evidence theory to obtain a falling probability evaluation value. Specifically: taking the first probability value calculated by the Bayesian network and the second probability value calculated by the long short-term memory network as evidence sources respectively, to obtain the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network, and combining all possible result sets of the evidence sources. All possible result sets of the evidence sources include falling and non-falling. Using the D-S evidence synthesis rule to fuse the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network to obtain the falling probability evaluation value.

[0056] First, use the maximum likelihood estimation or Bayesian estimation method to construct each feature parameter node and the conditional probability table. For the data to be detected, that is, the abnormal feature points, calculate the first probability value and the probability value that the object does not fall under the given observed features. Then, convert the first probability value output by the Bayesian network into the basic probability assignment function m1. Specifically: m1({falling}) = the first probability value, m1({non-falling}) = the probability value that the object does not fall under the given observed features, m1(Θ)= , is a measure of uncertainty, reflecting the degree of uncertainty of the Bayesian network's own judgment. Next, historical time-series data is used to train the LSTM network to learn the time-dependent relationship of feature parameters. For the time-series data window to be detected, that is, the time-series data window corresponding to the abnormal feature point, the second probability value is predicted, that is, the probability of the object falling at a future moment, and the predicted probability output by the LSTM network is converted into a basic probability assignment function m2, specifically: m2({fall}) = the second probability value, m2({non-fall}) = 1 - the second probability value, m2(Θ) = , is a measure of uncertainty, , is a preset constant (0 < < 1), is a dynamic adjustment factor, determined according to the performance of the LSTM model on the validation set, is the second probability value; then the D-S evidence combination rule is used to fuse m1 and m2: , where, is marked as the fall probability evaluation value, K is the normalization constant, is marked as the basic probability assignment value constructed based on the calculation result of the Bayesian network, that is, in the first evidence source (the basic probability assignment function transformed from the result of the Bayesian network), the basic probability assignment value of event A1, is marked as the basic probability assignment value constructed based on the calculation result of the long short-term memory network, that is, in the second evidence source (the basic probability assignment function transformed from the result of the LSTM network), the basic probability assignment value of event A2, A1 is an element in the event set involved in the first evidence source, A2 is an element in the event set involved in the second evidence source, A is marked as and 's intersection; the role of the normalization constant K is to ensure that the sum of the values of the fused basic probability assignment function is 1, avoiding deviation in the total probability value due to the calculation process.

[0057] As an uncertainty reasoning method, D-S evidence theory can handle incomplete information and uncertainty, providing a theoretical basis for multi-source information fusion.

[0058] In D-S evidence theory, the basic probability assignment function m is a mapping from the power set to [0, 1], satisfying Σm(A) = 1, where A Θ, Θ is the set of all possible results;

[0059] The evidence combination rule is the core formula of D-S evidence theory, used to fuse the basic probability assignment functions of multiple independent evidence sources.

[0060] The analysis module also includes: a judgment unit for comparing the fall probability assessment value with a pre-set assessment threshold to generate a command signal; if the fall probability assessment value exceeds the assessment threshold, it is determined that there is a fall trend, triggering an early warning instruction and a buffer mechanism, otherwise it is determined that there is no fall trend.

[0061] The command signal includes a trigger warning command and a non-trigger warning command;

[0062] In the embodiment of the present invention, the analysis module builds a high-precision risk assessment system through multi-model collaboration and deep data integration, improves the ability to predict the risk of falling objects in high-risk working environments, and can identify potential dangers more scientifically and accurately than traditional single model analysis. In the conditional analysis unit, a Bayesian network is constructed based on historical data, and the characteristic parameters are used as nodes and the conditional probability table is used as the edge weight to quantify the probability association between each feature and the falling of the object. For example, in the construction scene, past data show that when the tilt angle of the tower crane changes suddenly (abnormal feature point), the probability of the object falling is 40%, and the frequency of the abnormal feature point in the historical data is 15%, and there is a fall event every 50 hours when there is no abnormal feature point. When the sudden change of the tilt angle of the tower crane is monitored in real time, the No. 1 probability value is calculated through the Bayesian network, and the judgment of the falling risk is updated by combining prior knowledge and real-time evidence, providing a probability-level basis for risk assessment.

[0063] The time series analysis unit uses the LSTM network to deeply explore the time series laws of the object's motion state by using the forget gate, input gate, and output gate to filter and update information for the characteristic parameters in the continuous time window of the historical period. For example, in high-altitude power maintenance operations, the LSTM network continuously learns the time series changes of characteristic parameters such as vibration and displacement of equipment components, such as insulators and cable joints. When abnormal fluctuations (abnormal feature points) in the vibration frequency of a component are detected, the LSTM network outputs the second probability value of the component falling based on the learning of historical time series data, effectively capturing the dynamic trend of the object's movement and making up for the shortcomings of the Bayesian network in processing time series information.

[0064] The fusion unit adopts the DS evidence theory, taking the No. 1 probability value calculated by the Bayesian network and the No. 2 probability value output by the LSTM network as independent evidence sources, converting them into basic probability distribution values ​​for fusion. For example, in a certain working condition, the Bayesian network concluded that the probability of falling was 0.5, while the LSTM network concluded that it was 0.6, which is different from the two. Through the DS evidence synthesis rule, the judgments of the two models are comprehensively considered, the contradictions and uncertainties between the evidence are eliminated, and a more realistic fall probability assessment value is obtained to avoid misjudgment of a single model.

[0065] Finally, the determination unit compares the fused fall probability evaluation value with a preset threshold. If it exceeds the threshold, it determines that there is a falling trend and triggers a warning instruction and a buffer mechanism; otherwise, it determines safety.

[0066] Furthermore, the feedback module is used to generate an optimal path when the instruction signal is a warning instruction, superimpose the optimal path on the staff's field of vision in the form of AR virtual arrows through a head-mounted display for evasive operations, and during the evasive process, monitor the actual impact and control the integrity of the buffer structure.

[0067] The feedback module includes: a prompt unit for receiving a warning instruction, determining the passable paths around the staff wearing the safety helmet through a camera installed on the safety helmet. If there are multiple groups of passable paths, it selects the optimal path according to the path undulation degree in each passable path. The optimal path is the passable path corresponding to the minimum path undulation degree, and superimposes the optimal path on the staff's field of vision in the form of AR virtual arrows through a head-mounted display, and at the same time guides the staff to avoid through voice prompts, which is used to remind the staff to immediately leave the current position to avoid the risk of falling objects from above.

[0068] The laser altimeter calculates the distance by emitting laser and measuring the reflection light time. When the path is flat, the height change is small; when the path has slope undulations, the height will change significantly, and the height change amount is used as the path undulation degree;

[0069] The passable path represents the paths that can be passed around the current position;

[0070] It can be seen from Figure 2 that the inside of the safety helmet consists of a shell, a skeleton, a cavity and an inner support shell. Among them, the skeleton is arranged in the inner cavity of the safety helmet and is close to the shell. The gap between the inner support shell and the skeleton is the cavity, and the cavity is used to trigger the buffer layer. The buffer layer is used to form an airbag to form a complete buffer structure, so as to absorb the impact energy;

[0071] The feedback module also includes: an impact and buffer unit for monitoring the collapse situation through a fiber optic strain sensor adopted inside the safety helmet during the evasive process to obtain the deformation amount, judging whether to trigger the buffer mechanism again according to the deformation amount. If the deformation amount exceeds the preset deformation threshold, the buffer layer is triggered. The buffer layer adopts a multi-stage trigger mechanism, including a pre-inflated layer and a rapid expansion layer. The pre-inflated layer is partially inflated when it is determined that there is a falling trend. When the buffer mechanism is triggered again, the rapid expansion layer is inflated within 50 milliseconds to form a complete buffer structure and absorb the impact energy.

[0072] The fiber optic strain sensor is laid along the safety helmet skeleton. When the safety helmet is subjected to external impact, the deformation of the skeleton will cause tensile or compressive strain on the optical fiber. The change in the fiber optic strain will cause changes in the characteristics of light propagation in the optical fiber, such as wavelength, light intensity, and phase. Before laying the fiber optic strain sensor, it is necessary to conduct mechanical analysis and experimental calibration on the safety helmet skeleton to establish a mathematical relationship model between strain and skeleton deformation. The finite element analysis software can be used to simulate the strain and deformation of each part of the skeleton under different loading conditions of the safety helmet, and obtain the theoretical strain-deformation curve. At the same time, conduct actual loading experiments on the safety helmet, arrange high-precision displacement measurement devices (such as laser displacement sensors) on the skeleton, and synchronously measure the strain (obtained by the fiber optic strain sensor) and deformation of the skeleton during the loading process, and fit the empirical formula of strain and deformation through experimental data. For example, for a safety helmet skeleton of a certain material and structure, through experiments and analysis, it is obtained that within a certain strain range, the relationship between the deformation of a certain part of the skeleton and the fiber optic strain is , where k is the proportionality coefficient determined through experiments, is the original length of this part when not under force, is the fiber optic strain, is the deformation of a certain part of the skeleton.

[0073] In the embodiment of the present invention, the feedback module constructs an all-round and multi-level safety protection system through path planning, navigation guidance, and dynamic buffer protection, improving the emergency avoidance ability and safety guarantee level of operators in high-risk environments. When the determination unit issues a warning instruction, the prompt unit is immediately activated. The surrounding environment of the operator is identified through the camera on the safety helmet to obtain the passable path information;

[0074] The laser altimeter is used to measure the height change data of each path and calculate the path undulation. The optimal path is selected based on the minimum path undulation and superimposed on the head-mounted display in the form of an AR virtual arrow, supplemented by voice prompts to guide the operator to quickly evacuate the dangerous area. For example, in the high-altitude operation scenario on a construction site, when there is a risk of falling objects from above, in the traditional method, the operator may be panicked and difficult to quickly find a safe escape path, or the selected path may have obstacles, a large slope, etc., resulting in low escape efficiency. And this system can instantly plan the optimal path. For example, when a component loosening warning occurs near the tower crane, the system can avoid the rough path full of materials and guide the operator to evacuate along a flat and unobstructed passage, effectively reducing the risk of escape delay and secondary injury caused by improper path selection, and improving the timeliness and safety of emergency escape.

[0075] During the worker's evasion process, the impact and buffer unit uses the optical fiber strain sensor inside the helmet to monitor the collapse and deformation of the helmet in real time. When the deformation detected by the sensor exceeds the preset deformation threshold, the multi-level trigger mechanism of the buffer layer is triggered. The pre-inflated layer has been partially inflated during the early warning stage. At this time, the rapid expansion layer is quickly inflated within 50 milliseconds to form a complete structure with strong buffering capacity, which is used to absorb the impact energy of falling objects and protect the safety of the worker's head.

[0076] For example, in mining operations, even if workers have taken evasive measures, they may still be impacted by falling ore. Traditional helmets have limited buffering capacity and are difficult to withstand large impact forces. The dynamic buffering mechanism of this system can respond accurately according to the actual impact situation. For example, when the ore hits the helmet, the optical fiber strain sensor immediately captures the deformation of the helmet. If the deformation exceeds the standard, the rapid expansion layer will inflate quickly, greatly weakening the impact force that may have caused serious damage to the head. It is like instantly propping up an airbag for the head, effectively reducing the degree of head injury. Compared with the passive and fixed protection mode of traditional helmets, it improves the protection effect on workers.

[0077] Through the above two closely coordinated links, the feedback module realizes the whole process from efficient escape guidance after risk warning to intelligent protection when suffering impact, building a solid and reliable line of defense for the life safety of high-risk workers, while also improving the intelligence level and practical application efficiency of safety protection equipment.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An AR intelligent safety helmet assistance system for high-risk working environments, characterized in that: Including: The acquisition module is used to deploy monitoring devices in the safety helmet to obtain real-time environmental change data. After preprocessing, abnormal feature points are determined. The analysis module is used to perform probability analysis on the abnormal feature points by using historical data and real-time environmental change data to generate command signals. The feedback module is used to generate an optimal path when the command signal is a warning command, superimpose the optimal path on the staff's field of vision in the form of AR virtual arrows through the head-mounted display, perform evasive operations, and monitor actual impacts during the evasive process to control the integrity of the buffer structure.

2. The AR intelligent safety helmet assistance system for high-risk operation environments according to claim 1, characterized in that: The acquisition module includes: The acquisition unit is used to deploy sensor groups, cameras, and head-mounted displays on the top, side, and rear of the safety helmet in advance to obtain real-time environmental change data and perform spatio-temporal alignment on the real-time environmental change data. Specifically: by connecting each sensor device to a GPS receiver to obtain a unified time reference and complete the time synchronization operation, and by using a laser scanner to scan the high-risk operation environment site to obtain three-dimensional point cloud data, and by matching and aligning the point cloud data collected by different sensor devices at the same spatial position, the spatial registration operation is completed.

3. The AR intelligent safety helmet assistance system for high-risk working environments according to claim 2, wherein: The acquisition module further includes: The processing unit is used to remove outliers from the real-time environmental change data by using the median filtering algorithm, and extract the relative velocity change rate, distance change acceleration, curvature of the object's motion trajectory, sudden change angle of the trajectory direction, pressure fluctuation amplitude, and pressure fluctuation frequency of the object on the top of the safety helmet from the processed real-time environmental change data to generate a real-time feature set. The recognition unit is used to draw a histogram of the historical data based on the statistical analysis of the historical data, determine the distribution shape of the historical data, determine the model category that the historical data follows according to the distribution shape, calculate the normal distribution interval of each feature parameter extracted by the processing unit according to the model category, and compare the real-time feature set with the normal distribution interval. If the feature points in the real-time feature set are outside the normal distribution interval, they are marked as abnormal feature points; otherwise, no marking process is performed.

4. The AR intelligent safety helmet assistance system for high-risk working environments according to claim 3, characterized in that: The analysis module includes: The condition analysis unit is used to construct a Bayesian network based on historical data, taking each feature parameter in the historical data as a node and using the conditional probability table obtained by training with the historical data as the weight of the edge between each node. According to the historical data, it calculates the probability of the occurrence of abnormal feature points observed when an object falls and the frequency of the occurrence of abnormal feature points in the historical data, and counts the number of object fall events when no abnormal feature points appear. When an abnormal feature point appears in the recognition unit, the corresponding abnormal feature point is input into the Bayesian network, and the first probability value of the object falling on the top of the safety helmet is calculated based on probability inference. The way to obtain the first probability value is as follows: divide the number of object fall events when no abnormal feature points appear by the total time involved in the historical data to get the probability of object fall events when no abnormal feature points appear, and divide the product of the probability of the occurrence of abnormal feature points observed when an object falls and the probability of object fall events when no abnormal feature points appear by the frequency of the occurrence of abnormal feature points in the historical data to get the first probability value.

5. The AR intelligent safety helmet assistance system for high-risk working environments according to claim 4, characterized in that: The analysis module further includes: The time series analysis unit is used to input the feature parameters within consecutive time windows in the historical period into the long short-term memory network. The long short-term memory network captures the time series change law of the movement state of the object on the top of the safety helmet through the forget gate, input gate and output gate, and inputs the abnormal feature points into the long short-term memory network to output the second probability value from the output gate.

6. An AR intelligent safety helmet assistance system for high-risk working environments according to claim 5, characterized in that: The analysis module further includes: The fusion unit is used to fuse the results of the Bayesian network and the long short-term memory network using the D-S evidence theory to obtain the fall probability evaluation value. Specifically, the first probability value calculated by the Bayesian network and the second probability value of the long short-term memory network are respectively regarded as evidence sources to obtain the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network, and combined with all possible result sets of the evidence sources. All possible result sets of the evidence sources include fall and non-fall. The D-S evidence synthesis rule is used to fuse the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network to obtain the fall probability evaluation value.

7. An AR intelligent safety helmet assistance system for high-risk working environments according to claim 6, characterized in that: The analysis module further includes: The determination unit is used to compare the fall probability evaluation value with a preset evaluation threshold to generate an instruction signal. If the fall probability evaluation value exceeds the evaluation threshold, it is determined that there is a fall trend, triggering a warning instruction and a buffer mechanism; otherwise, it is determined that there is no fall trend.

8. An AR intelligent safety helmet assistance system for high-risk working environments according to claim 7, characterized in that: The feedback module includes: The prompt unit is used to receive the warning instruction, determine the passable paths around the staff wearing the safety helmet through the camera installed on the safety helmet. If there are multiple groups of passable paths, the optimal path is selected according to the path undulation degree in each passable path. The optimal path is the passable path corresponding to the minimum path undulation degree, and the optimal path is superimposed on the staff's field of vision in the form of an AR virtual arrow through the head-mounted display, and at the same time, the staff is guided to avoid through voice prompts.

9. An AR intelligent safety helmet assistance system for high-risk working environments according to claim 8, characterized in that: The feedback module further includes: The impact and buffer unit is used to monitor the collapse situation during the evasion process through the fiber optic strain sensors adopted inside the safety helmet to obtain the deformation amount, and judge whether to trigger the buffer mechanism again according to the deformation amount. If the deformation amount exceeds the preset deformation threshold, the buffer layer will be triggered. The buffer layer adopts a multi-stage trigger mechanism, including a pre-inflated layer and a rapid expansion layer. The pre-inflated layer is partially inflated when it is determined that there is a falling trend. When the buffer mechanism is triggered again, the rapid expansion layer completes inflation within 50 milliseconds to form a complete buffer structure and absorb the impact energy.

Images