AR intelligent safety helmet auxiliary system for high-risk working environment

Abstract

The application discloses an AR intelligent safety helmet auxiliary system for a high-risk operation environment, and relates to the technical field of construction safety.The system uses a Bayesian network to combine historical data to perform probability reasoning, uses an LSTM network to mine time sequence characteristics, and further uses D-S evidence theory fusion, so that the object falling trend can be more accurately recognized, early warning can be given, and valuable risk avoidance time can be obtained for operation personnel.Meanwhile, the dynamic decision mechanism of the system makes the triggering of the buffer mechanism more reasonable, reduces the occurrence rate of safety accidents, reduces unnecessary resource waste, and provides a scientific and efficient solution for high-risk operation safety protection.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of construction safety, in particular to an AR intelligent safety helmet auxiliary system for a high-risk operation environment. BACKGROUND

[0002] As the most basic and critical safety protection equipment in high-risk operation scenes such as industrial production and construction, the function of the safety helmet is changing from traditional passive protection to intelligent and active early warning. The AR intelligent safety helmet auxiliary system, as a product of cutting-edge technology in this field, aims to provide more intelligent and accurate safety protection and operation assistance for workers in high-risk fields such as construction, mining and high-altitude operation, and has become an important direction for promoting the intelligent upgrading of safety protection equipment.

[0003] However, the existing safety helmet has many shortcomings in dealing with potential risks in high-risk operation environments. For example, the traditional safety helmet only relies on physical structure to resist impact, but in the construction site, safety hazards such as high-altitude falling objects and equipment failure occur frequently. The traditional safety helmet can only provide passive physical protection when an accident occurs, and it is difficult to perceive danger in advance and issue warnings. For sudden object falling, workers often lack timely and effective prompts, making it difficult to make correct avoidance reactions, thereby increasing the risk of injury. SUMMARY

[0004] In view of the above problems existing in the prior art, the application provides an AR intelligent safety helmet auxiliary system for a high-risk operation environment.

[0005] The AR intelligent safety helmet auxiliary system for a high-risk operation environment provided by the embodiments of the present disclosure comprises:

[0006] The acquisition module is used to deploy a monitoring device in the middle of the safety helmet to obtain real-time environmental change data, and to determine abnormal feature points after preprocessing;

[0007] The analysis module is used to perform probability analysis on the abnormal feature points by using historical data and real-time environmental change data to generate an instruction signal;

[0008] The feedback module is used to generate an optimal path when the instruction signal is a warning instruction, superimpose the optimal path in the form of an AR virtual arrow in the field of view of the worker through a head-mounted display, perform avoidance operation, and monitor the actual impact and control the integrity of the buffer structure during the avoidance process.

[0009] Optionally, the acquisition module comprises:

[0010] The acquisition unit is configured to pre-deploy sensor groups, cameras and head-mounted displays on the top, sides and back of the safety helmet to acquire real-time environmental change data, and to perform space-time alignment on the real-time environmental change data, specifically, by connecting GPS receivers to each sensor device to acquire a unified time reference, thereby completing time synchronization, and by using a laser scanner to scan the high-risk work environment site to acquire three-dimensional point cloud data, and by matching and aligning the point cloud data collected by different sensor devices at the same spatial location to complete space registration.

[0011] Optionally, the acquisition module further comprises:

[0012] The processing unit is configured to remove outliers from the real-time environmental change data using a median filtering algorithm, and to extract the relative speed change rate of the object on the top of the safety helmet, the distance change acceleration, the curvature of the object motion trajectory, the trajectory direction mutation angle, the pressure fluctuation amplitude and the pressure fluctuation frequency from the processed real-time environmental change data to generate a real-time feature set.

[0013] The recognition unit is configured to draw a histogram of the historical data based on statistical analysis of the historical data, to determine the distribution shape of the historical data, to determine the model category to which the historical data conforms according to the distribution shape, to calculate the normal distribution interval of each feature parameter extracted by the processing unit according to the model category, and to compare the real-time feature set with the normal distribution interval, and if a feature point in the real-time feature set is outside the normal distribution interval, to mark it as an abnormal feature point, otherwise not to mark it.

[0014] Optionally, the analysis module comprises:

[0015] The conditional analysis unit is configured to construct a Bayesian network based on the historical data, by taking each feature parameter in the historical data as a node and using the conditional probability table trained from the historical data as the weight of the edge between each node, to calculate the probability of observing an abnormal feature point when an object falls and the frequency of occurrence of an abnormal feature point in the historical data, and to count the number of times an object falling event occurs without any abnormal feature point, and when an abnormal feature point appears in the recognition unit, to input the corresponding abnormal feature point into the Bayesian network and calculate a first probability value of the object falling from the top of the safety helmet based on probability reasoning, the first probability value being obtained by dividing the number of times an object falling event occurs without any abnormal feature point by the total time involved in the historical data to obtain the probability of an object falling event occurring without any abnormal feature point, and dividing the product of the probability of observing an abnormal feature point when an object falls and the probability of an object falling event occurring without any abnormal feature point by the frequency of occurrence of an abnormal feature point in the historical data to obtain the first probability value.

[0016] Optionally, the analysis module further comprises:

[0017] The time sequence analysis unit is configured to input the feature parameters in the continuous time windows in the historical time period into the long short-term memory network, and the long short-term memory network is configured to capture the time sequence change rule of the motion state of the object on the top of the safety helmet through the forget gate, the input gate and the output gate, and input the abnormal feature points into the long short-term memory network to output the second probability value from the output gate.

[0018] Optionally, the analysis module further comprises:

[0019] The fusion unit is configured to fuse the results of the Bayesian network and the long short-term memory network by using the D-S evidence theory to obtain the falling probability evaluation value, specifically: taking the first probability value calculated by the Bayesian network and the second probability value of the long short-term memory network as evidence sources to obtain the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network, combining all possible result sets of the evidence sources, the all possible result sets of the evidence sources including falling and non-falling, and using the D-S evidence synthesis rule to fuse the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network to obtain the falling probability evaluation value.

[0020] Optionally, the analysis module further comprises:

[0021] The determination unit is configured to compare the falling probability evaluation value with a pre-set evaluation threshold to generate an instruction signal; if the falling probability evaluation value exceeds the evaluation threshold, it is determined that there is a falling trend, and the warning instruction and the buffer mechanism are triggered, otherwise it is determined that there is no falling trend.

[0022] Optionally, the feedback module comprises:

[0023] The prompt unit is configured to receive the warning instruction, determine the passable paths around the worker wearing the safety helmet through the camera installed on the safety helmet, if there are multiple groups of passable paths, select the optimal path according to the path fluctuation degree in each passable path, the optimal path is the passable path corresponding to the minimum path fluctuation degree, and superimpose the optimal path in the form of AR virtual arrow in the worker's field of view through the head-mounted display, and guide the worker to avoid through voice prompt at the same time.

[0024] Optionally, the feedback module further comprises:

[0025] The impact and buffer unit is used to monitor the collapse condition by the optical fiber strain sensor adopted in the inside of the safety helmet during the avoidance process to obtain the deformation amount, and whether the buffer mechanism is triggered again is judged according to the deformation amount, if the deformation amount exceeds the preset deformation threshold, the buffer layer is triggered, the buffer layer adopts a multi-stage triggering mechanism, including a pre-inflation layer and a rapid expansion layer, the pre-inflation layer is partially inflated when it is determined that there is a falling trend, when the buffer mechanism is triggered again, the rapid expansion layer completes inflation within 50 milliseconds to form a complete buffer structure to absorb impact energy.

[0026] The system uses Bayesian network to combine historical data for probability reasoning, LSTM network to mine time sequence characteristics, and D-S evidence theory fusion to more accurately identify the falling trend of the object and issue an early warning to give valuable risk avoidance time to the operating personnel. Meanwhile, the dynamic decision mechanism of the system makes the triggering of the buffer mechanism more reasonable, reduces the unnecessary waste of resources while reducing the incidence of safety accidents, and provides a scientific and efficient solution for high-risk operation safety protection. The feedback module optimizes path planning and dynamic buffer control to improve the emergency response capability and safety protection level of the high-risk operating personnel when facing the falling risk. In terms of path planning, the prompt unit identifies the passable paths around based on the camera and calculates the undulation of each path by using the distance data obtained by the laser altimeter to accurately select the optimal path with the minimum undulation and guide the operating personnel to evacuate quickly and safely through AR virtual arrows and voice prompts. For example, in the construction site, after the warning instruction is triggered, the operating personnel can quickly obtain clear escape instructions in the complex environment, avoid delaying the escape opportunity due to the selection of rugged and obstacle-rich paths in panic, and reduce the risk of being hit by the falling object. In terms of impact protection, the impact and buffer unit monitors the deformation amount of the safety helmet in real time by means of the optical fiber strain sensor, and realizes efficient resolution of impact force in combination with the multi-stage triggering buffer mechanism. When the deformation amount exceeds the preset threshold, the pre-inflation layer and the rapid expansion layer work together to quickly form a complete buffer structure to effectively absorb impact energy and reduce the impact force on the head, thereby enhancing the protection effect on the operating personnel compared with the passive protection mode of the traditional safety helmet. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application.

[0028] Figure 1 The module diagram of the present application;

[0029] Figure 2 The safety helmet cross-sectional structure diagram of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0031] As shown in Figure 1 The embodiment of the present application proposes an AR intelligent safety helmet auxiliary system for high-risk operation environment, which comprises:

[0032] The acquisition module is used to deploy monitoring devices in the middle of the safety helmet to obtain real-time environmental change data, determine abnormal feature points after preprocessing, and determine abnormal feature points;

[0033] The acquisition unit is used to pre-deploy sensor groups, cameras and head-mounted displays on the top, sides and back of the safety helmet to obtain real-time environmental change data, and to perform spatio-temporal alignment on the real-time environmental change data. Specifically, by connecting GPS receivers to each sensor device, a unified time reference is obtained to complete time synchronization operation, and by using a laser scanner to scan the high-risk operation environment site, three-dimensional point cloud data is obtained. By matching and aligning the point cloud data collected by different sensor devices at the same spatial position, the space registration operation is completed.

[0034] Six-axis acceleration sensors, pressure sensor matrices, laser ranging sensors and high-definition cameras are uniformly deployed on the top, sides and back of the safety helmet. The six-axis acceleration sensor collects acceleration and angular velocity data in three directions, and the pressure sensor matrix is composed of multiple miniature pressure sensors and can capture pressure changes in different areas. The laser ranging sensor scans the area above the operation space in real time to detect the distance of objects, and the high-definition camera collects visual information of the operation environment.

[0035] The high-precision time function of the global positioning system is used to obtain accurate time information by receiving GPS satellite signals. The GPS receiver can provide nanosecond-level time accuracy, and each sensor device connects to the GPS receiver to obtain a unified time reference to achieve time synchronization.

[0036] The processing unit is used to remove outliers from the real-time environmental change data using a median filter algorithm, and to extract the relative speed change rate of objects on the top of the safety helmet, the distance change acceleration, the curvature of the object motion trajectory, the trajectory direction mutation angle, the pressure fluctuation amplitude and the pressure fluctuation frequency from the processed real-time environmental change data to generate a real-time feature set.

[0037] The identification unit is used to draw a histogram of the historical data based on statistical analysis of the historical data, and determine a distribution shape of the historical data, according to the distribution shape, determine a model category to which the historical data conforms, for example, if the data presents a bell-shaped curve similar to a Gaussian distribution, it is possible to conform to a Gaussian mixture model or a Gaussian distribution; or some statistical test methods can be used to judge whether the data conforms to a specific distribution. For example, for a Gaussian distribution, a Shapiro-Wilk test can be used, the null hypothesis of the test is that the data conforms to a normal distribution, if the p-value of the test result is greater than a given significance level, the null hypothesis is accepted, and it is considered that the data conforms to a normal distribution. For a Gaussian mixture model, a likelihood ratio-based test method can be used.

[0038] The normal distribution interval of each feature parameter extracted in the model category calculation processing unit is calculated, and the real-time feature set is compared with the normal distribution interval, if the feature point in the real-time feature set is outside the normal distribution interval, it is marked as an abnormal feature point, otherwise no marking processing is performed.

[0039] The historical data refers to the feature set collected in the historical period.

[0040] In the embodiment of the application, the acquisition module further improves the comprehensiveness and accuracy of safety hazard monitoring in high-risk operation environment through multi-dimensional data acquisition, accurate preprocessing and intelligent abnormality recognition. Compared with the traditional single monitoring method, potential risks can be captured more timely and accurately.

[0041] In the data acquisition link, the acquisition unit deploys sensor groups, cameras and other equipment at multiple positions of the safety helmet, and realizes space-time alignment combined with GPS and laser scanners. Taking mine exploitation as an example, the top pressure sensor, the side displacement sensor and the rear temperature and humidity sensor work cooperatively, cooperate with the camera to capture the scene picture, at the same time, the GPS ensures the time synchronization of each data, and the laser scanner completes the space registration, so that the system constructs a multi-dimensional perception network for the operation environment, avoids the monitoring deviation caused by different data synchronization and space position disorder, and ensures that the acquired environmental data truly reflects the on-site situation.

[0042] The processing unit adopts a median filter algorithm to remove abnormal values, effectively filtering error data generated by device noise, environmental interference and the like. For example, in a construction site, electromagnetic interference generated when a crane is running may cause abnormal fluctuations in sensor data, and the median filter algorithm can quickly eliminate these interference data, and then extract key features such as object relative speed change rate and pressure fluctuation frequency, accurately focus on information related to safety hazards, and provide reliable data support for subsequent analysis.

[0043] The recognition unit determines a normal distribution interval based on a historical data statistical analysis to determine whether real-time data is abnormal. For example, in a high-altitude operation scenario, historical data shows that the curvature of the object motion trajectory fluctuates within a certain range during normal operation. When the real-time monitored trajectory curvature exceeds this range, the system immediately marks it as an abnormal feature point. Compared with manual judgment or simple threshold setting, this judgment method based on data distribution rules can more sensitively detect subtle abnormalities, provide early warning of potential dangers, give workers more time to avoid risks, and significantly reduce the probability of safety accidents.

[0044] Further, the analysis module is configured to utilize historical data and real-time environmental change data to perform a probability analysis on the abnormal feature points to generate an instruction signal. The analysis module includes a conditional analysis unit configured to use historical data to take each feature parameter in the historical data as a node, use a conditional probability table trained from the historical data as the weight of the edge between each node to construct a Bayesian network, and calculate the probability of observing the abnormal feature points when the object falls and the frequency of the abnormal feature points in the historical data according to the historical data, and count the number of times the object falls without any abnormal feature points.

[0045] The probability of observing the abnormal feature points when the object falls can be estimated from the historical data. A large amount of feature parameter data corresponding to the object falling is collected, and the frequency of various value combinations of these feature parameters appearing under the object falling condition is counted to approximate the likelihood probability. For example, the values of the acceleration, displacement, and other feature parameters collected by the sensor are counted in multiple recorded object falling events, and the frequency of different values is calculated.

[0046] The frequency of the abnormal feature points in the historical data can be estimated by statistically analyzing a large amount of historical data (including feature data when the object falls and does not fall) to calculate the frequency of these feature parameters appearing in all data.

[0047] The number of times the object falls without any abnormal feature points is divided by the total time involved in the historical data to obtain the probability of the object falling without any abnormal feature points. This probability can be obtained based on historical statistical data. For example, the number of times the object falls within a certain period of time is counted, and the total number of events (or total time) is divided by the probability of the object falling, which is the prior probability.

[0048] When an abnormal feature point appears in the recognition unit, the corresponding abnormal feature point is input into the Bayesian network, and a first probability value of the object falling on the top of the safety helmet is calculated based on probability reasoning. The first probability value is obtained by dividing the number of times the object falling event occurs without any abnormal feature point by the total time involved in the historical data, obtaining the probability of the object falling event without any abnormal feature point, and dividing the product of the probability of the observed abnormal feature point when the object falls and the probability of the object falling event without any abnormal feature point by the frequency of the abnormal feature point in the historical data, obtaining the first probability value, which is the conditional probability, reflecting the possibility of the object falling actually occurring under the given observation feature condition, combining the new evidence (feature parameter) with the modified probability estimate of the prior knowledge (prior probability).

[0049] The time series analysis unit is used to input the feature parameters in the continuous time window in the historical period into the long short-term memory network (LSTM), and the long short-term memory network captures the time series change rule of the motion state of the object on the top of the safety helmet through the forget gate, the input gate and the output gate, and inputs the abnormal feature point into the long short-term memory network to output a second probability value from the output gate.

[0050] In the LSTM network, the forget gate, the input gate and the output gate are three core components, which solve the gradient disappearance problem in traditional recurrent neural network RNN by selectively retaining, updating and outputting information, so that the network can learn the time dependence in long sequence data. The working principle of them is explained in simple language as follows:

[0051] The forget gate decides which information to discard from the cell state. The forget gate calculates the previous hidden state and the current input through a Sigmoid function (output range 0-1) to get a vector between 0 and 1. Each value in this vector represents the degree to which the corresponding information in the cell state needs to be forgotten (0 means completely forgotten, and 1 means completely retained). For example, suppose the LSTM is analyzing the motion of an object in a video. When the object leaves the screen, the forget gate will forget the previously stored object position information to make room for new object information.

[0052] The input gate decides which new information will be added to the cell state. The input gate is divided into two parts: a Sigmoid function that decides which values need to be updated (output a vector between 0 and 1); and a Tanh function that creates a new candidate vector that may be added to the cell state. Finally, the input gate combines the results of these two parts to selectively update the cell state. For example, continuing the video analysis example above, when a new object enters the screen, the input gate will decide to add the initial position and features of the object as new information to the cell state.

[0053] The output gate decides which information to output from the cell state. The output gate first processes the previous hidden state and the current input through a Sigmoid function to get a vector between 0 and 1, which controls which parts of the cell state will be output. Then, the cell state is processed through a Tanh function (scaling values to -1 to 1) and multiplied by the Sigmoid output vector to get the final output. For example, in video analysis, the output gate will decide which information (such as the location and speed of a specific object) stored in the current cell state should be used to predict the content of the next frame. The forget gate, input gate, and output gate jointly act on the cell state to achieve selective flow of information.

[0054] In a complex environment monitoring and safety warning system, a single model often has limitations and is difficult to comprehensively and accurately capture the characteristics of the target event. For example, in the AR intelligent safety hat falling trend judgment scene, although the Bayesian network can perform probability reasoning based on prior knowledge, it has weak ability to capture time sequence characteristics. While the LSTM network is good at processing time-dependent relationships of sequence data, it lacks effective use of domain knowledge. Therefore, the outputs of multiple complementary models need to be effectively fused to improve the accuracy and reliability of decision-making.

[0055] The analysis module further includes a fusion unit configured to fuse the results of the Bayesian network and the long short-term memory network using D-S evidence theory to obtain the falling probability evaluation value. Specifically, the fusion unit is configured to take the first probability value calculated by the Bayesian network and the second probability value of the long short-term memory network as evidence sources, to obtain a basic probability assignment value constructed based on the calculation result of the Bayesian network and a basic probability assignment value constructed based on the calculation result of the long short-term memory network, and to combine all possible result sets of the evidence sources, including falling and non-falling, to fuse the basic probability assignment value constructed based on the calculation result of the Bayesian network and the basic probability assignment value constructed based on the calculation result of the long short-term memory network using D-S evidence synthesis rules to obtain the falling probability evaluation value.

[0056] First, the maximum likelihood estimation or Bayesian estimation method is used to construct the feature parameter nodes and the conditional probability table. For the to-be-detected data, i.e., the abnormal feature points, the first probability value and the probability value of the object not falling under the given observation feature condition are calculated. Then, the first probability value output by the Bayesian network is converted into a basic probability assignment function m1, specifically: m1({falling}) = the first probability value, m1({non-falling}) = the probability value of the object not falling under the given observation feature condition, and m1(Θ) = 0. ​is an uncertainty measure, reflecting the uncertainty of the Bayesian network in its own judgment. Then, the LSTM network is trained using historical time series data to learn the time dependence of the feature parameters. For the time series data window to be detected, i.e. the time series data window corresponding to the abnormal feature point, the second probability value, i.e. the probability of the object falling in the future time, is predicted, and the predicted probability output by the LSTM network is converted into the basic probability assignment function m2, specifically: m2({falling}) = the second probability value, m2({non-falling}) = 1-the second probability value, m2(Θ) = 0, , is an uncertainty measure, , is a preset constant (0 <1), is a dynamic adjustment factor determined according to the performance of the LSTM model on the validation set, is the second probability value; then m1 and m2 are fused using the D-S evidence combination rule: wherein, is marked as the falling probability evaluation value, and K is a normalization constant, is marked as the basic probability assignment value constructed based on the calculation result of the Bayesian network, i.e. the basic probability assignment value of event A1 in the first evidence source (the basic probability assignment function obtained by converting the result of the Bayesian network), is marked as the basic probability assignment value constructed based on the calculation result of the long short-term memory network, i.e. the basic probability assignment value of event A2 in the second evidence source (the basic probability assignment function obtained by converting the result of the LSTM network), A1 is an element in the event set involved in the first evidence source, and A2 is an element in the event set involved in the second evidence source, A is marked as and the intersection of and; the normalization constant K serves to ensure that the sum of the basic probability assignment function values after fusion is 1, avoiding deviation in the total probability value caused by the calculation process.

[0057] As an uncertainty reasoning method, the D-S evidence theory can handle incomplete information and uncertainty, and provides a theoretical basis for multi-source information fusion.

[0058] In the D-S evidence theory, the basic probability assignment function m is a mapping from the power set to [0, 1], satisfying Σm(A) = 1, where A Θ, Θ is the set of all possible results;

[0059] The evidence combination rule is the core formula of the D-S evidence theory, which is used to fuse the basic probability assignment functions of multiple independent evidence sources.

[0060] The analysis module further comprises a determination unit configured to compare the falling probability evaluation value with a pre-set evaluation threshold to generate an instruction signal; if the falling probability evaluation value exceeds the evaluation threshold, it is determined that there is a falling trend, and a warning instruction and a buffer mechanism are triggered; otherwise, it is determined that there is no falling trend.

[0061] The instruction signal comprises a warning instruction triggering signal and a warning instruction non-triggering signal.

[0062] In the embodiment of the present application, the analysis module constructs a high-precision risk evaluation system through multi-model cooperation and deep data fusion, improves the pre-judgment ability of object falling risk in high-risk operation environment, and can more scientifically and accurately identify potential dangers compared with traditional single model analysis. In the condition analysis unit, a Bayesian network is constructed based on historical data, the feature parameters are taken as nodes, and the conditional probability table is taken as edge weights to quantify the probability correlation between each feature and object falling. For example, in a construction scene, past data shows that when the inclination angle of the tower crane suddenly changes (an abnormal feature point), the object falling probability is 40%, the frequency of the abnormal feature point in the historical data is 15%, and a falling event occurs once every 50 hours when there is no abnormal feature point. When the inclination angle of the tower crane is suddenly changed in real time, the first probability value is calculated through the Bayesian network, the prior knowledge and real-time evidence are combined to update the judgment of the falling risk, and the probability level basis for risk evaluation is provided.

[0063] The time sequence analysis unit uses an LSTM network to deeply mine the time sequence rules of the object motion state by using the information filtering, updating mechanism of the forget gate, input gate and output gate for the feature parameters in the historical time period continuous time window. For example, in high-altitude power maintenance operation, the LSTM network continuously learns the time sequence changes of feature parameters such as the vibration, displacement and the like of equipment components such as insulators and cable joints. When an abnormal fluctuation (an abnormal feature point) of the vibration frequency of a component is monitored, the LSTM network outputs a second probability value of falling of the component according to the learning of the historical time sequence data, effectively captures the dynamic trend of the object motion, and makes up for the deficiency of the Bayesian network in processing time sequence information.

[0064] The fusion unit adopts D-S evidence theory, takes the first probability value calculated by the Bayesian network and the second probability value output by the LSTM network as independent evidence sources, converts them into basic probability assignment values, and then fuses them. For example, under certain working conditions, the Bayesian network obtains a falling probability of 0.5, and the LSTM network obtains a falling probability of 0.6, and there is a difference between them. Through the D-S evidence synthesis rule, the judgments of the two models are comprehensively considered to eliminate the contradictions and uncertainties between the evidences, obtain a falling probability evaluation value that is more in line with the actual situation, and avoid misjudgment of a single model.

[0065] Finally, the determination unit compares the fused falling probability evaluation value with a preset threshold value, and if the threshold value is exceeded, it is determined that there is a falling trend, and a warning instruction and a buffer mechanism are triggered; otherwise, it is determined to be safe.

[0066] Further, the feedback module is configured to generate an optimal path when the instruction signal is a warning instruction, superimpose the optimal path in the form of an AR virtual arrow in the field of view of the worker through a head-mounted display, and perform an avoidance operation, and monitor the actual impact and control the integrity of the buffer structure during the avoidance process.

[0067] The feedback module includes a prompt unit configured to receive a warning instruction, determine passable paths around a worker wearing a safety helmet through a camera installed on the safety helmet, select an optimal path according to path undulations in each passable path if there are multiple passable paths, superimpose the optimal path in the form of an AR virtual arrow in the field of view of the worker through a head-mounted display, and guide the worker to avoid through voice prompts, to prompt the worker to immediately leave the current position to avoid the risk of falling objects from above.

[0068] The laser altimeter calculates the distance by emitting laser light and measuring the time of reflected light, and the height changes little when the path is flat, and the height changes significantly when the path has slope undulations, and the height change is used as the path undulation;

[0069] The passable path represents a path that can be passed around the current position;

[0070] The safety helmet includes an outer shell, a framework, a cavity, and an inner support shell, wherein the framework is arranged in the inner cavity of the safety helmet and is arranged against the outer shell, the space between the framework and the inner support shell is the cavity, and the cavity is used to trigger a buffer layer, the buffer layer is used to form an airbag to form a complete buffer structure to absorb impact energy. Figure 2 As can be seen, the safety helmet has an outer shell, a framework, a cavity, and an inner support shell, wherein the framework is arranged in the inner cavity of the safety helmet and is arranged against the outer shell, the space between the framework and the inner support shell is the cavity, and the cavity is used to trigger a buffer layer, the buffer layer is used to form an airbag to form a complete buffer structure to absorb impact energy.

[0071] The feedback module further includes an impact and buffer unit configured to monitor the collapse during the avoidance process through a fiber optic strain sensor inside the safety helmet to obtain a deformation amount, determine whether to trigger the buffer mechanism again according to the deformation amount, and if the deformation amount exceeds a preset deformation threshold, trigger the buffer layer, the buffer layer uses a multi-stage triggering mechanism and includes a pre-charging layer and a rapid expansion layer, the pre-charging layer is partially inflated when it is determined that there is a falling trend, and the rapid expansion layer completes inflation within 50 milliseconds when the buffer mechanism is triggered again, forming a complete buffer structure to absorb impact energy.

[0072] The optical fiber strain sensor is laid along the safety helmet framework. When the safety helmet is impacted by external force, the deformation of the framework causes the optical fiber to be stretched or compressed, and the change of the optical fiber strain causes the wavelength, light intensity, phase and other characteristics of the light propagating in the optical fiber to change. Before laying the optical fiber strain sensor, the safety helmet framework needs to be mechanically analyzed and experimentally calibrated to establish a mathematical relationship model between the strain and the deformation amount of the framework. The strain-deformation amount curve can be obtained by simulating the strain and deformation of each part of the framework of the safety helmet under different stress conditions through the finite element analysis software. At the same time, the actual loading experiment is carried out on the safety helmet, and a high-precision displacement measuring device (such as a laser displacement sensor) is arranged on the framework to synchronously measure the strain (obtained by the optical fiber strain sensor) and the deformation amount of the framework during the loading process. The empirical formula of the strain and the deformation amount is fitted through the experimental data. For example, for a safety helmet framework of a certain material and structure, it is obtained through experiments and analysis that within a certain strain range, the deformation amount of a part of the framework and the optical fiber strain are related as follows wherein k is a proportional coefficient determined through experiments, is the original length of the part under no stress, is the optical fiber strain, is the deformation amount of the part of the framework.

[0073] In the embodiment of the application, the feedback module constructs a comprehensive and multi-level safety protection system through path planning, navigation guidance and dynamic buffer protection, thereby improving the emergency escape ability and safety protection level of the operating personnel in a high-risk environment. When the warning instruction is issued by the judgment unit, the prompt unit is immediately started. The camera on the safety helmet recognizes the environment around the operating personnel to obtain passable path information;

[0074] The height change data of each path is measured by using the laser altimeter to calculate the path undulation. The optimal path is selected according to the minimum path undulation as the standard, which is superimposed on the head-mounted display in the form of an AR virtual arrow, and is supplemented by voice prompts to guide the operating personnel to quickly evacuate the dangerous area. For example, in the high-altitude operation scene on the construction site, when there is a risk of object falling from above, the operating personnel may not be able to quickly find a safe escape path due to panic in the traditional way, or the selected path may have obstacles, a large slope and other problems, resulting in low escape efficiency. The system can plan the optimal path in an instant, for example, when a part loosening warning occurs near the tower crane, the system can avoid the rugged path full of materials and guide the operating personnel to evacuate along the flat and unobstructed path, effectively reducing the delay and secondary injury risk caused by improper path selection, and improving the timeliness and safety of emergency escape.

[0075] In the process of the worker avoiding, the impact and buffer unit monitors the collapse deformation of the safety helmet in real time by using the optical fiber strain sensor inside the safety helmet. When the deformation detected by the sensor exceeds the preset deformation threshold, the multi-stage triggering mechanism of the buffer layer is triggered. The pre-inflatable layer is partially inflated in the early warning stage, and at this time the rapid expansion layer rapidly completes inflation within 50 milliseconds to form a complete structure with strong buffering capacity for absorbing the impact energy of the falling object and protecting the worker's head.

[0076] For example, in the mining operation, even if the worker has taken evasive measures, he or she may still be impacted by falling ore. The buffering capacity of the traditional safety helmet is limited and it is difficult to resist larger impact forces. However, the dynamic buffering mechanism of the system can accurately respond to the actual impact situation. For example, when the ore hits the safety helmet, the optical fiber strain sensor immediately captures the deformation of the safety helmet. If the deformation exceeds the standard, the rapid expansion layer inflates rapidly, greatly reducing the impact force that would otherwise cause serious head injuries. It is like instantly supporting a safety airbag for the head, effectively reducing the degree of head injury. Compared with the passive and fixed protection mode of the traditional safety helmet, the protection effect for the worker is improved.

[0077] The feedback module realizes the whole-process protection from efficient escape guidance after risk warning to intelligent protection when impacted through the above two closely coordinated links, builds a solid and reliable defense line for the life safety of high-risk workers, and improves the intelligent level and actual application effectiveness of safety protection equipment.

[0078] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An AR smart helmet assistance system for high-risk working environments, characterized by: include: The acquisition module is used to deploy monitoring equipment in the helmet to obtain real-time environmental change data, and determine abnormal feature points after preprocessing; The analysis module is used to use historical data and real-time environmental change data to perform probability analysis on abnormal feature points to generate command signals; The analysis module includes a condition analysis unit, a timing analysis unit, a fusion unit and a judgment unit; The conditional analysis unit is used to construct a Bayesian network based on historical data, using feature parameters as nodes and conditional probability tables as weights of edges between nodes. When an abnormal feature point appears, the corresponding abnormal feature point is input into the Bayesian network, and the probability value of the object falling on the top of the helmet is calculated based on probabilistic reasoning; The time series analysis unit is used to input the characteristic parameters in the continuous time window of the historical period into the long short-term memory network. The long short-term memory network captures the temporal change pattern of the motion state of the object on the top of the helmet through the forget gate, input gate and output gate, and inputs the abnormal feature points into the long short-term memory network to output the second probability value from the output gate; The fusion unit is used to fuse the results of the Bayesian network and the long short-term memory network using the DS evidence theory to obtain the fall probability assessment value; The determination unit is used to compare the fall probability assessment value with a preset assessment threshold value and generate a command signal; If the fall probability assessment value exceeds the assessment threshold, it is determined that there is a fall trend, triggering the warning instruction and buffer mechanism; otherwise, it is determined that there is no fall trend; The feedback module is used to generate the optimal path when the command signal is a warning command, and superimpose the optimal path in the form of an AR virtual arrow in the staff's field of view through a head-mounted display to perform evasive operations. During the evasive process, the module also monitors the actual impact and controls the integrity of the buffer structure.

2. The AR smart helmet assistance system for high-risk working environments according to claim 1 is characterized by: The acquisition module includes: The acquisition unit is used to pre-deploy sensor groups, cameras and head-mounted displays on the top, sides and rear of the safety helmet to obtain real-time environmental change data and align the real-time environmental change data in time and space. Specifically, each sensor device is connected to a GPS receiver to obtain a unified time base to complete the time synchronization operation, and a laser scanner is used to scan the high-risk working environment site to obtain three-dimensional point cloud data. The point cloud data collected by different sensor devices at the same spatial position are matched and aligned to complete the spatial registration operation.

3. The AR smart helmet assistance system for high-risk working environments according to claim 2, characterized in that: The acquisition module also includes: The processing unit is used to remove outliers from the real-time environmental change data using a median filtering algorithm, and extract the relative velocity change rate of the object on the top of the helmet, the distance change acceleration, the curvature of the object's motion trajectory, the angle of sudden change in trajectory direction, the pressure fluctuation amplitude, and the pressure fluctuation frequency from the processed real-time environmental change data to generate a real-time feature set; The recognition unit is used to draw a histogram of the historical data based on the statistical analysis of the historical data, and determine the distribution shape of the historical data. According to the distribution shape, the model category to which the historical data obeys is determined. According to the model category, the normal distribution interval of each feature parameter extracted from the calculation processing unit is calculated, and the real-time feature set is compared with the normal distribution interval. If the feature point in the real-time feature set is outside the normal distribution interval, it will be marked as an abnormal feature point, otherwise it will not be marked.

4. The AR smart helmet assistance system for high-risk working environments according to claim 3 is characterized by: Analysis modules include: The conditional analysis unit is used to construct a Bayesian network based on historical data, using each feature parameter in the historical data as a node, and using the conditional probability table obtained by training with the historical data as the weight of the edge between each node. The conditional analysis unit calculates the probability of occurrence of abnormal feature points observed when the object falls and the frequency of occurrence of abnormal feature points in the historical data based on the historical data, and counts the number of object falling events when no abnormal feature points appear. When an abnormal feature point appears in the recognition unit, the corresponding abnormal feature point is input into the Bayesian network, and the probability value No. 1 of the object falling on the top of the safety helmet is calculated based on probabilistic reasoning. The method of obtaining the No. 1 probability value is as follows: the number of object falling events when no abnormal feature points appear is divided by the total time involved in the historical data to obtain the probability of the object falling event when no abnormal feature points appear, and the product of the probability of occurrence of abnormal feature points observed when the object falls and the probability of the object falling event when no abnormal feature points appear is divided by the frequency of occurrence of abnormal feature points in the historical data to obtain the No. 1 probability value.

5. The AR smart helmet assistance system for high-risk working environments according to claim 4, characterized in that: The analysis module also includes: The fusion unit is used to fuse the results of the Bayesian network and the long short-term memory network using the DS evidence theory to obtain a fall probability assessment value. Specifically, the probability value No. 1 calculated by the Bayesian network and the probability value No. 2 of the long short-term memory network are used as evidence sources to obtain a basic probability distribution value constructed based on the calculation results of the Bayesian network and a basic probability distribution value constructed based on the calculation results of the long short-term memory network, and combined with all possible result sets of the evidence source, all possible result sets of the evidence source include falling and non-falling, and the basic probability distribution value constructed based on the calculation results of the Bayesian network and the basic probability distribution value constructed based on the calculation results of the long short-term memory network are used to obtain a fall probability assessment value.

6. The AR smart helmet assistance system for high-risk working environments according to claim 5, characterized in that: The feedback module includes: The prompt unit is used to receive early warning instructions and determine the passable paths around the worker wearing the safety helmet through the camera installed on the safety helmet. If there are multiple sets of passable paths, the optimal path is selected according to the path undulations in each passable path. The optimal path is the passable path corresponding to the minimum path undulation, and the optimal path is superimposed on the worker's field of view in the form of an AR virtual arrow through the head-mounted display, and the worker is guided to avoid it through voice prompts.

7. The AR smart helmet assistance system for high-risk working environments according to claim 6, characterized in that: The feedback module also includes: The impact and buffering unit is used to monitor the collapse situation during the avoidance process through the optical fiber strain sensor used inside the helmet to obtain the deformation amount, and determine whether to trigger the buffering mechanism again based on the deformation amount. If the deformation exceeds the preset deformation threshold, the buffering layer is triggered. The buffering layer adopts a multi-stage triggering mechanism, including a pre-inflated layer and a rapid expansion layer. The pre-inflated layer is partially inflated when it is determined that there is a falling trend. When the buffering mechanism is triggered again, the rapid expansion layer completes the inflation within 50 milliseconds, forming a complete buffering structure to absorb impact energy.

Images