Construction site-oriented multivariate heterogeneous internet-of-things sensing data fusion method

Abstract

The invention belongs to the technical field of data fusion, and provides a multivariate heterogeneous Internet of Things sensing data fusion method for a construction site, and the method comprises the steps: obtaining a node abnormal state factor through analyzing the transformation condition of a position coordinate of a node of a worker in local frame data; and analyzing the change irregularity between the accelerations of all the nodes of the staff in the local frame data to obtain the relative abnormal degree of the nodes, and integrating the abnormal state factors of the nodes and the difference between the relative abnormal degree of the nodes and historical data to obtain the abnormal working state degree of the construction site. And combining the abnormal working state degree, the equipment risk factor and the project progress risk factor to assess the possibility of sudden danger on the construction site, and carrying out detection and early warning on the danger on the construction site. According to the comprehensive assessment, risk points of the construction site can be more comprehensively grasped, the accuracy of sudden danger prediction is improved, more accurate early warning and decision making are made, and misjudgment caused by a single data source is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data protection technology, and in particular to a method for fusing multi-heterogeneous IoT perception data for construction sites. Background Art

[0002] With the continuous expansion of the scale of construction projects and the increasing complexity of construction site management, tasks such as safety management, quality monitoring, and resource allocation at the construction site are increasingly becoming key factors in determining whether the project can proceed smoothly. In modern construction sites, a variety of sensors and devices are used to monitor the site environment, equipment status, personnel location, construction progress and other information in real time. These devices and sensors constitute a huge Internet of Things (IoT) system. In order to ensure efficient management and safe operation of the construction site, it is necessary to effectively integrate and analyze the multi-dimensional heterogeneous data from different sensors and devices to provide accurate and real-time decision support.

[0003] Workers are the core resource of the construction site, so data related to workers is crucial; and the construction site often changes dynamically, and the location and activities of workers may change at any time. Video data provides managers with continuous monitoring information. Through video data, the behavior, location and status of workers can be obtained in real time, which can help assess the safety risks on site. However, video data can only reflect the behavior and status of workers, and the dangers on the construction site are often the result of the combined effect of multiple factors. Relying solely on the abnormal behavior of workers in video data to assess the possibility of sudden dangers on the construction site has limitations, resulting in inaccurate predictions of sudden dangers. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a multi-heterogeneous IoT perception data fusion method for construction sites.

[0005] According to a method for fusion of multi-heterogeneous IoT sensing data for construction sites provided by the present invention, the method comprises: Collect video data of construction site workers, identify the node of each worker, obtain the position coordinates and acceleration of each node, and obtain equipment status data and project progress data at the construction site; Based on the position coordinates of the nodes, analyzing the transformation of the position coordinates of the nodes of each staff member in the local frame data of the video data, and obtaining the abnormal state factor of the nodes of each staff member; Based on the acceleration of the node, analyzing the irregularity of the acceleration changes between all nodes of each staff member in the local frame data of the video data, and obtaining the relative abnormality degree of the node of each staff member; The differences of the node abnormal state factors and the relative abnormal degree of the nodes of all workers are analyzed between the current local frame data and the historical data to obtain the abnormal working state degree of the construction site; According to the equipment status data and the project progress data, using a neural network, obtaining equipment risk factors and project progress risk factors; According to the degree of the abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained, and the danger at the construction site is detected and warned.

[0006] In some embodiments of the present invention, the transformation of the position coordinates of each worker's node in the local frame data of the video data is analyzed, and the process also includes: identifying the unstable node of each worker.

[0007] In some embodiments of the invention, identifying the unstable nodes of each worker includes: Recording all frame data before the current local frame data in the video data as historical data; Preset variance threshold; For any node of any staff member, if the variance of the distance between any two position coordinates of the node in the historical data is less than the variance threshold, the node is recorded as a stable node, otherwise, the node is recorded as an unstable node.

[0008] In some embodiments of the present invention, based on the position coordinates of the node, the transformation of the position coordinates of the node of each staff member in the local frame data of the video data is analyzed to obtain the node abnormal state factor of each staff member, including: Based on the position coordinates of the nodes, analyzing the Euclidean distance of the position coordinates of the unstable nodes of each staff member between any two adjacent frames in the local frame data of the video data, and obtaining a distance measure of the position coordinates; Based on the position coordinates of the nodes, the standard deviations of all the position coordinates of the unstable nodes of each staff member in the local frame data of the video data in the X-axis and Y-axis directions are analyzed to obtain the degree of discreteness of the position coordinates; According to the distance metric and the discrete degree, the transformation of the position coordinates of the unstable node of each staff member in the local frame data of the video data is obtained, and the node abnormal state factor of each staff member is obtained.

[0009] In some embodiments of the present invention, based on the acceleration of the node, the irregularity of the acceleration changes between all nodes of each staff member in the local frame data of the video data is analyzed to obtain the relative abnormality degree of the node of each staff member, including: Based on the acceleration of the node, analyze the The staff member The node is the first in the local frame data The acceleration anomaly factor at the frame time; Based on the acceleration anomaly factor, analyze the The irregularity of the acceleration changes of all nodes of a worker in the local frame data of the video data is analyzed to obtain the relative abnormality degree of each worker's node.

[0010] In some embodiments of the present invention, based on the acceleration of the node, analyzing the The staff member The node is the first in the local frame data Acceleration anomaly factors at the frame time, including: Based on the acceleration of the node, identify the The staff member All abnormal accelerations of nodes in the video data are obtained to obtain the average value of all the abnormal accelerations; According to The staff member The node is the first in the local frame data The acceleration at the frame time is combined with the mean value of the abnormal acceleration to obtain the The staff member The node is the first in the local frame data The acceleration anomaly factor at frame time.

[0011] In some embodiments of the present invention, based on the acceleration anomaly factor, the first The irregularity of the acceleration changes between all nodes of a worker in the local frame data of the video data includes: The first The staff member The acceleration anomaly factors of the nodes in all frames in the local frame data constitute a sequence, recorded as The staff member The acceleration time series of each node; The first All nodes of the staff are combined in any pairs to obtain node combinations; Analysis The Pearson correlation coefficient between the acceleration time series corresponding to two nodes in all the node combinations corresponding to the staff member is obtained. The relative abnormality of the nodes of each worker.

[0012] In some embodiments of the present invention, the differences in the node abnormal state factors and the differences in the relative abnormal degree of the nodes of all workers are analyzed respectively between the current local frame data and the historical data to obtain the abnormal working state degree of the construction site, including: Recording all frame data before the current local frame data in the video data as historical data; Based on the historical data, the mean values ​​of the abnormal state factors of the historical nodes and the relative abnormal degree of the historical nodes of all staff members are obtained; Analyze the difference between the node abnormal state factor of each worker and the mean value of the historical node abnormal state factor, and analyze the difference between the node relative abnormality degree of each worker and the mean value of the historical node relative abnormality degree, traverse all workers, and obtain the degree of abnormal working status at the construction site.

[0013] In some embodiments of the present invention, according to the equipment status data and the engineering progress data, using a neural network, obtaining equipment risk factors and engineering progress risk factors includes: Inputting the equipment status data and the project progress data into the trained CNN neural network respectively to obtain equipment risk factors and project progress risk factors; The training method of the CNN neural network is as follows: collect a large amount of equipment status data or project progress data at the construction site, score each equipment status data or project progress data, and record the scoring result as the label of each equipment status data or project progress data; the equipment status data or project progress data and their corresponding labels constitute a data set; use the data set to train the CNN neural network to obtain the equipment status data training neural network or the project progress data training neural network, and the loss function used in the training process is the cross loss function.

[0014] In some embodiments of the present invention, according to the degree of abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained, and the danger at the construction site is detected and warned, including: According to the degree of the abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained; Preset likelihood threshold; Determine whether the possibility of sudden danger at the construction site is greater than or equal to the possibility threshold; If so, it is determined that a sudden dangerous incident has occurred at the construction site at the current moment, and an alarm is sent immediately to remind relevant personnel to handle it in a timely manner.

[0015] It can be seen from the above embodiments that the multi-heterogeneous IoT sensing data fusion method for construction sites provided by the embodiments of the present invention has the following beneficial effects: 1. By analyzing the transformation of the position coordinates of the worker's node in the local frame, the node abnormal state factor of each worker is obtained. Considering that when a fault occurs, the worker may suddenly stop or start to move quickly, and in order to avoid dangerous areas, their movement trajectory is more chaotic, it provides a stable and reliable data basis for the subsequent analysis of abnormal working conditions at the construction site; 2. By analyzing the irregularity of the acceleration changes of the workers' nodes in the local frame, the relative abnormality of each worker's node is obtained. Considering that when a fault occurs, the workers may have abnormal body postures due to faults or dangerous actions, and the acceleration of different body parts may be different, resulting in irregular changes in the overall acceleration, providing a stable and reliable data basis for the subsequent analysis of abnormal working conditions at the construction site; 3. Compare historical data, analyze the differences in node abnormal state factors and node relative abnormality levels of workers, and obtain the degree of abnormal working state of the construction site. By fusing real-time image data and acceleration data and comparing the differences with the normal working state of workers in historical data, the abnormal working state of workers can be accurately and effectively identified; 4. By combining the abnormal working status of staff, equipment status data and project progress data, we can more comprehensively grasp the risk points of the construction site, so as to make more accurate warnings and decisions. This comprehensive assessment helps to improve the accuracy of sudden danger predictions and reduce misjudgments caused by a single data source.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of the basic flow of a method for fusion of multi-heterogeneous IoT perception data for construction sites provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a multi-heterogeneous IoT sensing data fusion method for a construction site proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. Terms such as "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of further restrictions, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the article or device including the element.

[0021] The following will introduce in detail a method for fusion of multi-heterogeneous IoT perception data for construction sites provided by this embodiment in conjunction with the accompanying drawings.

[0022] See also Figure 1 , which shows the basic process of a multi-heterogeneous IoT perception data fusion method for a construction site provided by an embodiment of the present invention.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a method for fusion of multi-heterogeneous IoT sensing data for a construction site, which specifically includes the following steps: S100: Collect video data of workers at the construction site, identify the node of each worker, obtain the position coordinates and acceleration of each node, and simultaneously obtain equipment status data and project progress data at the construction site.

[0024] The data sources of the construction site usually include: video data of the construction site, equipment status data of the construction site, project progress data of the construction site, etc.

[0025] Therefore, in this embodiment, several workers at the construction site are filmed using the monitoring of the construction site to obtain video data of the construction site. For any worker in the video data of the construction site, the head, shoulders, arms and legs of the worker are identified using a posture estimation algorithm, all of which are used as the nodes of the worker, and then all the nodes of each worker are obtained. Then, video analysis software such as Kinovea is used to obtain the position coordinates and acceleration of each node as the feature data of each worker. All nodes of each worker are arranged in order from top to bottom, and the feature data of all the arranged nodes are used as the behavior data of each worker.

[0026] At the same time, the equipment management software is used to monitor the equipment status in real time and obtain the equipment status data of the construction site. By combining the BIM model with the actual construction progress data, the project progress data of the construction site can be obtained.

[0027] At this point, multivariate heterogeneous data of the construction site have been obtained.

[0028] Since workers are the core resources of the construction site, and the construction site often changes dynamically, the location and activities of workers may change at any time, so the data related to workers is crucial. Video data provides managers with continuous monitoring information. Through video data, the behavior, location and status of workers can be obtained in real time; and acceleration data can reflect the most realistic behavior of workers in dangerous areas, so that the abnormal behavior of workers can be more accurately inferred, which can help assess the safety risks of the construction site. However, relying solely on the abnormal behavior of workers in video data to assess the possibility of sudden dangers at the construction site has limitations, because video data can only reflect the behavior and status of workers, and the dangers at the construction site are often the result of the combined effect of multiple factors. Therefore, by combining equipment status data and project progress data, the risk points of the construction site can be more comprehensively grasped, so as to make more accurate warnings and decisions. This method of integrating multi-heterogeneous data and comprehensive evaluation helps to improve the accuracy of sudden danger prediction and reduce misjudgments caused by a single data source.

[0029] Based on the above analysis, in an embodiment of the present invention, a method for detecting and warning of dangers at a construction site by fusing multivariate heterogeneous data includes steps S200 to S500.

[0030] S200: Based on the position coordinates of the nodes, the transformation of the position coordinates of the nodes of each worker in the local frame data of the video data is analyzed to obtain the node abnormal state factor of each worker.

[0031] For any frame in the video data, local video data consisting of the first 10 frames and the last 10 frames of the frame are recorded as local frame data.

[0032] When a fault occurs, the worker may suddenly stop or start to move quickly. For example, if a machine or equipment fails, the worker may quickly turn to check the equipment or jump away, resulting in drastic changes in the coordinates of the head, shoulders, arms, and legs. Therefore, based on the position coordinates of the nodes, the transformation of the position coordinates of each worker's node in the local frame data of the video data is analyzed to obtain the node abnormal state factor of each worker.

[0033] Since some of the nodes of the staff are constantly undergoing stable transformations due to work reasons when they are working, when analyzing these stable transformation nodes in the local frame data, the local frame data is short in time and will change itself, which leads to the mistaken belief that the position changes of these nodes are due to the dangerous state of the construction site, resulting in low credibility of the abnormal state of the staff's nodes. Therefore, before analyzing the transformation of the position coordinates of each staff member's node in the local frame data of the video data, it also includes: identifying the unstable nodes of each staff member. The specific implementation method is: all frame data before the current local frame data in the video data are recorded as historical data; a variance threshold is preset, and the value can be 0.2; for any node of any staff member, if the variance of the distance between any two position coordinates of the node in the historical data is less than the variance threshold (less than 0.2), or all the position coordinates of the node in the historical data constitute a curve, and the variance of the distance between two data points in the curve is less than 0.2, then the node is recorded as a stable node, otherwise, the node is recorded as an unstable node. Thus, all stable nodes and unstable nodes of each staff member can be obtained. A stable node is a node in which the worker's historical data under normal conditions has been undergoing stable position changes due to work reasons.

[0034] In order to improve the credibility of the abnormal node status of the staff, in some embodiments of the present invention, based on the position coordinates of the nodes, the transformation of the position coordinates of the unstable nodes of each staff member in the local frame data of the video data is analyzed to obtain the abnormal node status factor of each staff member. The specific implementation method is as follows: First, for any local frame data staff members, the adjacent frames The larger the distance between the position coordinates of the unstable nodes of a worker, the faster the worker moves in a very short time, that is, the greater the degree of mutation of the node position of the worker. Therefore, based on the position coordinates of the nodes, the Euclidean distance of the position coordinates of the unstable nodes of each worker between any two adjacent frames in the local frame data of the video data is analyzed to obtain the distance measure of the position coordinates.

[0035] Then, under normal circumstances, some workers may move quickly in a short period of time due to special work, but the movements will be relatively stable and regular; when a fault occurs, the movement trajectory of the workers may become more irregular and messy; the larger the standard deviation in the local frame data, the greater the fluctuation of the worker's movement trajectory, which may be due to the workers' movement to avoid danger, causing their movement trajectory to be more messy. Therefore, based on the position coordinates of the nodes, the standard deviation of all position coordinates of each worker's unstable node in the local frame data of the video data in the X-axis and Y-axis directions is analyzed to obtain the discrete degree of the position coordinates.

[0036] According to the distance metric and the degree of discreteness, the transformation of the position coordinates of the unstable node of each staff member in the local frame data of the video data is obtained. Furthermore, according to the transformation of the position coordinates of the unstable node of each staff member in the local frame data of the video data, the node abnormal state factor of each staff member is obtained. The calculation formula for the node abnormal status factor of a worker is: In the formula, Indicates the local frame data The node abnormal status factor of each worker; Indicates the number of all frame images in the local frame data; Indicates The number of unstable nodes per worker; Indicates The staff member in The first The position coordinates of unstable nodes; Indicates The staff member in The first The position coordinates of unstable nodes; Represents two coordinate positions ( and ) between ; Indicates The staff member The standard deviation of all position coordinates of the unstable nodes in the local frame data of the video data in the X-axis direction; Indicates The staff member The standard deviation of all position coordinates of the unstable nodes in the local frame data of the video data in the Y-axis direction; represents the linear normalization function.

[0037] The distance between the position coordinates of the worker's unstable nodes in adjacent frames The larger the value, the faster the worker moves in a short period of time, that is, the greater the mutation degree of the worker's node position. The larger the value, the greater the fluctuation of the staff's movement trajectory. It may be that the staff moves to avoid danger and their movement trajectory is relatively messy in order to avoid dangerous areas. Therefore, the messier the movement trajectory and the greater the degree of node position mutation, the greater the node abnormal state factor of the staff.

[0038] S300: Based on the acceleration of the nodes, analyze the irregularity of the acceleration changes of all nodes of each worker in the local frame data of the video data to obtain the relative abnormality degree of the nodes of each worker.

[0039] In this embodiment, based on the acceleration of the nodes, the irregularity of the acceleration changes of all nodes of each staff member in the local frame data of the video data is analyzed to obtain the relative abnormality of the nodes of each staff member. The staff member The node is the first in the local frame data The acceleration anomaly factor at the time of the frame; Based on the acceleration anomaly factor, analyze the The irregularity of the acceleration changes of all nodes of a worker in the local frame data of the video data is analyzed to obtain the relative abnormality degree of each worker's node.

[0040] When a fault occurs, the speed of the staff may suddenly slow down or stop due to injury, shock and other factors. For example, in the case of an explosion caused by a loss of control of the equipment, the synchronous staff may quickly retreat due to panic, and if the staff is injured, the movement speed may suddenly slow down. These abnormal behaviors or states can all be manifested as abnormal acceleration, that is, acceleration data can reflect the most realistic behavior of the staff in dangerous areas, so as to more accurately infer the abnormal behavior state of the staff. Therefore, firstly, based on the acceleration of the node, analyze the The staff member The node is the first in the local frame data The acceleration anomaly factor of the frame. The specific implementation is: Based on the node's acceleration, the local frame data The staff member Node, if The node is in the video data If the acceleration at the frame is negative, The frame is recorded as the acceleration burst moment, and the first The acceleration at the frame is recorded as abnormal acceleration; similarly, the The staff member All abnormal accelerations of nodes in the video data, and calculate the The staff member The average value of all abnormal accelerations of a node in the video data. Abnormal acceleration means that the staff encountered an emergency, which caused their behavior to slow down or stagnate at that moment.

[0041] According to The staff member The node is the first in the local frame data The acceleration at the time of the frame, combined with the average of the abnormal acceleration, is obtained. The staff member The node is the first in the local frame data The acceleration anomaly factor at the frame time. Construct the The staff member The node is the first in the local frame data The calculation formula of the acceleration abnormal factor at the frame time is: In the formula, Indicates The staff member The node is the first in the local frame data Acceleration anomaly factor at frame time; Indicates The staff member The node is the first in the local frame data Acceleration at frame time; Indicates The staff member The mean of all abnormal accelerations of nodes in the video data; Indicated by natural constant The exponential function of base .

[0042] No. The node in The acceleration of the frame The smaller the difference between the mean values ​​of abnormal acceleration of the nodes in the video data, the closer the worker's moving acceleration is to the abnormal acceleration, which means that an accident is more likely to occur at the construction site at this moment, resulting in abnormal node acceleration, and thus making the acceleration abnormal factor of the node at this moment larger.

[0043] When a worker has an abnormal body posture due to a malfunction or dangerous action (for example, falling or being unable to stand normally after being injured), the relative acceleration between the worker's nodes will be abnormal. For example, when falling, the positions of the head, shoulders, arms, and legs will change rapidly, but these changes may no longer be synchronized in time, resulting in significant differences in acceleration characteristics; that is, the acceleration of different parts of the body is different, resulting in irregular changes in the overall acceleration. Therefore, based on the acceleration abnormality factor, the first The irregularity of the acceleration changes between all nodes of a worker in the local frame data of the video data. The specific implementation is: The first The staff member The acceleration anomaly factors of the node in all frames in the local frame data constitute a sequence, recorded as The staff member The acceleration time series of the nodes; All nodes of the workers are combined in any pair to get the Node combination of workers; analyze the The Pearson correlation coefficient between the acceleration time series corresponding to two nodes in all node combinations corresponding to the worker is obtained. The relative abnormality of the nodes of each worker. The calculation formula for the relative abnormality degree of a worker's node is: In the formula, Indicates The relative abnormality of the nodes of each worker in the local frame data; Indicates The number of all node combinations for a worker; Indicates The staff member The Pearson correlation coefficient between the acceleration time series of two nodes in a node combination; Expressed as a natural constant The exponential function of base .

[0044] The smaller the Pearson correlation coefficient between the acceleration time series of two nodes in the local frame data of the worker, the greater the relative abnormality between the nodes, indicating that there are differences in the acceleration of different body parts of the worker, which may be caused by an accident at the construction site, resulting in irregular changes in the overall acceleration.

[0045] S400: Analyze the differences in node abnormal state factors and node relative abnormal degree of all workers between the current local frame data and the historical data, and obtain the abnormal working state degree of the construction site.

[0046] Historical data can show the relative abnormality between nodes during the normal work of staff, but the reliability of historical data compared with current local frame data is different. The more consistent the local frame data in the historical data is with the current local frame data, the higher the reliability, that is, the more consistent the working status corresponding to the historical data is with the current working status, so the higher the reliability, the more accurate the working status of the staff.

[0047] Therefore, between the current local frame data and the historical data, the differences in the node abnormal state factors and the differences in the relative abnormal degree of the nodes of all staff members are analyzed respectively to obtain the degree of abnormal working state of the construction site. The specific implementation method is: all frame data before the current local frame data in the video data are recorded as historical data; based on the historical data, the mean of the historical node abnormal state factors and the mean of the historical node relative abnormal degree of all staff members are obtained; the difference between the node abnormal state factor of each staff member and the mean of the historical node abnormal state factor, and the difference between the node relative abnormal degree of each staff member and the mean of the historical node relative abnormal degree are analyzed, and all staff members are traversed to obtain the degree of abnormal working state of the construction site. The calculation formula for constructing the degree of abnormal working state of the construction site is: In the formula, Indicates the degree of abnormal working status of workers at the construction site; Represents the number of all workers in the local frame data of the video data; Indicates The relative abnormality of the nodes of each worker in the local frame data; Indicates the local frame data The node abnormal status factor of each worker; Represents the mean value of the abnormal state factors of the historical nodes of all the staff members' nodes in the local frame data of the video data in the historical data; Represents the average of the relative abnormality of the nodes of all staff members in the local frame data of the video data in the historical data; represents the linear normalization function.

[0048] When a fault occurs, the working status of the staff will show characteristics that are significantly different from the normal state in the historical data, mainly manifested in abnormal behavior in position, acceleration, and relative abnormalities between nodes. Therefore, the working status of the staff is obtained through local frame data, and these features are fused with real-time image data and acceleration data. The differences are compared with the normal working status of the staff in the historical data, and the abnormal working status of the staff at the construction site can be effectively identified.

[0049] S500: Obtain equipment risk factors and project progress risk factors using a neural network based on the equipment status data and the project progress data.

[0050] There are limitations in relying solely on abnormal worker behavior in video data to assess the possibility of sudden dangers at construction sites, because video data can only reflect the behavior and status of workers, but cannot fully understand the entire construction environment, equipment operating conditions and project progress; dangers at construction sites are often the result of a combination of multiple factors, such as equipment failure or delayed construction progress. By combining the abnormal working status of workers, equipment status data (such as mechanical failure, working status) and project progress data, it is possible to more comprehensively grasp the sudden risk factors at the construction site.

[0051] Therefore, it is also necessary to use neural networks to obtain equipment risk factors and project progress risk factors based on equipment status data and project progress data. The specific implementation method is as follows: For any type of data except the construction site, such as the equipment status data at the construction site. Input the equipment status data at the construction site into the trained neural network to obtain the equipment risk factor at the construction site; wherein the neural network used in this embodiment is CNN, and the method for training the neural network data set is: collect a large amount of equipment status data at the construction site, score each piece of equipment status data, wherein the score range is 0-1, the larger the score, the higher the possibility of abnormal equipment status, and record the scoring result as the label of each piece of equipment status data; the equipment status data and its corresponding labels constitute a data set; use the data set to train the CNN neural network to obtain the equipment status data training neural network, and the loss function used in the training process is the cross loss function.

[0052] Similarly, the project progress risk factor is obtained.

[0053] S600: According to the degree of abnormal working status, combined with equipment risk factors and project progress risk factors, the possibility of sudden danger at the construction site is obtained, and detection and early warning of dangers at the construction site are carried out.

[0054] By combining the abnormal working status of staff, equipment status data and project progress data, we can more comprehensively grasp the risk points of the construction site, thereby making more accurate warnings and decisions; this comprehensive assessment helps to improve the accuracy of sudden danger predictions and reduce misjudgments caused by a single data source.

[0055] Therefore, according to the degree of abnormal working status, combined with equipment risk factors and project progress risk factors, the possibility of sudden danger at the construction site is obtained, and the danger at the construction site is detected and warned. According to the degree of abnormal working status, combined with equipment risk factors and project progress risk factors, the possibility of sudden danger at the construction site is obtained. The calculation formula for the possibility of sudden danger at the construction site is constructed as follows: In the formula, Indicates the possibility of sudden danger at the construction site; Indicates the degree of abnormal working status of workers at the construction site; Indicates the equipment risk factor at the construction site; Indicates the project progress risk factor of the construction site; represents the linear normalization function.

[0056] The greater the risk factor of the equipment status data and project progress data at the construction site, the greater the possibility of sudden danger at the construction site; the greater the degree of abnormal working status of the staff, the greater the abnormal behavior of the staff in the video data of the construction site, and the greater the possibility of sudden danger at the construction site.

[0057] The possibility threshold is preset, and the value can be 0.85; determine whether the possibility of sudden danger at the construction site is greater than or equal to the possibility threshold; if so, , it determines that there is an emergency dangerous incident at the construction site at the current moment, and immediately sends an alarm to remind relevant personnel to deal with it in time.

[0058] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0059] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for fusion of multi-heterogeneous IoT sensing data for construction sites, characterized in that: The method comprises: Collect video data of construction site workers, identify the node of each worker, obtain the position coordinates and acceleration of each node, and obtain equipment status data and project progress data at the construction site; Based on the position coordinates of the nodes, analyzing the transformation of the position coordinates of the nodes of each staff member in the local frame data of the video data, and obtaining the abnormal state factor of the nodes of each staff member; Based on the acceleration of the node, analyzing the irregularity of the acceleration changes between all nodes of each staff member in the local frame data of the video data, and obtaining the relative abnormality degree of the node of each staff member; The differences of the node abnormal state factors and the relative abnormal degree of the nodes of all workers are analyzed between the current local frame data and the historical data to obtain the abnormal working state degree of the construction site; According to the equipment status data and the project progress data, using a neural network, obtaining equipment risk factors and project progress risk factors; According to the degree of the abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained, and the danger at the construction site is detected and warned.

2. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 1 is characterized in that: Analyzing the transformation of the position coordinates of each worker's node in the local frame data of the video data also includes: identifying the unstable node of each worker.

3. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 2 is characterized in that: Identify the unstable nodes of each worker, including: Recording all frame data before the current local frame data in the video data as historical data; Preset variance threshold; For any node of any staff member, if the variance of the distance between any two position coordinates of the node in the historical data is less than the variance threshold, the node is recorded as a stable node, otherwise, the node is recorded as an unstable node.

4. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 3 is characterized in that: Based on the position coordinates of the nodes, the transformation of the position coordinates of the nodes of each staff member in the local frame data of the video data is analyzed to obtain the abnormal state factor of the nodes of each staff member, including: Based on the position coordinates of the nodes, analyzing the Euclidean distance of the position coordinates of the unstable nodes of each staff member between any two adjacent frames in the local frame data of the video data, and obtaining a distance measure of the position coordinates; Based on the position coordinates of the nodes, the standard deviations of all the position coordinates of the unstable nodes of each staff member in the local frame data of the video data in the X-axis and Y-axis directions are analyzed to obtain the degree of discreteness of the position coordinates; According to the distance metric and the discrete degree, the transformation of the position coordinates of the unstable node of each staff member in the local frame data of the video data is obtained, and the node abnormal state factor of each staff member is obtained.

5. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 1 is characterized in that: Based on the acceleration of the node, the irregularity of the acceleration changes between all nodes of each staff member in the local frame data of the video data is analyzed to obtain the relative abnormality degree of the node of each staff member, including: Based on the acceleration of the node, analyze the The staff member The node is the first in the local frame data The acceleration anomaly factor at the frame time; Based on the acceleration anomaly factor, analyze the The irregularity of the acceleration changes of all nodes of a worker in the local frame data of the video data is analyzed to obtain the relative abnormality degree of each worker's node.

6. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 5 is characterized in that: Based on the acceleration of the node, analyze the The staff member The node is the first in the local frame data Acceleration anomaly factors at the frame time, including: Based on the acceleration of the node, identify the The staff member All abnormal accelerations of nodes in the video data are obtained to obtain the average value of all the abnormal accelerations; According to The staff member The node is the first in the local frame data The acceleration at the frame time is combined with the mean value of the abnormal acceleration to obtain the The staff member The node is the first in the local frame data The acceleration anomaly factor at frame time.

7. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 5 or 6, characterized in that: Based on the acceleration anomaly factor, analyze the The irregularity of the acceleration changes between all nodes of a worker in the local frame data of the video data includes: The first The staff member The acceleration anomaly factors of the nodes in all frames in the local frame data constitute a sequence, recorded as The staff member The acceleration time series of each node; The first All nodes of the staff are combined in any pairs to obtain node combinations; Analysis The Pearson correlation coefficient between the acceleration time series corresponding to two nodes in all the node combinations corresponding to the staff member is obtained. The relative abnormality of the nodes of each worker.

8. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 1 is characterized in that: The differences in the node abnormal state factors and the relative abnormal degree of the nodes of all workers are analyzed between the current local frame data and the historical data to obtain the abnormal working state degree of the construction site, including: Recording all frame data before the current local frame data in the video data as historical data; Based on the historical data, the mean values ​​of the abnormal state factors of the historical nodes and the relative abnormal degree of the historical nodes of all staff members are obtained; Analyze the difference between the node abnormal state factor of each worker and the mean value of the historical node abnormal state factor, and analyze the difference between the node relative abnormality degree of each worker and the mean value of the historical node relative abnormality degree, traverse all workers, and obtain the degree of abnormal working status at the construction site.

9. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 1 is characterized in that: According to the equipment status data and the engineering progress data, using a neural network, equipment risk factors and engineering progress risk factors are obtained, including: Inputting the equipment status data and the project progress data into the trained CNN neural network respectively to obtain equipment risk factors and project progress risk factors; The training method of the CNN neural network is as follows: collect a large amount of equipment status data or project progress data at the construction site, score each equipment status data or project progress data, and record the scoring result as the label of each equipment status data or project progress data; the equipment status data or project progress data and their corresponding labels constitute a data set; use the data set to train the CNN neural network to obtain the equipment status data training neural network or the project progress data training neural network, and the loss function used in the training process is the cross loss function.

10. The method for fusion of multi-heterogeneous IoT sensing data for construction sites according to claim 1 is characterized in that: According to the degree of abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained, and the danger at the construction site is detected and warned, including: According to the degree of the abnormal working state, combined with the equipment risk factor and the project progress risk factor, the possibility of sudden danger at the construction site is obtained; Preset likelihood threshold; Determine whether the possibility of sudden danger at the construction site is greater than or equal to the possibility threshold; If so, it is determined that a sudden dangerous incident has occurred at the construction site at the current moment, and an alarm is sent immediately to remind relevant personnel to handle it in a timely manner.

Images