An intelligent early warning method for construction hazards

Abstract

The present invention relates to the technical field of data processing, and particularly to an intelligent early warning method for construction hazards; the method includes the following steps: S1, obtaining video data within a set time through a camera, where the video data includes multiple frames of pictures; S2, identifying a first region of interest and a second region of interest in each frame of the video data, and outputting the first pixel coordinates, second pixel coordinates, and convolution offset of each frame of the picture; S3, obtaining the first world correction coordinates and the second world correction coordinates of each frame of the picture, thereby calculating the world coordinates of the construction worker and the world coordinates of the construction machinery, and finally obtaining the distance value between the construction worker and the construction machinery; S4, obtaining the moving speed of the construction worker and the moving speed of the construction machinery; S5, performing a hazard early warning evaluation based on the distance value between the construction worker and the construction machinery, the moving speed of the construction worker, and the moving speed of the construction machinery; the present invention realizes a more accurate construction hazard early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent early warning method for construction hazards. Background Art

[0002] The rapid development of the construction industry has brought high risks and frequent accidents. Factors such as the dense flow of personnel on the construction site and complex cross-operation links have led to frequent safety accidents in construction projects. Among them, close-range dangerous area intrusion accidents, such as collisions between workers and equipment or equipment, are the main types of accidents. In particular, when workers and mobile machinery are in the same construction space, human-machine collisions caused by the intersection of work trajectories are an important cause of such accidents. Workers' unintentional blindness to nearby machinery or machine drivers' negligence in observation often cause construction entities with different predetermined trajectories to overlap their work trajectories, triggering dangerous intrusions and leading to safety accidents. Accurately supervising the interaction status between workers and mobile construction machinery and timely warning of the risk of intrusion into dangerous areas of construction machinery are crucial to reducing on-site safety accidents. However, traditional manual inspection methods are too one-sided and inefficient, and existing sensor monitoring methods require high costs and have poor on-site application effects.

[0003] In the article "Intelligent Warning Method for Construction Machinery Dangerous Area Intrusion Based on Deep Learning and Depth Estimation" published by Wu Han and Han Yu, a method consisting of three parts, namely, a dataset of images of on-site workers and construction machinery, a target detection network and a depth estimation network, was disclosed. The algorithm optimization takes into account detection accuracy, detection speed and operating cost, and realizes dynamic identification and risk warning of construction machinery dangerous area intrusion events, in order to improve the efficiency of early prevention of safety accidents and ensure the safety of workers' work. However, the above method does not take into account that the camera will shake during the movement of the construction machinery, so that the spatial coordinates of the workers and the construction machinery obtained based on the images taken by the camera are inaccurate, resulting in inaccurate risk judgment; at the same time, the influence of the movement speed of workers and construction machinery on risk judgment is not considered, which will also lead to inaccurate risk judgment.

[0004] Therefore, there is an urgent need to provide an intelligent early warning method for construction hazards, which can achieve more accurate construction hazard early warning compared with the existing technology. Summary of the invention

[0005] The present invention solves the technical problems existing in the prior art and provides an intelligent early warning method for construction hazards.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A construction hazard intelligent early warning method comprises the following steps:

[0008] S1. Obtain video data within a set time through a camera. The video data includes multiple frames of pictures;

[0009] S2. Identify the regions of interest in each frame of the video data, including the first region of interest and the second region of interest, and output the first pixel coordinates, second pixel coordinates, and convolution offset of each frame of the picture;

[0010] S3. Based on the first pixel coordinates, second pixel coordinates, and convolution offset of each frame of the picture obtained in step S2, obtain the first world correction coordinates and second world correction coordinates of each frame of the picture, thereby calculating the world coordinates of the construction worker and the world coordinates of the construction machinery, and finally obtaining the distance value between the construction worker and the construction machinery;

[0011] S4. Based on the first world correction coordinates and second world correction coordinates obtained in step S3, obtain the moving speed of the construction worker and the moving speed of the construction machinery;

[0012] S5. Conduct a risk warning evaluation based on the distance value between the construction worker and the construction machinery, the moving speed of the construction worker, and the moving speed of the construction machinery.

[0013] Further, the world coordinates of the construction worker are obtained through the first world correction coordinates of each frame of the picture. The i-th first world correction coordinate is calculated by the following formula:

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] In the above formula, represents the i-th first corrected world coordinate, represents the coordinate value along the X direction, represents the coordinate value along the Y direction, represents the coordinate value along the Z direction, , , respectively represent the coordinate values of the i-th first world coordinate along the X direction, Y direction, and Z direction; represents the correction weight, represents the first correction value.

[0019] Furthermore, the world coordinates of the construction machinery are obtained from the second world correction coordinates of each frame of image. The i-th second world correction coordinate is calculated by the following formula:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] In the above formula, represents the i-th second corrected world coordinate, represents the coordinate value along the X direction, represents the coordinate value along the Y direction, represents the coordinate value along the Z direction, represents the second correction value, 、 、 respectively represent the coordinate values of the i-th second world coordinate along the X direction, Y direction, and Z direction.

[0025] Furthermore, 、 、 are calculated by the following formula:

[0026] ;

[0027] ;

[0028] ;

[0029] In the above formula, represents a constant, 、 are respectively the abscissa and ordinate of the first first pixel coordinate, 、 are respectively the abscissa value and ordinate value of the N-th first pixel coordinate, 、 are respectively the abscissa value and ordinate value of the first second pixel coordinate, 、 are respectively the abscissa value and ordinate value of the N-th second pixel coordinate; represents the convolution offset of the i-th frame of image.

[0030] Furthermore, the i-th first world coordinate is obtained by transforming the i-th first pixel coordinate using the Monodepth2 monocular depth estimation network, and the i-th second world coordinate is obtained by transforming the i-th second pixel coordinate using the Monodepth2 monocular depth estimation network.

[0031] Furthermore, the world coordinates of the construction worker and the construction machinery are specifically calculated by the following formula:

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] In the above formula, represents the world coordinates of the construction worker, , , respectively represent the coordinate values along the X, Y, and Z directions, represents the world coordinates of the construction machinery, , , respectively represent the coordinate values along the X, Y, and Z directions.

[0041] Furthermore, the distance value between the construction worker and the construction machinery is calculated by the following formula:

[0042] ;

[0043] In the above formula, represents the distance value between the construction worker and the construction machinery.

[0044] Furthermore, the moving speed of the construction worker and the moving speed of the construction machinery are specifically calculated by the following formula:

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] In the above formula, represents the distance between the th first-world correction coordinate and the i-th first-world correction coordinate, represents the j-th first speed, represents the frequency of one frame, represents the moving speed of the construction worker; , , respectively represent the coordinate values of the th first-world correction coordinate along the X, Y, and Z directions; represents the distance between the th second-world correction coordinate and the i-th second-world correction coordinate, represents the j-th second speed, represents the moving speed of the construction machinery, , , respectively represent the coordinate values of the th second-world correction coordinate along the X, Y, and Z directions.

[0052] Furthermore, the specific method for performing the danger warning evaluation in step S5 is as follows: Set the speed setting value to V, the turning radius of the construction machinery to A, and the blind area distance of the construction machinery to B. Take the largest value between A and B as C, and take the smallest value between A and B as D;

[0053] (1) When, perform the danger warning evaluation according to the following content:

[0054] When C, it is judged as a high-risk state;

[0055] When it is judged as a medium-risk state;

[0056] When it is judged as a low-risk state;

[0057] When it is judged as a safe state;

[0058] (2) When, the following content is used for dangerous early warning evaluation:

[0059] When D, it is judged as a high-risk state;

[0060] When When, it is judged as a medium-risk state;

[0061] When When, it is judged as a low-risk state;

[0062] When When, it is judged as a safe state.

[0063] Furthermore, pieces form the first speed data set, denoted as , pieces form the second speed data set, denoted as ; The speed setting value is calculated by the following formula:

[0064] ;

[0065] In the above formula, , respectively represent the maximum first speed and the minimum first speed in the first speed data set, , respectively represent the maximum second speed and the minimum second speed in the second speed data set.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] The present invention takes into account that the camera will shake during the movement of the construction machinery, thereby calculating the first correction value and the second correction value to correct the world coordinates of the construction workers and the world coordinates of the construction machinery. At the same time, it also takes into account that the convolution offset of each frame of the picture is different, and sets the correction weight of each frame of the picture based on the convolution offset of each frame of the picture to correct the calculated world coordinates of the construction workers and the world coordinates of the construction machinery, obtaining more accurate world coordinates. At the same time, based on the corrected first world correction coordinates and the second world correction coordinates, the speeds of the construction workers and the construction machinery are calculated. When performing intelligent early warning of construction hazards, the impacts of the moving speed of the construction workers, the moving speed of the construction machinery, and the distance value between the construction workers and the construction machinery are considered simultaneously, improving the accuracy of the early warning of hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is the flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0069] The technical solution of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention. It should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0070] As Figure 1 shown, the present invention provides an intelligent early warning method for construction hazards, including the following steps:

[0071] S1. Obtain video data within a set time through a camera. The camera is set on a construction machine. The video data includes multiple frames of pictures, and each frame of picture includes construction workers and construction machines.

[0072] S2. Use the main architecture of the Backbone backbone network in Yolov8 to identify the regions of interest in each frame of the video data. The regions of interest include construction workers and construction machines, and semantic labels are assigned to different regions of interest according to the Label category information of the pictures. The construction workers are marked as the first region of interest, and the construction machines are marked as the second region of interest; at the same time, the central pixel coordinates of the first region of interest and the central pixel coordinates of the second region of interest are output for each frame of the picture. The central pixel coordinates of the first region of interest are denoted as the first pixel coordinates, and the central pixel coordinates of the second region of interest are denoted as the second pixel coordinates. The convolution offset of each frame of the picture is also output.

[0073] The first pixel coordinates of all frames of pictures form a first pixel data set, denoted as , where represents the first pixel data set, represents the first first pixel coordinate, represents the i-th first pixel coordinate, represents the N-th first pixel coordinate, and i takes 1 - N; the second pixel coordinates of all frames of pictures form a second pixel data set, denoted as , where represents the second pixel data set, represents the first second pixel coordinate, represents the i-th second pixel coordinate, represents the N-th second pixel coordinate; the convolution offsets of all frames of pictures form a convolution offset data set, denoted as , where Represents the convolutional offset dataset, Represents the convolutional offset of the first frame of the picture, Represents the convolutional offset of the i-th frame of the picture, Represents the convolutional offset of the N-th frame of the picture.

[0074] The first pixel coordinates of each frame are obtained by taking the weighted average of the pixel coordinates of all pixel points in the first region of interest, and the second pixel coordinates of each frame are obtained by taking the weighted average of the pixel coordinates of all pixel points in the second region of interest.

[0075] In the main architecture of the Backbone backbone network in Yolov8 used in step S2, the Conv module is replaced by the DConv2 module, and convolutional operations are performed through a deformable convolution kernel (deformableconv) and deformable region pooling (deformableROIpooling), so that the convolutional kernel generates a convolutional offset for the input picture in the way of 3×3×2, and after bias adjustment through a 3×3 ordinary convolution, it focuses on the region of interest and adaptively adjusts the receptive field size for objects that may correspond to different scales or deformations at different positions.

[0076] S3. According to what is obtained in step S2 、 、 , obtain the distance values between construction workers and construction machinery, which specifically include the following steps:

[0077] S31. Use the Monodepth2 monocular depth estimation network to convert all the first pixel coordinates into first world coordinates correspondingly, and convert all the second pixel coordinates into second world coordinates correspondingly.

[0078] All the first world coordinates form a first world coordinate set, denoted as , where Represents the first world coordinate set, Represents the first first world coordinate, Represents the i-th first world coordinate, Represents the N-th first world coordinate; all the second world coordinates form a second world coordinate set, denoted as , where Represents the second world coordinate set, Represents the first second world coordinate, Represents the i-th second world coordinate, Represents the N-th second world coordinate.

[0079] S32. Calculate the corresponding first corrected world coordinates and second corrected world coordinates according to each first world coordinate and each second world coordinate. The i-th first corrected world coordinate and the i-th second corrected world coordinate are calculated by the following formula:

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] In the above formula, represents the i-th first corrected world coordinate, represents the coordinate value along the X direction, represents the coordinate value along the Y direction, represents the coordinate value along the Z direction; represents the correction weight, represents the first correction value; represents the i-th second corrected world coordinate, represents the coordinate value along the X direction, represents the coordinate value along the Y direction, represents the coordinate value along the Z direction, represents the second correction value.

[0089] Among them, , , are calculated by the following formula:

[0090] ;

[0091] ;

[0092] ;

[0093] In the above formula, Represents a constant, and are respectively the abscissa and ordinate of and are respectively the abscissa value and ordinate value of and are respectively the abscissa value and ordinate value of and are respectively the abscissa value and ordinate value of.

[0094] S33. Based on all the first corrected world coordinates and all the second corrected world coordinates calculated in step S32, obtain the world coordinates of the construction worker and the world coordinates of the construction machinery, which are specifically calculated through the following formula:

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] ;

[0103] In the above formula, represents the world coordinates of the construction worker, and and respectively represent the coordinate values along the X, Y, and Z directions, represents the world coordinates of the construction machinery, and and respectively represent the coordinate values along the X, Y, and Z directions.

[0104] S34. The distance value between the construction worker and the construction machinery is calculated through the following formula:

[0105] ;

[0106] In the above formula, Represents the distance value between construction workers and construction machinery.

[0107] S4. Based on all the first-world corrected coordinates and all the second-world corrected coordinates obtained in step S3, obtain the moving speed of the construction workers and the moving speed of the construction machinery, which are specifically calculated by the following formula:

[0108] ;

[0109] ;

[0110] ;

[0111] In the above formula, Represents the distance between the th first-world corrected coordinate and the i-th first-world corrected coordinate, Represents the j-th first speed, Represents the frequency of one frame, Represents the moving speed of the construction workers; , , Respectively represent the coordinate values of the th first-world corrected coordinate along the X, Y, and Z directions.

[0112] ;

[0113] ;

[0114] ;

[0115] In the above formula, Represents the distance between the th second-world corrected coordinate and the i-th second-world corrected coordinate, Represents the j-th second speed, Represents the moving speed of the construction machinery, , , Respectively represent the coordinate values of the th second-world corrected coordinate along the X, Y, and Z directions.

[0116] S5. Conduct a danger warning evaluation based on the distance value between the construction workers and the construction machinery, the moving speed of the construction workers, and the moving speed of the construction machinery. Specifically:

[0117] Set the speed set value, the turning radius of the construction machinery, and the blind area distance of the construction machinery, which are denoted as V, A, and B respectively. Take the largest value between A and B as C, and take the smallest value between A and B as D;

[0118] (1) When, conduct a danger warning evaluation according to the following content:

[0119] When C, it is judged as a high-risk state.

[0120] When it is judged as a medium-risk state.

[0121] When it is judged as a low-risk state.

[0122] When it is judged as a safe state.

[0123] (2) When, conduct a danger warning evaluation according to the following content:

[0124] When D, it is judged as a high-risk state.

[0125] When it is judged as a medium-risk state.

[0126] When it is judged as a low-risk state.

[0127] When it is judged as a safe state.

[0128] pieces Form the first speed data set, denoted as , pieces Form the second speed data set, denoted as The speed setting value is calculated by the following formula:

[0129] ;

[0130] In the above formula, , respectively represent the maximum first speed and the minimum first speed in the first speed data set, , respectively represent the maximum second speed and the minimum second speed in the second speed data set.

[0131] In the present invention, considering that the camera will jitter during the movement of the construction machinery, the first correction value and the second correction value are calculated to correct the world coordinates of the construction worker and the world coordinates of the construction machinery. At the same time, considering that the convolution offset of each frame of picture is different, the correction weight of each frame of picture is set based on the convolution offset of each frame of picture, and the calculated world coordinates of the construction worker and the world coordinates of the construction machinery are corrected to obtain more accurate world coordinates. At the same time, based on the corrected first world correction coordinates and the second world correction coordinates, the speeds of the construction worker and the construction machinery are calculated. When the intelligent early warning of construction danger is carried out, the influences of the moving speed of the construction worker, the moving speed of the construction machinery, and the distance value between the construction worker and the construction machinery are considered simultaneously, which improves the accuracy of the danger early warning.

[0132] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than limiting the protection scope of the present invention. Any simple modification or equivalent replacement made by those of ordinary skill in the art to the technical solution of the present invention does not depart from the essence and scope of the technical solution of the present invention.

Claims

1. A construction hazard intelligent early warning method, characterized in that: The following steps are involved: S1. Obtain video data within a set time through a camera, where the video data includes multiple frames of images; S2, identifying a region of interest in each frame of the video data, including a first region of interest and a second region of interest, and outputting a first pixel coordinate, a second pixel coordinate, and a convolution offset of each frame of the video data; S3, according to the first pixel coordinates, the second pixel coordinates, and the convolution offset of each frame of the image obtained in step S2, the first world corrected coordinates and the second world corrected coordinates of each frame of the image are obtained, thereby calculating the world coordinates of the construction worker and the world coordinates of the construction machine, and finally obtaining the distance value between the construction worker and the construction machine; S4, obtaining the moving speed of the construction worker and the moving speed of the construction machinery according to the first world corrected coordinates and the second world corrected coordinates obtained in step S3; S5. Conduct risk warning evaluation based on the distance between the construction workers and the construction machinery, the moving speed of the construction workers, and the moving speed of the construction machinery; The world coordinates of the construction workers are obtained through the first world corrected coordinates of each frame. The i-th first world corrected coordinates are calculated by the following formula: ； ； ； ； In the above formula, represents the i-th first corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, , , Respectively represent the coordinate values ​​of the i-th first world coordinate along the X direction, Y direction, and Z direction; represents the correction weight, represents the first correction value; The world coordinates of the construction machinery are obtained through the second world corrected coordinates of each frame of the image. The i-th second world corrected coordinates are calculated by the following formula: ； ； ； ； In the above formula, represents the i-th second corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, represents the second correction value, , , Respectively represent the coordinate values ​​of the i-th second world coordinate along the X direction, Y direction, and Z direction.

2. The intelligent early warning method for construction hazards according to claim 1 is characterized in that: , , Calculated by the following formula: ； ； ； In the above formula, represents a constant, , are the horizontal and vertical coordinates of the first pixel coordinates, , are the horizontal and vertical coordinate values ​​of the Nth first pixel coordinates, respectively. , are the horizontal and vertical coordinate values ​​of the first and second pixel coordinates respectively. , are the horizontal coordinate value and the vertical coordinate value of the Nth second pixel coordinate respectively; Represents the convolution offset of the i-th frame.

3. The intelligent early warning method for construction hazards according to claim 1 is characterized in that: The i-th first world coordinate is obtained by transforming the i-th first pixel coordinate using the Monodepth2 monocular depth estimation network, and the i-th second world coordinate is obtained by transforming the i-th second pixel coordinate using the Monodepth2 monocular depth estimation network.

4. The intelligent early warning method for construction hazards according to claim 1 is characterized in that: The world coordinates of the construction workers and the world coordinates of the construction machinery are calculated by the following formula: ； ； ； ； ； ； ； ； In the above formula, represents the world coordinates of the construction worker, , , Respectively Coordinate values ​​along the X, Y, and Z directions, Represents the world coordinates of the construction machinery, , , Respectively Coordinate values ​​along the X, Y, and Z directions.

5. The intelligent early warning method for construction hazards according to claim 4 is characterized in that: The distance between construction workers and construction machinery is calculated by the following formula: ； In the above formula, Indicates the distance value between construction workers and construction machinery.

6. The intelligent early warning method for construction hazards according to claim 2 is characterized in that: The moving speed of construction workers and construction machinery is calculated by the following formula: ； ； ； ； ； ； In the above formula, Indicates The distance between the first world corrected coordinate and the i-th first world corrected coordinate, represents the jth first velocity, Indicates the frequency of a frame, Indicates the moving speed of the construction workers; , , Respectively represent The coordinate values ​​of the first world corrected coordinates along the X, Y, and Z directions; Indicates The distance between the second world corrected coordinate and the i-th second world corrected coordinate, represents the jth second speed, Indicates the moving speed of the construction machinery. , , Respectively represent The coordinate values ​​of the second world corrected coordinates along the X, Y, and Z directions.

7. The intelligent early warning method for construction hazards according to claim 6 is characterized in that: The specific method for performing the hazard warning evaluation in step S5 is: setting the speed setting value to V, the turning radius of the construction machinery to A, and the blind spot distance of the construction machinery to B, taking the largest value between A and B as C, and taking the smallest value between A and B as D; (1) When conducting a hazard warning assessment, the following contents shall be considered: when C, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when When , it is judged to be a safe state; (2) When conducting a hazard warning assessment, the following contents shall be considered: when D, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when , it is judged to be a safe state.

8. The intelligent early warning method for construction hazards according to claim 7 is characterized in that: indivual Form the first velocity data set, denoted as , indivual The second velocity data set is formed, denoted as ; The speed setting value is calculated by the following formula: ； In the above formula, , Respectively represent the maximum first speed and the minimum first speed in the first speed data set, , They respectively represent the maximum second speed and the minimum second speed in the second speed data set.

Images