Intelligent early warning method for construction danger

Abstract

The invention relates to the technical field of data processing, in particular to a construction danger intelligent early warning method. Comprising the following steps: S1, video data in a set time are acquired through a camera, and the video data comprise multiple frames of pictures; s2, identifying a first region of interest and a second region of interest in each frame of picture in the video data, and outputting a first pixel coordinate, a second pixel coordinate and a convolution offset of each frame of picture; s3, obtaining a first world correction coordinate and a second world correction coordinate of each frame of picture, calculating world coordinates of the construction worker and world coordinates of the construction machinery, and finally obtaining a distance value between the construction worker and the construction machinery; s4, the moving speed of the construction worker and the moving speed of the construction machine are obtained; s5, performing danger early warning evaluation according to the distance value between the construction worker and the construction machine, the moving speed of the construction worker and the moving speed of the construction machine; according to the invention, more accurate construction danger early warning is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent early warning method for construction hazards. Background Art

[0002] The rapid development of the construction industry has brought high risks and frequent accidents. Factors such as the dense flow of personnel on the construction site and complex cross-operation links have led to frequent safety accidents in construction projects. Among them, close-range dangerous area intrusion accidents, such as collisions between workers and equipment or equipment, are the main types of accidents. In particular, when workers and mobile machinery are in the same construction space, human-machine collisions caused by the intersection of work trajectories are an important cause of such accidents. Workers' unintentional blindness to nearby machinery or machine drivers' negligence in observation often cause construction entities with different predetermined trajectories to overlap their work trajectories, triggering dangerous intrusions and leading to safety accidents. Accurately supervising the interaction status between workers and mobile construction machinery and timely warning of the risk of intrusion into dangerous areas of construction machinery are crucial to reducing on-site safety accidents. However, traditional manual inspection methods are too one-sided and inefficient, and existing sensor monitoring methods require high costs and have poor on-site application effects.

[0003] In the article "Intelligent Warning Method for Construction Machinery Dangerous Area Intrusion Based on Deep Learning and Depth Estimation" published by Wu Han and Han Yu, a method consisting of three parts, namely, a dataset of images of on-site workers and construction machinery, a target detection network and a depth estimation network, was disclosed. The algorithm optimization takes into account detection accuracy, detection speed and operating cost, and realizes dynamic identification and risk warning of construction machinery dangerous area intrusion events, in order to improve the efficiency of early prevention of safety accidents and ensure the safety of workers' work. However, the above method does not take into account that the camera will shake during the movement of the construction machinery, so that the spatial coordinates of the workers and the construction machinery obtained based on the images taken by the camera are inaccurate, resulting in inaccurate risk judgment; at the same time, the influence of the movement speed of workers and construction machinery on risk judgment is not considered, which will also lead to inaccurate risk judgment.

[0004] Therefore, there is an urgent need to provide an intelligent early warning method for construction hazards, which can achieve more accurate construction hazard early warning compared with the existing technology. Summary of the invention

[0005] The present invention solves the technical problems existing in the prior art and provides an intelligent early warning method for construction hazards.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows: A construction hazard intelligent early warning method comprises the following steps: S1. Obtain video data within a set time through a camera, where the video data includes multiple frames of images; S2, identifying a region of interest in each frame of the video data, including a first region of interest and a second region of interest, and outputting a first pixel coordinate, a second pixel coordinate, and a convolution offset of each frame of the video data; S3, according to the first pixel coordinates, the second pixel coordinates, and the convolution offset of each frame of the image obtained in step S2, the first world corrected coordinates and the second world corrected coordinates of each frame of the image are obtained, thereby calculating the world coordinates of the construction worker and the world coordinates of the construction machine, and finally obtaining the distance value between the construction worker and the construction machine; S4, obtaining the moving speed of the construction worker and the moving speed of the construction machinery according to the first world corrected coordinates and the second world corrected coordinates obtained in step S3; S5. Conduct risk warning evaluation based on the distance between the construction workers and the construction machinery, the moving speed of the construction workers, and the moving speed of the construction machinery.

[0007] Furthermore, the world coordinates of the construction workers are obtained through the first world corrected coordinates of each frame of the image. The i-th first world corrected coordinates are calculated by the following formula: ; ; ; ; In the above formula, represents the i-th first corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, , , Respectively represent the coordinate values ​​of the i-th first world coordinate along the X direction, Y direction, and Z direction; represents the correction weight, Indicates the first correction value.

[0008] Furthermore, the world coordinates of the construction machinery are obtained through the second world corrected coordinates of each frame of the image, and the i-th second world corrected coordinates are calculated by the following formula: ; ; ; ; In the above formula, represents the i-th second corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, represents the second correction value, , , Respectively represent the coordinate values ​​of the i-th second world coordinate along the X direction, Y direction, and Z direction.

[0009] Furthermore, , , Calculated by the following formula: ; ; ; In the above formula, represents a constant, , are the horizontal and vertical coordinates of the first pixel coordinates, , are the horizontal and vertical coordinate values ​​of the Nth first pixel coordinates, respectively. , are the horizontal and vertical coordinate values ​​of the first and second pixel coordinates respectively. , are the horizontal coordinate value and the vertical coordinate value of the Nth second pixel coordinate respectively; Represents the convolution offset of the i-th frame.

[0010] Furthermore, the i-th first world coordinate is obtained by transforming the i-th first pixel coordinate using the Monodepth2 monocular depth estimation network, and the i-th second world coordinate is obtained by transforming the i-th second pixel coordinate using the Monodepth2 monocular depth estimation network.

[0011] Furthermore, the world coordinates of the construction workers and the world coordinates of the construction machinery are calculated by the following formula: ; ; ; ; ; ; ; ; In the above formula, represents the world coordinates of the construction worker, , , Respectively Coordinate values ​​along the X, Y, and Z directions, Represents the world coordinates of the construction machinery, , , Respectively Coordinate values ​​along the X, Y, and Z directions.

[0012] Furthermore, the distance between the construction workers and the construction machinery is calculated by the following formula: ; In the above formula, Indicates the distance value between construction workers and construction machinery.

[0013] Furthermore, the moving speed of the construction workers and the moving speed of the construction machinery are specifically calculated by the following formula: ; ; ; ; ; ; In the above formula, Indicates The distance between the first world corrected coordinate and the i-th first world corrected coordinate, represents the jth first velocity, Indicates the frequency of a frame, Indicates the moving speed of the construction workers; , , Respectively represent The coordinate values ​​of the first world corrected coordinates along the X, Y, and Z directions; Indicates The distance between the second world corrected coordinate and the i-th second world corrected coordinate, represents the jth second speed, Indicates the moving speed of the construction machinery. , , Respectively represent The coordinate values ​​of the second world corrected coordinates along the X, Y, and Z directions.

[0014] Furthermore, in step S5, the specific method for performing the danger warning evaluation is as follows: setting the speed setting value to V, the turning radius of the construction machinery to A, and the blind spot distance of the construction machinery to B, taking the largest value between A and B as C, and taking the smallest value between A and B as D; (1) When conducting a hazard warning assessment, the following contents shall be considered: when C, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when When , it is judged to be a safe state; (2) When conducting a hazard warning assessment, the following contents shall be considered: when D, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when , it is judged to be a safe state.

[0015] Furthermore, indivual Form the first velocity data set, denoted as , indivual The second velocity data set is formed, denoted as ; The speed setting value is calculated by the following formula: ; In the above formula, , Respectively represent the maximum first speed and the minimum first speed in the first speed data set, , They respectively represent the maximum second speed and the minimum second speed in the second speed data set.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention takes into account that the camera may shake during the movement of the construction machinery, thereby calculating the first correction value and the second correction value to correct the world coordinates of the construction worker and the world coordinates of the construction machinery. At the same time, the convolution offset of each frame of the picture is different, and the correction weight of each frame of the picture is set based on the convolution offset of each frame of the picture. The calculated world coordinates of the construction worker and the world coordinates of the construction machinery are corrected to obtain more accurate world coordinates. At the same time, the speed of the construction worker and the speed of the construction machinery are calculated based on the corrected first world corrected coordinates and the second world corrected coordinates. When the intelligent warning of construction hazards is given, the influence of the three aspects of the moving speed of the construction worker, the moving speed of the construction machinery, and the distance value between the construction worker and the construction machinery are considered at the same time, thereby improving the accuracy of the hazard warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention. It should be noted that the orientation or position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical", "horizontal", etc. is based on the orientation or position relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description. It does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0019] like Figure 1 As shown, the present invention provides a construction hazard intelligent early warning method, comprising the following steps: S1. Obtain video data within a set time through a camera. The camera is set on a construction machine. The video data includes multiple frames of pictures, and each frame of the picture includes a construction worker and a construction machine.

[0020] S2. Use the main structure of the Backbone network in Yolov8 to identify the regions of interest in each frame of the video data. The regions of interest include construction workers and construction machinery. Different regions of interest are semantically labeled according to the Label category information of the image. Construction workers are marked as the first region of interest, and construction machinery is marked as the second region of interest. At the same time, the center pixel coordinates of the first region of interest and the center pixel coordinates of the second region of interest are output for each frame of the image. The center pixel coordinates of the first region of interest are marked as the first pixel coordinates, and the center pixel coordinates of the second region of interest are marked as the second pixel coordinates. The convolution offset of each frame of the image is also output.

[0021] The first pixel coordinates of all frames form the first pixel data set, denoted as ,in, represents the first pixel data set, Indicates the first pixel coordinate, represents the i-th first pixel coordinate, represents the Nth first pixel coordinate, i is 1-N; the second pixel coordinates of all frame images form the second pixel data set, recorded as ,in, represents the second pixel data set, represents the first second pixel coordinate, represents the i-th second pixel coordinate, represents the Nth second pixel coordinate; the convolution offset of all frame images forms a convolution offset data set, recorded as ,in, represents the convolution offset dataset, Represents the convolution offset of the first frame of the image, Represents the convolution offset of the i-th frame image, Indicates the convolution offset of the Nth frame.

[0022] The first pixel coordinates of each frame are obtained by taking a weighted average of the pixel coordinates of all the pixels in the first region of interest, and the second pixel coordinates of each frame are obtained by taking a weighted average of the pixel coordinates of all the pixels in the second region of interest.

[0023] The main architecture of the Backbone network in Yolov8 used in step S2 replaces the Conv module with the DConv2 module. The convolution operation is performed through the adaptive geometric deformation convolution kernel (deformableconv) and deformable region pooling (deformableROIpooling), so that the convolution kernel generates a convolution offset for the input image in a 3×3×2 manner, and after the offset adjustment of the 3×3 ordinary convolution, it focuses on the area of ​​interest and adaptively adjusts the receptive field size for objects at different positions that may correspond to different scales or deformations.

[0024] S3, obtained according to step S2 , , , obtain the distance value between the construction workers and the construction machinery, specifically including the following steps: S31. Using the Monodepth2 monocular depth estimation network, all first pixel coordinates are converted into first world coordinates, and all second pixel coordinates are converted into second world coordinates.

[0025] All first-world coordinates form a first-world coordinate set, denoted as ,in, represents the first world coordinate set, Represents the first first world coordinate, represents the i-th first world coordinate, represents the Nth first world coordinate; all the second world coordinates form the second world coordinate set, recorded as ,in, represents the second world coordinate set, represents the first second world coordinate, represents the i-th second world coordinate, Indicates the Nth second world coordinate.

[0026] S32, according to each first world coordinate and each second world coordinate, calculate the corresponding first corrected world coordinate and second corrected world coordinate, the i-th first corrected world coordinate and the i-th second corrected world coordinate are calculated by the following formula: ; ; ; ; ; ; ; ; In the above formula, represents the i-th first corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express Coordinate value along the Z direction; represents the correction weight, represents the first correction value; represents the i-th second corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, Indicates the second correction value.

[0027] in, , , Calculated by the following formula: ; ; ; In the above formula, represents a constant, , They are The horizontal and vertical coordinates of , They are The horizontal and vertical coordinate values ​​of , They are The horizontal and vertical coordinate values ​​of , They are The horizontal and vertical coordinate values ​​of .

[0028] S33, according to all the first corrected world coordinates and all the second corrected world coordinates calculated in step S32, the world coordinates of the construction workers and the world coordinates of the construction machinery are obtained, which are specifically calculated by the following formula: ; ; ; ; ; ; ; ; In the above formula, represents the world coordinates of the construction worker, , , Respectively Coordinate values ​​along the X, Y, and Z directions, Represents the world coordinates of the construction machinery, , , Respectively Coordinate values ​​along the X, Y, and Z directions.

[0029] S34. The distance between construction workers and construction machinery is calculated by the following formula: ; In the above formula, Indicates the distance value between construction workers and construction machinery.

[0030] S4. According to all the first-world corrected coordinates and all the second-world corrected coordinates obtained in step S3, the moving speed of the construction workers and the moving speed of the construction machinery are obtained, which are specifically calculated by the following formula: ; ; ; In the above formula, Indicates The distance between the first world corrected coordinate and the i-th first world corrected coordinate, represents the jth first velocity, Indicates the frequency of a frame, Indicates the moving speed of the construction workers; , , Respectively represent The coordinate values ​​of the first world corrected coordinates along the X, Y, and Z directions.

[0031] ; ; ; In the above formula, Indicates The distance between the second world corrected coordinate and the i-th second world corrected coordinate, represents the jth second speed, Indicates the moving speed of the construction machinery. , , Respectively represent The coordinate values ​​of the second world corrected coordinates along the X, Y, and Z directions.

[0032] S5. Conduct risk warning evaluation based on the distance between construction workers and construction machinery, the moving speed of construction workers, and the moving speed of construction machinery. Specifically: Set the speed setting value, the turning radius of the construction machinery, and the blind spot distance of the construction machinery, and record them as V, A, and B respectively. Take the largest value between A and B as C, and take the smallest value between A and B as D; (1) When conducting a hazard warning assessment, the following contents shall be considered: when C, it is judged as a high-risk state.

[0033] when It is judged as a medium-risk state.

[0034] when It is judged as a low-risk state.

[0035] when , it is judged to be a safe state.

[0036] (2) When conducting a hazard warning assessment, the following contents shall be considered: when D, it is judged as a high-risk state.

[0037] when It is judged as a medium-risk state.

[0038] when It is judged as a low-risk state.

[0039] when , it is judged to be a safe state.

[0040] indivual Form the first velocity data set, denoted as , indivual The second velocity data set is formed, denoted as , the speed setting value is calculated by the following formula: ; In the above formula, , Respectively represent the maximum first speed and the minimum first speed in the first speed data set, , They respectively represent the maximum second speed and the minimum second speed in the second speed data set.

[0041] The present invention takes into account that the camera may shake during the movement of the construction machinery, thereby calculating the first correction value and the second correction value to correct the world coordinates of the construction worker and the world coordinates of the construction machinery. At the same time, the convolution offset of each frame of the picture is different, and the correction weight of each frame of the picture is set based on the convolution offset of each frame of the picture. The calculated world coordinates of the construction worker and the world coordinates of the construction machinery are corrected to obtain more accurate world coordinates. At the same time, the speed of the construction worker and the speed of the construction machinery are calculated based on the corrected first world corrected coordinates and the second world corrected coordinates. When the intelligent warning of construction hazards is given, the influence of the three aspects of the moving speed of the construction worker, the moving speed of the construction machinery, and the distance value between the construction worker and the construction machinery are considered at the same time, thereby improving the accuracy of the hazard warning.

[0042] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A construction hazard intelligent early warning method, characterized in that: The following steps are involved: S1. Obtain video data within a set time through a camera, where the video data includes multiple frames of images; S2, identifying a region of interest in each frame of the video data, including a first region of interest and a second region of interest, and outputting a first pixel coordinate, a second pixel coordinate, and a convolution offset of each frame of the video data; S3, according to the first pixel coordinates, the second pixel coordinates, and the convolution offset of each frame of the image obtained in step S2, the first world corrected coordinates and the second world corrected coordinates of each frame of the image are obtained, thereby calculating the world coordinates of the construction worker and the world coordinates of the construction machine, and finally obtaining the distance value between the construction worker and the construction machine; S4, obtaining the moving speed of the construction worker and the moving speed of the construction machinery according to the first world corrected coordinates and the second world corrected coordinates obtained in step S3; S5. Conduct risk warning evaluation based on the distance between the construction workers and the construction machinery, the moving speed of the construction workers, and the moving speed of the construction machinery.

2. The intelligent early warning method for construction hazards according to claim 1 is characterized in that: The world coordinates of the construction workers are obtained through the first world corrected coordinates of each frame. The i-th first world corrected coordinates are calculated by the following formula: ； ； ； ； In the above formula, represents the i-th first corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, , , Respectively represent the coordinate values ​​of the i-th first world coordinate along the X direction, Y direction, and Z direction; represents the correction weight, Indicates the first correction value.

3. The intelligent early warning method for construction hazards according to claim 2 is characterized in that: The world coordinates of the construction machinery are obtained through the second world corrected coordinates of each frame of the image. The i-th second world corrected coordinates are calculated by the following formula: ； ； ； ； In the above formula, represents the i-th second corrected world coordinate, express The coordinate value along the X direction, express The coordinate value along the Y direction, express The coordinate value along the Z direction, represents the second correction value, , , Respectively represent the coordinate values ​​of the i-th second world coordinate along the X direction, Y direction, and Z direction.

4. The intelligent early warning method for construction hazards according to claim 3 is characterized in that: , , Calculated by the following formula: ； ； ； In the above formula, represents a constant, , are the horizontal and vertical coordinates of the first pixel coordinates, , are the horizontal and vertical coordinate values ​​of the Nth first pixel coordinates, respectively. , are the horizontal and vertical coordinate values ​​of the first and second pixel coordinates respectively. , are the horizontal coordinate value and the vertical coordinate value of the Nth second pixel coordinate respectively; Represents the convolution offset of the i-th frame.

5. The intelligent early warning method for construction hazards according to claim 3 is characterized in that: The i-th first world coordinate is obtained by transforming the i-th first pixel coordinate using the Monodepth2 monocular depth estimation network, and the i-th second world coordinate is obtained by transforming the i-th second pixel coordinate using the Monodepth2 monocular depth estimation network.

6. The intelligent early warning method for construction hazards according to claim 3 is characterized in that: The world coordinates of the construction workers and the world coordinates of the construction machinery are calculated by the following formula: ； ； ； ； ； ； ； ； In the above formula, represents the world coordinates of the construction worker, , , Respectively Coordinate values ​​along the X, Y, and Z directions, Represents the world coordinates of the construction machinery, , , Respectively Coordinate values ​​along the X, Y, and Z directions.

7. The intelligent early warning method for construction hazards according to claim 6 is characterized in that: The distance between construction workers and construction machinery is calculated by the following formula: ； In the above formula, Indicates the distance value between construction workers and construction machinery.

8. The intelligent early warning method for construction hazards according to claim 4 is characterized in that: The moving speed of construction workers and construction machinery is calculated by the following formula: ； ； ； ； ； ； In the above formula, Indicates The distance between the first world corrected coordinate and the i-th first world corrected coordinate, represents the jth first velocity, Indicates the frequency of a frame, Indicates the moving speed of the construction workers; , , Respectively represent The coordinate values ​​of the first world corrected coordinates along the X, Y, and Z directions; Indicates The distance between the second world corrected coordinate and the i-th second world corrected coordinate, represents the jth second speed, Indicates the moving speed of the construction machinery. , , Respectively represent The coordinate values ​​of the second world corrected coordinates along the X, Y, and Z directions.

9. The intelligent early warning method for construction hazards according to claim 8 is characterized in that: The specific method for performing the hazard warning evaluation in step S5 is: setting the speed setting value to V, the turning radius of the construction machinery to A, and the blind spot distance of the construction machinery to B, taking the largest value between A and B as C, and taking the smallest value between A and B as D; (1) When conducting a hazard warning assessment, the following contents shall be considered: when C, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when When , it is judged to be a safe state; (2) When conducting a hazard warning assessment, the following contents shall be considered: when D, it is judged as a high-risk state; when When the patient is in a moderate-risk state, when When the risk is low, it is judged as a low-risk state; when , it is judged to be a safe state.

10. The intelligent early warning method for construction hazards according to claim 9, characterized in that: indivual Form the first velocity data set, denoted as , indivual The second velocity data set is formed, denoted as ; The speed setting value is calculated by the following formula: ； In the above formula, , Respectively represent the maximum first speed and the minimum first speed in the first speed data set, , They respectively represent the maximum second speed and the minimum second speed in the second speed data set.

Images