Fall detection method, device and equipment for high-altitude operation and storage medium

Abstract

The invention discloses a drop detection method, device and equipment for high-altitude operation and a storage medium, and relates to the technical field of electric digital data processing. According to the method, the mutual relation between the three-dimensional detection frame and the boundary of the constructor is visually analyzed, so that misjudgment caused by the fact that a two-dimensional detection frame generated by an existing visual algorithm has no world mapping relation is avoided, and meanwhile misjudgment of the constructor when the three-dimensional detection frame and the vertical boundary are instantaneously intersected under the normal condition is also avoided; according to the technical scheme, timing is started only when the midpoint of the three-dimensional detection frame exceeds the vertical boundary, and it is judged that falling is about to occur only when the three-dimensional detection frame continuously exceeds the vertical boundary for a certain duration (such as one second, two seconds, three seconds and the like), so that false triggering of constructors in normal work is avoided, and the falling detection result is accurate.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a method, device, equipment and storage medium for detecting a fall during high-altitude operations. Background Art

[0002] The construction industry is an important material production sector and one of the pillar industries of my country's national economy, playing an important role in improving living conditions, improving infrastructure, absorbing labor employment, and promoting economic growth. With the continuous advancement of my country's urbanization process, the scale of construction projects will continue to expand, and the quality and safety of construction cannot be taken lightly. Operation risks seriously threaten the lives of construction workers.

[0003] In aerial work, in addition to wearing safety protective gear on a daily basis to prevent accidental physical injuries, the falling behavior of construction workers and the subsequent risks such as falls, bumps, and sudden illness caused by falling behavior also threaten the personal safety of construction workers. Therefore, it is necessary to detect the fall risk of construction workers during the construction process.

[0004] At present, during high-altitude operations, there is a possibility that construction workers may accidentally fall out of hanging baskets, scaffolds, etc. due to unstable center of gravity, accidental bumps, or other reasons caused by other equipment. Since construction workers usually use safety ropes when working at high altitudes, the construction workers will not fall to the ground. Instead, the safety rope will be tightened after a short fall, so that the construction workers are suspended in mid-air. However, if the inspection personnel do not go to the site for inspection, or the construction workers in a suspended state are not convenient to seek help, it may not be discovered in time that the construction workers have fallen, thus affecting the personal safety of the construction workers. Summary of the invention

[0005] The main purpose of the present application is to provide a fall detection method, device, equipment and storage medium for high-altitude operations to solve the problem that the prior art cannot promptly detect the fall of construction workers.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A fall detection method for high-altitude work, the fall detection method is applied to construction workers located in a construction site, the construction site is separated from the external ground by a preset distance, and the fall detection method includes:

[0008] Step S1, obtaining the edge of the construction site and constructing a vertical boundary in a direction away from the external ground along the vertical direction;

[0009] Step S2, defining at least two response time lengths based on the size of the preset distance, and the values ​​of all response time lengths decrease linearly with the linear increase of the preset distance;

[0010] Step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm;

[0011] Step S4, obtaining the real-time distance between the construction site and the external ground;

[0012] Step S5, obtaining a response duration that matches the real-time distance as a real-time response duration;

[0013] Step S6, obtaining the intersection duration between the three-dimensional detection frame and the vertical boundary;

[0014] Step S7, determining whether the intersection duration is greater than or equal to the real-time response duration, if the intersection duration is greater than or equal to the real-time response duration, executing step S8;

[0015] Step S8, determining whether the midpoint of the three-dimensional detection frame crosses the vertical boundary, if the midpoint of the three-dimensional detection frame crosses the vertical boundary, executing step S9;

[0016] Step S9, determining that the construction worker is about to fall.

[0017] As a further improvement of the present application, step S1, obtaining the edge of the construction site and constructing a vertical boundary in a direction away from the external ground along the vertical direction, includes:

[0018] Step S11, obtaining a digital elevation model of the construction site through a preset strategy;

[0019] Step S12, obtaining the elevation coordinates of the edge of the construction site and defining it as the lower edge of the vertical boundary;

[0020] Step S13, obtaining the upper edge of the vertical boundary based on the z-axis coordinate of the elevation coordinate and a preset height value;

[0021] Step S14: vertically connect the upper edge and the lower edge to obtain the vertical boundary.

[0022] As a further improvement of the present application, the preset strategy includes one or more combinations of remote sensing interpretation, on-site surveying and measurement, drone photogrammetry, three-dimensional laser scanning, radar scanning, image recognition, and visual algorithms.

[0023] As a further improvement of the present application, step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm, includes:

[0024] Step S31, obtaining a two-dimensional detection frame of the construction worker through the visual algorithm;

[0025] Step S32, defining a world coordinate system according to any horizontal plane and any vertical plane of the construction site;

[0026] Step S33, mapping the two-dimensional detection frame to the world coordinate system through a homography matrix to form a mapping detection frame, and the mapping detection frame is parallel or coplanar with any vertical plane;

[0027] Step S34, obtaining the symmetry axis of the mapping detection frame in the vertical direction;

[0028] Step S35, rotating the mapping detection frame based on the symmetry axis;

[0029] Step S36, obtaining a complete rotation trajectory of the mapped detection frame, and defining the complete rotation trajectory as the three-dimensional detection frame.

[0030] As a further improvement of the present application, step S31, obtaining the two-dimensional detection frame of the construction worker through the visual algorithm, includes:

[0031] Step S311, in response to an entry signal of the construction worker entering the construction site, acquiring image data matching the entry signal through an external preset camera terminal;

[0032] Step S312, dividing the image data into a preset number of grids on average;

[0033] Step S313, predicting a plurality of bounding boxes for the construction workers in the image data according to the visual algorithm based on all grids;

[0034] Step S314, respectively obtain the confidence of each bounding box, obtain the bounding box with the largest confidence and mark it as the first-order bounding box;

[0035] Step S315, calculating the intersection-over-union ratio of each other bounding box with the first-order bounding box;

[0036] Step S316, selecting a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a second-order bounding box;

[0037] Step S317: Obtain a second-order bounding box with the highest confidence and define it as the two-dimensional detection box.

[0038] As a further improvement of the present application, in step S9, it is determined that the construction worker is about to fall, and then the following steps are included:

[0039] Step S10, generating an impending fall warning signal based on the impending fall behavior;

[0040] Step S20, sending the impending fall warning signal to an external receiving end.

[0041] As a further improvement of the present application, in step S9, it is determined that the construction worker is about to fall, and then the following steps are included:

[0042] Step S100, continuously acquiring the acceleration of the construction worker from the moment the falling behavior is about to occur;

[0043] Step S200, determining whether the acceleration exceeds a preset acceleration threshold, if the acceleration exceeds the preset acceleration threshold, executing step S30;

[0044] Step S300, generating a drop signal and sending it to an external receiving end.

[0045] In order to achieve the above objectives, this application also provides the following technical solutions:

[0046] A fall detection device for high-altitude work, the fall detection device for high-altitude work is applied to the above-mentioned fall detection method, and the fall detection device comprises:

[0047] A vertical boundary construction module, used to obtain the edge of the construction site and construct a vertical boundary in a direction away from the external ground along the vertical direction;

[0048] A response time definition module, used to define at least two response time lengths based on the size of the preset distance, and the values ​​of all response time lengths decrease linearly with the linear increase of the preset distance;

[0049] A three-dimensional detection frame acquisition module, used to acquire a three-dimensional detection frame of the construction worker based on a visual algorithm;

[0050] A real-time distance acquisition module is used to acquire the real-time distance between the construction site and the external ground;

[0051] A real-time response duration acquisition module, used to acquire a response duration matching the real-time distance as the real-time response duration;

[0052] An intersection duration acquisition module, used to acquire the intersection duration between the three-dimensional detection frame and the vertical boundary;

[0053] An intersection duration judgment module, used to judge whether the intersection duration is greater than or equal to the real-time response duration;

[0054] A three-dimensional detection frame judgment module, configured to judge whether the midpoint of the three-dimensional detection frame passes through the vertical boundary if the intersection duration is greater than or equal to the real-time response duration;

[0055] The module for determining that a construction worker is about to fall is used to determine that the construction worker is about to fall if the midpoint of the three-dimensional detection frame passes through the vertical boundary.

[0056] In order to achieve the above objectives, this application also provides the following technical solutions:

[0057] An electronic device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the above-mentioned fall detection method for high-altitude operations is implemented.

[0058] To achieve the above objectives, this application also provides the following technical solutions:

[0059] A storage medium stores program instructions, which, when executed by a processor, implement the above-mentioned fall detection method for high-altitude operations.

[0060] The present application constructs a vertical boundary by acquiring the edge of the construction site along the vertical direction away from the external ground; defines at least two response times based on the size of a preset distance, and the values ​​of all response times decrease linearly with the linear increase of the preset distance; acquires a three-dimensional detection frame of the construction personnel based on a visual algorithm; acquires the real-time distance between the construction site and the external ground; acquires a response time that matches the real-time distance as the real-time response time; acquires the intersection time of the three-dimensional detection frame and the vertical boundary; determines whether the intersection time is greater than or equal to the real-time response time, and if the intersection time is greater than or equal to the real-time response time, determines whether the midpoint of the three-dimensional detection frame crosses the vertical boundary, and if the midpoint of the three-dimensional detection frame crosses the vertical boundary, determines that the construction personnel are about to fall. The present application avoids misjudgment caused by the lack of world mapping relationship between the two-dimensional detection frame generated by the existing visual algorithm through visual analysis of the relationship between the construction workers in the three-dimensional detection frame. It also avoids misjudgment when the three-dimensional detection frame of the construction workers instantaneously intersects with the vertical boundary under normal circumstances. The timing starts only when the midpoint of the three-dimensional detection frame exceeds the vertical boundary, and it is determined that a fall is about to occur after the three-dimensional detection frame continues to exceed the vertical boundary for a certain period of time (for example, one second, two seconds, three seconds, etc.). This avoids false triggering of construction workers in normal work, making the fall detection results of the present application more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the process steps of an embodiment of a fall detection method for high-altitude work in this application;

[0062] Figure 2 This is a functional module diagram of an embodiment of a fall detection device for high-altitude operations of the present application;

[0063] Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application;

[0064] Figure 4 This is a schematic diagram of the structure of an embodiment of the storage medium of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The terms "first", "second" and "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of the present application (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or power device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or power devices.

[0067] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0068] like Figure 1 As shown, this embodiment provides an embodiment of a fall detection method for aerial work. In this embodiment, the fall detection method is applied to construction workers located in a construction site, which is separated from the external ground by a preset distance, such as a hanging basket, scaffolding, steel beam, unprotected high platform, etc.

[0069] The fall detection method comprises the following steps:

[0070] Step S1, obtaining the edge of the construction site and constructing a vertical boundary along the vertical direction away from the external ground.

[0071] Preferably, the construction site with a height difference from the ground is usually of regular shape, for example, the top view of a hanging basket or a scaffold is usually rectangular, so the shape of the vertical boundary is usually rectangular.

[0072] Step S2, defining at least two response time lengths based on the size of the preset distance, and the values ​​of all response time lengths decrease linearly with the linear increase of the preset distance.

[0073] For example, when the preset distance is (0,5) meters, the response time is 3 seconds; when the preset distance is [5,10) meters, the response time is 2 seconds; when the preset distance is more than 10 meters, the response time is 1 second.

[0074] Preferably, the design intention of this step is to take into account the diversity of high-altitude operations. When the preset distance is (0,5) meters, the types of construction work may be mostly stacking, painting, etc., and the construction workers may need to extend their hands beyond the vertical boundary in a short time. As the preset distance increases, the protective measures become more stringent. For example, when the preset distance exceeds 10 meters, the types of construction work are mostly cleaning of glass curtain walls, installation of window air conditioners and other equipment. At this time, there is no need to extend the limbs beyond the vertical boundary, so the response time is reduced accordingly.

[0075] Step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm.

[0076] Preferably, the three-dimensional detection frame needs to be obtained by mapping the two-dimensional detection frame to the world. Usually, the radar detection data of the unified shooting position is required, and the value of the homography matrix is ​​jointly solved by the image coordinates and the world coordinates. Then, the two-dimensional detection frame is converted into a three-dimensional detection frame that conforms to the world coordinates through the solved homography matrix.

[0077] Step S4, obtaining the real-time distance between the construction site and the external ground.

[0078] Preferably, the real-time distance can be achieved through existing technical means such as infrared ranging.

[0079] Step S5, obtaining a response duration that matches the real-time distance as the real-time response duration.

[0080] Step S6, obtaining the intersection duration between the three-dimensional detection frame and the vertical boundary.

[0081] Step S7, determining whether the intersection duration is greater than or equal to the real-time response duration. If the intersection duration is greater than or equal to the real-time response duration, executing step S8.

[0082] Step S8, determining whether the midpoint of the three-dimensional detection frame crosses the vertical boundary. If the midpoint of the three-dimensional detection frame crosses the vertical boundary, executing step S9.

[0083] Step S9, determining that the construction worker is about to fall.

[0084] Furthermore, in step S1, the edge of the construction site is obtained to construct a vertical boundary in a direction away from the external ground in a vertical direction. This step specifically includes the following steps:

[0085] Step S11, obtaining a digital elevation model of the construction site through a preset strategy.

[0086] Step S12, obtaining the elevation coordinates of the edge of the construction site and defining it as the lower edge of the vertical boundary.

[0087] Step S13, based on the z-axis coordinate of the elevation coordinate and the preset height value, obtain the upper edge of the vertical boundary.

[0088] Step S14: vertically connect the upper edge and the lower edge to obtain a vertical boundary.

[0089] Furthermore, the preset strategies include one or more combinations of remote sensing interpretation, on-site mapping and measurement, drone photogrammetry, three-dimensional laser scanning, radar scanning, image recognition, and visual algorithms.

[0090] Furthermore, step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm, includes:

[0091] Step S31, obtaining a two-dimensional detection frame of the construction worker through a visual algorithm.

[0092] Step S32, defining a world coordinate system according to any horizontal plane and any vertical plane of the construction site.

[0093] Step S33: Map the two-dimensional detection frame to the world coordinate system through the homography matrix to form a mapped detection frame, and the mapped detection frame is parallel or coplanar with any vertical plane.

[0094] Preferably, in practical applications, radar data of the same scene needs to be added to ensure the accuracy of detection box mapping.

[0095] Preferably, the homography matrix can be obtained by express.

[0096] Among them, u and v are the location information of the current device, x and y are the pixel coordinate values ​​of the current device in the image data, H is the homography matrix, and By solving the homography matrix, all radar points whose pixel coordinate system coincides with the world coordinate system of the same external scene are obtained.

[0097] Step S34, obtaining the symmetry axis of the mapped detection frame in the vertical direction.

[0098] Step S35, rotating the mapped detection frame based on the axis of symmetry.

[0099] Step S36, obtaining the complete rotation trajectory of the mapped detection frame, and defining the complete rotation trajectory as a three-dimensional detection frame.

[0100] Preferably, the three-dimensional detection frame is in a cylindrical shape, so that when the construction worker turns around, the size of the three-dimensional detection frame remains almost unchanged.

[0101] Furthermore, step S31, obtaining a two-dimensional detection frame of the construction worker through a visual algorithm, includes:

[0102] Step S311, in response to the entry signal of the construction personnel entering the construction site, image data matching the entry signal is acquired through an external preset camera terminal.

[0103] Step S312, dividing the image data into a preset number of grids on average.

[0104] Preferably, the size of the original picture of the image data may be adjusted to 448×448, and then the resized picture may be evenly divided into S×S (eg, 7×7) grids, and the size of each grid is 64×64.

[0105] Preferably, each grid is used to predict the horizontal coordinate, vertical coordinate, width, height of N detection boxes, and the confidence of each detection box, that is, each grid needs to predict N×(4+1) values.

[0106] Step S313, predicting several bounding boxes for the construction workers in the image data based on all grids according to a visual algorithm.

[0107] Preferably, if the center of an object lies on a certain grid, then the grid is responsible for predicting the bounding box of the object.

[0108] Step S314, respectively obtain the confidence of each bounding box, obtain the bounding box with the maximum confidence and mark it as the first-order bounding box.

[0109] Step S315 , calculating the intersection-over-union ratio of each other bounding box with the first-order bounding box.

[0110] Step S316: Select a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a second-order bounding box.

[0111] Step S317: Obtain a second-order bounding box with the highest confidence and define it as a two-dimensional detection box.

[0112] It can be understood that each grid needs to predict N (x, y, w, h, confidence); where (x, y) is the offset of the center of the detection box relative to the grid, (w, h) is the ratio of the detection box to the above resized image, and (confidence) is the confidence of the grid, which takes a value of 1 or 0.

[0113] Preferably, the confidence level can be understood as whether there is a target in the current grid and the accuracy of the detection frame.

[0114] For example: suppose there is an object in a resized image, and the width and height of the resized image are (w a ,h a )but:

[0115] Divide the image into 7×7 (S×S) grids evenly. There is a grid located at the center of the target. The coordinates of the grid are (x i ,y i ), let the coordinates of the center of the target be (x a ,y a ), the above offset (x b ,y b ):

[0116] Preferably, in actual detection, if the predicted detection box and the actual bounding box overlap perfectly, the intersection-and-union ratio is 1. In actual application, the value of the first preset threshold can generally be set to 0.5 to determine whether the predicted bounding box is correct, and the accuracy of the bounding box is positively correlated with the intersection-and-union ratio.

[0117] Preferably, the YOLO algorithm also needs to train the detection frame to improve the accuracy of target detection.

[0118] Next, the training model is trained using a preset pedestrian and vehicle training set, and the weights and biases of the training model are iteratively adjusted a first preset number of times using a back propagation algorithm to reduce the value of the loss function of the training model.

[0119] Preferably, the loss function is as follows:

[0120]

[0121] in, is the indicator function of whether the j detection box of the i-th grid is responsible for the target, and its value is 1 or 0; x i ,y i 、w i 、h i , C iThey correspond to the i-th (x, y, w, h, confidence) prediction values ​​respectively.

[0122] It can be understood that the loss function includes the coordinate value deviation of the detection box, the confidence deviation, and the prediction probability deviation (or category deviation).

[0123] in, is the detection frame midpoint loss in the coordinate value deviation, is the loss of detection box width and height in coordinate value deviation, is the confidence deviation, is the deviation of the predicted probability (or class deviation).

[0124] Among them, λ coord is the positioning error penalty, generally λ coord =5;S 2 That is, the S×S grids mentioned above; B is the number of bounding boxes; and is the estimated value of the horizontal and vertical coordinates of the midpoint of the i-th bounding box; and is the estimated value of the width and height of the i-th bounding box; C i is the confidence of the i-th bounding box; is the estimated value of the confidence of the i-th bounding box; noobj is the confidence prediction loss, usually λ noobj =0.5; p i (c) is the category probability of the i-th bounding box; is the estimated value of the category probability of the i-th bounding box; p i (c) with The c in it corresponds to classes.

[0125] It should be noted that since each grid does not necessarily contain a target, if there is no target in the grid, the value of (confidence) will be 0, making the gradient span in the subsequent back propagation algorithm too large, so λ is introduced coord To control the loss of the predicted position of the detection box, and introduce λ noobj Controls the penalty for objects not existing within a single grid.

[0126] Furthermore, in step S9, it is determined that the construction worker is about to fall, and then the following steps are included:

[0127] Step S10, generating an impending fall warning signal based on the impending fall behavior.

[0128] Step S20, sending a falling warning signal to an external receiving end.

[0129] Furthermore, in step S9, it is determined that the construction worker is about to fall, and then the following steps are included:

[0130] Step S100, continuously obtaining the acceleration of the construction worker from the moment the falling behavior is about to occur.

[0131] Preferably, the acceleration of the construction workers can be acquired by wearing an accelerometer.

[0132] Step S200, determining whether the acceleration exceeds a preset acceleration threshold, if the acceleration exceeds the preset acceleration threshold, executing step S30.

[0133] Step S300, generating a drop signal and sending it to an external receiving end.

[0134] This embodiment constructs a vertical boundary by acquiring the edge of the construction site along the vertical direction away from the external ground; defines at least two response times based on the size of a preset distance, and the values ​​of all response times decrease linearly with the linear increase of the preset distance; acquires a three-dimensional detection frame of the construction personnel based on a visual algorithm; acquires the real-time distance between the construction site and the external ground; acquires a response time matching the real-time distance as the real-time response time; acquires the intersection time of the three-dimensional detection frame and the vertical boundary; determines whether the intersection time is greater than or equal to the real-time response time, and if the intersection time is greater than or equal to the real-time response time, determines whether the midpoint of the three-dimensional detection frame crosses the vertical boundary, and if the midpoint of the three-dimensional detection frame crosses the vertical boundary, determines that the construction personnel is about to fall. This embodiment uses visual analysis to analyze the relationship between the construction workers in the three-dimensional detection frame and the boundary, thereby avoiding misjudgment caused by the lack of world mapping relationship in the two-dimensional detection frame generated by the existing visual algorithm. It also avoids misjudgment when the three-dimensional detection frame of the construction workers intersects with the vertical boundary instantaneously under normal circumstances. The timing starts only when the midpoint of the three-dimensional detection frame exceeds the vertical boundary, and it is determined that a fall is about to occur after the three-dimensional detection frame continues to exceed the vertical boundary for a certain period of time (for example, one second, two seconds, three seconds, etc.). This avoids false triggering of construction workers in normal work, making the fall detection result of this embodiment more accurate.

[0135] like Figure 2 As shown, this embodiment provides an embodiment of a fall detection device for high-altitude operations. In this embodiment, the fall detection device for high-altitude operations is applied to the fall detection method as described above.

[0136] Specifically, the fall detection device includes a vertical boundary construction module 1, a response time definition module 2, a three-dimensional detection frame acquisition module 3, a real-time distance acquisition module 4, a real-time response time acquisition module 5, an intersection time acquisition module 6, an intersection time judgment module 7, a three-dimensional detection frame judgment module 8, and an impending fall behavior judgment module 9, which are electrically connected in sequence.

[0137] Among them, the vertical boundary construction module 1 is used to obtain the edge of the construction site and construct a vertical boundary along the vertical direction away from the external ground; the response time definition module 2 is used to define at least two response times based on the size of the preset distance, and the values ​​of all response times decrease linearly with the linear increase of the preset distance; the three-dimensional detection frame acquisition module 3 is used to obtain the three-dimensional detection frame of the construction personnel based on the visual algorithm; the real-time distance acquisition module 4 is used to obtain the real-time distance between the construction site and the external ground; the real-time response time acquisition module 5 is used to obtain the response time matching the real-time distance as the real-time response time; the intersection time acquisition module 6 is used to obtain the intersection time of the three-dimensional detection frame and the vertical boundary; the intersection time judgment module 7 is used to judge whether the intersection time is greater than or equal to the real-time response time; the three-dimensional detection frame judgment module 8 is used to judge whether the midpoint of the three-dimensional detection frame passes through the vertical boundary if the intersection time is greater than or equal to the real-time response time; the module for judging whether a falling behavior is about to occur is used to judge that the construction personnel are about to fall if the midpoint of the three-dimensional detection frame passes through the vertical boundary.

[0138] Furthermore, the vertical boundary construction module 1 specifically includes a first vertical boundary construction submodule, a second vertical boundary construction submodule, a third vertical boundary construction submodule, and a fourth vertical boundary construction submodule which are electrically connected in sequence; the fourth vertical boundary construction submodule is electrically connected to the response duration definition module 2.

[0139] Among them, the first vertical boundary construction submodule is used to obtain the digital elevation model of the construction site through a preset strategy; the second vertical boundary construction submodule is used to obtain the elevation coordinates of the edge of the construction site and define it as the lower edge of the vertical boundary; the third vertical boundary construction submodule is used to obtain the upper edge of the vertical boundary based on the z-axis coordinates of the elevation coordinates and the preset height value; the fourth vertical boundary construction submodule is used to vertically connect the upper edge and the lower edge to obtain the vertical boundary.

[0140] Furthermore, the preset strategies carried by the first vertical boundary construction submodule include one or more combinations of remote sensing interpretation, on-site mapping and measurement, drone photogrammetry, three-dimensional laser scanning, radar scanning, image recognition, and visual algorithms.

[0141] Furthermore, the three-dimensional detection frame acquisition module 3 specifically includes a first three-dimensional detection frame acquisition submodule, a second three-dimensional detection frame acquisition submodule, a third three-dimensional detection frame acquisition submodule, a fourth three-dimensional detection frame acquisition submodule, a fifth three-dimensional detection frame acquisition submodule, and a sixth three-dimensional detection frame acquisition submodule, which are electrically connected in sequence; the first three-dimensional detection frame acquisition submodule is electrically connected to the response time definition module 2, and the sixth three-dimensional detection frame acquisition submodule is electrically connected to the real-time distance acquisition module 4.

[0142] Among them, the first three-dimensional detection frame acquisition submodule is used to acquire the two-dimensional detection frame of the construction workers through a visual algorithm; the second three-dimensional detection frame acquisition submodule is used to define the world coordinate system according to any horizontal plane and any vertical plane of the construction site; the third three-dimensional detection frame acquisition submodule is used to map the two-dimensional detection frame to the world coordinate system through a homography matrix to form a mapping detection frame, and the mapping detection frame is parallel or coplanar with any vertical plane; the fourth three-dimensional detection frame acquisition submodule is used to acquire the symmetry axis of the mapping detection frame in the vertical direction; the fifth three-dimensional detection frame acquisition submodule is used to rotate the mapping detection frame based on the symmetry axis; the sixth three-dimensional detection frame acquisition submodule is used to acquire the complete rotation trajectory of the mapping detection frame, and define the complete rotation trajectory as a three-dimensional detection frame.

[0143] Furthermore, the first three-dimensional detection frame acquisition submodule specifically includes a first three-dimensional detection frame acquisition unit, a second three-dimensional detection frame acquisition unit, a third three-dimensional detection frame acquisition unit, a fourth three-dimensional detection frame acquisition unit, a fifth three-dimensional detection frame acquisition unit, a sixth three-dimensional detection frame acquisition unit, and a seventh three-dimensional detection frame acquisition unit, which are electrically connected in sequence; the first three-dimensional detection frame acquisition unit is electrically connected to the response duration definition module 2, and the seventh three-dimensional detection frame acquisition unit is electrically connected to the second three-dimensional detection frame acquisition submodule.

[0144] Among them, the first three-dimensional detection frame acquisition unit is used to respond to the entry signal of the construction personnel entering the construction site, and obtain the image data matching the entry signal through the external preset camera terminal; the second three-dimensional detection frame acquisition unit is used to evenly divide the image data into a preset number of grids; the third three-dimensional detection frame acquisition unit is used to predict several bounding boxes for the construction personnel in the image data based on all grids according to the visual algorithm; the fourth three-dimensional detection frame acquisition unit is used to obtain the confidence of each bounding box respectively, and obtain the bounding box with the largest confidence and mark it as a first-order bounding box; the fifth three-dimensional detection frame acquisition unit is used to calculate the intersection and union ratio of each other bounding box with the first-order bounding box; the sixth three-dimensional detection frame acquisition unit is used to select the bounding box whose intersection and union ratio is greater than or equal to the preset threshold as the second-order bounding box; the seventh three-dimensional detection frame acquisition unit is used to obtain the second-order bounding box with the highest confidence and define it as a two-dimensional detection frame.

[0145] Furthermore, the fall detection device also includes an impending fall warning signal generating module and an impending fall warning signal sending module which are electrically connected in sequence; the impending fall warning signal generating module is electrically connected to the impending fall behavior determining module 9.

[0146] The imminent fall warning signal generating module is used to generate an imminent fall warning signal based on the imminent fall behavior; the imminent fall warning signal sending module is used to send the imminent fall warning signal and send it to an external receiving end.

[0147] Furthermore, the fall detection device also includes a construction worker acceleration acquisition module, a construction worker acceleration judgment module, and a fall signal generation and sending module which are electrically connected in sequence; the construction worker acceleration acquisition module is electrically connected to the impending fall behavior judgment module 9.

[0148] Among them, the construction worker acceleration acquisition module is used to continuously obtain the acceleration of the construction worker from the moment the fall is about to occur; the construction worker acceleration judgment module is used to judge whether the acceleration exceeds the preset acceleration threshold; the fall signal generation and sending module is used to generate a fall signal and send it to the external receiving end if the acceleration exceeds the preset acceleration threshold.

[0149] It should be noted that this embodiment is an apparatus embodiment based on the above method embodiment. Additional contents such as the preference, extension, limitation, and example illustration of this embodiment can be found in the above method embodiment, and will not be elaborated in this embodiment.

[0150] This embodiment constructs a vertical boundary by acquiring the edge of the construction site along the vertical direction away from the external ground; defines at least two response times based on the size of a preset distance, and the values ​​of all response times decrease linearly with the linear increase of the preset distance; acquires a three-dimensional detection frame of the construction personnel based on a visual algorithm; acquires the real-time distance between the construction site and the external ground; acquires a response time matching the real-time distance as the real-time response time; acquires the intersection time of the three-dimensional detection frame and the vertical boundary; determines whether the intersection time is greater than or equal to the real-time response time, and if the intersection time is greater than or equal to the real-time response time, determines whether the midpoint of the three-dimensional detection frame crosses the vertical boundary, and if the midpoint of the three-dimensional detection frame crosses the vertical boundary, determines that the construction personnel is about to fall. This embodiment uses visual analysis to analyze the relationship between the construction workers in the three-dimensional detection frame and the boundary, thereby avoiding misjudgment caused by the lack of world mapping relationship in the two-dimensional detection frame generated by the existing visual algorithm. It also avoids misjudgment when the three-dimensional detection frame of the construction workers intersects with the vertical boundary instantaneously under normal circumstances. The timing starts only when the midpoint of the three-dimensional detection frame exceeds the vertical boundary, and it is determined that a fall is about to occur after the three-dimensional detection frame continues to exceed the vertical boundary for a certain period of time (for example, one second, two seconds, three seconds, etc.). This avoids false triggering of construction workers in normal work, making the fall detection result of this embodiment more accurate.

[0151] Figure 3 An embodiment of the electronic device of the present application is shown. In this embodiment, the electronic device 9 includes a processor 91 and a memory 92 coupled to the processor 91.

[0152] The memory 92 stores program instructions for implementing the fall detection method for aerial work of any of the above-mentioned embodiments.

[0153] The processor 91 is used to execute program instructions stored in the memory 92 to perform fall detection for aerial work.

[0154] The processor 91 may also be referred to as a CPU (Central Processing Unit). The processor 91 may be an integrated circuit chip having a signal processing capability. The processor 91 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0155] Further, Figure 4 This is a schematic diagram of the structure of a storage medium of an embodiment of the present application. The storage medium 11 of this embodiment stores program instructions 111 that can implement all the above methods, wherein the program instructions 111 can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer power device (which can be a personal computer, server, or network power device, etc.) or a processor to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal power devices such as a computer, a server, a mobile phone, and a tablet.

[0156] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0158] The specific implementation methods of the invention are described in detail above, but they are only examples, and the present application is not limited to the specific implementation methods described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application, and therefore, the equalization, modification, and improvement made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A fall detection method for high-altitude work, the fall detection method is applied to construction workers located in a construction site, the construction site is separated from the external ground by a preset distance, characterized in that: The fall detection method comprises: Step S1, obtaining the edge of the construction site and constructing a vertical boundary in a direction away from the external ground along the vertical direction; Step S2, defining at least two response time lengths based on the size of the preset distance, and the values ​​of all response time lengths decrease linearly with the linear increase of the preset distance; Step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm; Step S4, obtaining the real-time distance between the construction site and the external ground; Step S5, obtaining a response duration that matches the real-time distance as a real-time response duration; Step S6, obtaining the intersection duration between the three-dimensional detection frame and the vertical boundary; Step S7, determining whether the intersection duration is greater than or equal to the real-time response duration, if the intersection duration is greater than or equal to the real-time response duration, executing step S8; Step S8, determining whether the midpoint of the three-dimensional detection frame crosses the vertical boundary, if the midpoint of the three-dimensional detection frame crosses the vertical boundary, executing step S9; Step S9, determining that the construction worker is about to fall.

2. The fall detection method according to claim 1, characterized in that: Step S1, obtaining the edge of the construction site and constructing a vertical boundary in a direction away from the external ground along the vertical direction, including: Step S11, obtaining a digital elevation model of the construction site through a preset strategy; Step S12, obtaining the elevation coordinates of the edge of the construction site and defining it as the lower edge of the vertical boundary; Step S13, obtaining the upper edge of the vertical boundary based on the z-axis coordinate of the elevation coordinate and a preset height value; Step S14: vertically connect the upper edge and the lower edge to obtain the vertical boundary.

3. The fall detection method according to claim 2, characterized in that: The preset strategy includes one or more combinations of remote sensing interpretation, on-site mapping and measurement, drone photogrammetry, three-dimensional laser scanning, radar scanning, image recognition, and visual algorithms.

4. The fall detection method according to claim 1, characterized in that: Step S3, obtaining a three-dimensional detection frame of the construction worker based on a visual algorithm, includes: Step S31, obtaining a two-dimensional detection frame of the construction worker through the visual algorithm; Step S32, defining a world coordinate system according to any horizontal plane and any vertical plane of the construction site; Step S33, mapping the two-dimensional detection frame to the world coordinate system through a homography matrix to form a mapping detection frame, and the mapping detection frame is parallel or coplanar with any vertical plane; Step S34, obtaining the symmetry axis of the mapping detection frame in the vertical direction; Step S35, rotating the mapping detection frame based on the symmetry axis; Step S36, obtaining a complete rotation trajectory of the mapped detection frame, and defining the complete rotation trajectory as the three-dimensional detection frame.

5. The fall detection method according to claim 4, characterized in that: Step S31, obtaining a two-dimensional detection frame of the construction worker through the visual algorithm, includes: Step S311, in response to an entry signal of the construction worker entering the construction site, acquiring image data matching the entry signal through an external preset camera terminal; Step S312, dividing the image data into a preset number of grids on average; Step S313, predicting a plurality of bounding boxes for the construction workers in the image data according to the visual algorithm based on all grids; Step S314, respectively obtain the confidence of each bounding box, obtain the bounding box with the largest confidence and mark it as the first-order bounding box; Step S315, calculating the intersection-over-union ratio of each other bounding box with the first-order bounding box; Step S316, selecting a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a second-order bounding box; Step S317: Obtain a second-order bounding box with the highest confidence and define it as the two-dimensional detection box.

6. The fall detection method according to claim 1, characterized in that: Step S9, determining that the construction worker is about to fall, then comprising: Step S10, generating an impending fall warning signal based on the impending fall behavior; Step S20, sending the impending fall warning signal to an external receiving end.

7. The fall detection method according to claim 1, characterized in that: Step S9, determining that the construction worker is about to fall, then comprising: Step S100, continuously acquiring the acceleration of the construction worker from the moment the falling behavior is about to occur; Step S200, determining whether the acceleration exceeds a preset acceleration threshold, if the acceleration exceeds the preset acceleration threshold, executing step S30; Step S300, generating a drop signal and sending it to an external receiving end.

8. A fall detection device for high-altitude work, the fall detection device for high-altitude work is applied to the fall detection method according to any one of claims 1 to 7, characterized in that: The fall detection device comprises: A vertical boundary construction module, used to obtain the edge of the construction site and construct a vertical boundary in a direction away from the external ground along the vertical direction; A response time definition module, used to define at least two response time lengths based on the size of the preset distance, and the values ​​of all response time lengths decrease linearly with the linear increase of the preset distance; A three-dimensional detection frame acquisition module, used to acquire a three-dimensional detection frame of the construction worker based on a visual algorithm; A real-time distance acquisition module is used to acquire the real-time distance between the construction site and the external ground; A real-time response duration acquisition module, used to acquire a response duration matching the real-time distance as the real-time response duration; An intersection duration acquisition module, used to acquire the intersection duration between the three-dimensional detection frame and the vertical boundary; An intersection duration judgment module, used to judge whether the intersection duration is greater than or equal to the real-time response duration; A three-dimensional detection frame judgment module, configured to judge whether the midpoint of the three-dimensional detection frame passes through the vertical boundary if the intersection duration is greater than or equal to the real-time response duration; The module for determining that a construction worker is about to fall is used to determine that the construction worker is about to fall if the midpoint of the three-dimensional detection frame passes through the vertical boundary.

9. An electronic device, characterized in that: It includes a processor and a memory coupled to the processor, the memory storing program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the fall detection method for high-altitude operations as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores program instructions, which, when executed by a processor, can implement a fall detection method for high-altitude operations as described in any one of claims 1 to 7.

Images