Fall detection method, device and equipment based on safety protector and storage medium

Abstract

The invention discloses a fall detection method, device and equipment based on a safety protector and a storage medium, and relates to the technical field of electric digital data processing. According to the method, the technical means of computer vision and acceleration slope dual verification is adopted, meanwhile, a framework in a figure detection frame is mapped to a construction site, the framework and the construction environment are located in the same perspective environment, perspective pulling caused by overlook shooting or look-up shooting of a shooting end is eliminated, and the safety of the construction site is improved. Therefore, the mapped skeleton conforms to a world coordinate system, the misjudgment rate of double verification is further reduced, and when the main trunk skeleton approaches to the transverse direction and the acceleration degree is too high, the main trunk skeleton is judged to fall down. According to the embodiment of the invention, the acceleration of the body movement of the main trunk of the constructor in the self-control range does not exceed the acceleration of the body when the body is out of control, so that misjudgment of the daily activity of the constructor caused by falling detection in the embodiment is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a fall detection method, device, equipment and storage medium based on safety gear. Background Art

[0002] The construction industry is an important material production sector and one of the pillar industries of my country's national economy, playing an important role in improving living conditions, improving infrastructure, absorbing labor employment, and promoting economic growth. With the continuous advancement of my country's urbanization process, the scale of construction projects will continue to expand, and the quality and safety of construction cannot be taken lightly. Operation risks seriously threaten the lives of construction workers.

[0003] In addition to wearing safety gear on a daily basis to prevent accidental physical injuries, construction workers' falls and subsequent risks such as falls, bumps, falls, and sudden illnesses caused by falls also threaten the personal safety of construction workers, so it is necessary to detect the risk of falls of construction workers during the construction process.

[0004] At present, the mainstream and intelligent fall detection method is computer vision detection, which uses a camera to capture image information of human motion and then uses computer vision technology to analyze fall events. However, since the motion characteristics of a fall may be very similar to daily activities (such as sitting, lying down, bending over, etc.), it is not easy for computer vision detection to distinguish between fall and non-fall events, resulting in a high misjudgment rate. Summary of the invention

[0005] The main purpose of the present application is to provide a fall detection method, device, equipment and storage medium based on safety gear to solve the problem in the prior art that computer vision detection is not easy to distinguish between fall and non-fall events, resulting in a high misjudgment rate.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A fall detection method based on safety gear, the fall detection method is applied to construction workers who are in a construction site and wearing safety gear, each safety gear is respectively equipped with an accelerometer, the construction site has at least one preset shooting end, and the fall detection method comprises:

[0008] Step S1, generating a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis;

[0009] Step S2, responding to an entry signal from the construction personnel entering the construction site;

[0010] Step S3, taking the triggering time of the entry signal as the starting point, obtaining the acceleration reading of the accelerometer at intervals of a preset time period;

[0011] Step S4, generating an acceleration coordinate point with the current preset time period as the horizontal coordinate and the current acceleration reading as the vertical coordinate;

[0012] Step S5, inputting all coordinate points into the rectangular coordinate system and sequentially connecting them to form an acceleration change curve;

[0013] Step S6, obtaining a person detection frame matching the entry signal through a target detection algorithm;

[0014] Step S7, obtaining a key point skeleton in the person detection frame by using a human key point detection algorithm;

[0015] Step S8, defining a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site;

[0016] Step S9, mapping the key point skeleton to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates;

[0017] Step S10, obtaining the vector direction of the main trunk of the three-dimensional skeleton at intervals of the preset time period, and obtaining the angle between the vector direction and any horizontal plane;

[0018] Step S11, determining whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, executing step S12;

[0019] Step S12, obtaining the slope of the acceleration change curve at the same time;

[0020] Step S13, determining whether the slope exceeds a preset slope threshold, if the slope exceeds the preset slope threshold, executing step S14;

[0021] Step S14, determining whether the construction worker has fallen.

[0022] As a further improvement of the present application, step S6, obtaining a person detection frame matching the entry signal through a target detection algorithm, includes:

[0023] Step S61, in response to the entry signal, acquiring image data matching the entry signal through an external preset camera terminal;

[0024] Step S62, dividing the image data into a preset number of grids on average;

[0025] Step S63, predicting a plurality of first-order bounding boxes for the moving object in the image data based on all the grids according to the object detection algorithm, each first-order bounding box including at least one grid;

[0026] Step S64, defining the construction worker as having the highest confidence level;

[0027] Step S65, respectively obtaining the confidence of each first-order bounding box, and obtaining the bounding box with the largest confidence and marking it as the second-order bounding box;

[0028] Step S66, calculating the intersection-over-union ratio between the second-order bounding box and each of the other first-order bounding boxes;

[0029] Step S67, selecting a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a third-order bounding box;

[0030] Step S68, obtaining a third-order bounding box with the highest confidence and defining it as the person detection box.

[0031] As a further improvement of the present application, step S7, obtaining a key point skeleton in the person detection frame by using a human key point detection algorithm, includes:

[0032] Step S71, training the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtaining a VGGNet prediction model after the training is completed;

[0033] Step S72, inputting the portrait data in each person detection frame into the VGGNet prediction model to extract human body features;

[0034] Step S73, extracting a plane confidence map of the position of the body part in the current portrait data through the first branch of the two-branch CNN model;

[0035] Step S74, extracting the plane affinity vector field of the body part positions in the current portrait data through the second branch in the two-branch CNN model;

[0036] Step S75, calculating the key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

[0037] As a further improvement of the present application, in step S9, the key point skeleton is mapped to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates, before; in step S10, the vector direction of the main trunk of the three-dimensional skeleton is obtained at intervals of the preset time period, and the angle between the vector direction and any horizontal plane is obtained, and then, it includes:

[0038] Step S100, obtaining a plurality of stereo skeleton world coordinate sets of the stereo skeleton at a plurality of preset time periods as acquisition time points, wherein one acquisition time point corresponds to one stereo skeleton world coordinate set;

[0039] Step S200, learning and training each stereo skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model;

[0040] Step S300, predicting the stereo skeleton as a future stereo skeleton at least one unit second in the future by using the prediction model;

[0041] Step S400: Apply the future stereo skeleton to the stereo skeleton used in step S10 to obtain the vector direction of the main trunk.

[0042] As a further improvement of the present application, step S200, learning and training each stereo skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model, including:

[0043] Step S2001, integrating all three-dimensional skeleton world coordinate sets to form a total coordinate data set;

[0044] Step S2002, dividing the total coordinate data set into a training set and a validation set;

[0045] Step S2003, defining a topological relationship of the neural network model, wherein the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected;

[0046] Step S2004, inputting the training set into the input layer, and iterating through the neural network model, obtaining a root mean square error between the validation set and the current training result based on each iteration;

[0047] Step S2005, obtaining the minimum value of all root mean square errors, and obtaining the training result corresponding to the minimum value as the prediction model.

[0048] As a further improvement of the present application, the neural network model is characterized by formula (1):

[0049]

[0050] Wherein, y is the preset neural network model; x is the input layer and x={x1,x2,…,x n}, x n is the nth input node of the input layer, each input node corresponds to a training set, is the weight from the mth input node of the input layer to the nth input node of the hidden layer; is the bias of the nth input node connected to the hidden layer; tansig(x) is the transfer function, and The numbers in brackets are the number of layers.

[0051] As a further improvement of the present application, in step S14, it is determined that the construction worker has fallen, and then the following steps are included:

[0052] Step S1000, obtaining the angle after the falling behavior occurs at intervals of the preset time period;

[0053] Step S2000, obtaining the duration of the angle after the falling behavior occurs;

[0054] Step S3000, determining whether the duration is greater than or equal to a preset duration threshold, if the duration is greater than or equal to the preset duration threshold, executing step S4000;

[0055] Step S4000: Generate a manual intervention request and send it to an external receiving end.

[0056] In order to achieve the above objectives, this application also provides the following technical solutions:

[0057] A fall detection device based on a safety gear, the fall detection device based on the safety gear is applied to the fall detection method as described above, the fall detection device comprises:

[0058] A rectangular coordinate system generation module, used to generate a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis;

[0059] A construction worker entry signal response module, used to respond to the construction worker's entry signal into the construction site;

[0060] An acceleration reading acquisition module, used to acquire the acceleration reading of the accelerometer at intervals of a preset time period starting from the trigger time of the entry signal;

[0061] An acceleration coordinate point generating module, used to generate an acceleration coordinate point with a current preset time period as a horizontal coordinate and a current acceleration reading as a vertical coordinate;

[0062] An acceleration change curve generating module, used for inputting all coordinate points into the rectangular coordinate system and sequentially connecting them to form an acceleration change curve;

[0063] A task detection frame acquisition module, used to acquire a person detection frame matching the entry signal through a target detection algorithm;

[0064] A key point skeleton acquisition module, used to acquire the key point skeleton in the person detection frame through a human key point detection algorithm;

[0065] A world coordinate system definition module, used to define a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site;

[0066] A three-dimensional skeleton generation module, used for mapping the key point skeleton to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates;

[0067] A vector direction and angle acquisition module, used to acquire the vector direction of the main trunk of the three-dimensional skeleton at intervals of the preset time period, and to acquire the angle between the vector direction and any horizontal plane;

[0068] An angle determination module, used to determine whether the angle is less than or equal to a preset angle threshold;

[0069] An acceleration change curve slope acquisition module, used for acquiring the slope of the acceleration change curve at the same time if the angle is less than or equal to the preset angle threshold;

[0070] A slope determination module, used to determine whether the slope exceeds a preset slope threshold;

[0071] The falling behavior determination module is used to determine that the construction worker has a falling behavior if the slope exceeds the preset slope threshold.

[0072] In order to achieve the above objectives, this application also provides the following technical solutions:

[0073] An electronic device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the fall detection method based on safety gear as described above is implemented.

[0074] To achieve the above objectives, this application also provides the following technical solutions:

[0075] A storage medium stores program instructions, and when the program instructions are executed by a processor, the above-mentioned fall detection method based on safety gear can be implemented.

[0076] The present application generates a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; responds to the entry signal of the construction personnel entering the construction site; takes the trigger time of the entry signal as the starting point, and obtains the acceleration reading of the accelerometer at intervals of a preset time period; generates acceleration coordinate points with the current preset time period as the horizontal axis and the current acceleration reading as the vertical axis; inputs all coordinate points into the rectangular coordinate system and connects them in sequence to form an acceleration change curve; obtains a person detection frame that matches the entry signal through a target detection algorithm; obtains a key point skeleton in the person detection frame through a human key point detection algorithm ; Define a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site; map the key point skeleton to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates; obtain the vector direction of the main trunk of the three-dimensional skeleton at preset time intervals, and obtain the angle between the vector direction and any horizontal plane; determine whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, obtain the slope of the acceleration change curve at the same time; determine whether the slope exceeds the preset slope threshold, if the slope exceeds the preset slope threshold, determine that the construction worker has fallen. The present application adopts the technical means of dual verification of computer vision and acceleration slope, and at the same time maps the skeleton in the person detection frame to the construction site, so that the skeleton and the construction environment are located in the same perspective environment, eliminating the perspective pull caused by the overhead or upward shooting at the shooting end, so that the mapped skeleton conforms to the world coordinate system, further reducing the misjudgment rate of the dual verification, and only determines it as a fall when the main trunk skeleton approaches the horizontal direction and the acceleration is too high. The present application also utilizes the fact that the acceleration of the main trunk body movement of the construction personnel within the self-control range will not exceed the acceleration when the body is out of control, avoiding the fall detection of the present application from misjudging the daily activities of the construction personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a schematic diagram of the process steps of an embodiment of a fall detection method based on safety gear of the present application;

[0078] Figure 2 This is a schematic diagram of the time periods of a fall acceleration signal according to an embodiment of a fall detection method based on safety gear of the present application;

[0079] Figure 3 This is a functional module diagram of an embodiment of a fall detection device based on safety gear of the present application;

[0080] Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application;

[0081] Figure 5 This is a schematic diagram of the structure of an embodiment of the storage medium of the present application. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0083] The terms "first", "second" and "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of the present application (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or power device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or power devices.

[0084] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0085] like Figure 1 As shown, a fall detection method based on safety gear is applied to construction workers who are in a construction site and wearing safety gear. Each safety gear is equipped with an accelerometer, and the construction site has at least one preset shooting end.

[0086] Preferably, the safety gear is mainly safety gear worn on the body such as a helmet and protective clothing. The accelerometer of this embodiment is a conventional miniature inertial sensor, which is a mature existing technology, and the detailed internal structure and principle thereof will not be described in detail in this embodiment.

[0087] Specifically, the fall detection method includes the following steps:

[0088] Step S1, generating a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis.

[0089] Step S2, responding to the entry signal of the construction personnel entering the construction site.

[0090] Preferably, the entry signal can be triggered by a GPS module, wherein the entry signal is generated when the GPS module coincides with or is sufficiently close to the map location of any entrance or exit, or when the elevation coordinates of the GPS module are within the elevation coordinate set of any entrance or exit.

[0091] Preferably, the entrances and exits of current construction sites are generally equipped with gates, and construction workers can also pass through the gates to enter the construction site to generate an entry signal.

[0092] Step S3, taking the triggering time of the entry signal as the starting point, obtaining the acceleration reading of the accelerometer at intervals of a preset time period.

[0093] Preferably, since the entire falling process is relatively fast, the preset time period can be set to a shorter time period, such as 0.1s, 0.5s, 1s, etc., and the acceleration reading can be a conventional meter per square second.

[0094] Step S4, generating an acceleration coordinate point with the current preset time period as the horizontal coordinate and the current acceleration reading as the vertical coordinate.

[0095] Step S5, input all coordinate points into a rectangular coordinate system and connect them in sequence to form an acceleration change curve.

[0096] Step S6: obtaining a person detection frame matching the entry signal through a target detection algorithm.

[0097] Preferably, the target detection algorithm can be implemented through target detection algorithms VJ, HOG, DPMDetector; deep learning Two-stage target detection algorithms RCNN, SPPNet, FastRCNN, FasterRCNN; target detection Trick algorithms FPN, CascadeRCNN; deep learning one-stage target detection algorithms Yolo, X, SSD, Ret i naNet; deep learning Anchor-free target detection algorithms CornerNet, CenterNet, FCOS; Transformer-based target detection algorithm DETR, etc.

[0098] Preferably, this embodiment prefers the Yolo algorithm.

[0099] Step S7, obtaining the key point skeleton in the person detection frame through a human key point detection algorithm.

[0100] Step S8, defining a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site.

[0101] It is worth noting that the above-mentioned rectangular coordinate system is a two-dimensional coordinate system, while the world coordinate system is a three-dimensional coordinate system.

[0102] Step S9, mapping the key point skeleton to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates.

[0103] Preferably, in practical applications, the key point skeleton in the two-dimensional detection frame obtained by the YOLO algorithm is converted into a three-dimensional skeleton in the world coordinate system. Usually, radar assistance is required, and radar data of construction workers in the same scene needs to be added to ensure the accuracy of the mapping of the person detection frame.

[0104] Preferably, the homography matrix can be obtained by express.

[0105] Among them, a and b are the location information of the current construction personnel, x and y are the pixel coordinate values ​​of the current construction personnel in the image data, H is the homography matrix, and By solving the homography matrix, all radar points whose pixel coordinate system coincides with the world coordinate system of the same external scene are obtained.

[0106] It should be noted that the symbolic meaning of the homography matrix is ​​not interchangeable with the other symbolic meanings in this embodiment.

[0107] Preferably, the existing 2D detection model can also be used to locate the region of interest (ROI) on the RGB image, and then a pixel mapping strategy is adopted in the point cloud, and finally the initial 2D bounding box is mapped to the 3D space.

[0108] Step S10, obtaining the vector direction of the main trunk of the three-dimensional skeleton at preset time intervals, and obtaining the angle between the vector direction and any horizontal plane.

[0109] Preferably, the main trunk is from the chest to the abdomen. Generally, the key point detection obtains two shoulder key points and two hip key points. The midpoints of each point can be connected, and then the two midpoints can be connected to obtain the vector of the main trunk.

[0110] Step S11, determining whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, executing step S12.

[0111] Preferably, the preset angle threshold can be set to 10°

[0112] Step S12, obtaining the slope of the acceleration change curve at the same time.

[0113] Step S13, determining whether the slope exceeds a preset slope threshold, if the slope exceeds the preset slope threshold, executing step S14.

[0114] Preferably, the slope of the acceleration change curve reflects the change trend of the acceleration. Since falling is a behavior of the human body out of control, the acceleration will be significantly greater than voluntary bending over, lying flat, etc.

[0115] Preferably, the experimental data on human falls can be directly obtained from public channels, see Figure 2 , the acceleration signal during a fall can be divided into five stages. Among them, S1 and S2 are non-falling static periods, W is the weightlessness period, P is the first peak segment of acceleration when touching the ground, and V is the vibration period. Figure 2 The data show that the acceleration of a human body when falling and touching the ground can reach 4.4g (g is the acceleration due to gravity). The entire touching-the-ground process takes 0.15 seconds, and the acceleration increases sharply from 0.7g in the weightlessness stage to 4.4g, and the slope increases sharply to about 26.44, so the slope threshold of this embodiment can be set to 20.

[0116] Step S14, determining whether the construction worker has fallen.

[0117] Furthermore, step S6, obtaining a person detection frame matching the entry signal through a target detection algorithm, includes:

[0118] Step S61, in response to the entry signal, acquiring image data matching the entry signal through an external preset camera terminal.

[0119] Step S62, dividing the image data into a preset number of grids on average.

[0120] Preferably, the size of the original image of the image data may be adjusted to 448×448, and then the adjusted image may be evenly divided into S×S (eg, 7×7) grids, and the size of each grid is 64×64.

[0121] Preferably, each grid is used to predict the coordinates and width and height of N detection boxes, as well as the confidence of each detection box, that is, each grid needs to predict N×(4+1) values.

[0122] It should be noted that the "N" in these N detection boxes is not interoperable with the "N" in the global optimization algorithm.

[0123] Step S63 , predicting a plurality of first-order bounding boxes for the moving target in the image data based on all the grids according to the target detection algorithm, each first-order bounding box including at least one grid.

[0124] Preferably, if the center of an object lies on a certain grid, then that grid is responsible for predicting the bounding box of the object.

[0125] Step S64, defining the construction personnel with the highest confidence.

[0126] Step S65 , respectively obtain the confidence of each first-order bounding box, obtain the bounding box with the largest confidence and mark it as the second-order bounding box.

[0127] Step S66, calculating the intersection-over-union ratio between the second-order bounding box and each of the other first-order bounding boxes.

[0128] Preferably, the intersection-over-union ratio is a ratio obtained by dividing the intersection of the optimal detection box and each bounding box by the union of the optimal detection box and each bounding box.

[0129] Step S67 , selecting a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a third-order bounding box.

[0130] Preferably, the second preset threshold may be set to 0.5.

[0131] Step S68: Obtain the third-order bounding box with the highest confidence and define it as the person detection box.

[0132] It can be understood that each grid needs to predict N (x, y, w, h, confidence); where (x, y) is the offset of the center of the detection box relative to the grid, (w, h) is the ratio of the detection box to the above resized image, and (confidence) is the confidence of the grid, which takes a value of 1 or 0.

[0133] Preferably, the confidence level can be understood as whether there is a target in the current grid and the accuracy of the detection box.

[0134] For example: suppose there is an object in a resized image, and the width and height of the resized image are (w a ,h a )but:

[0135] Divide the image into 7×7 (S×S) grids evenly. There is a grid located at the center of the target. The coordinates of the grid are (x i ,y i ), let the coordinates of the center of the target be (x a ,y a ), the above offset (x b ,y b ):

[0136] Preferably, in actual detection, if the predicted detection box and the actual bounding box overlap perfectly, the intersection-and-union ratio is 1. In actual application, the intersection-and-union ratio can generally be set to 0.5 to determine whether the predicted bounding box is correct, and the more accurate the bounding box is, the more positively correlated it is with the intersection-and-union ratio.

[0137] Preferably, the YOLO algorithm also needs to train the detection frame to improve the accuracy of target detection.

[0138] It should be noted that the training model of the person detection frame is generally implemented directly through code, for example:

[0139]

[0140]

[0141] Among them, “*” is the target category that needs to be identified, which can be set to words related to construction workers such as “person” and “human” to identify construction workers.

[0142] Next, the above training model is trained using a preset training set, and the weights and biases of the training model are iteratively adjusted using a back-propagation algorithm to reduce the value of the loss function of the training model.

[0143] Preferably, the loss function is as follows:

[0144]

[0145] in, is the indicator function of whether the j detection box of the i-th grid is responsible for the target, and its value is 1 or 0; x i ,y i 、w i 、h i , C i They correspond to the i-th (x, y, w, h, confidence) prediction value, that is, the N×(4+1) values ​​mentioned above.

[0146] It can be understood that the loss function includes the coordinate value deviation of the detection box, the confidence deviation, and the prediction probability deviation (or category deviation).

[0147] in, is the detection frame midpoint loss in the coordinate value deviation, is the loss of detection box width and height in coordinate value deviation, is the confidence deviation, is the deviation of the predicted probability (or class deviation).

[0148] Among them, λ coord is the positioning error penalty, generally λcoord =5;S 2 That is, the S×S grids mentioned above; B is the number of bounding boxes; and is the estimated value of the horizontal and vertical coordinates of the midpoint of the i-th bounding box; and is the estimated value of the width and height of the i-th bounding box; C i is the confidence of the i-th bounding box; is the estimated value of the confidence of the i-th bounding box; noobj is the confidence prediction loss, usually λ noobj =0.5; p i (c) is the category probability of the i-th bounding box; is the estimated value of the category probability of the i-th bounding box; p i (c) with The c in it corresponds to classes.

[0149] It should be noted that since each grid does not necessarily contain a target, if there is no target in the grid, the value of (confidence) will be 0, making the gradient span in the subsequent back propagation algorithm too large, so λ is introduced coord To control the loss of the predicted position of the detection box, and introduce λ noobj Controls the loss of objects not existing in a single grid.

[0150] It should be noted that the meanings of all symbols in the extended content of the above YOLO algorithm are not interchangeable with other symbol meanings.

[0151] Further, step S7, obtaining a key point skeleton in the person detection frame by using a human key point detection algorithm, includes:

[0152] Step S71, training the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtaining the VGGNet prediction model after the training is completed.

[0153] Preferably, commonly used training sets for human key points include COCO Keypoints chal lenge, MPI IHumanPose Dataset, VGG Pose Dataset, CMU Panoptic Dataset, and OpenPose Dataset.

[0154] Step S72, input the portrait data in each person detection frame into the VGGNet prediction model to extract human body features.

[0155] Step S73, extracting a planar confidence map of the positions of body parts in the current portrait data through the first branch in the two-branch CNN model.

[0156] Step S74, extracting the planar affinity vector field of the body part positions in the current portrait data through the second branch in the two-branch CNN model.

[0157] Step S75, calculating the key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

[0158] In simple terms, an image is input, and features are extracted through a convolutional network to obtain a set of feature maps, which are then divided into two branches, using a two-branch CNN network to extract Part Conf i dence Maps (plane confidence maps) and Part Affiliity Fields (plane affinity vector fields); after obtaining the aforementioned PCMs and PAFs, the Part Association (key point set) is obtained through Bipartite Matching of the Hungarian algorithm.

[0159] Preferably, the bipartite graph matching of the Hungarian algorithm refers to a directed graph G = (V, E) in which the vertex set V is divided into two independent sets U and And each edge (e = (u, w)) connects one end from U and one end from W, and the process of finding the maximum number of undirected edge pairs makes these edges have no intersections.

[0160] Among them, the bipartite graph matching steps of the Hungarian algorithm are as follows:

[0161] Construct an augmented matrix: Convert the graph into a weighted adjacency matrix, with each row representing a node in U and each column representing a node in W. If there is an edge (u, w), the corresponding matrix element is the weight of the edge; otherwise it is 0.

[0162] Initialization: Set all matches to unmatched and create an initial zero-filled augmented matrix.

[0163] Loop: Perform the following steps until an exact match is found or no more pairs can be found.

[0164] Solve linear programming: Use Huffman coding or the shortest augmenting path algorithm to find a minimum augmenting path, that is, starting from an unmatched node and walking along the edge with smaller weight until encountering another matched node.

[0165] Update matching: adjust the matching along this path, add a matching edge and delete this path.

[0166] Termination condition: If an augmenting path can be found, continue the next round of search. If not found, it means that the current best match has been found and the algorithm ends.

[0167] Return result: Output the number of maximum matches currently found and the corresponding matching solutions.

[0168] Further, in step S9, the key point skeleton is mapped to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates, before; in step S10, the vector direction of the main trunk of the three-dimensional skeleton is obtained at intervals of a preset time period, and the angle between the vector direction and any horizontal plane is obtained, and then, it includes:

[0169] Step S100 , obtaining a plurality of stereo skeleton world coordinate sets of the stereo skeleton using a plurality of preset time periods as acquisition time points, wherein one acquisition time point corresponds to one stereo skeleton world coordinate set.

[0170] Step S200, learning and training each stereo skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model.

[0171] Step S300 , predicting a stereo skeleton as a future stereo skeleton of at least one unit second in the future by using a prediction model.

[0172] Step S400 , applying the future stereo skeleton to the stereo skeleton used in step S10 to obtain the vector direction of the main trunk.

[0173] Furthermore, in step S200, each stereo skeleton world coordinate set is learned and trained by a machine learning algorithm to obtain a prediction model, including:

[0174] Step S2001, integrating all three-dimensional skeleton world coordinate sets to form a total coordinate data set.

[0175] Step S2002, dividing the total coordinate data set into a training set and a validation set.

[0176] Preferably, the image data is usually divided into a training set and a validation set in a ratio of 80%:20%, that is, 80% of the data is the training set and 20% of the data is the validation set.

[0177] Step S2003, defining the topological relationship of the neural network model, the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected.

[0178] Step S2004, input the training set to the input layer, and iterate through the neural network model, and obtain the root mean square error between the verification set and the current training result based on each iteration.

[0179] Preferably, the root mean square error is Among them, RMSE is the root mean square error, n is the number of training sets, x i is the true value of the i-th training set, is the training result after the i-th training set is completed.

[0180] It should be noted that the root mean square error formula here is only used for explanation of the principle, and its letter symbols and meanings are not interchangeable with other formulas in this embodiment.

[0181] Step S2005, obtaining the minimum value of all root mean square errors, and obtaining the training result corresponding to the minimum value as the prediction model.

[0182] Furthermore, the neural network model is represented by formula (1):

[0183]

[0184] Among them, y is the preset neural network model; x is the input layer and x={x1,x2,…,x n}, x n is the nth input node of the input layer, each input node corresponds to a training set, is the weight from the mth input node of the input layer to the nth input node of the hidden layer; is the bias of the nth input node connected to the hidden layer; tansig(x) is the transfer function, and The numbers in brackets are the number of layers.

[0185] Preferably, the number in the bracket of the symbol in the formula (1) of this embodiment is the number of layers, for example The superscript (2) in the equation is the second layer, i.e., the hidden layer. The superscript (1, 2) represents the first layer to the second layer, that is, the input layer to the hidden layer.

[0186] It should be noted that the symbolic meaning of the transfer function is not interchangeable with other places.

[0187] Preferably, training model training a neural network usually requires providing a large amount of data, namely a data set; the data set is generally divided into three categories, namely the above-mentioned training set, validation set and test set.

[0188] Among them, one epoch (increase in number) is equal to the process of training once with all the samples in the training set. The so-called training once refers to one forward pass and one back pass. When the number of samples in an epoch (i.e., training set) is too large, one training may consume too much time, and it is not necessary to use all the data in the training set for each training. In this case, the entire training set needs to be divided into multiple small blocks, that is, divided into multiple batches for training. An epoch consists of one or more batches, where a batch is a part of the training set. Each training process uses only a part of the data, i.e., a batch. The process of training a batch is an iteration.

[0189] Preferably, the neural network training specifically includes a perceptron, which is composed of two layers of neurons. The input layer receives external input signals and transmits them to the output layer. The output layer is MP neurons. Let y j =f(∑ i w i ·x i -θ i ) is a step function, and given a training data set, the weight w i (i=1,2,...,n) and training threshold θ i Can be obtained through learning.

[0190] It should be noted that the step function here has no interchangeable symbolic meaning with other formulas in the embodiment. This step function is only used for principle explanation and does not participate in the calculation of other formulas.

[0191] Preferably, in this embodiment, the number of neural network training times can be set to 500 times.

[0192] Preferably, the learning rate from the 1st to the 250th epoch can be set to 0.01, the learning rate from the 251st to the 325th epoch can be set to 0.001, and the learning rate from the 326th to the 1000th epoch can be set to 0.0001.

[0193] It can be understood that the neural network training of this embodiment mainly includes the following ideas:

[0194] ① Initialize the weights and bias items in the network, and initialize the parameter values ​​(the output unit weights, bias items and hidden unit weights and bias items are all parameters of the model) to activate forward propagation, obtain the output values ​​of each layer of elements, and then obtain the value of the loss function.

[0195] ②Activate forward propagation to obtain the output value of each layer and the expected value of the loss function of each layer.

[0196] ③ According to the loss function, calculate the error term of the output unit and the error term of the hidden unit, calculate various errors, calculate the gradient of the parameter with respect to the loss function or calculate the partial derivative according to the chain rule of calculus. For the partial derivative of the vector or matrix in the composite function, the derivative of the function inside the composite function is always left multiplied; for the partial derivative of the scalar in the composite function, the derivative of the function inside the composite function can be left multiplied or right multiplied.

[0197] ④Update the weights and bias items in the neural network.

[0198] ⑤ Repeat ② to ④ until the loss function is less than the preset threshold or the number of iterations is exhausted, and the parameters output at this time are the current optimal parameters.

[0199] Furthermore, in step S14, it is determined that the construction worker has fallen, and then the following steps are performed:

[0200] Step S1000, obtaining the angle after the falling behavior occurs at preset time intervals.

[0201] Step S2000, obtaining the duration of the angle after the falling behavior occurs.

[0202] Step S3000, determine whether the duration is greater than or equal to a preset duration threshold. If the duration is greater than or equal to the preset duration threshold, execute step S4000.

[0203] Preferably, the preset time threshold is set to give the construction workers time to get up by themselves. If the construction workers fail to get up within the time, it means that external manual intervention is required.

[0204] Step S4000: Generate a manual intervention request and send it to an external receiving end.

[0205] This embodiment generates a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; responds to the entry signal of the construction personnel entering the construction site; takes the trigger time of the entry signal as the starting point, and obtains the acceleration reading of the accelerometer at intervals of a preset time period; generates acceleration coordinate points with the current preset time period as the horizontal axis and the current acceleration reading as the vertical axis; inputs all coordinate points into the rectangular coordinate system and connects them in sequence to form an acceleration change curve; obtains a person detection frame matching the entry signal through a target detection algorithm; obtains a key point skeleton in the person detection frame through a human key point detection algorithm ; Define a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site; map the key point skeleton to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates; obtain the vector direction of the main trunk of the three-dimensional skeleton at preset time intervals, and obtain the angle between the vector direction and any horizontal plane; determine whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, obtain the slope of the acceleration change curve at the same time; determine whether the slope exceeds the preset slope threshold, if the slope exceeds the preset slope threshold, determine that the construction worker has fallen. This embodiment adopts the technical means of dual verification of computer vision and acceleration slope, and at the same time, by mapping the skeleton in the person detection frame to the construction site, the skeleton and the construction environment are located in the same perspective environment, eliminating the perspective pull caused by the overhead or upward shooting at the shooting end, so that the mapped skeleton conforms to the world coordinate system, further reducing the misjudgment rate of dual verification, and only determines it as a fall when the main trunk skeleton approaches the horizontal direction and the acceleration is too high. This embodiment also utilizes the fact that the acceleration of the main trunk body movement of the construction personnel within the self-control range will not exceed the acceleration when the body is out of control, avoiding the fall detection of this embodiment from causing misjudgment of the daily activities of the construction personnel.

[0206] like Figure 3 As shown, this embodiment provides an embodiment of a fall detection device based on safety gear. In this embodiment, the fall detection device is applied to the fall detection method as in the above embodiment, and the fall detection device includes a rectangular coordinate system generation module 1, a construction personnel entry signal response module 2, an acceleration reading acquisition module 3, an acceleration coordinate point generation module 4, an acceleration change curve generation module 5, a task detection frame acquisition module 6, a key point skeleton acquisition module 7, a world coordinate system definition module 8, a three-dimensional skeleton generation module 9, a vector direction and angle acquisition module 10, an angle judgment module 11, an acceleration change curve slope acquisition module 12, a slope judgment module 13, and a fall behavior judgment module 14, which are electrically connected in sequence.

[0207] Among them, the rectangular coordinate system generation module 1 is used to generate a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; the construction personnel entry signal response module 2 is used to respond to the entry signal of the construction personnel entering the construction site; the acceleration reading acquisition module 3 is used to obtain the acceleration reading of the accelerometer at intervals of a preset time period with the trigger time of the entry signal as the starting point; the acceleration coordinate point generation module 4 is used to generate acceleration coordinate points with the current preset time period as the horizontal coordinate and the current acceleration reading as the vertical coordinate; the acceleration change curve generation module 5 is used to input all coordinate points into the rectangular coordinate system and connect them in sequence to form an acceleration change curve; the task detection frame acquisition module 6 is used to obtain a person detection frame matching the entry signal through a target detection algorithm; the key point skeleton acquisition module 7 is used to obtain the key point skeleton in the person detection frame through a human key point detection algorithm The world coordinate system definition module 8 is used to define the world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site; the three-dimensional skeleton generation module 9 is used to map the key point skeleton to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates; the vector direction and angle acquisition module 10 is used to obtain the vector direction of the main trunk of the three-dimensional skeleton at intervals of a preset time period, and obtain the angle between the vector direction and any horizontal plane; the angle judgment module 11 is used to judge whether the angle is less than or equal to a preset angle threshold; the acceleration change curve slope acquisition module 12 is used to obtain the slope of the acceleration change curve at the same time if the angle is less than or equal to the preset angle threshold; the slope judgment module 13 is used to judge whether the slope exceeds the preset slope threshold; the fall behavior judgment module 14 is used to judge whether the construction personnel have a fall behavior if the slope exceeds the preset slope threshold.

[0208] Furthermore, the task detection frame acquisition module 6 specifically includes a first task detection frame acquisition submodule, a second task detection frame acquisition submodule, a third task detection frame acquisition submodule, a fourth task detection frame acquisition submodule, a fifth task detection frame acquisition submodule, a sixth task detection frame acquisition submodule, a seventh task detection frame acquisition submodule, and an eighth task detection frame acquisition submodule, which are electrically connected in sequence; the first task detection frame acquisition submodule is electrically connected to the acceleration change curve generation module, and the eighth task detection frame acquisition submodule is electrically connected to the key point skeleton acquisition module.

[0209] Among them, the first task detection frame acquisition submodule is used to respond to the entry signal and acquire image data matching the entry signal through an external preset camera terminal; the second task detection frame acquisition submodule is used to evenly divide the image data into a preset number of grids; the third task detection frame acquisition submodule is used to predict a number of first-order bounding boxes for moving targets in the image data based on all grids according to the target detection algorithm, and each first-order bounding box includes at least one grid; the fourth task detection frame acquisition submodule is used to define the construction personnel with the highest confidence; the fifth task detection frame acquisition submodule is used to respectively acquire the confidence of each first-order bounding box, and acquire the bounding box with the largest confidence and mark it as a second-order bounding box; the sixth task detection frame acquisition submodule is used to calculate the intersection and union ratio of the second-order bounding box with each other first-order bounding box; the seventh task detection frame acquisition submodule is used to select a bounding box whose intersection and union ratio is greater than or equal to a preset threshold as a third-order bounding box; the eighth task detection frame acquisition submodule is used to acquire the third-order bounding box with the highest confidence and define it as a person detection frame.

[0210] Furthermore, the key point skeleton acquisition module 7 specifically includes a first key point skeleton acquisition submodule, a second key point skeleton acquisition submodule, a third key point skeleton acquisition submodule, a fourth key point skeleton acquisition submodule, and a fifth key point skeleton acquisition submodule, which are electrically connected in sequence; the first key point skeleton acquisition submodule is electrically connected to the eighth task detection frame acquisition submodule, and the fifth key point skeleton acquisition submodule is electrically connected to the world coordinate system definition module.

[0211] Among them, the first key point skeleton acquisition submodule is used to train the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtain the VGGNet prediction model after the training is completed; the second key point skeleton acquisition submodule is used to input the portrait data in each character detection frame into the VGGNet prediction model to extract human body features; the third key point skeleton acquisition submodule is used to extract the plane confidence map of the body part position in the current portrait data through the first branch in the two-branch CNN model; the fourth key point skeleton acquisition submodule is used to extract the plane affinity vector field of the body part position in the current portrait data through the second branch in the two-branch CNN model; the fifth key point skeleton acquisition submodule is used to calculate the key points of the plane confidence map and the plane affinity vector field of the current portrait data through the bipartite graph matching of the Hungarian algorithm.

[0212] Furthermore, the fall detection device also includes a three-dimensional skeleton world coordinate set acquisition module, a prediction model acquisition module, a future three-dimensional skeleton acquisition module, and a future three-dimensional skeleton application module, which are electrically connected in sequence; the three-dimensional skeleton world coordinate set acquisition module is electrically connected to the three-dimensional skeleton generation module, and the future three-dimensional skeleton application module is electrically connected to the vector direction and angle acquisition module.

[0213] Among them, the three-dimensional skeleton world coordinate set acquisition module is used to acquire several three-dimensional skeleton world coordinate sets of the three-dimensional skeleton with several preset time periods as acquisition time points, and one acquisition time point corresponds to one three-dimensional skeleton world coordinate set; the prediction model acquisition module is used to learn and train each three-dimensional skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model; the future three-dimensional skeleton acquisition module is used to predict the three-dimensional skeleton as a future three-dimensional skeleton of at least one unit second in the future through the prediction model; the future three-dimensional skeleton application module is used to output the future three-dimensional skeleton to the vector direction and angle acquisition module to apply the future three-dimensional skeleton to the three-dimensional skeleton in the vector direction of the main trunk.

[0214] Furthermore, the prediction model acquisition module specifically includes a first prediction model acquisition submodule, a second prediction model acquisition submodule, a third prediction model acquisition submodule, a fourth prediction model acquisition submodule, and a fifth prediction model acquisition submodule, which are electrically connected in sequence; the first prediction model acquisition submodule is electrically connected to the three-dimensional skeleton world coordinate set acquisition module, and the fifth prediction model acquisition submodule is electrically connected to the future three-dimensional skeleton acquisition module.

[0215] Among them, the first prediction model acquisition submodule is used to integrate all stereo skeleton world coordinate sets to form a total coordinate data set; the second prediction model acquisition submodule is used to divide the total coordinate data set into a training set and a verification set; the third prediction model acquisition submodule is used to define the topological relationship of the neural network model, and the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected; the fourth prediction model acquisition submodule is used to input the training set into the input layer, and iterate through the neural network model, and obtain the root mean square error between the verification set and the current training result based on each iteration; the fifth prediction model acquisition submodule is used to obtain the minimum value of all root mean square errors, and obtain the training result corresponding to the minimum value as the prediction model.

[0216] Furthermore, the third prediction model acquisition submodule is equipped with a neural network model represented by formula (1):

[0217]

[0218] Among them, y is the preset neural network model; x is the input layer and x={x1,x2,…,x n}, x n is the nth input node of the input layer, each input node corresponds to a training set, is the weight from the mth input node of the input layer to the nth input node of the hidden layer; is the bias of the nth input node connected to the hidden layer; tansig(x) is the transfer function, and The numbers in brackets are the number of layers.

[0219] Furthermore, the fall detection device also includes a module for acquiring the angle after the fall behavior, a module for acquiring the duration after the fall, a module for determining the duration after the fall, and a module for generating and sending a manual intervention request, which are electrically connected in sequence; the module for acquiring the angle after the fall behavior is electrically connected to the module for determining the duration after the fall behavior.

[0220] Among them, the angle acquisition module after the fall behavior is used to obtain the angle after the fall behavior at intervals of a preset time period; the duration acquisition module after the fall is used to obtain the duration of the angle after the fall behavior; the duration judgment module after the fall is used to judge whether the duration is greater than or equal to the preset duration threshold; the manual intervention request generation and sending module is used to generate a manual intervention request and send it to the external receiving end if the duration is greater than or equal to the preset duration threshold.

[0221] It should be noted that this embodiment is a functional module item embodiment based on the above method embodiment. For additional contents such as the optimization, expansion, and example illustration of this embodiment, please refer to the above method embodiment, and this embodiment will not be repeated.

[0222] This embodiment generates a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; responds to the entry signal of the construction personnel entering the construction site; takes the trigger time of the entry signal as the starting point, and obtains the acceleration reading of the accelerometer at intervals of a preset time period; generates acceleration coordinate points with the current preset time period as the horizontal axis and the current acceleration reading as the vertical axis; inputs all coordinate points into the rectangular coordinate system and connects them in sequence to form an acceleration change curve; obtains a person detection frame matching the entry signal through a target detection algorithm; obtains a key point skeleton in the person detection frame through a human key point detection algorithm ; Define a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site; map the key point skeleton to the world coordinate system through the homography matrix to form a three-dimensional skeleton with world coordinates; obtain the vector direction of the main trunk of the three-dimensional skeleton at preset time intervals, and obtain the angle between the vector direction and any horizontal plane; determine whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, obtain the slope of the acceleration change curve at the same time; determine whether the slope exceeds the preset slope threshold, if the slope exceeds the preset slope threshold, determine that the construction worker has fallen. This embodiment adopts the technical means of dual verification of computer vision and acceleration slope, and at the same time, by mapping the skeleton in the person detection frame to the construction site, the skeleton and the construction environment are located in the same perspective environment, eliminating the perspective pull caused by the overhead or upward shooting at the shooting end, so that the mapped skeleton conforms to the world coordinate system, further reducing the misjudgment rate of dual verification, and only determines it as a fall when the main trunk skeleton approaches the horizontal direction and the acceleration is too high. This embodiment also utilizes the fact that the acceleration of the main trunk body movement of the construction personnel within the self-control range will not exceed the acceleration when the body is out of control, avoiding the fall detection of this embodiment from causing misjudgment of the daily activities of the construction personnel.

[0223] Figure 4 An embodiment of an electronic device of the present application is shown. In this embodiment, the electronic device 15 includes a processor 151 and a memory 152 coupled to the processor 151.

[0224] The memory 152 stores program instructions for implementing the fall detection method based on safety gear according to any of the above embodiments.

[0225] The processor 151 is used to execute program instructions stored in the memory 152 to perform fall detection based on safety gear.

[0226] The processor 151 may also be referred to as a CPU (Central Processing Unit). The processor 151 may be an integrated circuit chip having signal processing capabilities. The processor 151 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0227] Further, Figure 5 This is a schematic diagram of the structure of a storage medium of an embodiment of the present application. The storage medium 16 of this embodiment stores program instructions 161 that can implement all the above methods, wherein the program instructions 161 can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer power device (which can be a personal computer, server, or network power device, etc.) or a processor to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, or terminal power devices such as a computer, a server, a mobile phone, and a tablet.

[0228] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0229] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0230] The specific implementation methods of the invention are described in detail above, but they are only examples, and the present application is not limited to the specific implementation methods described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application, and therefore, the equalization, modification, and improvement made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A fall detection method based on safety gear, wherein the fall detection method is applied to construction workers wearing safety gear in a construction site, wherein each piece of safety gear is equipped with an accelerometer, and the construction site has at least one preset camera terminal, characterized in that: The fall detection method comprises: Step S1, generating a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; Step S2, responding to an entry signal from the construction worker entering the construction site; Step S3, taking the triggering time of the entry signal as the starting point, obtaining the acceleration reading of the accelerometer at intervals of a preset time period; Step S4, generating an acceleration coordinate point with the current preset time period as the horizontal coordinate and the current acceleration reading as the vertical coordinate; Step S5, inputting all coordinate points into the rectangular coordinate system and sequentially connecting them to form an acceleration change curve; Step S6, obtaining a person detection frame that matches the entry signal through a target detection algorithm; Step S7, obtaining a key point skeleton in the person detection frame using a human key point detection algorithm; Step S8, defining a world coordinate system based on any horizontal plane elevation and any vertical plane elevation of the construction site; Step S9, mapping the key point skeleton to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates; Step S10, obtaining the vector direction of the main trunk of the three-dimensional skeleton at intervals of the preset time period, and obtaining the angle between the vector direction and any horizontal plane; Step S11, determining whether the angle is less than or equal to a preset angle threshold, if the angle is less than or equal to the preset angle threshold, executing step S12; Step S12, obtaining the slope of the acceleration change curve at the same time; Step S13, determining whether the slope exceeds a preset slope threshold, if the slope exceeds the preset slope threshold, executing step S14; Step S14: determining whether the construction worker has fallen.

2. The fall detection method according to claim 1, wherein: Step S6, obtaining a person detection frame that matches the entry signal through a target detection algorithm, includes: Step S61, in response to the entry signal, acquiring image data matching the entry signal through an external preset camera terminal; Step S62, dividing the image data into a preset number of grids on average; Step S63, predicting a plurality of first-order bounding boxes for the moving object in the image data based on all the grids according to the object detection algorithm, each first-order bounding box including at least one grid; Step S64, defining the construction worker as having the highest confidence level; Step S65, obtaining the confidence of each first-order bounding box respectively, obtaining the bounding box with the largest confidence and marking it as the second-order bounding box; Step S66, calculating the intersection-over-union ratio between the second-order bounding box and each of the other first-order bounding boxes; Step S67, selecting a bounding box whose intersection-over-union ratio is greater than or equal to a preset threshold as a third-order bounding box; Step S68: Obtain the third-order bounding box with the highest confidence and define it as the person detection box.

3. The fall detection method according to claim 1, wherein: Step S7, obtaining a key point skeleton in the person detection frame using a human key point detection algorithm, includes: Step S71, training the standard VGGNet convolutional network for a preset number of times using the OpenPose library, and obtaining a VGGNet prediction model after the training is completed; Step S72, inputting the portrait data in each person detection frame into the VGGNet prediction model to extract human body features; Step S73, extracting a planar confidence map of the body part positions in the current portrait data through the first branch of the two-branch CNN model; Step S74, extracting the planar affinity vector field of the body part positions in the current portrait data through the second branch of the two-branch CNN model; Step S75 , calculating key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

4. The fall detection method according to claim 1, wherein: Step S9, mapping the key point skeleton to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates, before; Step S10, obtaining the vector direction of the main trunk of the three-dimensional skeleton at intervals of the preset time period, and obtaining the angle between the vector direction and any horizontal plane, after which, including: Step S100, obtaining a plurality of stereo skeleton world coordinate sets of the stereo skeleton at a plurality of preset time periods as acquisition time points, wherein one acquisition time point corresponds to one stereo skeleton world coordinate set; Step S200 , learning and training each stereo skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model; Step S300 , predicting the stereo skeleton as a future stereo skeleton at least one unit second in the future by using the prediction model; Step S400 : Applying the future stereo skeleton to the stereo skeleton used in step S10 to obtain the vector direction of the main trunk.

5. The fall detection method according to claim 4, characterized in that: Step S200, learning and training each stereo skeleton world coordinate set separately through a machine learning algorithm to obtain a prediction model, including: Step S2001, integrating all three-dimensional skeleton world coordinate sets to form a total coordinate data set; Step S2002, dividing the global coordinate data set into a training set and a validation set; Step S2003, defining a topological relationship of the neural network model, wherein the topological relationship includes an input layer, a hidden layer, and an output layer that are sequentially signal-connected; Step S2004: input the training set into the input layer, and iterate through the neural network model, obtaining a root mean square error between the validation set and the current training result based on each iteration; Step S2005: Obtain the minimum value of all root mean square errors, and obtain the training result corresponding to the minimum value as the prediction model.

6. The fall detection method according to claim 5, characterized in that: The neural network model is represented by formula (1): Wherein, y is the preset neural network model; x is the input layer and x={x1,x2,…,x n }, x n is the nth input node of the input layer, each input node corresponds to a training set, is the weight from the mth input node of the input layer to the nth input node of the hidden layer; is the bias of the nth input node connected to the hidden layer; tansig(x) is the transfer function, and The numbers in brackets are the layers.

7. The fall detection method according to claim 1, wherein: Step S14, determining whether the construction worker has fallen, then includes: Step S1000, obtaining the angle after the falling behavior occurs at intervals of the preset time period; Step S2000, obtaining the duration of the angle after the fall behavior occurs; Step S3000, determining whether the duration is greater than or equal to a preset duration threshold, if the duration is greater than or equal to the preset duration threshold, executing step S4000; Step S4000: Generate a manual intervention request and send it to an external receiving end.

8. A fall detection device based on safety gear, wherein the fall detection device based on safety gear is applied to the fall detection method according to any one of claims 1 to 7, characterized in that: The fall detection device comprises: A rectangular coordinate system generation module is used to generate a rectangular coordinate system with natural time as the horizontal axis and the acceleration value as the vertical axis; A construction worker entry signal response module, configured to respond to an entry signal from the construction worker entering the construction site; An acceleration reading acquisition module, configured to acquire acceleration readings of the accelerometer at intervals of a preset time period, starting from the triggering time of the entry signal; An acceleration coordinate point generating module, configured to generate an acceleration coordinate point using a current preset time period as a horizontal coordinate and a current acceleration reading as a vertical coordinate; an acceleration change curve generating module, for inputting all coordinate points into the rectangular coordinate system and sequentially connecting them to form an acceleration change curve; A task detection frame acquisition module is used to acquire a person detection frame that matches the entry signal through a target detection algorithm; A key point skeleton acquisition module, configured to acquire the key point skeleton in the person detection frame by using a human key point detection algorithm; A world coordinate system definition module, configured to define a world coordinate system based on any horizontal elevation and any vertical elevation of the construction site; A three-dimensional skeleton generation module, used for mapping the key point skeleton to the world coordinate system through a homography matrix to form a three-dimensional skeleton with world coordinates; a vector direction and angle acquisition module, configured to acquire the vector direction of the main trunk of the three-dimensional skeleton at intervals of the preset time period, and to acquire the angle between the vector direction and any horizontal plane; An angle determination module, used to determine whether the angle is less than or equal to a preset angle threshold; an acceleration change curve slope acquisition module, configured to acquire the slope of the acceleration change curve at the same time if the included angle is less than or equal to the preset angle threshold; A slope determination module, configured to determine whether the slope exceeds a preset slope threshold; The falling behavior determination module is used to determine that the construction worker has fallen if the slope exceeds the preset slope threshold.

9. An electronic device, characterized in that: The invention comprises a processor and a memory coupled to the processor, wherein the memory stores program instructions that can be executed by the processor; when the processor executes the program instructions stored in the memory, the fall detection method based on safety gear as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed by the processor, the fall detection method based on safety gear according to any one of claims 1 to 7 can be implemented.

Images