Safety evaluation method, device and equipment based on safety protector and storage medium

Abstract

The invention discloses a safety evaluation method, device and equipment based on a safety protector and a storage medium, and relates to the technical field of electric digital data processing. According to the method, the target detection algorithm is improved, and the key point detection algorithm is fused to detect each area which should have a safety protector, so that compared with existing computer vision, the target vision detection is only performed on the detection areas divided by the key point detection algorithm; according to the method, all models do not need to traverse the same whole image, the detection response time is shortened, scoring is performed according to the existence of the safety protectors in each area, and the obtained scores can reflect the wearing condition and safety awareness of the constructor, so that the wearing condition of the constructor has a quantitative basis, and the safety of the constructor is improved. And an effective means is provided for subsequent systematic management and overall planning.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a safety evaluation method, device, equipment and storage medium based on safety protective gear. Background Art

[0002] The scale of construction projects continues to expand, and the quality and safety of construction cannot be taken lightly. Operational risks seriously threaten the lives of construction workers, so construction workers are required to strictly follow safety regulations and wear corresponding safety gear, such as safety helmets, protective clothing, etc.

[0003] Correctly wearing complete safety gear can effectively reduce physical injuries. However, during the actual construction process, some construction workers fail to wear safety gear properly or completely due to negligence or carelessness, which makes the construction workers always have safety hazards during the construction process.

[0004] Mainstream construction site safety supervision adopts manual supervision, which is not only time-consuming and labor-intensive, but also unable to monitor at any time whether construction workers are not wearing helmets correctly or not wearing them, resulting in frequent safety accidents on construction sites. In addition, most construction sites only have clear requirements for wearing safety helmets, thus ignoring the protection of other parts of the construction workers' bodies.

[0005] At present, the computer vision detection method for safety gear mainly focuses on helmets. By embedding electronic tags and sensors on the helmets, real-time positioning and tracking are performed, and physical detection methods such as pressure, infrared beams, and thermals are used to confirm whether the helmet is in a worn state. This method is inefficient and costly. If it is applied to other safety gear, embedding electronic tags and sensors in other safety gear and establishing a fixed data base station will inevitably bring high installation and maintenance costs. At the same time, the application method has many inconveniences, which is not conducive to actual large-scale promotion and use. At the same time, the visual inspection of helmets only stays at the detection of presence or absence. At present, there is no evaluation system that can evaluate whether the construction workers are wearing complete safety gear. It cannot directly reflect the level of self-safety awareness of the construction workers, but only reminds the construction workers who are not wearing helmets to wear them. Summary of the invention

[0006] The main purpose of this application is to provide a safety evaluation method, device, equipment and storage medium based on safety protective gear, so as to solve the problem that the visual inspection of safety helmets in the prior art only stops at detecting the presence or absence, and there is no evaluation system that can evaluate whether the construction workers are wearing complete safety protective gear, and cannot intuitively reflect the level of self-safety awareness of the construction workers.

[0007] In order to achieve the above objectives, this application provides the following technical solutions: A safety assessment method based on safety protective gear, the safety assessment method is applied to construction workers wearing safety protective gear, each construction surface has at least one preset shooting end, and the current preset shooting end faces the current construction surface, the safety assessment method comprises: Step S1, defining a first standard model of a target detection algorithm, and training the first standard model based on a person image training set, and obtaining a person detection model after the training is completed; Step S2, acquiring a plurality of image data of all construction surfaces through all preset shooting terminals based on a preset time period; Step S3, performing person detection on each image data respectively by using the person detection model, and obtaining a person detection frame based on a complete person individual; Step S4, intercepting all image data with each person detection frame as the boundary, and obtaining a portrait data based on one person detection frame; Step S5, detecting key points in each portrait data respectively by using a human key point detection algorithm; Step S6, defining a wearing torso category based on each type of safety gear, and classifying all key points into all wearing torso categories through a classification algorithm; Step S7, obtaining the maximum area enclosed by all key points in the current wearing torso category, and defining the maximum area as the detection area of ​​the current wearing torso category; Step S8, defining a second standard model of the target detection algorithm, and training the second standard model based on a safety gear image training set for each type of safety gear, and obtaining a safety gear detection model based on one type of safety gear after the training is completed; Step S9, detecting whether the safety gear exists in the corresponding detection area through the safety gear detection model; Step S10, if the safety gear is present, it is scored as 1 point, and if the safety gear is not present, it is scored as 0 point; Step S11, obtaining the total score of all detection areas based on the current portrait data, and marking the portrait data whose total score does not reach the first preset threshold as unqualified.

[0008] As a further improvement of the present application, in step S3, person detection is performed on each image data by using the person detection model, and a person detection frame is obtained based on a complete person individual, including: Step S31, each image data is evenly divided into a plurality of first grids; Step S32, detecting a plurality of first priori frames based on all first grids by using the person detection model; Step S33, defining the construction worker as having the highest confidence level; Step S34, respectively obtaining the confidence of each first priori box, and obtaining the first priori box with the largest confidence and marking it as the first-order person bounding box; Step S35, calculating the intersection-over-union ratio between the first-order person bounding box and each other first priori box; Step S36, selecting a first-order person bounding box whose intersection-over-union ratio is greater than or equal to a second preset threshold as a second-order person bounding box; Step S37, obtaining a second-order person bounding box with the highest confidence and defining it as the person detection box.

[0009] As a further improvement of the present application, step S5, detecting key points in each portrait data respectively by using a human key point detection algorithm, includes: Step S51, training the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtaining a VGGNet prediction model after the training is completed; Step S52, inputting each portrait data into the VGGNet prediction model to extract human body features; Step S53, extracting a plane confidence map of the position of the body part in the current portrait data through the first branch of the two-branch CNN model; Step S54, extracting the plane affinity vector field of the body part positions in the current portrait data through the second branch in the two-branch CNN model; Step S55 , calculating key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

[0010] As a further improvement of the present application, in step S6, a wearing torso category is defined based on each type of safety gear, and all key points are classified into all wearing torso categories through a classification algorithm, including: Step S61: define a set of data to be classified based on all key points ,in The data set to be classified The Key points, is the number of all key points; Step S62, defining a category set according to the wearing torso category ,in, For the category set The Wear torso categories, is the number of all wearing torso categories; Step S63, calculate the conditional probability of each key point in each wearing torso category according to formula (1): (1); in, For the Under the category of wearing torso, the data set to be classified The conditional probability of For the The marginal probability of the wearing torso category; For the Under the category of wearing torso, The conditional probability of key points; Step S64, classify each key point into the wearing torso category with the highest conditional probability.

[0011] As a further improvement of the present application, the safety gear includes a helmet, protective clothing, gloves, protective pants, and protective shoes. Step S7, obtaining the maximum area enclosed by all key points in the current wearing torso category, and defining the maximum area as the detection area of ​​the current wearing torso category, includes: Step S71, using a human key point detection algorithm to detect at least one key point on the head, at least one key point on each shoulder, at least one key point on each hand, at least one key point on each side of the hips, at least one key point on each knee, and at least one key point on each foot in each portrait data; Step S72, defining the maximum area enclosed by all key points of the head as the detection area of ​​the helmet; Step S73, defining the maximum area enclosed by all shoulder key points and all hip key points as the detection area of ​​the protective clothing; Step S74, defining the maximum area enclosed by all the key points of the hand as the detection area of ​​the glove; Step S75, defining the maximum area enclosed by all the key points of the hips, all the key points of the knees and all the key points of the feet as the detection area of ​​the protective pants; Step S76, defining the maximum area enclosed by all the key points of the foot as the detection area of ​​the protective shoe.

[0012] As a further improvement of the present application, step S9, detecting whether the safety gear exists in the corresponding detection area through the safety gear detection model, includes: Step S91, dividing the detection area into a plurality of second grids on average; Step S92, detecting a plurality of second priori frames based on all second grids through the safety gear detection model; Step S93, defining the safety gear as having the highest confidence level; Step S94, respectively obtaining the confidence of each second priori frame, and marking the second priori frame with a confidence of 0 as not containing the safety gear; Step S95, obtaining the second priori frame with the highest confidence and marking it as the first-order safety gear bounding box; Step S96, calculating the intersection-and-union ratio of the first-order safety gear bounding box with each of the other second priori boxes; Step S97, selecting a first-order safety gear bounding box whose intersection-over-union ratio is greater than or equal to a third preset threshold as a second-order safety gear bounding box; Step S98, obtaining a second-order safety gear boundary box with the highest confidence and defining it as the existence of the safety gear.

[0013] As a further improvement of the present application, step S11, based on the current portrait data, obtains the total score of all detection areas, and marks the portrait data whose total score does not reach the first preset threshold as unqualified, and then includes: Step S12, generating a reminder signal based on the unqualified wearing; Step S13, sending the reminder signal to the construction personnel corresponding to the portrait data whose total score does not reach the first preset threshold; Step S14, counting the proportion of unqualified portrait data with a total score that does not reach a first preset threshold based on the preset time period; Step S15, determining whether the proportion is greater than or equal to a fourth preset threshold, if the proportion is greater than or equal to the fourth preset threshold, executing step S16; Step S16, generating a warning signal, and sending the warning signal to the construction personnel corresponding to whether the proportion is greater than or equal to the fourth preset threshold and the external coordination end.

[0014] In order to achieve the above objectives, this application also provides the following technical solutions: A safety evaluation device based on safety gear, the safety evaluation device based on safety gear is applied to the above-mentioned safety evaluation method, and the safety evaluation device comprises: A person detection model training and acquisition module is used to define a first standard model of the target detection algorithm, and train the first standard model based on a person image training set to obtain a person detection model after the training is completed; A construction surface image data acquisition module, used to acquire a number of image data of all construction surfaces through all preset shooting ends based on a preset time period; A person detection frame acquisition module is used to perform person detection on each image data by using the person detection model, and obtain a person detection frame based on a complete person individual; A portrait data acquisition module is used to intercept all image data with each person detection frame as the boundary, and obtain a portrait data based on a person detection frame; A portrait data key point extraction module is used to detect the key points in each portrait data by using a human key point detection algorithm; The portrait data key point classification module is used to define a wearing torso category based on each type of safety gear, and classify all key points into all wearing torso categories through a classification algorithm; A safety gear detection area definition module is used to obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; A safety gear detection model training and acquisition module, used to define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and obtain a safety gear detection model based on one type of safety gear after the training is completed; A safety gear detection module, used to detect whether the safety gear exists in the corresponding detection area through the safety gear detection model; A safety gear scoring module, used to score 1 point if the safety gear is present, and score 0 point if the safety gear is not present; The unqualified wearing marking module is used to obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified wearing.

[0015] In order to achieve the above objectives, this application also provides the following technical solutions: An electronic device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the safety assessment method based on safety gear as described above is implemented.

[0016] To achieve the above objectives, this application also provides the following technical solutions: A storage medium stores program instructions, and when the program instructions are executed by a processor, the safety assessment method based on safety gear as described above can be implemented.

[0017] The present application defines a first standard model of a target detection algorithm, and trains the first standard model based on a person image training set, and obtains a person detection model after the training is completed; obtains a number of image data of all construction surfaces through all preset shooting terminals based on a preset time period; performs person detection on each image data respectively through the person detection model, and obtains a person detection frame based on a complete person individual; intercepts all image data with each person detection frame as the boundary, and obtains a portrait data based on a person detection frame; detects the key points in each portrait data respectively through a human key point detection algorithm; defines a wearing torso category based on each type of safety protective gear, and classifies all key points into all In the wearing torso category; obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and after the training is completed, obtain a safety gear detection model based on a type of safety gear; detect whether there is safety gear in the corresponding detection area through the safety gear detection model; if there is safety gear, it is recorded as 1 point, and if there is no safety gear, it is recorded as 0 point; obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified for wearing. This application improves the target detection algorithm and integrates the key point detection algorithm to separately detect each area that should have a safety protective gear. Compared with the existing computer vision, this application only performs target visual detection on the detection area divided by the key point detection algorithm. It does not require all models to traverse the same entire image separately, shortening the detection response time. At the same time, it scores each area based on the presence or absence of safety protective gear. The obtained score can reflect the amount of wearing of the construction workers and the level of safety awareness, so that the wearing situation of the construction workers has a quantitative basis, providing an effective means for subsequent systematic management and coordination. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the process steps of an embodiment of the safety evaluation method based on safety protective gear of the present application; Figure 2 This is a functional module diagram of an embodiment of a safety evaluation device based on safety gear of the present application; Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application; Figure 4 This is a schematic diagram of the structure of an embodiment of the storage medium of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The terms "first", "second" and "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of the present application (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or power device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or power devices.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] like Figure 1 As shown, this embodiment provides an embodiment of a safety evaluation method based on safety protective gear. In this embodiment, the safety evaluation method is applied to construction workers wearing safety protective gear. Each construction surface has at least one preset shooting end, and the current preset shooting end faces the current construction surface.

[0023] Specifically, safety protective gear includes safety helmets, protective clothing, gloves, protective pants, and protective shoes. The above five types are universal safety protective gear. This embodiment will use the above five types of safety protective gear as the illustration object. If there is an application scenario that requires the detection of other types of safety protective gear such as goggles, masks, and face shields, it can be executed according to the contents of steps S1 to S11 below.

[0024] The safety assessment method includes the following steps: Step S1, defining a first standard model of a target detection algorithm, and training the first standard model based on a person image training set, and obtaining a person detection model after the training is completed.

[0025] Preferably, the target detection algorithm can be implemented by target detection algorithms VJ, HOG, DPMDetector; deep learning two-stage target detection algorithms RCNN, SPPNet, FastRCNN, FasterRCNN; target detection trick algorithms FPN, CascadeRCNN; deep learning one-stage target detection algorithms Yolo, X, SSD, RetinaNet; deep learning anchor-free target detection algorithms CornerNet, CenterNet, FCOS; Transformer-based target detection algorithm DETR, etc.

[0026] Preferably, this embodiment prefers the Yolo algorithm.

[0027] Preferably, the character image training set can be directly obtained from public channels, or a number of images of the construction surface can be taken as training samples.

[0028] Preferably, the first standard model and the second standard model described below are both Yolo's standard models, including Backbone, PANet, and Yolo Head. "First" and "Second" are only used to distinguish different application environments. The first standard model is applied to the entire image data to detect complete individual people, and the second standard model is applied to individual people to detect safety gear.

[0029] Step S2, acquiring a plurality of image data of all construction surfaces through all preset shooting terminals based on a preset time period.

[0030] Preferably, the preset time period can be set to 1min, 5min, 10min, 15min, etc., and can be selected according to actual needs or computer computing power.

[0031] Step S3, performing person detection on each image data respectively through a person detection model, and obtaining a person detection frame based on a complete person individual.

[0032] Preferably, if there are multiple construction workers in one image data, a person detection frame is obtained based on each construction worker.

[0033] Step S4, intercepting all image data with each person detection frame as the boundary, and obtaining a portrait data based on one person detection frame.

[0034] Preferably, the image data is deleted except for the portrait data to save computer memory.

[0035] Step S5, detecting key points in each portrait data respectively by using a human key point detection algorithm.

[0036] Step S6, defining a wearing torso category based on each type of safety gear, and classifying all key points into all wearing torso categories through a classification algorithm.

[0037] Step S7, obtaining the maximum area enclosed by all key points in the current wearing torso category, and defining the maximum area as the detection area of ​​the current wearing torso category.

[0038] Preferably, if the number of key points in the wearing torso category is small and insufficient to form an area that meets the size of the corresponding safety gear, the center of the minimum covering circle of all key points in the current wearing torso category can be obtained, and a circular detection area is formed with the center of the minimum covering circle as the center of the detection area and the commonly used size of the corresponding safety gear as the radius.

[0039] Preferably, the minimum covering circle can be obtained by using a three-point circle determination method, a random increment method, or Emo Welzl's linear programming algorithm based on RaimundSeidel.

[0040] It should be noted that the dimensions referred to here are the actual dimensions in the image data, not the metric dimensions of the helmet itself, i.e. the scale needs to be considered.

[0041] Step S8, defining a second standard model of the target detection algorithm, and training the second standard model based on a safety gear image training set for each type of safety gear, and obtaining a safety gear detection model based on one type of safety gear after the training is completed.

[0042] Step S9: Detect whether there is any safety gear in the corresponding detection area through the safety gear detection model.

[0043] Step S10: If there is a safety protective gear, it is scored as 1 point; if there is no safety protective gear, it is scored as 0 point.

[0044] Step S11, obtaining the total score of all detection areas based on the current portrait data, and marking the portrait data whose total score does not reach the first preset threshold as unqualified.

[0045] Preferably, the first preset threshold can be set to 3 points, using a 60% passing rate. For construction surfaces with more stringent or higher risk factors, it can be set to 4 points or even 5 points.

[0046] The design intention of this embodiment is to attach equal importance to various parts of the body rather than just the head. In order to prevent construction workers from getting 4 points by not wearing a safety helmet but wearing safety gear on other parts of the body, this embodiment can add a step after step S9 and before step S10: when it is recognized that there is a safety helmet in the detection area located on the head, the scoring step of step S10 is performed. If it is recognized that there is no safety helmet in the detection area located on the head, step S10 is skipped and 0 points are directly assigned.

[0047] Furthermore, in step S3, a person detection model is used to perform person detection on each image data, and a person detection frame is obtained based on a complete person individual, including: Step S31, each image data is evenly divided into a plurality of first grids.

[0048] Preferably, the size of the original picture of each image data can be adjusted to 448×448, and then the resized picture can be evenly divided into S×S (for example, 7×7) grids, and the size of each grid is 64×64.

[0049] Preferably, each grid is used to predict The coordinates and width and height of the detection box, as well as the confidence of each detection box, that is, each grid needs to predict value.

[0050] It should be noted that this " ” and “ "Not interoperable.

[0051] Step S32: Detect a plurality of first priori boxes based on all first grids through a person detection model.

[0052] Preferably, if the center of the object is located on a first grid, the first grid is responsible for predicting the bounding box of the object.

[0053] Step S33, defining the construction personnel with the highest confidence.

[0054] Step S34, respectively obtain the confidence of each first priori box, obtain the first priori box with the largest confidence and mark it as the first-order person bounding box.

[0055] Step S35 , calculating the intersection-over-union ratio between the first-order person bounding box and each of the other first priori boxes.

[0056] Preferably, the intersection-over-union ratio is a ratio obtained by dividing the intersection of the optimal detection box and each bounding box by the union of the optimal detection box and each bounding box.

[0057] Step S36 , selecting a first-order person bounding box whose intersection-over-union ratio is greater than or equal to a second preset threshold as a second-order person bounding box.

[0058] Preferably, the second preset threshold may be set to 0.5.

[0059] Step S37, obtaining a second-order person bounding box with the highest confidence and defining it as a person detection box.

[0060] It can be understood that each grid needs to predict indivual ;in, is the offset of the center of the detection box relative to the grid, is the ratio of the detection box to the above resized image, is the confidence of the grid, which can be 1 or 0.

[0061] Preferably, the confidence level can be understood as whether there is a target in the current grid and the accuracy of the detection box.

[0062] For example: suppose there is a target in a resized image, and the width and height of the resized image are but: Divide the image into 7×7 (S×S) grids evenly. There is a grid located at the center of the target. The coordinates of the grid are , let the coordinates of the center of the target be , the above offset can be calculated according to the following formula : .

[0063] Preferably, in actual detection, if the predicted detection box and the actual bounding box overlap perfectly, the intersection-and-union ratio is 1. In actual application, the intersection-and-union ratio can generally be set to 0.5 to determine whether the predicted bounding box is correct, and the more accurate the bounding box is, the more positively correlated it is with the intersection-and-union ratio.

[0064] Preferably, the YOLO algorithm also needs to train the detection frame to improve the accuracy of target detection.

[0065] It should be noted that the training model of the person detection frame is generally implemented directly through code, for example: “class Yolo(object): def __init__(self, weights_file, verbose=True): self.verbose = verbose # detection params self.S = 7 # cell size self.B = 2 # boxes_per_cell self.classes = [*] self.C = len(self.classes) # number of classes # offset for box center (top left point of each cell) self.x_offset= np.transpose(np.reshape(np.array([np.arange(self.S)]*self.S*self.B),[self.B, self.S, self.S]), [1, 2, 0]) self.y_offset = np.transpose(self.x_offset, [1, 0, 2]) self.threshold = 0.2# confidence scores threshold self.iou_threshold = 0.4 #the maximum number of boxes to be selected by non max suppression self.max_output_size = 10".

[0066] Among them, “*” is the target category that needs to be identified, which can be set to words related to construction workers such as “person” and “human” to identify construction workers.

[0067] Next, the training model is trained using a preset training set, and the weights and biases of the training model are iteratively adjusted a first preset number of times using a back propagation algorithm to reduce the value of the loss function of the training model.

[0068] Preferably, the loss function is as follows: .

[0069] in, For the Grid An indicator function of whether a detection box is responsible for the target, with a value of 1 or 0; , , , , Corresponding to indivual The predicted value, that is, the above value.

[0070] It can be understood that the loss function includes the coordinate value deviation of the detection box, the confidence deviation, and the prediction probability deviation (or category deviation).

[0071] in, is the detection frame midpoint loss in the coordinate value deviation, is the loss of detection box width and height in coordinate value deviation, is the confidence deviation, is the deviation of the predicted probability (or class deviation).

[0072] in, is the positioning error penalty. Generally, ; That is the S×S grids mentioned above; is the number of bounding boxes; and For the The estimated values ​​of the horizontal and vertical coordinates of the midpoint of the bounding box; and For the An estimate of the width and height of the bounding box; For the The confidence of the bounding box; For the An estimate of the confidence of the bounding box; is the confidence prediction loss, generally ; For the The class probabilities of the bounding boxes; For the An estimate of the class probability of each bounding box; and In correspond .

[0073] It should be noted that since each grid does not necessarily contain a target, if there is no target in the grid, it will lead to The value of is 0, which makes the gradient span in the subsequent back propagation algorithm too large, so we introduce To control the loss of the predicted position of the detection box, and introduce Controls the loss of objects not existing in a single grid.

[0074] It should be noted that the meanings of all symbols in the extended content of the above YOLO algorithm are not interchangeable with other symbol meanings.

[0075] Furthermore, step S5, detecting key points in each portrait data by using a human key point detection algorithm, includes: Step S51, training the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtaining the VGGNet prediction model after the training is completed.

[0076] Preferably, commonly used training sets for human key points include COCO Keypoints challenge, MPII HumanPose Dataset, VGG Pose Dataset, CMU Panoptic Dataset, and OpenPose Dataset.

[0077] Step S52, input each portrait data into the VGGNet prediction model to extract human body features.

[0078] Step S53, extracting a planar confidence map of the positions of body parts in the current portrait data through the first branch in the two-branch CNN model.

[0079] Step S54, extracting the plane affinity vector field of the body part positions in the current portrait data through the second branch in the two-branch CNN model.

[0080] Step S55 , calculating key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

[0081] In simple terms, an image is input, and features are extracted through a convolutional network to obtain a set of feature maps. Then, the process is divided into two branches, and a two-branch CNN network is used to extract Part Confidence Maps and Part Affinity Fields respectively. After obtaining the aforementioned PCMs and PAFs, the Part Association (key point set) is obtained through the Bipartite Matching of the Hungarian algorithm.

[0082] Preferably, the bipartite graph matching of the Hungarian algorithm refers to a process of finding the maximum number of undirected edge pairs in a directed graph G=(V,E), in which the vertex set V is divided into two independent sets U and W (U∩W=∅), and each edge (e=(u,w)) connects one end from U and one end from W, so that these edges have no intersections.

[0083] Among them, the bipartite graph matching steps of the Hungarian algorithm are as follows: Construct an augmented matrix: Convert the graph into a weighted adjacency matrix, with each row representing a node in U and each column representing a node in W. If there is an edge (u, w), the corresponding matrix element is the weight of the edge; otherwise it is 0.

[0084] Initialization: Set all matches to unmatched and create an initial zero-filled augmented matrix.

[0085] Loop: Perform the following steps until an exact match is found or no more pairs can be found.

[0086] Solve linear programming: Use Huffman coding or the shortest augmenting path algorithm to find a minimum augmenting path, which starts from an unmatched node and follows the edge with smaller weight until another matched node is encountered.

[0087] Update matching: adjust the matching along this path, add a matching edge and delete this path.

[0088] Termination condition: If an augmenting path can be found, continue the next round of search. If not found, it means that the current best match has been found and the algorithm ends.

[0089] Return result: Output the number of maximum matches currently found and the corresponding matching solutions.

[0090] Further, in step S6, a wearing torso category is defined based on each type of safety gear, and all key points are classified into all wearing torso categories through a classification algorithm, including: Step S61: define a set of data to be classified based on all key points ,in The data set to be classified The Key points, is the number of all key points.

[0091] Step S62, defining a category set according to the wearing torso category ,in, For category collection The Wear torso categories, The number of all torso categories.

[0092] Step S63, calculate the conditional probability of each key point in each wearing torso category according to formula (1): (1).

[0093] in, For the Under the category of wearing torso, the data set to be classified The conditional probability of For the The marginal probability of the wearing torso category; For the Under the category of wearing torso, The conditional probability of a key point.

[0094] Step S64, classify each key point into the wearing torso category with the highest conditional probability.

[0095] Preferably, the classification algorithm of this embodiment adopts Bayesian classification to classify each electrical device into the load level with the highest conditional probability, so as to prevent subjective errors in manual classification.

[0096] Specifically, Naive Bayes is defined as follows: ① Let x={a1, a2, a3,…, an} be an item to be classified, and each a is a feature of x.

[0097] ②There is a category set c={y1, y2, y3, ..., y m}.

[0098] ③ Calculate P(y1|x), P(y2|x), ..., P(y m |x).

[0099] ④If P(y k |x)=max{P(y1|x),P(y2|x),...,P(y m |x)}, then x∈y k .

[0100] Then calculate the conditional probabilities in step ③ through the following steps: 1. Find a set of items to be classified with known classifications. This set is called a training sample set.

[0101] 2. Statistically obtain the conditional probability estimate of each feature attribute under each category. That is: P(a1|y1),P(a2|y1),……,P(a n |y1) P(a1|y2),P(a2|y2),……,P(a n |y2); … P(a1|y m ),P(a2|y m),……,P(a n |y m ); 3. Assuming that each feature attribute is conditionally independent, according to the Bayesian principle: P(y i |x)=P(x|y i )P(y i ) / p(x).

[0102] Since the denominator is a constant for all categories, we only need to maximize the numerator. And because each feature attribute is conditionally independent, then: P(x|y i )P(y i )=P(a1|y i )P(a2|y i )……P(a n |y i )P(y i ).

[0103] It should be noted that the above preferred contents are also explanations of the principles, and the meanings of their symbols are not interchangeable with the meanings of the symbols of other formulas in this embodiment.

[0104] Further, in step S7, the maximum area enclosed by all key points in the current wearing torso category is obtained, and the maximum area is defined as the detection area of ​​the current wearing torso category, including: Step S71, using a human key point detection algorithm to detect at least one key point on the head, at least one key point on each shoulder, at least one key point on each hand, at least one key point on each side of the hips, at least one key point on each knee, and at least one key point on each foot in each portrait data.

[0105] Preferably, if the number of key points in the wearing torso category is small and insufficient to form an area that meets the size of the corresponding safety gear, the center of the minimum covering circle of all key points in the current wearing torso category can be obtained, and a circular detection area is formed with the center of the minimum covering circle as the center of the detection area and the commonly used size of the corresponding safety gear as the radius.

[0106] Preferably, the minimum covering circle can be obtained by using a three-point circle determination method, a random increment method, or Emo Welzl's linear programming algorithm based on RaimundSeidel.

[0107] It should be noted that the dimensions referred to here are the actual dimensions in the image data, not the metric dimensions of the helmet itself, i.e. the scale needs to be considered.

[0108] Step S72, defining the maximum area enclosed by all key points of the head as the detection area of ​​the helmet.

[0109] Step S73, the maximum area enclosed by all shoulder key points and all hip key points is defined as the detection area of ​​the protective clothing.

[0110] Step S74, defining the maximum area enclosed by all the key points of the hand as the detection area of ​​the glove.

[0111] Step S75, defining the maximum area enclosed by all hip key points, all knee key points and all foot key points as the detection area of ​​the protective pants.

[0112] Step S76, defining the maximum area enclosed by all the key points of the foot as the detection area of ​​the protective shoe.

[0113] Further, step S9, detecting whether there is safety gear in the corresponding detection area through the safety gear detection model, includes: Step S91, dividing the detection area into a plurality of second grids on average.

[0114] Step S92: Detect a plurality of second priori frames based on all second grids through a safety gear detection model.

[0115] Step S93, defining the safety gear with the highest confidence.

[0116] Step S94, respectively obtain the confidence of each second priori frame, and mark the second priori frame with a confidence of 0 as having no safety gear.

[0117] Step S95: Obtain the second priori box with the highest confidence and mark it as the first-order safety gear bounding box.

[0118] Step S96, calculating the intersection-and-union ratio between the first-order safety gear bounding box and each of the other second priori boxes.

[0119] Step S97, selecting a first-order safety gear bounding box whose intersection-over-union ratio is greater than or equal to a third preset threshold as a second-order safety gear bounding box.

[0120] Preferably, the third preset threshold may be set to 0.5.

[0121] Step S98, obtaining the second-order safety gear boundary box with the highest confidence and defining it as the existence of safety gear.

[0122] Preferably, the second standard model and the first standard model mentioned above are both Yolo's standard models, including Backbone, PANet, and Yolo Head. "First" and "Second" are only used to distinguish different application environments. The first standard model is applied to the entire image data to detect complete individual people, and the second standard model is applied to individual people to detect safety gear.

[0123] Preferably, the extended contents of step S91 to step S98 refer to the extended contents of step S31 to step S37 above.

[0124] Further, in step S11, the total score of all detection areas is obtained based on the current portrait data, and the portrait data whose total score does not reach the first preset threshold is marked as unqualified for wearing. After that, it includes: Step S12, generating a reminder signal based on unqualified wearing.

[0125] Step S13: sending a reminder signal to the construction personnel corresponding to the portrait data whose total score does not reach the first preset threshold.

[0126] Step S14, based on a preset time period, counting the proportion of portrait data with unqualified dressing whose total score does not reach a first preset threshold.

[0127] Step S15, determining whether the proportion is greater than or equal to a fourth preset threshold; if the proportion is greater than or equal to the fourth preset threshold, executing step S16.

[0128] Preferably, the fourth preset threshold may be set to 0.2.

[0129] Step S16, generating a warning signal, and sending the warning signal to the construction personnel corresponding to whether the proportion is greater than or equal to the fourth preset threshold and the external coordination end.

[0130] This embodiment defines a first standard model of the target detection algorithm, and trains the first standard model based on a person image training set, and obtains a person detection model after the training is completed; obtains a number of image data of all construction surfaces through all preset shooting terminals based on a preset time period; performs person detection on each image data through the person detection model, and obtains a person detection frame based on a complete person individual; intercepts all image data with each person detection frame as the boundary, and obtains a portrait data based on a person detection frame; detects the key points in each portrait data through a human key point detection algorithm; defines a wearing torso category based on each safety protective gear, and classifies all key points into all In the wearing torso category; obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and after the training is completed, obtain a safety gear detection model based on a type of safety gear; detect whether there is safety gear in the corresponding detection area through the safety gear detection model; if there is safety gear, it is recorded as 1 point, and if there is no safety gear, it is recorded as 0 point; obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified for wearing. This embodiment improves the target detection algorithm and integrates the key point detection algorithm to perform separate detection on each area that should have a type of safety protective gear. Compared with existing computer vision, this embodiment only performs target visual detection on the detection area divided by the key point detection algorithm. It does not require all models to traverse the same entire image separately, shortening the detection response time. At the same time, each area is scored based on the presence or absence of safety protective gear. The obtained score can reflect the amount of wearing of the construction workers and the level of safety awareness, so that the wearing situation of the construction workers has a quantitative basis, providing an effective means for subsequent systematic management and coordination.

[0131] like Figure 2 As shown, this embodiment provides an embodiment of a safety evaluation device based on safety protective gear. In this embodiment, the safety evaluation device is applied to the safety evaluation method in the above embodiment, and the safety evaluation device includes a person detection model training and acquisition module 1, a construction surface image data acquisition module 2, a person detection frame acquisition module 3, a portrait data acquisition module 4, a portrait data key point extraction module 5, a portrait data key point classification module 6, a safety protective gear detection area definition module 7, a safety protective gear detection model training and acquisition module 8, a safety protective gear detection module 9, a safety protective gear scoring module 10, and an unqualified wearing marking module 11, which are electrically connected in sequence.

[0132] Among them, the person detection model training and acquisition module 1 is used to define the first standard model of the target detection algorithm, and train the first standard model based on the person image training set, and obtain the person detection model after the training is completed; the construction surface image data acquisition module 2 is used to obtain a number of image data of all construction surfaces through all preset shooting ends based on a preset time period; the person detection frame acquisition module 3 is used to perform person detection on each image data respectively through the person detection model, and obtain a person detection frame based on a complete person individual; the portrait data acquisition module 4 is used to intercept all image data with each person detection frame as the boundary, and obtain a portrait data based on a person detection frame; the portrait data key point extraction module 5 is used to detect the key points in each portrait data respectively through the human body key point detection algorithm; the portrait data key point classification module 6 is used to define a wearing torso category based on each safety protective gear, and classify all key points through the classification algorithm. Classify into all wearing torso categories; the safety gear detection area definition module 7 is used to obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; the safety gear detection model training and acquisition module 8 is used to define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and after the training is completed, a safety gear detection model is obtained based on a type of safety gear; the safety gear detection module 9 is used to detect whether there is safety gear in the corresponding detection area through the safety gear detection model; the safety gear scoring module 10 is used to record 1 point if there is safety gear, and record 0 point if there is no safety gear; the unqualified wearing marking module 11 is used to obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified wearing.

[0133] Furthermore, the person detection frame acquisition module specifically includes a first person detection frame acquisition submodule, a second person detection frame acquisition submodule, a third person detection frame acquisition submodule, a fourth person detection frame acquisition submodule, a fifth person detection frame acquisition submodule, a sixth person detection frame acquisition submodule, and a seventh person detection frame acquisition submodule, which are electrically connected in sequence; the first person detection frame acquisition submodule is electrically connected to the construction surface image data acquisition module, and the seventh person detection frame acquisition submodule is electrically connected to the portrait data acquisition module.

[0134] Among them, the first person detection frame acquisition submodule is used to divide each image data into several first grids respectively; the second person detection frame acquisition submodule is used to detect several first prior frames based on all the first grids through the person detection model; the third person detection frame acquisition submodule is used to define the construction personnel with the highest confidence; the fourth person detection frame acquisition submodule is used to obtain the confidence of each first prior frame respectively, and obtain the first prior frame with the largest confidence and mark it as the first-order person bounding frame; the fifth person detection frame acquisition submodule is used to calculate the intersection and union ratio of the first-order person bounding frame with each other first prior frame respectively; the sixth person detection frame acquisition submodule is used to select the first-order person bounding box with an intersection and union ratio greater than or equal to a second preset threshold as the second-order person bounding box; the seventh person detection frame acquisition submodule is used to obtain the second-order person bounding box with the highest confidence and define it as the person detection frame.

[0135] Furthermore, the portrait data key point extraction module specifically includes a first portrait data key point extraction submodule, a second portrait data key point extraction submodule, a third portrait data key point extraction submodule, a fourth portrait data key point extraction submodule, and a fifth portrait data key point extraction submodule, which are electrically connected in sequence; the first portrait data key point extraction submodule is electrically connected to the portrait data acquisition module, and the fifth portrait data key point extraction submodule is electrically connected to the portrait data key point classification module.

[0136] Among them, the first portrait data key point extraction submodule is used to train the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtain the VGGNet prediction model after the training is completed; the second portrait data key point extraction submodule is used to input each portrait data into the VGGNet prediction model respectively to extract human body features; the third portrait data key point extraction submodule is used to extract the plane confidence map of the body part position in the current portrait data through the first branch in the two-branch CNN model; the fourth portrait data key point extraction submodule is used to extract the plane affinity vector field of the body part position in the current portrait data through the second branch in the two-branch CNN model; the fifth portrait data key point extraction submodule is used to calculate the key points of the plane confidence map and the plane affinity vector field of the current portrait data through the bipartite graph matching of the Hungarian algorithm.

[0137] Furthermore, the portrait data key point classification module specifically includes a first portrait data key point classification submodule, a second portrait data key point classification submodule, a third portrait data key point classification submodule, and a fourth portrait data key point classification submodule, which are electrically connected in sequence; the first portrait data key point classification submodule is electrically connected to the portrait data key point extraction module, and the fourth portrait data key point classification submodule is electrically connected to the safety gear detection area definition module.

[0138] Among them, the first portrait data key point classification submodule is used to define the data set to be classified according to all key points ,in The data set to be classified The Key points, is the number of all key points.

[0139] The second portrait data key point classification submodule is used to define a category set according to the wearing torso category. ,in, For category collection The Wear torso categories, The number of all torso categories.

[0140] The third portrait data key point classification submodule is used to calculate the conditional probability of each key point in each wearing torso category according to formula (1): (1).

[0141] in, For the Under the category of wearing torso, the data set to be classified The conditional probability of For the The marginal probability of the wearing torso category; For the Under the category of wearing torso, The conditional probability of a key point.

[0142] The fourth portrait data key point classification submodule is used to classify each key point into the wearing torso category with the highest conditional probability.

[0143] Furthermore, the safety gear detection area definition module specifically includes a first safety gear detection area definition submodule, a second safety gear detection area definition submodule, a third safety gear detection area definition submodule, a fourth safety gear detection area definition submodule, a fifth safety gear detection area definition submodule, and a sixth safety gear detection area definition submodule, which are electrically connected in sequence; the first safety gear detection area definition submodule is electrically connected to the fourth portrait data key point classification submodule, and the sixth safety gear detection area definition submodule is electrically connected to the safety gear detection model training and acquisition module.

[0144] Among them, the first safety gear detection area definition submodule is used to detect at least one key point of the head, at least one key point of each shoulder, at least one key point of each hand, at least one key point on each side of the crotch, at least one key point on each knee, and at least one key point on each foot in each portrait data through a human key point detection algorithm; the second safety gear detection area definition submodule is used to define the maximum area enclosed by all head key points as the detection area of ​​the helmet; the third safety gear detection area definition submodule is used to define the maximum area enclosed by all shoulder key points and all crotch key points as the detection area of ​​the protective clothing; the fourth safety gear detection area definition submodule is used to define the maximum area enclosed by all hand key points as the detection area of ​​the gloves; the fifth safety gear detection area definition submodule is used to define the maximum area enclosed by all crotch key points, all knee key points and all foot key points as the detection area of ​​the protective pants; the sixth safety gear detection area definition submodule is used to define the maximum area enclosed by all foot key points as the detection area of ​​the protective shoes.

[0145] Furthermore, the safety gear detection module specifically includes a first safety gear detection submodule, a second safety gear detection submodule, a third safety gear detection submodule, a fourth safety gear detection submodule, a fifth safety gear detection submodule, a sixth safety gear detection submodule, a seventh safety gear detection submodule, and an eighth safety gear detection submodule, which are electrically connected in sequence; the first safety gear detection submodule is electrically connected to the safety gear detection model training and acquisition module, and the eighth safety gear detection submodule is electrically connected to the safety gear scoring module.

[0146] Among them, the first safety gear detection submodule is used to divide the detection area into a number of second grids on average; the second safety gear detection submodule is used to detect a number of second prior frames based on all the second grids through the safety gear detection model; the third safety gear detection submodule is used to define the safety gear with the highest confidence; the fourth safety gear detection submodule is used to obtain the confidence of each second prior frame respectively, and mark the second prior frame with a confidence of 0 as the absence of safety gear; the fifth safety gear detection submodule is used to obtain the second prior frame with the largest confidence and mark it as the first-order safety gear boundary frame; the sixth safety gear detection submodule is used to calculate the intersection and union ratio of the first-order safety gear boundary frame with each other second prior frame respectively; the seventh safety gear detection submodule is used to select the first-order safety gear boundary frame with an intersection and union ratio greater than or equal to a third preset threshold as the second-order safety gear boundary frame; the eighth safety gear detection submodule is used to obtain the second-order safety gear boundary frame with the highest confidence and define it as the presence of safety gear.

[0147] Furthermore, the safety assessment device also includes a reminder signal generating module, a reminder signal sending module, an unqualified wearing ratio statistics module, an unqualified wearing ratio judgment module, and a warning signal generating and sending module, which are electrically connected in sequence; the reminder signal generating module is electrically connected to the unqualified wearing marking module.

[0148] Among them, the reminder signal generation module is used to generate a reminder signal based on unqualified dressing; the reminder signal sending module is used to send the reminder signal to the construction personnel corresponding to the portrait data whose total score does not reach the first preset threshold; the unqualified dressing ratio statistics module is used to count the unqualified dressing ratio of the portrait data whose total score does not reach the first preset threshold based on a preset time period; the unqualified dressing ratio judgment module is used to judge whether the ratio is greater than or equal to a fourth preset threshold; the warning signal generation and sending module is used to generate a warning signal if the ratio is greater than or equal to the fourth preset threshold, and send the warning signal to the construction personnel corresponding to whether the ratio is greater than or equal to the fourth preset threshold and the external coordination end.

[0149] It should be noted that this embodiment is a functional module item embodiment based on the above method embodiment. For additional contents such as the optimization, expansion, and example illustration of this embodiment, please refer to the above method embodiment, and this embodiment will not be repeated.

[0150] This embodiment defines a first standard model of the target detection algorithm, and trains the first standard model based on a person image training set, and obtains a person detection model after the training is completed; obtains a number of image data of all construction surfaces through all preset shooting terminals based on a preset time period; performs person detection on each image data through the person detection model, and obtains a person detection frame based on a complete person individual; intercepts all image data with each person detection frame as the boundary, and obtains a portrait data based on a person detection frame; detects the key points in each portrait data through a human key point detection algorithm; defines a wearing torso category based on each safety protective gear, and classifies all key points into all In the wearing torso category; obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and after the training is completed, obtain a safety gear detection model based on a type of safety gear; detect whether there is safety gear in the corresponding detection area through the safety gear detection model; if there is safety gear, it is recorded as 1 point, and if there is no safety gear, it is recorded as 0 point; obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified for wearing. This embodiment improves the target detection algorithm and integrates the key point detection algorithm to perform separate detection on each area that should have a type of safety protective gear. Compared with existing computer vision, this embodiment only performs target visual detection on the detection area divided by the key point detection algorithm. It does not require all models to traverse the same entire image separately, shortening the detection response time. At the same time, each area is scored based on the presence or absence of safety protective gear. The obtained score can reflect the amount of wearing of the construction workers and the level of safety awareness, so that the wearing situation of the construction workers has a quantitative basis, providing an effective means for subsequent systematic management and coordination.

[0151] Figure 3 An embodiment of the electronic device of the present application is shown, see Figure 3 The electronic device 12 includes a processor 121 and a memory 122 coupled to the processor 121.

[0152] The memory 122 stores program instructions for implementing the safety assessment method based on safety gear according to any of the above embodiments.

[0153] The processor 121 is used to execute program instructions stored in the memory 122 to perform safety assessment based on safety gear.

[0154] The processor 121 may also be referred to as a CPU (Central Processing Unit). The processor 121 may be an integrated circuit chip having signal processing capabilities. The processor 121 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0155] Further, Figure 4 This is a schematic diagram of the structure of a storage medium according to an embodiment of the present application. Figure 4 The storage medium 13 of the embodiment of the present application stores program instructions 131 that can implement all the above methods, wherein the program instructions 131 can be stored in the above storage medium in the form of a software product, including several instructions to enable a computer power device (which can be a personal computer, server, or network power device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes, or terminal power devices such as computers, servers, mobile phones, tablets, etc.

[0156] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0158] The specific implementation methods of the invention are described in detail above, but they are only examples, and the present application is not limited to the specific implementation methods described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application, and therefore, the equalization, modification, and improvement made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A safety assessment method based on safety protective gear, the safety assessment method is applied to construction workers wearing safety protective gear, each construction surface has at least one preset shooting end, and the current preset shooting end faces the current construction surface, characterized in that: The safety assessment method comprises: Step S1, defining a first standard model of a target detection algorithm, and training the first standard model based on a person image training set, and obtaining a person detection model after the training is completed; Step S2, acquiring a plurality of image data of all construction surfaces through all preset shooting terminals based on a preset time period; Step S3, performing person detection on each image data respectively by using the person detection model, and obtaining a person detection frame based on a complete person individual; Step S4, intercepting all image data with each person detection frame as the boundary, and obtaining a portrait data based on one person detection frame; Step S5, detecting key points in each portrait data respectively by using a human key point detection algorithm; Step S6, defining a wearing torso category based on each type of safety gear, and classifying all key points into all wearing torso categories through a classification algorithm; Step S7, obtaining the maximum area enclosed by all key points in the current wearing torso category, and defining the maximum area as the detection area of ​​the current wearing torso category; Step S8, defining a second standard model of the target detection algorithm, and training the second standard model based on a safety gear image training set for each type of safety gear, and obtaining a safety gear detection model based on one type of safety gear after the training is completed; Step S9, detecting whether the safety gear exists in the corresponding detection area through the safety gear detection model; Step S10, if the safety gear is present, it is scored as 1 point, and if the safety gear is not present, it is scored as 0 point; Step S11, obtaining the total score of all detection areas based on the current portrait data, and marking the portrait data whose total score does not reach the first preset threshold as unqualified.

2. The safety evaluation method according to claim 1, characterized in that: Step S3, performing person detection on each image data by using the person detection model, and obtaining a person detection frame based on a complete person individual, including: Step S31, each image data is evenly divided into a plurality of first grids; Step S32, detecting a plurality of first priori frames based on all first grids by using the person detection model; Step S33, defining the construction worker as having the highest confidence level; Step S34, respectively obtaining the confidence of each first priori box, and obtaining the first priori box with the largest confidence and marking it as the first-order person bounding box; Step S35, calculating the intersection-over-union ratio between the first-order person bounding box and each other first priori box; Step S36, selecting a first-order person bounding box whose intersection-over-union ratio is greater than or equal to a second preset threshold as a second-order person bounding box; Step S37, obtaining a second-order person bounding box with the highest confidence and defining it as the person detection box.

3. The safety evaluation method according to claim 1, characterized in that: Step S5, detecting key points in each portrait data by using a human key point detection algorithm, including: Step S51, training the standard VGGNet convolutional network for a preset number of times through the OpenPose library, and obtaining a VGGNet prediction model after the training is completed; Step S52, inputting each portrait data into the VGGNet prediction model to extract human body features; Step S53, extracting a plane confidence map of the position of the body part in the current portrait data through the first branch of the two-branch CNN model; Step S54, extracting the plane affinity vector field of the body part positions in the current portrait data through the second branch in the two-branch CNN model; Step S55 , calculating key points of the plane confidence graph and the plane affinity vector field of the current portrait data through bipartite graph matching of the Hungarian algorithm.

4. The safety evaluation method according to claim 1, characterized in that: Step S6, defining a wearing torso category based on each type of safety gear, and classifying all key points into all wearing torso categories through a classification algorithm, including: Step S61: define a set of data to be classified based on all key points ,in The data set to be classified The Key points, is the number of all key points; Step S62, defining a category set according to the wearing torso category ,in, For the category set The Wear torso categories, is the number of all wearing torso categories; Step S63, calculate the conditional probability of each key point in each wearing torso category according to formula (1): （1）； in, For the Under the category of wearing torso, the data set to be classified The conditional probability of For the The marginal probability of the wearing torso category; For the Under the category of wearing torso, The conditional probability of key points; Step S64, classify each key point into the wearing torso category with the highest conditional probability.

5. The safety evaluation method according to claim 1, wherein the safety gear comprises a helmet, protective clothing, gloves, protective pants, and protective shoes, characterized in that: Step S7, obtaining the maximum area enclosed by all key points in the current wearing torso category, and defining the maximum area as the detection area of ​​the current wearing torso category, including: Step S71, using a human key point detection algorithm to detect at least one key point on the head, at least one key point on each shoulder, at least one key point on each hand, at least one key point on each side of the hips, at least one key point on each knee, and at least one key point on each foot in each portrait data; Step S72, defining the maximum area enclosed by all key points of the head as the detection area of ​​the helmet; Step S73, defining the maximum area enclosed by all shoulder key points and all hip key points as the detection area of ​​the protective clothing; Step S74, defining the maximum area enclosed by all the key points of the hand as the detection area of ​​the glove; Step S75, defining the maximum area enclosed by all the key points of the hips, all the key points of the knees and all the key points of the feet as the detection area of ​​the protective pants; Step S76, defining the maximum area enclosed by all the key points of the foot as the detection area of ​​the protective shoe.

6. The safety evaluation method according to claim 1, characterized in that: Step S9, detecting whether the safety gear exists in the corresponding detection area through the safety gear detection model, comprises: Step S91, dividing the detection area into a plurality of second grids on average; Step S92, detecting a plurality of second priori frames based on all second grids through the safety gear detection model; Step S93, defining the safety gear as having the highest confidence level; Step S94, respectively obtaining the confidence of each second priori frame, and marking the second priori frame with a confidence of 0 as not containing the safety gear; Step S95, obtaining the second priori frame with the highest confidence and marking it as the first-order safety gear bounding box; Step S96, calculating the intersection-and-union ratio of the first-order safety gear bounding box with each of the other second priori boxes; Step S97, selecting a first-order safety gear bounding box whose intersection-over-union ratio is greater than or equal to a third preset threshold as a second-order safety gear bounding box; Step S98, obtaining a second-order safety gear boundary box with the highest confidence and defining it as the existence of the safety gear.

7. The safety evaluation method according to claim 1, characterized in that: Step S11, based on the current portrait data, the total score of all detection areas is obtained, and the portrait data whose total score does not reach the first preset threshold is marked as unqualified, and then, it includes: Step S12, generating a reminder signal based on the unqualified wearing; Step S13, sending the reminder signal to the construction personnel corresponding to the portrait data whose total score does not reach the first preset threshold; Step S14, counting the proportion of unqualified portrait data with a total score that does not reach a first preset threshold based on the preset time period; Step S15, determining whether the proportion is greater than or equal to a fourth preset threshold, if the proportion is greater than or equal to the fourth preset threshold, executing step S16; Step S16, generating a warning signal, and sending the warning signal to the construction personnel corresponding to whether the proportion is greater than or equal to the fourth preset threshold and the external coordination end.

8. A safety assessment device based on safety gear, wherein the safety assessment device based on safety gear is applied to the safety assessment method according to any one of claims 1 to 7, characterized in that: The safety evaluation device comprises: A person detection model training and acquisition module is used to define a first standard model of the target detection algorithm, and train the first standard model based on a person image training set to obtain a person detection model after the training is completed; A construction surface image data acquisition module, used to acquire a number of image data of all construction surfaces through all preset shooting ends based on a preset time period; A person detection frame acquisition module is used to perform person detection on each image data by using the person detection model, and obtain a person detection frame based on a complete person individual; A portrait data acquisition module is used to intercept all image data with each person detection frame as the boundary, and obtain a portrait data based on a person detection frame; A portrait data key point extraction module is used to detect the key points in each portrait data by using a human key point detection algorithm; The portrait data key point classification module is used to define a wearing torso category based on each type of safety gear, and classify all key points into all wearing torso categories through a classification algorithm; A safety gear detection area definition module is used to obtain the maximum area enclosed by all key points in the current wearing torso category, and define the maximum area as the detection area of ​​the current wearing torso category; A safety gear detection model training and acquisition module, used to define a second standard model of the target detection algorithm, and train the second standard model based on a safety gear image training set for each type of safety gear, and obtain a safety gear detection model based on one type of safety gear after the training is completed; A safety gear detection module, used to detect whether the safety gear exists in the corresponding detection area through the safety gear detection model; A safety gear scoring module, used to score 1 point if the safety gear is present, and score 0 point if the safety gear is not present; The unqualified wearing marking module is used to obtain the total score of all detection areas based on the current portrait data, and mark the portrait data whose total score does not reach the first preset threshold as unqualified wearing.

9. An electronic device, characterized in that: It includes a processor and a memory coupled to the processor, wherein the memory stores program instructions that can be executed by the processor; when the processor executes the program instructions stored in the memory, the safety assessment method based on safety gear as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed by the processor, the safety assessment method based on safety gear according to any one of claims 1 to 7 can be implemented.

Images