Semantic segmentation-based high-altitude operation safety belt and safety rope detection method

Abstract

The invention relates to the technical field of safety monitoring of machine vision, and particularly discloses a semantic segmentation-based high-altitude operation safety belt and safety rope detection method, which comprises the following steps of: performing polygonal labeling on figures, safety belts and safety ropes in a high-altitude operation picture by using a labeling tool, and distributing high-altitude operation category names; the method comprises the following steps: constructing a semantic segmentation model based on deep learning, carrying out convolution and down-sampling on an aerial work picture so as to obtain feature maps of different scales, using the multi-scale feature maps for predicting the size, position and category of masks, obtaining a prediction result, drawing the prediction result on the picture, and classifying all pixels in the picture by the masks of different categories; training the semantic segmentation model, and optimizing the parameter weight of the model; after training is completed, a semantic segmentation model is used for reasoning pictures, people, safety belts and safety ropes at different positions are distinguished, the method conforms to a current video monitoring system, and precision improvement is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring of machine vision, and more specifically, to a method for detecting a safety belt and a safety rope for aerial work based on semantic segmentation. Background Art

[0002] In industries such as electricity and construction, workers are often required to work at heights. For workers, there is a risk of falling when working at heights, which can cause harm to the life and health of workers. Therefore, they need to wear safety devices such as seat belts according to regulations. However, workers tend not to wear seat belts correctly as required for some reasons. Among those injured by falling, the proportion of those who do not wear seat belts correctly is high. In order to better protect the life and health of workers, it is necessary to pay attention to whether they wear seat belts correctly when working at heights at any time, and to remind and warn them in time when they find that the seat belts are not used correctly. At the current engineering site, by installing surveillance cameras and summarizing the collected video information, it is possible to supervise most workers with only a small amount of manpower, and by using equipment such as radio or intercom, it is possible to issue reminders or alarms to workers in time. However, paying attention to the status of a large number of workers in multiple images for a long time is undoubtedly a challenge to the supervisor's spirit, which may lead to a gradual decrease in efficiency and a gradual increase in judgment errors.

[0003] In the above situation, using machine vision to replace or assist human supervision of workers' safety is an alternative solution. In recent years, with the development of deep learning, neural network models have gradually begun to be applied in various industries, and the accuracy of various machine vision tasks has continued to improve due to the emergence of deep learning methods. In the scene of aerial work, the background is complex, the scale of workers and safety belts is relatively small, and the shapes are varied. Through semantic segmentation, the edges of objects are divided at the pixel level to accurately outline the shape of the object.

[0004] In view of this, the present invention provides a method for detecting aerial work safety belts and safety ropes based on semantic segmentation to solve the above problems. Summary of the invention

[0005] In order to overcome the problems in the prior art, the present invention proposes a method for detecting aerial work safety belts and safety ropes based on semantic segmentation, which can automatically identify aerial work workers, the safety belts they wear, and the safety ropes fixed at the anchor points, and accurately mark them with masks respectively.

[0006] In a first aspect, the present invention provides a high-altitude work safety belt detection algorithm based on semantic segmentation, comprising the following steps:

[0007] Step 1: Use the annotation tool to make polygon annotations on the people, safety belts and safety ropes in the aerial work pictures, and assign aerial work category names;

[0008] Step 2: Build a semantic segmentation model based on deep learning, perform convolution and downsampling on the aerial work pictures to obtain feature maps of different scales, use the multi-scale feature maps to predict the size, position and category of the mask, obtain the prediction results (mask) and draw them on the picture. Masks of different categories classify all pixels in the picture.

[0009] Step 3: Train the semantic segmentation model. The loss function in the Mask2Former framework of the segmentation head will compare the semantic segmentation results predicted by the model for the input image with the true value results annotated by humans, and optimize the parameter weights of the model.

[0010] Step 4: After training, use the semantic segmentation model to infer the image and distinguish people in different positions, seat belts, and safety ropes.

[0011] As a preferred technical solution of the first aspect of the present invention, the aerial work pictures are pictures related to aerial work obtained from power plants and the Internet, a total of 2016 pictures, and all aerial work pictures are integrated to establish an aerial work data set;

[0012] Use annotation tools to annotate all aerial work images with polygons, including people standing on the ground, people working at high altitudes, safety belts and safety ropes; and classify the annotation content into corresponding category names based on the aerial work scenes.

[0013] As a preferred technical solution of the first aspect of the present invention, the judgment standard of the safety rope is the part used by workers working at heights, which has been fixed at the anchor point and is outside the human body contour.

[0014] As a preferred technical solution of the first aspect of the present invention, based on the semantic segmentation model, ConvNeXt is used as the backbone network, the backbone network includes a Transformer deep learning framework, the Mask2Former framework is used in the segmentation head of the semantic segmentation model, and a multi-scale attention mechanism module is added to the end of the backbone network; the following steps are included:

[0015] Step 21: Preprocess all the aerial work images in the aerial work data set into aerial work annotation images of the same standard according to the aerial work category name, and record the file names of all the aerial work annotation images in the train.txt, val.txt, and test.txt files respectively as the training set, validation set, and test set;

[0016] Step 22: Capture the pattern between pixels in the image based on the ConvNeXt backbone network and generate a multi-scale feature map;

[0017] Step 23: The feature map output by the backbone is the input of the effective multi-scale attention EMA module. The two spatial attention maps are added and the multi-scale feature map of the output features is mapped according to the attention weight using the sigmoid function.

[0018] Step 24: Input the multi-scale feature map into the segmentation head Mask2Former framework for fusion. The fused feature map will be used for mask prediction to obtain the prediction result.

[0019] As a preferred technical solution of the first aspect of the present invention, the unified standard is to unify the images into specifications with 3 channels, height and width of 1024 pixels by shrinking or filling the boundaries based on the training program, and then mark the aerial work annotation map in the form of an array as training data.

[0020] As a preferred technical solution of the first aspect of the present invention, the ConvNeXt backbone network includes:

[0021] Based on the convolution block Conv1, a feature map of size 96×256×256 is generated according to the image;

[0022] The feature map does not change its size after passing through the ConvNeXtBlock layer and is output as the multi-scale feature map obtained by the backbone. After passing through Downsample modules 1, 2, and 3, the feature map sizes become 192×128×128, 384×64×64, and 768×32×32, respectively.

[0023] In the ConvNeXt Block1 layer, ConvNeXt Block2 layer, ConvNeXt Block3 layer and ConvNeXtBlock4 layer, 3, 3, 9 and 3 ConvNeXt Block infrastructures are stacked successively respectively;

[0024] In the basic structure, the first depth convolution uses a 7×7 convolution kernel and does not change the number of channels. The other two convolutions use a 1×1 convolution kernel. Conv1 expands the number of channels to 4 times the original number, and Conv2 restores the expanded number of channels.

[0025] The GELU activation function in the ConvNeXt Block is used in the ConvNeXt backbone network.

[0026] As a preferred technical solution of the first aspect of the present invention, the effective multi-scale attention EMA module includes:

[0027] After the feature map is input into the effective multi-scale attention EMA module, it is divided into n groups in the channel (C) direction, and then each group is divided into two branches for calculation. The feature map is symmetrically average-pooled along the two perpendicular directions of width (W) and height (H);

[0028] The pooled results are concatenated along the W(H) direction, and the concatenated size is C / / n×(W+H). The convolution is then performed, and the output of the convolution is split into two vectors along the previous concatenation direction. The two vectors are fitted with a sigmoid function to fit a two-dimensional binomial distribution, and then multiplied to make the channels interact.

[0029] In cross-spatial learning, the outputs of the two branches pass through a symmetrical structure to generate two spatial attention maps.

[0030] As a preferred technical solution of the first aspect of the present invention, the multi-scale feature map is input into the Pixel Decoder classification module in the Mask2Former framework for multi-scale feature fusion and pixel-level prediction, and is input into the Transformer Decoder at a single scale for prediction. The prediction results output by the two secondary modules are combined as the prediction results of the semantic segmentation model.

[0031] As a preferred technical solution of the first aspect of the present invention, Poly Loss and Log-Cosh Dice Loss are introduced into the loss function.

[0032] The Log-Cosh Dice Loss function is a loss function with the symmetry of Dice loss, and together with the binary cross entropy loss constitutes the mask loss function of the model.

[0033] In a second aspect, the present invention provides a high-altitude work safety belt, a high-altitude work safety belt, used in the first aspect, comprising: obtaining the marked points where the high-altitude work safety belt contacts the human body based on the high-altitude work safety belt detection algorithm.

[0034] The specific advantages of the present invention are as follows:

[0035] The present invention divides the safety belt into two parts, the safety belt worn on the body and the safety rope fixed at the anchor point, which not only avoids the incorrect usage of only considering the safety belt as a whole and ignoring the unfixed safety rope, but also avoids the situation where the structure of the safety belt is classified too much and the subdivided structure is completely blocked and difficult to detect.

[0036] The present invention adopts a distributed image acquisition and centralized image processing method, which is in line with current video monitoring systems and is conducive to improving accuracy.

[0037] The algorithm of the present invention is adapted to run in a central server instead of running in an edge or terminal device, which can avoid the situation where there are many distributed intelligent devices, high replacement costs and difficult maintenance and overhaul. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the high-altitude work safety belt detection algorithm based on semantic segmentation of the present invention;

[0039] Figure 2 It is the overall structure diagram of the CEMFormer model of the present invention;

[0040] Figure 3 This is the structural diagram of the ConvNeXt Block of the present invention;

[0041] Figure 4 It is the structure diagram of DETR Encoder of the present invention;

[0042] Figure 5 This is a structural diagram of the Decoder of the present invention;

[0043] Figure 6 It is the structural diagram of EMA of the present invention;

[0044] Figure 7 This is an example diagram of Embodiment 2 of the present invention;

[0045] Figure 8 This is an example diagram of the detection results of Example 2 of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example 1

[0048] See also Figure 1 The present invention provides a technical solution: a high-altitude work safety belt detection algorithm based on semantic segmentation to complete the high-altitude work worker safety belt detection task, including the following steps:

[0049] Step 1: Use the annotation tool to make polygon annotations on the people, safety belts and safety ropes in the aerial work pictures, and assign aerial work category names;

[0050] Specifically, the aerial work pictures are pictures related to aerial work obtained from power plants and the Internet, a total of 2016 pictures, and all aerial work pictures are integrated to establish an aerial work data set;

[0051] Use annotation tools to annotate the people standing on the ground, people working at height, safety belts and safety ropes in all aerial work pictures with polygons; and classify the annotation content into corresponding category names based on the aerial work scene;

[0052] More specifically, the annotation tool is ISAT_with_segment_anything, which is an open source and free annotation tool with a wider application prospect in practical applications. The ISAT_with_segment_anything annotation tool is used to annotate the people standing on the ground, people working at heights, safety belts and safety ropes in all pictures with polygons.

[0053] Ground, Offground, Safebelt, and Saferope are used as their category names to help the algorithm learn to distinguish different objects and scenes, so that in practical applications it can accurately detect and identify whether workers working at heights are wearing safety belts and safety ropes correctly. Areas that are not labeled in the image will be automatically labeled as Background by the tool. This detailed classification is crucial to improving the accuracy and reliability of the detection algorithm.

[0054] Among them, the judgment standard for the safety rope is the part used by workers working at heights, which has been fixed at the anchor point and is outside the human body outline.

[0055] Step 2: Build a semantic segmentation model based on deep learning, perform convolution and downsampling on the aerial work pictures to obtain feature maps of different scales, use the multi-scale feature maps to predict the size, position and category of the mask, obtain the prediction results (mask) and draw them on the picture. Masks of different categories classify all pixels in the picture.

[0056] Specifically, if Figure 2 As shown, based on the semantic segmentation model, ConvNeXt is used as the backbone network, the backbone network includes the Transformer deep learning framework, the Mask2Former framework is used in the segmentation head of the semantic segmentation model, and the Efficient Multi-Scale Attention (EMA) module is added to the end of the backbone network.

[0057] Step 21: pre-process all aerial work images in the aerial work data set into aerial work annotation images of the same standard according to the aerial work category name, and mark the aerial work annotation images in the form of an array as training data;

[0058] Specifically, the unified standard is to unify the images into 3 channels, 1024 pixels in height and width by shrinking or filling the boundaries based on the training program, and then mark the aerial work annotation images in array form as training data.

[0059] Step 22: Capture the pattern between pixels in the image based on the ConvNeXt backbone network and generate a multi-scale feature map;

[0060] Specifically, the feature map contains rich and important pattern information, and the ConvNeXt backbone network includes:

[0061] Based on the convolution block Conv1, a feature map of size 96×256×256 is generated according to the image;

[0062] The feature map does not change its size after passing through the ConvNeXtBlock layer and is output as the multi-scale feature map obtained by the backbone. After passing through Downsample modules 1, 2, and 3, the feature map sizes become 192×128×128, 384×64×64, and 768×32×32, respectively. The internal structure of ConvNeXt Block is shown in Figure 3 shown.

[0063] In the ConvNeXt Block1 layer, ConvNeXt Block2 layer, ConvNeXt Block3 layer and ConvNeXtBlock4 layer, 3, 3, 9 and 3 ConvNeXt Block infrastructures are stacked successively respectively;

[0064] In the basic structure, the first depth convolution uses a 7×7 convolution kernel and does not change the number of channels. The other two convolutions use a 1×1 convolution kernel. Conv1 expands the number of channels to 4 times the original number, and Conv2 restores the expanded number of channels.

[0065] In order to streamline the model, the GELU activation function in the ConvNeXt Block is used in the entire ConvNeXt backbone network.

[0066] Step 23: The feature map output by the backbone is the input of the effective multi-scale attention EMA module. The two spatial attention maps are added and the multi-scale feature map of the output features is mapped according to the attention weight using the sigmoid function. The structure of the effective multi-scale attention EMA module is as follows: Figure 6 shown.

[0067] After the feature map is input into the effective multi-scale attention EMA module, it is divided into n groups in the channel (C) direction, and then each group is divided into two branches for calculation. For example, n = 8. Conv1 uses a 3×3 convolution kernel, and the branch it is in is a 3×3 branch. Conv2 uses a 1×1 convolution kernel, and the branch it is in is a 1×1 branch.

[0068] In the 1×1 branch, the feature map is average pooled symmetrically along the two perpendicular directions of width (W) and height (H). This operation can effectively capture its vertical position information.

[0069] The pooled results are spliced ​​along the W(H) direction, and the size of the spliced ​​result is C / / n×(W+H), and convolution is performed. Conv2 does not change the size of the feature map, so there will be no dimensionality reduction, and no information loss will occur.

[0070] The output of the convolution is re-split into two vectors along the direction of the previous splicing, and each vector is passed through the sigmoid function to fit the two-dimensional binomial distribution, and then multiplied to make the channels interact. The 3×3 branch only captures multi-scale features through Conv1, and then enters cross-spatial learning.

[0071] In cross-spatial learning, the outputs of the two branches go through a symmetrical structure, namely, 2D global average pooling to encode spatial information, a softmax function, and finally vector multiplication to generate two spatial attention maps.

[0072] At the end of the effective multi-scale attention EMA module, the two spatial attention maps are added and the output features are mapped according to the attention weights using the sigmoid function.

[0073] In the entire effective multi-scale attention EMA module, these symmetrical structures capture spatial semantic information and perform inter-channel interactions, providing pixel-level global context for the feature map. In addition, the number of channels does not change, avoiding information loss caused by channel dimensionality reduction. These optimizations of the feature map will enhance the effect of semantic segmentation.

[0074] Step 24: Input the multi-scale feature map into the segmentation head Mask2Former framework for fusion. The fused feature map will be used for mask prediction to obtain the prediction result.

[0075] In the Mask2Former framework, the multi-scale feature map is input into the Pixel Decoder classification module for multi-scale feature fusion and pixel-level prediction, and is also input into the Transformer Decoder for prediction at a single scale. The prediction results output by these two secondary modules are combined to form the prediction result of semantic segmentation.

[0076] In the Pixel Decoder classification module, the multi-scale feature map is convolved to unify the number of channels to 256, and then enters the DETR Encoder after position encoding to capture the global context. Then, the nearest neighbor interpolation method is used for upsampling, and the multi-scale feature maps are fused. The fused feature maps will be used for mask prediction. The structure of the DETR Encoder is as follows: Figure 4 shown.

[0077] After the feature map enters the DETR Encoder, it is divided into three vectors K, Q, and V, where K is associated with V and the position code, and then the three vectors are used as the input of the multi-head self-attention (MHSA). The output of the MHSA is added to the feature map and layer-normalized, then divided into two branches, one of which passes through the feed-forward network (FFN) and is added to the other branch, and finally layer-normalized once before output.

[0078] In the Transformer Decoder, feature maps of each scale enter different decoder layers and are used together with the input query features associated with the positional encoding to predict the category and mask. The structure of the Decoder is as follows: Figure 5 As shown in Figure 2, its structure is similar to that of DETR Encoder, but it adds masked attention and corresponding layer normalization operations as a substitute for cross attention, which focuses the model's attention more on the foreground.

[0079] Step 3: Train the semantic segmentation model. The loss function in the Mask2Former framework of the segmentation head will compare the semantic segmentation results predicted by the model for the input image with the true value results annotated by humans, and optimize the parameter weights of the model.

[0080] Specifically, Poly Loss and Log-Cosh Dice Loss are introduced into the loss function. At this time, the loss function of the model is as shown in formula (1).

[0081] L=λ ce L ce +λ lc-dce L lc-dce +λ poly L poly (1)

[0082] L ce is the binary cross entropy loss, and its expression is shown in formula (2).

[0083]

[0084] Among them, y i is the true value of sample i, which is 0 or 1, σ(p i ) is the result of the predicted value of sample i after the sigmoid function, and N is the total number of samples. lc-dce is the Log-Cosh Dice Loss, and its expression is shown in formula (3).

[0085]

[0086] Among them, p i is the predicted value of pixel i, y i is the true value of pixel i, and N is the total number of pixels. The Log-Cosh Dice Loss function is a loss function that has the symmetry of Dice loss and is smooth and convenient for obtaining gradients. It and the binary cross entropy loss together constitute the mask loss function of the model. poly is PolyLoss, which is the category loss function of the model, and its expression is shown in formula (5).

[0087]

[0088] Among them, y i,c is the true value of sample i, p i,c is the probability that sample i is predicted to be of category c, C is the total number of categories, N is the total number of samples, and ε is a hyperparameter used to adjust the polynomial coefficient. On the dataset constructed by the present invention, setting ε to -1 can improve the training effect and the accuracy of the model. ce ,λ lc-dce and λ poly are the weights of the three loss functions, which are 5, 5 and 2 respectively.

[0089] Step 4: After training, use the semantic segmentation model to infer the image and distinguish people in different positions, seat belts, and safety ropes.

[0090] Specifically, during inference, the model will repeat steps 22-24 to output the result of semantic segmentation. Figure 7 The result of segmentation is as follows Figure 8 As shown in Figure 3, after training on a dataset using the new annotation principle, our model can distinguish people in different positions, seat belts and safety ropes with good segmentation results.

[0091] Example 2: A technical solution based on Example 1: a high-altitude work safety belt detection algorithm based on semantic segmentation, which further proves the effectiveness of the method described in Example 1.

[0092] The semantic segmentation model is compared with the baseline Mask2Former framework (using ResNet50 as the backbone) and five contemporary models PIDNet, SegNext, PoolFormer, ConvNeXt (using Uper Head as the segmentation head), and Deep Dual-resolution Network (DDRNet). Step 2 is repeated to generate the accuracy data of the semantic segmentation model on the test set. The results are shown in Table 1.

[0093] Table 1: Comparison of the accuracy of CEMFormer and contemporary semantic segmentation models

[0094]

[0095] In Table 1, Acc i The percentage of pixels predicted correctly for the i-th class to all pixels in that class is the pixel accuracy, and mAc is the average of all class Accs, i.e., the average pixel accuracy. CEMFormer achieves a mAc of 92.31%, surpassing PIDNet, SegNext, PoolFormer, ConvNeXt, DDRNet, and the baseline by 3.96%, 5.35%, 5.00%, 3.72%, 28.89%, and 3.15%, respectively. For the accuracy of a single category, our model loses 0.21%, 1.67%, and 0.49% in the background, high-altitude, and ground categories, respectively, compared to the baseline. However, it improves by 4.59% and 13.54% in the safety belt and safety rope categories, respectively. This trade-off is considered worthwhile, especially since the safety rope class achieves an Acc of 91.68%, which is the main focus of our method. Except for DDRNet, all other contemporary models achieve mAcs of over 85%, indicating that our annotation principle is effective.

[0096] The effectiveness of the proposed algorithm is demonstrated through ablation experiments. Part of the model structure is gradually reduced and step 2 is repeated. The results are shown in Table 2.

[0097] Table 2: CEMFormer ablation experiment results

[0098]

[0099] In Table 2, "√" indicates that the model includes the corresponding component in the experiment, while "×" indicates that the model does not include the component but uses the corresponding structure in the baseline. When comparing Experiments 1 and 2 in Table 2, replacing Log-Cosh-Dice Loss with Dice Loss in the baseline leads to a 2.29% reduction in mAcc. This drop is mainly attributed to the drop in Acc for the high-altitude, safety belt, and safety rope categories, which dropped by 6.36%, 4.87%, and 2.81%, respectively. This shows that the smoother Log Cosh Dice Loss has an advantage in training. When comparing Experiments 2 and 3, reverting Poly Loss to cross entropy loss reduced mAcc by 0.3%. Cross entropy loss showed advantages in the background, high-altitude, and safety belt categories, while Poly loss significantly improved the classification accuracy of the safety rope by 8.69%, representing a substantial improvement in identifying the safety rope.

[0100] Similarly, removing EMA results in a significant loss in Acc, especially for the safety rope category, which drops by 3.13% from 80.18% in Experiment 3 to 77.05% in Experiment 4. This reduction is due to the loss of high-resolution feature maps provided by EMA, which makes it more difficult to segment small-scale objects such as safety ropes. Finally, although ConvNeXt, as a backbone that incorporates many advanced designs, does not provide a clear advantage, it is still worth using compared to the classic ResNet. When comparing Experiments 4 and 5, removing ConvNeXt results in an Acc change of less than 2% for a single category and a 0.28% drop in mAcc.

[0101] Embodiment 3: The embodiment of the present invention further provides a high-altitude work safety belt, which is used in a high-altitude work safety belt detection algorithm based on semantic segmentation, including: obtaining the annotation points where the high-altitude work safety belt contacts the human body based on the high-altitude work safety belt detection algorithm.

[0102] Embodiment 4: The embodiment of the present invention further provides an electronic device, comprising: a processor and a temperature control component, wherein the temperature control component stores a computer program that can be called by the processor;

[0103] The processor executes the above-mentioned high-altitude work safety belt detection algorithm based on semantic segmentation by calling the computer program stored in the temperature control component.

[0104] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more temperature control components, wherein the temperature control component stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the stock algorithm trading method based on deep neural network provided by the above-mentioned various method embodiments. The electronic device may also include other components for realizing the functions of the device, for example, the electronic device may also have components such as wired or wireless network interfaces and input and output interfaces for input and output. The embodiments of this application are not described in detail here.

[0105] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0106] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0107] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0108] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0111] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0112] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0113] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-altitude work safety belt detection algorithm based on semantic segmentation, characterized in that: The following steps are involved: Step 1: Use the annotation tool to make polygon annotations on the people, safety belts and safety ropes in the aerial work pictures, and assign aerial work category names; Step 2: Build a semantic segmentation model based on deep learning, perform convolution and downsampling on the aerial work pictures to obtain feature maps of different scales, use the multi-scale feature maps to predict the size, position and category of the mask, obtain the prediction results and draw them on the picture, and use masks of different categories to classify all pixels in the picture; Step 3: Train the semantic segmentation model. The loss function in the Mask2Former framework of the segmentation head will compare the semantic segmentation results predicted by the model for the input image with the true value results annotated by humans, and optimize the parameter weights of the model. Step 4: After training, use the semantic segmentation model to infer the image and distinguish people in different positions, seat belts, and safety ropes.

2. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 1 is characterized in that: The aerial work pictures are pictures related to aerial work obtained from power plants and the Internet, a total of 2016 pictures, and all the aerial work pictures are integrated to establish a aerial work data set; Use annotation tools to annotate all aerial work images with polygons, including people standing on the ground, people working at high altitudes, safety belts and safety ropes; and classify the annotation content into corresponding category names based on the aerial work scenes.

3. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 2 is characterized in that: The criterion for judging a safety rope is the portion used by workers working at heights that has been secured at an anchor point and is outside the outline of the human body.

4. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 3 is characterized in that: Based on the semantic segmentation model, ConvNeXt is used as the backbone network, the backbone network includes the Transformer deep learning framework, the Mask2Former framework is used in the segmentation head of the semantic segmentation model, and a multi-scale attention mechanism module is added to the end of the backbone network; the following steps are included: Step 21: Preprocess all aerial work images in the aerial work data set into aerial work labeled images of the same standard according to the aerial work category name, and record the file names of all aerial work labeled images in files as training set, validation set and test set respectively; Step 22: Capture the pattern between pixels in the image based on the ConvNeXt backbone network and generate a multi-scale feature map; Step 23: The feature map output by the backbone is the input of the effective multi-scale attention EMA module. The two spatial attention maps are added and the multi-scale feature map of the output features is mapped according to the attention weight using the sigmoid function. Step 24: Input the multi-scale feature map into the segmentation head Mask2Former framework for fusion. The fused feature map will be used for mask prediction to obtain the prediction result.

5. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 4 is characterized in that: The ConvNeXt backbone network includes: Based on the convolution block Conv1, a feature map of size 96×256×256 is generated according to the image; The feature map does not change its size after passing through the ConvNeXtBlock layer and is output as the multi-scale feature map obtained by the backbone. After passing through Downsample modules 1, 2, and 3, the feature map sizes become 192×128×128, 384×64×64, and 768×32×32, respectively. In the ConvNeXt Block1 layer, the ConvNeXt Block2 layer, the ConvNeXt Block3 layer, and the ConvNeXt Block4 layer, 3, 3, 9, and 3 ConvNeXt Block infrastructure structures are stacked successively, respectively; In the basic structure, the first depth convolution uses a 7×7 convolution kernel without changing the number of channels. The two convolutions use a 1×1 convolution kernel. Conv1 expands the number of channels to 4 times the original number, and Conv2 restores the expanded number of channels. The GELU activation function in ConvNeXtBlock is used in the ConvNeXt backbone network.

6. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 5 is characterized in that: The effective multi-scale attention EMA module includes: After the feature map is input into the effective multi-scale attention EMA module, it is divided into n groups in the channel (C) direction, and then each group is divided into two branches for calculation. The feature map is symmetrically average-pooled along the two perpendicular directions of width (W) and height (H); The pooled results are concatenated along the W(H) direction, and the concatenated size is C / / n×(W+H). The convolution is then performed, and the output of the convolution is split into two vectors along the previous concatenation direction. The two vectors are fitted with a sigmoid function to fit a two-dimensional binomial distribution, and then multiplied to make the channels interact. In cross-spatial learning, the outputs of the two branches pass through a symmetrical structure to generate two spatial attention maps.

7. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 6 is characterized in that: In the Mask2Former framework, the multi-scale feature map is input into the PixelDecoder classification module for multi-scale feature fusion and pixel-level prediction, and is input into the Transformer Decoder for prediction at a single scale. The prediction results output by the two secondary modules are combined into the prediction results of the semantic segmentation model.

8. The high-altitude work safety belt detection algorithm based on semantic segmentation according to claim 7 is characterized in that: Introduce Poly Loss and Log-Cosh Dice Loss into the loss function, The Log-Cosh Dice Loss function is a loss function with the symmetry of Dice loss, and together with the binary cross entropy loss constitutes the mask loss function of the model.

9. A safety belt for aerial work, characterized in that: The semantic segmentation-based aerial work safety belt detection algorithm used in any one of claims 1 to 8 comprises: obtaining the marked points where the aerial work safety belt contacts the human body based on the aerial work safety belt detection algorithm.

Images