A method and system for detecting illegal wearing of safety belts for high-altitude operations

Abstract

The present invention discloses a method and system for detecting seat belt violations for aerial work, the method comprising: obtaining an aerial work image to be detected, and preprocessing the aerial work image to be detected to obtain a target image to be detected; inputting the target image to be detected into a target seat belt violation detection model for feature extraction and semantic guidance to obtain a violation prediction probability; wherein the target seat belt violation detection model comprises an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module; based on the violation prediction probability, determining the violation corresponding to the target image to be detected. The present invention can effectively identify and classify various incorrect seat belt wearing behaviors by performing multi-level feature extraction and semantic guidance on images in aerial work scenes, thereby improving the safety monitoring level of aerial work and reducing safety hazards.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for detecting illegal wearing of safety belts for aerial work. Background Art

[0002] When performing aerial work, the correct wearing of safety belts is crucial to ensure the safety of workers. Therefore, the detection of illegal wearing of safety belts for aerial work is particularly critical. It can effectively monitor and warn workers of illegal wearing of safety belts in real time, thereby promoting the standardized use of safety belts and further enhancing the safety protection of workers.

[0003] However, in the actual high-altitude working environment, accurately identifying the irregular wearing of seat belts faces great challenges. First, due to the presence of a large amount of interference information such as wires in the high-altitude working scene, and the relatively small visual area occupied by the seat belt in the entire scene, it becomes a major challenge to accurately extract the semantic information of the seat belt in a complex background. When the features are not obvious and the details are missing, the difficulty of the existing detection methods increases significantly. At the same time, the wearing of the seat belt has a considerable correlation with the body's posture. When working at high altitude, the posture and position of the operator change frequently, such as bending, stretching, climbing and other dynamic behaviors. These posture changes significantly increase the difficulty of the detection method to accurately judge whether there is illegal wearing of the seat belt, thereby complicating the detection task. In addition, compared with conventional image recognition tasks, it is particularly difficult to collect training samples for high-altitude working scenes, which makes it more likely to have insufficient generalization ability of deep learning models. At present, there are few related research applications of deep learning in the field of high-altitude working seat belt violation detection. The existing seat belt violation detection methods only focus on detecting a single violation, and the research scope is limited to the basic recognition task of whether there is a seat belt. Summary of the invention

[0004] In order to solve the deficiencies of the prior art in identifying various seat belt violation wearing behaviors, the present invention provides a seat belt violation wearing detection method and system for high-altitude operations.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting illegal wearing of a safety belt for high-altitude operations, comprising:

[0006] Acquire the high-altitude work image to be detected, and pre-process the high-altitude work image to be detected to obtain the target image to be detected;

[0007] Input the target image to be detected into the target seat belt violation wearing detection model for feature extraction and semantic guidance to obtain the violation item prediction probability; wherein the target seat belt violation wearing detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module, and the target seat belt violation wearing detection model is configured to perform feature extraction on the target image to be detected through the overall image feature extraction module and the seat belt segmentation module to obtain overall image feature information and seat belt semantic information respectively, and input the overall image feature information and the seat belt semantic information into the semantic guidance module for semantic guidance to obtain the violation item prediction probability;

[0008] Based on the violation item prediction probability, a violation item corresponding to the target image to be detected is determined.

[0009] Preferably, the preprocessing of the high-altitude work image to be detected to obtain the target image to be detected includes:

[0010] Extracting the region of interest from the high-altitude work image to be detected to obtain a high-altitude work personnel detection result;

[0011] The detection result of the high-altitude workers is expanded and cropped to obtain a target image to be detected.

[0012] Preferably, before inputting the target image to be detected into the target seat belt violation wearing detection model for feature extraction and semantic guidance to obtain the violation item prediction probability, the method further includes:

[0013] Obtain a data set of aerial work images;

[0014] Labeling the high-altitude work image dataset for violation items, and dividing the labeled high-altitude work image dataset into a training set and a test set;

[0015] Using the training set, the semantic guidance module and the seat belt segmentation module are trained based on the loss function to obtain an initial seat belt violation wearing detection model;

[0016] The initial seat belt violation detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module; the loss function is a weighted sum of the violation classification loss and the seat belt segmentation loss, the violation classification loss includes a binary cross entropy loss, and the seat belt segmentation loss includes a Dice loss;

[0017] The test set is used to verify the initial seat belt violation wearing detection model to obtain a target seat belt violation wearing detection model.

[0018] Preferably, the overall image feature extraction module includes a multi-layer convolution module; wherein each layer of the convolution module includes a first convolution layer, a second convolution layer, a third convolution layer and a fusion module connected in sequence, the convolution kernel size of the first convolution layer and the third convolution layer is the same and different from the convolution kernel size of the second convolution layer, the second convolution layer is a depth-separable convolution layer, the first convolution layer is used to adjust the number of channels of the input feature map, the second convolution layer is used to extract the spatial features of the input feature map, the third convolution layer is used to integrate the feature information of the input feature map, and the fusion module is used to directly add the input feature map of the first convolution layer to the output feature map of the third convolution layer.

[0019] Preferably, the third convolution module to the last convolution module of the overall image feature extraction module are integrated with a seat belt adaptive convolution module;

[0020] Among them, the adaptive seat belt convolution module is integrated between the second convolution layer and the third convolution layer of the corresponding convolution module, the adaptive seat belt convolution module includes a parallel first convolution module, a second convolution module, a third convolution module and a feature splicing module, the first convolution module includes a first convolution layer and a non-square convolution layer connected in sequence, the non-square convolution layer includes two parallel fourth convolution layers and a fifth convolution layer, the convolution kernel sizes of the fourth convolution layer and the fifth convolution layer are different, the fourth convolution layer is used to capture the vertical morphological features of the seat belt, and the fifth convolution layer is used to capture the horizontal morphological features of the seat belt, the second convolution module includes a first convolution layer and a sixth convolution layer connected in sequence, the convolution kernel sizes of the sixth convolution layer and the second convolution layer are the same, the sixth convolution layer is used to capture the non-vertical horizontal morphological features of the seat belt, the third convolution module is a first convolution layer, and the feature splicing module is used to splice the vertical morphological features, horizontal morphological features and non-vertical horizontal morphological features after activation processing in the channel dimension to obtain a fusion feature map.

[0021] Preferably, the seat belt segmentation module includes an encoder and a decoder, and the seat belt segmentation module is configured to perform feature extraction on an input feature map through the encoder, and input the obtained output feature map into the decoder for feature recovery to obtain seat belt semantic information.

[0022] Preferably, the encoder is the first N layers of the backbone of a target detection model, N is an integer greater than 5, and the decoder includes a first deconvolution layer, a second deconvolution layer, a third deconvolution layer and a fourth deconvolution layer connected in sequence;

[0023] The step of extracting features from the input feature map by the encoder and inputting the obtained output feature map into the decoder for feature recovery to obtain seat belt semantic information includes:

[0024] Extracting features from the input feature map through the encoder to obtain an output feature map, and retaining the first layer output feature map, the third layer output feature map, and the fifth layer output feature map of the encoder;

[0025] Inputting the output feature map of the encoder into the first deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the fifth layer after upsampling to obtain the output feature map of the first deconvolution layer;

[0026] Inputting the output feature map of the first deconvolution layer into the second deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the third layer after upsampling to obtain the output feature map of the second deconvolution layer;

[0027] Inputting the output feature map of the second deconvolution layer into the third deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the first layer after upsampling to obtain the output feature map of the third deconvolution layer;

[0028] The output feature map of the third deconvolution layer is input into the fourth deconvolution layer for up-sampling to obtain seat belt semantic information.

[0029] Preferably, the semantic guidance module comprises a first attention mechanism layer, a multi-layer perceptron, a second attention mechanism layer and a fully connected layer connected in sequence;

[0030] Among them, the first attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt semantic information to obtain seat belt representation information, the seat belt semantic information is a query vector, and the overall image feature information is a key and a value; the multi-layer perceptron is used to adjust the dimension of the seat belt representation information; the second attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt representation information to obtain the overall image representation information, the overall image feature information is a query vector, and the seat belt representation information is a key and a value; the fully connected layer is used to perform nonlinear mapping on the overall image representation information to obtain the prediction probability of the violation item.

[0031] Preferably, the determining the violation item corresponding to the target image to be detected based on the violation item prediction probability includes:

[0032] Determine whether the predicted probability of the violation is higher than a preset threshold, and if so, determine that the target image to be detected has a corresponding violation; wherein the violation at least includes the quick-insert seat belt not being fastened, the lap seat belt not being fastened, the quick-insert seat belt being hung low and used high, the lap seat belt being hung low and used high, the quick-insert seat belt being positioned incorrectly, and the quick-insert seat belt being loose.

[0033] In a second aspect, an embodiment of the present invention provides a safety belt violation wearing detection system for high-altitude operations, comprising:

[0034] A target image acquisition unit is used to acquire the high-altitude work image to be detected, and pre-process the high-altitude work image to be detected to obtain a target image to be detected;

[0035] A seat belt violation wearing detection unit, used for inputting the target image to be detected into a target seat belt violation wearing detection model for feature extraction and semantic guidance, and obtaining a violation item prediction probability; wherein the target seat belt violation wearing detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module, and the target seat belt violation wearing detection model is configured to perform feature extraction on the target image to be detected through the overall image feature extraction module and the seat belt segmentation module, respectively obtain overall image feature information and seat belt semantic information, and input the overall image feature information and the seat belt semantic information into the semantic guidance module for semantic guidance, and obtain a violation item prediction probability;

[0036] The violation item determination unit is used to determine the violation item corresponding to the target image to be detected based on the violation item prediction probability.

[0037] Compared with the prior art, the embodiment of the present invention is a method and system for detecting illegal wearing of seat belts for aerial work, and its beneficial effects are as follows: the overall image feature extraction module can effectively capture and analyze the feature details of the seat belt while maintaining low computational complexity by integrating the adaptive seat belt convolution module; the seat belt segmentation module ensures that the output feature map integrates detailed information and global information through feature fusion; the semantic guidance module enables the model to pay more attention to the seat belt area in the image, while reducing the attention to other irrelevant information, thereby improving the accuracy and robustness of feature extraction. By performing multi-level feature extraction and semantic guidance on images in aerial work scenes, various incorrect seat belt wearing behaviors can be effectively identified and classified, thereby improving the safety monitoring level of aerial work and reducing safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of a method for detecting illegal wearing of safety belts for aerial work according to an embodiment of the present invention;

[0039] Figure 2is a schematic diagram of a process for obtaining a target image to be detected according to an embodiment of the present invention;

[0040] Figure 3 is a structural schematic diagram of a target safety belt violation wearing detection model according to an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a process of constructing a target safety belt violation wearing detection model according to an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of the structure of the first layer convolution module or the second layer convolution module in the overall image feature extraction module of an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of the structure of each convolution module from the third convolution module to the last convolution module in the overall image feature extraction module of an embodiment of the present invention;

[0044] Figure 7 is a schematic diagram of the structure of a seat belt convolution module adapted to an embodiment of the present invention;

[0045] Figure 8 is a schematic structural diagram of a seat belt segmentation module according to an embodiment of the present invention;

[0046] Fig. 9 is a schematic diagram of the structure of a semantic guidance module according to an embodiment of the present invention;

[0047] Fig.10 It is a structural schematic diagram of a safety belt violation wearing detection system for high-altitude operations according to an embodiment of the present invention;

[0048] Figure 8 In the figure, 10 is the first deconvolution layer; 20 is the second deconvolution layer; 30 is the third deconvolution layer; 40 is the fourth deconvolution layer. DETAILED DESCRIPTION

[0049] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0050] In the description of the present invention, it should be understood that the terms "first" and "second" etc. are used in the present invention to distinguish different objects rather than to describe a specific order.

[0051] like Figure 1 As shown, the embodiment of the present invention provides a method for detecting illegal wearing of seat belts for high-altitude operations, including the steps of:

[0052] S1. Acquire the high-altitude working image to be detected, and pre-process the high-altitude working image to be detected to obtain the target image to be detected;

[0053] Specifically, Figure 2 As shown, step S1 includes:

[0054] S101, extracting the region of interest from the high-altitude working image to be detected, and obtaining the high-altitude working personnel detection result;

[0055] Due to the high complexity of the aerial work scene, the aerial workers and safety belts occupy a relatively small area of ​​the aerial work image to be detected. In this embodiment, the YOLOv8 model is used to extract the region of interest of the aerial work image to be detected to obtain the aerial worker detection result.

[0056] S102, expanding and cropping the detection results of the high-altitude workers to obtain the target image to be detected.

[0057] In this embodiment, the detection results of high-altitude workers are expanded outward by 0.8 times to ensure that the safety belt area is included, and the target image to be detected is obtained after cropping, which serves as the input of the target safety belt violation wearing detection model of the present invention.

[0058] S2, inputting the target image to be detected into the target seat belt violation wearing detection model for feature extraction and semantic guidance to obtain the violation item prediction probability;

[0059] Specifically, Figure 3 As shown, the target seat belt violation wearing detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module. Further, the target seat belt violation wearing detection model is configured to perform feature extraction on the target image to be detected through the overall image feature extraction module and the seat belt segmentation module, respectively obtain overall image feature information and seat belt semantic information, and input the overall image feature information and seat belt semantic information into the semantic guidance module for semantic guidance to obtain the violation item prediction probability. It can be understood that the overall image feature extraction module is used to perform feature extraction on the target image to be detected to obtain overall image feature information; the seat belt segmentation module is used to perform feature extraction on the target image to be detected to obtain seat belt semantic information; the semantic guidance module is used to perform semantic guidance on the overall image feature information and seat belt semantic information to obtain the violation item prediction probability.

[0060] It should be noted that, before step S2, Figure 4 As shown, it also includes:

[0061] Step 1: Obtain a high-altitude work image dataset;

[0062] The aerial work image dataset obtained in this embodiment is derived from construction site pictures of power grids, and the dataset contains 1,451 high-resolution images.

[0063] Step 2: annotate the violation items in the aerial work image dataset, and divide the annotated aerial work image dataset into a training set and a test set;

[0064] In the case where multiple violations appear in a picture, the present invention uses the idea of ​​multi-label tasks to mark the violations, and the violations include at least the quick-insert safety belt not being fastened, the lap safety belt not being fastened, the quick-insert safety belt being hung low and used high, the lap safety belt being hung low and used high, the quick-insert safety belt being positioned incorrectly, and the quick-insert safety belt being loose. This embodiment uses the EISeg annotation tool for annotation, and for each violation, if it exists, it is marked as 1, otherwise it is marked as 0. After the annotation is completed, this embodiment uses 1161 images in the data set as a training set to ensure that the model can fully learn the features and patterns in various situations, and the remaining 290 images are used as a test set to verify the performance of the model to evaluate its generalization ability on unseen data.

[0065] Step 3: Using the training set, the semantic guidance module and the seat belt segmentation module are trained based on the loss function to obtain an initial seat belt violation detection model;

[0066] Specifically, refer to Figure 3 ,The initial seat belt violation wearing detection model includes the overall ,image feature extraction module, seat belt segmentation module and semantic ,guidance module.

[0067] Furthermore, the overall image feature extraction module is specifically described:

[0068] The present invention proposes an overall image feature extraction module including an adaptive seat belt convolution module, which uses the extracted overall image feature information as one of the inputs of a semantic guidance module.

[0069] The overall image feature extraction module includes a multi-layer convolution module, which can extract high-quality features while maintaining low computational complexity. Specifically, Figure 5As shown, each convolution module includes a first convolution layer, a second convolution layer, a third convolution layer and a fusion module connected in sequence. The convolution kernel size of the first convolution layer and the third convolution layer is the same and different from the convolution kernel size of the second convolution layer. The second convolution layer is a depth-separable convolution layer. The first convolution layer is used to adjust the number of channels of the input feature map, the second convolution layer is used to extract the spatial features of the input feature map, and the third convolution layer is used to integrate the feature information of the input feature map. The fusion module is used to add the input feature map of the first convolution layer directly to the output feature map of the third convolution layer to maintain the integrity of the features and enhance the gradient flow. The overall image feature extraction module of this embodiment includes 7 layers of convolution modules. Each convolution module first adjusts the number of channels of the input feature map through the first convolution layer with a convolution kernel size of 1×1, then uses the second convolution layer with a convolution kernel size of 3×3 to extract spatial features, and then uses the third convolution layer with a convolution kernel size of 1×1 to integrate feature information, and uses the fusion module to directly add the input feature map of the first convolution layer to the output feature map of the third convolution layer using a jump connection.

[0070] Since seat belts often appear to be long and thin in images, conventional feature extraction networks often find it difficult to effectively capture and parse their feature details. In order to solve this problem, the present invention proposes an adaptive seat belt convolution module, and integrates it in the third to last convolution module of the overall image feature extraction module, so as to more effectively extract seat belt features in the image. It should be noted that since the asymmetric convolution kernel is not suitable for being placed at the beginning of the network, the adaptive seat belt convolution module is not integrated in the first two layers of convolution modules, but is integrated in the third to last layers of convolution modules. For this embodiment, there is no adaptive seat belt convolution module in the first two layers of convolution modules, and there is an adaptive seat belt convolution module in the last five layers of convolution modules.

[0071] Specifically, Figure 6 As shown, the adaptive seat belt convolution module is integrated between the second convolution layer and the third convolution layer of the corresponding convolution module. It can be understood that Figure 5 In fact, it is a structural diagram of the first layer convolution module or the second layer convolution module in the overall image feature extraction module. Figure 6 In fact, it is a structural diagram of each convolution module from the third convolution module to the last convolution module in the overall image feature extraction module.

[0072] like Figure 7As shown, the adaptive seat belt convolution module includes a parallel first convolution module, a second convolution module, a third convolution module and a feature splicing module. Among them, the first convolution module includes a first convolution layer and a non-square convolution layer connected in sequence, and the non-square convolution layer includes two parallel fourth convolution layers and fifth convolution layers. The convolution kernel sizes of the fourth convolution layer and the fifth convolution layer are different. The fourth convolution layer is used to capture the vertical morphological features of the seat belt, and the fifth convolution layer is used to capture the horizontal morphological features of the seat belt. The second convolution module includes a first convolution layer and a sixth convolution layer connected in sequence. The convolution kernel sizes of the sixth convolution layer and the second convolution layer are the same, and the sixth convolution layer is used to capture the non-vertical horizontal morphological features of the seat belt. The third convolution module is a first convolution layer, and the feature splicing module is used to splice the vertical morphological features, horizontal morphological features and non-vertical horizontal morphological features after activation processing in the channel dimension to obtain a fused feature map. Specifically, this embodiment captures the slender features of the seat belt in the vertical and horizontal directions by applying the fourth convolution layer with a 2×5 convolution kernel size and the fifth convolution layer with a 5×2 convolution kernel size in parallel, and also uses the sixth convolution layer with a 3×3 convolution kernel size to supplement the feature information. The step size of all convolution kernel sizes is 1, and after being processed by the ReLU activation function, they are spliced ​​in the channel dimension to generate a fused feature map. Compared with the conventional feature extraction network, this method improves the accuracy of the model by 3%.

[0073] Furthermore, the seat belt segmentation module is specifically described:

[0074] The seat belt semantic information obtained by the seat belt segmentation module is used as another input of the semantic guidance module. Figure 8 As shown, the seat belt segmentation module includes an encoder and a decoder. The seat belt segmentation module is configured to extract features from an input feature map through the encoder, and input the obtained output feature map into the decoder for feature recovery to obtain seat belt semantic information.

[0075] Specifically, the encoder is the first N layers of the backbone of an object detection model, where N is an integer greater than 5, and the decoder includes a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, and a fourth deconvolution layer connected in sequence. Figure 8 ,The workflow of the seat belt segmentation module is as follows:

[0076] 1) Extract features from the input feature map through the encoder to obtain the output feature map, and retain the first layer output feature map, the third layer output feature map, and the fifth layer output feature map of the encoder;

[0077] 2) Input the output feature map of the encoder into the first deconvolution layer for feature recovery, and perform feature fusion on the restored output feature map and the upsampled output feature map of the fifth layer to obtain the output feature map of the first deconvolution layer;

[0078] 3) The output feature map of the first deconvolution layer is input into the second deconvolution layer for feature recovery, and the restored output feature map and the upsampled output feature map of the third layer are fused to obtain the output feature map of the second deconvolution layer;

[0079] 4) The output feature map of the second deconvolution layer is input into the third deconvolution layer for feature recovery, and the restored output feature map and the upsampled output feature map of the first layer are fused to obtain the output feature map of the third deconvolution layer;

[0080] 5) The output feature map of the third deconvolution layer is input into the fourth deconvolution layer for upsampling to obtain the seat belt semantic information.

[0081] In this embodiment, the first 7 layers of the backbone of the YOLOv8 model are used as encoders, and the output feature map size of the encoder is 40×40. The decoder contains 4 upsamplings, and high-quality feature recovery and image reconstruction are achieved through multi-layer deconvolution and multi-scale feature fusion. First, the spatial resolution of the feature map is gradually restored using four deconvolution layers; at the same time, the output feature maps of the first, third, and fifth layers in the encoder are retained, and these feature maps and the output feature maps of the deconvolution layer are fused to the same scale by upsampling the output feature maps of these three different resolutions, ensuring that the output feature maps integrate detail information and global information; finally, the output feature map of the third deconvolution layer is upsampled to the same size as the input feature map of the encoder to obtain seat belt semantic information.

[0082] Furthermore, the semantic guidance module is specifically described:

[0083] The semantic guidance module utilizes the seat belt semantic information output by the seat belt segmentation module and the overall image feature information output by the overall image feature extraction module, enabling the model to pay more attention to the seat belt area in the image while reducing the attention to other irrelevant information, thereby improving the accuracy and robustness of feature extraction.

[0084] like Fig. 9 As shown, the semantic guidance module includes a first attention mechanism layer, a multi-layer perceptron, a second attention mechanism layer and a fully connected layer connected in sequence.

[0085] Specifically, the first attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt semantic information to obtain the seat belt representation information. First, the seat belt semantic information is used as the query vector, and then the overall image feature information is used as the key and value to perform attention fusion, thereby updating and enriching the seat belt representation information. This process ensures that the seat belt can effectively capture the information related to it in the overall image through the attention weight. This embodiment uses the following formula to obtain the seat belt representation information:

[0086]

[0087] in, Indicates seat belt identification information. Represents the semantic information of seat belts, Represents the overall image feature information, , and represents the weight matrix, represents the attention scaling factor.

[0088] The multi-layer perceptron is used to adjust the dimension of the seat belt representation information. In this embodiment, a two-layer multi-layer perceptron is used to perform feature transformation on the seat belt representation information to adjust the dimension for subsequent calculation.

[0089] The second attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt representation information to obtain the overall image representation information. First, the overall image feature information is used as the query vector, and then the seat belt representation information is used as the key and value to perform attention fusion to ensure that the overall image can focus on the important information in the seat belt more accurately. This embodiment uses the following formula to obtain the overall image representation information:

[0090]

[0091] in, Represents the overall image representation information.

[0092] The fully connected layer is used to perform nonlinear mapping on the overall image representation information to obtain the prediction probability of the violation item. The number of neurons in the fully connected layer of this embodiment is 6, and 6 eigenvalues ​​corresponding to the overall image representation information can be obtained. Each eigenvalue is mapped to a value of 0~1 through the Sigmoid function, and the prediction probability of 6 violation items can be obtained.

[0093] The following is a detailed description of the model's loss function:

[0094] The loss function is the weighted sum of the violation classification loss and the seat belt segmentation loss. The violation classification loss includes the binary cross entropy loss, and the seat belt segmentation loss includes the Dice loss.

[0095] Specifically, this embodiment uses the following formula to express the classification loss of the violation item:

[0096]

[0097] Among them, BCELoss represents the classification loss of violation items, N represents the number of samples, represents the loss of the nth sample, represents the true label of the nth sample, represents the predicted value of the nth sample, Represents the Sigmoid function.

[0098] Furthermore, this embodiment uses the following formula to express the seat belt segmentation loss:

[0099]

[0100] Among them, DiceLoss represents the seat belt segmentation loss, represents the true label of the i-th sample, Represents the predicted value of the i-th sample.

[0101] Finally, this embodiment uses the following formula to express the loss function of the model:

[0102]

[0103] Among them, Loss represents the loss function, and Indicates the weight coefficient. Set to 0.6, Set to 0.4.

[0104] The following is a detailed description of the model training process:

[0105] In this embodiment, the number of iterative training times is 200 times, the basic learning rate is 0.001, the learning rate decreases by 10 times every 20 times, the model is saved once after each iteration, and the model of the last iteration is selected as the initial seat belt violation wearing detection model of the present invention.

[0106] Step 4: Use the test set to verify the initial seat belt violation detection model to obtain the target seat belt violation detection model.

[0107] The initial seat belt violation detection model is optimized through the test set to obtain the target seat belt violation detection model, thereby improving the generalization ability of the model on unseen data.

[0108] S3. Determine the violation item corresponding to the target image to be detected based on the predicted probability of the violation item.

[0109] Specifically, determine whether the predicted probability of the violation is higher than the preset threshold. If so, determine that the target image to be detected has the corresponding violation. Among them, the violation items at least include the quick-insertion seat belt not being fastened, the waist seat belt not being fastened, the quick-insertion seat belt being hung low and used high, the waist seat belt being hung low and used high, the quick-insertion seat belt being positioned incorrectly, and the quick-insertion seat belt being loose. The preset threshold of this embodiment is 0.5. Of course, the preset threshold of this embodiment can be adaptively adjusted according to the detection requirements of the illegal wearing of the seat belt. If the detection requirements are high, a smaller preset threshold can be set. Conversely, the preset threshold can be increased accordingly.

[0110] The embodiment of the present invention is a method for detecting illegal wearing of seat belts for aerial work. The overall image feature extraction module can effectively capture and analyze the feature details of the seat belt while maintaining low computational complexity by integrating the adaptive seat belt convolution module; the seat belt segmentation module ensures that the output feature map integrates detailed information and global information through feature fusion; the semantic guidance module enables the model to pay more attention to the seat belt area in the image, while reducing the attention to other irrelevant information, thereby improving the accuracy and robustness of feature extraction. By performing multi-level feature extraction and semantic guidance on images in aerial work scenes, various incorrect seat belt wearing behaviors can be effectively identified and classified, thereby improving the safety monitoring level of aerial work and reducing safety hazards.

[0111] Based on the above seat belt violation detection method, Fig.10 As shown, an embodiment of the present invention provides a safety belt violation wearing detection system for high-altitude operations, comprising:

[0112] The target image acquisition unit 1 is used to acquire the high-altitude working image to be detected, and pre-process the high-altitude working image to be detected to obtain the target image to be detected;

[0113] The seat belt violation detection unit 2 is used to input the target image to be detected into the target seat belt violation detection model for feature extraction and semantic guidance to obtain the violation item prediction probability; wherein the target seat belt violation detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module, and the target seat belt violation detection model is configured to extract features of the target image to be detected through the overall image feature extraction module and the seat belt segmentation module to obtain overall image feature information and seat belt semantic information respectively, and input the overall image feature information and the seat belt semantic information into the semantic guidance module for semantic guidance to obtain the violation item prediction probability;

[0114] The violation item determination unit 3 is used to determine the violation item corresponding to the target image to be detected based on the violation item prediction probability.

[0115] It should be noted that each unit in the above-mentioned safety belt violation detection system for high-altitude operations can be fully or partially implemented by software, hardware and their combination. The above-mentioned units can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules. For the specific definition of a safety belt violation detection system for high-altitude operations, please refer to the definition of a safety belt violation detection method for high-altitude operations above. The two have the same functions and effects, which will not be repeated here.

[0116] In summary, the embodiment of the present invention is a method and system for detecting illegal wearing of seat belts for aerial work. The overall image feature extraction module can effectively capture and analyze the feature details of the seat belt while maintaining low computational complexity by integrating the adaptive seat belt convolution module; the seat belt segmentation module ensures that the output feature map integrates detailed information and global information through feature fusion; the semantic guidance module enables the model to pay more attention to the seat belt area in the image, while reducing the attention to other irrelevant information, thereby improving the accuracy and robustness of feature extraction. By performing multi-level feature extraction and semantic guidance on images in aerial work scenes, various incorrect seat belt wearing behaviors can be effectively identified and classified, thereby improving the safety monitoring level of aerial work and reducing safety hazards.

[0117] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting illegal wearing of safety belts for high-altitude operations, characterized in that: include: Acquire the high-altitude work image to be detected, and pre-process the high-altitude work image to be detected to obtain the target image to be detected; Input the target image to be detected into the target seat belt violation wearing detection model for feature extraction and semantic guidance to obtain the violation item prediction probability; wherein the target seat belt violation wearing detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module, and the target seat belt violation wearing detection model is configured to perform feature extraction on the target image to be detected through the overall image feature extraction module and the seat belt segmentation module to obtain overall image feature information and seat belt semantic information respectively, and input the overall image feature information and the seat belt semantic information into the semantic guidance module for semantic guidance to obtain the violation item prediction probability; Determining a violation item corresponding to the target image to be detected based on the predicted probability of the violation item; The overall image feature extraction module includes a multi-layer convolution module; wherein each layer of the convolution module includes a first convolution layer, a second convolution layer, a third convolution layer and a fusion module connected in sequence, the convolution kernel size of the first convolution layer and the third convolution layer is the same and different from the convolution kernel size of the second convolution layer, the second convolution layer is a depth-separable convolution layer, the first convolution layer is used to adjust the number of channels of the input feature map, the second convolution layer is used to extract the spatial features of the input feature map, the third convolution layer is used to integrate the feature information of the input feature map, and the fusion module is used to directly add the input feature map of the first convolution layer to the output feature map of the third convolution layer; The third convolution module to the last convolution module of the overall image feature extraction module are integrated with a seat belt adaptation convolution module; Wherein, the adaptive seat belt convolution module is integrated between the second convolution layer and the third convolution layer of the corresponding convolution module, the adaptive seat belt convolution module includes a parallel first convolution module, a second convolution module, a third convolution module and a feature splicing module, the first convolution module includes a first convolution layer and a non-square convolution layer connected in sequence, the non-square convolution layer includes two parallel fourth convolution layers and a fifth convolution layer, the convolution kernel sizes of the fourth convolution layer and the fifth convolution layer are different, the fourth convolution layer is used to capture the vertical morphological features of the seat belt, the fifth convolution layer is used to capture the horizontal morphological features of the seat belt, the second convolution module includes a first convolution layer and a sixth convolution layer connected in sequence, the convolution kernel sizes of the sixth convolution layer and the second convolution layer are the same, the sixth convolution layer is used to capture the non-vertical horizontal morphological features of the seat belt, the third convolution module is a first convolution layer, and the feature splicing module is used to splice the vertical morphological features, horizontal morphological features and non-vertical horizontal morphological features after activation processing in the channel dimension to obtain a fusion feature map; The seat belt segmentation module includes an encoder and a decoder, and the seat belt segmentation module is configured to extract features from an input feature map through the encoder, and input the obtained output feature map into the decoder for feature recovery to obtain seat belt semantic information; The encoder is the first N layers of the backbone of a target detection model, where N is an integer greater than 5, and the decoder includes a first deconvolution layer, a second deconvolution layer, a third deconvolution layer, and a fourth deconvolution layer connected in sequence; The step of extracting features from the input feature map by the encoder and inputting the obtained output feature map into the decoder for feature recovery to obtain seat belt semantic information includes: Extracting features from the input feature map through the encoder to obtain an output feature map, and retaining the first layer output feature map, the third layer output feature map, and the fifth layer output feature map of the encoder; Inputting the output feature map of the encoder into the first deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the fifth layer after upsampling to obtain the output feature map of the first deconvolution layer; Inputting the output feature map of the first deconvolution layer into the second deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the third layer after upsampling to obtain the output feature map of the second deconvolution layer; Inputting the output feature map of the second deconvolution layer into the third deconvolution layer for feature recovery, and performing feature fusion on the output feature map after feature recovery and the output feature map of the first layer after upsampling to obtain the output feature map of the third deconvolution layer; Inputting the output feature map of the third deconvolution layer into the fourth deconvolution layer for upsampling to obtain seat belt semantic information; The semantic guidance module includes a first attention mechanism layer, a multi-layer perceptron, a second attention mechanism layer and a fully connected layer connected in sequence; Among them, the first attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt semantic information to obtain seat belt representation information, the seat belt semantic information is a query vector, and the overall image feature information is a key and a value; the multi-layer perceptron is used to adjust the dimension of the seat belt representation information; the second attention mechanism layer is used to perform attention fusion based on the overall image feature information and the seat belt representation information to obtain the overall image representation information, the overall image feature information is a query vector, and the seat belt representation information is a key and a value; the fully connected layer is used to perform nonlinear mapping on the overall image representation information to obtain the prediction probability of the violation item.

2. The seat belt violation detection method according to claim 1, characterized in that: The preprocessing of the high-altitude work image to be detected to obtain the target image to be detected includes: Extracting the region of interest from the high-altitude work image to be detected to obtain a high-altitude work personnel detection result; The detection result of the high-altitude workers is expanded and cropped to obtain a target image to be detected.

3. The seat belt violation detection method according to claim 1, characterized in that: Before inputting the target image to be detected into the target seat belt violation wearing detection model for feature extraction and semantic guidance to obtain the violation item prediction probability, the method further includes: Obtain a data set of aerial work images; Labeling the high-altitude work image dataset for violation items, and dividing the labeled high-altitude work image dataset into a training set and a test set; Using the training set, the semantic guidance module and the seat belt segmentation module are trained based on the loss function to obtain an initial seat belt violation wearing detection model; The initial seat belt violation detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module; the loss function is a weighted sum of the violation classification loss and the seat belt segmentation loss, the violation classification loss includes a binary cross entropy loss, and the seat belt segmentation loss includes a Dice loss; The test set is used to verify the initial seat belt violation wearing detection model to obtain a target seat belt violation wearing detection model.

4. The seat belt violation detection method according to claim 1, characterized in that: The determining, based on the predicted probability of the violation item, the violation item corresponding to the target image to be detected includes: Determine whether the predicted probability of the violation is higher than a preset threshold, and if so, determine that the target image to be detected has a corresponding violation; wherein the violation at least includes the quick-insert seat belt not being fastened, the lap seat belt not being fastened, the quick-insert seat belt being hung low and used high, the lap seat belt being hung low and used high, the quick-insert seat belt being positioned incorrectly, and the quick-insert seat belt being loose.

5. A safety belt violation detection system for high-altitude operations, the safety belt violation detection system is applied to the safety belt violation detection method according to any one of claims 1 to 4, characterized in that: include: A target image acquisition unit is used to acquire the high-altitude work image to be detected, and pre-process the high-altitude work image to be detected to obtain a target image to be detected; A seat belt violation wearing detection unit, used for inputting the target image to be detected into a target seat belt violation wearing detection model for feature extraction and semantic guidance, and obtaining a violation item prediction probability; wherein the target seat belt violation wearing detection model includes an overall image feature extraction module, a seat belt segmentation module and a semantic guidance module, and the target seat belt violation wearing detection model is configured to perform feature extraction on the target image to be detected through the overall image feature extraction module and the seat belt segmentation module, respectively obtain overall image feature information and seat belt semantic information, and input the overall image feature information and the seat belt semantic information into the semantic guidance module for semantic guidance, and obtain a violation item prediction probability; The violation item determination unit is used to determine the violation item corresponding to the target image to be detected based on the violation item prediction probability.

Images