Construction environment safety belt identification method based on multi-scale fusion

Abstract

The invention discloses a construction environment safety belt identification method based on multi-scale fusion, and the method comprises the following steps: S100, constructing an improved yolov8 safety belt identification model based on IAFPN, namely introducing an AIFI mixed feature encoder and a BiFusion module on a high-level feature map for calculation; s200, deploying a reasoning strategy based on an SAHI algorithm, a t-iou matching strategy and a double-camera linkage strategy; s300, collecting and training data, and comparing and judging the tracking target; the method comprises the following steps: improving a yolov8 model through an IAFPN, fusing a Transform layer, optimizing an up-sampling structure and a feature fusion structure, increasing the feature fusion capability of the model for a small target on the basis of reducing the parameter quantity, and improving the recognition precision; based on an SAHI algorithm, a t-iou matching strategy, a safety belt judgment strategy and a double-camera linkage strategy, slice reasoning is carried out on the image, and the judgment accuracy of an angle shielding image target image is improved; and a feature data set is constructed through a large amount of data acquisition and training, so that the overall discrimination accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for recognizing safety belts in a construction environment based on multi-scale fusion. Background Art

[0002] In high-altitude work scenarios in the petroleum and petrochemical industries, workers are required to wear safety belts and hang the safety hooks attached to the safety belts at a position higher than the safety belts to ensure the safety of the work process.

[0003] To achieve this goal, current technologies mainly rely on 2D object detection algorithms to detect seat belts. However, in actual oilfield engineering application scenarios, there are three complex challenges in the detection of seat belts and hooks that have not been solved by current technologies:

[0004] ① Difficulty in detecting small targets: Due to the large number of high-altitude work points and the need to take into account multiple intelligent tasks, surveillance cameras are usually located at a relatively far distance, resulting in smaller targets in the picture. The human body and safety belts only occupy less than one-tenth of the picture, and the safety hook is a pixel-level target, which is extremely difficult to identify.

[0005] ②Occlusion problem: In the high-altitude operation scenes of petroleum and petrochemical industries, the safety hook is often blocked by railings, walls and winding safety ropes of similar colors, which increases the difficulty of identification. At the same time, the movement of personnel during the operation will also cause the human body and safety belts to be frequently blocked. How to accurately identify the target and correctly match the safety hook and safety belt of the same person under severe occlusion is also a major challenge.

[0006] ③ Data scarcity: Petroleum and petrochemical construction scenes are proprietary industrial scenes, and there is a lack of dedicated high-altitude work safety belt and safety hook data sets. Although there are some similar data and algorithms, due to the large differences in the environment, background and target characteristics of different high-altitude work scenes, the existing data and algorithms are difficult to directly apply to large-scale training for this task.

[0007] Therefore, it is necessary to design a safety belt recognition method for construction environments based on multi-scale fusion. Summary of the invention

[0008] The purpose of the present invention is to provide a construction environment safety belt identification method based on multi-scale fusion to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] The construction environment safety belt recognition method based on multi-scale fusion includes the following steps:

[0011] S100, build an improved yolov8 seat belt recognition model based on IAFPN, that is, introduce AIFI hybrid feature encoder and BiFusion module for calculation on high-level feature map;

[0012] S200, deployment reasoning strategy based on SAHI algorithm, t-iou matching strategy and dual-camera linkage strategy;

[0013] S300: data collection and training, comparing and judging the tracking target.

[0014] According to the above technical solution, the specific calculation method of the AIFI hybrid feature encoder in S100 is as follows:

[0015] S111, generating 2D sine-cosine position embeddings for the input data to capture the spatial relationships in the image;

[0016] S112, using 1x1 convolution and flattening operations to process context information in the image data and perform modeling;

[0017] S113, feature interaction and fusion through the Transformer layer, that is, by using a single-scale Transformer encoder to perform intra-scale interaction only on the S5 feature layer.

[0018] According to the above technical solution, the specific calculation method of the BiFusion module in S100 is as follows:

[0019] S121, use 1x1 convolution to reduce the dimension of feature maps of the same scale;

[0020] S122, first use 1x1 convolution to reduce the dimension of the large-scale feature map, then use 3x3 convolution with a step size of 2 for down-sampling, and use 2x2 transposed convolution to up-sample the small-scale feature map;

[0021] S123. Concatenate the feature maps obtained from these three parts and use 1x1 convolution to reduce the dimension again.

[0022] According to the above technical solution, the specific algorithm of the SAHI algorithm in S200 is as follows:

[0023] S201, perform data augmentation by extracting image patches from the images of the fine-tuning dataset, each image IF 1 , IF 2 , ...IF j Divided into overlapping image blocks PF 1 , PF 2 , ...PF k , whose sizes M and N are in the predefined range [M min, M max ] and [N min , N max ], the above range is a hyperparameter;

[0024] S202: During the fine-tuning process, the image block is resized while maintaining the aspect ratio so that the image width is between 800 and 1333 pixels, thereby obtaining enhanced images I1, I 2 ,...I j And the original image IF 1 , IF 2 , ...IF j ,Both are used together in the fine-tuning process, where the original image is used to assist the detection of large objects;

[0025] S203, dividing the original query image I into l M*N overlapping image blocks And resize each image block while keeping the aspect ratio unchanged;

[0026] S204, performing forward propagation of target detection on each overlapping image block respectively. If a larger target is detected in a local image block, no local processing is performed, but full-image reasoning is performed on the entire original image to ensure the integrity and accuracy of the detection;

[0027] S205: The prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size through non-maximum suppression. In the non-maximum suppression process, the prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size. m Match the boxes with the intersection-over-union ratio and remove those whose detection probability is lower than the threshold T d The test results.

[0028] According to the above technical solution, the t-iou matching strategy and the dual-camera linkage strategy in S200 are specifically as follows:

[0029] The t-iou matching strategy matches the safety belt and the human body, that is, the intersection-and-union ratio of each safety belt and the human body target is calculated in turn, and the pair with the highest intersection-and-union ratio is the matching pair;

[0030] Dual-camera linkage strategy, through the cooperation of secondary camera and main camera.

[0031] According to the above technical solution, the data collection and training of S300 are specifically as follows:

[0032] The data collection is images under different construction environments, different lighting conditions, different shooting angles, and different equipment and tools, and the images contain safety belts, safety ropes, hooks, and people;

[0033] The data training is to call the Transformer layer to perform feature interaction and fusion training on the image.

[0034] According to the above technical solution, the specific method of comparing and judging the tracking target in S300 is:

[0035] The T-IOU seat belt judgment strategy based on the target tracking algorithm makes judgments based on the overlap ratio between the seat belt target and the whole, and adds a tracking algorithm to reduce the impact of visual occlusion.

[0036] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0037] 1) Improve the yolov8 model through IAFPN, integrate the Transformer layer, optimize the upsampling structure and feature fusion structure, increase the model's feature fusion ability for small targets on the basis of reducing the number of parameters, and improve recognition accuracy;

[0038] 2) Based on the SAHI algorithm, t-iou matching strategy, seat belt judgment strategy, and dual-camera linkage strategy, the image is sliced ​​and inferred to improve the accuracy of the recognition of the target image of the angle-occluded image;

[0039] 3) Through large amounts of data collection and training, we build feature data sets to improve the overall accuracy of discrimination. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of the present invention;

[0041] Figure 2 It is a flowchart of the AIFI hybrid feature encoder of the present invention;

[0042] Figure 3 It is the structure diagram of the BiFusion module process of the present invention;

[0043] Figure 4 It is a SAHI algorithm flow structure diagram of the present invention;

[0044] Figure 5 This is a flowchart of the dual-camera linkage strategy execution process of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] like Figure 1-5 As shown, the present invention provides a technical solution: a construction environment safety belt identification method based on multi-scale fusion, comprising the following steps:

[0047] S100, build an improved yolov8 seat belt recognition model based on IAFPN, that is, introduce AIFI hybrid feature encoder and BiFusion module for calculation on high-level feature map;

[0048] S200, deployment reasoning strategy based on SAHI algorithm, t-iou matching strategy and dual-camera linkage strategy;

[0049] S300: data collection and training, comparing and judging the tracking target.

[0050] Specifically, the specific calculation method of the AIFI hybrid feature encoder in S100 is as follows:

[0051] S111, generating 2D sine-cosine position embeddings for the input data to capture the spatial relationships in the image;

[0052] S112, using 1x1 convolution and flattening operations to process context information in the image data and perform modeling;

[0053] S113, feature interaction and fusion through the Transformer layer, that is, by using a single-scale Transformer encoder to perform intra-scale interaction only on the S5 feature layer.

[0054] Specifically, the specific calculation method of the BiFusion module in S100 is as follows:

[0055] S121, use 1x1 convolution to reduce the dimension of feature maps of the same scale;

[0056] S122, first use 1x1 convolution to reduce the dimension of the large-scale feature map, then use 3x3 convolution with a step size of 2 for down-sampling, and use 2x2 transposed convolution to up-sample the small-scale feature map;

[0057] S123. Concatenate the feature maps obtained from these three parts and use 1x1 convolution to reduce the dimension again.

[0058] Specifically, the specific algorithm of the SAHI algorithm in S200 is as follows:

[0059] S201, perform data augmentation by extracting image patches from the images of the fine-tuning dataset, each image IF 1 , IF 2 , ...IF j Divided into overlapping image blocks PF 1, PF 2 , ...PF k , whose sizes M and N are in the predefined range [M min , M max ] and [N min , N max ], the above range is a hyperparameter;

[0060] S202: During the fine-tuning process, the image block is resized while maintaining the aspect ratio so that the image width is between 800 and 1333 pixels, thereby obtaining an enhanced image I 1 , I 2 ,...I j And the original image IF 1 , IF 2 , ...IF j ,Both are used together in the fine-tuning process, where the original image is used to assist the detection of large objects;

[0061] S203, dividing the original query image I into l M*N overlapping image blocks And resize each image block while keeping the aspect ratio unchanged;

[0062] S204, performing forward propagation of target detection on each overlapping image block respectively. If a larger target is detected in a local image block, no local processing is performed, but full-image reasoning is performed on the entire original image to ensure the integrity and accuracy of the detection;

[0063] S205: The prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size through non-maximum suppression. In the non-maximum suppression process, the prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size. m Match the boxes with the intersection-over-union ratio and remove those whose detection probability is lower than the threshold T d The test results.

[0064] Specifically, the t-iou matching strategy and the dual-camera linkage strategy in S200 are as follows:

[0065] The ti ou matching strategy matches the safety belt and the human body, that is, the intersection and union ratio of each safety belt and the human body target is calculated in turn, and the pair with the highest intersection and union ratio is the matching pair;

[0066] Dual-camera linkage strategy, through the cooperation of secondary camera and main camera.

[0067] Specifically, the data collection and training of S300 are as follows:

[0068] The data collection is images under different construction environments, different lighting conditions, different shooting angles, and different equipment and tools, and the images contain safety belts, safety ropes, hooks, and people;

[0069] The data training is to call the Transformer layer to perform feature interaction and fusion training on the image.

[0070] Specifically, the specific method of comparing and judging the tracking target in S300 is as follows:

[0071] The T-IOU seat belt judgment strategy based on the target tracking algorithm makes judgments based on the overlap ratio between the seat belt target and the whole, and adds a tracking algorithm to reduce the impact of visual occlusion.

[0072] The overall method was verified and tested based on the existing mainstream target detection algorithms. The comparison of the effects of each algorithm is shown in Table 1:

[0073]

[0074] Table 1 Performance comparison

[0075] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0076] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A construction environment safety belt recognition method based on multi-scale fusion, characterized in that: The steps include: S100, build an improved yolov8 seat belt recognition model based on IAFPN, that is, introduce AIFI hybrid feature encoder and BiFusion module for calculation on high-level feature map; S200, deployment reasoning strategy based on SAHI algorithm, t-iou matching strategy and dual-camera linkage strategy; S300: data collection and training, comparing and judging the tracking target.

2. The construction environment safety belt identification method based on multi-scale fusion according to claim 1 is characterized in that: The specific calculation method of the AIFI hybrid feature encoder in S100 is as follows: S111, generating 2D sine-cosine position embeddings for the input data to capture the spatial relationships in the image; S112, using 1x1 convolution and flattening operations to process context information in the image data and perform modeling; S113, feature interaction and fusion through the Transformer layer, that is, by using a single-scale Transformer encoder to perform intra-scale interaction only on the S5 feature layer.

3. The construction environment safety belt identification method based on multi-scale fusion according to claim 1 is characterized in that: The specific calculation method of the BiFusion module in the S100 is as follows: S121, use 1x1 convolution to reduce the dimension of feature maps of the same scale; S122, first use 1x1 convolution to reduce the dimension of the large-scale feature map, then use 3x3 convolution with a step size of 2 for down-sampling, and use 2x2 transposed convolution to up-sample the small-scale feature map; S123. Concatenate the feature maps obtained from these three parts and use 1x1 convolution to reduce the dimension again.

4. The construction environment safety belt identification method based on multi-scale fusion according to claim 1 is characterized in that: The specific algorithm of the SAHI algorithm in S200 is as follows: S201, perform data augmentation by extracting image patches from the images of the fine-tuning dataset, each image IF 1 , IF 2 , ...IF j Divided into overlapping image blocks PF 1 , PF 2 , ...PF k , whose dimensions M and N are in the predefined range [M min , M max ] and [N min , N max ], the above range is a hyperparameter; S202: During the fine-tuning process, the image block is resized while maintaining the aspect ratio so that the image width is between 800 and 1333 pixels, thereby obtaining an enhanced image I 1 , I 2 ,...I j And the original image IF 1 , IF 2 , ...IF j ,Both are used together in the fine-tuning process, where the original image is used to assist the detection of large objects; S203, dividing the original query image I into l M*N overlapping image blocks And resize each image block while keeping the aspect ratio unchanged; S204, performing forward propagation of target detection on each overlapping image block respectively. If a larger target is detected in a local image block, no local processing is performed, but full-image reasoning is performed on the entire original image to ensure the integrity and accuracy of the detection; S205: The prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size through non-maximum suppression. In the non-maximum suppression process, the prediction results of all overlapping image blocks and the results of full-image reasoning are merged back to the original size. m Match the boxes with the intersection-over-union ratio and remove those whose detection probability is lower than the threshold T d The test results.

5. The construction environment safety belt identification method based on multi-scale fusion according to claim 1 is characterized in that: The t-iou matching strategy and dual-camera linkage strategy in S200 are specifically as follows: The t-iou matching strategy matches the safety belt and the human body, that is, the intersection-and-union ratio of each safety belt and the human body target is calculated in turn, and the pair with the highest intersection-and-union ratio is the matching pair; Dual-camera linkage strategy, through the cooperation of secondary camera and main camera.

6. The construction environment safety belt identification method based on multi-scale fusion according to claim 1 is characterized in that: The data collection and training of S300 are specifically as follows: The data collection is images under different construction environments, different lighting conditions, different shooting angles, and different equipment and tools, and the images contain safety belts, safety ropes, hooks, and people; The data training is to call the Transformer layer to perform feature interaction and fusion training on the image.

7. The method for identifying safety belts in construction environments based on multi-scale fusion according to claim 1, characterized in that: The specific method of comparing and judging the tracking target in S300 is as follows: The T-IOU seat belt judgment strategy based on the target tracking algorithm makes judgments based on the overlap ratio between the seat belt target and the whole, and adds a tracking algorithm to reduce the impact of visual occlusion.

Images