A helmet detection system and method based on transformer

Abstract

The present invention provides a transformer-based helmet detection system and method, wherein the system includes: a model training module, which is used to train a transformer model based on transfer learning and utilize a helmet detection data set to obtain a helmet detection model; an image acquisition module, which is used to acquire an image to be detected of a construction scene; and a helmet detection module, which is used to perform helmet detection on the image to be detected based on the helmet detection model. The transformer-based helmet detection system and method of the present invention greatly improves the accuracy of helmet detection.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a transformer-based helmet detection system and method. Background Art

[0002] In the field of vision-based object detection, convolutional neural networks are mainly used for feature extraction. For example, VGG extracts richer features by deepening the network, ResNet designs residual networks to solve the problem of gradient disappearance, and PPM provides multi-scale context information. Although these networks have achieved good results, due to the limited receptive field of convolutional neural networks, only local features can be extracted. In the target detection model, the use of convolutional neural networks as the backbone network still has certain defects, which reduces the accuracy of helmet detection. The self-attention module in the Transformer structure can effectively extract global features and has obvious advantages over convolutional neural networks. Summary of the invention

[0003] One of the purposes of the present invention is to provide a transformer-based helmet detection system to improve the accuracy of helmet detection.

[0004] An embodiment of the present invention provides a transformer-based helmet detection system, comprising:

[0005] The model training module is used to train the transformer model based on transfer learning and use the helmet detection dataset to obtain the helmet detection model;

[0006] An image acquisition module, used to acquire the image to be detected of the construction scene;

[0007] The helmet detection module is used to perform helmet detection on the image to be detected based on the helmet detection model.

[0008] Preferably, the transformer model includes:

[0009] Pixel Block Embedding Module:

[0010] Define the input as W, H, and C represent the width, height, and number of channels of the input;

[0011] Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used;

[0012] O 11 Flatten the first two dimensions to get the output in

[0013] O 12 Normalize to get O1;

[0014] Multi-scale channel attention structure for processing O1:

[0015] Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m};

[0016] For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation;

[0017] Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′;

[0018] Dual-path gated attention structure for processing O2:

[0019] Use 1×1 convolution to double the channel dimension of O2, and get O 31 ;

[0020] For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ;

[0021] Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function;

[0022] Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2;

[0023] Backbone network:

[0024] Assume the input is x,

[0025] Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5};

[0026] x is processed by M1 to get out1;

[0027] out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2;

[0028] out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3;

[0029] out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4;

[0030] out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

[0031] Preferably, the image acquisition module acquires the image to be detected of the construction scene, including:

[0032] Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons;

[0033] Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments;

[0034] Acquire multimodal information of multiple construction workers in the construction scene;

[0035] Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future;

[0036] Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence;

[0037] Based on the shooting parameters, a mobile shooting device in the construction scene is controlled and scheduled to shoot the target path segment to obtain the image to be detected.

[0038] Preferably, the path segmentation condition includes one or more of the following combinations:

[0039] The path length of each path segment falls within the standard length interval;

[0040] The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein 2≤N≤K, K is the first quantity threshold;

[0041] The total number of path segments in the same sub-scene in the construction scene is less than or equal to a second number threshold.

[0042] Preferably, the shooting parameter determination conditions include:

[0043] The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters;

[0044] There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment;

[0045] The mobile shooting device shoots the target path segment without obstruction based on the shooting angle of view when shooting based on the shooting parameters.

[0046] The embodiment of the present invention provides a transformer-based helmet detection method, comprising:

[0047] Based on transfer learning, the helmet detection dataset is used to train the transformer model to obtain the helmet detection model;

[0048] Acquire the image to be detected of the construction scene;

[0049] Based on the helmet detection model, helmet detection is performed on the image to be detected.

[0050] Preferably, the transformer model includes:

[0051] Pixel Block Embedding Module:

[0052] Define the input as W, H, and C represent the width, height, and number of channels of the input;

[0053] Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used;

[0054] O 11 Flatten the first two dimensions to get the output in

[0055] O 12 Normalize to get O1;

[0056] Multi-scale channel attention structure for processing O1:

[0057] Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m};

[0058] For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation;

[0059] Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′;

[0060] Dual-path gated attention structure for processing O2:

[0061] Use 1×1 convolution to double the channel dimension of O2, and get O 31 ;

[0062] For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ;

[0063] Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function;

[0064] Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33, output O3 = O 33 +O2;

[0065] Backbone network:

[0066] Assume the input is x,

[0067] Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5};

[0068] x is processed by M1 to get out1;

[0069] out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2;

[0070] out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3;

[0071] out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4;

[0072] out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

[0073] Preferably, the step of acquiring the image to be detected of the construction scene includes:

[0074] Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons;

[0075] Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments;

[0076] Acquire multimodal information of multiple construction workers in the construction scene;

[0077] Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future;

[0078] Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence;

[0079] Based on the shooting parameters, a mobile shooting device in the construction scene is controlled and scheduled to shoot the target path segment to obtain the image to be detected.

[0080] Preferably, the path segmentation condition includes one or more of the following combinations:

[0081] The path length of each path segment falls within the standard length interval;

[0082] The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein 2≤N≤K, K is the first quantity threshold;

[0083] The total number of path segments in the same sub-scene in the construction scene is less than or equal to a second number threshold.

[0084] Preferably, the shooting parameter determination conditions include:

[0085] The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters;

[0086] There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment;

[0087] The mobile shooting device shoots the target path segment without obstruction based on the shooting angle of view when shooting based on the shooting parameters.

[0088] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0089] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0091] Figure 1 Schematic diagram of a transformer-based helmet detection system in an embodiment of the present invention;

[0092] Figure 2 Schematic diagram of helmet detection results when the helmet detection model in the embodiment of the present invention is actually applied;

[0093] Figure 3 Schematic diagram of a transformer-based helmet detection method in an embodiment of the present invention. DETAILED DESCRIPTION

[0094] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0095] The embodiment of the present invention provides a transformer-based helmet detection system, such as Figure 1 As shown, including:

[0096] Model training module 1 is used to train the transformer model based on transfer learning and use the helmet detection data set to obtain a helmet detection model;

[0097] Image acquisition module 2, used to acquire the image to be detected of the construction scene;

[0098] The helmet detection module 3 is used to perform helmet detection on the image to be detected based on the helmet detection model.

[0099] There are a large number of images of people wearing helmets in the helmet detection data set. Using the helmet detection data set, the transformer model is trained to obtain the helmet detection model. Based on the helmet detection model, helmet detection is performed on the image to be detected. The backbone network of the standard detection network Mask-RCNN is replaced with the backbone network in the transformer model of the present invention, and AdamW is selected as the optimizer with a decay weight of 0.05 and a learning rate of 0.0001. The model is trained and tested on the MSCOCO target detection data set for 100 epochs. The test results show that the average detection accuracy reaches 44.2%, while the average detection accuracy of the model using ResNet-50 as the backbone network is 38.0%, and the average detection accuracy of the model using SWIN transformer as the backbone network is 42.2%. The results show that the transformer model of the present invention has a better effect. The accuracy of helmet detection is improved.

[0100] In one embodiment, the transformer model includes:

[0101] Pixel Block Embedding Module:

[0102] Define the input as W, H, and C represent the width, height, and number of channels of the input;

[0103] Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used;

[0104] O 11 Flatten the first two dimensions to get the output in

[0105] O 12 Normalize to get O1;

[0106] Multi-scale channel attention structure for processing O1:

[0107] Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m};

[0108] For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation;

[0109] Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′;

[0110] Dual-path gated attention structure for processing O2:

[0111] Use 1×1 convolution to double the channel dimension of O2, and get O 31 ;

[0112] For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ;

[0113] Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function;

[0114] Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2;

[0115] Backbone network:

[0116] Assume the input is x,

[0117] Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5};

[0118] x is processed by M1 to get out1;

[0119] out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2;

[0120] out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3;

[0121] out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4;

[0122] out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

[0123] 1. Design a pixel block embedding module and define the input as W, H, and C represent the width, height, and number of channels of the input;

[0124] 1.1 Using convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used;

[0125] 1.2 O 11 Flatten the first two dimensions to get the output in

[0126] 1.3 O 12 Normalize to get O1

[0127] 2. Design a multi-scale channel attention structure to handle O1

[0128] 2.1 Divide O1 into m channels, and obtain O1 = {O 1_1 ,O 1_2 ,…,O 1_m};

[0129] 2.2 vs. O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation

[0130] 2.3 Execute 2.2 for each channel of O1 and get the output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m '

[0131] 3. Design a dual-path gated attention structure to handle O2

[0132] 3.1 Use 1×1 convolution to double the channel dimension of O2 to get O 31 ;

[0133] 3.2 vs. O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ;

[0134] 3.3 Calculation of O33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function;

[0135] 3.4 Use 1×1 convolution to transform O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2;

[0136] 4. Build the backbone network, assuming the input is x,

[0137] 4.1 Construct the transformer basic module M. M is composed of steps 1-3 in series, that is, the input is processed by steps 1, 2 and 3 respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5}

[0138] 4.2x is processed by M1 to get out1

[0139] 4.3out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2;

[0140] 4.4out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3;

[0141] 4.5out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4;

[0142] 4.5out4 first performs upsampling, doubles the feature width and height to obtain out41, then takes out41+out1 as input, and then processes it through M5 to obtain output out5. In one embodiment, the image acquisition module 2 acquires the image to be detected of the construction scene, including:

[0143] Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons;

[0144] Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments;

[0145] Acquire multimodal information of multiple construction workers in the construction scene;

[0146] Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future;

[0147] Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence;

[0148] Based on the shooting parameters, a mobile shooting device in the construction scene is controlled and scheduled to shoot the target path segment to obtain the image to be detected.

[0149] The mobile shooting device may be a drone, etc. Construction workers need to wear safety helmets when they are at the target scene location, for example: the target scene location is a scene location within the construction scene where there may be falling objects from high altitudes, etc.; the scene map is a geographical map of the construction scene; when planning the necessary path for wearing safety helmets, the target scene location can be first determined based on the various geographical information on the scene map, and then the target scene locations are connected in sequence to obtain the previous target scene location in the necessary path for wearing safety helmets and the target scene location closest to its surroundings are connected as the next adjacent target scene location; the necessary path for wearing safety helmets is divided into multiple path segments; multimodal information includes: the current location of the construction worker in the construction scene, the historical movement trajectory of the construction worker in the construction scene within the last 100 seconds, etc., the construction tasks assigned to the construction worker on the day, etc.; based on multimodal information, it is possible to predict the target path segments that construction workers will pass through in the future, for example: it can be based on The current position of the construction worker in the construction scene and the historical movement trajectory of the construction worker in the construction scene in the last 100 seconds can be predicted. It can also be predicted based on the construction tasks assigned to the construction workers on that day (for example, a task execution location passes through a certain path segment, which is used as the target path segment). The movement position sequence of the construction workers when they pass through the target path segment in the future can also be predicted. For example: the construction task assigned to the construction workers on that day is to maintain each device passing through the target path segment, then the movement position in the movement position sequence is the position of each device in turn; according to the movement position sequence, it can be reasonably determined how the mobile shooting device captures the image to be detected for detecting the safety helmet when the construction workers pass through the target path segment, that is, the shooting parameters, which include shooting position, shooting posture, etc.; based on the shooting parameters, the mobile shooting device in the construction scene is controlled and scheduled to shoot the target path segment to obtain the image to be detected.

[0150] Generally, the scenes in which helmet wearing detection is required in construction scenes will change with the changes in daily construction tasks. If the image to be detected is acquired for each scene, it will be a waste of acquisition resources. The embodiment of the present invention can solve this problem by screening out the target path segment from the path segment, determining the shooting parameters, and triggering the control and scheduling of the mobile shooting device to go to the target path segment to acquire the image to be detected based on the shooting parameters, thereby reducing acquisition resources. Secondly, when using a mobile shooting device to acquire images in a scene, the mobile shooting device is mostly controlled to patrol along a fixed route. This is relatively rigid and cannot guarantee the rationality of the control and scheduling of the mobile shooting device. The embodiment of the present invention can also solve this problem by adaptively determining the target path segment and shooting parameters to fully guarantee the rationality of the control and scheduling of the mobile shooting device.

[0151] In one embodiment, the path segmentation condition includes one or more combinations of the following:

[0152] The path length of each path segment falls within a standard length range; the standard length range may be 100 meters to 200 meters; setting this path segmentation condition can ensure that the path segment photographed by the mobile photographing device is not too long to exceed the photographing range of the mobile photographing device, nor too short to waste the photographing resources of the mobile photographing device;

[0153] The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein, 2≤N≤K, K is the first quantity threshold; the first quantity threshold may be 5; the total cost of the mobile camera device tracking transfer is the total distance that the mobile camera device flies along the N consecutive adjacent path segments, etc.; the cost threshold may be set in advance by the technical personnel, and its unit is consistent with the unit of the total cost of the mobile camera device tracking transfer; setting this path segmentation condition can ensure that when the mobile camera device temporarily needs to track the construction personnel (for example: when it is detected that the construction personnel have not worn the safety helmet for a long time, it is necessary to continuously detect whether the personnel have worn the safety helmet afterwards), the total cost of the mobile camera device tracking transfer will not be too large;

[0154] The total number of path segments in the same sub-scene in the construction scene is less than or equal to the second number threshold. The sub-scene is a local scene in the construction collection; the second number threshold can be 4; when setting this path segmentation condition, it can be ensured that there will not be too many path segments entering the same sub-scene, resulting in too many tasks for the mobile shooting device to shoot the same sub-scene and causing unreasonable control and scheduling of the mobile shooting device.

[0155] In one embodiment, the shooting parameter determination condition includes:

[0156] The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters; when the mobile shooting device shoots based on the shooting parameters, a shooting angle of view is generated, so that the target path segment completely falls within the shooting angle of view, which can ensure that the mobile shooting device shooting the target path segment can completely capture the entire picture when the construction personnel pass through the target path segment;

[0157] There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment; the angle interval can be 150 degrees to 180 degrees; the ratio threshold can be 0.1; setting this shooting parameter determination condition can ensure that the mobile shooting device can capture the faces of construction workers, so that when it is detected that the construction workers are not wearing safety helmets, the facial information of the construction workers can be obtained, the identity can be determined, and subsequent reminders or recording operations are convenient;

[0158] The mobile shooting device shoots the target path segment without any obstruction at the shooting angle when shooting based on the shooting parameters. Setting this shooting parameter determination condition can ensure that there is no obstruction when the mobile shooting device shoots the target path segment.

[0159] The embodiment of the present invention provides a transformer-based helmet detection method, which is characterized by comprising:

[0160] S1. Based on transfer learning, the helmet detection dataset is used to train the transformer model to obtain the helmet detection model.

[0161] S2, obtaining the image to be detected of the construction scene;

[0162] S3. Perform helmet detection on the image to be detected based on the helmet detection model.

[0163] The transformer model consists of:

[0164] Pixel Block Embedding Module:

[0165] Define the input as W, H, and C represent the width, height, and number of channels of the input;

[0166] Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used;

[0167] O 11 Flatten the first two dimensions to get the output in

[0168] O 12 Normalize to get O1;

[0169] Multi-scale channel attention structure for processing O1:

[0170] Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m};

[0171] For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation;

[0172] Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′;

[0173] Dual-path gated attention structure for processing O2:

[0174] Use 1×1 convolution to double the channel dimension of O2, and get O 31 ;

[0175] For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ;

[0176] Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function;

[0177] Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2;

[0178] Backbone network:

[0179] Assume the input is x,

[0180] Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5};

[0181] x is processed by M1 to get out1;

[0182] out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2;

[0183] out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3;

[0184] out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4;

[0185] out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

[0186] The step of obtaining the image to be detected of the construction scene includes:

[0187] Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons;

[0188] Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments;

[0189] Acquire multimodal information of multiple construction workers in the construction scene;

[0190] Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future;

[0191] Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence;

[0192] Based on the shooting parameters, a mobile shooting device in the construction scene is controlled and scheduled to shoot the target path segment to obtain the image to be detected.

[0193] The path segmentation condition includes one or more of the following combinations:

[0194] The path length of each path segment falls within the standard length interval;

[0195] The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein 2≤N≤K, K is the first quantity threshold;

[0196] The total number of path segments in the same sub-scene in the construction scene is less than or equal to a second number threshold.

[0197] The shooting parameter determination conditions include:

[0198] The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters;

[0199] There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment;

[0200] The mobile shooting device shoots the target path segment without obstruction based on the shooting angle of view when shooting based on the shooting parameters.

[0201] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A transformer-based helmet detection system, characterized in that: include: The model training module is used to train the transformer model based on transfer learning and use the helmet detection dataset to obtain the helmet detection model; An image acquisition module, used to acquire the image to be detected of the construction scene; A helmet detection module, used for performing helmet detection on the image to be detected based on the helmet detection model; The image acquisition module acquires the image to be detected of the construction scene, including: Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons; Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments; Acquire multimodal information of multiple construction workers in the construction scene; Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future; Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence; Based on the shooting parameters, controlling and scheduling a mobile shooting device in the construction scene to shoot the target path segment to obtain the image to be detected; The shooting parameter determination conditions include: The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters; There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment; The mobile shooting device shoots the target path segment without obstruction based on the shooting angle of view when shooting based on the shooting parameters.

2. The transformer-based helmet detection system according to claim 1, characterized in that: The transformer model consists of: Pixel Block Embedding Module: Define the input as W, H, and C represent the width, height, and number of channels of the input; Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used; O 11 Flatten the first two dimensions to get the output in O 12 Normalize to get O1; Multi-scale channel attention structure for processing O1: Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m }; For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation; Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′; Dual-path gated attention structure for processing O2: Use 1×1 convolution to double the channel dimension of O2, and get O 31 ; For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ; Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function; Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2; Backbone network: Assume the input is x, Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5}; x is processed by M1 to obtain out1; out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2; out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3; out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4; out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

3. The transformer-based helmet detection system according to claim 1, characterized in that: The path segmentation condition includes one or more of the following combinations: The path length of each path segment falls within the standard length interval; The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein 2≤N≤K, K is the first quantity threshold; The total number of path segments in the same sub-scene in the construction scene is less than or equal to a second number threshold.

4. A transformer-based helmet detection method, characterized in that: include: Based on transfer learning, the helmet detection dataset is used to train the transformer model to obtain the helmet detection model; Acquire the image to be detected of the construction scene; Based on the helmet detection model, performing helmet detection on the image to be detected; The step of obtaining the image to be detected of the construction scene includes: Planning a necessary path for wearing a hard hat in a scene map of the construction scene; the necessary path for wearing a hard hat is formed by sequentially connecting a plurality of target scene positions in the construction scene where construction workers need to wear hard hats for safety reasons; Based on the path segmentation condition, the necessary path for wearing the helmet is divided into a plurality of path segments; Acquire multimodal information of multiple construction workers in the construction scene; Based on the multimodal information, predicting a target path segment in the path segment that a construction worker will pass through in the future and a moving position sequence of the construction worker when passing through the target path segment in the future; Based on a shooting parameter determination condition, determining a shooting parameter of the target path segment according to the moving position sequence; Based on the shooting parameters, controlling and scheduling a mobile shooting device in the construction scene to shoot the target path segment to obtain the image to be detected; The shooting parameter determination conditions include: The target path segment completely falls within the shooting angle of view when the mobile shooting device shoots based on the shooting parameters; There are at least T groups of target scene positions in the target path segment, and the efficiency angle between the moving vectors and the shooting vectors of the mobile shooting device when shooting based on the shooting parameters falls within the angle interval; wherein each group of target scene positions is composed of two adjacent target scene positions in the target path segment, and the moving vector of each group of target scene positions is constructed based on the previous target scene position in the group of target scene positions and the direction from the previous target scene position to the next target scene position in the group of target scene positions; the shooting vector of the mobile shooting device when shooting based on the shooting parameters is constructed based on the shooting position of the mobile shooting device when shooting based on the shooting parameters and the shooting direction of the mobile shooting device when shooting based on the shooting parameters; L≤T / X-1≤1, L is the ratio threshold, and X is the total number of target scene positions in the target path segment; The mobile shooting device shoots the target path segment without obstruction based on the shooting angle of view when shooting based on the shooting parameters.

5. The transformer-based helmet detection method according to claim 4, characterized in that: The transformer model consists of: Pixel Block Embedding Module: Define the input as W, H, and C represent the width, height, and number of channels of the input; Use the convolution kernel W k×k , where k represents the convolution kernel size, the convolution operation is performed on the input I, and the output is C1 represents the number of convolution kernels used; O 11 Flatten the first two dimensions to get the output in O 12 Normalize to get O1; Multi-scale channel attention structure for processing O1: Divide O1 into m channels, and get O1 = {O 1_1 ,O 1_2 ,…,O 1_m }; For O 1_i Use depth-wise separable convolution to get Q, K and V, and output O 1_i ′=W p (V · softmax(K · Q)) + O 1_i , W p Represents the convolution operation; Output O2 = O 1_1 ′+O 1_2 ′+…+O 1_m ′; Dual-path gated attention structure for processing O2: Use 1×1 convolution to double the channel dimension of O2, and get O 31 ; For O 31 Use depth-wise separable convolution and split the result into O in the channel dimension. 32_0 and O 32_1 ; Calculate O 33 =G(O 32_0 )·O 32_1 , where G represents the GELU activation function; Use 1×1 convolution to convert O 33 The channel dimension is restored to be consistent with O2, and O is obtained. 33 , output O3 = O 33 +O2; Backbone network: Assume the input is x, Construct the transformer basic module M, which is composed of a pixel block embedding module, a multi-scale channel attention structure, and a dual-path gated attention structure in series. That is, the input is processed by the pixel block embedding module, the multi-scale channel attention structure, and the dual-path gated attention structure respectively. The backbone network consists of 5 basic modules {M1, M2, M3, M4, M5}; x is processed by M1 to get out1; out1 is first downsampled to reduce the feature width and height to half, and then processed by M2 to obtain out2; out2 is first downsampled to reduce the feature width and height to half, and then processed by M3 to obtain out3; out3 is first upsampled to double the feature width and height to get out31, then out31+out2 is used as input and processed by M4 to get out4; out4 is first upsampled to double the feature width and height to get out41, then out41+out1 is used as input, and then processed by M5 to get output out5.

6. The transformer-based helmet detection method according to claim 4, characterized in that: The path segmentation condition includes one or more of the following combinations: The path length of each path segment falls within the standard length interval; The total cost of the mobile camera device tracking transfer of N consecutive adjacent path segments is less than or equal to the cost threshold; wherein 2≤N≤K, K is the first quantity threshold; The total number of path segments in the same sub-scene in the construction scene is less than or equal to a second number threshold.

Images