A method and device for detecting hot work

Abstract

The present application provides a method and device for detecting hot work, and performs the following processing on each frame image in a video stream of hot work: extracting an image sequence containing the current frame image from the video stream; extracting temporal features used to characterize the dynamic changes of the target object from the image sequence; using a multi-layer cross-fusion feature pyramid network to process the extracted temporal features to obtain the pyramid temporal features containing the current frame image; extracting global features and local features from the pyramid temporal features, and determining the probability of hot work in the current frame image through feature fusion; and outputting the current frame image when it is detected that the probability is greater than a preset hot work probability threshold. Through this application, the detection accuracy of hot work is improved, and hot work that has not been approved and reported is discovered in a timely manner, which improves the safety of hot work construction, thereby reducing casualties and equipment loss.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for detecting hot work operations. Background Art

[0002] At present, hot work, such as welding and cutting, is a common operation in industrial production and construction activities. However, non-standard hot work (such as hot work in a restricted area) is a high-risk factor for fire accidents. Therefore, supervision of hot work will help reduce the risk of fire.

[0003] However, in a complex and ever-changing working environment, the existing technology has the following defects when detecting local areas in a single image:

[0004] 1) It does not consider the temporal information of the hot work behavior and the peripheral visual information other than the hot work in the picture, and it is easy to misdetect the interference factors such as complex lighting and fireworks in the background as hot work. Therefore, it is not robust enough in complex working environments;

[0005] 2) When using the feature pyramid network to extract multi-scale features of dynamic fire, only adjacent layers are considered. Therefore, as the number of layers increases, information dissipates, making the problem of missed detection of small targets not well alleviated;

[0006] 3) Hot work operations are diverse in form, resulting in very irregular shapes of hot work. The current method does not have a corresponding module to deal with this, resulting in inaccurate positioning of the hot work. Summary of the invention

[0007] In view of this, the purpose of the present application is to provide a method and device for detecting hot work to overcome at least one of the above-mentioned defects.

[0008] In a first aspect, an embodiment of the present application provides a method for detecting hot work operations, and performs the following processing on each frame image in a video stream of hot work operations: extracting an image sequence including a current frame image from the video stream; extracting temporal features for characterizing dynamic changes of a target object from the image sequence; and processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature, and a fifth pyramid feature, each pyramid feature including a preset number of feature layers, and each feature in the first pyramid feature is a pyramid feature. The feature layer is a time series feature, each feature layer in the second pyramid feature is a cross fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid time series feature; global features and local features are extracted from the pyramid time series features through a classification network, and the probability of the existence of hot work in the current frame image is determined through feature fusion; when it is detected that the probability is greater than a preset hot work probability threshold, the current frame image is output.

[0009] In an optional embodiment of the present application, a positioning network is used to extract shape features from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

[0010] In an optional embodiment of the present application, the specific location of the hot work in the current frame image is determined in the following manner: the corresponding original convolution layer in the positioning network is replaced by multiple asymmetric deformable convolutions; the shape of the hot work in the current frame image is identified through the asymmetric deformable convolutions, the position parameters of the hot work in the current frame image are output, and the specific location of the hot work in the current frame image is determined and displayed in the form of a rectangular frame.

[0011] In an optional embodiment of the present application, the current frame image that frames the location of the hot work is determined in the following manner: determine whether the probability is greater than the preset hot work probability threshold; if the probability is greater than the preset hot work probability threshold, it is considered that there is hot work in the current frame image, and the position of the rectangular frame is determined according to the position parameters, and the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame; according to the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame, the specific location of the hot work is marked with a rectangular frame on the current frame image; and the current frame image that frames the location of the hot work is output.

[0012] In an optional embodiment of the present application, the probability that hot work exists in the current frame image is determined in the following manner: the pyramid time series features are input into a convolutional layer, and global features of the current frame image are determined by global pooling feature extraction after the convolutional layer; the pyramid time series features of the current frame image are divided into multiple local areas, and local features of each local area are obtained from the pyramid time series features corresponding to the current frame image; for each local area, the global features are added to the local features of the local area, and feature fusion of the local features of the local area and the global features is performed; and the probability that hot work exists in each local area in the current frame image is output.

[0013] In an optional embodiment of the present application, the position parameters are determined in the following manner: an asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions; the shape of the hot work that meets the preset conditions in the current frame image is identified through the deformable convolution, and the position parameters of the hot work in the current frame image are output to calibrate the specific position of the hot work in the current frame image.

[0014] In an optional embodiment of the present application, an image sequence is determined in the following manner: determining a current frame image to be detected, a target number of frames before and after the current frame image in the extracted image sequence, and a total number of frames to be extracted; extracting each frame image read from the video stream to form a candidate image sequence, wherein the candidate image sequence corresponds one-to-one to the current frame image to be detected; for each frame image, determining whether the current frame image is a desired preset frame image, wherein the preset frame image is one of the following situations: the current frame image, one of the images of the target number of frames before the current frame, and one of the images of the target number of frames after the current frame; for each frame image, if the current frame image is the desired preset frame image, then the current frame image is added to the target image sequence, the order of frames in the target image sequence is consistent with that during extraction, and the frame in the middle position of the target image sequence is the current frame image; for each frame image, if the current frame image is not the desired preset frame image, then the current frame image continues to be stored in the candidate image sequence; when it is detected that the number of frames in the target image sequence meets the total number of frames, the target image sequence is determined as an image sequence.

[0015] In a second aspect, an embodiment of the present application further provides a hot work detection device, the device comprising: an image sequence extraction module, used to extract an image sequence including a current frame image from a video stream; a temporal feature extraction module, used to extract temporal features for characterizing dynamic changes of a target object from the image sequence; a pyramid temporal feature determination module, used to process the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, the multi-layer cross-fusion network comprising a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature comprising a preset number of feature layers, each feature in the first pyramid feature The layer is a time series feature, each feature layer in the second pyramid feature is a cross fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid time series feature; a probability determination module is used to extract global features and local features from the pyramid time series features through a classification network, and determine the probability of the existence of hot work in the current frame image through feature fusion; a current frame image output module is used to output the current frame image when it is detected that the probability is greater than a preset hot work probability threshold.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method described above are performed.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are executed.

[0018] The hot work detection method and device provided in the embodiment of the present application perform the following processing for each frame image in the video stream of the hot work: extract the image sequence containing the current frame image from the video stream; extract the time series features used to characterize the dynamic changes of the target object from the image sequence; use a multi-layer cross-fusion feature pyramid network to process the extracted time series features to obtain the pyramid time series features containing the current frame image, extract global features and local features from the pyramid time series features, and determine the probability of the current frame image containing the hot work through feature fusion; when the probability detected is greater than the preset hot work probability threshold, output the current frame image. Through the present application, by extracting the time series features used to characterize the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is hot work in the current frame image, extract global features and local features from the pyramid time series features, and determine the probability of the current frame image containing the hot work through feature fusion, which can comprehensively consider various information in the image and improve the reliability and stability of detection.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flow chart of a method for detecting hot work provided in an embodiment of the present application;

[0022] Figure 2 A flowchart for determining an image sequence provided in an embodiment of the present application;

[0023] Figure 3 A flow chart for determining the probability of hot work existing in the current frame image provided by an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a hot work detection method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of a hot work detection device provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0028] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of image processing.

[0029] Hot work, such as welding and cutting, is a common operation in industrial production and construction activities. It has the characteristics of generating sparks and high temperature, which increases the possibility of fire. When hot work is carried out in a confined area, if the operation is not standardized, such as failure to strictly abide by safety regulations and inadequate protective measures, it is very easy to cause fire accidents. The confined area may contain flammable and explosive items, complex lighting conditions, etc. These factors further increase the risk of fire caused by hot work.

[0030] Research has found that traditional monitoring methods rely on manual inspections, which are labor-intensive and prone to omissions. Irregular hot work operations are a high-risk factor for fire accidents. In complex and changeable operating environments, existing technologies have problems such as high false alarm rates and insufficient recognition accuracy when detecting local areas in a single image, especially in complex and changeable operating environments.

[0031] Based on this, the embodiment of the present application provides a method and device for detecting hot work, and performs the following processing for each frame image in the video stream of hot work: extracting an image sequence containing the current frame image from the video stream; extracting the time series features used to characterize the dynamic changes of the target object from the image sequence; using a multi-layer cross-fusion feature pyramid network to process the extracted time series features to obtain a pyramid time series feature containing the current frame image, extracting global features and local features from the pyramid time series features, and determining the probability of the current frame image containing hot work through feature fusion; when the probability detected is greater than a preset hot work probability threshold, outputting the current frame image. Through the present application, by extracting the time series features used to characterize the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is hot work in the current frame image, extract global features and local features from the pyramid time series features, and determine the probability of the current frame image containing hot work through feature fusion, and comprehensively consider various information in the image to improve the reliability and stability of detection.

[0032] See also Figure 1 , Figure 1 This is a flow chart of the hot work detection method provided in the embodiment of the present application. Figure 1 As shown in , the hot work detection method provided in the embodiment of the present application includes:

[0033] S101: extracting an image sequence including a current frame image from a video stream.

[0034] Specifically, the image sequence is determined in the following way:

[0035] See also Figure 2 , Figure 2 This is a flow chart for determining an image sequence provided by an embodiment of the present application. Figure 2 As shown in , the hot work detection method provided in the embodiment of the present application includes:

[0036] S201, determining a current frame image to be detected, a target number of frames before and after the current frame image in an extracted image sequence, and a total number of frames to be extracted;

[0037] Read the video stream, and obtain multiple frames before and after the current frame according to the predefined current frame image, target frame number and total frame number, that is, the image sequence containing timing information, the current frame is recorded as It, and the extracted image sequence is [I t -n,I t -n+1,……,I t , ..., I t +n-1,I t +n], where n is the target number of frames before and after the current frame image to be extracted.

[0038] For example, n=2, the current frame image is determined to be the 3rd frame, the target frame number is 2 frames, and the total frame number is 5 frames. Then, the image sequence of the 3rd frame is the 5 frames from the 1st frame image to the 5th frame image.

[0039] Specifically, determine the current frame image to be detected, which is the starting point of the entire detection process, and set the target number of frames before and after the current frame image to be extracted. This value determines the size of the time window we will observe in order to capture the dynamic changes of hot work.

[0040] According to the target number of frames, calculate the total number of frames to be extracted. The total number of frames is equal to the sum of the current frame and the target number of frames before and after it (if the current frame is calculated only once, the total number of frames is 2×target number of frames+1).

[0041] S202: extract each frame of the read image from the video stream to form a candidate image sequence.

[0042] Here, the candidate image sequence corresponds one-to-one to the current frame image to be detected.

[0043] The images in the video stream are read frame by frame to form a preliminary candidate image sequence. This sequence includes all the frames in the video stream, but will be filtered later according to needs.

[0044] It is ensured that each frame in the candidate image sequence has a one-to-one correspondence with the current frame image to be detected, that is, each frame can be located at a specific position in the video stream.

[0045] S203 . For each frame image, determine whether the current frame image is a desired preset frame image.

[0046] The preset frame image is one of the following: the current frame image, one of the images with a target number of frames before the current frame, and one of the images with a target number of frames after the current frame.

[0047] Traverse each frame in the candidate image sequence to determine whether the frame image is the required preset frame image, the preset frame image includes the current frame image itself, one or more images of the target frame number before the current frame, and one or more images of the target frame number after the current frame.

[0048] The basis for the determination may be the frame sequence number, timestamp or other attributes that can uniquely identify the frame.

[0049] S204 . For each frame image, if the current frame image is a required preset frame image, add the current frame image to the candidate image sequence.

[0050] The order of frames in the candidate image sequence remains consistent with that during extraction, and the frame in the middle of the candidate image sequence is the current frame image.

[0051] S205 . For each frame image, if the current frame image is not the required preset frame image, the current frame image continues to be stored in the candidate image sequence.

[0052] Here, for the frames that meet the preset conditions, they are added to the final image sequence, and this sequence will be used for subsequent hot work detection.

[0053] The order of frames in the candidate image sequence is kept unchanged to ensure the correct temporal order in the image sequence. In particular, the middle position of the image sequence should be the current frame image to reflect the time point we are mainly concerned with.

[0054] For example, since the time of the first two frames of images is too short and not conducive to detection, they are ignored here. The detection starts from the 3rd frame image. Then the 6th frame is not the preset frame image in the image sequence corresponding to the 3rd frame image. If we detect the 4th frame image, the preset frame image in the image sequence corresponding to the 4th frame image can be the 2nd to 6th frame images, and so on. Determine each frame image after the 3rd frame as the image sequence when the current frame image is used, so as to detect the hot work operation of each frame image.

[0055] S206: When it is detected that the number of frames in the candidate image sequence meets the total number of frames, the candidate image sequence is determined as an image sequence.

[0056] When it is detected that the number of frames in the candidate image sequence meets the total number of frames, the candidate image sequence is determined as the final image sequence, and this image sequence is output for use by the subsequent hot work detection algorithm.

[0057] In an optional embodiment, factors such as the stability of the video stream, the consistency of the frame rate, and the image quality are also considered to ensure that the extracted image sequence can accurately reflect the actual situation of the hot work operation. If the video stream is long or the frame rate is high, the extracted image sequence may be very large. At this time, it may be necessary to consider downsampling or compressing the image to reduce storage and computing costs. In addition, in order to improve the accuracy and efficiency of detection, other technical means, such as motion detection, target tracking, etc., can be combined to further optimize the image sequence extraction process.

[0058] S102: extracting temporal features for characterizing dynamic changes of the target object from the image sequence.

[0059] First, extract the image sequence corresponding to the current frame image from the video stream. According to the characteristics of hot work, select frames containing key dynamic information, such as the appearance, growth, and extinction of flames. Adjust each frame in the image sequence to a fixed size for the ResNet-50 model input. For example, the image resolution can be adjusted to 640x640. In order to increase the generalization ability of the model, the image sequence can be subjected to data enhancement operations such as random cropping, rotation, and flipping.

[0060] Then, load the pre-trained ResNet-50 model, which has been trained on a large number of image classification tasks and has learned to extract high-level features of images, input each frame in the image sequence into the ResNet-50 model, and extract the feature vectors before its fully connected layer. These feature vectors contain high-level semantic information of the image, such as the shape and texture of the object.

[0061] Since the dynamic features of hot work have obvious changes in time series, while static similar objects in the background (such as lights) are relatively stable, through temporal feature extraction, we can effectively reduce the interference of static similar objects in the background and improve the recognition accuracy of hot work.

[0062] In an optional embodiment, a sequence of images containing hot work and background interference can be used as training data and input into the model for training. During the training process, the model parameters are updated by optimizing the loss function (such as cross entropy loss), and the model is evaluated using a test data set to calculate its accuracy, recall rate and other indicators for identifying hot work. The performance of the model is continuously optimized by adjusting the model structure, parameters, etc.

[0063] Finally, the trained model is deployed to the monitoring system to process the image sequence in the video stream to achieve real-time detection and early warning of hot work. By inputting the image sequence into ResNet-50, the visual temporal features in the image sequence can be effectively extracted, the dynamic features of hot work can be captured, and the interference of static similar objects in the background can be reduced.

[0064] Specifically, when using image sequences to detect hot work, extracting visual temporal features is a key step. By inputting the image sequence into a classic image classification model (such as ResNet-50), we can obtain the temporal features. Combined with the temporal analysis method, we can effectively capture the dynamic characteristics of hot work, thereby more accurately identifying hot work behavior and reducing the interference of static similar objects (such as lights) in the background.

[0065] S103 , using a multi-layer cross-fusion feature pyramid network to process the extracted temporal features to obtain pyramid temporal features of the current frame image.

[0066] Here, the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is the sum of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is a cross-fusion of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature.

[0067] First, we extract temporal features from the image sequence. These features usually contain dynamic information of hot work, such as the flickering and spread of flames.

[0068] Each feature layer in the first pyramid feature corresponds to a layer in the temporal feature.

[0069] Each feature layer in the second pyramid feature is obtained by cross-fusion of all feature layers in the first pyramid feature, which realizes the preliminary fusion of cross-layer features. However, this simple summation operation may not fully utilize the complementary information between different layers. Therefore, we introduce a cross-fusion strategy, that is, on the basis of cross-fusion, a specific fusion operation (such as convolution, attention mechanism, etc.) is used to further enhance the fusion effect of features. The cross-fused feature layer will contain the information of all feature layers in the first pyramid feature, but it is more compact and effective.

[0070] The third pyramid feature is obtained by summing the first pyramid feature and the second pyramid feature, which further enhances the degree of feature fusion, so that each feature layer in the third pyramid feature contains the information of all feature layers in the first pyramid feature and the second pyramid feature. Similarly, we can introduce a cross-fusion strategy to optimize this fusion process.

[0071] The fourth pyramid feature is constructed in a similar way to the second pyramid feature, but it is based on the third pyramid feature. Each feature layer in the fourth pyramid feature is obtained by cross-fusion of all feature layers in the third pyramid feature. In this way, the feature layer in the fourth pyramid feature will contain the information of all feature layers in the third pyramid feature (and indirectly the first pyramid feature and the second pyramid feature).

[0072] The fifth pyramid feature is obtained by summing the third pyramid feature and the fourth pyramid feature. As the final pyramid time series feature, the fifth pyramid feature will contain the information in all the previous pyramid features, and due to the existence of multi-layer cross fusion, this information will be more fully utilized and integrated.

[0073] Specifically, in the process of constructing pyramid features, the element-wise feature fusion operation is also used. Specifically, after obtaining the second pyramid feature, we perform element-wise addition feature fusion with the original time series feature first pyramid feature to obtain the third pyramid feature. Similarly, after obtaining F4, we perform element-wise addition feature fusion with F3 to obtain the fifth pyramid feature. This operation not only enhances the feature fusion effect, but also makes the entire network module plug-and-play. In addition, the module can be reused indefinitely, and the number of layers and fusion methods of the pyramid features can be adjusted according to the needs of specific tasks.

[0074] In neural networks, addition operations can be used for residual connections, such as short-circuit connections in ResNet. This connection allows the network to directly pass the output of the previous layer to the next layer, and merge with the output features of the next layer, which helps to alleviate the gradient vanishing problem in deep networks. The advantage of element-wise operations is that they are simple, efficient, and easy to implement. Since they are performed element by element, they can make full use of the parallel computing power of modern computing devices to speed up the calculation process. In addition, element-wise operations are also very scalable and can be easily applied to input tensors of different sizes.

[0075] Compared with the current mainstream method that only considers the fusion method of adjacent layer features, the multi-layer cross-fusion pyramid feature extraction method proposed in this application uses multi-layer cross-fusion, and each layer of features contains the feature information of other layers, thereby enhancing the robustness and expression ability of the features. The multi-layer cross-fusion strategy helps to retain and fuse the complementary information between different layers, avoiding the information dissipation problem during feature fusion. It can be plug-and-play, and the number of layers and fusion methods of the pyramid features can be adjusted according to specific tasks. It has strong scalability.

[0076] The multi-layer cross-fusion network of the present application has broad application prospects in the fields of hot work detection, target tracking, video analysis, etc. By extracting multi-layer cross-fused pyramid features, we can more accurately capture the dynamic characteristics of hot work and reduce the impact of size changes on detection effects, thereby improving the recognition accuracy and robustness of hot work. Compared with the current mainstream method that only considers the fusion of adjacent layer features, the fusion is more complete, alleviating the problem of information dissipation during feature fusion in the current method.

[0077] S104, extracting global features and local features from the pyramid time series features through a classification network, and determining the probability of hot work existing in the current frame image through feature fusion.

[0078] See also Figure 3 , Figure 3 This is a flow chart for determining the probability of hot work in the current frame image provided by the embodiment of the present application. Figure 3 As shown in , the hot work detection method provided in the embodiment of the present application includes:

[0079] S301, inputting the pyramid temporal features into the convolution layer, and extracting the global features of the current frame image through global pooling after the convolution layer.

[0080] One or more layers of images in the pyramid temporal features are input into the convolution layer. The convolution layer extracts the spatial features of the image through convolution operations and activation functions. The size of the feature map is further reduced through the pooling layer (such as maximum pooling or average pooling), thereby extracting global features. These global features contain the overall structure and key information of the image.

[0081] Preferably, in the classification network, a visual context feature extraction module is added after each convolution layer, that is, global maximum pooling (Global MaxPooling) is used in the channel dimension to extract the global features of the image for the features obtained by the convolution layer.

[0082] S302: Divide the pyramid temporal features of the current frame image into a plurality of local regions, and obtain local features of each local region from the pyramid temporal features corresponding to the current frame image.

[0083] That is, each local area is the area covered by the 3x3 convolution with that position as the center. Local features refer to the features of a specific area in an image, and they are usually used to capture the details and local changes of the image.

[0084] Here, the current frame image is divided into multiple local regions, each of which can be regarded as an independent image block, and for each local region, the corresponding local features are extracted from the pyramid temporal features. These local features describe the visual information in the region.

[0085] S303: For each local area, add the global feature to the local feature of the local area, and perform feature fusion of the local feature of the local area and the global feature.

[0086] Feature fusion is the process of combining global features and local features. It aims to make full use of the advantages of both features and improve recognition accuracy.

[0087] The global features are added to the local features of each local area to achieve the fusion of local features and global features. This addition operation can be element-by-element addition or weighted addition. After the features are added, the fused total features are obtained. These total features contain both global information and local details, so they can better describe the content of the image.

[0088] The global feature is added to the feature vector of each position to achieve the fusion of the feature of the current position and the global feature. However, this global feature mainly contains the semantic information of the scene and lacks local detail information. During hot work operations, the hot work area often has an important relationship with the surrounding local areas. For example, there are welding or cutting tools, workers with protective masks or automated equipment around the hot work. Therefore, it is necessary to better model and fuse the global features with the local features.

[0089] S304: Output the probability of hot work existing in each local area in the current frame image.

[0090] By repeating the feature fusion, convolution is finally used to output the probability of the existence of hot work at each position in the current frame image.

[0091] In summary, extracting global and local features through pyramid temporal features and performing feature fusion can effectively improve the accuracy of hot work identification. This method combines the advantages of multi-scale analysis and global-local feature fusion, providing an effective solution for target detection in complex scenes.

[0092] S105: When the detected probability is greater than a preset fire probability threshold, output the current frame image.

[0093] Here, it is determined whether the probability is greater than a preset fire probability threshold;

[0094] If the probability is greater than the preset fire probability threshold, it is considered that there is fire operation in the current frame image, and the current frame image is output; if the probability is not greater than the preset fire probability threshold, it is considered that there is no fire operation in the current frame image, and the next frame image is taken as the current frame image, and the image sequence corresponding to the current frame image is obtained to perform a new round of detection.

[0095] Specifically, the positioning network can also be used to extract deformation-robust features from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

[0096] In an optional embodiment, the corresponding original convolutional layer in the positioning network is replaced by multiple asymmetric deformable convolutions;

[0097] In the pyramid temporal feature extraction stage, we used the traditional convolution layer to capture the spatial features of the image. In order to introduce deformable convolution, we need to convert the original convolution layer, which usually involves adjusting the size, step size, padding and other parameters of the convolution kernel to ensure that the converted convolution layer is compatible with the deformable convolution layer.

[0098] Deformable convolution is a convolution operation that can adaptively adjust the sampling position. It allows the convolution kernel to sample in a non-uniform manner on the input feature map. Compared with traditional convolution, deformable convolution can more flexibly capture geometric deformation and shape changes in images, which is especially important for identifying targets with complex shapes and dynamic changes such as hot work.

[0099] In an optional embodiment, in the hot work positioning network, the original 3x3 convolution in YOLOv8 is replaced by 1x3, 3x3, and 3x1 deformable convolutions, because the deformable convolution can automatically adjust the receptive field according to the shape of the object and has a certain adaptability to objects with large shape changes.

[0100] Preferably, YOLOv8 is a newer version of the YOLO (You Only Look Once) object detection and image segmentation model, which is a SOTA (State Of The Art) model.

[0101] The 3x3 convolution used in the original YOLOv8 was replaced by 1x3, 3x3, and 3x1 deformable convolutions. Deformable convolution is a convolution method that can automatically adjust the receptive field according to the shape of the object. Therefore, it has good adaptability to objects with large shape changes. In the positioning of hot work, since the shape of hot work often changes greatly, the use of deformable convolution can better capture the characteristics of hot work.

[0102] Asymmetric deformable convolutions (such as 1x3 and 3x1) are introduced to better adapt to objects with different aspect ratios. The 1x3 convolution kernel can capture feature changes in the horizontal direction, while the 3x1 convolution kernel focuses on feature changes in the vertical direction. By combining these two asymmetric convolution kernels, the network can more accurately identify the shape changes of objects in different directions.

[0103] The original convolutional layer corresponding to the pyramid temporal feature is converted into a convolutional layer, that is, the traditional 3x3 convolution is replaced by the above-mentioned deformable convolution. This conversion process enables the network to handle the shape changes of hot work more flexibly.

[0104] Through asymmetric deformable convolution, the shape of the hot work in the current frame image is identified, the position parameters of the hot work in the current frame image are output, and the specific position of the hot work in the current frame image is determined in the form of a rectangular frame.

[0105] The position parameters are determined by:

[0106] Asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions. Through deformable convolution, the shape of the hot work that meets the preset conditions in the current frame image is identified, and the position parameters of the hot work in the current frame image are output to calibrate the specific position of the hot work in the current frame image.

[0107] Here, through asymmetric deformable convolution, we can more accurately capture the shape features of hot work, which includes key information such as the shape, size, and color of the flame. Based on the results of shape recognition, we output the position parameters of the hot work in the current frame image. These parameters usually include the coordinates of the center point of the hot work, the length and width of the rectangular box, etc.

[0108] Through the transformed deformable convolutional layer, the shape of the hot work that meets the preset conditions in the current frame image is identified, and the network outputs the position parameters of the hot work. These position parameters include the center point coordinates of the rectangular box, the length and width of the rectangular box. These position parameters are used to calibrate the specific position of the hot work in the image.

[0109] The current frame image of the frame fire operation position is determined by the following method:

[0110] Determine whether the probability is greater than the preset fire probability threshold;

[0111] If the probability is greater than the preset fire probability threshold, it is considered that there is fire operation in the current frame image, and the position of the rectangular frame is determined according to the position parameters. Here, the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame.

[0112] First, we determine whether the probability of hot work in the current frame image is greater than the preset hot work probability threshold based on the output of the classification layer. If the probability is greater than the threshold, it is considered that hot work exists in the current frame image. At this time, we determine the position of the rectangular box based on the position parameters. The position of the rectangular box is determined by the center point coordinates, length and width.

[0113] According to the coordinates of the center point of the rectangular frame, the length of the rectangular frame and the width of the rectangular frame, the specific position of the hot work is marked with a rectangular frame on the current frame image.

[0114] Output the current frame image of the fire operation position.

[0115] According to the determined position parameters, we use a rectangular frame to mark the specific position of the hot work on the current frame image. This usually involves drawing a rectangular frame with a specified length and width on the image with the center point as the reference.

[0116] Finally, we will output the current frame image of the fire operation location, which can be used in application scenarios such as real-time monitoring and alarm prompts.

[0117] If the probability is not greater than the preset fire probability threshold, it is considered that there is no fire operation in the current frame image, and no output is required, or the image obtained from the video stream is directly output.

[0118] Applying the above method to the video surveillance system can realize real-time monitoring and early warning of hot work. By analyzing the location of hot work in historical frame images, the spatiotemporal distribution characteristics of hot work can be extracted to provide data support for safety management. Combined with other sensor data (such as temperature, smoke concentration, etc.), a more intelligent alarm system can be realized to improve the accuracy and timeliness of fire warning.

[0119] In summary, by introducing deformable convolution and position parameter output, we can more accurately identify the shape and position of hot work and mark it with a rectangular frame in the current frame image. This provides strong technical support for the monitoring and management of hot work.

[0120] See also Figure 4 , Figure 4 Schematic diagram of the hot work detection method provided in the embodiment of the present application. Figure 4 As shown in , the schematic diagram includes an image sequence 41, a temporal feature 42, a multi-layer cross-fusion network 43, a classification network 44, a positioning network 45 and an output image 46.

[0121] like Figure 4 In the figure, the image sequence 41 is input into the classic image classification model ResNet-50 for temporal feature extraction to obtain temporal feature 42. The temporal feature 42 is cross-fused through a multi-layer cross-fusion network 43. Here, F1 corresponds to the first pyramid feature, F2 corresponds to the second pyramid feature, F3 corresponds to the third pyramid feature, F4 corresponds to the fourth pyramid feature, and F5 corresponds to the fifth pyramid feature.

[0122] The “2×” in the figure indicates that the corresponding modules in the classification network and the localization network need to be repeated twice, and the output results after two processings of the classification network 44 and the localization network 45 are summarized to obtain the output image 46.

[0123] like Figure 4 As shown, in the dashed box corresponding to the classification network 44, the input is the convolution layer 440, the parameters of the Conv of the convolution layer 440 are "k=3, p=1, s=1", the convolution kernel size (k=3), and the convolution kernel (also called the filter) is the core component in the convolution layer, which is used to slide on the input data (such as an image) and perform convolution operations to extract features. Figure 4 In the figure, the size of the convolution kernel is 3x3, which means that the convolution kernel is a matrix with 3 rows and 3 columns. The elements in this matrix (also called weights) will be optimized by the back-propagation algorithm during the training process.

[0124] Padding size (p=1), padding refers to adding extra pixel values ​​(usually zero) around the input data to control the size of the output feature map, Figure 4 In the example, the padding size is 1, which means that an extra circle of pixel values ​​is added to each boundary of the input data. This ensures that the convolution kernel can cover the edge of the input data, thereby avoiding the loss of edge information during the convolution process.

[0125] The step size (s=1) refers to the number of pixels that the convolution kernel moves each time it slides over the input data. Figure 4 In the example, the stride is 1, which means that the convolution kernel moves only one pixel each time it slides. This ensures that the convolution kernel can traverse every position of the input data and extract as many features as possible.

[0126] The parameters of the convolution layer Conv "k=3, p=1, s=1" indicate that the convolution layer uses a 3x3 convolution kernel, adds a circle of padding around the input data, and the convolution kernel moves one pixel at a time when sliding. These parameters together determine how the convolution layer extracts features from the input data.

[0127] The classification network 44 successively uses global max pooling 441, non-local attention mechanism 442 and channel attention 443 in CBAM to obtain a convolution layer 444. The parameters of Conv of the convolution layer 444 are "k=1, p=0, s=1" to achieve full integration of the local features around the features of the current position and the global features of the image, capture the feeling of dynamic and surrounding features, improve robustness, and reduce the interference of local similar objects in the background.

[0128] Preferably, in the dotted box corresponding to the positioning network 45, the input is three deep convolutional layers 451, 452 and 453 with different convolution kernel sizes. The deep convolutional layer 453DConv (Depthwise ConvolutionalLayer) represents the convolution kernel size of the deformable convolution layer (k=3×1): this means that the convolution kernel is a matrix of 3 rows and 1 column, or more accurately, a two-dimensional filter with 3 heights and 1 width. Convolution kernels of this shape are usually used to process data with a specific spatial structure, such as images. The convolution kernel sizes of the deformable convolution layers 451 and 452 can be (k=1×3) and (k=3×3), respectively. The specific explanation is the same as that of the deep convolution layer 453, and will not be repeated here.

[0129] In the classification network 44 and the localization network 45, the output results are the convolution layer 444 (Con2d, k=1, p=0, s=1) and the convolution layer 454 (Con2d, k=1, p=0, s=1), where the convolution kernel size (k=1) is 1x1, which means that the convolution kernel is a matrix with 1 row and 1 column. This size of convolution kernel is usually used to change the number of channels of the input data (i.e., to perform cross-channel feature fusion or transformation) without changing the size of the input data in the spatial dimension, the padding size (p=0) is 0, which means that no padding is performed on the input data. Therefore, the convolution kernel will slide directly on the original input data without considering the boundary effect, and the step size (s=1) is 1, which means that the convolution kernel moves only one pixel each time it slides. This ensures that the convolution kernel can traverse every position of the input data, but since the convolution kernel size is 1x1, the size of the output feature map will actually be the same as the size of the input data (without considering padding).

[0130] In summary, the parameters of Con2d (Conv2D) of convolutional layer 444 and convolutional layer 454 are both "k=1, p=0, s=1", which means that the convolutional layer uses a 1x1 convolution kernel, does not pad the input data, and the convolution kernel moves a distance of one pixel each time when sliding. This configuration of convolutional layers is usually used to change the number of channels of input data or perform cross-channel feature fusion without changing the spatial size of the input data.

[0131] This application proposes a feature pyramid network (FPN) based on cross-fusion of multiple layers of features. The feature pyramid network is a technical means for target detection, which aims to capture targets of different sizes through feature maps of different scales. It uses different layers of convolutional neural networks (CNN) to extract features of different scales and fuse these features to improve the performance of target detection, so that each layer of features contains the semantic information of the remaining layers, reduces the information dissipation problem of the current feature pyramid network during feature fusion, and improves the model's ability to detect multi-scale dynamic fire operations, especially the detection effect of small-scale dynamic fire.

[0132] The present application proposes a context feature extraction module for hot work based on global and local feature attention weighting, and integrates it into the classification network 44 of hot work to achieve modeling of the relationship between objects related to hot work, improve the robustness of the model, and reduce false detections caused by complex lighting, etc.

[0133] The present application proposes a positioning network 45 for dynamic fire operations based on asymmetric deformable convolution, which can automatically adjust the receptive field according to the different shapes of dynamic fire, so that the parameters fall on the dynamic fire area, and then extract more effective features, thereby improving the positioning accuracy of the model for dynamic fire operations with large shape changes.

[0134] The hot work detection method and device provided in the embodiment of the present application perform the following processing for each frame image in the video stream of the hot work: extract the image sequence containing the current frame image from the video stream; extract the time series features used to characterize the dynamic changes of the target object from the image sequence; use a multi-layer cross-fusion feature pyramid network to process the extracted time series features to obtain the pyramid time series features containing the current frame image, extract global features and local features from the pyramid time series features, and determine the probability of the current frame image containing the hot work through feature fusion; when the probability is greater than the preset hot work probability threshold, output the current frame image. Through the present application, by extracting the time series features used to characterize the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is a hot work in the current frame image, timely discover the hot work that has not been approved and reported, extract global features and local features from the pyramid time series features, and determine the probability of the current frame image containing the hot work through feature fusion, which can comprehensively consider various information in the image and improve the reliability and stability of detection.

[0135] The present invention obtains the timing information in the video stream by inputting a sequence of pictures, captures the dynamic features of the fire during the fire operation, and introduces visual context features in the classification network. Based on the attention mechanism, the weighted fusion of local features and global features is realized to extract the relationship between the fire and the surrounding objects. By extracting the above timing features and context features, the interference of similar objects in the background can be effectively reduced, and the accuracy of fire detection can be improved. In the positioning network, the introduction of asymmetric deformable convolution can automatically adjust the size and shape of the receptive field for different shapes of fire, so as to achieve a better positioning effect for the fire.

[0136] Here, in convolutional neural networks (CNNs), the receptive field refers to the size of the area of ​​the input image that a specific neuron in a certain layer of the network can receive. In CNNs, the receptive field gradually increases through the stacking of convolutional layers and pooling layers. This means that the deeper the network layer is, the larger the input area can be captured, which is crucial for detecting larger patterns or objects in the image. The size of the receptive field has an important impact on both classification tasks and target detection tasks. For classification tasks, the receptive field size of the last layer of feature maps must be greater than or equal to the input image size, otherwise the classification performance may not be ideal. For target detection tasks, the receptive field needs to match the size of the target to ensure that the model can accurately detect the target.

[0137] In summary, it is possible to better classify and locate the hot fire generated during hot fire operations, thereby improving the detection effect of hot fire operations.

[0138] Based on the same inventive concept, an embodiment of the present application also provides a hot work detection device corresponding to the hot work detection method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned hot work detection method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0139] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of the hot work detection device provided in the embodiment of the present application. Figure 5 As shown in , the hot work detection device 500 includes:

[0140] An image sequence extraction module 501 is used to extract an image sequence including a current frame image from a video stream;

[0141] A time series feature extraction module 502, used to extract time series features for characterizing dynamic changes of a target object from the image sequence;

[0142] The pyramid temporal feature determination module 503 is used to process the extracted temporal features by using a multi-layer cross-fusion feature pyramid network to obtain the pyramid temporal features of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature;

[0143] The probability determination module 504 is used to extract global features and local features from the pyramid time series features through a classification network, and determine the probability of the current frame image having hot work through feature fusion;

[0144] The current frame image output module 505 is used to output the current frame image when it is detected that the probability is greater than a preset fire probability threshold.

[0145] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 6 As shown in , the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

[0146] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figure 1 The specific implementation of the steps of the hot work detection method in the method embodiment shown can be found in the method embodiment, which will not be repeated here.

[0147] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps of the hot work detection method in the method embodiment shown can be found in the method embodiment, which will not be repeated here.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0152] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0153] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting hot work, characterized in that: The following processing is performed on each frame of the hot work video stream: Extracting an image sequence including a current frame image from the video stream; Extracting temporal features for characterizing dynamic changes of a target object from the image sequence; Using a multi-layer cross-fusion feature pyramid network, the extracted temporal features are processed to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion feature pyramid network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature; Extracting global features and local features from the pyramid time series features through a classification network, and determining the probability of hot work in the current frame image through feature fusion; When it is detected that the probability is greater than a preset fire probability threshold, outputting the current frame image; The image sequence is determined in the following way: Determine the current frame image to be detected, the target number of frames before and after the current frame image in the extracted image sequence, and the total number of frames to be extracted; Extracting each frame of image read from the video stream to form a candidate image sequence, wherein the candidate image sequence corresponds to the current frame image to be detected one by one; For each frame image, determine whether the current frame image is a desired preset frame image, wherein the preset frame image is one of the following: the current frame image, one of the images of a target number of frames before the current frame, and one of the images of a target number of frames after the current frame; For each frame image, if the current frame image is the desired preset frame image, the current frame image is added to the target image sequence, the order of the frames in the target image sequence is consistent with that during extraction, and the frame in the middle position of the target image sequence is the current frame image; For each frame image, if the current frame image is not the required preset frame image, the current frame image continues to be stored in the candidate image sequence; When it is detected that the number of frames in the target image sequence meets the total number of frames, the target image sequence is determined as an image sequence.

2. The method according to claim 1, characterized in that The method further comprises: Through the positioning network, shape features are extracted from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

3. The method according to claim 2, characterized in that Determine the specific location of the fire in the current frame image by the following method: Replacing the corresponding original convolutional layer in the positioning network with multiple asymmetric deformable convolutions; Through the asymmetric deformable convolution, the shape of the hot work in the current frame image is identified, the position parameters of the hot work in the current frame image are output, and the specific position of the hot work in the current frame image is determined in the form of a rectangular frame.

4. The method according to claim 3, characterized in that The current frame image of the frame fire operation position is determined by the following method: Determine whether the probability is greater than the preset fire probability threshold; If the probability is greater than the preset hot work probability threshold, it is considered that there is hot work in the current frame image, and the position of the rectangular frame is determined according to the position parameters, where the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame; According to the coordinates of the center point of the rectangular frame, the length of the rectangular frame and the width of the rectangular frame, a specific position of the hot work is marked with a rectangular frame on the current frame image; Output the current frame image of the fire operation position.

5. The method according to claim 1, characterized in that The probability of hot work in the current frame image is determined by the following method: Inputting the pyramid temporal features into a convolutional layer, and extracting the global features of the current frame image by global pooling after the convolutional layer; Dividing the pyramid temporal features of the current frame image into a plurality of local regions, and obtaining local features of each local region from the pyramid temporal features corresponding to the current frame image; For each local area, the global feature is added to the local feature of the local area to perform feature fusion of the local feature of the local area and the global feature; Output the probability of hot work existing in each local area of ​​the current frame image.

6. The method according to claim 3, characterized in that: The position parameters are determined by: Asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions; Through the deformable convolution, the hot work shape that meets the preset conditions in the current frame image is identified, and the position parameters of the hot work in the current frame image are output to calibrate the specific position of the hot work in the current frame image.

7. A hot work detection device, characterized in that: The device comprises: An image sequence extraction module is used to extract an image sequence including a current frame image from a video stream; A time series feature extraction module, used to extract time series features for characterizing dynamic changes of a target object from the image sequence; A pyramid temporal feature determination module is used to process the extracted temporal features by using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion feature pyramid network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature; A probability determination module is used to extract global features and local features from the pyramid time series features through a classification network, and determine the probability that a hot work exists in the current frame image through feature fusion; A current frame image output module, configured to output the current frame image when it is detected that the probability is greater than a preset fire probability threshold; The image sequence is determined in the following way: Determine the current frame image to be detected, the target number of frames before and after the current frame image in the extracted image sequence, and the total number of frames to be extracted; Extracting each frame of image read from the video stream to form a candidate image sequence, wherein the candidate image sequence corresponds to the current frame image to be detected one by one; For each frame image, determine whether the current frame image is a desired preset frame image, wherein the preset frame image is one of the following: the current frame image, one of the images of a target number of frames before the current frame, and one of the images of a target number of frames after the current frame; For each frame image, if the current frame image is the desired preset frame image, the current frame image is added to the target image sequence, the order of the frames in the target image sequence is consistent with that during extraction, and the frame in the middle position of the target image sequence is the current frame image; For each frame image, if the current frame image is not the required preset frame image, the current frame image continues to be stored in the candidate image sequence; When it is detected that the number of frames in the target image sequence meets the total number of frames, the target image sequence is determined as an image sequence.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images