Fire operation detection method and device

Abstract

The invention provides a live operation detection method and device. For each frame of image in a video stream of live operation, the following processing is executed: extracting an image sequence containing the current frame of image from the video stream; extracting time sequence characteristics used for representing the dynamic change of the target object from the image sequence; processing the extracted time sequence features by using a multi-layer cross fusion feature pyramid network to obtain pyramid time sequence features containing the current frame image; extracting global features and local features from the pyramid time sequence features, and through feature fusion, determining the probability that the current frame of image has live operation; and when it is detected that the probability is greater than a preset live probability threshold, outputting the current frame image. According to the method and the device, the detection precision of the fire operation is improved, the fire operation which is not approved and reported can be found in time, and the safety of the fire operation construction is improved, so that casualties and equipment loss are reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for detecting hot work. Background Art

[0002] Currently, hot work, such as welding and cutting, is a common operation in industrial production and construction activities. However, non-standard hot work (such as hot work in restricted areas) is a high-risk factor for fire accidents. Therefore, supervision of hot work can help reduce the risk of fire.

[0003] However, in complex and changing working environments, existing technologies have the following drawbacks when detecting local areas in a single image: 1) It fails to consider the temporal information of hot work activities and the surrounding visual information outside of the hot work in the image. This makes it easy to misidentify interference factors such as complex background lighting and fireworks as hot work activities. Therefore, it is not robust enough in complex working environments. 2) When using the feature pyramid network to extract multi-scale features of moving objects, only adjacent layers are considered. As a result, information dissipates as the number of layers increases, which does not effectively alleviate the problem of missed detection of small objects. 3) Hot work operations are diverse in form, resulting in very irregular shapes. The current method does not have a corresponding module to deal with this, resulting in inaccurate positioning of the hot work. Summary of the Invention In view of this, the purpose of the present application is to provide a hot work detection method and device to overcome at least one of the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting hot work operations, which performs the following processing on each frame image in a video stream of hot work operations: extracting an image sequence including a current frame image from the video stream; extracting temporal features for characterizing the dynamic changes of a target object from the image sequence; processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature, and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, and each feature in the first pyramid feature is processed. The feature layer is a time series feature, each feature layer in the second pyramid feature is a cross fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid time series feature; global features and local features are extracted from the pyramid time series features through a classification network, and the probability of hot work in the current frame image is determined through feature fusion; when it is detected that the probability is greater than a preset hot work probability threshold, the current frame image is output.

[0005] In an optional embodiment of the present application, a positioning network is used to extract shape features from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

[0006] In an optional embodiment of the present application, the specific location of the hot work in the current frame image is determined by: replacing the corresponding original convolution layer in the positioning network with multiple asymmetric deformable convolutions; through the asymmetric deformable convolution, the shape of the hot work in the current frame image is identified, the position parameters of the hot work in the current frame image are output, and the specific location of the hot work in the current frame image is determined and displayed in the form of a rectangular frame.

[0007] In an optional embodiment of the present application, the current frame image of the hot work location is determined in the following manner: judging whether the probability is greater than the preset hot work probability threshold; if the probability is greater than the preset hot work probability threshold, it is considered that there is hot work in the current frame image, and the position of the rectangular frame is determined according to the position parameters, and the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame; according to the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame, the specific location of the hot work is marked with a rectangular frame on the current frame image; and the current frame image of the hot work location is output.

[0008] In an optional embodiment of the present application, the probability that hot work exists in the current frame image is determined in the following manner: the pyramid temporal features are input into a convolutional layer, and global pooling features are extracted after the convolutional layer to determine the global features of the current frame image; the pyramid temporal features of the current frame image are divided into multiple local areas, and local features of each local area are obtained from the pyramid temporal features corresponding to the current frame image; for each local area, the global features are added to the local features of the local area, and feature fusion of the local features of the local area and the global features is performed; and the probability that hot work exists in each local area in the current frame image is output.

[0009] In an optional embodiment of the present application, the position parameters are determined in the following manner: an asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions; the shape of the hot work that meets the preset conditions in the current frame image is identified through the deformable convolution, and the position parameters of the hot work in the current frame image are output to calibrate the specific position of the hot work in the current frame image.

[0010] In an optional embodiment of the present application, an image sequence is determined in the following manner: determining a current frame image to be detected, a target number of frames before and after the current frame image in the extracted image sequence, and a total number of frames to be extracted; extracting each frame image read from the video stream to form a candidate image sequence, wherein the candidate image sequence corresponds one-to-one to the current frame image to be detected; for each frame image, determining whether the current frame image is a desired preset frame image, wherein the preset frame image is one of the following: the current frame image, one of images a target number of frames before the current frame, and one of images a target number of frames after the current frame; for each frame image, if the current frame image is the desired preset frame image, then adding the current frame image to the target image sequence, wherein the order of frames in the target image sequence remains consistent with that during extraction, and the frame at the middle position of the target image sequence is the current frame image; for each frame image, if the current frame image is not the desired preset frame image, then continuing to store the current frame image in the candidate image sequence; and when it is detected that the number of frames in the target image sequence meets the total number of frames, determining the target image sequence as an image sequence.

[0011] In a second aspect, an embodiment of the present application further provides a hot work detection device, the device comprising: an image sequence extraction module for extracting an image sequence including a current frame image from a video stream; a temporal feature extraction module for extracting temporal features for characterizing dynamic changes of a target object from the image sequence; a pyramid temporal feature determination module for processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, the multi-layer cross-fusion network comprising a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature comprising a preset number of feature layers, each feature in the first pyramid feature The layer is a time series feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid time series feature; a probability determination module is used to extract global features and local features from the pyramid time series features through a classification network, and determine the probability of the presence of hot work in the current frame image through feature fusion; a current frame image output module is used to output the current frame image when it is detected that the probability is greater than a preset hot work probability threshold. In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method described above are performed.

[0012] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are executed.

[0013] The hot work detection method and device provided in the embodiment of the present application performs the following processing on each frame image in the video stream of the hot work: extracting an image sequence containing the current frame image from the video stream; extracting temporal features for characterizing the dynamic changes of the target object from the image sequence; processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature containing the current frame image, extracting global features and local features from the pyramid temporal features, and determining the probability of the existence of hot work in the current frame image through feature fusion; outputting the current frame image when the probability is greater than a preset hot work probability threshold. Through the present application, by extracting temporal features for characterizing the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is hot work in the current frame image, extract global features and local features from the pyramid temporal features, and determine the probability of the existence of hot work in the current frame image through feature fusion, and comprehensively consider various information in the image to improve the reliability and stability of detection.

[0014] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flow chart of a hot work detection method provided in an embodiment of the present application; Figure 2 A flowchart for determining an image sequence provided in an embodiment of the present application; Figure 3 A flowchart for determining the probability of hot work occurring in the current frame image provided by an embodiment of the present application; Figure 4 A schematic diagram of a hot work detection method provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a hot work detection device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0018] First, the application scenarios to which this application is applicable are introduced. This application can be applied in the field of image processing.

[0019] Hot work, such as welding and cutting, is a common operation in industrial production and construction activities. It has the characteristics of generating sparks and high temperature, which increases the possibility of fire. When hot work is carried out in a confined area, if the operation is not standardized, such as failure to strictly abide by safety regulations and inadequate protective measures, it is very easy to cause fire accidents. The confined area may contain flammable and explosive items, complex lighting conditions, etc. These factors further increase the risk of fire caused by hot work.

[0020] Research has found that traditional monitoring methods rely on manual inspections, which are labor-intensive and prone to omissions. Irregular hot work operations are a high-risk factor in causing fire accidents. In complex and changing operating environments, existing technologies have problems such as high false alarm rates and insufficient recognition accuracy when detecting local areas in a single image, especially in complex and changing operating environments.

[0021] Based on this, an embodiment of the present application provides a method and device for detecting hot work, which performs the following processing on each frame image in a video stream of hot work: extracting an image sequence containing the current frame image from the video stream; extracting temporal features for characterizing the dynamic changes of the target object from the image sequence; processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature containing the current frame image, extracting global features and local features from the pyramid temporal features, and determining the probability of the presence of hot work in the current frame image through feature fusion; outputting the current frame image when the probability is greater than a preset hot work probability threshold. Through the present application, by extracting temporal features for characterizing the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is hot work in the current frame image, extract global features and local features from the pyramid temporal features, and determine the probability of the presence of hot work in the current frame image through feature fusion, and comprehensively consider various information in the image to improve the reliability and stability of detection.

[0022] See also Figure 1 , Figure 1 This is a flow chart of the hot work detection method provided in the embodiment of the present application. Figure 1 As shown in , the hot work detection method provided by the embodiment of the present application includes: S101: Extract an image sequence including a current frame image from a video stream.

[0023] Specifically, the image sequence is determined in the following way: See also Figure 2 , Figure 2 This is a flow chart for determining an image sequence provided by an embodiment of the present application. Figure 2 As shown in , the hot work detection method provided by the embodiment of the present application includes: S201, determining a current frame image to be detected, a target number of frames before and after the current frame image in an extracted image sequence, and a total number of frames to be extracted; Read the video stream and obtain multiple frames before and after the current frame according to the predefined current frame image, target frame number and total frame number, that is, the image sequence containing timing information. The current frame is recorded as It, and the extracted image sequence is [I t -n,I t -n+1, ​​..., I t ,……,I t +n-1,I t +n], where n is the target number of frames before and after the current frame image to be extracted.

[0024] For example, n=2, the current frame image is determined to be the 3rd frame, the target frame number is 2 frames, and the total frame number is 5 frames. Then, the image sequence of the 3rd frame is the 5 frames from the 1st frame image to the 5th frame image.

[0025] Specifically, we determine the current frame image to be detected, which is the starting point of the entire detection process. We also set the target number of frames before and after the current frame image to be extracted. This value determines the size of the time window we will observe in order to capture the dynamic changes of the hot work.

[0026] Based on the target number of frames, calculate the total number of frames to be extracted. The total number of frames is equal to the sum of the current frame and the target frames before and after it (if the current frame is calculated only once, the total number of frames is 2 × target number of frames + 1).

[0027] S202: Extract each frame of image read from the video stream to form a candidate image sequence.

[0028] Here, the candidate image sequence corresponds one-to-one to the current frame image to be detected.

[0029] The images in the video stream are read frame by frame to form a preliminary candidate image sequence. This sequence contains all the frames in the video stream, but will be filtered later according to needs.

[0030] Ensure that each frame in the candidate image sequence has a one-to-one correspondence with the current frame image to be detected, that is, each frame can be located at a specific position in the video stream.

[0031] S203 : For each frame image, determine whether the current frame image is a desired preset frame image.

[0032] The preset frame image is one of the following: the current frame image, one of images that are a target number of frames before the current frame, and one of images that are a target number of frames after the current frame.

[0033] Traverse each frame in the candidate image sequence to determine whether the frame image is the required preset frame image. The preset frame image includes the current frame image itself, one or more images of the target number of frames before the current frame, and one or more images of the target number of frames after the current frame.

[0034] The basis for judgment may be the frame sequence number, timestamp or other attributes that can uniquely identify the frame.

[0035] S204 : For each frame image, if the current frame image is a required preset frame image, add the current frame image to the candidate image sequence.

[0036] The order of frames in the candidate image sequence remains consistent with that during extraction, and the frame in the middle of the candidate image sequence is the current frame image.

[0037] S205 : For each frame image, if the current frame image is not the desired preset frame image, then the current frame image continues to be stored in the candidate image sequence.

[0038] Here, for frames that meet the preset conditions, they are added to the final image sequence, which will be used for subsequent hot work detection.

[0039] The order of frames in the candidate image sequence is kept unchanged to ensure the correct temporal order in the image sequence. In particular, the middle position of the image sequence should be the current frame image to reflect the time point we are mainly interested in.

[0040] For example, because the time of the first two frames of images is too short and not conducive to detection, they are ignored here. Starting detection from the 3rd frame image, the 6th frame is not the preset frame image in the image sequence corresponding to the 3rd frame image. If we detect the 4th frame image, the preset frame images in the image sequence corresponding to the 4th frame image can be the 2nd to 6th frames, and so on. Determine each frame image after the 3rd frame as the image sequence of the current frame image, so as to detect the hot work operation of each frame image.

[0041] S206 : When it is detected that the number of frames in the candidate image sequence meets the total number of frames, the candidate image sequence is determined as an image sequence.

[0042] When it is detected that the number of frames in the candidate image sequence meets the total number of frames, the candidate image sequence is determined as the final image sequence, and this image sequence is output for use by the subsequent hot work detection algorithm.

[0043] In an optional embodiment, factors such as the stability of the video stream, the consistency of the frame rate, and the image quality are also considered to ensure that the extracted image sequence can accurately reflect the actual situation of the hot work operation. If the video stream is long or the frame rate is high, the extracted image sequence may be very large. At this time, it may be necessary to consider downsampling or compressing the image to reduce storage and computing costs. In addition, in order to improve the accuracy and efficiency of detection, other technical means, such as motion detection, target tracking, etc., can be combined to further optimize the image sequence extraction process.

[0044] S102: extracting temporal features for characterizing dynamic changes of the target object from the image sequence.

[0045] First, extract the image sequence corresponding to the current frame image from the video stream. Based on the characteristics of hot work, select frames containing key dynamic information, such as the flame appearance, growth, and extinction process. Adjust each frame in the image sequence to a fixed size for the ResNet-50 model input. For example, the image resolution can be adjusted to 640x640. To increase the generalization ability of the model, perform data augmentation operations such as random cropping, rotation, and flipping on the image sequence.

[0046] Next, we load a pre-trained ResNet-50 model, which has been trained on a wide range of image classification tasks and has learned to extract high-level features from images. We then feed each frame in the image sequence into the ResNet-50 model and extract its feature vectors before the fully connected layer. These feature vectors contain high-level semantic information about the image, such as object shape and texture.

[0047] Since the dynamic features of hot work have obvious changes in time series, while static similar objects in the background (such as lights) are relatively stable, through temporal feature extraction, we can effectively reduce the interference of static similar objects in the background and improve the recognition accuracy of hot work.

[0048] In an optional embodiment, a sequence of images containing hot work and background interference can be used as training data and fed into the model for training. During the training process, the model parameters are updated by optimizing a loss function (such as cross-entropy loss). The model is then evaluated using a test dataset, calculating metrics such as accuracy and recall for identifying hot work. By adjusting the model structure and parameters, the model's performance can be continuously optimized.

[0049] Finally, the trained model is deployed into the monitoring system to process the image sequences in the video stream to achieve real-time detection and early warning of hot work. By inputting the image sequence into ResNet-50, the visual temporal features in the image sequence can be effectively extracted, the dynamic characteristics of hot work can be captured, and the interference of static similar objects in the background can be reduced.

[0050] Specifically, when using image sequences to detect hot work, extracting visual temporal features is a key step. By inputting the image sequence into a classic image classification model (such as ResNet-50), we can obtain temporal features. Combined with the temporal analysis method, we can effectively capture the dynamic characteristics of hot work, thereby more accurately identifying hot work behavior and reducing the interference of static similar objects (such as lights) in the background.

[0051] S103 , using a multi-layer cross-fusion feature pyramid network to process the extracted temporal features to obtain pyramid temporal features of the current frame image.

[0052] Here, the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a time series feature, each feature layer in the second pyramid feature is the sum of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the cross fusion of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid time series feature.

[0053] First, we extract temporal features from the image sequence. These features usually contain dynamic information of hot work, such as the flickering and spread of flames.

[0054] Each feature layer in the first pyramid feature corresponds to a layer in the temporal feature.

[0055] Each feature layer in the second pyramid is obtained by cross-fusing all feature layers in the first pyramid. This achieves a preliminary fusion of cross-layer features. However, this simple summation operation may not fully utilize the complementary information between different layers. Therefore, we introduce a cross-fusion strategy. Based on cross-fusion, specific fusion operations (such as convolution and attention mechanisms) are used to further enhance the fusion effect of features. The cross-fused feature layer will contain information from all feature layers in the first pyramid, but it will be more compact and efficient.

[0056] The third pyramid feature is obtained by summing the first pyramid feature and the second pyramid feature, which further enhances the degree of feature fusion, so that each feature layer in the third pyramid feature contains the information of all feature layers in the first pyramid feature and the second pyramid feature. Similarly, we can introduce a cross-fusion strategy to optimize this fusion process.

[0057] The fourth pyramid feature is constructed in a similar way to the second pyramid feature, but it is based on the third pyramid feature. Each feature layer in the fourth pyramid feature is obtained by cross-fusion of all feature layers in the third pyramid feature. In this way, the feature layer in the fourth pyramid feature will contain information of all feature layers in the third pyramid feature (and indirectly the first pyramid feature and the second pyramid feature).

[0058] The fifth pyramid feature is obtained by summing the third pyramid feature and the fourth pyramid feature. As the final pyramid time series feature, the fifth pyramid feature will contain the information in all the previous pyramid features, and due to the existence of multi-layer cross fusion, this information will be more fully utilized and integrated.

[0059] Specifically, element-wise feature fusion is employed during the pyramid feature construction process. Specifically, after obtaining the second pyramid feature, we perform element-wise additive feature fusion with the original time series feature, the first pyramid feature, to obtain the third pyramid feature. Similarly, after obtaining F4, we perform element-wise additive feature fusion with F3 to obtain the fifth pyramid feature. This operation not only enhances the feature fusion effect but also makes the entire network module plug-and-play. Furthermore, this module can be reused indefinitely, and the number of pyramid feature layers and fusion method can be adjusted according to specific task requirements.

[0060] In neural networks, addition operations can be used in residual connections, such as the shortcut connections in ResNet. This connection allows the network to directly pass the output of the previous layer to the next layer, fusing the features with the output of the next layer. This helps alleviate the vanishing gradient problem in deep networks. Element-wise operations are simple, efficient, and easy to implement. Because they operate element by element, they can fully utilize the parallel computing power of modern computing devices to accelerate the process. Element-wise operations are also highly scalable and can be easily applied to input tensors of varying sizes.

[0061] Compared with the current mainstream method that only considers the fusion method of adjacent layer features, the multi-layer cross-fusion pyramid feature extraction method proposed in this application uses multi-layer cross-fusion, and each layer of features contains the feature information of other layers, thereby enhancing the robustness and expressiveness of the features. The multi-layer cross-fusion strategy helps to retain and fuse the complementary information between different layers, avoiding the information dissipation problem during feature fusion. It can be plug-and-play, and the number of layers and fusion methods of the pyramid features can be adjusted according to specific tasks, and it has strong scalability.

[0062] The multi-layer cross-fusion network of this application has broad application prospects in the fields of hot work detection, target tracking, video analysis, etc. By extracting the pyramid features of multi-layer cross-fusion, we can more accurately capture the dynamic characteristics of hot work and reduce the impact of size changes on the detection effect, thereby improving the recognition accuracy and robustness of hot work. Compared with the current mainstream method that only considers the fusion method of adjacent layer features, the fusion is more complete, alleviating the problem of information dissipation during feature fusion in the current method.

[0063] S104: extracting global features and local features from the pyramid temporal features through a classification network, and determining the probability of hot work occurring in the current frame image through feature fusion.

[0064] See also Figure 3 , Figure 3 This is a flow chart for determining the probability of hot work in the current frame image provided by the embodiment of the present application. Figure 3 As shown in , the hot work detection method provided by the embodiment of the present application includes: S301: Input the pyramid temporal features into the convolution layer, and extract the global features of the current frame image through global pooling feature extraction after the convolution layer.

[0065] One or more layers of images in the pyramid temporal features are input into the convolution layer. The convolution layer extracts the spatial features of the image through convolution operations and activation functions. The size of the feature map is further reduced through the pooling layer (such as maximum pooling or average pooling), thereby extracting global features. These global features contain the overall structure and key information of the image.

[0066] Preferably, in the classification network, a visual context feature extraction module is added after each convolutional layer, that is, global max pooling is used in the channel dimension to extract the global features of the image based on the features obtained by the convolutional layer.

[0067] S302: Divide the pyramid temporal features of the current frame image into multiple local regions, and obtain local features of each local region from the pyramid temporal features corresponding to the current frame image.

[0068] That is, each local area is the area covered by the 3x3 convolution with that position as the center. Local features refer to the features of a specific area in an image, which are usually used to capture the details and local changes of the image.

[0069] Here, the current frame image is divided into multiple local regions, each of which can be regarded as an independent image block. For each local region, the corresponding local features are extracted from the pyramid temporal features. These local features describe the visual information within the region.

[0070] S303 : For each local area, add the global feature to the local feature of the local area to perform feature fusion of the local feature of the local area and the global feature.

[0071] Feature fusion is the process of combining global features and local features. It aims to fully utilize the advantages of both features and improve recognition accuracy.

[0072] The global features are added to the local features of each local area to achieve the fusion of local and global features. This addition operation can be element-by-element or weighted addition. After the feature addition, the fused total features are obtained. These total features contain both global information and local details, thus better describing the content of the image.

[0073] The global feature is added to the feature vector of each position to achieve the fusion of the feature of the current position and the global feature. However, this global feature mainly contains the semantic information of the scene and lacks local detail information. During hot work operations, the hot work area often has an important relationship with the surrounding local areas. For example, there are welding or cutting tools, workers with protective masks, or automated equipment around the hot work. Therefore, it is necessary to better model and fuse the global features with the local features.

[0074] S304: Output the probability of hot work occurring in each local area of ​​the current frame image.

[0075] By repeating the feature fusion, convolution is finally used to output the probability of the presence of hot work at each position in the current frame image.

[0076] In summary, extracting global and local features through pyramid temporal features and performing feature fusion can effectively improve the accuracy of hot work identification. This method combines the advantages of multi-scale analysis and global-local feature fusion, providing an effective solution for target detection in complex scenarios.

[0077] S105: When the detected probability is greater than a preset fire probability threshold, output the current frame image.

[0078] Here, it is determined whether the probability is greater than the preset fire probability threshold; If the probability is greater than the preset fire probability threshold, it is considered that there is fire operation in the current frame image, and the current frame image is output; if the probability is not greater than the preset fire probability threshold, it is considered that there is no fire operation in the current frame image, and the next frame image is used as the current frame image. The image sequence corresponding to the current frame image is obtained and a new round of detection is performed.

[0079] Specifically, the positioning network can also be used to extract deformation-robust features from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

[0080] In an optional embodiment, the corresponding original convolution layer in the localization network is replaced with multiple asymmetric deformable convolutions; In the pyramid temporal feature extraction stage, we use traditional convolutional layers to capture the spatial features of the image. In order to introduce deformable convolution, we need to convert the original convolutional layer, which usually involves adjusting parameters such as the size of the convolution kernel, stride, and padding to ensure that the converted convolutional layer is compatible with the deformable convolutional layer.

[0081] Deformable convolution is a convolution operation that can adaptively adjust the sampling position. It allows the convolution kernel to sample in a non-uniform manner on the input feature map. Compared with traditional convolution, deformable convolution can more flexibly capture geometric deformation and shape changes in images, which is especially important for identifying targets with complex shapes and dynamic changes such as hot work.

[0082] In an optional embodiment, in the hot work positioning network, the original 3x3 convolution in YOLOv8 is replaced by 1x3, 3x3, and 3x1 deformable convolutions, because the deformable convolution can automatically adjust the receptive field according to the shape of the object and has a certain adaptability to objects with large shape changes.

[0083] Preferably, YOLOv8 is a newer version of the YOLO (You Only Look Once) object detection and image segmentation model, which is a SOTA (State Of The Art) model.

[0084] The 3x3 convolution used in the original YOLOv8 has been replaced with 1x3, 3x3, and 3x1 deformable convolutions. Deformable convolution is a convolution method that can automatically adjust the receptive field according to the shape of the object. Therefore, it has good adaptability to objects with large shape changes. In the positioning of hot work operations, since the shape of hot work often varies greatly, the use of deformable convolution can better capture the characteristics of hot work.

[0085] Asymmetric deformable convolutions (such as 1x3 and 3x1) are introduced to better adapt to images with different aspect ratios. The 1x3 convolution kernel can capture feature changes in the horizontal direction, while the 3x1 convolution kernel focuses on feature changes in the vertical direction. By combining these two asymmetric convolution kernels, the network can more accurately identify the shape changes of images in different directions.

[0086] The original convolutional layer corresponding to the pyramid temporal features is converted into a convolutional layer, that is, the traditional 3x3 convolution is replaced with the above-mentioned deformable convolution. This conversion process enables the network to handle the shape changes of hot work more flexibly.

[0087] Through asymmetric deformable convolution, the shape of the hot work in the current frame image is recognized, the position parameters of the hot work in the current frame image are output, and the specific position of the hot work in the current frame image is determined in the form of a rectangular frame.

[0088] The position parameters are determined as follows: Asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions. Through deformable convolution, the shape of the hot work that meets the preset conditions in the current frame image is identified, and the position parameters of the hot work in the current frame image are output to calibrate the specific location of the hot work in the current frame image.

[0089] Here, through asymmetric deformable convolution, we can more accurately capture the shape features of hot work, including key information such as the shape, size, and color of the flame. Based on the results of shape recognition, we output the position parameters of the hot work in the current frame image. These parameters usually include the coordinates of the center point of the hot work, the length and width of the rectangular box, etc.

[0090] Through the transformed deformable convolutional layer, the shape of the hot work that meets the preset conditions in the current frame image is identified, and the network outputs the position parameters of the hot work. These position parameters include the coordinates of the center point of the rectangular box, the length and width of the rectangular box. These position parameters are used to calibrate the specific location of the hot work in the image.

[0091] The current frame image of the frame fire operation position is determined by the following method: Determine whether the probability is greater than the preset fire probability threshold; If the probability is greater than the preset fire probability threshold, it is considered that there is fire operation in the current frame image, and the position of the rectangular frame is determined according to the position parameters. Here, the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame.

[0092] First, we determine whether the probability of hot work in the current frame image is greater than the preset hot work probability threshold based on the output of the classification layer. If the probability is greater than the threshold, it is considered that hot work exists in the current frame image. At this time, we determine the position of the rectangular box based on the position parameters. The position of the rectangular box is determined by the center point coordinates, length, and width.

[0093] The specific location of the hot work is marked with a rectangular frame on the current frame image according to the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame.

[0094] Output the current frame image of the fire operation position.

[0095] Based on the determined location parameters, we use a rectangular box to mark the specific location of the hot work on the current frame image. This usually involves drawing a rectangular box with a specified length and width based on the center point on the image.

[0096] Finally, we will output the current frame image of the fire operation location, which can be used in application scenarios such as real-time monitoring and alarm prompts.

[0097] If the probability is not greater than the preset fire probability threshold, it is considered that there is no fire operation in the current frame image and no output is required, or the image obtained from the video stream is directly output.

[0098] Applying the above method to video surveillance systems can achieve real-time monitoring and early warning of hot work operations. By analyzing the location of hot work operations in historical frame images, the spatiotemporal distribution characteristics of hot work operations can be extracted to provide data support for safety management. Combined with other sensor data (such as temperature, smoke concentration, etc.), a more intelligent alarm system can be implemented to improve the accuracy and timeliness of fire warnings.

[0099] In summary, by introducing deformable convolution and position parameter output, we can more accurately identify the shape and location of hot work and mark it with a rectangular box in the current frame image. This provides powerful technical support for the monitoring and management of hot work.

[0100] See also Figure 4 , Figure 4 This is a schematic diagram of the hot work detection method provided in the embodiment of the present application. Figure 4 As shown in , the schematic diagram includes an image sequence 41, temporal features 42, a multi-layer cross fusion network 43, a classification network 44, a positioning network 45 and an output image 46.

[0101] like Figure 4 In the figure, the image sequence 41 is input into the classic image classification model ResNet-50 for temporal feature extraction to obtain temporal features 42. The temporal features 42 are cross-fused through a multi-layer cross-fusion network 43. Here, F1 corresponds to the first pyramid feature, F2 corresponds to the second pyramid feature, F3 corresponds to the third pyramid feature, F4 corresponds to the fourth pyramid feature, and F5 corresponds to the fifth pyramid feature.

[0102] The “2×” in the figure indicates that the corresponding modules in the classification network and the localization network need to be repeated twice, and the output results after the two processings of the classification network 44 and the localization network 45 are summarized to obtain the output image 46.

[0103] like Figure 4As shown, in the dotted box corresponding to the classification network 44, the input is the convolution layer 440, the parameters of the Conv of the convolution layer 440 are "k=3, p=1, s=1", the convolution kernel size is (k=3), and the convolution kernel (also called the filter) is the core component of the convolution layer. It is used to slide on the input data (such as an image) and perform convolution operations to extract features. Figure 4 In the example, the size of the convolution kernel is 3x3, which means that the convolution kernel is a matrix with 3 rows and 3 columns. The elements in this matrix (also called weights) will be optimized by the backpropagation algorithm during the training process.

[0104] Padding size (p=1), padding refers to adding extra pixel values ​​(usually zero) around the input data to control the size of the output feature map, Figure 4 In the example, the padding size is 1, which means that an extra circle of pixel values ​​is added to each boundary of the input data. This ensures that the convolution kernel can cover the edge of the input data, thereby avoiding the loss of edge information during the convolution process.

[0105] Step size (s=1), which refers to the number of pixels that the convolution kernel moves each time it slides on the input data. Figure 4 In the example, the stride is 1, which means that the convolution kernel moves only one pixel each time it slides. This ensures that the convolution kernel can traverse every position of the input data and extract as many features as possible.

[0106] The parameters "k=3, p=1, s=1" for the convolutional layer (Conv) indicate that the layer uses a 3x3 convolution kernel, adds padding around the input data, and slides the kernel one pixel at a time. These parameters together determine how the convolutional layer extracts features from the input data.

[0107] The classification network 44 successively uses global max pooling 441, non-local attention mechanism 442, and channel attention 443 in CBAM to obtain a convolutional layer 444. The parameters of Conv in the convolutional layer 444 are "k=1, p=0, s=1" to fully integrate the local features around the features at the current position with the global features of the image, capture the feeling of dynamic and surrounding features, improve robustness, and reduce the interference of similar local objects in the background.

[0108] Preferably, in the dotted box corresponding to the positioning network 45, the input is three depthwise convolutional layers 451, 452 and 453 with different convolution kernel sizes. The depthwise convolutional layer 453DConv (Depthwise ConvolutionalLayer) represents the convolution kernel size of the deformable convolution layer (k=3×1): this means that the convolution kernel is a matrix with 3 rows and 1 column, or more accurately, it is a two-dimensional filter with 3 heights and 1 width. Convolution kernels of this shape are usually used to process data with a specific spatial structure, such as images. The convolution kernel sizes of the deformable convolution layers 451 and 452 can be (k=1×3) and (k=3×3), respectively. The specific explanation is the same as that of the depthwise convolution layer 453 and will not be repeated here.

[0109] In the classification network 44 and the localization network 45, the outputs are convolutional layers 444 (Con2d, k=1, p=0, s=1) and 454 (Con2d, k=1, p=0, s=1). The convolution kernel size (k=1) is 1x1, meaning the kernel is a 1-row, 1-column matrix. This kernel size is often used to change the number of channels in the input data (i.e., perform cross-channel feature fusion or transformation) without changing the spatial dimensions of the input data. The padding size (p=0) is 0, meaning no padding is applied to the input data. Therefore, the kernel slides directly over the original input data without considering boundary effects. The stride (s=1) is 1, meaning the kernel moves only one pixel at a time. This ensures that the kernel traverses every position in the input data. However, since the kernel size is 1x1, the output feature map is effectively the same size as the input data (ignoring padding).

[0110] In summary, the Conv2D (Conv2D) parameters for both convolutional layers 444 and 454 are "k=1, p=0, s=1," indicating that the convolutional layer uses a 1x1 kernel, does not pad the input data, and slides the kernel one pixel at a time. This convolutional layer configuration is typically used to change the number of input channels or perform cross-channel feature fusion without changing the spatial size of the input data.

[0111] This application proposes a feature pyramid network (FPN) based on the cross-fusion of multiple layers of features. The feature pyramid network is a technical means for target detection, which aims to capture targets of different sizes through feature maps of different scales. It uses different layers of convolutional neural networks (CNN) to extract features of different scales and fuse these features to improve the performance of target detection, so that each layer of features contains the semantic information of the remaining layers, reducing the information dissipation problem of the current feature pyramid network during feature fusion, and improving the model's ability to detect multi-scale hot work, especially the detection effect of small-scale hot work.

[0112] This application proposes a hot work context feature extraction module based on global and local feature attention weighting, and integrates it into the hot work classification network 44 to achieve modeling of the relationship between objects related to the hot work, improve the robustness of the model, and reduce false detections caused by complex lighting, etc.

[0113] This application proposes a positioning network 45 for dynamic fire operations based on asymmetric deformable convolution, which automatically adjusts the receptive field according to the different shapes of dynamic fire, so that the parameters fall on the dynamic fire area, and then extracts more effective features, thereby improving the positioning accuracy of the model for dynamic fire operations with large shape changes.

[0114] The embodiment of the present application provides a method and device for detecting hot work, which performs the following processing on each frame image in a video stream of hot work: extracting an image sequence containing the current frame image from the video stream; extracting temporal features for characterizing the dynamic changes of the target object from the image sequence; processing the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature containing the current frame image, extracting global features and local features from the pyramid temporal features, and determining the probability of the current frame image containing hot work through feature fusion; outputting the current frame image when the probability is greater than a preset hot work probability threshold. Through the present application, by extracting temporal features for characterizing the dynamic changes of the target object and processing them using a multi-layer cross-fusion network, it is possible to accurately identify whether there is hot work in the current frame image, promptly discover hot work that has not been approved and reported, extract global features and local features from the pyramid temporal features, and determine the probability of the current frame image containing hot work through feature fusion, which can comprehensively consider multiple information in the image and improve the reliability and stability of detection.

[0115] The present invention obtains the timing information in the video stream by inputting a sequence of pictures, captures the dynamic characteristics of the fire during the fire operation, and introduces visual context features in the classification network. Based on the attention mechanism, it realizes the weighted fusion of local features and global features, and extracts the relationship between the fire and the surrounding objects. By extracting the above timing features and context features, the interference of similar objects in the background can be effectively reduced, and the accuracy of fire detection can be improved. In the positioning network, the introduction of asymmetric deformable convolution can automatically adjust the size and shape of the receptive field for different shapes of fire, so as to achieve a better positioning effect for the fire.

[0116] Here, in convolutional neural networks (CNNs), the receptive field refers to the size of the input image region that a specific neuron in a certain layer of the network can receive. In CNNs, the receptive field gradually increases through the stacking of convolutional and pooling layers. This means that deeper layers in the network can capture information from larger input regions, which is crucial for detecting larger patterns or objects in images. The size of the receptive field has a significant impact on both classification and object detection tasks. For classification tasks, the receptive field size of the feature map in the last layer must be greater than or equal to the input image size; otherwise, the classification performance may be unsatisfactory. For object detection tasks, the receptive field needs to match the size of the target to ensure that the model can accurately detect the target.

[0117] In summary, it is possible to better classify and locate hot work fires generated during hot work operations, thereby improving the detection effect of hot work operations.

[0118] Based on the same inventive concept, an embodiment of the present application also provides a hot work detection device corresponding to the hot work detection method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned hot work detection method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0119] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of the hot work detection device provided in the embodiment of the present application. Figure 5 As shown in , the hot work detection device 500 includes: An image sequence extraction module 501 is used to extract an image sequence including a current frame image from a video stream; A temporal feature extraction module 502 is configured to extract temporal features for characterizing dynamic changes of a target object from the image sequence; A pyramid temporal feature determination module 503 is configured to process the extracted temporal features using a multi-layer cross-fusion feature pyramid network to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature, and a fifth pyramid feature, each pyramid feature including a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature; The probability determination module 504 is configured to extract global features and local features from the pyramid temporal features through a classification network, and determine the probability of hot work occurring in the current frame image through feature fusion; The current frame image output module 505 is configured to output the current frame image when it is detected that the probability is greater than a preset fire probability threshold.

[0120] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 6 As shown in FIG, the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

[0121] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figure 1 The specific implementation of the steps of the hot work detection method in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0122] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps of the hot work detection method in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0123] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for detecting hot work, characterized in that: The following processing is performed on each frame of the hot work video stream: Extracting an image sequence including a current frame image from the video stream; Extracting temporal features for characterizing dynamic changes of a target object from the image sequence; Using a multi-layer cross-fusion feature pyramid network, the extracted temporal features are processed to obtain a pyramid temporal feature of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature, and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is the cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature; Extracting global features and local features from the pyramid time series features through a classification network, and determining the probability of hot work in the current frame image through feature fusion; When it is detected that the probability is greater than a preset fire probability threshold, the current frame image is output.

2. The method according to claim 1, characterized in that The method further comprises: Through the positioning network, shape features are extracted from the pyramid temporal features using asymmetric deformable convolution to determine the specific location of the fire in the current frame image.

3. The method according to claim 2, characterized in that Determine the specific location of the fire in the current frame image by the following method: Replacing the corresponding original convolutional layer in the positioning network with multiple asymmetric deformable convolutions; Through the asymmetric deformable convolution, the shape of the hot work in the current frame image is identified, the position parameters of the hot work in the current frame image are output, and the specific position of the hot work in the current frame image is determined in the form of a rectangular frame.

4. The method according to claim 3, characterized in that The current frame image of the frame fire operation position is determined by the following method: Determine whether the probability is greater than the preset fire probability threshold; If the probability is greater than the preset hot work probability threshold, it is considered that there is hot work in the current frame image, and the position of the rectangular frame is determined according to the position parameters, where the position parameters are the coordinates of the center point of the rectangular frame, the length of the rectangular frame, and the width of the rectangular frame; According to the coordinates of the center point of the rectangular frame, the length of the rectangular frame and the width of the rectangular frame, a specific position of the hot work is marked with a rectangular frame on the current frame image; Output the current frame image of the fire operation position.

5. The method according to claim 1, characterized in that The probability of hot work in the current frame image is determined by the following method: Inputting the pyramid temporal features into a convolutional layer, and extracting the global features of the current frame image by global pooling after the convolutional layer; Dividing the pyramid temporal features of the current frame image into a plurality of local regions, and obtaining local features of each local region from the pyramid temporal features corresponding to the current frame image; For each local area, the global feature is added to the local feature of the local area to perform feature fusion of the local feature of the local area and the global feature; Output the probability of hot work existing in each local area of ​​the current frame image.

6. The method according to claim 3, characterized in that The position parameters are determined by: Asymmetric deformable convolution is introduced to capture the feature changes of the current frame image in the horizontal and vertical directions; Through the deformable convolution, the hot work shape that meets the preset conditions in the current frame image is identified, and the position parameters of the hot work in the current frame image are output to calibrate the specific position of the hot work in the current frame image.

7. The method according to claim 1, characterized in that The image sequence is determined by: Determine the current frame image to be detected, the target number of frames before and after the current frame image in the extracted image sequence, and the total number of frames to be extracted; Extracting each frame of image read from the video stream to form a candidate image sequence, wherein the candidate image sequence corresponds to the current frame image to be detected one by one; For each frame image, determine whether the current frame image is a desired preset frame image, wherein the preset frame image is one of the following: the current frame image, one of the images of a target number of frames before the current frame, and one of the images of a target number of frames after the current frame; For each frame image, if the current frame image is the desired preset frame image, the current frame image is added to the target image sequence, the order of the frames in the target image sequence is consistent with that during extraction, and the frame in the middle position of the target image sequence is the current frame image; For each frame image, if the current frame image is not the required preset frame image, the current frame image continues to be stored in the candidate image sequence; When it is detected that the number of frames in the target image sequence meets the total number of frames, the target image sequence is determined as an image sequence.

8. A hot work detection device, characterized in that: The device comprises: An image sequence extraction module is used to extract an image sequence including a current frame image from a video stream; A time series feature extraction module, used to extract time series features for characterizing dynamic changes of a target object from the image sequence; A pyramid temporal feature determination module is used to process the extracted temporal features by using a multi-layer cross-fusion feature pyramid network to obtain the pyramid temporal features of the current frame image, wherein the multi-layer cross-fusion network includes a first pyramid feature, a second pyramid feature, a third pyramid feature, a fourth pyramid feature and a fifth pyramid feature, each pyramid feature includes a preset number of feature layers, each feature layer in the first pyramid feature is a temporal feature, each feature layer in the second pyramid feature is a cross-fusion of all feature layers in the first pyramid feature, the third pyramid feature is the sum of the first pyramid feature and the second pyramid feature, each feature layer in the fourth pyramid feature is a cross-fusion of all feature layers in the third pyramid feature, the fifth pyramid feature is the sum of the third pyramid feature and the fourth pyramid feature, and the fifth pyramid feature is a pyramid temporal feature; A probability determination module is used to extract global features and local features from the pyramid time series features through a classification network, and determine the probability that a hot work exists in the current frame image through feature fusion; The current frame image output module is used to output the current frame image when it is detected that the probability is greater than a preset fire probability threshold.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.

Images