Fire detection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a fire detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: inputting a to-be-detected image into a residual network of a fire detection model for feature extraction to obtain feature data of the to-be-detected image, inputting the feature data of the to-be-detected image into an encoder of the fire detection model for feature extraction to obtain a target feature extraction result, and determining a fire detection result based on the target feature extraction result. According to the embodiment of the invention, the number of times of feature extraction of the high-level feature data in the encoder is greater than the number of times of feature extraction of the low-level feature data in the encoder, so that the number of times of feature extraction of the low-level feature data is reduced, the calculation amount of a fire detection model can be reduced while the model detection precision is maintained, and the fire detection efficiency is improved. The operation speed of the model is improved, and the requirements of high calculation and low storage of edge calculation equipment are met, so that the fire detection model can be deployed on the edge calculation equipment.

Description

Technical Field

[0001] The present application relates to the technical field of fire detection, and in particular to a fire detection method, device, computer equipment, storage medium and program product. Background Art

[0002] In contemporary society, fire safety is extremely critical, whether in industrial or residential areas. For power transmission systems, fires may not only cause significant property losses and interrupt power supply, but in severe cases, they may also endanger personnel safety. Therefore, it is crucial to be able to detect and warn of potential fire risks in a timely manner.

[0003] However, traditional fire detection models have high requirements for computing resources and are not suitable for deployment on edge computing devices with limited resources. Therefore, how to perform fire detection on edge computing devices has become a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] Based on this, it is necessary to provide a fire detection method, device, computer equipment, storage medium and program product that can perform fire detection on edge computing devices to address the above technical problems.

[0005] In a first aspect, the present application provides a fire detection method. The method comprises:

[0006] Inputting the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data;

[0007] Inputting the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder;

[0008] The fire detection result is determined based on the target feature extraction result.

[0009] In one embodiment, the encoder includes a plurality of deformable attention modules, which input feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result, including:

[0010] Inputting the feature data of the image to be detected into the multiple deformable attention modules in sequence, and performing feature extraction on the high-level feature data and / or low-level feature data in the feature data to obtain the target feature extraction result;

[0011] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0012] In one embodiment, the plurality of deformable attention modules include the M deformable attention modules and the N deformable attention modules, the feature data of the image to be detected is sequentially input into the plurality of deformable attention modules, and the high-level feature data and / or low-level feature data in the feature data are subjected to feature extraction to obtain the target feature extraction result, including:

[0013] Inputting the feature data of the image to be detected into the N deformable attention modules in sequence, performing N feature extractions on the high-level feature data in the feature data to obtain a first intermediate feature extraction result;

[0014] The first intermediate feature extraction result is sequentially input into the M deformable attention modules, and the low-level feature data in the first intermediate feature extraction result is subjected to M feature extractions to obtain the target feature extraction result.

[0015] In one embodiment, the feature data of the image to be detected is sequentially input into the multiple deformable attention modules, and the high-level feature data and / or low-level feature data in the feature data are subjected to feature extraction to obtain the target feature extraction result, including:

[0016] The feature data is sequentially input into the M deformable attention modules, and feature extraction is performed M times on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result;

[0017] The second intermediate feature extraction result is sequentially input into NM deformable attention modules, and NM feature extractions are performed on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0018] In one embodiment, the feature data of the image to be detected is sequentially input into the multiple deformable attention modules, and feature extraction is performed on the high-level feature data and / or low-level feature data in the feature data, including:

[0019] Determine a target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected;

[0020] Feature extraction is performed based on the target sampling point.

[0021] In one embodiment, determining the target sampling point according to the feature data of the image to be detected includes:

[0022] Determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected;

[0023] According to the sampling range and the offset, the target sampling point is determined based on the reference point; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

[0024] In a second aspect, the present application also provides a fire detection device. The device comprises:

[0025] A first feature extraction module is used to input the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data;

[0026] A second feature extraction module is used to input the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder;

[0027] The determination module is used to determine the fire detection result based on the target feature extraction result.

[0028] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0031] The fire detection method, apparatus, computer equipment, storage medium and program product described above input the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected, input the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain the target feature extraction result, and the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder, and finally determine the fire detection result based on the target feature extraction result. Since in the embodiment of the present application, the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder, the number of feature extractions of the low-level feature data is reduced, thereby being able to reduce the computational complexity of the fire detection model while maintaining the detection accuracy of the model, improve the running speed of the model, and meet the high computation and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0033] FIG2 is a flow chart of a fire detection method provided in an embodiment of the present application;

[0034] FIG3 is a schematic diagram of a flow chart of a method for obtaining target feature extraction results provided in an embodiment of the present application;

[0035] FIG4 is a schematic diagram of a feature extraction method provided in an embodiment of the present application;

[0036] FIG5 is a flow chart of another method for obtaining target feature extraction results provided in an embodiment of the present application;

[0037] FIG6 is a flow chart of a feature extraction method provided in an embodiment of the present application;

[0038] FIG7 is a schematic diagram of a flow chart of a method for determining a target sampling point provided in an embodiment of the present application;

[0039] FIG8 is a schematic diagram of the structure of a deformable attention module provided in an embodiment of the present application;

[0040] FIG9 is a flow chart of a lightweight fire smoke detection method for edge devices provided in an embodiment of the present application;

[0041] FIG. 10 is a structural block diagram of a fire detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] In contemporary society, fire safety is extremely critical, whether in industrial or residential areas. For power transmission systems, fires may not only cause significant property losses and interrupt power supply, but in severe cases, they may also endanger personnel safety. Therefore, it is crucial to be able to detect and warn of potential fire risks in a timely manner.

[0044] However, traditional fire detection models have high requirements for computing resources and are not suitable for deployment on edge computing devices with limited resources. Therefore, how to perform fire detection on edge computing devices has become a technical problem that needs to be solved urgently in this field.

[0045] The fire detection method provided in the embodiment of the present application can be applied in the application environment as shown in Figure 1. Figure 1 is an internal structure diagram of a computer device provided in the embodiment of the present application. The computer device can be a server, and its internal structure diagram can be as shown in Figure 1. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fire detection method is implemented.

[0046] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0047] In one embodiment, as shown in FIG. 2 , FIG. 2 is a flow chart of a fire detection method provided in an embodiment of the present application. The method can be applied to the computer device in FIG. 1 . The method includes the following steps:

[0048] S201, inputting the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected.

[0049] The feature data includes high-level feature data and low-level feature data.

[0050] In a possible implementation, the fire detection model can be obtained in the following manner: the fire detection data set is divided into 70% training set, 15% validation set and 15% test set, and then appropriate model hyperparameters are set, such as learning rate, batch size, training cycle, etc., and the initial fire detection model is trained based on the training set, while monitoring the performance of the initial fire detection model on the validation set to adjust the hyperparameters and avoid overfitting. The performance of the model is then evaluated on the test set, and the performance of the fire detection model is evaluated using indicators such as accuracy, recall rate, and F1 score, and the model is fine-tuned according to the results of the performance evaluation to obtain the fire detection model.

[0051] Exemplarily, the fire detection dataset can be obtained in the following ways: obtain images and video resources related to fire smoke through network search engines, professional databases, and social media platforms, and then filter the images and video resources according to image clarity, scene diversity, smoke density, and lighting conditions, and convert the filtered images and video resources into a unified format (such as JPEG, PNG, MP4) to obtain an initial fire image. Then, the initial fire image is input into the generative adversarial network to obtain the fire generated image output by the generative adversarial network, and finally the initial fire image and the fire generated image are annotated to obtain the fire detection dataset.

[0052] The generative adversarial network can be, for example, a CycleGAN network, which consists of two pairs of generators and discriminators, namely generators gen_AB and gen_BA, and discriminators dis_A and dis_B. The generator consists of downsampling layers and upsampling layers, specifically including 6 downsampling layers for gradually reducing the spatial dimension of the image and increasing the number of channels to capture the deep features of the image, and 6 upsampling layers for gradually restoring the spatial dimension of the image while reducing the number of channels to reconstruct the image of the target domain. In the upsampling process, a jump connection is used to combine the features of the downsampling layer with the features of the upsampling layer to retain more image details. The discriminator includes two downsampling layers for feature extraction, and maps the features to a single probability value through the convolution of the last layer, indicating the possibility that the image is a real image.

[0053] In an embodiment of the present application, the residual network may be, for example, a ResNet50 network, which may be used to extract feature data from an image to be detected, and divide the extracted feature data into high-level feature data and low-level feature data.

[0054] S202, inputting feature data of the image to be detected into an encoder of a fire detection model for feature extraction to obtain a target feature extraction result.

[0055] The number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder.

[0056] In the embodiment of the present application, low-level feature data usually contains too much unnecessary information, while high-level feature data usually carries richer semantic information and is less in number. Therefore, the number of feature extraction times for low-level feature data is reduced, and the number of feature extraction times for high-level feature data is made greater than that for low-level feature data, thereby optimizing the performance and efficiency of the fire detection model.

[0057] In a possible implementation, if there are M+N deformable attention modules in the encoder, the feature data of the image to be detected can be sequentially input into the N deformable attention modules of the encoder, and the high-level feature data in the feature data can be subjected to N feature extractions to obtain a first intermediate feature extraction result, and then the first intermediate feature extraction result can be sequentially input into the M deformable attention modules of the encoder, and the low-level feature data in the first intermediate feature extraction result can be subjected to M feature extractions to obtain a target feature extraction result, where M is less than N.

[0058] Alternatively, if there are N deformable attention modules in the encoder, the feature data can be input into the M deformable attention modules of the encoder in sequence, and feature extraction is performed M times on the high-level feature data and low-level feature data in the feature data to obtain a second intermediate feature extraction result, and then the second intermediate feature extraction result is input into the NM deformable attention modules of the encoder in sequence, and feature extraction is performed NM times on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0059] S203: Determine a fire detection result based on the target feature extraction result.

[0060] In a possible implementation, the target feature extraction result may be classified based on a classifier, and the classification result of the classifier is used as the fire detection result.

[0061] Exemplarily, the classification result of the classifier may be, for example, a probability value of the existence of a fire in the image to be detected.

[0062] Optionally, a preset threshold may be set, and when the fire detection result exceeds the preset threshold, prompt information is generated to indicate through the prompt information that a fire may exist in the area shown in the image to be detected.

[0063] In the embodiment of the present application, the image to be detected is input into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected, and the feature data of the image to be detected is input into the encoder of the fire detection model for feature extraction to obtain the target feature extraction result, and the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder, and finally the fire detection result is determined based on the target feature extraction result. Since in the embodiment of the present application, the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder, the number of feature extractions of the low-level feature data is reduced, thereby being able to reduce the computational complexity of the fire detection model while maintaining the detection accuracy of the model, improve the running speed of the model, and meet the high computation and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device.

[0064] Based on the above embodiment, the encoder includes multiple deformable attention modules, and the above S202 includes the following steps:

[0065] The feature data of the image to be detected is sequentially input into a plurality of deformable attention modules, and feature extraction is performed on high-level feature data and / or low-level feature data in the feature data to obtain target feature extraction results.

[0066] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0067] In a possible implementation, if there are M+N deformable attention modules in the encoder, the feature data of the image to be detected can be sequentially input into the N deformable attention modules of the encoder, and the high-level feature data in the feature data can be subjected to N feature extractions to obtain a first intermediate feature extraction result, and then the first intermediate feature extraction result can be sequentially input into the M deformable attention modules of the encoder, and the low-level feature data in the first intermediate feature extraction result can be subjected to M feature extractions to obtain a target feature extraction result, where M is less than N.

[0068] Alternatively, if there are N deformable attention modules in the encoder, the feature data can be input into the M deformable attention modules of the encoder in sequence, and feature extraction is performed M times on the high-level feature data and low-level feature data in the feature data to obtain a second intermediate feature extraction result, and then the second intermediate feature extraction result is input into the NM deformable attention modules of the encoder in sequence, and feature extraction is performed NM times on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0069] It should be noted that if there are M+N deformable attention modules in the encoder, then the M deformable attention modules are only used to extract features from low-level feature data, and the N deformable attention modules are only used to extract features from high-level feature data. If there are N deformable attention modules in the encoder, then the M deformable attention modules among the N deformable attention modules can extract features from both low-level feature data and high-level feature data, and the remaining NM deformable attention modules are only used to extract features from high-level feature data.

[0070] In an embodiment of the present application, the feature data of the image to be detected is sequentially input into a plurality of deformable attention modules, and feature extraction is performed on high-level feature data and / or low-level feature data in the feature data to obtain a target feature extraction result. Among them, M deformable attention modules among the plurality of deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the plurality of deformable attention modules are used to extract features from high-level feature data, thereby reducing the number of feature extractions of low-level feature data, thereby being able to reduce the amount of computation of the fire detection model while maintaining the detection accuracy of the model, improving the running speed of the model, and meeting the high computation and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device.

[0071] Referring to Figure 3, Figure 3 is a flow chart of a method for obtaining target feature extraction results provided by an embodiment of the present application. This embodiment involves a possible implementation method of how to input feature data of an image to be detected into multiple deformable attention modules in sequence, and extract high-level feature data and / or low-level feature data in the feature data to obtain target feature extraction results.

[0072] On the basis of the above embodiment, the multiple deformable attention modules include M deformable attention modules and N deformable attention modules. Then, the feature data of the image to be detected is sequentially input into the multiple deformable attention modules, and the high-level feature data and / or low-level feature data in the feature data are subjected to feature extraction to obtain the target feature extraction result. This can be achieved by the following steps:

[0073] S301, inputting the feature data of the image to be detected into N deformable attention modules in sequence, performing N feature extractions on the high-level feature data in the feature data to obtain a first intermediate feature extraction result.

[0074] Exemplarily, as shown in FIG4 , FIG4 is a schematic diagram of a feature extraction provided in an embodiment of the present application. The image to be detected can be first input into the residual network Resnet for feature extraction to obtain feature data S (S1-S4) of the image to be detected. Among them, S1-S3 are high-level feature data, and S4 is low-level feature data.

[0075] In an embodiment of the present application, the feature data S can be input into N deformable attention modules, and high-level feature data (S1-S3) in the feature data can be subjected to N feature extractions to obtain a first intermediate feature extraction result.

[0076] S302, input the first intermediate feature extraction result into M deformable attention modules in sequence, perform M feature extractions on the low-level feature data in the first intermediate feature extraction result, to obtain a target feature extraction result.

[0077] Exemplarily, the first intermediate feature extraction result may be sequentially input into M deformable attention modules, and feature extraction may be performed M times on the low-level feature data S4 in the first intermediate feature extraction result to obtain the target feature extraction result S'.

[0078] In an embodiment of the present application, the feature data of the image to be detected is sequentially input into N deformable attention modules, and the high-level feature data in the feature data is subjected to N feature extractions to obtain a first intermediate feature extraction result. The first intermediate feature extraction result is sequentially input into M deformable attention modules, and the low-level feature data in the first intermediate feature extraction result is subjected to M feature extractions to obtain a target feature extraction result. This reduces the number of feature extractions of the low-level feature data, thereby reducing the computational complexity of the fire detection model while maintaining the model detection accuracy, thereby improving the running speed of the model and meeting the high computing and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device.

[0079] Referring to Figure 5, Figure 5 is a flow chart of another method for obtaining target feature extraction results provided by an embodiment of the present application. This embodiment involves how to sequentially input feature data of an image to be detected into multiple deformable attention modules, perform feature extraction on high-level feature data and / or low-level feature data in the feature data, and obtain another possible implementation method of the target feature extraction result.

[0080] On the basis of the above embodiment, the feature data of the image to be detected is sequentially input into a plurality of deformable attention modules, and feature extraction is performed on high-level feature data and / or low-level feature data in the feature data to obtain a target feature extraction result, which can also be achieved by the following steps:

[0081] S501, input the feature data into M deformable attention modules in sequence, perform M feature extractions on the high-level feature data and low-level feature data in the feature data to obtain a second intermediate feature extraction result.

[0082] In a possible implementation, there are N deformable attention modules in the encoder, and M of the N deformable attention modules can perform feature extraction on both low-level feature data and high-level feature data, and the remaining NM deformable attention modules are only used to extract features from high-level feature data. Therefore, the feature data can be sequentially input into the M deformable attention modules, and feature extraction is performed M times on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result.

[0083] S502, input the second intermediate feature extraction result into NM deformable attention modules in sequence, perform NM feature extractions on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0084] After the feature data is sequentially input into M deformable attention modules to obtain the second intermediate feature extraction result, the second intermediate feature extraction result can be input into NM deformable attention modules again to perform NM feature extractions on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0085] In the embodiment of the present application, feature extraction is performed N times on the high-level feature data, and feature extraction is performed M times on the low-level feature data, and M is less than N.

[0086] In an embodiment of the present application, the feature data is sequentially input into M deformable attention modules, and feature extraction is performed M times on high-level feature data and low-level feature data in the feature data to obtain a second intermediate feature extraction result. The second intermediate feature extraction result is then sequentially input into NM deformable attention modules, and feature extraction is performed NM times on the high-level feature data in the second intermediate feature extraction result to obtain a target feature extraction result, thereby reducing the number of feature extractions of the low-level feature data. This reduces the computational complexity of the fire detection model while maintaining the model detection accuracy, improves the running speed of the model, and meets the high computing and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device.

[0087] Referring to Figure 6, Figure 6 is a flow chart of a feature extraction method provided in an embodiment of the present application. This embodiment relates to a possible implementation method of how to sequentially input feature data of an image to be detected into multiple deformable attention modules to extract features from high-level feature data and / or low-level feature data in the feature data.

[0088] On the basis of the above embodiment, the feature data of the image to be detected is sequentially input into a plurality of deformable attention modules, and feature extraction is performed on high-level feature data and / or low-level feature data in the feature data, including the following steps:

[0089] S601, determining target sampling points according to feature data of the image to be detected.

[0090] The target sampling point is at least one pixel point in the feature data of the image to be detected.

[0091] In one embodiment, the approximate range of the fire smoke area in the image to be detected can be determined according to the feature data of the image to be detected, and then all pixel points within the range can be determined as target sampling points.

[0092] S602: Extract features based on target sampling points.

[0093] In a possible implementation, when extracting features from the high-level feature data and the low-level feature data, feature extraction may be performed only on feature data of target sampling points in the high-level feature data and the low-level feature data.

[0094] In an embodiment of the present application, the target sampling points are determined according to the feature data of the image to be detected, and feature extraction is performed based on the target sampling points, so as to avoid feature extraction of all pixel points in the image to be detected as much as possible, further reduce the computational complexity of the fire detection model, improve the running speed of the model, and meet the high computing and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device.

[0095] Referring to FIG. 7 , FIG. 7 is a flow chart of a method for determining a target sampling point provided in an embodiment of the present application. This embodiment relates to a possible implementation method of determining a target sampling point based on feature data of an image to be detected. Based on the above embodiment, the above S601 includes the following steps:

[0096] S701, determining a reference point according to feature data of an image to be detected, and determining a sampling range and an offset.

[0097] The reference point is any pixel point in the feature data of the image to be detected.

[0098] Exemplarily, as shown in FIG8, FIG8 is a schematic diagram of the structure of a deformable attention module provided in an embodiment of the present application. The approximate range of the fire smoke area in the image to be detected can be determined based on the feature data of the image to be detected, and then a reference point is determined in the range, and the sampling range and offset are determined based on the range. That is, the query key and value are dynamically sampled according to the feature data of the image to be detected.

[0099] S702: Determine a target sampling point based on a reference point according to a sampling range and an offset.

[0100] The number of target sampling points is less than or equal to the number of pixel points in the feature data.

[0101] In one embodiment, the sampling range is used to indicate the distance between the target sampling point and the reference point, and the offset is used to indicate the direction of each target sampling point relative to the reference point. Then, based on the sampling range and the offset, the target sampling point is determined based on the reference point. That is, the keys and values ​​obtained by the above dynamic sampling are used together with the original query to participate in the standard scaled dot product attention operation, which enhances the adaptability of the model to feature data of different scales and forms, and effectively compensates for the performance degradation that may occur when detecting small targets.

[0102] In the embodiment of the present application, the reference point is determined according to the feature data of the image to be detected, and the sampling range and offset are determined. According to the sampling range and offset, the target sampling point is determined based on the reference point, which further reduces the computational complexity of the fire detection model, improves the running speed of the model, and meets the high computation and low storage requirements of the edge computing device, so that the fire detection model can be deployed on the edge computing device. In addition, the proposed dynamic sampling strategy can also improve the adaptability of the fire detection model, further improving the accuracy and robustness of the fire detection model.

[0103] Referring to FIG. 9 , FIG. 9 is a flow chart of a lightweight fire smoke detection method for edge devices provided in an embodiment of the present application. The method comprises the following steps:

[0104] S901, obtaining an initial fire image.

[0105] S902: Input the initial fire image into a generative adversarial network to obtain a fire generated image output by the generative adversarial network.

[0106] S903: annotate the initial fire image and the fire generated image to obtain a fire detection data set.

[0107] S904: training a fire detection model based on the fire detection data set, and testing the trained fire detection model to obtain a fire detection model for fire detection.

[0108] S905: Input the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected.

[0109] S906, inputting the feature data of the image to be detected into N deformable attention modules in sequence, performing N feature extractions on the high-level feature data in the feature data to obtain a first intermediate feature extraction result.

[0110] S907, input the first intermediate feature extraction result into M deformable attention modules in sequence, and perform M feature extractions on the low-level feature data in the first intermediate feature extraction result to obtain the target feature extraction result.

[0111] S908, determining a fire detection result based on the target feature extraction result.

[0112] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0113] Based on the same inventive concept, the embodiment of the present application also provides a fire detection device for implementing the fire detection method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more fire detection device embodiments provided below can refer to the limitations on the fire detection method above, and will not be repeated here.

[0114] In one embodiment, as shown in FIG. 10 , FIG. 10 is a structural block diagram of a fire detection device provided in an embodiment of the present application, and the device 1000 includes:

[0115] The first feature extraction module 1001 is used to input the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data.

[0116] The second feature extraction module 1002 is used to input the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain the target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder.

[0117] The determination module 1003 is used to determine the fire detection result based on the target feature extraction result.

[0118] In one embodiment, the second feature extraction module 1002 includes:

[0119] The feature extraction unit is used to input the feature data of the image to be detected into multiple deformable attention modules in sequence, and perform feature extraction on the high-level feature data and / or low-level feature data in the feature data to obtain the target feature extraction result.

[0120] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0121] In one embodiment, the feature extraction unit includes:

[0122] The first feature extraction subunit is used to input the feature data of the image to be detected into N deformable attention modules in sequence, and perform N feature extractions on the high-level feature data in the feature data to obtain a first intermediate feature extraction result.

[0123] The second feature extraction subunit is used to input the first intermediate feature extraction result into M deformable attention modules in sequence, and perform M feature extractions on the low-level feature data in the first intermediate feature extraction result to obtain the target feature extraction result.

[0124] In one embodiment, the feature extraction unit further includes:

[0125] The third feature extraction subunit is used to input the feature data into M deformable attention modules in sequence, and perform M feature extractions on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result.

[0126] The fourth feature extraction subunit is used to input the second intermediate feature extraction result into NM deformable attention modules in sequence, and perform NM feature extractions on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0127] In one embodiment, the feature extraction unit further includes:

[0128] The determination subunit is used to determine the target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected.

[0129] The fifth feature extraction subunit is used to extract features based on the target sampling points.

[0130] In one embodiment, the determination subunit is specifically used to determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected;

[0131] According to the sampling range and the offset, the target sampling points are determined based on the reference points; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

[0132] Each module in the fire detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0133] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0134] The image to be detected is input into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data;

[0135] Inputting feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder;

[0136] The fire detection result is determined based on the target feature extraction result.

[0137] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0138] Inputting feature data of the image to be detected into a plurality of deformable attention modules in sequence, performing feature extraction on high-level feature data and / or low-level feature data in the feature data to obtain a target feature extraction result;

[0139] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0140] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0141] Inputting the feature data of the image to be detected into N deformable attention modules in sequence, performing N feature extractions on high-level feature data in the feature data to obtain a first intermediate feature extraction result;

[0142] The first intermediate feature extraction result is sequentially input into M deformable attention modules, and feature extraction is performed M times on the low-level feature data in the first intermediate feature extraction result to obtain the target feature extraction result.

[0143] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0144] Input the feature data into M deformable attention modules in sequence, and perform M feature extractions on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result;

[0145] The second intermediate feature extraction result is sequentially input into NM deformable attention modules, and NM feature extractions are performed on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0146] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0147] Determine a target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected;

[0148] Feature extraction is performed based on target sampling points.

[0149] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0150] Determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected;

[0151] According to the sampling range and the offset, the target sampling points are determined based on the reference points; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

[0152] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0153] The image to be detected is input into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data;

[0154] Inputting feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder;

[0155] The fire detection result is determined based on the target feature extraction result.

[0156] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0157] Inputting feature data of the image to be detected into a plurality of deformable attention modules in sequence, performing feature extraction on high-level feature data and / or low-level feature data in the feature data to obtain a target feature extraction result;

[0158] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0159] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0160] Inputting the feature data of the image to be detected into N deformable attention modules in sequence, performing N feature extractions on high-level feature data in the feature data to obtain a first intermediate feature extraction result;

[0161] The first intermediate feature extraction result is sequentially input into M deformable attention modules, and feature extraction is performed M times on the low-level feature data in the first intermediate feature extraction result to obtain the target feature extraction result.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] Input the feature data into M deformable attention modules in sequence, and perform M feature extractions on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result;

[0164] The second intermediate feature extraction result is sequentially input into NM deformable attention modules, and NM feature extractions are performed on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Determine a target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected;

[0167] Feature extraction is performed based on target sampling points.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0169] Determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected;

[0170] According to the sampling range and the offset, the target sampling points are determined based on the reference points; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

[0171] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0172] The image to be detected is input into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data;

[0173] Inputting feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder;

[0174] The fire detection result is determined based on the target feature extraction result.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0176] Inputting feature data of the image to be detected into a plurality of deformable attention modules in sequence, performing feature extraction on high-level feature data and / or low-level feature data in the feature data to obtain a target feature extraction result;

[0177] Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, where N and M are integers greater than zero, and N is greater than M.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0179] Inputting the feature data of the image to be detected into N deformable attention modules in sequence, performing N feature extractions on high-level feature data in the feature data to obtain a first intermediate feature extraction result;

[0180] The first intermediate feature extraction result is sequentially input into M deformable attention modules, and feature extraction is performed M times on the low-level feature data in the first intermediate feature extraction result to obtain the target feature extraction result.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] Input the feature data into M deformable attention modules in sequence, and perform M feature extractions on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result;

[0183] The second intermediate feature extraction result is sequentially input into NM deformable attention modules, and NM feature extractions are performed on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0185] Determine a target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected;

[0186] Feature extraction is performed based on target sampling points.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0188] Determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected;

[0189] According to the sampling range and the offset, the target sampling points are determined based on the reference points; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

[0190] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0191] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0192] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fire detection method, characterized in that: The method comprises: Inputting the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data; Inputting the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder; A fire detection result is determined based on the target feature extraction result.

2. The method according to claim 1, characterized in that The encoder includes a plurality of deformable attention modules, and the feature data of the image to be detected is input into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result, including: Inputting the feature data of the image to be detected into the multiple deformable attention modules in sequence, and performing feature extraction on the high-level feature data and / or low-level feature data in the feature data to obtain the target feature extraction result; Among them, M deformable attention modules among the multiple deformable attention modules are used to extract features from low-level feature data, and N deformable attention modules among the multiple deformable attention modules are used to extract features from high-level feature data, N and M are integers greater than zero, and N is greater than M.

3. The method according to claim 2, characterized in that The multiple deformable attention modules include the M deformable attention modules and the N deformable attention modules, the feature data of the image to be detected is sequentially input into the multiple deformable attention modules, and the high-level feature data and / or low-level feature data in the feature data are subjected to feature extraction to obtain the target feature extraction result, including: Inputting the feature data of the image to be detected into the N deformable attention modules in sequence, and performing N feature extractions on the high-level feature data in the feature data to obtain a first intermediate feature extraction result; The first intermediate feature extraction result is sequentially input into the M deformable attention modules, and the low-level feature data in the first intermediate feature extraction result is subjected to M feature extractions to obtain the target feature extraction result.

4. The method according to claim 2, characterized in that: The step of sequentially inputting the feature data of the image to be detected into the plurality of deformable attention modules, and extracting the high-level feature data and / or the low-level feature data in the feature data to obtain the target feature extraction result includes: Inputting the feature data into the M deformable attention modules in sequence, performing M feature extractions on the high-level feature data and the low-level feature data in the feature data to obtain a second intermediate feature extraction result; The second intermediate feature extraction result is sequentially input into NM deformable attention modules, and NM feature extractions are performed on the high-level feature data in the second intermediate feature extraction result to obtain the target feature extraction result.

5. The method according to claim 2, characterized in that: The step of sequentially inputting the feature data of the image to be detected into the plurality of deformable attention modules, and extracting features from high-level feature data and / or low-level feature data in the feature data comprises: Determine a target sampling point according to the feature data of the image to be detected; the target sampling point is at least one pixel point in the feature data of the image to be detected; Feature extraction is performed based on the target sampling points.

6. The method according to claim 3, characterized in that The step of determining the target sampling point according to the feature data of the image to be detected comprises: Determine a reference point according to the feature data of the image to be detected, and determine a sampling range and an offset; the reference point is any pixel point in the feature data of the image to be detected; The target sampling points are determined based on the reference points according to the sampling range and the offset; the number of the target sampling points is less than or equal to the number of pixel points in the feature data.

7. A fire detection device, characterized in that: The device comprises: A first feature extraction module is used to input the image to be detected into the residual network of the fire detection model for feature extraction to obtain feature data of the image to be detected; the feature data includes high-level feature data and low-level feature data; A second feature extraction module is used to input the feature data of the image to be detected into the encoder of the fire detection model for feature extraction to obtain a target feature extraction result; the number of feature extractions of the high-level feature data in the encoder is greater than the number of feature extractions of the low-level feature data in the encoder; A determination module is used to determine a fire detection result based on the target feature extraction result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images