Smoke detection method and device

Abstract

The invention provides a smoke detection method and device which can be applied to the technical field of target detection and computer vision. The method comprises the following steps: utilizing a smoke detection model to pre-process and vectorize an environment image acquired by image acquisition equipment, and determining input data for smoke detection; multiple dynamic convolution kernels are used for conducting multiple times of feature extraction on the input data, multiple data features of different dimensions are obtained, and the multiple dynamic convolution kernels are determined according to a dynamic convolution kernel library; using a weighted feature fusion module of the smoke detection model to perform multi-round weighted aggregation on the plurality of data features to obtain a plurality of fusion features of different scales; respectively inputting the plurality of fusion features into a detection head, and determining a plurality of smoke detection sub-results of different scales; and determining a smoke detection result of the environment image based on the plurality of smoke detection sub-results.

Description

Technical Field

[0001] The present disclosure relates to the fields of target detection and computer vision technology, and in particular to a smoke detection method and device. Background Art

[0002] With the development of target detection technology, it is becoming increasingly common to use computers to monitor and analyze areas that are difficult to detect manually. Due to the high frequency and destructiveness of forest fires, accurate and timely monitoring is necessary to reduce losses or damage. In related technologies, forest conditions are usually monitored through manual observation, satellite imaging, and aerial monitoring. However, the accuracy and timeliness of manual observation are difficult to guarantee, and the labor cost is high. Satellite imaging is difficult to guarantee accuracy because its resolution may be affected by the complex environment of the forest. Although aerial monitoring has a wide coverage and good real-time performance, it is difficult to promote its use due to its cost and cannot be used for continuous monitoring.

[0003] In the process of implementing the present disclosure, the inventors found that when the related technology processes the collected images, it is difficult to configure it on the image acquisition device because the image processing network parameters are very large. Therefore, it is usually necessary to transmit the environmental images collected by the image acquisition device back to the server, and the server will perform image processing to determine whether there is a fire. This will result in a large number of pictures that do not contain fire conditions being transmitted to the server, resulting in a waste of transmission resources. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides a smoke detection method and device.

[0005] According to a first aspect of the present disclosure, a smoke detection method is provided, comprising: utilizing a smoke detection model to preprocess and vectorize an environmental image acquired by an image acquisition device to determine input data for smoke detection; utilizing multiple dynamic convolution kernels to perform multiple feature extractions on the input data to obtain multiple data features of different dimensions, wherein the multiple dynamic convolution kernels are determined based on a dynamic convolution kernel library; utilizing a weighted feature fusion module of the smoke detection model to perform multiple rounds of weighted aggregation on the multiple data features to obtain multiple fused features of different scales; respectively inputting the multiple fused features into a detection head to determine multiple smoke detection sub-results of different scales; and determining a smoke detection result of the environmental image based on the multiple smoke detection sub-results.

[0006] According to an embodiment of the present disclosure, a weighted feature fusion module is used to perform multiple rounds of weighted aggregation on multiple data features to obtain multiple fused features of different scales, including: determining the feature scales of each of the multiple data features, where the feature scales include dimensions and number of channels; based on the multiple feature scales, determining high-dimensional features and low-dimensional features from the multiple data features; and based on the high-dimensional features and low-dimensional features, performing multiple rounds of weighted aggregation on the multiple data features to obtain multiple fused features.

[0007] According to an embodiment of the present disclosure, multiple rounds of weighted aggregation are performed on multiple data features based on high-dimensional features and low-dimensional features to obtain multiple fused features, including: for each data feature, based on the feature scale of the data feature, data alignment processing is performed on the high-dimensional features, low-dimensional features and data features to obtain aligned data features; the data features are divided into a preset number of partitions according to the channel dimension to obtain a preset number of partition features; for each partition, the partition features of the partition are processed using an activation function to determine a fusion weight; the high-dimensional features and low-dimensional features are summed using the fusion weight to obtain the processed features of the partition; and the processed features of each preset number of partitions are processed to determine a fused feature.

[0008] According to an embodiment of the present disclosure, the processed features of each of a preset number of partitions are processed to determine a fusion feature, including: splicing the processed features of each of the preset number of partitions to obtain a spliced ​​feature, wherein the feature scale of the spliced ​​feature is the same as the feature scale of the data feature corresponding to the spliced ​​feature; performing convolution processing on the spliced ​​feature to obtain a convolution feature; performing interpolation processing on the convolution feature to obtain an interpolation feature; and processing the interpolation feature using an activation function to obtain a fusion feature.

[0009] According to an embodiment of the present disclosure, multiple dynamic convolution kernels are used to perform multiple feature extractions on input data to obtain multiple data features of different dimensions, including: determining multiple kernel units and multiple groups of attention weights included in a dynamic convolution kernel library, wherein each group of attention weights includes the respective attention weights of multiple kernel units; determining multiple dynamic convolution kernels based on the multiple groups of attention weights and the multiple kernel units; and using the multiple dynamic convolution kernels to perform feature extraction on the input data respectively to obtain data features.

[0010] According to an embodiment of the present disclosure, multiple fused features are respectively input into a detection head to determine multiple smoke detection sub-results of different scales, including: inputting multiple fused features into a detection head corresponding to the scale of the fused features, and outputting the smoke detection sub-results corresponding to the scale of the fused features.

[0011] According to an embodiment of the present disclosure, the smoke detection model is trained in the following manner: using the initial smoke detection model, the sample environment image is preprocessed and vectorized to obtain sample input data; multiple groups of sample attention weights are determined based on the sample input data; based on the multiple groups of sample attention weights and the dynamic convolution kernel library, multiple sample dynamic convolution kernels are determined; multiple features are extracted from the sample input data using multiple sample dynamic convolution kernels to obtain multiple sample data features of different dimensions; using the initial weighted feature fusion module of the sample smoke detection model, multiple rounds of weighted aggregation are performed on the multiple sample data features to obtain multiple sample fusion features of different scales; the multiple sample fusion features are respectively input into the detection head to determine multiple sample smoke detection sub-results of different scales; based on the multiple sample smoke detection sub-results, the sample smoke detection result is determined; and based on the sample smoke detection result and the sample label of the sample environment image, the model parameters of the initial smoke detection model are adjusted until the prediction error between the sample smoke detection result and the sample label is less than a preset threshold, and the smoke detection model is determined based on the current model parameters of the initial smoke detection model.

[0012] According to an embodiment of the present disclosure, the model parameters of the initial smoke detection model are adjusted based on the sample smoke detection results and the sample labels of the sample environment images, including: determining the prediction error between the sample smoke detection results and the sample labels of the sample environment images; determining the current learning rate based on an adaptive learning rate determination method; and adjusting the model parameters of the initial smoke detection model based on the current learning rate and the prediction error.

[0013] According to an embodiment of the present disclosure, the prediction error between the sample smoke detection result and the sample label of the sample environment image is determined, including: determining the edge line of the predicted anchor frame, the edge line of the sample anchor frame, and the anchor frame width and anchor frame height of the sample anchor frame based on the predicted anchor frame corresponding to the sample smoke detection result and the sample anchor frame corresponding to the sample label; determining the edge line distance between the edge line of the predicted anchor frame and the corresponding edge line of the sample anchor frame; and determining the prediction error based on the edge line distance, the anchor frame width and the anchor frame height.

[0014] A second aspect of the present disclosure provides a smoke detection device, comprising: a data preprocessing module, used to use a smoke detection model to preprocess and vectorize an environmental image acquired by an image acquisition device, and determine input data for smoke detection; a feature extraction module, used to use multiple dynamic convolution kernels to perform multiple feature extractions on the input data, and obtain multiple data features of different dimensions, wherein the multiple dynamic convolution kernels are determined based on a dynamic convolution kernel library; a feature aggregation module, used to use a weighted feature fusion module of the smoke detection model to perform multiple rounds of weighted aggregation on the multiple data features, and obtain multiple fused features of different scales; a sub-result determination module, used to input the multiple fused features into a detection head respectively, and determine multiple smoke detection sub-results of different scales; and a result determination module, used to determine the smoke detection result of the environmental image based on the multiple smoke detection sub-results.

[0015] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0016] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0017] The fifth aspect of the present disclosure further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0018] According to the embodiments of the present disclosure, a smoke detection model is used to preprocess and vectorize environmental data, a dynamic convolution kernel library is used to determine a dynamic convolution kernel, and the dynamic convolution kernel is used to perform multiple feature extractions on the input data. Multiple kernel units in the dynamic convolution kernel library can be used to determine dynamic convolution kernels of different scales, thereby reducing the number of model parameters, balancing parameter efficiency and model performance, and improving the model's lightweightness and usability. The multiple data features obtained from feature extraction are weighted and aggregated, and the fused features are then input into the detection head to further determine the smoke detection results. This allows the fused features to include features of multiple scales, thereby improving the accuracy of smoke detection using the fused features. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0020] Figure 1Schematically illustrates an application scenario diagram of the smoke detection method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0021] Figure 2 The flowchart of the smoke detection method according to the embodiment of the present disclosure is schematically shown;

[0022] Figure 3 The module structure of the dimension-aware selective integration module according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 4 The model structure of the smoke detection model according to the embodiment of the present disclosure is schematically shown;

[0024] Figure 5 The figure shows the positional relationship between the sample anchor box and the predicted anchor box according to an embodiment of the present disclosure;

[0025] Figure 6 Schematically shows a structural block diagram of a smoke detection device according to an embodiment of the present disclosure; and

[0026] Figure 7 The block diagram schematically shows an electronic device suitable for implementing the smoke detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solutions disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] In scenarios where personal information is used for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure all provide users with corresponding operation portals for them to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge, and skills, and have reached a certain level of professionalism.

[0033] An embodiment of the present disclosure provides a smoke detection method, including: using a smoke detection model to preprocess and vectorize an environmental image captured by an image acquisition device to determine input data for smoke detection; using multiple dynamic convolution kernels to perform multiple feature extractions on the input data to obtain multiple data features of different dimensions, wherein the multiple dynamic convolution kernels are determined based on a dynamic convolution kernel library; using a weighted feature fusion module of the smoke detection model to perform multiple rounds of weighted aggregation on the multiple data features to obtain multiple fused features of different scales; inputting the multiple fused features into a detection head respectively to determine multiple smoke detection sub-results of different scales; and determining a smoke detection result of the environmental image based on the multiple smoke detection sub-results.

[0034] Figure 1 The application scenario diagram of the smoke detection method, device, equipment, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0035] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0036] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0037] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0038] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0039] It should be noted that the smoke detection method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the smoke detection device provided in the embodiments of the present disclosure can generally be set in the server 105. The smoke detection method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the smoke detection device provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0041] The following will be based on Figure 1 The scene described by Figures 2 to 5 The smoke detection method of the embodiment of the present disclosure is described in detail.

[0042] Figure 2 The flowchart of the smoke detection method according to the embodiment of the present disclosure is schematically shown.

[0043] like Figure 2 As shown, the smoke detection method of this embodiment includes operations S210 to S250.

[0044] In operation S210 , the environment image captured by the image capture device is preprocessed and vectorized using a smoke detection model to determine input data for smoke detection.

[0045] According to an embodiment of the present disclosure, an image acquisition device may include a device for acquiring environmental images in a forest environment, such as a drone, a fixed surveillance camera, etc. The environmental images may include visual images of the forest environment, which are used to monitor the presence of smoke and thereby determine whether a fire is occurring within the forest area shown in the environmental image.

[0046] According to embodiments of the present disclosure, preprocessing of environmental images may include data filtering and data enhancement. Data filtering can be used to filter out environmental images with resolutions below a threshold, while data enhancement can further improve the clarity of environmental images with resolutions above the threshold through methods such as denoising algorithms. Preprocessing may also include image resizing, for example, by cropping or scaling multiple environmental images to a uniform size for subsequent processing.

[0047] According to an embodiment of the present disclosure, vectorization can be performed by converting the environment image into a grayscale image, and then converting the pixel values ​​of the grayscale image into a one-dimensional array, and determining the one-dimensional vector corresponding to the one-dimensional array as input data.

[0048] In operation S220 , multiple dynamic convolution kernels are used to perform multiple feature extractions on the input data to obtain multiple data features of different dimensions.

[0049] According to an embodiment of the present disclosure, multiple dynamic convolution kernels are determined based on a dynamic convolution kernel library, which includes multiple kernel units of the same scale. By performing multiple linear combinations of multiple kernel units of the same dimension using different weight groups, multiple dynamic convolution kernels of the same scale but different values ​​can be obtained.

[0050] According to an embodiment of the present disclosure, since the values ​​of multiple dynamic convolution kernels are different, multiple feature extractions are performed on the input data through multiple dynamic convolution kernels, and data features of multiple different dimensions of the input data can be obtained, wherein the dimensions of the data features can be used to characterize the granularity of the smoke. For example, for the same input data, the data features with larger dimensions include the smoke conditions with a large impact range in the environmental image, and the data features with smaller dimensions include the smoke conditions with a small impact range in the environmental image.

[0051] In operation S230 , a weighted feature fusion module of the smoke detection model is used to perform multiple rounds of weighted aggregation on the multiple data features to obtain multiple fused features of different scales.

[0052] According to the embodiments of the present disclosure, multiple data features of different dimensions can be aligned to ensure dimensional consistency, and then the aligned data features can be divided in the channel dimension. The high-dimensional and low-dimensional data can be fused through data processing methods such as convolution to obtain multiple fused features of different scales.

[0053] In operation S240 , the multiple fused features are respectively input to the detection head to determine multiple smoke detection sub-results of different scales.

[0054] According to an embodiment of the present disclosure, multiple fused features are respectively input into the detection head, and the detection head is used to locate and visualize the fused features, so that the anchor frame corresponding to the smoke detection sub-result is represented on the environment image.

[0055] In operation S250 , a smoke detection result of the environment image is determined based on the plurality of smoke detection sub-results.

[0056] According to an embodiment of the present disclosure, based on multiple smoke detection sub-results, multiple anchor frames are represented on the same environment image to obtain a smoke detection result of the environment image.

[0057] According to the embodiments of the present disclosure, a smoke detection model is used to preprocess and vectorize environmental data, a dynamic convolution kernel library is used to determine a dynamic convolution kernel, and the dynamic convolution kernel is used to perform multiple feature extractions on the input data. Multiple kernel units in the dynamic convolution kernel library can be used to determine dynamic convolution kernels of different scales, thereby reducing the number of model parameters, balancing parameter efficiency and model performance, and improving the model's lightweightness and usability. The multiple data features obtained from feature extraction are weighted and aggregated, and the fused features are then input into the detection head to further determine the smoke detection results. This allows the fused features to include features of multiple scales, thereby improving the accuracy of smoke detection using the fused features.

[0058] According to an embodiment of the present disclosure, a weighted feature fusion module is used to perform multiple rounds of weighted aggregation on multiple data features to obtain multiple fused features of different scales, including: determining the feature scales of each of the multiple data features, where the feature scales include dimensions and number of channels; based on the multiple feature scales, determining high-dimensional features and low-dimensional features from the multiple data features; and based on the high-dimensional features and low-dimensional features, performing multiple rounds of weighted aggregation on the multiple data features to obtain multiple fused features.

[0059] According to embodiments of the present disclosure, the dimensions of data features include feature width (Weight, W) and feature height (Height, H). Based on the characteristic scales of multiple data features, the data feature with the highest characteristic scale is determined as a high-dimensional feature, and the data feature with the lowest characteristic scale is determined as a low-dimensional feature.

[0060] According to an embodiment of the present disclosure, each data feature is processed according to high-dimensional features and low-dimensional features to achieve weighted aggregation of features and obtain fused features.

[0061] According to an embodiment of the present disclosure, each data feature is processed using high-dimensional features and low-dimensional features, so that the processed fusion features include data features of multiple dimensions, thereby ensuring that the smoke detection results can take into account smoke conditions of different scales at the same time and improve the accuracy of monitoring.

[0062] According to an embodiment of the present disclosure, multiple rounds of weighted aggregation are performed on multiple data features based on high-dimensional features and low-dimensional features to obtain multiple fused features, including: for each data feature, based on the feature scale of the data feature, data alignment processing is performed on the high-dimensional features, low-dimensional features and data features to obtain aligned data features; the data features are divided into a preset number of partitions according to the channel dimension to obtain a preset number of partition features; for each partition, the partition features of the partition are processed using an activation function to determine a fusion weight; the high-dimensional features and low-dimensional features are summed using the fusion weight to obtain the processed features of the partition; and the processed features of each preset number of partitions are processed to determine a fused feature.

[0063] According to an embodiment of the present disclosure, for each data feature, the high-dimensional features and the low-dimensional features are processed according to the feature scale of the current data feature, so that the high-dimensional features and the low-dimensional features are consistent with the feature scale of the current data feature, and the aligned data features are obtained, wherein the aligned data features include the current data feature, and the high-dimensional features and the low-dimensional features that are consistent with the feature scale of the current data feature.

[0064] According to an embodiment of the present disclosure, the aligned data features are evenly divided according to the channel dimension to obtain a preset number of partitions. Preferably, the preset number can be 4. For example, when the feature scale of the aligned data features is H0×W0×C0, the current data features, high-dimensional features, and low-dimensional features in the climate data features are each divided into four partition features with a feature scale of H0×W0×(1 / 4)C0.

[0065] According to an embodiment of the present disclosure, for each partition, an activation function is used to process the partition features of the partition in the current data features to determine the fusion weight. Preferably, the activation function can be a sigmoid function.

[0066] For example, for the i-th partition, the fusion weight a can be determined by formula (1): i :

[0067] (1)

[0068] Among them, u i is the i-th partition of the current data feature.

[0069] According to the embodiment of the present disclosure, based on the fusion weight, the partition features of the i-th partition of each of the high-dimensional features and the low-dimensional features can be summed to obtain the processed features of the i-th partition. , as shown in formula (2):

[0070] (2)

[0071] Among them, l i is the i-th partition of the low-dimensional feature, h i is the i-th partition of the high-dimensional feature.

[0072] According to an embodiment of the present disclosure, based on the processed features of each of the multiple partitions, further processing is performed using feature splicing, convolution, etc. to obtain fused features.

[0073] According to the embodiments of the present disclosure, by processing data features, the characteristic scales of the data features, low-dimensional features, and high-dimensional features are aligned to facilitate subsequent feature processing, thereby improving the efficiency and accuracy of the feature processing process. The processed features determined by the low-dimensional and high-dimensional features can more effectively represent the characteristics of smoke in complex forest environments, thereby ensuring the accuracy of monitoring.

[0074] According to an embodiment of the present disclosure, the above-mentioned feature fusion operation may be performed in a Dimension-Aware Selective Integration Module (DASI) in a smoke detection model.

[0075] Figure 3 The module structure of the dimension-aware selective integration module according to an embodiment of the present disclosure is schematically shown.

[0076] like Figure 3 As shown, low-dimensional features, high-dimensional features, and current data features are input into the DASI module. Their respective characteristic scales are determined, and the low-dimensional features, high-dimensional features, and current data features are aligned according to the characteristic scale of the current data features. After alignment, the climate data features are evenly divided into four groups of partitioned features along the channel dimension. For each partition, a fusion weight is determined based on the current data features using a sigmoid activation function. Based on the fusion weight, the high-dimensional and low-dimensional features are summed to obtain the processed features for that partition. After calculating the processed features corresponding to each of the four groups of partitioned features, the four processed features are concatenated along the channel dimension to obtain a fused feature with the same characteristic scale as the current data features.

[0077] According to an embodiment of the present disclosure, the processed features of each of a preset number of partitions are processed to determine a fusion feature, including: splicing the processed features of each of the preset number of partitions to obtain a spliced ​​feature, wherein the feature scale of the spliced ​​feature is the same as the feature scale of the data feature corresponding to the spliced ​​feature; performing convolution processing on the spliced ​​feature to obtain a convolution feature; performing interpolation processing on the convolution feature to obtain an interpolation feature; and processing the interpolation feature using an activation function to obtain a fusion feature.

[0078] According to the embodiment of the present disclosure, a preset number of partitions are arranged in order and spliced ​​to obtain splicing features. Taking 4 partitions as an example, To perform splicing, use the splicing feature , as shown in formula (3):

[0079] (3)

[0080] According to an embodiment of the present disclosure, the splicing feature The concatenated features are processed with the same feature scale as the current data features, and the activation function is used to process them to obtain the fused features. , preferably, the activation function can use the sigmoid function, as shown in formula (4):

[0081] (4)

[0082] Among them, δ is the sigmoid function, B(·) is the interpolation process, and Conv(·) is the convolution process.

[0083] According to the embodiments of the present disclosure, by splicing features and performing further processing, the obtained fused features can include features across multiple scales at the same time, thereby improving the accuracy of smoke detection.

[0084] According to an embodiment of the present disclosure, multiple dynamic convolution kernels are used to perform multiple feature extractions on input data to obtain multiple data features of different dimensions, including: determining multiple kernel units and multiple groups of attention weights included in a dynamic convolution kernel library, wherein each group of attention weights includes the respective attention weights of multiple kernel units; determining multiple dynamic convolution kernels based on the multiple groups of attention weights and the multiple kernel units; and using the multiple dynamic convolution kernels to perform feature extraction on the input data respectively to obtain data features.

[0085] According to an embodiment of the present disclosure, a plurality of core units included in a dynamic convolution kernel library is determined, wherein the plurality of core units are randomly generated when the dynamic convolution kernel library is generated. A plurality of groups of attention weights are determined, wherein the plurality of groups of attention weights are obtained by training in the model training phase. For example, a dynamic convolution kernel library E includes n core units, namely E1, E2, ..., E n , the attention weight of group j is w 1j 、w 2j 、…、w nj , then the dynamic convolution kernel W is determined according to the kernel unit and the group of attention weights j =w 1j E1+w 2j E2+…+w nj E n .

[0086] According to an embodiment of the present disclosure, a dynamic convolution kernel can be obtained by performing weighted summation on multiple kernel units using each set of attention weights in the multiple sets of attention weights. Therefore, multiple dynamic convolution kernels can be determined based on the multiple sets of attention weights.

[0087] According to an embodiment of the present disclosure, multiple dynamic convolution kernels are used to convolve the input data respectively to achieve feature extraction and determine the data features corresponding to the dynamic convolution kernels.

[0088] According to the embodiments of the present disclosure, by using a dynamic convolution kernel library and attention weights, determining a dynamic convolution kernel and performing feature extraction on the input data, the dimension of the convolution kernel can be reduced, thereby maintaining or even improving the representation capability of the smoke detection model while reducing the number of parameters.

[0089] According to an embodiment of the present disclosure, multiple fused features are respectively input into a detection head to determine multiple smoke detection sub-results of different scales, including: inputting multiple fused features into a detection head corresponding to the scale of the fused features, and outputting the smoke detection sub-results corresponding to the scale of the fused features.

[0090] According to an embodiment of the present disclosure, the detection head can determine the range of smoke existing in the environmental image represented by the fusion feature based on the fusion feature, and represent the smoke range on the environmental image with an anchor frame, thereby obtaining a smoke detection sub-result, wherein the detection heads corresponding to the fusion features of different scales can be the same detection head or different detection heads.

[0091] Figure 4 The model structure of the smoke detection model according to the embodiment of the present disclosure is schematically shown.

[0092] like Figure 4 As shown in the figure, the smoke detection model consists of a backbone network for feature extraction, a neck network for feature fusion, and a head network for determining smoke detection sub-results. The backbone includes five interface modules (KWConv), four C2f_KW modules for feature processing, and an improved version of spatial pyramid pooling (SPPF). In the KWConv module, the convolution kernel is 3*3 with a stride of 2, and the convolution layers are padded with zeros.

[0093] According to an embodiment of the present disclosure, the neck portion includes two DASI modules, two KWConv modules, and four C2f_KW modules. After DASI processing, data is input into the KWConv module for convolution and other processing, and then concatenated through the Concat module. The concatenated features are further processed by the C2f_KW module, and the processed features are upsampled by the Unsample module. After further concatenation through the Concat module, they are input into the C2f_KW module for processing. The scales of the features output by the C2f_KW modules at different levels vary.

[0094] According to an embodiment of the present disclosure, the head part includes several detection heads (Detect), and different Detects are used to process fusion features of different scales to determine the smoke detection sub-results of the scale.

[0095] According to an embodiment of the present disclosure, the smoke detection model is trained in the following manner: using the initial smoke detection model, the sample environment image is preprocessed and vectorized to obtain sample input data; multiple groups of sample attention weights are determined based on the sample input data; based on the multiple groups of sample attention weights and the dynamic convolution kernel library, multiple sample dynamic convolution kernels are determined; multiple features are extracted from the sample input data using multiple sample dynamic convolution kernels to obtain multiple sample data features of different dimensions; using the initial weighted feature fusion module of the sample smoke detection model, multiple rounds of weighted aggregation are performed on the multiple sample data features to obtain multiple sample fusion features of different scales; the multiple sample fusion features are respectively input into the detection head to determine multiple sample smoke detection sub-results of different scales; based on the multiple sample smoke detection sub-results, the sample smoke detection result is determined; and based on the sample smoke detection result and the sample label of the sample environment image, the model parameters of the initial smoke detection model are adjusted until the prediction error between the sample smoke detection result and the sample label is less than a preset threshold, and the smoke detection model is determined based on the current model parameters of the initial smoke detection model.

[0096] According to an embodiment of the present disclosure, the sample environment image is preprocessed and vectorized in the same way as the environment image, and multiple groups of sample attention weights are determined based on the obtained sample input data, and multiple sample dynamic convolution kernels are determined with multiple kernel units in the dynamic convolution kernel library to perform feature extraction on the sample input data.

[0097] According to the embodiments of the present disclosure, the extracted sample data features are subjected to multiple rounds of weighted aggregation using the same method as the weighted feature fusion module to obtain multiple sample fusion features. These sample fusion features are then input into the detection head to determine multiple sample smoke detection sub-results. The anchor boxes of these multiple sample smoke detection sub-results are then displayed on the same sample environment image to obtain the sample smoke detection result.

[0098] According to an embodiment of the present disclosure, after obtaining the sample smoke detection result, the prediction error is determined based on one or more anchor frames corresponding to the sample label of the sample environment image and one or more anchor frames of the sample smoke detection result, so as to adjust the model parameters in the sample smoke detection model according to the prediction error, wherein the adjusted model parameters may include the sample attention weight, the parameters of the detection head, etc.

[0099] According to the embodiments of the present disclosure, based on the anchor frames in the sample labels and the anchor frames obtained through monitoring, the prediction error is determined and the network parameters are adjusted, which can make the anchor frame prediction of the network more accurate after the parameter adjustment, thereby improving the detection accuracy of the smoke detection model.

[0100] According to an embodiment of the present disclosure, the model parameters of the initial smoke detection model are adjusted based on the sample smoke detection results and the sample labels of the sample environment images, including: determining the prediction error between the sample smoke detection results and the sample labels of the sample environment images; determining the current learning rate based on an adaptive learning rate determination method; and adjusting the model parameters of the initial smoke detection model based on the current learning rate and the prediction error.

[0101] According to an embodiment of the present disclosure, the adaptive learning rate determination method can determine the current learning rate by utilizing the Adam optimizer and adopting adaptive learning rate scheduling strategies such as cosine annealing with warm restarts or one-cycle learning rate (OneCycleLR), and determine the adjustment amplitude of the model parameters based on the current learning rate and the prediction error, and adjust the model parameters according to the adjustment amplitude until the prediction error between the sample smoke detection result and the sample label is less than a preset threshold.

[0102] According to the embodiments of the present disclosure, by adjusting the adaptive learning rate, the model can select a more appropriate learning rate during the parameter adjustment process, thereby improving the efficiency of parameter adjustment.

[0103] According to an embodiment of the present disclosure, the prediction error between the sample smoke detection result and the sample label of the sample environment image is determined, including: determining the edge line of the predicted anchor frame, the edge line of the sample anchor frame, and the anchor frame width and anchor frame height of the sample anchor frame based on the predicted anchor frame corresponding to the sample smoke detection result and the sample anchor frame corresponding to the sample label; determining the edge line distance between the edge line of the predicted anchor frame and the corresponding edge line of the sample anchor frame; and determining the prediction error based on the edge line distance, the anchor frame width and the anchor frame height.

[0104] According to embodiments of the present disclosure, in the regression task of delineating anchor frames, the Intersection over Union Loss (IoU) loss function used for bounding box regression is typically used to determine prediction errors. However, this loss function cannot adequately account for the shape differences between the anchor frames and the smoke. To address this issue, a gradient adjustment mechanism can be introduced to prioritize the loss function over anchor frames of medium quality. The quality of the anchor frames can be used to characterize the degree of match between the anchor frames and the smoke in the surrounding image.

[0105] Figure 5 The figure schematically shows the positional relationship between the sample anchor box and the predicted anchor box according to an embodiment of the present disclosure.

[0106] like Figure 5 As shown, based on the predicted anchor box and the sample anchor box, the anchor box width w of the sample anchor box can be determined respectively gt and the anchor box height hgt , and determine the edge line position of the predicted anchor frame and the edge line position of the sample anchor frame. According to the edge line position of the predicted anchor frame and the edge line position of the sample anchor frame, the distances between the left edge line, right edge line, upper edge line and lower edge line can be determined respectively, which are recorded as dw1, dw2, dh1, dh2 respectively. Based on the edge line distance, the parameter p used to characterize the position deviation can be determined according to formula (5):

[0107] (5)

[0108] According to an embodiment of the present disclosure, the loss function (Pixels-IoULoss, PIoU) of the bounding box regression with the gradient adjustment mechanism is as shown in formula (6):

[0109] (6)

[0110] Among them, IoU is the value of the IoU loss function.

[0111] According to an embodiment of the present disclosure, based on PIoU, a loss function as shown in formula (7) can be determined:

[0112] (7)

[0113] According to an embodiment of the present disclosure, in order to enhance the ability to focus on medium-quality predicted anchor boxes, the non-monotonic attention mechanism can be combined with PIoU to obtain the PIoU2 loss function, as shown in formula (8):

[0114] (8)

[0115] Where λ is the scaling factor, u(·) is the non-monotonic attention mechanism function, and q=e -p .

[0116] According to the embodiments of the present disclosure, by improving the loss function, the model can pay more attention to medium-quality anchor frames during the parameter adjustment process, so that medium-quality anchor frames can be used more for feature learning during training. The effective gradient signals provided by medium-quality anchor frames can be used to enable the model to learn a wider range of features, reduce dependence on high-quality anchor frames, reduce noise in the training process, and improve the generalization ability of the model.

[0117] Based on the above smoke detection method, the present disclosure also provides a smoke detection device. Figure 6 The device is described in detail.

[0118] Figure 6 The structural block diagram of the smoke detection device according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 6 As shown, the smoke detection device 600 of this embodiment includes a data preprocessing module 610 , a feature extraction module 620 , a feature aggregation module 630 , a sub-result determination module 640 and a result determination module 650 .

[0120] The data preprocessing module 610 is used to preprocess and vectorize the environmental image captured by the image acquisition device using the smoke detection model to determine the input data for smoke detection. In one embodiment, the data preprocessing module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0121] Feature extraction module 620 is configured to perform multiple feature extractions on the input data using multiple dynamic convolution kernels to obtain multiple data features of different dimensions. The multiple dynamic convolution kernels are determined based on a dynamic convolution kernel library. In one embodiment, feature extraction module 620 can be configured to perform operation S220 described above, which will not be further described here.

[0122] The feature aggregation module 630 is used to utilize the weighted feature fusion module of the smoke detection model to perform multiple rounds of weighted aggregation on multiple data features to obtain multiple fused features of different scales. In one embodiment, the feature aggregation module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0123] The sub-result determination module 640 is used to input the multiple fusion features into the detection head respectively to determine multiple smoke detection sub-results of different scales. In one embodiment, the sub-result determination module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0124] The result determination module 650 is used to determine the smoke detection result of the environment image based on the multiple smoke detection sub-results. In one embodiment, the result determination module 650 can be used to perform the operation S250 described above, which will not be repeated here.

[0125] According to an embodiment of the present disclosure, the feature aggregation module 630 includes a scale determination submodule, a feature determination submodule, and a feature aggregation submodule.

[0126] The scale determination submodule is used to determine the characteristic scale of each of the multiple data features, wherein the characteristic scale includes the dimension and the number of channels.

[0127] The feature determination submodule is used to determine high-dimensional features and low-dimensional features from multiple data features based on multiple feature scales.

[0128] The feature aggregation submodule is used to perform multiple rounds of weighted aggregation on multiple data features based on high-dimensional features and low-dimensional features to obtain multiple fused features.

[0129] According to an embodiment of the present disclosure, the feature aggregation submodule includes a feature alignment unit, a partition division unit, a weight determination unit, a feature processing unit and a feature fusion unit.

[0130] The feature alignment unit is used to perform data alignment processing on the high-dimensional features, low-dimensional features and data features based on the feature scale of the data features for each data feature to obtain the aligned data features.

[0131] The partition division unit is used to divide the data features into a preset number of partitions according to the channel dimension to obtain a preset number of partition features.

[0132] The weight determination unit is used to process the partition features of each partition using an activation function to determine a fusion weight.

[0133] The feature processing unit is used to use the fusion weight to sum the high-dimensional features and the low-dimensional features to obtain the processed features of the partition.

[0134] The feature fusion unit is used to process the processed features of a preset number of partitions to determine the fusion features.

[0135] According to an embodiment of the present disclosure, the feature fusion unit includes a feature splicing subunit, a feature convolution subunit, a feature difference subunit and a feature activation subunit.

[0136] The feature splicing subunit is used to splice the processed features of a preset number of partitions to obtain a spliced ​​feature, wherein the feature scale of the spliced ​​feature is the same as the feature scale of the data feature corresponding to the spliced ​​feature.

[0137] The feature convolution subunit is used to perform convolution processing on the spliced ​​features to obtain convolution features.

[0138] The feature difference subunit is used to interpolate the convolution features to obtain interpolated features.

[0139] The feature activation subunit is used to process the interpolation features using the activation function to obtain the fusion features.

[0140] According to an embodiment of the present disclosure, the feature extraction module 620 includes a kernel unit determination submodule, a convolution kernel determination submodule, and a feature extraction submodule.

[0141] The kernel unit determination submodule is used to determine multiple kernel units and multiple groups of attention weights included in the dynamic convolution kernel library, wherein each group of attention weights includes the respective attention weights of multiple kernel units.

[0142] The convolution kernel determination submodule is used to determine multiple dynamic convolution kernels based on multiple sets of attention weights and multiple kernel units.

[0143] The feature extraction submodule is used to use multiple dynamic convolution kernels to extract features from the input data and obtain data features.

[0144] According to an embodiment of the present disclosure, the sub-result determination module 640 includes a feature input sub-module.

[0145] The feature input submodule is used to input multiple fusion features into the detection head corresponding to the scale of the fusion features, and output the smoke detection sub-result corresponding to the scale of the fusion features.

[0146] According to an embodiment of the present disclosure, the smoke detection device 600 also includes a sample data preprocessing module, a sample weight determination module, a sample convolution kernel determination module, a sample feature extraction module, a sample feature aggregation module, a sample sub-result determination module, a sample result determination module and a parameter adjustment module.

[0147] The sample data preprocessing module is used to preprocess and vectorize the sample environment image using the initial smoke detection model to obtain sample input data.

[0148] The sample weight determination module is used to determine multiple groups of sample attention weights based on sample input data.

[0149] The sample convolution kernel determination module is used to determine multiple sample dynamic convolution kernels based on multiple groups of sample attention weights and a dynamic convolution kernel library.

[0150] The sample feature extraction module is used to perform multiple feature extractions on the sample input data using multiple sample dynamic convolution kernels to obtain multiple sample data features of different dimensions.

[0151] The sample feature aggregation module is used to use the initial weighted feature fusion module of the sample smoke detection model to perform multiple rounds of weighted aggregation on multiple sample data features to obtain multiple sample fusion features of different scales.

[0152] The sample sub-result determination module is used to input multiple sample fusion features into the detection head respectively to determine multiple sample smoke detection sub-results of different scales.

[0153] The sample result determination module is used to determine the sample smoke detection result based on multiple sample smoke detection sub-results.

[0154] The parameter adjustment module is used to adjust the model parameters of the initial smoke detection model based on the sample smoke detection results and the sample labels of the sample environment images until the prediction error between the sample smoke detection results and the sample labels is less than a preset threshold, and determine the smoke detection model based on the current model parameters of the initial smoke detection model.

[0155] According to an embodiment of the present disclosure, the parameter adjustment module includes an error determination submodule, a learning rate determination submodule and a parameter adjustment submodule.

[0156] The error determination submodule is used to determine the prediction error between the sample smoke detection result and the sample label of the sample environment image.

[0157] The learning rate determination submodule is used to determine the current learning rate based on an adaptive learning rate determination method.

[0158] The parameter adjustment submodule is used to adjust the model parameters of the initial smoke detection model based on the current learning rate and prediction error.

[0159] According to an embodiment of the present disclosure, the error determination submodule includes an anchor frame determination unit, a distance determination unit, and an error determination unit.

[0160] The anchor frame determination unit is used to determine the edge line of the predicted anchor frame, the edge line of the sample anchor frame, and the anchor frame width and anchor frame height of the sample anchor frame based on the predicted anchor frame corresponding to the sample smoke detection result and the sample anchor frame corresponding to the sample label.

[0161] The distance determination unit is used to determine the edge line distance between the edge line of the predicted anchor box and the corresponding edge line of the sample anchor box.

[0162] The error determination unit is used to determine the prediction error based on the edge line distance, the anchor box width and the anchor box height.

[0163] According to an embodiment of the present disclosure, any multiple modules among the data preprocessing module 610, feature extraction module 620, feature aggregation module 630, sub-result determination module 640, and result determination module 650 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present disclosure, at least one of the data preprocessing module 610, feature extraction module 620, feature aggregation module 630, sub-result determination module 640, and result determination module 650 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of these. Alternatively, at least one of the data preprocessing module 610, the feature extraction module 620, the feature aggregation module 630, the sub-result determination module 640 and the result determination module 650 can be at least partially implemented as a computer program module, which can perform the corresponding function when it is executed.

[0164] Figure 7 The block diagram schematically shows an electronic device suitable for implementing the smoke detection method according to an embodiment of the present disclosure.

[0165] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0166] Various programs and data required for the operation of the electronic device 700 are stored in the RAM 703. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0167] According to an embodiment of the present disclosure, electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to bus 704. Electronic device 700 may also include one or more of the following components connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or modem. Communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 710 as needed, so that computer programs read from the removable media can be installed into storage section 708 as needed.

[0168] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0169] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above, and / or one or more memories other than ROM 702 and RAM 703.

[0170] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiments of the present disclosure.

[0171] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the processor 701 executes the computer program. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0172] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0173] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the processor 701, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0174] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0176] Those skilled in the art will appreciate that the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of the present disclosure. All such combinations and / or couplings fall within the scope of the present disclosure.

[0177] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A smoke detection method, characterized in that: The method comprises: Using the smoke detection model, preprocess and vectorize the environmental image collected by the image acquisition device to determine the input data for smoke detection; Performing multiple feature extractions on the input data using multiple dynamic convolution kernels to obtain multiple data features of different dimensions, wherein the multiple dynamic convolution kernels are determined according to a dynamic convolution kernel library; Using the weighted feature fusion module of the smoke detection model, the multiple data features are subjected to multiple rounds of weighted aggregation to obtain multiple fusion features of different scales; Inputting the multiple fused features into the detection head respectively to determine multiple smoke detection sub-results of different scales; and A smoke detection result of the environment image is determined based on the multiple smoke detection sub-results.

2. The method according to claim 1, characterized in that The weighted feature fusion module is used to perform multiple rounds of weighted aggregation on the multiple data features to obtain multiple fusion features of different scales, including: Determining a characteristic scale of each of the plurality of data features, wherein the characteristic scale includes a dimension and a number of channels; Based on the plurality of feature scales, determining high-dimensional features and low-dimensional features from the plurality of data features; and Based on the high-dimensional features and the low-dimensional features, multiple rounds of weighted aggregation are performed on the multiple data features to obtain the multiple fusion features.

3. The method according to claim 2, characterized in that The performing multiple rounds of weighted aggregation on the multiple data features based on the high-dimensional features and the low-dimensional features to obtain the multiple fusion features includes: For each of the data features, Based on the feature scale of the data feature, performing data alignment processing on the high-dimensional feature, the low-dimensional feature and the data feature to obtain aligned data features; Dividing the data features into a preset number of partitions according to the channel dimension to obtain a preset number of partition features; For each of the partitions, Processing the partition features of the partitions using an activation function to determine a fusion weight; Using the fusion weight, summing the high-dimensional features and the low-dimensional features to obtain processed features of the partition; and The processed features of each of the preset number of partitions are processed to determine the fusion features.

4. The method according to claim 3, characterized in that The processing of the processed features of the preset number of partitions to determine the fusion features includes: Splicing the processed features of the preset number of partitions to obtain a spliced ​​feature, wherein a feature scale of the spliced ​​feature is the same as a feature scale of a data feature corresponding to the spliced ​​feature; Performing convolution processing on the splicing features to obtain convolution features; Performing interpolation processing on the convolution feature to obtain an interpolation feature; and The interpolation feature is processed by using the activation function to obtain the fusion feature.

5. The method according to claim 1, characterized in that The method uses multiple dynamic convolution kernels to perform multiple feature extractions on the input data to obtain multiple data features of different dimensions, including: Determine a plurality of core units and a plurality of groups of attention weights included in the dynamic convolution core library, wherein each group of attention weights includes respective attention weights of a plurality of the plurality of core units; Determining the multiple dynamic convolution kernels based on the multiple groups of attention weights and the multiple kernel units; The multiple dynamic convolution kernels are used to extract features of the input data to obtain the data features.

6. The method according to claim 1, characterized in that The step of inputting the plurality of fused features into the detection head respectively to determine a plurality of smoke detection sub-results of different scales includes: The multiple fused features are respectively input into detection heads corresponding to the scales of the fused features, and smoke detection sub-results corresponding to the scales of the fused features are output.

7. The method according to claim 1, characterized in that The smoke detection model is trained in the following way: Using the initial smoke detection model, the sample environment image is preprocessed and vectorized to obtain sample input data; Determine multiple groups of sample attention weights based on the sample input data; Determine a plurality of sample dynamic convolution kernels based on the plurality of groups of sample attention weights and the dynamic convolution kernel library; Using the multiple sample dynamic convolution kernels to perform multiple feature extractions on the sample input data to obtain multiple sample data features of different dimensions; Using the initial weighted feature fusion module of the sample smoke detection model, the multiple sample data features are subjected to multiple rounds of weighted aggregation to obtain multiple sample fusion features of different scales; Inputting the plurality of sample fusion features into the detection head respectively to determine the plurality of sample smoke detection sub-results of different scales; determining a sample smoke detection result based on the plurality of sample smoke detection sub-results; as well as Based on the sample smoke detection result and the sample label of the sample environment image, the model parameters of the initial smoke detection model are adjusted until the prediction error between the sample smoke detection result and the sample label is less than a preset threshold, and the smoke detection model is determined based on the current model parameters of the initial smoke detection model.

8. The method according to claim 7, characterized in that The adjusting the model parameters of the initial smoke detection model based on the sample smoke detection result and the sample label of the sample environment image includes: Determining the prediction error between the sample smoke detection result and the sample label of the sample environment image; Determining a current learning rate based on an adaptive learning rate determination method; and Based on the current learning rate and the prediction error, model parameters of the initial smoke detection model are adjusted.

9. The method according to claim 8, characterized in that The determining the prediction error between the sample smoke detection result and the sample label of the sample environment image comprises: Based on the predicted anchor frame corresponding to the sample smoke detection result and the sample anchor frame corresponding to the sample label, determining the edge line of the predicted anchor frame, the edge line of the sample anchor frame, and the anchor frame width and anchor frame height of the sample anchor frame; Determining an edge line distance between an edge line of the predicted anchor frame and a corresponding edge line of the sample anchor frame; and The prediction error is determined based on the edge line distance, the anchor box width, and the anchor box height.

10. A smoke detection device, characterized in that: The device comprises: A data preprocessing module is used to preprocess and vectorize the environmental image collected by the image acquisition device using the smoke detection model to determine the input data for smoke detection; A feature extraction module, used to perform multiple feature extractions on the input data using multiple dynamic convolution kernels to obtain multiple data features of different dimensions, wherein the multiple dynamic convolution kernels are determined according to a dynamic convolution kernel library; A feature aggregation module, used to perform multiple rounds of weighted aggregation on the multiple data features using the weighted feature fusion module of the smoke detection model to obtain multiple fusion features of different scales; a sub-result determination module, configured to input the plurality of fused features into a detection head respectively, and determine a plurality of smoke detection sub-results of different scales; and The result determination module is used to determine the smoke detection result of the environment image based on the multiple smoke detection sub-results.

Images