Network model optimization method and system for fire smoke detection, and storage medium

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a network model optimization method and system for fire smoke detection and a storage medium, and the method comprises the steps: collecting a fire smoke data set which comprises a positive sample and a negative sample, and calculating the similarity between different frames in a fire smoke video, the method comprises the following steps: extracting fire smoke feature images capable of representing different states from a fire smoke video, adding the fire smoke feature images into a fire smoke data set, analyzing all the images in the fire smoke data set to obtain an analysis result, and performing different processing modes on different images based on the analysis result, the method comprises the following steps of: processing a target detection model of fire smoke, taking processed images as a training data set to train the target detection model for identifying the fire smoke, fusing a CBAM attention mechanism to extract image features in the process of training the target detection model, and optimizing the target detection model by adjusting parameters. According to the invention, the recognition speed and recognition accuracy of the model can be improved by optimizing the target detection model.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a network model optimization method, system and storage medium for fire smoke detection. Background Art

[0002] With the acceleration of urbanization, fire safety issues are becoming increasingly prominent. Traditional fire smoke detection methods have problems such as slow detection speed and low accuracy. In recent years, deep learning technology has made significant progress in the field of image recognition, especially the YOLO series models have performed well in target detection tasks. However, the detection of fire smoke requires higher real-time and accuracy of the model, so it is necessary to further optimize the model to adapt to complex environments. A similar prior art Chinese patent application with publication number CN110096942A provides a smoke detection algorithm based on video analysis, the production of a training set, first obtaining pictures containing smoke from existing forest smoke videos, manually marking the smoke area, producing label images, training a full-volume network model, using the trained network model, obtaining suspected smoke areas in single-frame images, dynamic growth analysis of suspected smoke area areas, calculating the formula for the sum of suspected smoke area areas corresponding to continuous video sequences, and calculating the formula for the relative growth of suspected smoke area areas. However, the document does not consider the problem that the insufficient training set will reduce the accuracy of the model. Similar prior art also includes a Chinese patent application with publication number CN112507925A, which discloses a fire detection method based on slow feature analysis, extracting frame images from real-time video data of buildings, and preprocessing the frame images, using slow feature analysis to extract flame parameter features in the frame images, and based on the global features of the flame parameters, combining fuzzy logic algorithms to quickly locate the suspected flame area in the extracted frame images, and perform local feature analysis of the flame parameters, and grid segment the preprocessed frame images to obtain sub-images, and match the feature points of each sub-image with the standard flame image, and accurately judge whether a fire has occurred based on the results of the local feature analysis of the flame parameters and the feature point matching results. However, the document does not consider the problem of slow processing speed when processing images, and thus, the present invention provides a network model optimization method, system and storage medium for fire smoke detection. Summary of the invention

[0003] The present invention increases the richness of fire smoke data by preparing a fire smoke dataset, including positive samples and negative samples. In order to further increase the richness of the fire smoke dataset, fire smoke videos are also collected, and fire smoke feature images in different states are extracted from the fire smoke videos. Since the fire smoke dataset has a large amount of data, in order to quickly process the images in the fire smoke dataset, different image processing methods are selected for the images according to different image features, thereby achieving the purpose of improving fire smoke detection performance.

[0004] In order to achieve the above-mentioned object of the invention, the present invention provides the following network model optimization method for fire smoke detection, which is implemented by executing the following steps:

[0005] Collect a fire smoke dataset, wherein the fire smoke dataset includes positive samples and negative samples, wherein the positive samples include fire smoke images and simulated smoke images in different environments and conditions, and the negative samples include similar images similar to the fire smoke images, and also collect fire smoke videos;

[0006] For the acquired fire smoke video, by calculating the similarity between different frames in the fire smoke video, extracting fire smoke feature images that can represent different states from the fire smoke video, and adding the extracted fire smoke feature images to the fire smoke dataset;

[0007] Analyze all images in the fire smoke dataset to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as training datasets to train a target detection model for identifying fire smoke;

[0008] In the process of training the target detection model, the CBAM attention mechanism is also integrated to extract image features, the target detection model is optimized by adjusting parameters, and the model pruning technology is also applied to reduce the computational complexity of the model.

[0009] As a preferred technical solution of the present invention, extracting fire smoke feature images that can represent different states from the fire smoke video includes the following steps:

[0010] Based on a preset time period, a plurality of first images are intercepted from the fire smoke video, a first of the first images is used as a standard image, the standard image is converted into a second image, the second image is a single-channel image containing only brightness information, a scaling algorithm is used to adjust the second image to a preset size, a third image is extracted from the second image using a preset method, a contour is extracted from the third image to obtain a fourth image, and an attribute code is generated based on the fourth image;

[0011] Acquire a second first image, generate the attribute code of the second first image, calculate the similarity between the second first image and the first first image based on the attribute code, if the similarity is greater than or equal to a first threshold, continue to acquire a third first image and generate the corresponding attribute code, calculate the similarity between the third first image and the first first image, if the similarity is less than the first threshold, add the standard image as the smoke feature image to the fire smoke dataset, and use the third first image as a new standard image, and continue to compare it with the remaining first images until all the first images are compared.

[0012] As a preferred technical solution of the present invention, generating attribute code based on the fourth image includes the following steps:

[0013] The fourth image is evenly divided into a plurality of squares, and for each of the squares, the difference in grayscale value between a central pixel and an adjacent pixel is calculated to generate a plurality of the differences, and the average of the absolute values ​​of the plurality of the differences is also calculated, and for each pixel in the square, the corresponding difference and the average are compared, and if the difference is greater than the average, the value of the corresponding pixel is set to one, otherwise, the value of the corresponding pixel is set to zero, and the values ​​corresponding to each pixel in each of the squares are combined to generate a first attribute code corresponding to the square, and the third image corresponds to a plurality of the first attribute codes, and a plurality of the first attribute codes are combined to generate the attribute code.

[0014] As a preferred technical solution of the present invention, all images in the fire smoke data set are analyzed to obtain analysis results, including the following steps:

[0015] A first analysis is performed on the images in the fire smoke dataset to obtain a first analysis result, wherein the first analysis refers to analyzing basic attribute information of the image, and the first analysis result includes type data, size data, number of pixels, brightness data, and clarity of the image; a second analysis is performed on the images in the fire smoke dataset to obtain a second analysis result, wherein the second analysis refers to dividing the brightness data into multiple categories, analyzing the number of pixels in each brightness category in the image, segmenting the image to obtain multiple segmented areas, analyzing the number of pixels in each brightness category in each segmented area, and the second analysis result includes brightness distribution data and contrast data of the image; a third analysis is performed on the images in the fire smoke dataset to obtain a third analysis result, wherein the third analysis refers to analyzing features and quality of the image, and the third analysis result refers to edge and texture features of the image, color distribution, and whether the image is blurred.

[0016] As a preferred technical solution of the present invention, different processing methods are used for different images based on the analysis results, including the following steps:

[0017] A plurality of processing methods are predefined, and the plurality of processing methods include image value normalization, brightness adjustment, clarity adjustment and histogram smoothing. A plurality of different images with different analysis results are selected from the fire smoke dataset as first test images, and the first test images are processed using different processing methods respectively, and the processed images are obtained as second test images. Different second test images are input into the target detection model to obtain detection results, and influences of different processing methods on the detection results are obtained based on the detection results, and optimal processing methods corresponding to different analysis results are determined. An optimal processing method matching table is created based on the analysis results and the corresponding optimal processing methods, and images in the fire smoke dataset are processed based on the optimal processing method matching table.

[0018] As a preferred technical solution of the present invention, training the target detection model includes the following steps:

[0019] The training data set is divided into multiple data types, a number of first data and a number of second data are selected from the training data set of each data type as training sample data, a feature vector of each training sample in the training sample data is extracted, and the proportion of the first data and the second data in each data type is estimated based on the extracted feature vector, and a first difference, a second difference and a third difference corresponding to each data type are calculated based on the proportion, and a total difference is calculated based on the first difference, the second difference and the third difference. In the process of calculating the total difference, the total difference is reduced by updating parameters, and this step is repeated until the total difference is less than or equal to a preset second threshold.

[0020] As a preferred technical solution of the present invention, respectively calculating the first difference, the second difference and the third difference based on the proportions includes the following steps:

[0021] Randomly selecting three first data from data belonging to the same data type, and obtaining feature vectors of the three first data, calculating sub-differences between the three feature vectors using a preset method, obtaining all combinations of the three first data and calculating sub-differences of each combination, and summing all calculated sub-differences to obtain a first difference;

[0022] Randomly select two of the first data and one of the second data from the same data type, and obtain feature vectors, use the preset method to calculate sub-differences between the three feature vectors, and also obtain all combinations of two of the first data and one of the second data and calculate the sub-differences of each combination, sum all the sub-differences to obtain a first sub-difference sum C1, and use a first formula to calculate the second difference D2, the first formula is: D2 = C1-r*a, where r refers to the proportion of the second data in the current data type, and a refers to the preset maximum difference between any two data in the same data type;

[0023] Select one of the first data and one of the second data in the same data type, obtain one of the second data in other data types, obtain the corresponding feature vector, use the preset method to calculate the sub-differences between the three feature vectors, obtain all combinations of one of the first data, one of the second data and one of the second data in any other data type belonging to the same data type, and calculate the sub-differences of each combination, sum all the calculated sub-differences to obtain a second sub-difference sum C2, and use the second formula to calculate the third difference D3, the second formula is: D3 = C2-(1-r)*a.

[0024] As a preferred technical solution of the present invention, calculating the total difference based on the first difference, the second difference and the third difference comprises the following steps:

[0025] The total difference D is calculated using a first formula, which is:

[0026]

[0027] Wherein n is the number of data types, D1, D2, D3 are the first difference, the second difference and the third difference respectively, and α refers to an adjustment parameter.

[0028] The present invention also provides a network model optimization system for fire smoke detection, comprising the following modules:

[0029] A collection module is used to collect a fire smoke dataset, wherein the fire smoke dataset includes positive samples and negative samples, wherein the positive samples include fire smoke images and simulated smoke images in different environments and conditions, and the negative samples include similar images similar to the fire smoke images, and fire smoke videos are also collected;

[0030] an expansion module, for extracting fire smoke feature images representing different states from the fire smoke video by calculating the similarity between different frames in the fire smoke video, and adding the extracted fire smoke feature images to the fire smoke dataset;

[0031] A processing module, used to analyze all images in the fire smoke data set to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as training data sets to train a target detection model for identifying fire smoke;

[0032] The training module is used to extract image features by incorporating the CBAM attention mechanism during the training of the target detection model, optimize the target detection model by adjusting parameters, and apply model pruning technology to reduce the amount of model calculation.

[0033] The present invention also provides a storage medium, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above methods.

[0034] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0035] In the present invention, a fire smoke dataset is first prepared, including positive samples and negative samples, thereby increasing the richness of the fire smoke data. In order to further increase the richness of the fire smoke dataset, fire smoke videos are also collected, and fire smoke feature images in different states are extracted from the fire smoke videos. Since the fire smoke dataset has a large amount of data, in order to quickly process the images in the fire smoke dataset, different image processing methods are selected for the images according to different image features, thereby achieving the purpose of improving the fire smoke detection performance. Finally, in the process of training the target detection model, the attention mechanism is integrated into the extraction of image features. Through the combination of channel attention and spatial attention, the model's recognition ability of fire smoke features is enhanced. The deep learning efficiency is also improved through model pruning, so that the target detection model can reduce the computational complexity and resource consumption while maintaining performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of the steps of the network model optimization method for fire smoke detection of the present invention;

[0037] Figure 2 This is a structural diagram of the network model optimization system for fire smoke detection of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0040] The present invention provides Figure 1 The network model optimization method for fire smoke detection shown is implemented by executing the following steps:

[0041] Step S1, collecting a fire smoke dataset, the fire smoke dataset includes positive samples and negative samples, the positive samples include fire smoke images and simulated smoke images in different environments and conditions, the negative samples include similar images similar to the fire smoke images, and fire smoke videos are also collected.

[0042] Specifically, in order to monitor fire smoke, we first need to collect relevant data on fire smoke to create a fire smoke dataset. In order to provide rich data resources for training and optimizing fire smoke detection, positive samples and negative samples are collected. Positive samples include fire smoke images and simulated smoke images in different environments and under different conditions. Fire smoke images refer to images taken when a real fire occurs. Simulated smoke images refer to fire smoke images artificially created and simulated under certain environmental conditions. In order to improve the accuracy of fire smoke detection, images that are not fire smoke images but are similar to fire smoke images are also collected as negative samples. In order to ensure the richness of the fire smoke dataset, videos of fires are also collected, and video data can be used to obtain image data at various stages of the fire process.

[0043] Step S2: for the acquired fire smoke video, by calculating the similarity between different frames in the fire smoke video, extracting fire smoke feature images that can represent different states from the fire smoke video, and adding the extracted fire smoke feature images to the fire smoke dataset.

[0044] Specifically, after obtaining the fire smoke video data, due to the continuity of the fire smoke video, there is great similarity between adjacent frames of the fire smoke video data. If a large amount of similar image data is obtained as training data, it will not only reduce the efficiency of fire smoke detection but also reduce the accuracy of fire smoke detection. Therefore, based on the similarity between different frames in the fire smoke video, fire smoke feature images that can represent different states are extracted from the fire smoke video, and the extracted fire smoke feature images are added to the fire smoke dataset to increase the data volume of the fire smoke dataset, which serves as data support for the subsequent training of the fire smoke detection model and model optimization.

[0045] Step S3: Analyze all images in the fire smoke dataset to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as a training dataset to train a target detection model for identifying fire smoke.

[0046] Specifically, a large number of images are collected in the fire smoke dataset, and the images in the fire smoke dataset need to be analyzed and processed. In order to improve the performance of fire smoke detection, an appropriate processing method can be selected according to the characteristics of the image to solve the problems of long processing time and data feature loss that may be caused by using the same processing method. Therefore, the analysis results are obtained based on the analysis of the images in the fire smoke dataset, and different processing methods are used for different images based on the analysis results. The specific analysis and processing method selection will be explained in detail later, and the processed images are used as a training dataset to train a target detection model for identifying fire smoke.

[0047] Step S4: In the process of training the target detection model, the CBAM attention mechanism is also integrated to extract image features, the target detection model is optimized by adjusting parameters, and the model pruning technology is also applied to reduce the computational complexity of the model.

[0048] Specifically, the target detection model is trained using the training data set. The specific training process will be explained in detail later. In the process of training the target detection model, due to the complexity and diversity of fire smoke data, the target detection has problems of inaccurate positioning and false detection. In order to enhance the accuracy and robustness of model detection, the CBAM attention mechanism is integrated when extracting image features. Due to the limitation of hardware resources, the target detection model has high requirements on computing power. In order to reduce the computational complexity and resource consumption while maintaining high performance, model pruning is also used to reduce the complexity and computational amount of the target detection model to improve the efficiency of deep learning.

[0049] Through the above method, a fire smoke dataset is first prepared, including positive samples and negative samples, which increases the richness of the fire smoke data. In order to further increase the richness of the fire smoke dataset, fire smoke videos are also collected, and fire smoke feature images in different states are extracted from the fire smoke videos. Since the fire smoke dataset has a large amount of data, in order to quickly process the images in the fire smoke dataset, different image processing methods are selected for the images according to different image features, thereby achieving the purpose of improving the fire smoke detection performance. Finally, in the process of training the target detection model, the attention mechanism is integrated into the image feature extraction to enhance the accuracy and robustness of the model detection. The deep learning efficiency is improved through model pruning, so that the target detection model can reduce the computational complexity and resource consumption while maintaining the performance.

[0050] Furthermore, extracting fire smoke feature images that can represent different states from the fire smoke video includes the following steps:

[0051] Based on a preset time period, multiple first images are captured from a fire smoke video, the first first image is used as a standard image, the standard image is converted into a second image, the second image is a single-channel image containing only brightness information, a scaling algorithm is used to adjust the second image to a preset size, a third image is extracted from the second image using a preset method, contour extraction is performed on the third image to obtain a fourth image, and attribute coding is generated based on the fourth image.

[0052] Specifically, in order to quickly and accurately extract fire smoke feature images from fire smoke videos, since the differences between video frames at adjacent times are not much or even exactly the same, multiple first images are first captured from the fire smoke video based on a preset time period. The preset period is generally set to 500ms or 1s. Although there is a time interval between the multiple first images, there is also a problem of no difference between the multiple first images. If there are many identical images in the fire smoke dataset, the image processing speed will be reduced, thereby reducing the training speed of the target detection model and the recognition accuracy of the target detection model. Therefore, images with larger differences are selected from the multiple first images as fire smoke feature images and added to the fire smoke dataset.

[0053] Multiple first images are stored in the order in which they are captured. First, the first image is used as a standard image, and then the standard image is converted into a single-channel image containing only brightness information, namely a second image, which can also be called a grayscale image. In order to reduce computational complexity and improve processing speed, an image scaling algorithm is used to adjust the second image to a preset size, such as uniformly adjusting it to 720×480 pixels. The scaling algorithm is a prior art and has a variety of options such as bilinear difference. Then, in order to identify and separate the area of ​​the main object from the second image, a preset method such as Gaussian normal distribution is used to extract a third image from the second image. The third image refers to the foreground image of the second image. Then, edge mask detection is used to extract the contour of the third image to obtain a fourth image and generate attribute coding based on the fourth image. The attribute coding refers to the feature vector of the image, and the specific generation method will be explained in detail later.

[0054] The second first image is obtained, and an attribute code of the second first image is generated. The similarity between the second first image and the first first image is calculated based on the attribute code. If the similarity is greater than or equal to a first threshold, the third first image is obtained and a corresponding attribute code is generated. The similarity between the third first image and the first first image is calculated. If the similarity is less than the first threshold, the standard image is added to the fire smoke dataset as a smoke feature image, and the third first image is used as a new standard image, and is continued to be compared with the remaining first images until all first images are compared.

[0055] Specifically, in order to detect the similarity between two adjacent first images, after generating the attribute code of the first first image, the same method is used to generate the second first image and generate the corresponding attribute code, and the similarity between the two is calculated based on the attribute code. If the similarity is greater than or equal to the first threshold, it means that the second first image and the first first image are highly similar, and there is no need to add it as a smoke feature image to the fire smoke dataset. Then, the attribute code of the third first image is obtained, and the similarity is calculated with the attribute code of the first first image, which is the standard image. If the similarity between the two is less than the first threshold, it means that the similarity between the two is not high, and the standard image, which is the first first image, is added to the fire smoke dataset. The third first image is also used as a new standard image to continue to be compared with the remaining first images. After all the first images are compared, all first images with low similarity are added as fire smoke feature images to the fire smoke dataset, providing a data basis for subsequent training of target detection models for identifying fire smoke.

[0056] Further, generating the attribute code based on the fourth image includes the following steps:

[0057] The fourth image is evenly divided into multiple squares, and the difference in grayscale value between the central pixel and the adjacent pixel of each square is calculated to generate multiple differences. The average of the absolute values ​​of the multiple differences is also calculated. For each pixel in the square, the corresponding difference and the average are compared. If the difference is greater than the average, the value of the corresponding pixel is set to one, otherwise the value of the corresponding pixel is set to zero. The values ​​corresponding to each pixel in each square are combined to generate a first attribute code corresponding to the square. The third image corresponds to multiple first attribute codes, and the multiple first attribute codes are combined to generate an attribute code.

[0058] Specifically, in order to quickly obtain the attribute coding of the image, the fourth image is first divided into multiple squares. Taking the fourth image being divided into multiple squares of 3×3 pixels as an example, the difference between the central pixel and the adjacent pixels is calculated for each square. For example, the grayscale value of the central pixel is 32, and the grayscale values ​​of the adjacent pixels adjacent to the central pixel are 75, 80, 90, 26, 56, 24, 18, and 39, respectively. The grayscale value differences between the central pixel and the adjacent pixels are 43, 48, 58, -6, 24, -8, -14, and 7, respectively. The average of the absolute values ​​of the multiple differences is 26. The difference between each pixel and the average value, the value of the pixel whose difference is greater than the average value is set to 1, otherwise it is set to 0, and the corresponding first attribute code obtained is (1, 1, 1, 0, 1, 1, 0, 0, 1), and the corresponding first attribute code is calculated for each square using the same method as above, and all the first attribute codes are combined to generate the attribute code corresponding to the fourth image. Subsequently, the attribute code can be used as a feature vector, and the cosine similarity calculation formula can be used to calculate the similarity between the two attribute codes, and the calculated similarity is used as the similarity between the two first images. The cosine similarity calculation formula is a prior art and will not be explained in detail here.

[0059] Furthermore, all images in the fire smoke dataset are analyzed to obtain analysis results, including the following steps:

[0060] A first analysis is performed on the images in the fire smoke dataset to obtain a first analysis result. The first analysis refers to analyzing the basic attribute information of the image. The first analysis result includes the type data, size data, number of pixels, brightness data and clarity of the image. A second analysis is also performed on the images in the fire smoke dataset to obtain a second analysis result. The second analysis refers to dividing the brightness data into multiple categories, analyzing the number of pixels in each brightness category in the image, segmenting the image to obtain multiple segmented areas, analyzing the number of pixels in each brightness category in each segmented area, and the second analysis result includes the brightness distribution data and contrast data of the image. A third analysis is also performed on the images in the fire smoke dataset to obtain a third analysis result. The third analysis refers to analyzing the characteristics and quality of the image. The third analysis result refers to the edge and texture characteristics of the image, the color distribution and whether the image is blurred.

[0061] Specifically, in order to select appropriate image processing methods for different image features and to solve the problems of long processing time and loss of important image data that may result from using a unified image processing method, the first, second and third analyses are first used to perform similar feature analyses on the image and obtain analysis results.

[0062] Furthermore, different processing methods are used for different images based on the analysis results, including the following steps:

[0063] A plurality of processing methods are predefined, including image value normalization, brightness adjustment, clarity adjustment and histogram smoothing. A plurality of different images with different analysis results are selected from the fire smoke dataset as the first test image. Different processing methods are used to perform image processing on the first test image respectively, and the processed image is obtained as the second test image. The different second test images are input into the target detection model to obtain the detection result. The influence of the different processing methods on the detection result is obtained based on the detection result. The optimal processing method corresponding to the different analysis results is determined. An optimal processing method matching table is created based on the analysis result and the corresponding optimal processing method. The image in the fire smoke dataset is processed based on the optimal processing method matching table.

[0064] Specifically, in order to ensure that the most appropriate preprocessing algorithm is selected under different image features, multiple processing methods are predefined, and several images are selected from the fire smoke data set as the first test image. The first test image is processed using the predefined multiple processing methods to obtain a processed second test image, and the second test image is input into the target detection model to obtain the detection result, and the influence of different processing methods on the detection result is obtained. It is determined which processing method can make the detection result of the target detection model more accurate, and the corresponding processing method is used as the optimal processing method of the analysis result corresponding to the first test image. Based on the analysis result and the optimal processing method, an optimal processing method matching table is created. Subsequently, the optimal processing method can be selected for different images according to the optimal processing method matching table. The above method can ensure that the most appropriate preprocessing algorithm is selected under different image features, thereby further improving the accuracy and efficiency of object recognition.

[0065] Furthermore, training the target detection model includes the following steps:

[0066] The training data set is divided into multiple data types, and a number of first data and a number of second data are selected from the training data set of each data type as training sample data, and the feature vector of each training sample in the training sample data is extracted. The proportion of the first data and the second data in each data type is estimated based on the extracted feature vector, and the first difference, the second difference and the third difference corresponding to each data type are calculated based on the proportion. The total difference is calculated based on the first difference, the second difference and the third difference. In the process of calculating the total difference, the total difference is reduced by updating parameters, and this step is repeated until the total difference is less than or equal to a preset second threshold.

[0067] Specifically, when training the target detection model, in order to avoid the overfitting problem caused by the imbalance of the data type of the training data, for example, the amount of positive samples in the training data set is much larger than the amount of negative samples, the parameters of the model are optimized in combination with the different proportions of data types in the training data set, so as to achieve the purpose of effectively training the target detection model even when the proportions of data of different data types are different, thereby improving the accuracy of the target detection model.

[0068] A number of first data and second data are obtained from the training data set, where the first data refers to image data with predefined labels added, and the second data refers to image data without predefined labels added. The feature vector of each sample data in the training sample data is extracted. The first difference, the second difference and the third difference refer to using the loss function to calculate the loss in different situations in the same data type, and considering the proportion of the first data and the second data in the same data type. The specific first difference, second difference and third difference calculation methods will be explained in detail later. When calculating the total difference, the proportion is also used as a weight to weight the loss of each data type, so that for the data type with a larger proportion of the second data, the calculation of its difference contributes more to the total difference. The total difference refers to the total loss, which reflects the performance of the model under the current parameters. In the process of calculating the total difference, the total difference is reduced by updating the parameters. The smaller the total difference, the better the performance of the model. Repeat the above process until the total difference is less than or equal to the preset second threshold, indicating that the performance of the model has reached expectations and the training model can be stopped.

[0069] Further, the first difference, the second difference and the third difference are calculated based on the proportions, including the following steps:

[0070] Randomly select three first data from data belonging to the same data type, and obtain feature vectors of the three first data, calculate sub-differences between the three feature vectors using a preset method, obtain all combinations of the three first data and calculate sub-differences of each combination, and sum all calculated sub-differences to obtain a first difference;

[0071] Randomly select two first data and one second data from the same data type, and obtain feature vectors, use a preset method to calculate sub-differences between three feature vectors, obtain all combinations of two first data and one second data and calculate sub-differences of each combination, sum all sub-differences to obtain a first sub-difference sum C1, and use a first formula to calculate a second difference D2, the first formula is: D2 = C1-r*a, where r refers to the proportion of the second data in the current data type, and a refers to a preset maximum difference between any two data in the same data type;

[0072] Select a first data and a second data in the same data type, obtain a second data in other data types, obtain the corresponding eigenvectors, use a preset method to calculate the sub-differences between the three eigenvectors, obtain all combinations of a first data, a second data in the same data type and a second data in any other data type, and calculate the sub-differences of each combination, sum all the calculated sub-differences to obtain a second sub-difference sum C2, and use the second formula to calculate the third difference D3, the second formula is D3=C2-(1-r)*a.

[0073] Specifically, the preset method refers to obtaining three feature vectors x, y, and z, calculating the difference S(x, y) between x and y, and the difference S(x, z) between x and z through the similarity calculation formula, and using the following formula to calculate the sub-difference d=max{(S(x, y)-S(x, z)+a), 0}. The first difference, the second difference, and the third difference are calculated through the above steps. The first difference refers to the difference calculated for the first data of the same data type, that is, the labeled data. The second difference takes into account the first data and the second data, and aims to use the second data to improve the learning process. The third difference also takes into account the first data and the second data, but it focuses on the difference between different data types. The calculation of the first difference, the second difference, and the third difference all depend on a specific combination of data. Through these combinations, the model learns how to distinguish the differences within the same data type and the differences between different data types during the training process, so as to achieve the purpose of enhanced learning.

[0074] Further, calculating the total difference based on the first difference, the second difference and the third difference includes the following steps:

[0075] The total difference D is calculated using the third formula, which is:

[0076]

[0077] Where n is the number of data types, D1, D2, D3 are the first difference, second difference and third difference respectively, and α refers to the adjustment parameter.

[0078] Specifically, the third formula is used to calculate the total difference. In the process of calculating the total difference, the adjustment parameters are updated to minimize the total difference. In the iterative process, the adjustment parameters are gradually optimized to improve the target detection model's ability to recognize fire smoke.

[0079] According to another aspect of the embodiment of the present invention, Figure 2 As shown, a network model optimization system for fire smoke detection is also provided, including a collection module, an expansion module, a processing module and a training module, which are used to implement the network model optimization method for fire smoke detection described above. The specific functions of each module are as follows:

[0080] A collection module is used to collect a fire smoke dataset. The fire smoke dataset includes positive samples and negative samples. The positive samples include fire smoke images and simulated smoke images in different environments and conditions. The negative samples include images similar to the fire smoke images. Fire smoke videos are also collected.

[0081] An extension module is used to extract fire smoke feature images that can represent different states from the acquired fire smoke video by calculating the similarity between different frames in the fire smoke video, and add the extracted fire smoke feature images to the fire smoke dataset;

[0082] A processing module is used to analyze all images in the fire smoke data set to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as training data sets to train the target detection model for identifying fire smoke;

[0083] The training module is used to extract image features by integrating the CBAM attention mechanism during the training of the target detection model, optimize the target detection model by adjusting parameters, and apply model pruning technology to reduce the computational complexity of the model.

[0084] According to another aspect of an embodiment of the present invention, a storage medium is further provided, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned network model optimization methods for fire smoke detection.

[0085] In summary, the network model optimization method, system and storage medium for fire smoke detection of the present invention include collecting a fire smoke data set, the data set includes positive samples and negative samples, and also extracting fire smoke feature images that can represent different states from the fire smoke video by calculating the similarity between different frames in the fire smoke video, and adding them to the fire smoke data set, analyzing all images in the fire smoke data set to obtain analysis results, and using different processing methods for different images based on the analysis results, and using the processed images as a training data set to train a target detection model for identifying fire smoke, and in the process of training the target detection model, integrating the CBAM attention mechanism to extract image features, and optimizing the target detection model by adjusting parameters. The present invention can improve the recognition speed and recognition accuracy of the model by optimizing the target detection model.

[0086] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0088] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0090] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A network model optimization method for fire smoke detection, characterized in that: The steps include: Collect a fire smoke dataset, wherein the fire smoke dataset includes positive samples and negative samples, wherein the positive samples include fire smoke images and simulated smoke images in different environments and conditions, and the negative samples include similar images similar to the fire smoke images, and also collect fire smoke videos; For the acquired fire smoke video, by calculating the similarity between different frames in the fire smoke video, extracting fire smoke feature images that can represent different states from the fire smoke video, and adding the extracted fire smoke feature images to the fire smoke dataset; Analyze all images in the fire smoke dataset to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as training datasets to train a target detection model for identifying fire smoke; In the process of training the target detection model, the CBAM attention mechanism is also integrated to extract image features, the target detection model is optimized by adjusting parameters, and the model pruning technology is also applied to reduce the computational complexity of the model.

2. The method according to claim 1, characterized in that Extracting fire smoke feature images that can represent different states from the fire smoke video includes the following steps: Based on a preset time period, a plurality of first images are intercepted from the fire smoke video, a first of the first images is used as a standard image, the standard image is converted into a second image, the second image is a single-channel image containing only brightness information, a scaling algorithm is used to adjust the second image to a preset size, a third image is extracted from the second image using a preset method, a contour is extracted from the third image to obtain a fourth image, and an attribute code is generated based on the fourth image; Acquire a second first image, generate the attribute code of the second first image, calculate the similarity between the second first image and the first first image based on the attribute code, if the similarity is greater than or equal to a first threshold, continue to acquire a third first image and generate the corresponding attribute code, calculate the similarity between the third first image and the first first image, if the similarity is less than the first threshold, add the standard image as the smoke feature image to the fire smoke dataset, and use the third first image as a new standard image, and continue to compare it with the remaining first images until all the first images are compared.

3. The method according to claim 2, characterized in that Generating an attribute code based on the fourth image comprises the following steps: The fourth image is evenly divided into a plurality of squares, and for each of the squares, the difference in grayscale value between a central pixel and an adjacent pixel is calculated to generate a plurality of the differences, and the average of the absolute values ​​of the plurality of the differences is also calculated, and for each pixel in the square, the corresponding difference and the average are compared, and if the difference is greater than the average, the value of the corresponding pixel is set to one, otherwise, the value of the corresponding pixel is set to zero, and the values ​​corresponding to each pixel in each of the squares are combined to generate a first attribute code corresponding to the square, and the third image corresponds to a plurality of the first attribute codes, and a plurality of the first attribute codes are combined to generate the attribute code.

4. The method according to claim 1, characterized in that: Analyzing all images in the fire smoke dataset to obtain analysis results includes the following steps: A first analysis is performed on the images in the fire smoke dataset to obtain a first analysis result, wherein the first analysis refers to analyzing basic attribute information of the image, and the first analysis result includes type data, size data, number of pixels, brightness data, and clarity of the image; a second analysis is performed on the images in the fire smoke dataset to obtain a second analysis result, wherein the second analysis refers to dividing the brightness data into multiple categories, analyzing the number of pixels in each brightness category in the image, segmenting the image to obtain multiple segmented areas, analyzing the number of pixels in each brightness category in each segmented area, and the second analysis result includes brightness distribution data and contrast data of the image; a third analysis is performed on the images in the fire smoke dataset to obtain a third analysis result, wherein the third analysis refers to analyzing features and quality of the image, and the third analysis result refers to edge and texture features of the image, color distribution, and whether the image is blurred.

5. The method according to claim 1, characterized in that Based on the analysis results, different processing methods are used for different images, including the following steps: A plurality of processing methods are predefined, and the plurality of processing methods include image value normalization, brightness adjustment, clarity adjustment and histogram smoothing. A plurality of different images with different analysis results are selected from the fire smoke dataset as first test images, and the first test images are processed using different processing methods respectively, and the processed images are obtained as second test images. Different second test images are input into the target detection model to obtain detection results, and influences of different processing methods on the detection results are obtained based on the detection results, and optimal processing methods corresponding to different analysis results are determined. An optimal processing method matching table is created based on the analysis results and the corresponding optimal processing methods, and images in the fire smoke dataset are processed based on the optimal processing method matching table.

6. The method according to claim 1, characterized in that Training the target detection model includes the following steps: The training data set is divided into multiple data types, a number of first data and a number of second data are selected from the training data set of each data type as training sample data, a feature vector of each training sample in the training sample data is extracted, and the proportion of the first data and the second data in each data type is estimated based on the extracted feature vector, and a first difference, a second difference and a third difference corresponding to each data type are calculated based on the proportion, and a total difference is calculated based on the first difference, the second difference and the third difference. In the process of calculating the total difference, the total difference is reduced by updating parameters, and this step is repeated until the total difference is less than or equal to a preset second threshold.

7. The method according to claim 6, characterized in that Calculating the first difference, the second difference and the third difference based on the proportions respectively includes the following steps: Randomly selecting three first data from data belonging to the same data type, and obtaining feature vectors of the three first data, calculating sub-differences between the three feature vectors using a preset method, obtaining all combinations of the three first data and calculating sub-differences of each combination, and summing all calculated sub-differences to obtain a first difference; Randomly select two of the first data and one of the second data from the same data type, and obtain feature vectors, use the preset method to calculate sub-differences between the three feature vectors, and also obtain all combinations of two of the first data and one of the second data and calculate the sub-differences of each combination, sum all the sub-differences to obtain a first sub-difference sum C1, and use a first formula to calculate the second difference D2, the first formula is: D2 = C1-r*a, where r refers to the proportion of the second data in the current data type, and a refers to the preset maximum difference between any two data in the same data type; Select one of the first data and one of the second data in the same data type, obtain one of the second data in other data types, obtain the corresponding feature vector, use the preset method to calculate the sub-differences between the three feature vectors, obtain all combinations of one of the first data, one of the second data and one of the second data in any other data type belonging to the same data type, and calculate the sub-differences of each combination, sum all the calculated sub-differences to obtain a second sub-difference sum C2, and use the second formula to calculate the third difference D3, the second formula is: D3 = C2-(1-r)*a.

8. The method according to claim 6, characterized in that Calculating a total difference based on the first difference, the second difference and the third difference comprises the following steps: The total difference D is calculated using a third formula, which is: Wherein n is the number of data types, D1, D2, D3 are the first difference, the second difference and the third difference respectively, and α refers to an adjustment parameter.

9. A network model optimization system for fire smoke detection, used to implement the network model optimization method for fire smoke detection according to any one of claims 1 to 8, characterized in that: Includes the following modules: A collection module is used to collect a fire smoke dataset, wherein the fire smoke dataset includes positive samples and negative samples, wherein the positive samples include fire smoke images and simulated smoke images in different environments and conditions, and the negative samples include similar images similar to the fire smoke images, and fire smoke videos are also collected; an expansion module, for extracting fire smoke feature images representing different states from the fire smoke video by calculating the similarity between different frames in the fire smoke video, and adding the extracted fire smoke feature images to the fire smoke dataset; A processing module, used to analyze all images in the fire smoke data set to obtain analysis results, and use different processing methods for different images based on the analysis results, and use the processed images as training data sets to train a target detection model for identifying fire smoke; The training module is used to extract image features by incorporating the CBAM attention mechanism during the training of the target detection model, optimize the target detection model by adjusting parameters, and apply model pruning technology to reduce the amount of model calculation.

10. A storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the network model optimization method for fire smoke detection according to any one of claims 1 to 8.

Images