A multi-scale flame image detection method based on autonomous learning

Abstract

The present invention discloses a multi-scale flame image detection method based on autonomous learning, which belongs to the field of image detection technology. It includes flame image acquisition to obtain an initial flame image; inputting the obtained initial flame image into an autonomous learning convolutional neural network model for flame image detection; analyzing the flame image detection result and obtaining an analysis result set; feeding back the analysis result set into the autonomous learning convolutional neural network model to adjust and train the autonomous learning convolutional neural network model; wherein the convolutional neural network model is a multi-scale convolutional neural network model. By further feeding back the flame image detection result to the autonomous learning convolutional neural network model, the model is corrected, adjusted and retrained to form a closed-loop feedback; at the same time, the BIM technology is combined with an unsupervised learning algorithm to restore the flame image. The present invention restores the flame scene to the greatest extent, further improving the accuracy of fire detection and the safety and reliability of fire warning decision-making.

Description

Technical Field

[0001] The invention relates to the technical field of image detection, and in particular to a multi-scale flame image detection method based on autonomous learning. Background Art

[0002] Fire is a catastrophic combustion phenomenon in which flames lose control in time and space, causing the flames to spread. The occurrence of fire causes a certain degree of damage to nature and society. Research shows that the occurrence and development and spread of fire are deterministic and random. Determinism means that the spread of fire and the flow and propagation of flames and smoke in a certain occasion and a certain development path follow the laws of fluid flow, heat transmission, and mass conservation. However, fire is also random, and there is uncertainty about when, where, and what factors cause the fire. Therefore, the detection of fire and its prevention and early warning have become the research focus of safety engineering fields including construction, forests, and even space.

[0003] In recent years, as image vision detection technology has the advantages of early detection time, high accuracy, flexible system installation, and the ability to effectively detect fires in large and complex building structures, fire image detection technology has become a research hotspot for fire detection. Early fire image detection methods were based on color recognition when a fire occurred, but this method was very sensitive to brightness and shadows, often causing false alarms and a high false alarm rate. With the development of machine learning, the application of deep learning networks in image recognition and classification has made significant progress, making fire detection algorithms based on deep learning more and more common. For example, convolutional neural networks (CNN) are used for early fire detection, but they are relatively complex, fast convolutional neural networks (R-CNN) are used for regional fire detection, but the detection speed is relatively slow, and smoke detection algorithms such as YOLO-V3 and YOLO-V4 still have problems with flame positioning errors and poor shielding effects in small-scale flame detection.

[0004] The concept of AI autonomous learning has existed in the field of machine learning for a long time, but it has only begun to attract widespread attention in recent years. There have been many studies on deep learning models based on autonomous learning, and scholars are also trying to apply autonomous learning and deep learning to the field of image recognition. However, in actual image recognition applications, autonomous learning still has some limitations and challenges, such as how to conduct autonomous learning in the face of a constantly changing environment and how to ensure the safety and reliability of AI system decisions. It can be seen that fire image detection technology based on autonomous learning is bound to become one of the hot research technologies for fire detection. Summary of the invention

[0005] In order to solve the problems existing in the above background technology, the present invention adopts the following technical solutions:

[0006] The present application provides a multi-scale flame image detection method based on autonomous learning, comprising the following steps:

[0007] Flame image acquisition, obtaining an initial flame image; the flame image acquisition includes acquiring a suspected flame image of a target area, performing a first preprocessing on the acquired suspected flame image to obtain a first preprocessing result, and then transmitting the first preprocessing result to a controller for analysis and judgment;

[0008] The obtained initial flame image is input into the self-learning convolutional neural network model for flame image detection;

[0009] Analyze the flame image detection results and obtain an analysis result set;

[0010] Feedback the analysis result set into the self-learning convolutional neural network model, and adjust and train the self-learning convolutional neural network model;

[0011] Among them, the convolutional neural network model is a multi-scale convolutional neural network model.

[0012] As a preferred solution of the present application, the first preprocessing includes extracting the suspected flame image features to initially determine whether there is a flame, specifically including the following steps:

[0013] Compare the extracted suspected flame features with the corresponding flame features stored in the database of the controller, and output the comparison results;

[0014] Performing flame characteristic correlation calculation on the comparison results;

[0015] The correlation calculation result is compared with the preset flame judgment threshold; if it exceeds the threshold, it is judged that there is a flame and an initial flame image is generated; if it does not exceed the threshold, it is judged that there is no flame and the result is sent to the controller, which issues an instruction to continue flame image acquisition.

[0016] As a preferred solution of the present application, the first preprocessing includes restoring the suspected flame image and judging whether there is a flame according to the restored flame image, which specifically includes the following steps:

[0017] Extracting architectural information from suspected fire images;

[0018] Call the BIM model based on the extracted building information;

[0019] Obstruction information in the suspected flame image and internal structure information of the building where the suspected flame image is located are obtained according to the called BIM model;

[0020] According to the obtained obstruction information and the internal structure information of the building, the obstruction of the suspected flame image is removed, the flame image is restored, and the initial flame image is obtained;

[0021] Wherein, obtaining the initial flame image further comprises: fusing the restored flame image obtained by removing the obstructions with the internal structure information of the building obtained by calling the BIM model.

[0022] As a preferred solution of the present application, the initial flame image obtained is input into the autonomous learning convolutional neural network model for flame image detection, which specifically includes the following process:

[0023] Performing a second preprocessing on the obtained initial flame image to obtain a target flame image;

[0024] Image features of target flame images are extracted through autonomous learning of flame images;

[0025] The extracted image features are subjected to convolution calculation to obtain the flame image detection results.

[0026] As a preferred solution of the present application, the autonomous learning of flame images specifically includes the following steps:

[0027] Flame calibration, determining the position and outline of the flame and other flame parameter information in a single image;

[0028] Flame correction: correct the flame information after calibration and store the relevant correction result parameters;

[0029] Flame feature segmentation: segment the flame features one by one to obtain different flame feature characters;

[0030] Flame feature classification and recognition: the segmented flame feature characters are compared with the flame feature characters stored in the model to perform flame feature classification and recognition, and the flame image features are obtained and stored.

[0031] As a preferred solution of the present application, the convolution calculation of the extracted image features to obtain the flame image detection result specifically includes the following sub-steps:

[0032] Perform convolution operation on the initial flame feature image through a multi-scale convolutional neural network model to obtain a convolution feature image;

[0033] Iteratively process the convolution feature image to obtain feature images under different process conditions;

[0034] The feature images under different process conditions are bidirectionally fused from bottom to top and from top to bottom to obtain the fused target feature image;

[0035] The fused target feature image is detected to obtain the flame image detection result.

[0036] As a preferred solution of the present application, the flame correction specifically includes the following steps:

[0037] Find out the calibrated flame information, including the recorded flame information and non-flame information;

[0038] Perform image enhancement on the found flame information;

[0039] Binarize the enhanced image feature information;

[0040] Analyze the image feature information after removing interference to determine the flame image parameters of the flame position and contour;

[0041] The flame image parameters obtained after analysis are stored to complete flame correction.

[0042] As a preferred solution of the present application, the step of feeding back the analysis result set to the autonomous learning convolutional neural network model specifically includes the following steps:

[0043] Inputting the analysis result set into the training set, experience set and test set of the autonomous learning convolutional neural network model;

[0044] Extract the data in the analysis result set according to the data formats of the training set, experience set and test set of the autonomous learning convolutional neural network model to obtain the result training set, result experience set and result test set corresponding to the training set, experience set and test set respectively;

[0045] Compare the result training set, result experience set and result test set with the training set, experience set and test set of the autonomous learning convolutional neural network model, and output the comparison result;

[0046] According to the comparison results, the training set, experience set and test set of the autonomous learning convolutional neural network model are optimized and adjusted respectively to obtain an optimized training set, an optimized experience set and an optimized test set;

[0047] Using the optimized training set to train the autonomous learning convolutional neural network model;

[0048] Using the optimized validation set to validate the trained self-learning convolutional neural network model, and adjusting parameters according to the validation results;

[0049] The optimized test set is used to evaluate the verified self-learning convolutional neural network model.

[0050] As a preferred solution of the present application, the evaluation indicators include the accuracy of the model, the recall rate of the model and the F1 score of the model.

[0051] As a preferred solution of the present application, the first preprocessing includes initially determining whether there is a flame by extracting features of a suspected flame image; if there is, generating the suspected flame image as an initial flame image; if not, repeating the flame image acquisition step.

[0052] As a preferred solution of the present application, the extracted suspected flame image features include the color, shape and texture features of the suspected target, as well as the area change, edge contour change, flashing frequency and overall movement change of the suspected target.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The multi-scale flame image detection method based on autonomous learning provided by the present application further feeds back the flame image detection results to the autonomous learning convolutional neural network model to correct, adjust and retrain the model. This feedback forms a closed loop between the entire detection method and the autonomous learning convolutional neural network model training. Compared with the prior art that only uses the convolutional neural network model for open-loop detection, the continuous autonomous learning of the model itself plus the result feedback to correct the autonomous learning again is not only conducive to the continuous optimization and correction of the autonomous learning convolutional neural network model, but also makes the image detection results more accurate and closer to the actual scene, further improving the safety and reliability of decision-making.

[0055] This application performs a first preprocessing on the suspected flame image before the self-learning convolutional neural network model, and generates an initial flame image only after confirming the presence of flame. The collected flame image is initially identified to determine whether the flame exists, which greatly reduces the risk of false alarms and waste of manpower costs.

[0056] The flame image detection method of the present invention performs the above-mentioned first preprocessing on the collected suspected flame image, combines the BIM technology with the unsupervised learning algorithm to restore the flame image, and generates an initial flame image by removing the obstructions and the internal structure information of the building at the fire source point. The flame scene can be restored to the greatest extent, greatly improving the efficiency of fire warning and reducing the risk of misjudgment of fire.

[0057] The present application further feeds back the features and judgment results extracted in the first preprocessing process to the autonomous learning convolutional neural network model to correct, adjust and retrain the model. This feedback forms a closed loop for the entire detection method, including the first stage of preprocessing and the autonomous learning convolutional neural network model training therein. This not only improves the efficiency of autonomous learning, but also improves the detection accuracy of flame images.

[0058] This application also introduces an autonomous learning mechanism in each process stage of the construction of the autonomous learning convolutional neural network model, and feeds back the actual detection stage data to the parameter adjustment and correction of each stage of the model, thereby greatly reducing the operating pressure of the detection equipment and improving the reliability of autonomous learning and the safety of decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 A schematic diagram of a process flow of a multi-scale flame image detection method based on autonomous learning provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of a process of performing flame image detection by using a self-learning convolutional neural network model in a multi-scale flame image detection method based on self-learning provided by an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of a process for extracting image features of a multi-scale flame image detection method based on autonomous learning provided by an embodiment of the present invention;

[0063] Figure 4 A schematic diagram of a flame correction process of a multi-scale flame image detection method based on autonomous learning provided by an embodiment of the present invention;

[0064] Figure 5 A schematic diagram of a flow chart of obtaining flame image detection results by convolution calculation of a multi-scale flame image detection method based on autonomous learning provided by an embodiment of the present invention;

[0065] Figure 6 A schematic diagram of a process of correcting, adjusting and retraining an autonomous learning convolutional neural network model in a multi-scale flame image detection method based on autonomous learning provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0066] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0067] The present invention provides a multi-scale flame image detection method based on autonomous learning. It is understandable that in the art, fire flames have their own image features, and the basic features are divided into static features and dynamic features. Specifically, static features mainly include flame color, flame shape, and flame texture, etc.; dynamic features mainly include flame area changes, flame edge contour changes, flame flashing frequency, flame shape changes, and flame overall movement changes, etc.

[0068] The color of the flame depends on two conditions: the temperature of the flame. Different temperatures will produce different colors of the flame; the element of the burning material. Combustion of materials with different elements will produce different spectra. The shape of the flame includes the flame circumference, flame length, flame thickness and flame integrity. The shape of the flame is determined by the shape of the surrounding objects. The flame circumference and flame length indicate the residence time of the flame, the flame thickness indicates the filling degree of the flame, and the flame integrity indicates the closed shape of the flame on any cross section.

[0069] Among them, the dynamic characteristics of flames, such as the change in flame area, the change in flame edge contour, the flame flicker frequency, the change in flame shape, and the change in overall flame movement, can well separate the fire image from other interferences. The change in flame area indicates the extent of flame spread, the change in flame edge contour indicates the law of flame edge change, the flame flicker frequency indicates the energy state of the flame, the change in flame shape reflects the distribution state of the flame in space, and the overall movement change of the flame can reflect the change in the flame spread path and trend.

[0070] AI autonomous learning means that AI systems can explore, learn, and evolve autonomously without any external guidance or supervision. AI autonomous learning has been applied in many fields. For example, in the fields of speech recognition, natural language processing, image recognition, and autonomous driving, AI autonomous learning can already achieve a certain degree of autonomous decision-making and action. In the field of image recognition, AI systems can recognize different objects through autonomous learning, and autonomously learn to generate new images and models.

[0071] The execution subject of the flame image detection method can be a computer, an intelligent terminal, a cloud processor or other intelligent processing equipment capable of logical operations and image processing functions.

[0072] Embodiment 1

[0073] See also Figure 1 The present invention provides a multi-scale flame image detection method based on autonomous learning, comprising the steps of:

[0074] S1, flame image acquisition, obtaining an initial flame image; the flame image acquisition includes acquiring a suspected flame image of a target area, performing a first preprocessing on the acquired suspected flame image to obtain a first preprocessing result, and then transmitting the first preprocessing result to a controller for analysis and judgment.

[0075] The first preprocessing includes extracting features of a suspected flame image to initially determine whether a flame exists; if so, generating the suspected flame image as an initial flame image; if not, repeating the flame image acquisition step.

[0076] The flame image acquisition can be an image captured by an external intelligent camera and a wide-angle monitoring camera; the image can be a picture image or a video image, and there is no restriction on the static or dynamic form of the image.

[0077] The electronic eyes and monitoring cameras may be monitoring equipment installed in certain fire-prone areas, or may be filming equipment or monitoring cameras mounted on aerial photography drones.

[0078] The target area refers to the area suspected of fire danger monitored by the flame image acquisition device, which is determined by the actual monitoring coverage of the flame image acquisition device. For example, the target area can be a forest, an external structure area of ​​a building, and an internal area of ​​a building such as a public facility area or an area prone to fire, etc.

[0079] The suspected flame image refers to the image features of the collected image that have suspected flame features. Since the occurrence of fire is often accompanied by randomness, it is impossible to monitor it manually in real time, otherwise the investment cost is too high. For example, the area involved in the forest area is very wide, and it is impossible to monitor the fire in real time and effectively manually. The existing intelligent monitoring equipment, such as the aforementioned intelligent camera electronic eye and wide-angle monitoring camera, can solve this problem well. After installation, these intelligent monitoring devices can monitor the fire in the area around the clock without interruption and in real time. Once a suspected target is found, the above information will be immediately transmitted to the control end for decision-making. In this way, the losses and harm caused by the failure to monitor and warn the fire in time can be greatly reduced.

[0080] The extracted features of the suspected flame image include the color, shape and texture features of the suspected target, as well as the area change, edge contour change, flashing frequency and overall movement change of the suspected target. The process of extracting the suspected flame image features to initially determine whether there is a flame specifically includes the following steps:

[0081] S11, comparing the extracted suspected flame features with corresponding flame features stored in a database of the controller;

[0082] S12, calculating the flame characteristic correlation of the comparison results;

[0083] S13, comparing the correlation calculation result with the preset flame judgment threshold; if it exceeds the threshold, it is judged that there is a flame and an initial flame image is generated; if it does not exceed the threshold, it is judged that there is no flame, and the result is sent to the controller, and the controller issues an instruction to continue flame image acquisition.

[0084] The flame image detection method of the present invention performs a first preprocessing on the collected suspected flame image to confirm the presence of flame before generating an initial flame image. The collected flame image is initially identified as to whether the flame exists, which greatly reduces the risk of false alarm of fire and waste of manpower costs.

[0085] S2, inputting the obtained initial flame image into the autonomous learning convolutional neural network model for flame image detection; the autonomous learning convolutional neural network model is an autonomous learning multi-scale convolutional neural network model. Figure 2 In one embodiment of the present application, step S2 specifically includes the following process:

[0086] S21, performing a second preprocessing on the obtained initial flame image to obtain a target flame image;

[0087] The second preprocessing includes normalizing the input initial flame image so that the input flame image is adjusted to a uniform size. The normalization process may adopt a normalization method conventionally used in the art, such as extracting frame sequence features of each of the input multiple images and performing normalization calculation on the frame sequence features of the multiple images. The normalization calculation formula may be, for example: norm =(x now -x min ) / (x max -x min );

[0088] where x norm is the normalized eigenvalue, x now is the current eigenvalue, x max is the maximum value of the feature, x minis the minimum value of the feature.

[0089] The target flame image refers to an image obtained after the initial flame image is subjected to a second preprocessing. The flame image detection method of the present invention normalizes the flame image to the same size by performing a second preprocessing on the initial flame image that has been preliminarily confirmed to have a flame. This helps to extract the features of the flame image later.

[0090] S22, extracting image features from the target flame image; extracting image features can further speed up the flame image detection process, save convolution calculation time, and thus alleviate the high performance requirements for image detection equipment.

[0091] In one embodiment of the present application, the image feature extraction of the target flame image is performed in an autonomous learning manner. The autonomous learning includes autonomously learning the flame image, identifying the flame image feature parameters and storing the identified flame image feature parameters. By autonomously learning the flame image feature parameters, the flame confidence of the flame image is obtained, and the flame recognition result is determined according to the flame confidence and the identified feature parameters.

[0092] Specifically, Figure 3 As shown, the autonomous learning of flame images in step S22 specifically includes the following steps:

[0093] S221, flame calibration, determining the position and outline of the flame and other flame parameter information in a single image.

[0094] The flame image feature parameters learned and stored in the flame calibration autonomous learning process include: static and dynamic feature parameters of the flame, specifically including the color, shape and texture feature parameters of the flame, the area change of the flame, the change of the edge contour of the flame, the jitter frequency, the shape change and the overall movement change of the flame. In addition, other non-flame information in the flame image must be learned, which may affect the recognition of the flame image and lead to misjudgment. For example, smoke information, obstruction information, etc. The smoke information includes the color, shape and texture of the smoke, as well as the contour and movement change of the smoke. The obstruction information includes the obstruction visibility information, the obstruction coverage range and the material information of the obstruction. All feature information related to flame image recognition will be learned and stored.

[0095] S222, flame correction, correcting and learning the flame information calibrated in step S221 and storing relevant correction result parameters.

[0096] Specifically, see Figure 4 , the flame correction specifically includes the following steps:

[0097] S2221, finding out the flame information calibrated in step S221, including the flame information and non-flame information recorded above.

[0098] S2222, performing image enhancement on the flame information found, and performing image enhancement processing on the above information features.

[0099] S2223, binarization processing is performed on the enhanced image feature information to remove interference.

[0100] S2224, analyzing the image feature information after the interference is removed to determine flame image parameters such as the flame position and contour.

[0101] S2225, storing the flame image parameters obtained in the above step S2224, and completing the flame correction.

[0102] S223, flame feature segmentation, segmenting the flame features one by one to obtain different flame feature characters.

[0103] S224, flame feature classification and recognition, comparing the segmented flame feature characters with the flame feature characters stored in the model to perform flame feature classification and recognition, further accurately obtain and store flame image features.

[0104] The flame feature classification and recognition is specifically performed as follows: the different flame feature characters obtained in step S223 are compared with the flame feature characters stored in the model, and the confidence of the flame feature classification and recognition is determined according to the similarity of the comparison; and according to the confidence result, it is determined whether to use the flame image feature parameters in steps S221-S223 as the recognition parameters in step S224.

[0105] S23, performing convolution calculation on the extracted image features to obtain flame image detection results; the convolution calculation can be completed using a multi-scale convolutional neural network model.

[0106] It can be understood that the multi-scale convolutional neural network model described in the present invention can adopt an improved yolov3 model, which uses the LeakyReLU activation function. Specifically, a multi-scale grouped convolution structure is added to the backbone network of the yolov3 model. Since the multi-scale grouped convolution structure has parallel convolution kernels of different scales, the grouped multi-scale convolutional neural network model can perform multi-scale convolution calculations on the initial flame feature image without missing the key information of the small target flame, and pass the target flame feature image to the feature extraction stage.

[0107] In other embodiments of the present invention, the multi-scale convolutional neural network model can also use the Fire-YOLO model. The Fire-YOLO model uses effecentnet to extract features from the input image, which promotes the feature learning of the model, improves the network performance, and optimizes the detection process of the YOLO-V3 model for extremely small targets; unlike the residual block used in the feature extraction network Darknet53 in YOLO-V3, the feature extraction network of Fire-YOLO uses a moving inverted bottleneck convolution (MBConv), which consists of deep convolution and squeeze-activation network (SENet). The structure of the MBConv block is, most importantly, that a 1×1 convolution kernel is used to increase the dimension of the image, followed by cutting the convolution and SENet in turn, and finally using a 1×1 convolution kernel to reduce the dimension of the image output. After the input image passes through the feature extraction network and is upsampled, the corresponding feature map is obtained, that is, the vector feature matrix corresponding to the image. In order to obtain high-level feature information of small targets, the feature map will be processed by a convolution set, which consists of 5 convolution layers, and the convolution kernels are alternately 1×1 and 3×3. Then the high-dimensional vectors are flattened into one-dimensional vectors using a pooling layer and processed by an activation function. These vectors are input into the activation function to obtain the corresponding classification results, and the most likely results are selected as the output of the feature extraction network. In view of the performance defects of the Darknet-53 feature extraction network in terms of the balance of the three dimensions of depth, width, and resolution, the improved Effecentnet feature extraction network is used on the fire dataset to make up for the performance deficiencies of Darknet-53 in small target detection and improve the feature extraction ability of small target detection. For the input small target image, the learning ability of small target features is enhanced, and the performance of the small target feature extraction network is improved. Fire-YOLO has better detection efficiency for fire-like and smoke-like targets. Except for Fire-YOLO, other models tend to misjudge light and clouds in the image as fire and smoke. Obviously, Fire-YOLO is more sensitive to image texture features, which is due to the combination of 1×1 convolution kernels and SE modules in the feature extraction network. Finally, the robustness of the model in detecting confusing targets is improved.

[0108] Furthermore, if Figure 5 As shown, the step S23 specifically includes the following sub-steps:

[0109] S231. Perform a convolution operation on the initial flame feature image through a multi-scale convolutional neural network model to obtain a convolution feature image; wherein the convolution operation is performed according to the convolution operation rules of the aforementioned multi-scale convolutional neural network model.

[0110] S232, iteratively process the convolution feature image to obtain feature images under different process conditions; wherein, according to the current process feature and the extraction order of the flame image, each convolution feature image is iterated once to generate a feature image of a process. Thus, feature images under different process conditions are generated according to the number of iterations.

[0111] S233, bidirectionally fuse the feature images under different process conditions from bottom to top and from top to bottom to obtain a fused target feature image; the bidirectional fusion method can enrich high-level semantic information and is more helpful in detecting small flame features.

[0112] S234, detect the fusion target feature image to obtain the flame image detection result. Through the above-mentioned autonomous learning process combined with the multi-scale convolutional neural network model, the fusion target image detection is performed to obtain the flame image detection result. Due to the above-mentioned continuous self-learning, the training and adjustment of the multi-scale convolutional neural network model are continuously carried out, and the multi-scale convolutional neural network model based on autonomous learning can obtain the optimal result of flame image detection.

[0113] S3. Analyze the flame image detection results and obtain an analysis result set. The analysis of the flame image detection results includes analyzing: flame image features, flame hazard levels, and fire rescue decisions, and correspondingly grouping the above three to form an analysis result set. The analysis result set includes flame image features corresponding to different flame hazard levels and corresponding fire rescue decisions.

[0114] In some embodiments of the present application, the flame hazard level is set to high, medium, low and no risk level in sequence according to the danger; the high level indicates that the fire is urgent, the harm caused is extremely great, and the potential risk is very high; the medium level indicates that the fire hazard is still within the controllable range, but the potential risk is also high, and if measures are not taken in time, it may turn into a high-level hazard; the low level indicates that the fire is within the scope of discretionary observation, the harm caused may be small, the potential risk is low, and it can be controlled through continuous observation; no risk means that the flame is harmless and does not fall within the scope of the disaster.

[0115] In some embodiments of the present application, the flame image features include the flame image features listed above, such as color, shape, texture, contour and movement, etc., which will not be repeated here.

[0116] In some embodiments of the present application, the fire rescue decision includes dialing the fire alarm number, sounding the fire detection alarm, taking emergency measures such as spraying water, and intelligently connecting to the telephone of the relevant management department.

[0117] The present invention collects flame image features, flame hazard levels and fire rescue decisions in a one-to-one correspondence by analyzing the result set, which helps to obtain flame image detection results more quickly and can alleviate the operating pressure of corresponding detection equipment.

[0118] S4, feeding the analysis result set into the self-learning convolutional neural network model, and modifying, adjusting and retraining the self-learning convolutional neural network model. Figure 6 The step of feeding back the analysis result set to the autonomous learning convolutional neural network model specifically includes the following steps:

[0119] S41, inputting the analysis result set into the training set, experience set and test set of the autonomous learning convolutional neural network model;

[0120] S42, extracting the data in the analysis result set according to the data formats of the training set, experience set and test set of the autonomous learning convolutional neural network model, to obtain the result training set, result experience set and result test set corresponding to the training set, experience set and test set respectively;

[0121] S43, performing data comparison on the result training set, result experience set and result test set and the training set, experience set and test set of the autonomous learning convolutional neural network model, and outputting the comparison result;

[0122] S44, optimizing and adjusting the training set, experience set and test set of the autonomous learning convolutional neural network model according to the comparison results to obtain an optimized training set, an optimized experience set and an optimized test set;

[0123] S45, using the optimized training set to train the autonomous learning convolutional neural network model;

[0124] S46, using the optimized verification set to verify the autonomous learning convolutional neural network model trained in step S45, and adjusting parameters according to the verification results;

[0125] S47. Use the optimized test set to evaluate the verified autonomous learning convolutional neural network model, and the evaluation indicators include: model accuracy, model recall rate, model F1 score and other indicators.

[0126] The flame image detection method of the present invention further feeds back the flame image detection result to the self-learning convolutional neural network model to correct, adjust and retrain the model. The feedback forms a closed loop between the entire detection method and the self-learning convolutional neural network model training. Compared with the prior art that only uses the convolutional neural network model for open-loop detection, the continuous self-learning of the model itself plus the result feedback to correct the self-learning again not only facilitates the continuous optimization and correction of the self-learning convolutional neural network model, but also makes the image detection result more accurate and closer to the actual scene.

[0127] Embodiment 2

[0128] Another embodiment of the present application provides an application of a multi-scale flame image detection method based on autonomous learning in building structure fire detection. In this embodiment, except for step S1, the remaining steps are the same as those in the first embodiment and are not repeated here. In this embodiment, the present application uses BIM technology combined with an unsupervised learning algorithm of a context editor to identify and restore the real flame image of the flame image obtained in the building structure fire detection, thereby not only accurately detecting the flame image but also making efficient fire handling decisions based on the detection results.

[0129] The building structures include civil residential buildings, commercial buildings such as shopping malls, office buildings, etc., and industrial buildings such as factory buildings, etc. For fire detection in building structures, since the building structure is a three-dimensional structure with a complex internal structure, and the outside of the building structure is wrapped by walls, etc. Therefore, once a fire occurs inside the building structure, the fire spreads rapidly, which can easily damage the electronic circuits inside the building structure, resulting in the monitoring equipment inside the building structure being unable to obtain real-time fire images. Photos or videos taken by peripheral monitoring equipment can only be obtained through unobstructed structures such as doors and windows of the building structure, and the fire situation cannot be obtained for the part blocked by the building wall.

[0130] The core of BIM technology is to establish a virtual three-dimensional model of the construction project and use digital technology to provide this model with a complete construction project information database that is consistent with the actual situation. This information database not only fully contains the geometric information describing the building components and decoration materials, but also includes the combustion performance, calorific value, smoke production rate and other detailed parameters of these materials. Through this model, it can assist material inspection agencies in analyzing the fire hazards of various areas of the building, determine the dangerous areas and the scope of building components and decoration materials that need to be tested for combustion performance. At the same time, the BIM database can also automatically export and generate a component / material list, which should include the combustion properties, usage location and usage of the materials. This list can also be used as the basis for material inspection, which not only improves work efficiency, but also helps inspection / approval agencies to strengthen their understanding of the project and reduce fire risks.

[0131] Based on this, this application proposes to use BIM technology combined with the unsupervised learning algorithm of the context editor to identify and restore the real flame image obtained in the fire detection of building structures, thereby not only accurately performing flame image detection but also making efficient fire handling decisions based on the detection results.

[0132] Specifically, step S1 is:

[0133] S1, flame image acquisition, obtaining an initial flame image; the flame image acquisition includes acquiring a suspected flame image of a target area, performing a first preprocessing on the acquired suspected flame image to obtain a first preprocessing result, and then transmitting the first preprocessing result to a controller for analysis and judgment.

[0134] The flame image acquisition can be an image captured by an external intelligent camera and a wide-angle monitoring camera; the image can be a picture image or a video image, and there is no restriction on the static or dynamic form of the image.

[0135] The electronic eyes and monitoring cameras can be monitoring devices installed in the peripheral area of ​​the building structure, or they can be shooting devices or monitoring cameras mounted on aerial photography drones.

[0136] The target area refers to the area suspected of fire danger monitored by the flame image acquisition device, which is determined by the actual monitoring coverage of the flame image acquisition device. For example, the target area is the external structure area of ​​the building, and the internal structure of the building such as the public facilities area or the fire prone area.

[0137] The suspected flame image refers to the image features of the collected image that have suspected flame features. Since the occurrence of fire is often accompanied by randomness, it is impossible to monitor it manually in real time, otherwise the investment cost is too high. Moreover, once a fire occurs, the fire spreads rapidly, which can easily damage the electronic circuits inside the building structure, resulting in the monitoring equipment inside the building structure being unable to obtain real-time fire images. Intelligent monitoring equipment installed on the periphery of the building structure, such as the aforementioned intelligent camera electronic eye and wide-angle monitoring camera, can solve this problem well. After installation, these intelligent monitoring devices can monitor the fire in the area around the clock and in real time. Once a suspected target is found, the above information will be immediately transmitted to the control end for decision-making. In this way, the losses and harm caused by the failure to monitor and warn the fire in time can be greatly reduced.

[0138] The first preprocessing includes restoring the suspected flame image and judging whether there is a flame according to the restored flame image; if there is, generating the restored flame image as the initial flame image; if not, repeating the flame image acquisition step.

[0139] The restoration process of the suspected flame image specifically includes the following steps:

[0140] S101, extracting building information from the suspected flame image;

[0141] S102, calling the BIM model according to the extracted building information;

[0142] S103, obtaining information about obstructions in the suspected flame image and internal structure information of the building where the suspected flame image is located according to the called BIM model;

[0143] S104: removing obstructions of the suspected flame image according to the obtained obstruction information and the internal structure information of the building, restoring the flame image, and obtaining an initial flame image.

[0144] The building information in step S101 includes the location of the building structure, the wall features of the building structure, and the surrounding environment features of the building structure. The location of the building structure includes residential areas, shopping malls, office buildings, or factories; the wall features of the building structure include the exterior walls of the building structure, the wall frames of the building structure, and the number of doors and windows on the walls of the building structure; the surrounding environment features of the building structure include other building structures around the building structure, the distance between building structures, the green belt structure, the number and height of trees, and other objects placed around the building structure. Step S101 extracts the building information in the suspected flame image in order to determine the type of building where the suspected flame image occurs, and judges whether the suspected flame image occurs in a residential building, a shopping mall, an office building, a factory, etc. by the building type.

[0145] In step S102, the building type where the suspected flame image occurs is determined by the above-mentioned building information in the suspected flame image extracted in step S101, and then the matching building BIM model and its information are retrieved from the BIM model database according to the building type. The BIM model database can be an existing BIM model that has been used as open source and can be obtained from the Internet or the official website of the corresponding department, or it can be a database formed by the BIM model obtained by retraining the above-mentioned existing BIM model combined with the autonomous learning mechanism of the present application. By retrieving the matching building BIM model and its information, it is possible to determine the internal and external structural information of the building structure where the fire occurs, locate the fire source point information of the suspected flame image, obtain the smoke information, fire load information and ventilation structure information of the building structure, etc., to help make fire emergency decisions faster and more accurately.

[0146] In step S103, the obstruction information in the suspected flame image and the internal structure information of the building where the suspected flame image is located are obtained by calling the BIM model; the fire source point where the suspected flame image occurs is located, and the smoke information, fire load information and ventilation structure information of the building structure are obtained at the same time. By locating the fire source information, the origin and boundary basis can be provided for the subsequent restoration of the flame image; by obtaining the smoke information of the building structure, the smoke concentration and obstruction degree of the flame image can be assisted in judging, and then the flame situation can be obtained in combination with the smoke information; the fire load information of the building structure can assist in the potential risk assessment of the flame, such as the explosion or spread situation estimation; and the ventilation structure information of the building structure can help judge the possible spread path and spread rate of the actual flame image after the obstruction is removed.

[0147] The step S104 removes the obstructions of the suspected flame image according to the obtained obstruction information and the internal structure information of the building, restores the flame image, and obtains the initial flame image. The removal of obstructions of the suspected flame image and restoration of the flame image is achieved by using an unsupervised learning algorithm based on a context editor.

[0148] The unsupervised learning algorithm based on the context editor is an unsupervised visual feature learning algorithm driven by context pixel prediction, which uses the surrounding image information to infer the missing image. The main idea of ​​the context editor is to combine the Encoder-Decoder network structure and GAN (Generative Adversarial Networks). The Encoder-Decoder stage is used to learn image features and generate prediction maps corresponding to the image area to be repaired. The GAN part is used to judge the possibility that the prediction map comes from the training set and the prediction set. When the generated prediction map is consistent with the Ground Truth in image content, and the GAN discriminator cannot judge whether the prediction map comes from the training set or the prediction set, it is considered that the network model parameters have reached the optimal state. The unsupervised learning algorithm based on the context editor places a channel fully connected layer between the encoder and the decoder to obtain better semantic features (also known as the "bottleneck"). Finally, the decoder uses the "bottleneck" feature to reconstruct the missing part of the input image.

[0149] The main advantages of the unsupervised learning algorithm based on the context editor include: first, the context encoder has an outstanding ability to fill in missing areas; second, the context encoder is used as a pre-training step for image classification, object detection and semantic segmentation, demonstrating that its learned features can be transferred to other tasks.

[0150] In combination with the characteristics and advantages of the above-mentioned context encoder, the present application specifically includes the following steps in the process of removing the obstruction of the suspected flame image and restoring the flame image:

[0151] Perform image encoding; encode the suspected flame image containing occluders through convolution to generate an image encoding set.

[0152] The image coding set is decoded; and the generated image coding set is deconvolved so as to restore the size of the image to be consistent with the size of the original suspected flame image.

[0153] Output the restored flame image; the flame image output after the above encoding and decoding is the flame image restored after removing the obstruction.

[0154] The training process of the convolutional neural network adopted by the algorithm is specifically carried out using a loss function, including the following process:

[0155] Establish a reconstruction loss; the reconstruction loss refers to directly calculating the pixel-wise error between the decoded generated image and the original suspected flame image. The reconstruction loss is used to capture the overall structure of the missing area (including the area of ​​the building shielding in this embodiment), so that the repaired area has structural consistency with the surrounding area. The reconstruction loss is calculated by the following reconstruction loss function It is expressed as:

[0156]

[0157] in, represents the reconstruction loss function, x represents the input image, M is the binary mask (1 represents the missing area, 0 represents the input pixel), F is the context encoder, and ⊙ represents the element-by-element product operation.

[0158] Establish an adversarial loss; the adversarial loss is used to select a mode, where the mode refers to a blank area in an image, there may be multiple filling methods that can conform to the overall structure of the image, and can also be well integrated with other surrounding information, called different modes. The adversarial loss is the loss function for establishing GAN, that is, the adversarial loss function It is expressed as:

[0159]

[0160] in, Denotes the adversarial loss function, and D denotes the discriminator, which is used to distinguish between filled images and real images. The output of the discriminator is a binary value 0 or 1, which is 1 if the input is a real image and 0 if the input is a filled image.

[0161] Obtain a loss function; the loss function is obtained by weighting the two losses of formula (1) and formula (2). Specifically expressed as:

[0162]

[0163] Among them, λ rec Represents the reconstruction loss function The weighting coefficient, λ adv Denotes the adversarial loss function The weighting coefficient of .

[0164] In a preferred embodiment of the present invention, after removing the obstruction to obtain the restored flame image, before generating the initial flame image, it also includes: fusing the restored flame image obtained by removing the obstruction with the internal structure information of the building obtained by calling the BIM model. The internal structure information of the building includes the internal structure information of the building at the fire source point or area where the suspected flame image occurs by calling the BIM model, including the internal space pattern distribution and ventilation structure information of the area. The initial flame image is generated by fusing the above-mentioned internal structure information of the building with the flame image obtained by removing the obstruction. The initial flame image includes not only the flame area image after the obstruction is restored, but also the internal building structure image of the flame area is restored.

[0165] The flame image detection method of the present invention performs the above-mentioned first preprocessing on the collected suspected flame image, combines the BIM technology with the unsupervised learning algorithm to restore the flame image, and generates an initial flame image by removing the obstructions and the internal structure information of the building at the fire source point. The flame scene can be restored to the greatest extent, greatly improving the efficiency of fire warning and reducing the risk of misjudgment of fire.

[0166] Embodiment 3

[0167] Another embodiment of the present application provides a multi-scale flame image detection method based on autonomous learning, wherein the first preprocessing in step S1 is specifically:

[0168] The extracted suspected flame image features are converted into texture features to obtain a texture feature image. The conversion of the extracted suspected flame image features into texture features can be performed using a filter.

[0169] The texture feature image is binarized to obtain local texture features.

[0170] It is determined whether there is a flame in the suspected target area according to the local texture feature.

[0171] In this embodiment of the present invention, the features and judgment results extracted by the first preprocessing process are also fed back to the autonomous learning convolutional neural network model, and the autonomous learning convolutional neural network model is corrected, adjusted and retrained. The specific process of correcting, adjusting and retraining the autonomous learning convolutional neural network model is referred to step S4 of the first embodiment, which will not be repeated here.

[0172] In this embodiment, the flame image detection method of the present invention converts the extracted suspected flame image features into texture features through a first preprocessing, and processes the texture features to further determine whether a flame exists, thereby reducing the amount of calculation and further improving the detection accuracy of the flame image, greatly reducing the risk of false alarms of fire and waste of manpower costs.

[0173] At the same time, in this embodiment, the features and judgment results extracted by the first preprocessing process are further fed back to the autonomous learning convolutional neural network model to correct, adjust and retrain the model. This feedback forms a closed loop for the entire detection method, including the preprocessing in the first stage and the autonomous learning convolutional neural network model training therein. Compared with the prior art that only uses the convolutional neural network model for open-loop detection, the continuous autonomous learning of the model itself plus the result feedback to correct the autonomous learning again is not only conducive to the continuous optimization and correction of the autonomous learning convolutional neural network model, but also makes the image detection results more accurate and closer to the actual scene.

[0174] Embodiment 4

[0175] Another embodiment of the present application provides a multi-scale flame image detection method based on autonomous learning. In this embodiment, step S2 further includes the construction of an autonomous learning convolutional neural network model, which specifically includes the following steps:

[0176] Data preparation includes collecting a large amount of image data, including flame image data and non-flame image data; extracting, classifying and annotating the collected image data, and dividing the image data into training sets, experience sets and test sets.

[0177] The collected image data also includes the analysis result data of the image detection in the aforementioned embodiment and the first preprocessing result data. The data source for constructing the autonomous learning convolutional neural network model of the present invention incorporates external data and result data after the model is applied in the data preparation stage. Under the framework of closed-loop data feedback, an autonomous learning mechanism is introduced. The data collection of the model of the present invention is closer to the real scene, further improving the accuracy and reliability of the flame image detection results.

[0178] Model construction: including constructing a layer structure of an autonomous learning convolutional neural network model, wherein the layer structure adopts a general convolutional neural network model architecture, such as including multiple convolutional layers, pooling layers, fully connected layers, etc. Specifically, the construction of convolutional layers, pooling layers, fully connected layers, etc. can adopt existing conventional construction methods, and this solution does not limit this.

[0179] Model training: Use the training set generated in the data preparation stage to train the model, and adjust the parameters of the model through the back propagation algorithm to minimize the prediction error. As described in the above embodiment of the present invention, the analysis result data of the image detection and the first preprocessing result data are further fed back into the model, and the training set data therein are extracted for comparison to generate an optimized training set, and then the model is trained again using the optimized training set. In this way, the present invention introduces an autonomous learning mechanism in the model training stage, optimizes the training effect, and further improves the accuracy and reliability of the flame image detection results.

[0180] Model verification: Use the verification set generated in the data preparation stage to verify the trained model and adjust the parameters of the model to obtain better performance. As described in the above embodiments of the present invention, the analysis result data of the image detection and the first preprocessing result data are further fed back into the model, and the verification set data are extracted for comparison to generate an optimized verification set, and then the model is trained again using the optimized verification set. In this way, the present invention introduces an autonomous learning mechanism in the model verification stage, optimizes and adjusts the verification parameters to improve the verification effect, and further improves the accuracy and reliability of the flame image detection results.

[0181] Model testing: Use the test set generated in the data preparation phase to evaluate the performance of the model, and calculate the model's accuracy, recall rate, F1 score and other indicators. As described in the above embodiments of the present invention, the image detection analysis result data and the first preprocessing result data are further fed back into the model, and the test set data are extracted for comparison to generate an optimized test set, and then the optimized test set is used to train the model again. In this way, the present invention introduces an autonomous learning mechanism in the model testing phase, optimizes and adjusts the test parameters to improve the test effect, and further improves the accuracy and reliability of the flame image detection results.

[0182] Model application: The trained model is applied to actual flame image detection, which can be integrated into the monitoring system, or used to analyze images taken by satellites or drones, etc. Moreover, the present invention introduces an autonomous learning mechanism, which can be fed back at each stage of the model for autonomous model learning, correct the parameters of the entire model, and thus improve the accuracy of image detection.

[0183] It is understandable that the autonomous learning convolutional neural network model constructed by the present invention can be used not only in flame image detection, but also in other image detection fields. Of course, the autonomous learning convolutional neural network model constructed by the present invention is not limited to the field of image detection, but also applicable to other fields that require the use of neural network models. The present invention does not limit the application of the model.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, the indirect coupling or communication connection of the modules can be electrical, mechanical or other forms.

[0185] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0187] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A multi-scale flame image detection method based on autonomous learning, Features: The steps include: Flame image acquisition, obtaining an initial flame image; the flame image acquisition includes acquiring a suspected flame image of a target area, performing a first preprocessing on the acquired suspected flame image to obtain a first preprocessing result, and then transmitting the first preprocessing result to a controller for analysis and judgment; The obtained initial flame image is input into the self-learning convolutional neural network model for flame image detection; Analyze the flame image detection results and obtain an analysis result set; Feedback the analysis result set into the self-learning convolutional neural network model, and adjust and train the self-learning convolutional neural network model; Wherein, the convolutional neural network model is a multi-scale convolutional neural network model; The initial flame image is input into the autonomous learning convolutional neural network model to perform flame image detection, which specifically includes the following process: Performing a second preprocessing on the obtained initial flame image to obtain a target flame image; Image features of target flame images are extracted through autonomous learning of flame images; Perform convolution calculation on the extracted image features to obtain flame image detection results; The autonomous learning of flame images specifically comprises the following steps: Flame calibration, determining the position and outline of the flame and other flame parameter information in a single image; Flame correction: correct the flame information after calibration and store the relevant correction result parameters; Flame feature segmentation: segment the flame features one by one to obtain different flame feature characters; Flame feature classification and recognition: the segmented flame feature characters are compared with the flame feature characters stored in the model to perform flame feature classification and recognition, and the flame image features are obtained and stored.

2. According to the multi-scale flame image detection method based on autonomous learning in claim 1, Features: The first preprocessing includes extracting the suspected flame image features to initially determine whether there is a flame, specifically including the following steps: Compare the extracted suspected flame features with the corresponding flame features stored in the database of the controller, and output the comparison results; Performing flame characteristic correlation calculation on the comparison results; Compare the correlation calculation result with the preset flame judgment threshold; If it exceeds the threshold, it is determined that there is a flame and an initial flame image is generated; If the threshold is not exceeded, it is determined that there is no flame, and the result is sent to the controller, which issues an instruction to continue flame image acquisition.

3. According to the multi-scale flame image detection method based on autonomous learning in claim 1, Features: The first preprocessing includes restoring the suspected flame image and judging whether there is a flame according to the restored flame image, which specifically includes the following steps: Extracting architectural information from suspected fire images; Call the BIM model based on the extracted building information; Obstruction information in the suspected flame image and internal structure information of the building where the suspected flame image is located are obtained according to the called BIM model; According to the obtained obstruction information and the internal structure information of the building, the obstruction of the suspected flame image is removed, the flame image is restored, and the initial flame image is obtained; Wherein, obtaining the initial flame image further comprises: fusing the restored flame image obtained by removing the obstructions with the internal structure information of the building obtained by calling the BIM model.

4. The multi-scale flame image detection method based on autonomous learning according to claim 1, Features: The step of performing convolution calculation on the extracted image features to obtain the flame image detection result specifically includes the following sub-steps: Perform convolution operation on the initial flame feature image through a multi-scale convolutional neural network model to obtain a convolution feature image; Iteratively process the convolution feature image to obtain feature images under different process conditions; The feature images under different process conditions are bidirectionally fused from bottom to top and from top to bottom to obtain the fused target feature image; The fused target feature image is detected to obtain the flame image detection result.

5. The multi-scale flame image detection method based on autonomous learning according to claim 4, Features: The flame correction specifically comprises the following steps: Find out the calibrated flame information, including the recorded flame information and non-flame information; Perform image enhancement on the found flame information; Binarize the enhanced image feature information; Analyze the image feature information after removing interference to determine the flame image parameters of the flame position and contour; The flame image parameters obtained after analysis are stored to complete flame correction.

6. A multi-scale flame image detection method based on autonomous learning according to any one of claims 1 to 3, Features: The step of feeding back the analysis result set to the autonomous learning convolutional neural network model specifically includes the following steps: Inputting the analysis result set into the training set, experience set and test set of the autonomous learning convolutional neural network model; Extracting data from the analysis result set according to the data formats of the training set, experience set, and test set of the autonomous learning convolutional neural network model to obtain a result training set, a result experience set, and a result test set corresponding to the training set, the experience set, and the test set, respectively; Compare the result training set, result experience set and result test set with the training set, experience set and test set of the autonomous learning convolutional neural network model, and output the comparison result; According to the comparison results, the training set, experience set and test set of the autonomous learning convolutional neural network model are optimized and adjusted respectively to obtain an optimized training set, an optimized experience set and an optimized test set; Using the optimized training set to train the autonomous learning convolutional neural network model; The trained autonomous learning convolutional neural network model is verified using the optimized experience set, and parameters are adjusted according to the verification results; The optimized test set is used to evaluate the verified self-learning convolutional neural network model.

7. The multi-scale flame image detection method based on autonomous learning according to claim 6, Features: The evaluation indicators include the accuracy of the model, the recall rate of the model and the F1 score of the model.

8. The multi-scale flame image detection method based on autonomous learning according to claim 3, Features: The building information includes the location of the building structure, the wall characteristics of the building structure, and the surrounding environment characteristics of the building structure; The method of removing obstructions of the suspected flame image and restoring the flame image is implemented by adopting an unsupervised learning algorithm based on a context editor.

Images