Flame area identification method based on improved Unet
By improving the Unet network, combining ResNet and Transformer, and using Laplace image enhancement algorithm in image preprocessing, the problem of low flame recognition accuracy in the prior art is solved, and higher recognition accuracy is achieved.
Patent Information
- Application Number
- CN202411950024.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-05-02
AI Technical Summary
In the prior art, the Unet network is used to identify flame areas. However, the feature extraction capability of the Unet backbone network is limited and the convergence is slow during training, resulting in a decrease in recognition accuracy.
The improved Unet network is adopted, combined with ResNet as the encoder and Transformer as the decoder, and the Laplace image enhancement algorithm is used in the image preprocessing stage to enhance the region boundaries and improve the recognition accuracy.
Through the improved Unet network and Laplace image enhancement algorithm, the accuracy of flame image recognition is improved, and the shortcomings of traditional Unet networks in feature extraction and training convergence are solved.
Smart Images
Figure CN119919877A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of flame image recognition, and in particular to a flame area recognition method based on improved Unet. Background Art
[0002] In the waste-to-energy process, the combustion process needs to be monitored in real time, and safety warnings or combustion adjustments can be made through monitoring. Generally, it can be identified and analyzed based on the flame. Flame image recognition can monitor the combustion conditions in the waste incinerator in real time, and it can also provide early warning of potential problems by monitoring the changes in the flame.
[0003] Moreover, combustion can be controlled based on the flame area, and by controlling combustion, environmental pollution can be reduced, the combustion process can be controlled, etc. Therefore, the identification of the flame area is very important.
[0004] In the prior art, the Unet network is generally used to identify the flame area. However, the traditional Unet backbone network has limited feature extraction capabilities and slow convergence during training. At the same time, the boundaries of different temperature areas in the real-time flame images collected are not obvious, resulting in reduced subsequent recognition accuracy. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a flame area recognition method based on improved Unet, which automatically recognizes the flame in the image through the improved Unet network recognition, and further adopts the Laplace image enhancement algorithm to increase the area boundary and improve the accuracy of image recognition.
[0006] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a flame area recognition method based on improved Unet, in which the flame image is collected and sent to the improved segmentation network model based on Unet after preprocessing, and the segmentation network model performs image segmentation and output; wherein the preprocessing includes enhancing the image using the Laplace image enhancement processing algorithm.
[0007] The Laplace image enhancement processing algorithm is used to enhance the image, including: processing the sudden change of grayscale in the image, when the grayscale of the central pixel in the neighborhood is lower than the average grayscale of other pixels in the neighborhood where the central pixel is located, the grayscale of the central pixel is reduced; when it is higher, the grayscale of the central pixel is increased, thereby realizing image sharpening processing to obtain a Laplace image, and superimposing the original image and the Laplace image to obtain a preprocessed image.
[0008] Image preprocessing also includes denoising, where the collected image is first denoised and then enhanced using the Laplace image enhancement algorithm.
[0009] The segmentation network model is improved based on the Unet network: the Unet network includes an encoder and a decoder, with the ResNet network as the encoder and the Transformer network as the decoder; the input image passes through the ResNet network encoder and the Transformer network decoder for image segmentation output.
[0010] The segmentation network model is trained and verified through pre-built training sets and validation sets; the loss function is calculated during the training process, and the model parameters are updated through back propagation to complete the model training process.
[0011] During the training process, the loss function uses a multi-classification cross entropy loss function to evaluate the training results to guide the training optimization direction of the model.
[0012] The collection of flame images in the training stage includes: collecting flame video streams on site and extracting flame pictures at the set frame rate. According to the temperature range required for actual waste-to-energy, the flame images are divided into four categories according to RGB image features: high-temperature flame, medium-temperature flame, low-temperature flame, and no flame. At the same time, the background is identified, and a total of five categories are segmented. Each category is labeled independently and annotated using the segmentation and annotation tool. The annotated images are used as training sets to train the model.
[0013] During training, hyperparameter tuning includes learning rate, batch size, number of iterations, etc. Grid search, random search, or Bayesian optimization tuning methods are used to find the best hyperparameter combination.
[0014] During the training process, monitoring indicators are set, including IoU coefficient and Dice coefficient. The segmentation accuracy of the model is evaluated based on the monitoring indicators to adjust the training parameters of the model.
[0015] The advantages of the present invention are that the flame in the image is automatically identified through improved Unet network recognition, and the Laplace image enhancement algorithm is further adopted to increase the region boundary, thereby improving the accuracy of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The following is a brief description of the contents expressed in the drawings of the present invention and the symbols in the drawings:
[0017] Figure 1 It is a deployment flow chart of the method of the present invention;
[0018] Figure 2 This is a schematic diagram of the improved Unet network model of the present invention;
[0019] Figure 3 This is a diagram of the encoder structure of Resnet of the present invention;
[0020] Figure 4 It is the decoder of the Transformer of the present invention. DETAILED DESCRIPTION
[0021] The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.
[0022] This embodiment provides a flame area recognition method based on improved Unet, and its core improvements include at least:
[0023] 1. Network model modification: Based on the Unet network structure, the ResNet network is introduced as the encoder, and the transformer network structure is used as the decoder to achieve accurate segmentation of flame images.
[0024] 2. Preprocess the image input into the network model, including denoising first and then Laplace enhancement, so as to improve the boundary of different temperature areas of the flame image and make the boundary more obvious.
[0025] like Figure 1-4 As shown, in this embodiment, the collected flame image is sent to the improved segmentation network model based on Unet after preprocessing, and the segmentation network model performs image segmentation and output; wherein the preprocessing includes enhancing the image using the Laplace image enhancement processing algorithm.
[0026] The Laplace image enhancement algorithm is used to enhance the image, including: processing the sudden change of grayscale in the image, when the grayscale of the central pixel in the neighborhood is lower than the average grayscale of other pixels in the neighborhood where the central pixel is located, the grayscale of the central pixel is reduced; when it is higher, the grayscale of the central pixel is increased, so as to achieve image sharpening processing to obtain the Laplace image, and the original image and the Laplace image are superimposed together to obtain the preprocessed image. The image preprocessing also includes denoising processing, and the collected image is first denoised and then enhanced by the Laplace image enhancement processing algorithm.
[0027] The segmentation network model is improved based on the Unet network: the Unet network includes an encoder and a decoder, with the ResNet network as the encoder and the Transformer network as the decoder; the input image passes through the ResNet network encoder and the Transformer network decoder for image segmentation output.
[0028] The segmentation network model is trained and verified through the pre-built training set and validation set. The loss function is calculated during the training process, and the model parameters are updated through back propagation to complete the model training process. During the training process, the loss function uses a multi-classification cross entropy loss function to evaluate the training results to guide the training optimization direction of the model.
[0029] In this embodiment, the Laplace image enhancement algorithm is proposed to solve the problem that the boundaries of different temperature regions in the actual flame image are not obvious. In order to solve the problem that the traditional Unet backbone network has limited feature extraction capabilities and slow convergence during training, a segmentation network that combines Transformer and Unet is proposed, such as Figure 1 shown.
[0030] 1.1 Flame Image Dataset Creation
[0031] First, the flame video stream in the waste-to-energy furnace is collected on site, and the flame image data is extracted from it. The abnormal images are initially eliminated manually or by program, and the flames in different temperature ranges are represented by mask images, and manually segmented and labeled to form an annotated image data set. The video stream is collected on site, and the flame image is extracted at a reasonable frame rate. According to the temperature range required for actual waste-to-energy, the flame image is divided into four categories according to the RGB image features: high-temperature flame, medium-temperature flame, low-temperature flame, and no flame. The background is also identified, and a total of five categories are segmented. Each category is independently labeled and annotated using the segmentation and annotation tool.
[0032] 1.2 Flame image preprocessing
[0033] The flame image is first denoised using a program, and then sharpened using the Laplace operator. The annotated image is preprocessed using the Laplace enhancement algorithm. The Laplace operator is a differential operator that mainly processes the sudden change of grayscale in the image. When the grayscale of the central pixel in the neighborhood is lower than the average grayscale of other pixels in its neighborhood, the grayscale of this central pixel should be further reduced; when it is higher, the grayscale of the central pixel should be further increased to achieve image sharpening. Laplace sharpening can effectively enhance the edges and details of the image, but it will also amplify the noise in the image. Therefore, before sharpening the image, denoising is usually required.
[0034] Finally, superimposing the original image and the Laplace image together can restore the background characteristics and maintain the effect of Laplace sharpening.
[0035] The Laplace sharpening formula is as follows:
[0036]
[0037] in,
[0038] f(x, y) represents the original image;
[0039] g(x, y) represents the sharpened image;
[0040] represents the Laplace operator, The transformation is as follows:
[0041]
[0042] In order to make the image a discrete digital signal,
[0043] The above formula is simplified to:
[0044]
[0045] Get 4 neighborhood convolution kernels, the template is expressed as:
[0046]
[0047] or:
[0048]
[0049] The 8-neighborhood convolution kernel is obtained, and the template is expressed as:
[0050]
[0051] Then use the template to perform convolution calculation. The main steps are:
[0052] (1) Move the template around in the image and make the center of the template coincide with a certain pixel position in the image;
[0053] (2) Multiply the coefficient on the template by the corresponding pixel under the template;
[0054] (3) Add all the products;
[0055] (4) Assign the sum (output response of the template) to the pixel in the image corresponding to the center position of the template.
[0056] 1.3 Model construction and training
[0057] Unet is a classic image segmentation network, which consists of an encoder and a decoder, and can effectively capture features of different scales. Transformer is a neural network model based on the self-attention mechanism, which has strong modeling capabilities. In order to improve the segmentation effect of traditional Unet flame images, the two are fused to obtain an improved Unet model.
[0058] The encoder part of the improved model adopts the Unet structure, including multi-layer convolution and pooling operations, to extract high-level semantic information of the image. At the same time, in order to improve the efficiency of feature extraction, the Unet encoding part is introduced into the ResNet network.
[0059] The decoder part introduces the Transformer structure, which uses the self-attention mechanism to integrate and adjust the features extracted by the encoder to obtain more accurate segmentation results. The specific network structure is as follows: Figure 2 shown.
[0060] The encoder structure of Resnet is as follows Figure 3 As shown, the decoder structure of Transformer is as follows Figure 4 shown.
[0061] During model training, the training process of the improved Unet generally includes steps such as data loading, model construction, loss calculation, back propagation and parameter update. During training, optimizers such as stochastic gradient descent (SGD) or Adam, as well as strategies such as learning rate decay, model preservation and validation set verification are usually used. This is not the focus of the present invention.
[0062] The loss function in the training process designs the loss function and adopts the multi-classification cross entropy loss function as follows:
[0063]
[0064] This loss function detects each pixel separately, and m is the number of categories;
[0065] The predicted pixel vector
[0066]
[0067] with one hot encoded target vector
[0068]
[0069] Make a comparison.
[0070] During training, hyperparameters can be tuned as needed, including learning rate, batch size, number of iterations, etc. Grid search, random search, or Bayesian optimization can usually be used to find the best hyperparameter combination. Pre-trained models, transfer learning, or model fusion strategies can also be used to improve the generalization ability of the model on specific tasks.
[0071] In order to better monitor the training of the model, some effective monitoring indicators, such as IoU (Intersect ion over Union ion) and D ice coefficient, can be used to evaluate the segmentation accuracy of the model. In addition, visualization tools, such as TensorBoard, can be used to observe the performance of the model on the training set and validation set in real time, and adjust the model structure and parameters in time to accelerate the optimization process.
[0072] 1.4 Model testing and deployment
[0073] After completing the model training, it is necessary to conduct sufficient model evaluation and verification. Use different evaluation methods such as cross-validation, holdout method, and bootstrap method to fully verify the generalization ability and stability of the model. It is also necessary to fully understand the performance of the model in different categories and scenarios, and conduct performance analysis and comparison. Evaluation indicators such as IoU (Intersection over Union) and Dice coefficient are used to evaluate the segmentation accuracy of the model.
[0074] During the model deployment phase, we need to consider issues such as model storage and loading, inference speed, and memory consumption. We need to select appropriate model compression and acceleration methods, such as pruning, quantization, and model distillation, to improve inference efficiency while ensuring accuracy. We also need to fully consider the deployment requirements of the model on different evaluation platforms in order to achieve model migration and application.
[0075] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.
Claims
1. A flame area recognition method based on improved Unet, characterized by: The collected flame image is sent to the segmentation network model improved based on Unet after preprocessing, and the segmentation network model performs image segmentation and output; The preprocessing includes enhancing the image using the Laplace image enhancement algorithm.
2. The flame area recognition method based on improved Unet according to claim 1, characterized in that: The Laplace image enhancement processing algorithm is used to enhance the image, including: processing the sudden change of grayscale in the image, when the grayscale of the central pixel in the neighborhood is lower than the average grayscale of other pixels in the neighborhood where the central pixel is located, the grayscale of the central pixel is reduced; when it is higher, the grayscale of the central pixel is increased, thereby realizing image sharpening processing to obtain a Laplace image, and superimposing the original image and the Laplace image to obtain a preprocessed image.
3. A flame area recognition method based on improved Unet as claimed in claim 1 or 2, characterized in that: Image preprocessing also includes denoising, where the collected image is first denoised and then enhanced using the Laplace image enhancement algorithm.
4. A flame area recognition method based on improved Unet as claimed in claim 1 or 2, characterized in that: The segmentation network model is improved based on the Unet network: the Unet network includes an encoder and a decoder, with the ResNet network as the encoder and the Transformer network as the decoder; The input image is then passed through the ResNet network encoder and the Transformer network decoder for image segmentation output.
5. The flame area recognition method based on improved Unet as claimed in claim 4, characterized in that: The segmentation network model is trained and verified through pre-built training sets and validation sets. The loss function is calculated during the training process, and the model parameters are updated through back propagation to complete the model training process.
6. The flame area recognition method based on improved Unet as claimed in claim 5, characterized in that: During the training process, the loss function uses a multi-classification cross entropy loss function to evaluate the training results to guide the training optimization direction of the model.
7. The flame area recognition method based on improved Unet as claimed in claim 5, characterized in that: The collection of flame images in the training stage includes: collecting flame video streams on site and extracting flame pictures at the set frame rate. According to the temperature range required for actual waste-to-energy, the flame images are divided into four categories according to RGB image features: high-temperature flame, medium-temperature flame, low-temperature flame, and no flame. At the same time, the background is identified, and a total of five categories are segmented. Each category is labeled independently and annotated using the segmentation and annotation tool. The annotated images are used as training sets to train the model.
8. The flame area recognition method based on improved Unet as claimed in claim 5, characterized in that: During training, hyperparameter tuning includes learning rate, batch size, number of iterations, etc. Grid search, random search, or Bayesian optimization tuning methods are used to find the best hyperparameter combination.
9. The flame area recognition method based on improved Unet as claimed in claim 5, characterized in that: During the training process, monitoring indicators are set, including IoU coefficient and Dice coefficient. The segmentation accuracy of the model is evaluated based on the monitoring indicators to adjust the training parameters of the model.