Smoke and flame recognition model training method, smoke and flame recognition method and smoke and flame recognition device

Abstract

The invention relates to a training method of a smoke and flame recognition model, and a smoke and flame recognition method and device. The method comprises the following steps: acquiring a refinery smoke flame data set; acquiring an initial smoke flame recognition model; performing convolution and feature processing on each picture in the smoke flame data set based on a ConvBNSiLU module and a C3 module of the Backbone unit, and performing feature extraction by using a dual attention mechanism module to obtain first hidden features of different scales; performing secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales; identifying the second hidden features of different scales through the Head unit, and generating a predicted value; and calculating a loss function value according to the predicted value, updating parameters of the smoke flame identification model until a preset condition is met, and obtaining the smoke flame identification model.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a smoke and flame recognition model training method, a smoke and flame recognition method and a smoke and flame recognition device. Background Art

[0002] Oil refining is a complex and dangerous industrial production process, in which smoke and flames are common safety hazards. Therefore, the detection and identification of smoke and flames is crucial to the safe production of the oil refining industry.

[0003] Smoke and fire detection algorithms are usually based on image processing technology. In the past, traditional image processing methods usually relied on hand-designed feature extractors and classifiers, which often performed poorly in complex scenes. Deep learning-based methods automatically learn feature representations through deep neural networks, making smoke and fire detection more accurate and robust. In recent years, the rise of deep learning technology has brought new solutions to smoke and fire detection. Among them, object detection models such as YOLO and Faster R-CNN can effectively detect the location of smoke and fire in images, making significant progress in this field. Summary of the invention

[0004] In order to obtain more accurate smoke and flame recognition results, the embodiments of the present invention provide a smoke and flame recognition model training method, a smoke and flame recognition method and a smoke and flame recognition device.

[0005] In a first aspect, an embodiment of the present application provides a method for training a smoke and flame recognition model, the method comprising:

[0006] Get the refinery smoke and fire dataset;

[0007] Acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; introduce a dual attention mechanism module into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module;

[0008] The ConvBNSiLU module and the C3 module based on the Backbone unit perform convolution and feature processing on each image in the smoke and flame dataset, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales;

[0009] Performing secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales;

[0010] Identifying the second hidden features of different scales by the Head unit to generate prediction values;

[0011] The loss function value is calculated according to the predicted value, and the parameters of the smoke and flame recognition model are updated until the preset conditions are met to obtain the smoke and flame recognition model.

[0012] In one or some optional implementations of the embodiment of the present application, the first hidden features of different scales include first hidden features of the first scale, first hidden features of the second scale, and first hidden features of the third scale; the Backbone unit also includes an SPPF module;

[0013] The ConvBNSiLU module and the C3 module based on the Backbone unit perform convolution and feature processing on each image in the smoke and flame dataset, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales, including:

[0014] For each picture in the smoke and flame dataset, input the picture into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a third hidden feature;

[0015] Inputting the third hidden feature into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a first hidden feature of a first scale;

[0016] Inputting the first hidden feature of the first scale into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain the first hidden feature of the second scale;

[0017] The first hidden feature of the second scale is input into the ConvBNSiLU module, the C3 module, the dual attention mechanism module and the SPPF module to obtain the first hidden feature of the third scale.

[0018] In one or some optional implementations of the embodiments of the present application, the dual attention mechanism module in the Backbone unit is an A2-Net dual attention mechanism module;

[0019] The process of using the dual attention mechanism module to extract features specifically includes:

[0020] Convolving the feature maps output by the ConvBNSiLU module and the C3 module to obtain feature maps A, B and V;

[0021] The feature map A is dimensionally transformed to obtain a fourth hidden feature; the feature maps B and V are dimensionally transformed and subjected to a Softmax operation to obtain an attention map and an attention vector;

[0022] Performing matrix multiplication of the fourth hidden feature and the attention map to obtain a global feature description;

[0023] Performing matrix multiplication of the attention vector and the global feature description to generate a feature distribution;

[0024] The feature distribution is dimensionally transformed to obtain a first hidden feature of a corresponding scale.

[0025] In one or some optional implementations of the embodiments of the present application, the SPPF module includes a ConvBNSiLU module and a maximum pooling layer;

[0026] The step of inputting the first hidden feature of the second scale into the ConvBNSiLU module, the C3 module, the dual attention mechanism module, and the SPPF module to obtain the first hidden feature of the third scale includes:

[0027] Inputting the first hidden feature of the second scale into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a fifth hidden feature;

[0028] Inputting the fifth hidden feature into the ConvBNSiLU module of the SPPF module to obtain a sixth hidden feature;

[0029] Inputting the sixth hidden feature into the maximum pooling layers connected in sequence to obtain the seventh hidden feature, the eighth hidden feature and the ninth hidden feature;

[0030] The sixth hidden feature, the seventh hidden feature, the eighth hidden feature and the ninth hidden feature are fused and input into the next ConvBNSiLU module to obtain the first hidden feature of the third scale.

[0031] In one or some optional implementations of the embodiment of the present application, before obtaining the initial smoke and flame recognition model, the method further includes: performing at least one of the following data enhancement operations on any image:

[0032] Based on the Mosaic method, multiple random images in the smoke and flame dataset are spliced;

[0033] Performing random affine transformation on the smoke and flame data set based on the Random affine method;

[0034] Based on the MixUP method, two random images in the smoke and flame dataset are weighted mixed;

[0035] The images in the smoke and flame dataset are adjusted in HSV color space based on the Augment HSV method.

[0036] In one or some optional implementations of the embodiment of the present application, the obtaining of a refinery smoke and flame dataset includes:

[0037] Collect smoke and flame images in oil refining scenes;

[0038] Obtain annotated image data obtained by annotating the smoke and flame image using an image annotation tool; wherein the annotation includes the position, size and label of the smoke and flame in the image;

[0039] The labeled image data is cleaned to obtain the refinery smoke and flame data set.

[0040] In a second aspect, an embodiment of the present application provides a smoke and flame recognition method, the method comprising:

[0041] Acquire the smoke and flame image of the refinery to be identified;

[0042] The image to be recognized is input into the smoke and flame recognition model obtained by using the training method of the smoke and flame recognition model to obtain a recognition result.

[0043] In one or some optional implementations of the embodiment of the present application, after obtaining the refinery smoke and flame image to be identified, the method further includes:

[0044] The refinery smoke and flame image to be identified is preprocessed to obtain a preprocessed refinery smoke and flame image to be identified.

[0045] In a third aspect, an embodiment of the present application provides a training device for a smoke and flame recognition model, the device comprising:

[0046] The first acquisition module is used to acquire the refinery smoke and flame data set;

[0047] The second acquisition module is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module;

[0048] A first processing module, configured to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to perform feature extraction to obtain first hidden features of different scales;

[0049] A second processing module, configured to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales;

[0050] A third processing module, configured to respectively identify the second hidden features of different scales through the Head unit to generate prediction values;

[0051] The third acquisition module is used to calculate a preset loss function value according to the predicted value, update the parameters of the smoke and flame recognition model until a preset condition is met, and obtain the smoke and flame recognition model.

[0052] In a fourth aspect, an embodiment of the present application provides a smoke and flame identification device, the device comprising:

[0053] The first acquisition module is used to acquire the refinery smoke and flame data set;

[0054] The second acquisition module is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module;

[0055] A first processing module, configured to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to perform feature extraction to obtain first hidden features of different scales;

[0056] A second processing module, configured to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales;

[0057] A third processing module, configured to respectively identify the second hidden features of different scales through the Head unit to generate prediction values;

[0058] The third acquisition module is used to calculate the preset loss function value according to the predicted value, update the parameters of the smoke and flame recognition model until the preset conditions are met, and obtain the smoke and flame recognition model

[0059] A fourth acquisition module is used to acquire the smoke and flame image of the refinery to be identified;

[0060] The first recognition module is used to input the image to be recognized into the smoke and flame recognition model to obtain a recognition result.

[0061] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned training method for the smoke and flame recognition model and / or the above-mentioned smoke and flame recognition method.

[0062] In a sixth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the smoke and flame recognition model as described above and / or the smoke and flame recognition method as described above are implemented.

[0063] In a seventh aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is executed on a computer device, the computer device executes the training method of the smoke and flame recognition model as described above, and / or the smoke and flame recognition method as described above.

[0064] In an eighth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the above-mentioned training method of the smoke and flame recognition model and / or the above-mentioned smoke and flame recognition method.

[0065] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0066] The training method of the smoke and flame recognition model provided in the embodiment of the present application uses Backbone units, Neck units and Head units to form an improved YOLOv5 smoke and flame recognition model, and introduces a dual attention mechanism module therein. The YOLOv5 model combined with the dual attention mechanism can effectively enhance the model's ability to pay attention to the position of smoke and flames. The dual attention mechanism can learn the spatial relationship between different regions, perform space-time relationship learning, consider the relationship between frames, capture the time-series evolution of smoke and flames, and enable the model to more accurately locate smoke and flames. For a small sample of smoke and flame data sets, the dual attention mechanism can help the YOLOv5 model focus on useful features in the data, enable the model to better eliminate adverse factors such as insufficient lighting and shadows, and improve the ability to extract smoke and flame features, so that the YOLOv5 model can be more powerful and efficient in smoke and flame detection.

[0067] The training method of the smoke and flame recognition model provided in the embodiment of the present application introduces a dual attention mechanism to a model that can achieve lower computing cost and memory usage when modeling dependencies, thereby improving the efficiency and practicality of the model.

[0068] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flowchart of a method for training a smoke and flame recognition model provided in an embodiment of the present application;

[0070] Figure 2 It is a schematic diagram of the network structure of the smoke and flame recognition model provided in the embodiment of the present application;

[0071] Figure 3 It is a schematic diagram of the structure of the C3 module in the Backbone unit provided in an embodiment of the present application;

[0072] Figure 4 It is a structural schematic diagram of the SPP module provided in the embodiment of the present application;

[0073] Figure 5 It is a structural diagram of the SPPF module provided in the embodiment of the present application;

[0074] Figure 6 A schematic diagram of the steps of the smoke and flame identification method provided in an embodiment of the present application;

[0075] Figure 7 A schematic diagram of the structure of a training device for a smoke and flame recognition model provided in an embodiment of the present application;

[0076] Figure 8 A schematic diagram of the structure of a smoke and flame identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0078] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or clusters thereof.

[0079] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0080] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0081] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0082] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0083] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0084] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0085] The inventors found that in the prior art, due to the particularity of the oil refining scene, the traditional target detection and recognition algorithm still has some problems in smoke and flame detection. The smoke and flame data set is small in size, and water vapor often appears in the refinery, which is easily confused with smoke. At the same time, the smoke and flame in the refinery are quite different from those in other places in terms of color and shape. The existing smoke and flame recognition model is not adaptable enough to the color and shape changes of smoke and flame, and is prone to misjudgment and missed detection. Therefore, there is an urgent need for a training method for a smoke and flame recognition model to meet the prediction accuracy requirements required in practical applications and overcome the defects of the prior art. Based on this, the inventors made the present invention after further research, providing a training method for a smoke and flame recognition model, a smoke and flame recognition method and a device.

[0086] Embodiment 1

[0087] See also Figure 1 , is a training method for a smoke and flame recognition model provided in Example 1 of the present application, referring to Figure 1 As shown, the method includes:

[0088] S101: Obtain a dataset of smoke and fire from a refinery.

[0089] In the embodiment of the present application, in the above step S101, the step of obtaining the refinery smoke and flame dataset includes:

[0090] Collect smoke and flame images in oil refining scenes;

[0091] Obtain annotated image data obtained by annotating the smoke and flame image using an image annotation tool; wherein the annotation includes the position, size and label of the smoke and flame in the image;

[0092] The labeled image data is cleaned to obtain the refinery smoke and flame data set.

[0093] In the embodiment of the present application, in order to accurately reflect the characteristics of smoke and flames in the oil refining scene, it is necessary to determine the collection scene, select some representative oil refining scenes, such as oil tank areas, refineries, etc., shoot the smoke and flames, ensure that the image is clear, well-lit, and at an appropriate angle, use professional annotation tools such as labelimg, labelme, etc. to annotate the image, annotate the position, size, and label of the smoke and flames, and perform data cleaning on the annotated data, delete images with incorrect or unclear annotations, and ensure the accuracy and reliability of the data set. The refinery smoke and flame dataset created through the above steps provides accurate and comprehensive data support for the subsequent training of the smoke and flame recognition model, and provides a basis for the optimization and improvement of the algorithm.

[0094] In an embodiment of the present application, it also includes performing data enhancement operations on the smoke and flame dataset, specifically including: splicing multiple random images in the smoke and flame dataset based on the Mosaic method; performing random affine transformation on the smoke and flame dataset based on the Random affine method; weighted mixing two random images in the smoke and flame dataset based on the MixUP method; and performing HSV color space adjustment on the images in the smoke and flame dataset using the Augment HSV method.

[0095] Mosaic data augmentation is a widely used image augmentation method that can effectively expand the data set and improve the robustness and generalization ability of the model. The specific implementation steps are as follows: Randomly select four images from the data set and store their position, size, label and other information respectively. The four images are randomly rotated, scaled, translated and other transformations are performed, and overlapped to a certain extent to form a mosaic image. In the generated mosaic image, the labels of the four images are adjusted according to the position and size of the original image to adapt to the size and shape of the mosaic image.

[0096] Random affine (Rotation, Scale, Translation and Shear) data enhancement is a data enhancement method for affine transformation of images, which is used to expand the data set, improve the robustness and generalization ability of the model. It is specifically implemented by selecting a center point of the original image and using it as the transformation center. A 2D affine transformation matrix is ​​randomly generated, including transformations such as translation, rotation, scaling and shearing. In a specific embodiment, Scale and Transaction, i.e. scaling and translation transformations, are used to transform the input image using a randomly generated 2D affine transformation matrix to obtain a data-enhanced image, and the coordinates of the annotation box are adjusted according to the conversion relationship between the transformed image and the coordinates of the annotation box to ensure that the correspondence between the annotation box and the image is correct. MixUp and Augment HSV (Hue, Saturation, Value) data enhancement, the former superimposes two different images at a certain ratio to generate a new image; the latter adjusts the original image in the HSV color space to obtain images of various colors and brightness. For the same image, multiple enhancement combinations can be used. For example, two of the enhancement methods can be used to obtain a blurred and reversed image. In addition, three of the enhancement methods are performed on the original image to obtain a blurred, reversed, and brightness-adjusted image. Thus, images with more enhanced effects can be combined. Therefore, in addition to the effect of increasing recognition accuracy brought by the enhancement itself, the technical effect of data enhancement will also produce richer and more diverse data sets from the perspective of model training.

[0097] S102: Obtain an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module, and a dual attention mechanism module.

[0098] S103: Based on the ConvBNSiLU module and the C3 module of the Backbone unit, convolution and feature processing are performed on each image in the smoke and flame dataset, and feature extraction is performed using the dual attention mechanism module to obtain first hidden features of different scales.

[0099] In the embodiment of the present application, the smoke and flame recognition model is improved based on the YOLOv5 model, including a Backbone unit, a Neck unit and a Head unit connected in sequence. The structure of the smoke and flame recognition model is as follows: Figure 2 As shown in the figure, the left column is the Backbone unit, the middle two columns are the Neck units, and the rightmost column is the Head unit. The images in the dataset are input into the Backbone unit, and then pass through a ConvBNSiLU module containing a 6×6 convolution layer, and then pass through three combination modules, where the combination module consists of a ConvBNSiLU module, a C3 module, and a dual attention mechanism module. A SPPF module is connected at the end of the Backbone unit. Three first hidden features of different sizes are obtained after the second and third dual attention mechanism modules and SPPF modules respectively. The Backbone unit is the backbone unit of YOLOv5 and is the core component responsible for extracting features from the input image. The Backbone unit uses operations such as convolution to gradually reduce the resolution of the feature map. The deeper the module in the Backbone unit outputs the features, the lower the resolution of the feature map, while gradually increasing the number of channels and the size of the receptive field.

[0100] In a specific embodiment, the Backbone unit in the present application replaces the Focus module with a ConvBNSiLU module containing a 6×6 convolutional layer based on the original YOLOv5 network. Specifically, the original Focus module divides each 2×2 pixel block into a patch, and splices the pixels at the same position together in each patch to obtain 4 feature maps. The same position here refers to pixels of the same color, so in the same patch, pixels of the same color will be spliced ​​together to form a feature map. Then a 3×3 convolutional layer is connected to these four feature maps to finally obtain the output feature map. In the present application, a 6×6 convolutional layer is directly used to replace the original Focus module, and the input data is convolved according to the size of 6×6, which is similar to using the Focus module to divide the data into 2×2 patches and then perform convolution operations. This method does not require patch division, can reduce calculation operations, reduce memory consumption, and better parallelism, so it is more efficient and faster to calculate on existing GPU devices.

[0101] The ConvBNSiLU module in the Backbone unit consists of three components: convolutional layer, batch normalization layer, and activation function SiLU. The activation function uses SiLU to improve feature extraction capabilities. This enables the smoke and flame recognition model to perform target detection more powerfully and efficiently. The C3 module in the Backbone unit contains the ConvBNSiLU module and the Bottleneck module, such as Figure 3 As shown in the figure, in the C3 module, feature maps at different levels are connected across stages to fuse information of different scales and enrich the feature representation.

[0102] In the Backbone unit, after each C3 module, a layer of A2-Nets dual attention mechanism module is added to enhance the ability to focus on specific spatial locations. A2-Nets is a dual attention network module designed as a two-step dual attention mechanism. Its core idea is that the first step collects the key features of the entire space into a compact set through a secondary attention pool, and the second step adaptively selects and allocates features to each position through another attention, so that the subsequent convolutional layer can perceive the features of the entire space even without a large receptive field. The first step of the attention operation selectively collects key features from the entire space, while the second step of the attention operation uses another attention mechanism to adaptively allocate subsets of key features that help perform complex tasks.

[0103] The main steps of A2-Nets are as follows: convolve the input feature map to generate three feature maps A, B, and V of the same scale; after forward propagating the three features A, B, and V, feature A is dimensionally transformed to obtain the hidden feature tmpA, feature B is dimensionally transformed and passes through Softmax to obtain the attention map Attention_Maps, and C is also dimensionally transformed and passes through a layer of softmax to obtain the attention vector Attention_Vectors; Feature Gathering module: matrix multiplication of the hidden vector tmpA and the attention map Attention_Maps is obtained to obtain the global feature description global_descriptors; Feature Distribution module: matrix multiplication of the global feature description global_descriptors and the attention vector Attention_Vectors is performed, and the feature map is output after dimension conversion.

[0104] The SPPF module in the Backbone unit is derived from the SPP module in the original YOLOv5 model to improve efficiency. The structural diagram of the SPP module and the SPPF module is as follows: Figure 4 , Figure 5 As shown in the figure. The two modules have the same function, but the latter is more efficient. The SPP module passes the input through multiple maximum pooling layers of different scales in parallel, and then further fuses them, which can solve the problem of multi-scale targets in target detection to a certain extent. The SPPF module passes the input through multiple 5×5 maximum pooling layers in series. The two modules have the same function, but the SPPF structure is more efficient and faster.

[0105] S104: performing secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales.

[0106] In the embodiment of the present application, in the above step S105, the first hidden features of three different scales obtained in step S104 are subjected to secondary feature extraction to obtain second hidden features of different scales. The Neck unit structure is as follows: Figure 2 The Neck unit is responsible for fusing and processing the features of different scales from the Backbone unit to generate a feature map with richer information, and fusing the feature maps of different scales together through upsampling and downsampling operations. Specifically, Figure 2 The second column in is the bottom-up part, which convolves the first hidden feature of the bottom layer, fuses the convolved feature with the first hidden feature of the previous layer, and performs upward fusion of features. These two steps are repeated until the highest layer is reached. Figure 2The third column in is the top-down part, which downsamples the features from the previous layer to obtain the second hidden features of different scales.

[0107] After the bottom-up and top-down calculations of the Neck unit, three second hidden features of different scales are output, corresponding to the second hidden features of lower resolution, medium resolution, and higher resolution from shallow to deep. The shallow graphic features (features from the shallow layer of the convolutional neural network, which only contain simple graphics and lack semantic shapes) and the deep semantic features (features from the deep layer of the convolutional neural network, which have strong semantic shapes but lose some simple graphics) are better fused. The output second hidden features of lower resolution are the feature maps closest to the original image resolution, which are used to detect small-sized targets, because the second hidden features of lower resolution can retain more graphic features and help capture the global features of the target; the second hidden features of medium resolution aim to detect medium-sized targets, and there is a balance between graphic features and semantic features; the second hidden features of higher resolution are obtained by upsampling and are used to detect larger targets, paying more attention to the details and local features of the target. By fusing the first hidden features from different scales into the three output second hidden features, the Neck unit can provide rich multi-scale information, which helps to improve the model's detection ability for targets of different sizes, making the smoke and flame recognition model more accurate and more robust.

[0108] S105: Identify the second hidden features of different scales through the Head unit to generate prediction values.

[0109] In the embodiment of the present application, in the above step S106, the Head unit is a detection module, such as Figure 2 As shown in the leftmost column, the network structure of the Head unit is very simple, consisting of only three 1*1 convolutional layers, corresponding to the second hidden features of three different scales of the input. The Head unit also contains anchor boxes (Anchors), which are a set of predefined bounding boxes used to generate candidate boxes on the feature map. The convolutional layer predicts the position, category, and confidence of the second hidden features of three different scales obtained in step S105 according to the anchor boxes. Through these predictions, multiple objects of different sizes and proportions can be detected from the image.

[0110] Then, a series of subsequent processing is performed on the position, category and confidence generated by each anchor box to obtain the final target detection result. These subsequent processing include: decoding prediction, which requires decoding the predicted bounding box coordinates into actual coordinates on the image based on the predicted position information and the size of the anchor box; category screening, for each anchor box, the category with the highest probability is selected as the final category label based on the predicted category probability; NMS (non-maximum suppression): Since the same target may be detected by multiple anchor boxes, it is necessary to screen bounding boxes with large overlaps; threshold screening: Bounding boxes with confidence scores below a certain threshold can be filtered to remove low-confidence detection results. Through the above steps, the final prediction value is obtained.

[0111] S106: Calculate the loss function value according to the predicted value, and update the parameters of the smoke and flame recognition model until a preset condition is met to obtain the smoke and flame recognition model.

[0112] In the embodiment of the present application, in the above step S107, the predicted value is used to calculate the loss function value, and then the gradient of the loss function is propagated back to the smoke and flame recognition model through the back propagation algorithm, so as to update the parameters of the model. This parameter update process will be iterative until the preset number of training iterations or the condition of loss function convergence is met. Through this training process, the parameters of the smoke and flame recognition model are gradually optimized so that it can more accurately identify smoke and flame targets. Finally, after multiple iterations of training, a trained smoke and flame recognition model is obtained.

[0113] Embodiment 2

[0114] Based on the same inventive concept, the present application also provides a smoke and flame identification method, referring to Figure 6 As shown, the method includes:

[0115] S201: Acquire a smoke and flame image of a refinery to be identified;

[0116] S202: Inputting the image to be recognized into the smoke and flame recognition model obtained by the training method of the smoke and flame recognition model in the first embodiment to obtain a recognition result.

[0117] The refinery smoke and flame image to be identified is preprocessed to obtain a preprocessed refinery smoke and flame image to be identified, wherein the preprocessing mainly includes operations such as image format conversion, image scaling, and standardization.

[0118] Embodiment 3

[0119] Based on the same inventive concept, the present application embodiment also provides a training device for a smoke and flame recognition model, referring to Figure 7 As shown, the device comprises:

[0120] The first acquisition module 101 is used to acquire a smoke and flame dataset of a refinery;

[0121] The second acquisition module 102 is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module;

[0122] A first processing module 103 is used to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales;

[0123] A second processing module 104 is used to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales;

[0124] A third processing module 105 is used to respectively identify the second hidden features of different scales through the Head unit to generate prediction values;

[0125] The third acquisition module 106 is used to calculate a preset loss function value according to the predicted value, and update the parameters of the smoke and flame recognition model until a preset condition is met to obtain the smoke and flame recognition model.

[0126] Embodiment 4

[0127] Based on the same inventive concept, the present application also provides a smoke and flame identification device, referring to Figure 8 As shown, the device comprises:

[0128] The first acquisition module 201 is used to acquire a smoke and flame dataset of a refinery;

[0129] The second acquisition module 202 is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module;

[0130] A first processing module 203 is used to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales;

[0131] A second processing module 204 is used to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales;

[0132] A third processing module 205 is used to respectively identify the second hidden features of different scales through the Head unit to generate prediction values;

[0133] The third acquisition module 206 is used to calculate a preset loss function value according to the predicted value, update the parameters of the smoke and flame recognition model until the preset conditions are met, and obtain the smoke and flame recognition model

[0134] The fourth acquisition module 207 is used to acquire the smoke and flame image of the refinery to be identified;

[0135] The first recognition module 208 is used to input the image to be recognized into the smoke and flame recognition model to obtain a recognition result.

[0136] Embodiment 5

[0137] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the training method for the smoke and flame recognition model described in the above embodiment 1 and / or the smoke and flame recognition method described in the above embodiment 2 are implemented.

[0138] Embodiment 6

[0139] Based on the same inventive concept, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the training method for the smoke and flame recognition model described in the above-mentioned embodiment 1 and / or the smoke and flame recognition method described in the above-mentioned embodiment 2 is implemented.

[0140] Embodiment 7

[0141] Based on the same inventive concept, an embodiment of the present application further provides a computer program product comprising instructions. When the computer program product is executed on a computer device, the computer device executes the training method for the smoke and flame recognition model described in the above embodiment 1, and / or the smoke and flame recognition method described in the above embodiment 2.

[0142] Embodiment 8

[0143] Based on the same inventive concept, an embodiment of the present application further provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the training method of the smoke and flame recognition model described in the above embodiment 1, and / or the smoke and flame recognition method described in the above embodiment 2.

[0144] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiment when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0145] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the steps in the above-mentioned method embodiment can be implemented when the computer device executes it.

[0146] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0148] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for training a smoke and flame recognition model, characterized in that: include: Get the refinery smoke and fire dataset; Acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; introduce a dual attention mechanism module into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module; The ConvBNSiLU module and the C3 module based on the Backbone unit perform convolution and feature processing on each image in the smoke and flame dataset, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales; Performing secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales; Identifying the second hidden features of different scales by the Head unit to generate prediction values; The loss function value is calculated according to the predicted value, and the parameters of the smoke and flame recognition model are updated until the preset conditions are met to obtain the smoke and flame recognition model.

2. The method according to claim 1, characterized in that The first hidden features of different scales include first hidden features of a first scale, first hidden features of a second scale, and first hidden features of a third scale; The Backbone unit also includes an SPPF module; The ConvBNSiLU module and the C3 module based on the Backbone unit perform convolution and feature processing on each image in the smoke and flame dataset, and use the dual attention mechanism module to extract features to obtain first hidden features of different scales, including: For each picture in the smoke and flame dataset, input the picture into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a third hidden feature; Inputting the third hidden feature into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a first hidden feature of a first scale; Inputting the first hidden feature of the first scale into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain the first hidden feature of the second scale; The first hidden feature of the second scale is input into the ConvBNSiLU module, the C3 module, the dual attention mechanism module and the SPPF module to obtain the first hidden feature of the third scale.

3. The method according to claim 2, characterized in that The dual attention mechanism module in the Backbone unit is an A2-Net dual attention mechanism module; The process of using the dual attention mechanism module to extract features specifically includes: Convolving the feature maps output by the ConvBNSiLU module and the C3 module to obtain feature maps A, B and V; The feature map A is dimensionally transformed to obtain a fourth hidden feature; the feature maps B and V are dimensionally transformed and subjected to a Softmax operation to obtain an attention map and an attention vector; Performing matrix multiplication of the fourth hidden feature and the attention map to obtain a global feature description; Performing matrix multiplication of the attention vector and the global feature description to generate a feature distribution; The feature distribution is dimensionally transformed to obtain a first hidden feature of a corresponding scale.

4. The method according to claim 3, characterized in that The SPPF module includes a ConvBNSiLU module and a maximum pooling layer; The step of inputting the first hidden feature of the second scale into the ConvBNSiLU module, the C3 module, the dual attention mechanism module, and the SPPF module to obtain the first hidden feature of the third scale includes: Inputting the first hidden feature of the second scale into the ConvBNSiLU module, the C3 module and the dual attention mechanism module to obtain a fifth hidden feature; Inputting the fifth hidden feature into the ConvBNSiLU module of the SPPF module to obtain a sixth hidden feature; Inputting the sixth hidden feature into the maximum pooling layers connected in sequence to obtain the seventh hidden feature, the eighth hidden feature and the ninth hidden feature; The sixth hidden feature, the seventh hidden feature, the eighth hidden feature and the ninth hidden feature are fused and input into the next ConvBNSiLU module to obtain the first hidden feature of the third scale.

5. The method according to claim 1, characterized in that Before obtaining the initial smoke and flame recognition model, the method further includes: for any image, performing at least one of the following data enhancement operations: Based on the Mosaic method, multiple random images in the smoke and flame dataset are spliced; Performing random affine transformation on the smoke and flame data set based on the Random affine method; Based on the MixUP method, two random images in the smoke and flame dataset are weighted mixed; The images in the smoke and flame dataset are adjusted in HSV color space based on the Augment HSV method.

6. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a refinery smoke and flame dataset includes: Collect smoke and flame images in oil refining scenes; Obtain annotated image data obtained by annotating the smoke and flame image using an image annotation tool; wherein the annotation includes the position, size and label of the smoke and flame in the image; The labeled image data is cleaned to obtain the refinery smoke and flame data set.

7. A smoke and flame identification method, characterized in that: include: Acquire the smoke and flame image of the refinery to be identified; The image to be recognized is input into a smoke and flame recognition model obtained by the method according to any one of claims 1 to 6 to obtain a recognition result.

8. The method according to claim 7, characterized in that After obtaining the smoke and flame image of the refinery to be identified, it also includes: The refinery smoke and flame image to be identified is preprocessed to obtain a preprocessed refinery smoke and flame image to be identified.

9. A training device for a smoke and flame recognition model, characterized in that: include: The first acquisition module is used to acquire the refinery smoke and flame data set; The second acquisition module is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module; A first processing module, configured to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to perform feature extraction to obtain first hidden features of different scales; A second processing module, configured to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales; A third processing module, configured to respectively identify the second hidden features of different scales through the Head unit to generate prediction values; The third acquisition module is used to calculate a preset loss function value according to the predicted value, update the parameters of the smoke and flame recognition model until a preset condition is met, and obtain the smoke and flame recognition model.

10. A smoke and flame identification device, characterized in that: include: The first acquisition module is used to acquire the refinery smoke and flame data set; The second acquisition module is used to acquire an initial smoke and flame recognition model; wherein the initial smoke and flame recognition model includes a Backbone unit, a Neck unit and a Head unit connected in sequence; a dual attention mechanism module is introduced into the Backbone unit; the Backbone unit includes a ConvBNSiLU module, a C3 module and a dual attention mechanism module; A first processing module, configured to perform convolution and feature processing on each image in the enhanced data based on the ConvBNSiLU module and the C3 module of the Backbone unit, and use the dual attention mechanism module to perform feature extraction to obtain first hidden features of different scales; A second processing module, configured to perform secondary feature extraction on the first hidden features of different scales based on the Neck unit to obtain second hidden features of different scales; A third processing module, configured to respectively identify the second hidden features of different scales through the Head unit to generate prediction values; The third acquisition module is used to calculate the preset loss function value according to the predicted value, update the parameters of the smoke and flame recognition model until the preset conditions are met, and obtain the smoke and flame recognition model A fourth acquisition module is used to acquire the smoke and flame image of the refinery to be identified; The first recognition module is used to input the image to be recognized into the smoke and flame recognition model to obtain a recognition result.

11. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the instructions are executed on a terminal, the terminal executes the smoke and flame recognition model training method according to any one of claims 1 to 6, and / or the smoke and flame recognition method according to claim 7 or 8.

12. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the smoke and flame recognition model according to any one of claims 1 to 6 and / or the smoke and flame recognition method according to claim 7 or 8 is implemented.

13. A computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the smoke and flame recognition model training method according to any one of claims 1 to 6, and / or the smoke and flame recognition method according to claim 7 or 8.

14. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a computer program or instruction to implement the smoke and flame recognition model training method according to any one of claims 1 to 6, and / or the smoke and flame recognition method according to claim 7 or 8.

Images