Image smoke segmentation model, training method, and image smoke segmentation method

Abstract

This invention discloses an image smoke segmentation model, which includes a backbone network, a dense multi-scale context module, a detail information enhancement module, and a feature fusion module. A first unit extracts first feature information from the image to be identified. A second unit obtains second feature information based on the first feature information and sends it to the detail information enhancement module, which extracts edge information of the smoke from the second feature information. A third unit calculates the spatial attention coefficient of the image to be identified based on the third feature information. A fourth unit sends fourth feature information to the dense multi-scale context module, which obtains context information for each sub-region of the image to be identified from the fourth feature information and sends it to the feature fusion module. The feature fusion module fuses the edge information, spatial attention coefficient, and context information to obtain a feature map of the image to be identified.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image segmentation technology, and in particular to an image smoke segmentation model, training method, and image smoke segmentation method. Background Technology

[0002] Fire is an extremely devastating disaster, causing not only immense property damage but also endangering lives and disrupting the ecological balance. Unlike other disasters, fires exhibit very clear visual cues, such as smoke and flames. In most cases, fires develop relatively slowly, potentially lasting for a considerable period before a major fire breaks out. Therefore, smoke detection provides earlier fire warnings than flame detection, giving firefighters and other relevant personnel ample time to extinguish and combat the fire. In many applications, accurate smoke detection plays a crucial role in fire early warning and firefighting efforts, significantly reducing fire-related losses. Traditional smoke detection technologies are typically based on sensing temperature or particle concentration. However, these sensors are ineffective in open spaces.

[0003] In existing technologies, visual smoke detection methods utilize computer vision technology to monitor and locate smoke from images, making them more suitable for smoke detection in open, outdoor scenes than traditional sensors. However, the irregular shape, blurred edges, and semi-transparency of smoke significantly increase the difficulty of accurately segmenting smoke regions. Summary of the Invention

[0004] Therefore, it is necessary to address the above problems by proposing an image smoke segmentation model, training method, and image smoke segmentation method. The aforementioned model and method can effectively segment smoke targets of different scales in the image to be detected.

[0005] An image smoke segmentation model includes a backbone network, a dense multi-scale context module, a detail enhancement module, and a feature fusion module. The backbone network comprises a first unit, a second unit, a third unit, and a fourth unit. The first unit extracts first feature information from the image to be identified. The second unit receives the first feature information, obtains second feature information based on it, and sends the second feature information to the detail enhancement module, which extracts edge information of the smoke from the second feature information. The third unit receives the second feature information, obtains third feature information based on it, and calculates spatial attention coefficients for the image to be identified based on the third feature information. The fourth unit receives the third feature information, obtains fourth feature information based on it, and sends the fourth feature information to the dense multi-scale context module, which extracts context information for each sub-region of the image to be identified from the fourth feature information and sends it to the feature fusion module. The feature fusion module fuses the edge information, spatial attention coefficients, and context information to obtain a feature map of the image to be identified.

[0006] Optionally, the model also includes an upsampling module, which is used to obtain a feature map and then perform convolution and upsampling on the feature map to obtain a smoke segmentation prediction map of the image to be identified.

[0007] Optionally, the spatial attention coefficients of the image to be recognized are calculated using the following formula:

[0008] X1′=(F 1×1 ((GAP(X1),GMP(X1))))

[0009] Where X1′ is the spatial attention coefficient, σ is the Sigmoid function, and F 1×1 It is a 1×1 convolution, GAP is global average pooling, GMP is global max pooling, and X1 is the third feature information.

[0010] A training method for an image smoke segmentation model, the method comprising:

[0011] Obtain the training set;

[0012] Set training parameters;

[0013] The target smoke segmentation model is obtained by training the image smoke segmentation model described above using the training set and training parameters.

[0014] Optionally, the training parameters include the learning policy, loss function, initialization method, and activation function. Specifically, setting the training parameters includes:

[0015] The learning strategy is configured as Poly;

[0016] Configure the loss function as the cross-entropy loss function;

[0017] Set the initialization method to Kaiming;

[0018] Set the activation function device to ReLU.

[0019] Optionally, the training parameters also include image size, batch size, weight decay, momentum, initial learning rate, and number of iterations. Specifically, setting the training parameters includes:

[0020] Set the image size to 256 pixels × 256 pixels;

[0021] Set the batch data size to 16;

[0022] Set the weight decay to 1e-5;

[0023] Set the momentum to 0.9;

[0024] Set the initial learning rate to 1e-2;

[0025] The number of iterations is set to 30000.

[0026] Optionally, the training set is obtained, specifically including:

[0027] Extract the first smoke image from historical smoke videos and retrieve the second smoke image from the database;

[0028] The first smoke image and the second smoke image are filtered to obtain the pre-training image;

[0029] The pre-training images are labeled to obtain the training images;

[0030] The training images are combined into a training set.

[0031] An image smoke segmentation method, the method comprising:

[0032] Acquire the image to be recognized;

[0033] The image to be identified is input into the target smoke segmentation model obtained by the training method described above, and the smoke segmentation result is obtained.

[0034] Optionally, the image to be recognized is acquired, specifically including:

[0035] Obtain real-time video of the area where smoke conditions need to be monitored;

[0036] Extract the image to be identified from real-time video;

[0037] The image to be recognized is obtained by converting the format of the image to be recognized.

[0038] A smoke alarm method, the method comprising:

[0039] Based on the smoke segmentation results obtained from the above image smoke segmentation method, determine whether there is smoke in the area where smoke needs to be monitored.

[0040] An alarm will be triggered when smoke is detected in an area where smoke monitoring is required.

[0041] The image smoke segmentation model comprises a backbone network, a dense multi-scale context module, an asymmetric pooling detail enhancement module, and a feature fusion module. The backbone network includes four units: Unit 1, Unit 2, Unit 3, and Unit 4. Unit 1 acquires the image to be recognized. Unit 2, combined with the asymmetric pooling detail enhancement module, acquires edge information of irregular objects in the image, enhancing the focus on edge details. Unit 3 generates spatial attention coefficients for the image. Unit 4, combined with the dense multi-scale context module, captures contextual information of different sub-regions of objects in the image. The deep fusion of spatial attention coefficients with multi-scale contextual and detail information supplements a large amount of spatial information, improving the network's robustness.

[0042] In the above embodiments, the feature fusion module is used to fuse the edge information of irregular objects, spatial attention coefficients and contextual information of different sub-regions of objects to obtain a feature map of the image to be identified. Furthermore, the feature map is used to obtain the smoke segmentation result map of the image to be identified, and finally the effective segmentation of smoke targets at different scales is achieved. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] in:

[0045] Figure 1 This is a schematic diagram of the image smoke segmentation model in the first embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the image smoke segmentation model in the second embodiment of the present invention;

[0047] Figure 3 This is a flowchart of the training method for the image smoke segmentation model in the first embodiment of the present invention;

[0048] Figure 4This is a flowchart of step S401 in the first embodiment of the present invention;

[0049] Figure 5 This is a flowchart of the image smoke segmentation method in the first embodiment of the present invention;

[0050] Figure 6 This is a flowchart of step S801 in the first embodiment of the present invention;

[0051] Figure 7 This is a flowchart of the smoke alarm method in the first embodiment of the present invention;

[0052] Figure 8 This is a schematic diagram of the internal structure of the computer device in the first embodiment of the present invention;

[0053] Figure 9 This is a schematic diagram of the image detection results in the first embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Figure 1 This is a schematic diagram of the image smoke segmentation model in the first embodiment of the present invention. Please refer to... Figure 1 The image smoke segmentation model comprises a backbone network, a dense multi-scale context module, a detail enhancement module, and a feature fusion module. The backbone network includes a first unit, a second unit, a third unit, and a fourth unit. Specifically, the backbone network in this application is a ResNet50 network pre-trained on ImageNet. The backbone network feeds features at various levels to the dense multi-scale context module, the detail enhancement module, and the feature fusion module for further processing. Furthermore, the ResNet50 network in this application incorporates dilated convolutional layers. The feature map size output by the ResNet50 network is 1 / 8 of the image to be recognized. The dilated convolutional layers can expand the receptive field of the backbone network, thereby acquiring more visual information.

[0056] The first unit is used to extract the first feature information of the image to be recognized. Specifically, the first unit performs preliminary feature extraction on the image to be recognized to obtain the first shallow features.

[0057] The second unit receives the first feature information, obtains the second feature information based on the first feature information, and sends the second feature information to the detail information enhancement module. The detail information enhancement module extracts the edge information of the smoke from the second feature information. Specifically, the detail information enhancement module, also known as the detail information enhancement module with embedded asymmetric pooling, further extracts the first shallow feature to obtain the second shallow feature, and then sends the second shallow feature to the detail information enhancement module with embedded asymmetric pooling to obtain the edge information of the smoke. The detail information enhancement module with embedded asymmetric pooling can enhance the model's attention to the edge detail information of irregular objects; in this embodiment, the irregular object is smoke.

[0058] In this embodiment, the embedded asymmetric pooling detail enhancement module is a module that uses an asymmetric pooling window to extract features of the object in the horizontal and vertical directions.

[0059] For example, given a feature information of C×H×W, asymmetric pooling takes the average of all pixels in the row or column of each channel, resulting in spatial shapes of C×H×1 and C×1×W, respectively. In this embodiment, matrix multiplication is used during feature recovery to separate the input second feature information into channels, reducing computational complexity. More specifically, the asymmetric pooling methods in this application are n×1 and 1×n, taking into account the feature information of irregular targets. Furthermore, the embedded asymmetric pooling detail enhancement module also includes residual branches and grouped convolution branches. Max pooling and grouped convolution are used in the residual branch. Max pooling effectively highlights the main response, which is beneficial for judging and locating smoke categories, while the use of the grouped convolution branch reduces the number of model parameters. Finally, the fused information from the two branches is output through a nonlinear operation, which greatly alleviates gradient vanishing or gradient exploding, ensuring the stability of the feature information output.

[0060] The third unit receives the second feature information, obtains the third feature information based on the second feature information, and calculates the spatial attention coefficient of the image to be recognized based on the third feature information. Specifically, the third unit further extracts the second shallow features to obtain the third shallow features, and calculates the spatial attention coefficient of the image to be recognized based on the third shallow features. The spatial attention coefficient can improve the model's recognition of spatial information in the image to be recognized, thus supplementing a large amount of spatial information.

[0061] The fourth unit receives the third feature information, obtains the fourth feature information based on the third feature information, and sends the fourth feature information to the Dense Multi-scale Context Module (DMCM). The DMCM extracts the context information of each sub-region of the image to be recognized from the fourth feature information and sends it to the feature fusion module. Specifically, the DMCM takes the fourth feature information output from the fourth unit of the backbone network as its input. The DMCM obtains context information at different scales based on the context aggregation method of the pyramid region, performs dense connections between different sub-regions, generates a stronger receptive field, and extracts multi-scale context information more effectively, compensating for information loss caused by excessively large sampling intervals. The DMCM uses asymmetric convolution to enhance the features of irregular targets and fuses them with the dense multi-scale information, reducing the limitations of pyramid pooling.

[0062] Furthermore, after concatenating the feature extraction information at different scales, convolution (Conv), batch normalization (BN), and the ReLU activation function are used to reduce the dimensionality of the overall output, obtaining feature information with rich and dense multi-scale contextual information. In addition to smoke, the image to be identified may also include environmental information such as trees and houses; this environmental information constitutes the contextual information of the image to be identified. Supplementing the contextual information helps to better identify the environment in which the smoke occurs, thereby accurately determining the probability of smoke appearing in the current environment.

[0063] Furthermore, this application upsamples the feature maps of the pyramid sub-regions, which are 1×1 and 3×3 in size, to scales of 5×5 and 7×7, respectively, and then concatenates and reduces their dimensionality. This compensates for the information loss caused by the large pooling kernel and focuses on small details. On the other hand, considering the differences in features across different regions, information fusion is performed using skip-layer connections. For example, the 1×1 sub-region is upsampled to 5×5 and then concatenated with the 5×5 scale information. In addition, a series of asymmetric convolutional blocks are used to extract the original features, aiming to focus on the feature information of irregular objects and complement the scale recovery information at each stage, thereby reducing the varying degrees of limitation caused by the local receptive field of the model.

[0064] The feature fusion module is used to fuse edge information, spatial attention coefficients, and contextual information to obtain a feature map of the image to be recognized. Specifically, the feature information of the fourth unit of the backbone network... First, the feature is processed through a dense multi-scale context module, then through bilinear interpolation upsampling, resulting in a feature output of the same size as the enhanced detail information. Simultaneously, the feature output embedded with the asymmetric pooling detail information enhancement module is element-wise added to the above-processed feature to obtain a feature map containing both multi-scale context information and enhanced detail information.

[0065] In other feasible embodiments, the model can also perform element-wise weighted multiplication using spatial attention coefficients after obtaining the enhanced feature map to obtain the final feature map. In this embodiment, the feature information processed by the feature fusion module contains rich spatial location information and deep semantic feature information, significantly improving the ability to represent pixel-level features. Furthermore, the feature fusion module fuses edge information, spatial attention coefficients, and contextual information to obtain rich spatial information, irregular object edge detail information, and contextual information, ensuring the robustness of the model.

[0066] In this embodiment, the input to the feature fusion module is mainly divided into three branches: the spatial attention coefficient branch, the output branch of the dense multi-scale context module, and the output branch of the embedded asymmetric pooling detail information enhancement module. First, the backbone network outputs features in the third stage. The data are fed into a global average pooling layer and a global max pooling layer, respectively. Global average pooling evaluates the entire feature map from a global perspective, while global max pooling maximizes the prominence of the main response features. The two pooling layers are concatenated along the channel dimension, and the fused features are then subjected to 1×1 convolution for dimensionality reduction and a sigmoid function to generate spatial attention coefficients, denoted as X1′. The spatial attention coefficients of the image to be recognized are calculated using the following formula:

[0067] X1′=(F 1×1 ((GAP(X1),GMP(X1))))

[0068] Where X1′ is the spatial attention coefficient, σ is the Sigmoid function, and F 1×1 It is a 1×1 convolution, GAP is global average pooling, GMP is global max pooling, and X1 is the third feature information.

[0069] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the image smoke segmentation model in the first embodiment of the present invention. The smoke segmentation model provided in the second embodiment differs from the smoke segmentation model provided in the first embodiment in that it further includes an upsampling module. The smoke segmentation model provided in the second embodiment also includes the following steps: The upsampling module is used to acquire a feature map, and then performs convolution and upsampling on the feature map to obtain a smoke segmentation prediction map of the image to be identified. Specifically, the feature map is then subjected to a 1×1 convolution and upsampling operation to restore it to the original image size, and the segmentation prediction map is output. This embodiment can more intuitively demonstrate the results of the model after smoke segmentation.

[0070] In the above embodiments, smoke targets of different scales can be effectively detected, and good recognition and segmentation effects are achieved for smoke with undesirable visual characteristics such as amorphous shape, blurred edges, and semi-transparency. The smoke region is accurately located and segmented while testing whether smoke exists in the image to be identified.

[0071] Please refer to the following: Figure 3 This is a flowchart of the training method for the image smoke segmentation model in the first embodiment of the present invention. The training method for the image smoke segmentation model specifically includes...

[0072] Step S401: Obtain the training set. In this embodiment, some images are obtained from existing videos with smoke, or from some existing smoke images, as the training set. Please refer to steps S4011-S4014 for details.

[0073] Step S402: Set training parameters. In this embodiment, the training parameters include a learning strategy, a loss function, an initialization method, and an activation function. Specifically, setting the training parameters includes setting the learning strategy to Poly, the loss function to cross-entropy loss, the initialization method to Kaiming, and the activation function to ReLU.

[0074] Furthermore, the training parameters also include image size, batch size, weight decay, momentum, initial learning rate, and number of iterations. Specifically, the training parameters are set as follows: image size is set to 256 pixels × 256 pixels, batch size is set to 16, weight decay is set to 1e-5, momentum is set to 0.9, initial learning rate is set to 1e-2, and number of iterations is set to 30000.

[0075] Step S403: Train the above-mentioned image smoke segmentation model using the training set and training parameters to obtain the target smoke segmentation model. In this embodiment, different models are trained to obtain different target smoke segmentation models. Specifically, training the model in Embodiment 1 yields a target smoke segmentation model capable of outputting feature maps. Training the model in Embodiment 2 yields a target smoke segmentation model capable of outputting smoke segmentation maps. In practical applications, the required image smoke segmentation model can be trained according to business needs.

[0076] Please refer to the following: Figure 4 This is a flowchart of step S401 in the first embodiment of the present invention. Step S401, obtaining the training set, specifically includes the following steps.

[0077] Step S4011: Extract a first smoke image from historical smoke videos and retrieve a second smoke image from the database. Specifically, smoke images are extracted from forest scene videos containing smoke. Since the smoke changes slowly between frames, one image is extracted every four frames as a sample. The currently constructed dataset contains smoke from hundreds of real-world scenes, including indoors, factories, forests, and villages. The above data is for illustrative purposes only and is not intended to be limiting.

[0078] Step S4012 involves filtering the first and second smoke images to obtain pre-training images. Specifically, images without smoke are deleted, and the image format and size of images with smoke are modified. Furthermore, new smoke images are generated using techniques such as random cropping, translation, and flipping. These operations enrich the smoke dataset, thereby improving the generalization ability of the training model and preventing network overfitting.

[0079] Step S4013: Annotate the pre-training images to obtain training images. Specifically, Adobe Photoshop is used for annotation; in this embodiment, pixel-level manual annotation of the smoke targets is required.

[0080] Step S4014: The training images are used to form a training set. To increase the richness of the samples, some synthetic smoke data and difficult samples are added as the final smoke dataset. In this embodiment, the dataset is divided into a training set, a validation set, and a test set according to a certain ratio. The training set is used to train the model's ability to recognize smoke images. The validation set is used to verify the model's ability to recognize smoke images. The test set is used to test whether the model's parameters meet the standards of the target model.

[0081] Please refer to the following: Figure 5 This is a flowchart of an image smoke segmentation method according to one embodiment of the present invention. The image smoke segmentation method specifically includes the following steps.

[0082] Step S801: Obtain the image to be identified. In this implementation, an image is captured from the real-time video of the area where smoke needs to be monitored as the image to be identified. For details, please refer to steps S8011-S8013.

[0083] Step S802: Input the image to be recognized into the target smoke segmentation model obtained according to the above training method to obtain the smoke segmentation result. Specifically, training the model in Example 1 yields a target smoke segmentation model capable of outputting feature maps. Inputting the image to be recognized into the target smoke segmentation model in Example 1 obtained according to the above training method yields a smoke segmentation result, which is a feature map of the smoke image. Training the model in Example 2 yields a target smoke segmentation model capable of outputting a smoke segmentation map. Inputting the image to be recognized into the target smoke segmentation model in Example 2 obtained according to the above training method yields a smoke segmentation result, which is a smoke image segmentation map. In practical applications, the required image smoke segmentation model can be trained according to business needs.

[0084] In this embodiment, the trained weight parameter file is loaded into the network model to initialize the smoke segmentation model. Video frame data is input into the model, and the smoke segmentation network outputs two-channel feature information through the trained dense multi-scale feature fusion. Each channel represents a class, with 0 representing the background and 1 representing the smoke target. Finally, the data is upsampled to the original image size, and the segmentation result is output. Some test results are shown below. Figure 9 As shown. Further, based on the segmentation results, the predicted probabilities of the background and smoke target are compared. If the area of ​​the smoke region exceeds a set threshold, it is determined that smoke exists, and the smoke is located and segmented in the original video, and the frame image is saved.

[0085] In other feasible embodiments, an alarm is triggered upon detecting the presence of smoke, and feedback is sent to monitoring personnel. The above steps are repeated until all frames of video data have been traversed.

[0086] In other feasible embodiments, appropriate monitoring equipment can be installed to monitor in real time and issue an alarm if smoke is detected.

[0087] Please refer to the following: Figure 6 This is a flowchart of step S801 in the first embodiment of the present invention. Step S801, acquiring the image to be identified, specifically includes the following steps.

[0088] Step S8011: Obtain real-time video of the area where smoke needs to be monitored. Specifically, build a smoke video image monitoring system, utilize the trained smoke segmentation model described above, and load it into the corresponding device to retrieve real-time images from the camera.

[0089] Step S8012: Extract the image to be recognized from the real-time video. Specifically, the real-time video is captured by taking screenshots at preset time intervals to obtain the image to be recognized.

[0090] Step S8013 involves converting the format of the image to be identified to obtain the image to be identified. Specifically, the captured image is processed frame by frame to convert it into the required image format. Then, it is input frame by frame into the trained smoke segmentation model for detection and segmentation.

[0091] Please refer to the following: Figure 7 This is a flowchart of the smoke alarm method in the first embodiment of the present invention. The smoke alarm method provided in this embodiment specifically includes the following steps.

[0092] Step S1001: Based on the smoke segmentation results obtained by the above image smoke segmentation method, determine whether there is smoke in the area where smoke needs to be monitored.

[0093] Step S1002: When smoke is detected in the area where smoke needs to be monitored, an alarm is triggered. Specifically, when smoke is detected, a signal is sent to the corresponding alarm device to issue an alarm and notify relevant personnel.

[0094] This invention analyzes the results of a smoke segmentation model built on a neural network to accurately determine whether smoke exists in the area requiring monitoring. If smoke is present, an alarm is triggered. Timely and accurate feedback on smoke conditions helps in the prompt extinguishing of fires.

[0095] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program causes the processor to perform the steps of any of the methods described above. Specifically, the program may be stored in a non-volatile computer-readable storage medium. When executed, the program may include the processes described in the embodiments of the methods above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0096] Please refer to the following: Figure 8 This is a schematic diagram of the internal structure of a computer device in the first embodiment of the present invention. The computer device 900 includes a memory 910 and a processor 920. The memory 910 stores a computer program. When the computer program is executed by the processor, the processor 920 performs the steps of any of the above-described methods.

[0097] The computer device 900 also includes a processor 920, a memory 910, and a network interface 940 connected via a system bus 930. The memory 910 includes a non-volatile storage medium and internal memory. The non-volatile storage medium of the computer device 900 stores an operating system and may also store a computer program. When executed by the processor 920, this computer program enables the processor 920 to implement a training method for an image smoke segmentation model. The internal memory 910 may also store a computer program, which, when executed by the processor, enables the processor to implement a training method for an image smoke segmentation model.

[0098] The memory 910 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 910 can be an internal storage unit of the computer device 900, such as the hard disk of the computer device 900. In other embodiments, the memory 910 can be an external storage device of the computer device 900, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 900. Furthermore, the memory 910 can include both internal and external storage units of the computer device 900. The memory 910 can be used not only to store application software and various types of data installed on the computer device 900, such as computer programs for training methods of image smoke segmentation models, but also to temporarily store data that has been output or will be output, such as data generated during the execution of the image smoke segmentation model training method. In some feasible embodiments, the processor 920 may be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0099] Specifically, the processor 920 executes a computer program for training an image smoke segmentation model to control the computer device 900 to implement the training method for the image smoke segmentation model.

[0100] Furthermore, the computer device 900 may also include a system bus 930, which may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0101] Specifically, the computer device 900 may also include a network interface 940. The network interface may optionally include a wired network interface and / or a wireless network interface (such as a Wi-Fi network interface, a Bluetooth network interface, etc.), which is typically used to establish communication connections between the computer device 900 and other devices, such as a communication connection between the computer device 900 and a waveform display device.

[0102] In other feasible embodiments, the computer device 900 may also include a display component (not shown). The display component may be an LED (Light Emitting Diode) display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display component may also be appropriately referred to as a display device or display unit, used to display information processed in the computer device 900 and to display a visual user interface.

[0103] Figure 8 Only a computer device 900 with components 910-940 and a training method for implementing an image smoke segmentation model is shown. Those skilled in the art will understand that... Figure 8 The structure shown does not constitute a limitation on the computer device 900, and may include fewer or more components than shown, or combine certain components, or have different component arrangements. Since the computer device 900 employs all the technical solutions of the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0104] Furthermore, the method according to the present invention can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the above-described method of the present invention.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image smoke segmentation model, characterized in that, The model includes a backbone network, a dense multi-scale context module, a detail enhancement module, and a feature fusion module. The backbone network comprises a first unit, a second unit, a third unit, and a fourth unit. The first unit extracts first feature information from the image to be recognized. The second unit receives the first feature information, obtains second feature information based on it, and sends the second feature information to the detail enhancement module, which extracts edge information of the smoke from the second feature information. The third unit receives the second feature information, obtains third feature information based on it, and calculates the spatial attention coefficient of the image to be recognized based on the third feature information. The fourth unit receives the third feature information, obtains fourth feature information based on it, and sends the fourth feature information to the dense multi-scale context module, which obtains context information for each sub-region of the image to be recognized from the fourth feature information and sends it to the feature fusion module. The feature fusion module fuses the edge information, the spatial attention coefficient, and the context information to obtain a feature map of the image to be recognized. The detail information enhancement module is a detail information enhancement module embedded with asymmetric pooling, which is used to extract the horizontal and vertical features of the object through the asymmetric pooling window, and combine the residual branch and the grouped convolution branch to fuse and output the edge information of the smoke. The model further includes an upsampling module, which is used to acquire the feature map and then perform convolution and upsampling on the feature map to obtain a smoke segmentation prediction map of the image to be identified.

2. The image smoke segmentation model according to claim 1, characterized in that, The spatial attention coefficient of the image to be recognized is calculated using the following formula: in, The spatial attention coefficient is... For the Sigmoid function, For 1×1 convolution, GAP For global average pooling, GMP For global max pooling, X 1 represents the third feature information.

3. A training method for an image smoke segmentation model, characterized in that, The method includes: Obtain the training set; Set training parameters; The target smoke segmentation model is obtained by training the image smoke segmentation model as described in claim 1 based on the training set and the training parameters.

4. The training method for the image smoke segmentation model according to claim 3, characterized in that, The training parameters include the learning strategy, loss function, initialization method, and activation function. Specifically, setting the training parameters includes: Set the learning strategy to Poly; Set the loss function to the cross-entropy loss function; Set the initialization method to Kaiming; Set the activation function to ReLU.

5. The training method for the image smoke segmentation model according to claim 4, characterized in that, The training parameters also include image size, batch data size, weight decay, momentum, initial learning rate, and number of iterations. Specifically, setting the training parameters includes: Set the image size to 256 pixels × 256 pixels; Set the batch data size to 16; Set the weight decay to 1e-5; Set the momentum to 0.9; Set the initial learning rate to 1e-2; The number of iterations is set to 30,000.

6. The training method for the image smoke segmentation model according to claim 3, characterized in that, The acquisition of the training set specifically includes: Extract the first smoke image from historical smoke videos and retrieve the second smoke image from the database; The first smoke image and the second smoke image are filtered to obtain pre-trained images; The pre-trained images are labeled to obtain training images; The training images are used to form the training set.

7. A method for image smoke segmentation, characterized in that, The method includes: Acquire the image to be recognized; The image to be identified is input into the target smoke segmentation model obtained by the training method according to claim 3 to obtain the smoke segmentation result.

8. The image smoke segmentation method according to claim 7, characterized in that, Obtaining the image to be recognized specifically includes: Obtain real-time video of the area where smoke conditions need to be monitored; Extract the image to be identified from the real-time video; The image to be identified is obtained by converting the format of the image to be identified.

9. A smoke alarm method, characterized in that, The method includes: Based on the smoke segmentation result obtained by the image smoke segmentation method as described in claim 7, determine whether there is smoke in the area where smoke needs to be monitored; An alarm will be triggered when smoke is detected in an area where smoke monitoring is required.

Images