Smoke image semantic segmentation method based on geometric regularization and core semantic optimization

Abstract

This invention relates to the field of image segmentation and proposes a semantic segmentation method for smoke images based on geometric regularization and core semantic optimization. By designing a prototype generator to obtain multi-granularity prototypes, it ensures accurate semantic guidance and alignment of feature levels, establishing a clear semantic bridge that enables the model to utilize category knowledge while maintaining sensitivity to spatial features. A cascaded core semantic optimization module is then designed to extract representative core semantic vectors, efficiently capturing global context while maintaining spatial coherence and structural locality, thereby generating context-rich labels and further enhancing the model's representational capabilities. A segmentation decoder and a centroid-driven regularized weighted sum are also designed to explicitly mitigate spatial attention drift, providing strong geometric supervision, forcing spatial consistency, and improving the localization accuracy of semi-transparent objects. This invention improves the accuracy of semantic segmentation of smoke images.

Description

Technical Field

[0001] This invention relates to the field of image segmentation, and in particular to a semantic segmentation method for smoke images based on geometric regularization and core semantic optimization. Background Technology

[0002] Fire is one of the most frequent natural and social disasters, seriously threatening human life and public safety, and causing extremely serious damage to environmental protection and sustainable development. Real-time monitoring and timely alarm of fire-prone areas are among the most effective means to prevent its large-scale spread. Traditional fire alarm methods use fire detectors, which have the advantages of low cost and high accuracy. However, they generally rely on the concentration and temperature of smoke particles in the air as judgment conditions, and the detectors need to be installed near the ignition point to be effective, which can easily lead to delays in fire alarms. With the continuous improvement of urban "smartness" and "security," video surveillance has become very common in daily life. Therefore, utilizing existing video surveillance resources to achieve intelligent fire monitoring has very high feasibility and application value.

[0003] Currently, fire investigation mainly relies on manual interpretation. However, due to the massive amount of video data and limited human capacity, this method is inefficient. Since fires typically begin with smoldering smoke before flames appear, and smoke is more observable, fire alarms based on smoke composition detection have a better prospect for practical application. Smoke is a non-rigid target. Unlike rigid objects, smoke exhibits three inherent characteristics that fundamentally challenge traditional segmentation paradigms: First, its semi-transparency leads to blurred boundaries and severe mixing with the background; second, its amorphous and dynamically changing shape lacks consistent structural priors; and third, it exhibits significant intra-class heterogeneity and inter-class similarity in color, texture, and density. Therefore, segmentation methods designed for standardized objects often perform poorly when faced with the ethereal and non-rigid characteristics of smoke.

[0004] Therefore, designing a semantic segmentation method for smoke images to avoid the influence of smoke characteristics and improve segmentation accuracy has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, this invention proposes a semantic segmentation method for smoke images based on geometric regularization and core semantic optimization. It designs a prototype generator to obtain multi-granular prototypes, ensuring accurate semantic guidance and alignment with feature levels. The generated prototypes establish a clear semantic bridge between classification and segmentation, enabling the overall segmentation model to utilize category knowledge while maintaining sensitivity to spatial features. A cascaded core semantic optimization module is then designed, extracting representative core semantic vectors through a core semantic aggregation submodule. This efficiently captures global context while maintaining spatial coherence and structural locality, thereby generating context-rich labels crucial for robust core semantic extraction. The semantic optimization submodule further enhances the model's representational capabilities. A segmentation decoder and a weighted sum with centroid-driven regularization are also designed to explicitly mitigate spatial attention drift. By penalizing the deviation between the predicted geometric centroid of the smoke region and the ground truth labeled centroid, strong geometric supervision is provided, anchoring the model's attention to the coherent core of the target, thereby forcing spatial consistency and improving the localization accuracy of semi-transparent objects. This invention improves the accuracy of semantic segmentation of smoke images.

[0006] This invention proposes a semantic segmentation method for smoke images based on geometric regularization and core semantic optimization, comprising: A smoke image dataset is acquired and input into a backbone network to obtain multi-scale features, including low-scale features, medium-scale features and high-scale features. The backbone network is a pre-trained ResNet101 network. The low-scale features are input into the prototype generator to obtain multi-granularity prototypes, which include global category prototypes, regional semantic prototypes, and structural context prototypes. The prototype generator is based on a multi-branch architecture. The multi-scale features and the multi-granularity prototype are input into the cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation sub-module and a semantic optimization sub-module. The core semantic aggregation sub-module is based on an inter-block interactive tokenization mechanism. The core semantic optimization features are input into the segmentation decoder to obtain the final segmentation result. The segmentation decoder also includes centroid-driven regularization.

[0007] In summary, based on the aforementioned semantic segmentation method for smoke images using geometric regularization and core semantic optimization, this invention designs a prototype generator to obtain multi-granular prototypes, ensuring accurate semantic guidance and alignment of feature levels. The generated prototypes establish a clear semantic bridge between classification and segmentation, enabling the overall segmentation model to utilize category knowledge while maintaining sensitivity to spatial features. Furthermore, a cascaded core semantic optimization module is designed, extracting representative core semantic vectors through a core semantic aggregation submodule. This efficiently captures global context while maintaining spatial coherence and structural locality, thereby generating context-rich labels crucial for robust core semantic extraction. The semantic optimization submodule further enhances the model's representational capabilities. Additionally, a segmentation decoder and a weighted sum with centroid-driven regularization are designed to explicitly mitigate spatial attention drift. By penalizing the deviation between the predicted geometric centroid of the smoke region and the ground truth labeled centroid, strong geometric supervision is provided, anchoring the model's attention to the coherent core of the target, thereby forcing spatial consistency and improving the localization accuracy of semi-transparent objects. This invention improves the accuracy of semantic segmentation of smoke images. Specifically, a smoke image dataset is acquired and input into a backbone network to obtain multi-scale features, including low-scale, medium-scale, and high-scale features. The backbone network is a pre-trained ResNet101 network. The low-scale features are then input into a prototype generator to obtain multi-granularity prototypes, including global category prototypes, region semantic prototypes, and structural context prototypes. The prototype generator is based on a multi-branch architecture, ensuring accurate semantic guidance and alignment with feature levels. The generated prototypes establish a clear semantic bridge between classification and segmentation, enabling the overall segmentation model to utilize category knowledge while maintaining sensitivity to spatial features. The multi-scale features and multi-granularity prototypes are then input into a cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation submodule and a language... The semantic optimization submodule, based on an inter-block interactive tokenization mechanism, extracts representative core semantic vectors, efficiently capturing global context while maintaining spatial coherence and structural locality. This generates context-rich tags crucial for robust core semantic extraction. The semantic optimization submodule further enhances the model's representational capabilities. The optimized core semantic features are input into the segmentation decoder to obtain the final segmentation result. Furthermore, a centroid-driven regularized weighted sum explicitly mitigates spatial attention drift. By penalizing the deviation between the predicted geometric centroid and the ground truth centroid of the smoke region, robust geometric supervision is provided, anchoring the model's attention to the coherent core of the target. This forces spatial consistency and improves the localization accuracy of semi-transparent objects. Therefore, this invention improves the accuracy of semantic segmentation of smoke images.

[0008] Furthermore, the step of inputting the low-scale features into the prototype generator to obtain multi-granularity prototypes specifically includes: The prototype generator includes a classifier head and a multilayer perceptron. The prototype generator is based on a multi-branch architecture. The classifier head compresses low-scale features through 1×1 convolutions and then passes them through three consecutive 3×3 convolutional blocks to obtain the output features of the classifier head. Each pair of 3×3 convolutional blocks has a 2×2 average pooling block between them. The multilayer perceptron obtains multi-granularity prototypes based on feature resolution. The multi-granularity prototypes include a global category prototype, a regional semantic prototype, and a structural context prototype. The global category prototype is generated by global aggregation and projection of low-resolution classifier head output features through the multilayer perceptron. The regional semantic prototype is generated by flattening and projection of medium-resolution classifier head output features through the multilayer perceptron. The structural context prototype is generated by transformation of high-resolution classifier head output features. The specific algorithm for the prototype generator is as follows: , in, Represents the global category prototype. Represents the semantic prototype of a region. Represents the structural context prototype. This indicates the output by category. Represents a prototype generator. It represents low-scale features.

[0009] Furthermore, the step of inputting the multi-scale features and the multi-granularity prototype into the cascaded core semantic optimization module to obtain core semantic optimization features specifically includes: The cascaded core semantic optimization module includes a core semantic aggregation submodule and a semantic optimization submodule; The core semantic aggregation submodule performs channel compression on the multi-scale features using 1×1 convolution, and then divides the multi-scale features into multiple blocks and superblocks, with no overlap between each block and superblock; Inter-block interaction tokenization is performed based on the block and superblock to obtain block tokens and superblock tokens. The inter-block interaction tokenization is based on an inter-block interaction tokenization mechanism, including inter-block interaction convolution processing and tokenization processing. Global semantic aggregation is performed on the multi-scale features after channel compression to obtain global labels; The superblock markers are optimized to obtain optimized superblock markers. The specific algorithm for obtaining optimized superblock markers is as follows: , in, This indicates an optimization of the superblock label. This indicates enhanced cross-attention. , , These represent the query features, key features, and value features of cross-attention, respectively. Indicates a superblock marker. Indicates a block marker; Based on the optimized superblock label and global label, the core semantic features are obtained. The specific algorithm for obtaining the core semantic features is as follows: , in, Represents core semantic features, Indicates a global flag; The semantic optimization submodule optimizes the core semantic features based on multi-granularity prototypes to obtain core semantic optimization features. The semantic optimization submodule is based on a dual-path fusion mechanism, which is used to optimize spatial and channel dimensions respectively.

[0010] Furthermore, the step of performing inter-block interaction tokenization based on the block and superblock to obtain block tags and superblock tags specifically includes: Inter-block interactive convolution processing is performed on blocks and superblocks to obtain a new block set. The inter-block interactive convolution processing is based on learnable structured convolution kernels. The specific algorithm for the spatial location of the new block set is as follows: , in, Indicates the spatial location of the new block set. Indicates the total number of blocks. Represents ordinal numbers. Indicates the position in the sampling grid. A predefined sampling grid is used to define the receptive field and expansion pattern of the interaction. This represents the convolution kernel corresponding to each block. Represents the source block set, Indicates spatial location; The new block set is tokenized using a depthwise convolutional kernel to obtain block tags and superblock tags. The specific algorithm for this tokenization process is as follows: , , in, Indicates block marker, Indicates a superblock marker. Represents the new block set, This represents a depthwise separable convolution operation, where the kernel size for performing a depthwise separable convolution operation on a block is... The kernel size for depthwise separable convolution operations on superblocks is , H and W represent the height and width of the feature, and r represents the segmentation of the feature into... Non-overlapping blocks.

[0011] Furthermore, the step of performing global semantic aggregation on the multi-scale features after channel compression to obtain global labels specifically includes: Based on the learnable spatial weight tensor and the channel-compressed multi-scale features, spatial weighted features are obtained. The specific algorithm for obtaining spatial weighted features is as follows: , in, Represents spatial weighted features. This represents the multi-scale features after channel compression. Represents the learnable space weight tensor. This indicates element-wise multiplication; A temporary global label is generated based on the spatial weighted features. The specific algorithm for generating the temporary global label is as follows: , in, Indicates a temporary global marker. , These represent the feature height and feature width, respectively. , These represent the summation ordinal numbers of the feature height and feature width, respectively. The temporary global label is projected using a multilayer perceptron to obtain a global label. The specific algorithm for obtaining the global label is as follows: , in, Indicates a global flag. This represents a multilayer perceptron.

[0012] Furthermore, the semantic optimization submodule optimizes the core semantic features based on multi-granularity prototypes to obtain core semantic optimized features, specifically including: The semantic optimization submodule is based on a dual-path fusion mechanism. It fuses multi-granularity prototypes with core semantic features according to a similarity-gated mixing mechanism to obtain core semantic fusion features. The specific algorithm for the core semantic fusion features is as follows: , in, Indicates core semantic fusion features, Represents cosine similarity. Represents core semantic features at different scales. This represents multi-granularity prototypes at different scales. This indicates element-wise multiplication; The core semantic fusion features are dimensionally expanded using a multilayer perceptron and then segmented into complementary components. These complementary components include spatial and channel components. The specific algorithm for segmenting into complementary components is as follows: , in, Represents spatial components, Indicates channel components, This indicates a segmentation process. Represents a multilayer perceptron; The similarity maps of the spatial components and channel components are calculated separately to obtain the core semantic optimization features. The specific algorithm for obtaining the core semantic optimization features is as follows: , , , in, Represents the spatial component similarity map. This represents the activation function. This represents the core semantic features after reshaping. T Represents the transpose matrix. This represents the core semantic optimization features.

[0013] Furthermore, the step of inputting the core semantic optimization features into the segmentation decoder to obtain the final segmentation result specifically includes: The segmentation decoder includes two upsampled convolutional submodules, one convolutional submodule, and a sigmoid nonlinear activation. The upsampled convolutional submodule includes a 2x bilinear interpolation upsampling, a 3×3 convolution, a BN, and a ReLU operation. The convolutional submodule includes a 3×3 convolution, a BN, and a ReLU operation. The segmentation decoder is followed by centroid-driven regularization, and the specific algorithm for centroid-driven regularization is as follows: , , , , , , , in, The x-coordinate representing the predicted centroid position, The ordinate representing the predicted centroid position, , express , Represents smoke pixels row index, Represents smoke pixels column index, Represents pixels The predicted value, Represents pixels The true value, Indicates an indicator function, This represents the centroid loss function.

[0014] This invention proposes a semantic segmentation system for smoke images based on geometric regularization and core semantic optimization, comprising: The backbone module is used to acquire smoke image datasets and input them into the backbone network to obtain multi-scale features, including low-scale features, medium-scale features and high-scale features. The backbone network is a pre-trained ResNet101 network. The prototype generation module is used to input the low-scale features into the prototype generator to obtain multi-granularity prototypes, which include global category prototypes, regional semantic prototypes and structural context prototypes. The prototype generator is based on a multi-branch architecture. The optimization module is used to input the multi-scale features and the multi-granularity prototype into the cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation sub-module and a semantic optimization sub-module. The core semantic aggregation sub-module is based on an inter-block interaction tokenization mechanism. The segmentation decoding module is used to input the core semantic optimization features into the segmentation decoder to obtain the final segmentation result. The segmentation decoder also includes centroid-driven regularization.

[0015] The present invention also provides a storage medium that stores one or more programs, which, when executed by a processor, implement the above-described semantic segmentation method for smoke images based on geometric regularization and core semantic optimization.

[0016] The present invention also provides a computer device, the computer device including a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the smoke image semantic segmentation method based on geometric regularization and core semantic optimization as described above. Attached Figure Description

[0017] Figure 1 This is a flowchart of the smoke image semantic segmentation method based on geometric regularization and core semantic optimization proposed in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the smoke image semantic segmentation system based on geometric regularization and core semantic optimization proposed in the second embodiment of the present invention; Figure 3 This is a network model structure diagram of the smoke image semantic segmentation method based on geometric regularization and core semantic optimization proposed in the first embodiment of the present invention. Figure 4 This is a logical diagram illustrating the global semantic aggregation proposed in the first embodiment of the present invention; Figure 5 This is a schematic diagram of the core semantic aggregation submodule proposed in the first embodiment of the present invention; Figure 6 This is a schematic diagram of the semantic optimization submodule proposed in the first embodiment of the present invention; Figure 7 This is a schematic diagram of the segmentation decoder proposed in the first embodiment of the present invention; Figure 8 This is a flowchart illustrating the logic of inter-block interactive convolution processing and tokenization processing proposed in the first embodiment of the present invention.

[0018] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0020] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Please see Figure 1 The diagram shows a flowchart of the smoke image semantic segmentation method based on geometric regularization and core semantic optimization proposed in the first embodiment of the present invention. This smoke image semantic segmentation method based on geometric regularization and core semantic optimization includes steps S01 to S04, wherein: Step S01: Obtain the smoke image dataset and input it into the backbone network to obtain multi-scale features; It should be noted that the complete network model structure in this embodiment can be found in [link to documentation]. Figure 3 The multi-scale features include low-scale features, medium-scale features and high-scale features, and the backbone network is a pre-trained ResNet101 network.

[0023] Step S02: Input the low-scale features into the prototype generator to obtain multi-granularity prototypes; It should be noted that in this embodiment, the multi-granularity prototype includes a global category prototype, a regional semantic prototype, and a structural context prototype. The prototype generator is based on a multi-branch architecture and includes a classifier head and a multilayer perceptron. The classifier head compresses low-scale features through 1×1 convolutions and then passes them through three consecutive 3×3 convolutional blocks to obtain the output features of the classifier head. Each pair of 3×3 convolutional blocks has a 2×2 average pooling block between them. The multilayer perceptron obtains multi-granularity prototypes based on feature resolution. The multi-granularity prototypes include a global category prototype, a regional semantic prototype, and a structural context prototype. The global category prototype is generated by global aggregation and projection of low-resolution classifier head output features through the multilayer perceptron. The regional semantic prototype is generated by flattening and projection of medium-resolution classifier head output features through the multilayer perceptron. The structural context prototype is generated by transformation of high-resolution classifier head output features. The specific algorithm for the prototype generator is as follows: , in, Represents the global category prototype. Represents the semantic prototype of a region. Represents the structural context prototype. This indicates the output by category. Represents a prototype generator. It represents low-scale features.

[0024] Step S03: Input the multi-scale features and multi-granularity prototypes into the cascaded core semantic optimization module to obtain core semantic optimization features; It should be noted that in this embodiment, the cascaded core semantic optimization module (CSR) includes a core semantic aggregation submodule (CSA) and a semantic optimization submodule (SR). The core semantic aggregation submodule is based on an inter-block interaction tokenization mechanism. The cascaded core semantic optimization module includes a core semantic aggregation submodule and a semantic optimization submodule. In this embodiment, please refer to the specific structure of the core semantic aggregation submodule. Figure 5 For the specific structure of the semantic optimization submodule, please refer to [link / reference]. Figure 6 ; The core semantic aggregation submodule performs channel compression on the multi-scale features using 1×1 convolution, and then divides the multi-scale features into multiple blocks and superblocks, with no overlap between each block and superblock; Inter-block interaction tokenization is performed based on the block and superblock to obtain block tokens and superblock tokens. The inter-block interaction tokenization is based on an inter-block interaction tokenization mechanism, including inter-block interaction convolution processing and tokenization processing. For the specific logic flow of inter-block interactive convolution processing and tokenization processing in this embodiment, please refer to [link / reference]. Figure 8 ; Global semantic aggregation is performed on the multi-scale features after channel compression to obtain global labels; The superblock markers are optimized to obtain optimized superblock markers. The specific algorithm for obtaining optimized superblock markers is as follows: , in, This indicates an optimization of the superblock label. This indicates enhanced cross-attention. , , These represent the query features, key features, and value features of cross-attention, respectively. Indicates a superblock marker. Indicates a block marker; Based on the optimized superblock label and global label, the core semantic features are obtained. The specific algorithm for obtaining the core semantic features is as follows: , in, Represents core semantic features, Indicates a global flag; The semantic optimization submodule optimizes the core semantic features based on multi-granularity prototypes to obtain core semantic optimization features. The semantic optimization submodule is based on a dual-path fusion mechanism, which is used to optimize spatial and channel dimensions respectively.

[0025] Inter-block interactive convolution processing is performed on blocks and superblocks to obtain a new block set. The inter-block interactive convolution processing is based on learnable structured convolution kernels. The specific algorithm for the spatial location of the new block set is as follows: , in, Indicates the spatial location of the new block set. Indicates the total number of blocks. Represents ordinal numbers. Indicates the position in the sampling grid. A predefined sampling grid is used to define the receptive field and expansion pattern of the interaction. This represents the convolution kernel corresponding to each block. Represents the source block set, Indicates spatial location; The new block set is tokenized using a depthwise convolutional kernel to obtain block tags and superblock tags. The specific algorithm for this tokenization process is as follows: , , in, Indicates block marker, Indicates a superblock marker. Represents the new block set, This represents a depthwise separable convolution operation, where the kernel size for performing a depthwise separable convolution operation on a block is... The kernel size for depthwise separable convolution operations on superblocks is , H and W represent the height and width of the feature, and r represents the segmentation of the feature into... Non-overlapping blocks.

[0026] Based on the learnable spatial weight tensor and the channel-compressed multi-scale features, spatial weighted features are obtained. The specific algorithm for obtaining spatial weighted features is as follows: , in, Represents spatial weighted features. This represents the multi-scale features after channel compression. Represents the learnable space weight tensor. This indicates element-wise multiplication; A temporary global label is generated based on the spatial weighted features. The specific algorithm for generating the temporary global label is as follows: , in, Indicates a temporary global marker. , These represent the feature height and feature width, respectively. , These represent the summation ordinal numbers of the feature height and feature width, respectively. The temporary global label is projected using a multilayer perceptron to obtain a global label. The specific algorithm for obtaining the global label is as follows: , in, Indicates a global flag. This represents a multilayer perceptron.

[0027] For the specific logical flow of global semantic aggregation in this embodiment, please refer to [link / reference needed]. Figure 4 The semantic optimization submodule is based on a dual-path fusion mechanism. It fuses multi-granularity prototypes with core semantic features according to a similarity-gated mixing mechanism to obtain core semantic fusion features. The specific algorithm for the core semantic fusion features is as follows: , in, Indicates core semantic fusion features, Represents cosine similarity. Represents core semantic features at different scales. This represents multi-granularity prototypes at different scales. This indicates element-wise multiplication; The core semantic fusion features are dimensionally expanded using a multilayer perceptron and then segmented into complementary components. These complementary components include spatial and channel components. The specific algorithm for segmenting into complementary components is as follows: , in, Represents spatial components, Indicates channel components, This indicates a segmentation process. Represents a multilayer perceptron; The similarity maps of the spatial components and channel components are calculated separately to obtain the core semantic optimization features. The specific algorithm for obtaining the core semantic optimization features is as follows: , , , in, Represents the spatial component similarity map. This represents the activation function. This represents the core semantic features after reshaping. T Represents the transpose matrix. This represents the core semantic optimization features.

[0028] Step S04: Input the core semantic optimization features into the segmentation decoder to obtain the final segmentation result; It should be noted that, in this embodiment, the specific structure of the segmentation decoder is described in the following reference: Figure 7 The segmentation decoder is further followed by centroid-driven regularization. The segmentation decoder includes two upsampling convolutional sub-modules, one convolutional sub-module and one sigmoid nonlinear activation. The upsampling convolutional sub-module includes a 2x bilinear interpolation upsampling, a 3×3 convolution, a BN and a ReLU operation. The convolutional sub-module includes a 3×3 convolution, a BN and a ReLU operation. The segmentation decoder is followed by centroid-driven regularization, and the specific algorithm for centroid-driven regularization is as follows: , , , , , , , in, The x-coordinate representing the predicted centroid position, The ordinate representing the predicted centroid position, , express , Represents smoke pixels row index, Represents smoke pixels column index, Represents pixels The predicted value, Represents pixels The true value, Indicates an indicator function, This represents the centroid loss function.

[0029] In summary, based on the aforementioned semantic segmentation method for smoke images using geometric regularization and core semantic optimization, this invention designs a prototype generator to obtain multi-granular prototypes, ensuring accurate semantic guidance and alignment of feature levels. The generated prototypes establish a clear semantic bridge between classification and segmentation, enabling the overall segmentation model to utilize category knowledge while maintaining sensitivity to spatial features. Furthermore, a cascaded core semantic optimization module is designed, extracting representative core semantic vectors through a core semantic aggregation submodule. This efficiently captures global context while maintaining spatial coherence and structural locality, thereby generating context-rich labels crucial for robust core semantic extraction. The semantic optimization submodule further enhances the model's representational capabilities. Additionally, a segmentation decoder and a weighted sum with centroid-driven regularization are designed to explicitly mitigate spatial attention drift. By penalizing the deviation between the predicted geometric centroid of the smoke region and the ground truth labeled centroid, strong geometric supervision is provided, anchoring the model's attention to the coherent core of the target, thereby forcing spatial consistency and improving the localization accuracy of semi-transparent objects. This invention improves the accuracy of semantic segmentation of smoke images. Specifically, a smoke image dataset is acquired and input into a backbone network to obtain multi-scale features, including low-scale, medium-scale, and high-scale features. The backbone network is a pre-trained ResNet101 network. The low-scale features are then input into a prototype generator to obtain multi-granularity prototypes, including global category prototypes, region semantic prototypes, and structural context prototypes. The prototype generator is based on a multi-branch architecture, ensuring accurate semantic guidance and alignment with feature levels. The generated prototypes establish a clear semantic bridge between classification and segmentation, enabling the overall segmentation model to utilize category knowledge while maintaining sensitivity to spatial features. The multi-scale features and multi-granularity prototypes are then input into a cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation submodule and a language... The semantic optimization submodule, based on an inter-block interactive tokenization mechanism, extracts representative core semantic vectors, efficiently capturing global context while maintaining spatial coherence and structural locality. This generates context-rich tags crucial for robust core semantic extraction. The semantic optimization submodule further enhances the model's representational capabilities. The optimized core semantic features are input into the segmentation decoder to obtain the final segmentation result. Furthermore, a centroid-driven regularized weighted sum explicitly mitigates spatial attention drift. By penalizing the deviation between the predicted geometric centroid and the ground truth centroid of the smoke region, robust geometric supervision is provided, anchoring the model's attention to the coherent core of the target. This forces spatial consistency and improves the localization accuracy of semi-transparent objects. Therefore, this invention improves the accuracy of semantic segmentation of smoke images.

[0030] Please see Figure 2 The figure shows a schematic diagram of the structure of the smoke image semantic segmentation system based on geometric regularization and core semantic optimization proposed in the second embodiment of the present invention. The system includes: Backbone module 10 is used to acquire smoke image dataset and input it into backbone network to obtain multi-scale features, including low-scale features, medium-scale features and high-scale features, and the backbone network is a pre-trained ResNet101 network. The prototype generation module 20 is used to input the low-scale features into the prototype generator to obtain multi-granularity prototypes, which include global category prototypes, regional semantic prototypes and structural context prototypes. The prototype generator is based on a multi-branch architecture. Optimization module 30 is used to input the multi-scale features and the multi-granularity prototype into the cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation sub-module and a semantic optimization sub-module. The core semantic aggregation sub-module is based on an inter-block interaction tokenization mechanism. The segmentation decoding module 40 is used to input the core semantic optimization features into the segmentation decoder to obtain the final segmentation result. The segmentation decoder also includes centroid-driven regularization.

[0031] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the aforementioned semantic segmentation method for smoke images based on geometric regularization and core semantic optimization.

[0032] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned smoke image semantic segmentation method based on geometric regularization and core semantic optimization.

[0033] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0034] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0035] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0036] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0037] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A semantic segmentation method for smoke images based on geometric regularization and core semantic optimization, characterized in that, include: A smoke image dataset is acquired and input into a backbone network to obtain multi-scale features, including low-scale features, medium-scale features and high-scale features. The backbone network is a pre-trained ResNet101 network. The low-scale features are input into the prototype generator to obtain multi-granularity prototypes, which include global category prototypes, regional semantic prototypes, and structural context prototypes. The prototype generator is based on a multi-branch architecture. The multi-scale features and the multi-granularity prototype are input into the cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation sub-module and a semantic optimization sub-module. The core semantic aggregation sub-module is based on an inter-block interactive tokenization mechanism. The core semantic optimization features are input into the segmentation decoder to obtain the final segmentation result. The segmentation decoder also includes centroid-driven regularization.

2. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 1, characterized in that, The step of inputting the low-scale features into the prototype generator to obtain a multi-granularity prototype specifically includes: The prototype generator includes a classifier head and a multilayer perceptron. The prototype generator is based on a multi-branch architecture. The classifier head compresses low-scale features through 1×1 convolutions and then passes them through three consecutive 3×3 convolutional blocks to obtain the output features of the classifier head. Each pair of 3×3 convolutional blocks has a 2×2 average pooling block between them. The multilayer perceptron obtains multi-granularity prototypes based on feature resolution. The multi-granularity prototypes include a global category prototype, a regional semantic prototype, and a structural context prototype. The global category prototype is generated by global aggregation and projection of low-resolution classifier head output features through the multilayer perceptron. The regional semantic prototype is generated by flattening and projection of medium-resolution classifier head output features through the multilayer perceptron. The structural context prototype is generated by transformation of high-resolution classifier head output features. The specific algorithm for the prototype generator is as follows: ， in, Represents the global category prototype. Represents the semantic prototype of a region. Represents the structural context prototype. This indicates the output by category. Represents a prototype generator. It represents low-scale features.

3. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 1, characterized in that, The step of inputting the multi-scale features and the multi-granularity prototype into the cascaded core semantic optimization module to obtain core semantic optimization features specifically includes: The cascaded core semantic optimization module includes a core semantic aggregation submodule and a semantic optimization submodule; The core semantic aggregation submodule performs channel compression on the multi-scale features using 1×1 convolution, and then divides the multi-scale features into multiple blocks and superblocks, with no overlap between each block and superblock; Inter-block interaction tokenization is performed based on the block and superblock to obtain block tokens and superblock tokens. The inter-block interaction tokenization is based on an inter-block interaction tokenization mechanism, including inter-block interaction convolution processing and tokenization processing. Global semantic aggregation is performed on the multi-scale features after channel compression to obtain global labels; The superblock markers are optimized to obtain optimized superblock markers. The specific algorithm for obtaining optimized superblock markers is as follows: ， in, This indicates an optimization of the superblock label. This indicates enhanced cross-attention. , , These represent the query features, key features, and value features of cross-attention, respectively. Indicates a superblock marker. Indicates a block marker; Based on the optimized superblock label and global label, the core semantic features are obtained. The specific algorithm for obtaining the core semantic features is as follows: ， in, Represents core semantic features, Indicates a global flag; The semantic optimization submodule optimizes the core semantic features based on multi-granularity prototypes to obtain core semantic optimization features. The semantic optimization submodule is based on a dual-path fusion mechanism, which is used to optimize spatial and channel dimensions respectively.

4. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 3, characterized in that, The step of performing inter-block interaction tokenization based on the block and superblock to obtain block tags and superblock tags specifically includes: Inter-block interactive convolution processing is performed on blocks and superblocks to obtain a new block set. The inter-block interactive convolution processing is based on learnable structured convolution kernels. The specific algorithm for the spatial location of the new block set is as follows: ， in, Indicates the spatial location of the new block set. Indicates the total number of blocks. Represents ordinal numbers. Indicates the position in the sampling grid. This represents a predefined sampling grid, which is used to define the receptive field and expansion pattern of the interaction. This represents the convolution kernel corresponding to each block. Represents the source block set, Indicates spatial location; The new block set is tokenized using a depthwise convolutional kernel to obtain block tags and superblock tags. The specific algorithm for this tokenization process is as follows: ， ， in, Indicates block marker, Indicates a superblock marker. Represents the new block set, This represents a depthwise separable convolution operation, where the kernel size for performing a depthwise separable convolution operation on a block is... The kernel size for depthwise separable convolution operations on superblocks is , H and W represent the height and width of the feature, and r represents the segmentation of the feature into... Non-overlapping blocks.

5. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 3, characterized in that, The step of performing global semantic aggregation on the multi-scale features after channel compression to obtain global labels specifically includes: Based on the learnable spatial weight tensor and the channel-compressed multi-scale features, spatial weighted features are obtained. The specific algorithm for obtaining spatial weighted features is as follows: ， in, Represents spatial weighted features. This represents the multi-scale features after channel compression. Represents the learnable space weight tensor. This indicates element-wise multiplication; A temporary global label is generated based on the spatial weighted features. The specific algorithm for generating the temporary global label is as follows: ， in, Indicates a temporary global marker. , These represent the feature height and feature width, respectively. , These represent the summation ordinal numbers of the feature height and feature width, respectively. The temporary global label is projected using a multilayer perceptron to obtain a global label. The specific algorithm for obtaining the global label is as follows: ， in, Indicates a global flag. This represents a multilayer perceptron.

6. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 3, characterized in that, The semantic optimization submodule optimizes the core semantic features based on multi-granularity prototypes to obtain core semantic optimized features, specifically including: The semantic optimization submodule is based on a dual-path fusion mechanism. It fuses multi-granularity prototypes with core semantic features according to a similarity-gated mixing mechanism to obtain core semantic fusion features. The specific algorithm for the core semantic fusion features is as follows: ， in, Indicates core semantic fusion features, Represents cosine similarity. Represents core semantic features at different scales. This represents multi-granularity prototypes at different scales. This indicates element-wise multiplication; The core semantic fusion features are dimensionally expanded using a multilayer perceptron and then segmented into complementary components. These complementary components include spatial and channel components. The specific algorithm for segmenting into complementary components is as follows: ， in, Represents spatial components, Indicates channel components, This indicates a segmentation process. This represents a multilayer perceptron; The similarity maps of the spatial components and channel components are calculated separately to obtain the core semantic optimization features. The specific algorithm for obtaining the core semantic optimization features is as follows: ， ， ， in, Represents the spatial component similarity map. This represents the activation function. This represents the core semantic features after reshaping. T Represents the transpose matrix. This represents the core semantic optimization features.

7. The smoke image semantic segmentation method based on geometric regularization and core semantic optimization according to claim 1, characterized in that, The step of inputting the core semantic optimization features into the segmentation decoder to obtain the final segmentation result specifically includes: The segmentation decoder includes two upsampled convolutional submodules, one convolutional submodule, and a sigmoid nonlinear activation. The upsampled convolutional submodule includes a 2x bilinear interpolation upsampling, a 3×3 convolution, a BN, and a ReLU operation. The convolutional submodule includes a 3×3 convolution, a BN, and a ReLU operation. The segmentation decoder is followed by centroid-driven regularization, and the specific algorithm for centroid-driven regularization is as follows: ， ， ， ， ， ， ， in, The x-coordinate representing the predicted centroid position, The ordinate representing the predicted centroid position, , express , Represents smoke pixels row index, Represents smoke pixels column index, Represents pixels The predicted value, Represents pixels The true value, Indicates an indicator function, This represents the centroid loss function.

8. A semantic segmentation system for smoke images based on geometric regularization and core semantic optimization, characterized in that, include: The backbone module is used to acquire smoke image datasets and input them into the backbone network to obtain multi-scale features, including low-scale features, medium-scale features and high-scale features. The backbone network is a pre-trained ResNet101 network. The prototype generation module is used to input the low-scale features into the prototype generator to obtain multi-granularity prototypes, which include global category prototypes, regional semantic prototypes and structural context prototypes. The prototype generator is based on a multi-branch architecture. The optimization module is used to input the multi-scale features and the multi-granularity prototype into the cascaded core semantic optimization module to obtain core semantic optimization features. The cascaded core semantic optimization module includes a core semantic aggregation sub-module and a semantic optimization sub-module. The core semantic aggregation sub-module is based on an inter-block interaction tokenization mechanism. The segmentation decoding module is used to input the core semantic optimization features into the segmentation decoder to obtain the final segmentation result. The segmentation decoder also includes centroid-driven regularization.

9. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the smoke image semantic segmentation method based on geometric regularization and core semantic optimization as described in any one of claims 1-7.

10. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the smoke image semantic segmentation method based on geometric regularization and core semantic optimization as described in any one of claims 1-7.

Images