Smoke and fire detection sample image generation method and device, storage medium and product

Abstract

The invention discloses a smoke and fire detection sample image generation method and device, a storage medium and a product, and relates to the technical field of training sample generation, and the smoke and fire detection sample image generation method comprises the steps: obtaining a smoke and fire-free scene image; performing scene understanding analysis on the smokeless scene image, and determining a combustible area and an expected smokeless type which meet preset conditions; according to the combustible region and the expected smoke and fire type, determining a synthesis region for synthesizing a smoke and fire target, and generating a corresponding region mask; and based on the area mask, generating a smoke and fire target in the synthesis area in the smoke and fire-free scene image, and obtaining a smoke and fire detection sample image. According to the method, scene understanding analysis is adopted to determine the combustible area and the expected smoke and fire type, and reasonable generation of the smoke and fire target in the sample image is realized, so that generation of the high-quality smoke and fire sample image is realized.

Description

Technical Field

[0001] This application relates to the field of training sample generation technology, and in particular to methods, devices, storage media and products for generating smoke and fire detection sample images. Background Technology

[0002] Deep learning-based smoke detection technology is a current research hotspot in the fields of security and early warning. However, the performance of such data-driven methods heavily relies on large-scale, high-quality labeled datasets for model training. In practical applications, obtaining high-quality smoke sample data faces fundamental challenges: First, real fire data is sporadic and dangerous, resulting in an extreme scarcity of positive samples (smoke data), especially data from critical stages such as the initial traces of smoke and small flames. Second, existing data augmentation methods have significant shortcomings: traditional image enhancement techniques (such as rotation and cropping) cannot generate new and diverse smoke patterns; while early data synthesis methods (such as GANs) often produce smoke images that blend awkwardly with the background, resulting in distorted lighting effects and poor realism and scene adaptability of the synthesized data, leading to low-quality training samples. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device, storage medium and product for generating smoke and fire detection sample images, aiming to solve the technical problem of low training sample quality.

[0004] To achieve the above objectives, this application proposes a method for generating smoke and fire detection sample images, the method comprising: Acquire images of a smokeless fire scene; Scene understanding analysis is performed on the smokeless scene images to determine the combustible areas and expected smoke and fire types that meet the preset conditions; Based on the combustible area and the expected pyrotechnic type, determine the synthesis area of ​​the synthetic pyrotechnic target and generate the corresponding area mask; Based on the region mask, a smoke and fire target is generated within the synthetic region in the smoke-free scene image to obtain a smoke and fire detection sample image.

[0005] In one embodiment, the step of performing scene understanding analysis on the smokeless fire scene image includes: The smokeless fire scene image is analyzed and identified to obtain the object material information in the smokeless fire scene image. Based on the object material information, the scene logic in the smokeless fire scene image is determined through a preset multimodal large model. The system queries a pre-defined fireworks feature knowledge base to determine the expected fireworks type that matches the scene logic. The fireworks feature knowledge base is obtained by analyzing pre-defined literature samples and pre-defined real fireworks image samples, parametrically representing the visual and physical characteristics of different combustibles at different combustion stages, and storing them as fireworks types for the corresponding scenes in a pre-defined structured database.

[0006] In one embodiment, the step of performing scene understanding analysis on the smokeless fire scene image includes: The semantic analysis of each item in the smokeless scene image is performed by a preset semantic segmentation model to determine the pixel-level semantic information in the smokeless scene image; By using a pre-set multimodal large model to perform scene understanding on the pixel-level semantic information, the main scene in the smokeless scene image is determined. Based on a preset set of rationality rules, logical reasoning is performed on the pixel-level semantic information and the main scene to determine the flammable area in the smokeless scene image.

[0007] In one embodiment, the step of determining the flammable area in the smokeless scene image by performing logical reasoning on the pixel-level semantic information and the main scene based on a preset set of rationality rules includes: Based on preset flammability rules, flammability scores are assigned to each object category in the pixel-level semantic information to obtain a first flammability score. Based on preset position rules, the candidate smoke and flame starting positions in the main scene are determined according to the physical characteristics of smoke and flame in the pixel-level semantic information. The starting position of smoke is limited to the top area of ​​the candidate combustible object, and the starting position of flame is limited to the entire area of ​​the candidate combustible object. Based on preset exclusion rules, the locations in the main scenarios where fire cannot start are identified as the first exclusion locations; Combustible areas are determined by combining the first flammability score, the candidate smoke initiation position, and the first exclusion position to identify the flammable areas in the smokeless scene image.

[0008] In one embodiment, the step of determining the composite region of the composite pyrotechnic target based on the combustible region and the expected pyrotechnic type, and generating a corresponding region mask, includes: The initial synthesis area is determined based on the combustible area and the expected pyrotechnic type; The initial mask corresponding to the initial synthesis region is subjected to edge optimization processing; The initial synthesis region is expanded. Based on the optimized initial mask and the expanded initial composite region, the composite region of the composite pyrotechnic target is determined, and the region mask corresponding to the composite region is generated.

[0009] In one embodiment, the step of generating a smoke target within the synthetic region in the smokeless scene image based on the region mask to obtain a smoke detection sample image includes: Based on the object category in the combustible area, query the fireworks feature knowledge base to obtain the fireworks visual feature parameters of the fireworks type corresponding to the object category; The visual feature parameters of the fireworks are converted into synthetic parameters; The region mask, the synthesis parameters, and the smokeless scene image are input into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image, thus obtaining a smoke detection sample image.

[0010] In one embodiment, the step of inputting the region mask, the synthesis parameters, and the smokeless scene image into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image to obtain a smoke detection sample image includes: The region mask is used as a spatial guidance input to a preset diffusion model; The synthesis parameters are converted into text prompts and used as the content conditions for generation, which are then input into the diffusion model. The diffusion model performs a local redrawing operation within the area specified by the region mask to generate a pyrotechnic target that meets the preset requirements. The image of the generated fireworks target is optimized to obtain a fireworks detection sample image.

[0011] In addition, to achieve the above objectives, this application also proposes a fireworks detection sample image generation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fireworks detection sample image generation method described above.

[0012] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the fireworks detection sample image generation method described above.

[0013] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the fireworks detection sample image generation method described above.

[0014] One or more technical solutions proposed in this application have at least the following technical effects: In contrast to related technologies, traditional image enhancement techniques (such as rotation and cropping) cannot generate new and diverse smoke and fire patterns; while early data synthesis methods (such as GANs) often produce smoke and fire images that blend awkwardly with the background and have distorted lighting effects, resulting in poor realism and scene adaptability of the synthesized data and low training sample quality. This application addresses this by acquiring smokeless scene images; performing scene understanding analysis on the smokeless scene images to determine combustible areas and expected smoke and fire types that meet preset conditions; determining the synthesis region of the synthesized smoke and fire target based on the combustible areas and the expected smoke and fire types, and generating a corresponding region mask; and generating smoke and fire targets within the synthesis region of the smokeless scene images based on the region mask, thus obtaining smoke and fire detection sample images. It is understood that this application employs scene understanding analysis to determine combustible areas and expected smoke and fire types, enabling the rational generation of smoke and fire targets in the sample images, thereby achieving the generation of high-quality smoke and fire sample images. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an embodiment of the method for generating sample images for fireworks detection in this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the method for generating sample images for fireworks detection in this application. Figure 3 This is a schematic diagram of the hardware operating environment involved in the method for generating smoke detection sample images in this application embodiment.

[0018] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0020] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0021] The main solution in this application's embodiments is: Acquire images of a smokeless fire scene; Scene understanding analysis is performed on the smokeless scene images to determine the combustible areas and expected smoke and fire types that meet the preset conditions; Based on the combustible area and the expected pyrotechnic type, determine the synthesis area of ​​the synthetic pyrotechnic target and generate the corresponding area mask; Based on the region mask, a smoke and fire target is generated within the synthetic region in the smoke-free scene image to obtain a smoke and fire detection sample image.

[0022] In this embodiment, the application uses a fireworks detection sample image generation device as the execution subject. For ease of description, it will be referred to as "device" in detail below.

[0023] Because existing technologies cannot generate new and diverse fireworks patterns, and early data synthesis methods (such as GANs) often produce fireworks images that blend awkwardly with the background and have distorted lighting effects, the realism and scene adaptability of the synthesized data are poor, resulting in low quality training samples.

[0024] This application provides a solution that employs scene understanding analysis to determine combustible areas and expected pyrotechnic types, thereby enabling the rational generation of pyrotechnic targets in sample images and achieving the generation of high-quality pyrotechnic sample images.

[0025] Based on this, embodiments of this application provide a method for generating smoke detection sample images, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method for generating sample images for fireworks detection according to this application.

[0026] In this embodiment, the method for generating smoke detection sample images includes steps S10 to S40: Step S10: Obtain an image of a smokeless fire scene; It should be noted that smokeless scene images refer to original background images that do not contain any flames or smoke phenomena, including but not limited to still images or video frames of indoor environments, outdoor landscapes, or industrial scenes. These images are usually derived from surveillance cameras or public datasets.

[0027] Understandably, this step provides the basic input for subsequent compositing operations, ensuring that the compositing process begins with an undisturbed blank scene, thereby guaranteeing that the generated fireworks samples have a clear background context and supporting data augmentation in diverse environments.

[0028] Step S20: Perform scene understanding analysis on the smokeless scene image to determine the combustible area and expected smoke type that meet the preset conditions; It should be noted that scene understanding analysis refers to the process of parsing and reasoning about image content using computer vision techniques and artificial intelligence models. This includes, but is not limited to, using semantic segmentation models to extract pixel-level semantic information and utilizing multimodal large models to identify spatial relationships and scene logic between objects. Preset conditions refer to predefined rule sets, such as flammability rules, location rules, and exclusion rules, used to assess the ignition probability of image regions. A flammable area refers to an object or region in the image identified as having flammable properties, such as an object made of wood, fabric, or plastic. Expected smoke and fire types refer to possible smoke and fire categories derived from scene analysis results and a smoke and fire feature knowledge base, such as thin white smoke from initial smoldering or orange-yellow flames from open flames.

[0029] Understandably, this step ensures that the location and type of the synthesized fireworks conform to physical laws and scene context through automated scene analysis and logical reasoning, thereby improving the rationality and realism of the synthesized samples and reducing logical errors in subsequent model training.

[0030] Step S30: Determine the composite region of the composite pyrotechnic target based on the combustible region and the expected pyrotechnic type, and generate the corresponding region mask; It should be noted that a synthetic pyrotechnic target refers to a flame or smoke entity planned to be generated in an image. A synthetic region refers to a specific range of image pixels calculated based on the combustible area and the expected pyrotechnic type, used to precisely place the pyrotechnic target. A region mask is a binary or grayscale matrix used to mark the position and shape of the synthetic region, including but not limited to masks optimized through edge feathering and expansion operations to support spatial guidance during the generation process.

[0031] Understandably, this step ensures that the pyrotechnic targets blend naturally with the background image by precisely defining the synthesis range and optimizing the mask boundaries, thus avoiding abrupt or distorted synthesis content and improving the overall visual quality and usability of the synthesized image.

[0032] Step S40: Based on the region mask, generate a smoke target within the synthetic region in the smokeless scene image to obtain a smoke detection sample image.

[0033] It should be noted that generating pyrotechnic targets refers to using generative artificial intelligence models, such as diffusion models, to create flame or smoke effects within a specified area that conform to visual and physical characteristics. Pyrotechnic detection samples refer to the final output image data containing synthetic pyrotechnics, along with corresponding annotation information, including bounding boxes or segmentation masks, used to directly train or evaluate the pyrotechnic detection model.

[0034] Understandably, this step achieves high-quality sample output through controllable generation technology, effectively compensating for the scarcity of real data and providing diverse and scenario-based training data for the detection model, thereby enhancing the model's recognition ability and generalization performance in complex environments.

[0035] This embodiment provides a method for generating smoke and fire detection sample images. It employs scene understanding analysis to determine the combustible area and the expected smoke and fire type, thereby achieving the rational generation of smoke and fire targets in the sample image and thus generating high-quality smoke and fire sample images.

[0036] In one feasible implementation, the step of performing scene understanding analysis on the smokeless fire scene image includes: The smokeless fire scene image is analyzed and identified to obtain the object material information in the smokeless fire scene image. Based on the object material information, the scene logic in the smokeless fire scene image is determined through a preset multimodal large model. The system queries a pre-defined fireworks feature knowledge base to determine the expected fireworks type that matches the scene logic. The fireworks feature knowledge base is obtained by analyzing pre-defined literature samples and pre-defined real fireworks image samples, parametrically representing the visual and physical characteristics of different combustibles at different combustion stages, and storing them as fireworks types for the corresponding scenes in a pre-defined structured database.

[0037] It should be noted that object material information refers to the properties of the materials that make up objects in a scene, inferred through image recognition technology, including but not limited to wood, plastic, fabric, and metal. The pre-set multimodal large model refers to a pre-trained artificial intelligence model capable of simultaneously processing and understanding image and text information, such as a model based on CLIP or a similar architecture. Scene logic refers to the semantic relationships and physical laws inferred from image content, such as the electrical fire risk implied by the spatial proximity of computers and sockets in an office scene. The smoke and fire feature knowledge base refers to a structured database whose content originates from the systematic analysis of pre-set literature samples and pre-set real smoke and fire image samples. Pre-set literature samples include, but are not limited to, fire science monographs, research papers, and safety regulations. Pre-set real smoke and fire image samples refer to visual data collected from real fire cases or controlled experiments. Parametric representation refers to converting visual and physical features, such as flame color, smoke transparency, and dynamic shape, into structured data fields that can be queried and accessed by computers. Storing in the pre-set structured database refers to organizing and storing the parametrically represented data in a database such as a relational database or a NoSQL database.

[0038] Understandably, this implementation method provides a solid logical foundation for fireworks synthesis by automatically analyzing the material of objects in the scene and using a large model to understand the scene context; furthermore, by querying a structured feature knowledge base, it ensures that the selected fireworks type is highly matched with the current scene in terms of physical characteristics and visual performance, thereby guaranteeing the rationality and scientific nature of the synthesized data from the source.

[0039] Understandably, the knowledge base's content originates from the analysis and summarization of professional knowledge in the field of fire science. Its core is to parameterize the visual characteristics of combustion phenomena under different conditions and store them in relational databases or easily searchable structured files (such as JSON and XML). Specifically, this includes: Classification System Construction: A classification and labeling system for fires and fire situations will be established, referencing international standards and specifications. Physical Feature Analysis: Scientific literature will be analyzed to extract characteristics such as flame color and temperature produced by different fuels at different stages (complete and incomplete combustion), as well as the composition, color, and optical characteristics of smoke. Visual Feature Analysis: Computer vision technology will be used to analyze real fire videos and images to quantify the dynamic, shape, and diffusion characteristics of flames and smoke. Knowledge Base Structure Storage: The visual characteristics of smoke and flames will be stored in structured data tables, including but not limited to type, color characteristics, and dynamic characteristics. Table 1 shows some sample content.

[0040] Table 1. Structured storage table of knowledge base

[0041] In one feasible implementation, the step of performing scene understanding analysis on the smokeless fire scene image includes: The semantic analysis of each item in the smokeless scene image is performed by a preset semantic segmentation model to determine the pixel-level semantic information in the smokeless scene image; By using a pre-set multimodal large model to perform scene understanding on the pixel-level semantic information, the main scene in the smokeless scene image is determined. Based on a preset set of rationality rules, logical reasoning is performed on the pixel-level semantic information and the main scene to determine the flammable area in the smokeless scene image.

[0042] It should be noted that the preset semantic segmentation model refers to a trained deep learning model capable of assigning a semantic category label to each pixel of the input image. Pixel-level semantic information refers to the densely labeled map output by the semantic segmentation model, which annotates the object category (e.g., table, wall, window) to which each pixel in the image belongs. The main scene refers to a generalized judgment of the overall environment and function of the image, such as an office, kitchen, forest, or workshop. The preset set of reasonableness rules refers to a series of predefined logical judgment conditions used to evaluate whether it is reasonable to synthesize what kind of smoke and fire in a specific scene, specifically including combustibility rules, location rules, and exclusion rules. Combustibility rules are criteria for judging the combustibility of objects based on their category and assigning corresponding scores. Location rules are criteria for limiting the starting location of smoke and fire based on physical characteristics such as smoke rising and flame attachment. Exclusion rules are criteria that explicitly prohibit synthesis in areas where ignition is impossible, such as the sky or water bodies. Logical reasoning refers to the process of comprehensively analyzing pixel-level semantic information and the main scene based on the above rules to deduce potential ignition points. The combustible area refers to one or more image areas that are ultimately determined through logical reasoning and are suitable for synthesizing fireworks.

[0043] Understandably, this implementation method ensures a precise grasp of the scene composition through pixel-level semantic analysis, and provides accurate context for subsequent reasoning by combining a multimodal large model to interpret the global scene. Furthermore, it uses systematic rationality rules for automatic logical reasoning, which can intelligently and reliably identify flammable areas that conform to physical laws and common sense, thereby ensuring the correctness of the synthesis scheme in terms of spatial location and scene logic, and laying the foundation for generating highly realistic samples.

[0044] In one feasible implementation, the step of determining the combustible area in the smokeless scene image by performing logical reasoning on the pixel-level semantic information and the main scene based on a preset set of rationality rules includes: Based on preset flammability rules, flammability scores are assigned to each object category in the pixel-level semantic information to obtain a first flammability score. Based on preset position rules, the candidate smoke and flame starting positions in the main scene are determined according to the physical characteristics of smoke and flame in the pixel-level semantic information. The starting position of smoke is limited to the top area of ​​the candidate combustible object, and the starting position of flame is limited to the entire area of ​​the candidate combustible object. Based on preset exclusion rules, the locations in the main scenarios where fire cannot start are identified as the first exclusion locations; Combustible areas are determined by combining the first flammability score, the candidate smoke initiation position, and the first exclusion position to identify the flammable areas in the smokeless scene image.

[0045] It should be noted that the preset flammability rules are a set of quantitative criteria for assessing the ignition probability of objects based on their material and type. For example, high scores are assigned to wood and paper, while low scores are assigned to metal and glass. The first flammability score is a numerical ignition probability calculated for each object category based on these rules. The preset location rules are location constraints based on physical characteristics such as smoke rising due to hot air convection and flames typically adhering to the combustible material itself. The candidate smoke initiation location is the initial area in the image coordinates where smoke may be synthesized, obtained after applying the location rules. Candidate combustible objects refer to objects with potential ignition probability that have been initially screened by the flammability rules. The smoke initiation location is limited to the top area of ​​the candidate combustible object, simulating the physical phenomenon of hot smoke accumulating above the ignition point. The flame initiation location is limited to the entire area of ​​the candidate combustible object, simulating the combustion characteristic of flames typically spreading from the surface of the combustible material. The preset exclusion rules are a set of rules used to identify and filter out areas where a fire is logically impossible. The first exclusion location is an image area explicitly defined after applying exclusion rules, where pyrotechnic synthesis is prohibited, such as the sky, water bodies, or metal structure surfaces. Comprehensive flammability zone determination refers to the process of weighting or logically performing calculations on three types of information: the first flammability score, the candidate pyrotechnic starting location, and the first exclusion location, to arrive at the final synthesis area decision.

[0046] Understandably, this implementation transforms fuzzy scene understanding into a precise, calculable region selection problem by introducing quantified flammability scoring, physically consistent location constraints, and a clear region exclusion mechanism. This multi-rule collaborative reasoning logic can systematically narrow down the candidate range, significantly improving the credibility of the identified flammable areas in terms of physical rationality and spatial accuracy, providing crucial spatial positioning assurance for generating highly realistic fireworks samples.

[0047] In one feasible implementation, the step of determining the composite region of the composite pyrotechnic target based on the combustible region and the expected pyrotechnic type, and generating a corresponding region mask, includes: The initial synthesis area is determined based on the combustible area and the expected pyrotechnic type; The initial mask corresponding to the initial synthesis region is subjected to edge optimization processing; The initial synthesis region is expanded. Based on the optimized initial mask and the expanded initial composite region, the composite region of the composite pyrotechnic target is determined, and the region mask corresponding to the composite region is generated.

[0048] It should be noted that the initial synthesis region refers to the image pixel range initially delineated based on the boundaries of the combustible area and the initial morphological features of the expected fireworks type. The initial mask is a binary mask matrix corresponding to the initial synthesis region, where the target region and the background region have different identifier values. Edge optimization processing refers to softening the boundaries of the initial mask, including but not limited to applying a Gaussian blur filter to feather the edges, making the mask edges exhibit a smooth transition from 0 to 1. The expansion operation refers to the image processing procedure of moderately expanding the pixel range of the initial synthesis region, the purpose of which is to provide a slightly larger canvas space than the original object area for subsequent fireworks generation.

[0049] Understandably, this implementation method optimizes and moderately expands the initial area, so that the final determined composite area not only ensures that the firework target can naturally blend into the background and avoid harsh boundaries, but also reserves enough space for light and shadow interaction and physical effects (such as flames illuminating the surrounding area and smoke spreading at the edge) during the generation process, thereby significantly improving the visual integration and physical rationality of the composite firework with the original scene.

[0050] In one feasible implementation, the step of generating a smoke target within the synthetic region in the smokeless scene image based on the region mask to obtain a smoke detection sample image includes: Based on the object category in the combustible area, query the fireworks feature knowledge base to obtain the fireworks visual feature parameters of the fireworks type corresponding to the object category; The visual feature parameters of the fireworks are converted into synthetic parameters; The region mask, the synthesis parameters, and the smokeless scene image are input into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image, thus obtaining a smoke detection sample image.

[0051] It should be noted that the visual feature parameters of fireworks refer to the quantitative descriptions of visual attributes retrieved from the fireworks feature knowledge base and associated with specific object categories and combustion stages. These include, but are not limited to, the color value range, opacity, and texture features of smoke, as well as the typical hue, brightness distribution, and shape contour of flames. Synthesis parameters refer to a set of conditional control variables prepared to drive the generative model. These variables are derived from the visual feature parameters of fireworks and include natural language descriptive prompts used in text encoders, or image conditions (such as edge maps and depth maps) used to control the network. The pre-defined diffusion model refers to a generative artificial intelligence model based on a denoising diffusion probability model. This model can perform local redrawing within a spatial range specified by a region mask based on given conditional inputs to generate high-fidelity image content.

[0052] It is understandable that this implementation method achieves precise control over the appearance attributes of fireworks by transforming the structured physical features in the knowledge base into control parameters that can be recognized by the generative model; and then, by utilizing the powerful generative capability of the diffusion model and the spatial constraints of the region mask, it can synthesize fireworks targets that conform to physical laws and are visually coordinated with the background scene within a specified area, thereby efficiently and automatically producing sample data that can be used to train a high-quality fireworks detection model.

[0053] In one feasible implementation, the step of inputting the region mask, the synthesis parameters, and the smokeless scene image into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image to obtain a smoke detection sample image includes: The region mask is used as a spatial guidance input to a preset diffusion model; The synthesis parameters are converted into text prompts and used as the content conditions for generation, which are then input into the diffusion model. The diffusion model performs a local redrawing operation within the area specified by the region mask to generate a pyrotechnic target that meets the preset requirements. The image of the generated fireworks target is optimized to obtain a fireworks detection sample image.

[0054] It should be noted that spatial guidance refers to using region masks to explicitly instruct the diffusion model on the specific spatial location and extent of image generation or modification. Text prompts are natural language descriptions converted from synthesis parameters, used to detail the desired visual features of the generated fireworks, such as thin white smoke or flickering yellow flames. Content conditions are conditional signals input into the diffusion model to control and guide the semantic information of the generated content. Local redrawing is a generation mode of the diffusion model that generates content only for the portion of the image specified by the region mask, while leaving the rest of the image unchanged. A fireworks target that meets the preset requirements is a fireworks image whose visual appearance matches the features described by the text prompts and is consistent with the surrounding scene in terms of lighting, shadows, and physical logic. Optimization processing refers to subsequent processing steps performed on the generated image, including but not limited to color consistency adjustments, removal of local artifacts, or super-resolution enhancement to improve the visual quality and usability of the image.

[0055] Understandably, this implementation method, by combining spatial guidance with semantic content conditions, can precisely control the location and appearance attributes of the generated fireworks; the local redrawing capability of the diffusion model ensures seamless integration of the synthesized content with the original background; and the final optimization process further improves the visual fidelity of the synthesized samples, thereby systematically ensuring that the generated fireworks detection samples have high quality and high realism, and can be directly used to train a robust fireworks detection model.

[0056] In one feasible implementation, the actual utility of the synthetic data generated by this application in improving the performance of the fire and smoke detection model is verified: This application uses the Fraser Initial Distance (FID) as an evaluation metric for image generation quality. The core idea of ​​FID is to compare the statistical distribution of two sets of images in the high-dimensional feature space extracted by the deep neural network.

[0057] Specifically, calculate the mean vector and covariance matrix of the two feature distributions, and then calculate the second-order Wasserstein distance between the mean vector and the covariance matrix using the following formula:

[0058] Among them, the above formula and These are the feature mean and covariance of the real image set and the generated image set, respectively. The lower the FID score, the closer the feature distribution of the generated image set is to the distribution of the real image set, meaning the generated images are of higher quality and more diverse.

[0059] Comparative experiments demonstrate that the synthetic data generated in this application can significantly improve the actual performance of the target detection model. The experimental design strictly follows the controlled variable method. Experimental steps: First, build the dataset: Benchmark dataset (Dataset A): Collect all available real-world scenario data and related open-source datasets, and after cleaning and labeling, form the benchmark dataset; Augmented Dataset (Dataset B): The baseline dataset is mixed with a large amount of high-quality synthetic data generated by the system in this application. The mixing ratio is an adjustable hyperparameter that can be set to different values ​​according to the experiment. For example, the ratio of real data to synthetic data is 1:1 or 1:2. Test dataset: A portion of real images that are reserved independently and not used in any training will be used for the final evaluation.

[0060] Then, a comparative experiment was set up: using a standard object detection framework (such as YOLOv8): Baseline Model (Model A): Trained only on the baseline dataset, this model represents existing best practices without using the techniques described in this application; Augmented models (Model B, C): Trained on augmented datasets using the exact same YOLOv8 architecture and hyperparameters.

[0061] Finally, performance evaluation and result analysis were performed: all trained models were evaluated on independent test datasets, and the following core performance metrics were calculated: Fireworks detection rate: The proportion of real fire and smoke events correctly identified by the model out of the total number of real fire and smoke events; False alarm rate in business scenarios: The proportion of non-fire events incorrectly identified as fire events by the model out of the total number of non-fire events; By comparing the performance metrics of Model A with those of Models B and C, the value of the method proposed in this invention is clearly evident, with a significant improvement in model performance. As shown in Table 2, experimental results demonstrate that after incorporating the synthetic data generated in this application, the model significantly outperforms the benchmark model trained solely on real data in both the detection rate and false alarm rate. This proves that the technical solution proposed in this application effectively addresses the pain points of existing technologies and provides a practical and feasible path for improving the actual application performance of fire and smoke detection.

[0062] Table 2 Experimental Data

[0063] For example, to help understand the implementation flow of the fireworks detection sample image generation method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 2 , Figure 2 A simplified flowchart illustrating a method for generating sample images for fireworks detection is provided, specifically: Obtain a smokeless scene image as the base background. This image serves as the carrier for all subsequent compositing operations, ensuring that the generated samples have realistic and diverse scene contexts.

[0064] The system performs deep analysis on the input image. First, it identifies each object and its precise contour in the image using a semantic segmentation model. Then, it combines a multimodal large model with a pre-set set of reasonableness rules (including flammability, location, and exclusion rules) to perform logical reasoning. The output of this step is the synthesis plan generated by intelligent decision-making, which clearly answers the two key questions of "where to synthesize" and "what kind of fireworks to synthesize," that is, determining the flammable area and the expected type of fireworks.

[0065] The abstract planning from the previous stage is transformed into specific inputs required for the generative model. Based on the planning, the system generates a precise region mask to define the area in the image that needs to be modified; at the same time, it queries the fireworks feature knowledge base to convert the specified fireworks type into synthesis parameters (such as text prompts) that control the generated content.

[0066] The original image, region mask, and synthesis parameters are input into the diffusion model. The model performs local redrawing within the regions specified by the mask, generating smoke targets that blend naturally with the background in terms of lighting, shadows, and physical logic. The final output is a high-quality smoke detection sample, which can be directly used to train and improve the performance of the smoke detection model.

[0067] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for generating sample images for fireworks detection in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0068] This application provides a fireworks detection sample image generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the fireworks detection sample image generation method in the above embodiment 1.

[0069] The following is for reference. Figure 3 The diagram illustrates a structural schematic of a fireworks detection sample image generation device suitable for implementing embodiments of this application. The fireworks detection sample image generation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 3 The illustrated smoke detection sample image generation device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0070] like Figure 3As shown, the fireworks detection sample image generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the fireworks detection sample image generation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the fireworks detection sample image generation device to communicate wirelessly or wiredly with other devices to exchange data. Although a fireworks detection sample image generation device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0071] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0072] The fireworks detection sample image generation device provided in this application, employing the fireworks detection sample image generation method in the above embodiments, can solve the technical problem of low training sample quality. Compared with the prior art, the beneficial effects of the fireworks detection sample image generation device provided in this application are the same as those of the fireworks detection sample image generation method provided in the above embodiments, and other technical features in this fireworks detection sample image generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0073] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0074] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0075] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the fireworks detection sample image generation method in the above embodiments.

[0076] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0077] The aforementioned computer-readable storage medium may be included in the fireworks detection sample image generation device; or it may exist independently and not be assembled into the fireworks detection sample image generation device.

[0078] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the fireworks detection sample image generating device, cause the fireworks detection sample image generating device to: Acquire images of a smokeless fire scene; Scene understanding analysis is performed on the smokeless scene images to determine the combustible areas and expected smoke and fire types that meet the preset conditions; Based on the combustible area and the expected pyrotechnic type, determine the synthesis area of ​​the synthetic pyrotechnic target and generate the corresponding area mask; Based on the region mask, a smoke and fire target is generated within the synthetic region in the smoke-free scene image to obtain a smoke and fire detection sample image.

[0079] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0080] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0081] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0082] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for generating sample images for firework detection, thereby solving the technical problem of low training sample quality. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the firework detection sample image generation method provided in the above embodiments, and will not be repeated here.

[0083] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the fireworks detection sample image generation method described above.

[0084] The computer program product provided in this application can solve the technical problem of low training sample quality. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the smoke detection sample image generation method provided in the above embodiments, and will not be repeated here.

[0085] All acquisition of signals, information, or actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the relevant device owner.

[0086] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for generating sample images for fireworks detection, characterized in that, The method includes: Acquire images of a smokeless fire scene; Scene understanding analysis is performed on the smokeless scene images to determine the combustible areas and expected smoke and fire types that meet the preset conditions; Based on the combustible area and the expected pyrotechnic type, determine the synthesis area of ​​the synthetic pyrotechnic target and generate the corresponding area mask; Based on the region mask, a smoke and fire target is generated within the synthetic region in the smoke-free scene image to obtain a smoke and fire detection sample image.

2. The method as described in claim 1, characterized in that, The steps for scene understanding analysis of the smokeless fire scene image include: The smokeless fire scene image is analyzed and identified to obtain the object material information in the smokeless fire scene image. Based on the object material information, the scene logic in the smokeless fire scene image is determined through a preset multimodal large model. The system queries a pre-defined fireworks feature knowledge base to determine the expected fireworks type that matches the scene logic. The fireworks feature knowledge base is obtained by analyzing pre-defined literature samples and pre-defined real fireworks image samples, parametrically representing the visual and physical characteristics of different combustibles at different combustion stages, and storing them as fireworks types for the corresponding scenes in a pre-defined structured database.

3. The method as described in claim 1, characterized in that, The steps for scene understanding analysis of the smokeless fire scene image include: The semantic analysis of each item in the smokeless scene image is performed by a preset semantic segmentation model to determine the pixel-level semantic information in the smokeless scene image; By using a pre-set multimodal large model to perform scene understanding on the pixel-level semantic information, the main scene in the smokeless scene image is determined. Based on a preset set of rationality rules, logical reasoning is performed on the pixel-level semantic information and the main scene to determine the flammable area in the smokeless scene image.

4. The method as described in claim 3, characterized in that, The step of determining the flammable area in the smokeless scene image by performing logical reasoning on the pixel-level semantic information and the main scene based on a preset set of rationality rules includes: Based on preset flammability rules, flammability scores are assigned to each object category in the pixel-level semantic information to obtain a first flammability score. Based on preset position rules, the candidate smoke and flame starting positions in the main scene are determined according to the physical characteristics of smoke and flame in the pixel-level semantic information. The starting position of smoke is limited to the top area of ​​the candidate combustible object, and the starting position of flame is limited to the entire area of ​​the candidate combustible object. Based on preset exclusion rules, the locations in the main scenarios where fire cannot start are identified as the first exclusion locations; Combustible areas are determined by combining the first flammability score, the candidate smoke initiation position, and the first exclusion position to identify the flammable areas in the smokeless scene image.

5. The method as described in claim 1, characterized in that, The step of determining the composite region of the composite pyrotechnic target based on the combustible region and the expected pyrotechnic type, and generating the corresponding region mask, includes: The initial synthesis area is determined based on the combustible area and the expected pyrotechnic type; The initial mask corresponding to the initial synthesis region is subjected to edge optimization processing; The initial synthesis region is expanded. Based on the optimized initial mask and the expanded initial composite region, the composite region of the composite pyrotechnic target is determined, and the region mask corresponding to the composite region is generated.

6. The method as described in claims 1-5, characterized in that, The step of generating a smoke target within the synthetic region in the smokeless scene image based on the region mask to obtain a smoke detection sample image includes: Based on the object category in the combustible area, query the fireworks feature knowledge base to obtain the fireworks visual feature parameters of the fireworks type corresponding to the object category; The visual feature parameters of the fireworks are converted into synthetic parameters; The region mask, the synthesis parameters, and the smokeless scene image are input into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image, thus obtaining a smoke detection sample image.

7. The method as described in claim 6, characterized in that, The step of inputting the region mask, the synthesis parameters, and the smokeless scene image into a preset diffusion model to generate a smoke target within the synthesis region of the smokeless scene image to obtain a smoke detection sample image includes: The region mask is used as a spatial guidance input to a preset diffusion model; The synthesis parameters are converted into text prompts and used as the content conditions for generation, which are then input into the diffusion model. The diffusion model performs a local redrawing operation within the area specified by the region mask to generate a pyrotechnic target that meets the preset requirements. The image of the generated fireworks target is optimized to obtain a fireworks detection sample image.

8. A device for generating sample images for smoke and fire detection, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for generating a fireworks detection sample image as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for generating a fireworks detection sample image as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method for generating a fireworks detection sample image as described in any one of claims 1 to 7.

Images