Forest fire prevention unmanned aerial vehicle smoke fire point detection method based on dynamic query adaptation

Abstract

The application relates to the technical field of computer vision and forest fire prevention monitoring, and provides a forest fire prevention unmanned aerial vehicle smoke fire point detection method based on dynamic query self-adaption. The method comprises the following steps: acquiring a visible light image of a forest area collected by an unmanned aerial vehicle, performing size normalization and standardization processing on the image, and extracting three layers of appearance features of different scales; inputting the three layers of appearance features of different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, obtaining coding features of three layers of corresponding scales, inputting semantic vectors of the three scales into three groups of two-layer fully connected network branches which are independent of each other, mapping to a unified dimension query embedding space, and outputting a fixed number of query vectors from each branch; splicing the query vectors output by the three groups of branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vector, and outputting a smoke fire point detection result on the image through a detection head.

Description

Technical Field

[0001] This application relates to the fields of computer vision and forest and grassland fire prevention monitoring technology, and in particular to a method for detecting smoke and fire points of forest and grassland fire prevention drones based on dynamic query adaptive method. Background Technology

[0002] Early detection of forest and grassland fires is crucial for reducing disaster losses, and drones combined with target detection technology are currently the mainstream monitoring method for wide-area forest and grassland patrols. Existing technologies include Transformer-based end-to-end detection solutions with fast inference speeds, suitable for real-time smoke and fire detection scenarios.

[0003] Existing technologies use fixed, learnable query vectors as decoder input, meaning the query features cannot be dynamically adjusted based on the scale distribution of smoke targets in the input image. This results in problems such as a large scale range of smoke targets under aerial forest and grassland photography, insufficient detection accuracy for small-area fire points at long distances and weak smoke, and a high false negative rate, making it unsuitable for detecting multi-scale smoke targets. Summary of the Invention

[0004] This application provides a dynamic query-based adaptive method for detecting smoke and fire points on forest and grassland fire prevention drones, aiming to solve the problems of existing technologies, such as the large scale of smoke and fire targets under the aerial photography perspective of forest and grassland, insufficient detection accuracy of small-area fire points at long distances and weak smoke, and high false negative rate.

[0005] In a first aspect, embodiments of this application provide a method for detecting smoke and fire points using forest and grassland fire prevention drones based on dynamic query adaptation, the method comprising: Visible light images of forest and grassland areas collected by drones are acquired, and the images are subjected to size normalization and standardization processing to extract appearance features at three different scales. The appearance features at three different scales are then input into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, resulting in encoded features at the three corresponding scales. Global average pooling is performed on the encoded features of each layer to obtain semantic vectors corresponding to the three scales. The semantic vectors of the three scales are then input into three independent two-layer fully connected network branches and mapped to a query embedding space of the same dimension. Each branch outputs a fixed number of query vectors. The query vectors output from the three branches are concatenated along the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors. The dynamic query matrix is ​​then input into the three-layer encoded feature converter decoder to complete cross-attention interaction, and the smoke and fire point detection results are output by the detection head to the image.

[0006] In some embodiments, the acquisition of visible light images of forest and grassland areas collected by a drone, the performance of image size normalization and standardization processing, and the extraction of appearance features at three different scales include: acquiring aerial images of forest and grassland areas collected by a drone equipped with a visible light imaging device; scaling the image to a preset input size and performing pixel value normalization and channel-specific standardization processing; inputting the processed image into a pre-trained hybrid backbone feature extraction network; and outputting three layers of appearance features corresponding to different downsampling ratios after multi-layer convolution operations and progressive downsampling processing.

[0007] In some embodiments, the step of inputting appearance features of three different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtaining coded features of three corresponding scales, includes: inputting appearance features of three different scales into an efficient hybrid encoder; completing cross-scale information interaction of features at different levels through a bidirectional multi-scale fusion path of the neck coding structure; completing global context information modeling through multi-head self-attention operation of the converter encoder; and outputting coded features of three corresponding scales.

[0008] In some embodiments, performing global average pooling calculation on each layer of encoded features to obtain semantic vectors corresponding to the three scales includes: performing global average pooling operation on each layer of two-dimensional feature map in the three layers of encoded features; calculating the average value of all pixels in each channel of a single-layer feature map; arranging the average values ​​of all channels in channel order to generate a one-dimensional semantic vector for the corresponding layer; and obtaining independent semantic vectors corresponding to the three scales.

[0009] In some embodiments, the step of inputting semantic vectors of the three scales into three sets of independent two-layer fully connected network branches and mapping them to a query embedding space of a unified dimension, with each branch outputting a fixed number of query vectors, includes: inputting semantic vectors of the three scales into three sets of two-layer fully connected network branches with mutually exclusive network parameters; each branch first maps the semantic vectors to a hidden layer space of a preset dimension, performs nonlinear activation processing, and then maps them to a query embedding space of a unified dimension; each branch outputs a fixed number of query vectors, and all query vectors have the same dimension.

[0010] In some embodiments, the step of concatenating the query vectors output by the three branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors includes: concatenating the query vectors output by the three fully connected network branches in the sequence dimension according to the order of their corresponding feature scales from smallest to largest; generating a dynamic query matrix containing all query vectors; and generating all parameters of the dynamic query matrix in real time by calculating the features of the current input image to replace the original fixed learnable query vectors in the detection network.

[0011] In some embodiments, the step of inputting the dynamic query matrix and the three-layer encoded features into the converter decoder to complete cross-attention interaction and output the smoke and fire point detection results on the image through the detection head includes: inputting the dynamic query matrix as the query end input and the three-layer encoded features as the key end input, and inputting them together into the multi-layer converter decoder; the decoder internally completes the information interaction between the query vector and the encoded features through a cross-attention mechanism to optimize the feature representation of the query vector; the optimized query vector is input into the dual-branch detection head, which outputs the category confidence and bounding box position parameters of the smoke and fire targets respectively, to obtain the smoke and fire point detection results on the image.

[0012] In some embodiments, the method further includes: acquiring flight altitude parameters collected in real time by the UAV, and matching a corresponding scale weight strategy according to the flight altitude; when the flight altitude is higher than a preset altitude threshold, increasing the confidence weight of the query vector output by the fully connected branch corresponding to the small scale and decreasing the confidence weight of the query vector output by the fully connected branch corresponding to the large scale; when the flight altitude is lower than the preset altitude threshold, balancing the confidence weights of the query vector output by the three fully connected branches; correcting the category confidence value output by the detection head based on the adjusted confidence weights, and then outputting the final fireworks fire point detection result.

[0013] In some embodiments, the method further includes: for the detected fire point targets, matching the source fully connected branch of the corresponding query vector to determine the scale level of the target; for fire targets determined to be of small scale level, extracting local detail features of the corresponding encoded feature region for secondary feature verification; if the secondary verification result corresponding to the secondary feature verification conforms to the fire feature distribution rules, then retaining the corresponding detection result; if the secondary verification result corresponding to the secondary feature verification does not conform to the fire feature distribution rules, then removing the corresponding detection box.

[0014] In some embodiments, the method further includes: obtaining the fire risk level of the current patrol area of ​​the UAV, configuring a query quantity matching strategy according to the risk level; increasing the number of query vector outputs for small-scale fully connected branches in high fire risk areas to enhance the detection capability of small fire points; reducing the total number of query vector outputs for all fully connected branches in low fire risk areas to improve the inference speed of the edge computing unit; generating a corresponding dynamic query matrix based on the adjusted number of query vectors, and executing subsequent decoding and detection processes.

[0015] This application has the following beneficial effects: 1. Based on the multi-scale coding features of the input image, a dynamic query matrix is ​​generated in real time, which completely replaces the fixed learnable query vector in the traditional detection scheme. The feature representation of the query can be dynamically adjusted according to the scale distribution of the current input fire target, effectively adapting to the detection scenario with a large scale span of fire targets under the aerial photography perspective of forest and grassland.

[0016] 2. Three sets of independent fully connected mapping branches are used to process the encoded features at different scales. The query generation process at each scale does not interfere with each other, which specifically enhances the feature matching ability of small-scale fire points and weak smoke, effectively reducing the false detection rate of small targets and improving the overall detection accuracy.

[0017] 3. The overall solution is compatible with the existing Transformer end-to-end detection framework. Only the query generation logic needs to be adjusted, without significantly changing the original network structure. It will not significantly increase inference time and is suitable for the real-time deployment requirements of UAV onboard edge computing units.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart illustrating the steps of a method for detecting smoke and fire points of forest and grassland fire prevention drones based on dynamic query adaptive method, provided in one embodiment of this application. Figure 2 This is a schematic diagram illustrating the principle of a dynamic query adaptive method for detecting smoke and fire points of forest and grassland fire prevention drones, provided in one embodiment of this application. Figure 3 This is a schematic block diagram of a forest and grassland fire prevention drone smoke and fire spot detection system based on dynamic query adaptive method, provided in one embodiment of this application. Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0024] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0025] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] Early detection of forest and grassland fires is crucial for reducing disaster losses, and drones combined with target detection technology are currently the mainstream monitoring method for wide-area forest and grassland patrols. Existing technologies include Transformer-based end-to-end detection solutions with fast inference speeds, adaptable to real-time smoke and fire detection scenarios.

[0028] The above scheme uses a fixed learnable query vector as the decoder input, and the query features cannot be dynamically adjusted according to the scale distribution of the smoke targets in the input image. This results in problems such as a large scale range of smoke targets under aerial forest and grassland photography, insufficient detection accuracy for small-area fire points at long distances and weak smoke, and a high false negative rate, making it unsuitable for the detection needs of multi-scale smoke targets.

[0029] Please refer to Figure 1 This application provides a method for detecting smoke and fire points using a forestry and grassland fire prevention drone based on dynamic query adaptation, applied to computer equipment. The computer equipment can be deployed on a single server or a server cluster. It can also be deployed on handheld terminals, laptops, wearable devices, or robots, etc. It should be noted that all information involved in the method provided in this application is extracted with the authorization of the relevant user and in accordance with relevant regulations, and will not infringe on user privacy.

[0030] The method for detecting smoke and fire points on forest and grassland fire prevention drones based on dynamic query adaptive provided in this application can be deployed on various electronic devices with computing capabilities, including independent servers, server clusters, handheld terminals, laptops, wearable devices, drone-borne edge computing units, and ground forest and grassland monitoring terminals. It is suitable for aerial patrol and fire detection scenarios of wide-area forest and grassland resources such as forest areas, grasslands, and wetlands.

[0031] The provided method for detecting smoke and fire spots using UAVs for forest and grassland fire prevention based on dynamic query adaptation includes steps S101 to S103. Details are as follows: Step S101. Acquire visible light images of forest and grassland areas collected by UAVs, perform size normalization and standardization processing on the images, and extract appearance features at three different scales; input the appearance features at three different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtain the encoded features at the corresponding scales of the three layers.

[0032] Specifically, aerial visible light images of forest and grassland areas, captured by a drone equipped with a visible light imaging device, are used as the raw input for the detection method. First, preprocessing is performed on the input images, consisting of two steps: size normalization and pixel standardization. Size normalization uses a bilinear interpolation algorithm, calculating the new pixel value based on the weighted distances to the four neighboring points around each pixel. This scales the input image of any resolution to a fixed input size adapted to the feature extraction network, ensuring uniformity in the size of all input images while balancing scaling speed and image detail preservation. Pixel standardization first linearly maps the red, green, and blue channel values ​​of each pixel from an integer grayscale range to a floating-point range of 0 to 1. Then, according to the preset mean and standard deviation of each color channel, a linear transformation is performed on the pixel value of each channel. This involves subtracting the mean of the corresponding channel from each pixel value and dividing by the standard deviation of the corresponding channel, ensuring that the distribution of the input data is completely consistent with the data distribution during the pre-training stage of the feature extraction network, improving the stability and transfer accuracy of feature extraction.

[0033] After preprocessing, the image is input into a pre-trained hybrid backbone feature extraction network. The backbone network consists of multiple residual convolutional blocks and depthwise separable convolutional blocks stacked sequentially. It performs progressive downsampling through convolutional layers with a stride of 2. With each downsampling, the feature map's width and height are halved, while the number of channels increases accordingly. The network ultimately extracts three layers of appearance features at different scales: the feature map with the lowest downsampling ratio has the highest resolution, preserving rich shallow details such as edges and textures, but with a smaller receptive field, suitable for detecting small targets such as distant, small-area fire spots and faint smoke; the feature map with an intermediate downsampling ratio balances detailed information with deep semantic information, with a moderate receptive field, suitable for detecting medium-sized smoke targets; and the feature map with the highest downsampling ratio has the largest receptive field, containing rich high-level semantic information, suitable for detecting large targets such as large-area open flames and widely diffused smoke.

[0034] Three layers of appearance features at different scales are input into the high-efficiency hybrid encoder to achieve cross-scale fusion and global context modeling. The high-efficiency hybrid encoder consists of a neck encoder structure and a converter encoder connected in series. The neck coding structure employs a bidirectional, multi-scale fusion architecture, comprising a top-down upsampling path and a bottom-up downsampling path. The top-down path starts with the deepest high-semantic features, upsamples them by a factor of 2 to enlarge the feature size, and then adds them element-wise with the shallow features of the previous layer, progressively transferring deep semantic information to the shallow layers and enhancing the semantic expressiveness of the shallow features. The bottom-up path starts with the shallowest high-detail features, downsamples them by a factor of 2 to reduce the feature size, and then adds them element-wise with the deep features of the next layer, progressively transferring shallow detail information to the deep layers and enhancing the detail precision of the deep features. After bidirectional fusion, all three layers of features simultaneously contain both detail and semantic information, improving the expressive power of features at different scales.

[0035] The converter encoder first flattens the two-dimensional feature map into a one-dimensional feature sequence and adds a learnable positional code to each location. The positional code length is the same as the feature sequence length and is directly superimposed on the feature sequence to preserve spatial positional information and avoid losing positional correlations after sequence unpacking. Then, a multi-head self-attention mechanism is used to model the pixel correlations globally. Multiple independent attention heads focus on different feature subspaces, capturing different types of feature correlations. The outputs of all attention heads are concatenated and linearly transformed to obtain the global context features. Multi-head self-attention can capture the dependency relationship between any two locations in the feature map, and is particularly suitable for smoke targets with blurred edges and irregular shapes, effectively improving the feature recognition ability for large-scale smoke diffusion and small open flames. The efficient hybrid encoder finally outputs three layers of encoded features corresponding to the same scale as the input layers.

[0036] Step S102. Perform global average pooling calculation on each layer of encoded features to obtain semantic vectors corresponding to the three scales respectively; input the semantic vectors of the three scales into three sets of independent two-layer fully connected network branches respectively, map them to a query embedding space of the same dimension, and each branch outputs a fixed number of query vectors.

[0037] Specifically, for the three-layer encoded features output by the efficient hybrid encoder, global average pooling is performed independently for each layer. The logic of global average pooling is as follows: for each layer of two-dimensional feature map, each feature channel is traversed, the corresponding two-dimensional spatial feature map is extracted, the values ​​of all pixels in the map are summed and divided by the total number of pixels to obtain the average response value of that channel; the average response values ​​of all channels are arranged in channel order to form a one-dimensional semantic vector. The dimension of each semantic vector is completely consistent with the number of channels of the corresponding layer of encoded features, and it can represent the overall semantic distribution, target proportion and scale attribute of the corresponding scale features. The three layers of encoded features correspond to three independent one-dimensional semantic vectors, which correspond to global feature information at small, medium and large scales, respectively.

[0038] The semantic vectors at the three scales are respectively input into three independent two-layer fully connected network branches. The network parameters of the three branches are completely independent and do not share with each other, and are adapted to the feature distribution characteristics of different scales respectively. This avoids the interference between the distribution differences of features at different scales and ensures that the query vector at each scale can accurately adapt to the target characteristics of the corresponding size.

[0039] Each fully connected branch contains two linear transformation layers and one non-linear activation layer: The first layer is a hidden layer linear transformation layer, with the input dimension consistent with the dimension of the corresponding semantic vector, and the output dimension being the preset unified hidden layer dimension. After linear transformation, a non-linear activation function is applied to introduce non-linear transformation capability, improve the upper limit of the network's feature representation, and avoid the limitations of pure linear mapping. The second layer is an output layer linear transformation layer, with the input dimension consistent with the hidden layer dimension, and the output dimension being the product of a fixed number of queries and the query dimension. The output results are reshaped to obtain a fixed number of unified dimension query vectors. All query vectors output by the three branches have completely consistent dimensions, with only the feature representation adapted to the target characteristics at the corresponding scale. The query vectors output by each branch correspond to target detection tasks at different scales.

[0040] Step S103. Concatenate the query vectors output from the three branches along the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors; input the dynamic query matrix and the three-layer encoded feature into the converter decoder to complete cross-attention interaction, and output the smoke and fire point detection results on the image through the detection head.

[0041] Specifically, the query vectors output by the three fully connected network branches are arranged in ascending order of their corresponding feature scales, i.e., the query vectors output by the small-scale branch come first, the query vectors output by the medium-scale branch are in the middle, and the query vectors output by the large-scale branch come last. After the arrangement is completed, a concatenation operation is performed on the three sets of query vectors in the sequence dimension, merging the three independent query vector sequences into a single query vector sequence, forming a complete dynamic query matrix.

[0042] All parameters of the dynamic query matrix are calculated and generated in real time from the features of the current input image. There are no pre-trained fixed learnable parameters, and the feature representation of the query can be automatically adjusted according to the scale distribution of the fireworks targets in each input image. The overall dimension of the dynamic query matrix is ​​completely consistent with the dimension of the fixed learnable query matrix in traditional converter detection schemes. It can directly replace the original fixed query input decoder without modifying the decoder structure, and has extremely strong compatibility.

[0043] The generated dynamic query matrix and the three-layer encoded features are input together into the multi-layer converter decoder. The dynamic query matrix serves as the query input to the decoder, and the flattened three-layer encoded features serve as the key and value inputs to the decoder. The converter decoder contains multiple decoding layers connected in series. Each decoding layer contains a self-attention module, a cross-attention module, and a feedforward network module. Each module is preceded by a layer normalization structure, and residual connections are used between modules to ensure training stability. The self-attention module performs self-attention calculation on the query vector sequence, models the interrelationships between query vectors, optimizes the internal distribution of the query sequence, avoids multiple query vectors repeatedly focusing on the same target region, and reduces the generation of subsequent redundant detection boxes. The cross-attention module completes the information interaction between the query vector and the encoded features, enabling each query vector to focus on the corresponding target region in the feature map as needed, and extracts the target's specific features. The feedforward network module consists of two layers of linear transformation and activation function, which performs non-linear transformation on the features of each query vector to further improve the expressive power of the features.

[0044] After iterative optimization by a multi-layer decoder, the final query feature sequence is output, and this sequence is then input into a dual-branch detection head. The dual-branch detection head contains independent classification and regression branches: the classification branch outputs the confidence score of each query vector corresponding to a target belonging to the fireworks category, mapped to a probability value between 0 and 1 by an activation function; a higher value indicates a greater probability that the corresponding area is a fireworks target. The regression branch outputs the bounding box position and size parameters of each query vector corresponding to the target, including the horizontal and vertical coordinates of the bounding box center, the width, and the height of the bounding box. All parameters are normalized coordinates relative to the input image; multiplying them by the actual size of the input image converts them to the absolute pixel coordinates of the original image. The detection head ultimately outputs complete fireworks fire point detection results, including the category confidence scores and bounding box positions of all valid targets.

[0045] In some embodiments, the acquisition of visible light images of forest and grassland areas collected by a drone, the performance of image size normalization and standardization processing, and the extraction of appearance features at three different scales include: acquiring aerial images of forest and grassland areas collected by a drone equipped with a visible light imaging device; scaling the image to a preset input size and performing pixel value normalization and channel-specific standardization processing; inputting the processed image into a pre-trained hybrid backbone feature extraction network; and outputting three layers of appearance features corresponding to different downsampling ratios after multi-layer convolution operations and progressive downsampling processing.

[0046] This embodiment provides a detailed explanation of the specific implementation methods for visible light image preprocessing and three-layer multi-scale appearance feature extraction.

[0047] First, aerial images of forest and grassland areas were acquired using a drone equipped with a visible light imaging device. The acquisition scenario involved the drone conducting forest and grassland patrols along a preset route, with the gimbal angle maintained between 30 and 60 degrees and the flight altitude ranging from 80 to 150 meters. The acquired images were in three-channel color format with an original resolution of no less than 1920×1080 pixels to ensure sufficient image detail for identifying small fires and faint smoke at a distance.

[0048] The input image undergoes size normalization, employing bilinear interpolation to scale it to a fixed input size of 640×640 pixels. Bilinear interpolation calculates a new pixel value by weighting the distances to the four neighboring pixels surrounding the target pixel. Compared to nearest-neighbor interpolation, it preserves more edge details and is faster than bicubic interpolation, balancing detection accuracy and inference efficiency. After size normalization, pixel value normalization is performed, linearly mapping the integer range of 0 to 255 for each pixel's red, green, and blue channels to a floating-point range of 0 to 1 by dividing by 255. Subsequently, channel-specific normalization is performed, standardizing the pixel values ​​of each channel according to the mean and standard deviation of the three channels corresponding to the pre-trained backbone network. Each pixel value is subtracted from the mean of the corresponding channel and divided by the standard deviation of the corresponding channel, ensuring that the data distribution of each channel meets the requirements of a mean of 0 and a variance of 1, perfectly matching the data distribution during the pre-training stage of the backbone network, thus improving the stability and transfer effect of feature extraction.

[0049] The pre-processed image is input into a pre-trained hybrid backbone feature extraction network. The backbone network is pre-trained on a large-scale general image dataset and has mature general visual feature extraction capabilities, which can be directly transferred to the smoke and fire detection scenario. The backbone network adopts a four-stage hierarchical convolutional structure. The initial stage is a single convolutional layer with a stride of 2. The subsequent four stages are composed of multiple residual convolutional blocks and depthwise separable convolutional blocks stacked together. The first convolutional block in each stage has a stride of 2 and completes one downsampling operation. The network extracts the outputs from the second, third, and fourth stages as three-layer multi-scale appearance features, with corresponding downsampling ratios of 8x, 16x, and 32x, respectively: the 8x downsampled feature map has a size of 80×80 pixels and 128 channels, preserving rich edge and texture details, with a smaller receptive field, suitable for small-sized fireworks target detection; the 16x downsampled feature map has a size of 40×40 pixels and 256 channels, balancing details and semantic information, suitable for medium-sized target detection; and the 32x downsampled feature map has a size of 20×20 pixels and 512 channels, containing rich high-level semantic information, with the largest receptive field, suitable for large-sized fireworks target detection.

[0050] This embodiment outputs three layers of appearance features corresponding to different downsampling rates, which are then input into a subsequent high-efficiency hybrid encoder for processing.

[0051] In some embodiments, the step of inputting appearance features of three different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtaining coded features of three corresponding scales, includes: inputting appearance features of three different scales into an efficient hybrid encoder; completing cross-scale information interaction of features at different levels through a bidirectional multi-scale fusion path of the neck coding structure; completing global context information modeling through multi-head self-attention operation of the converter encoder; and outputting coded features of three corresponding scales.

[0052] This embodiment provides a detailed explanation of the specific implementation method of a high-efficiency hybrid encoder to achieve cross-scale fusion and global context modeling.

[0053] The appearance features of three different scales are input into the high-efficiency hybrid encoder, which consists of a neck encoder structure and a converter encoder connected in series.

[0054] The neck coding structure employs a bidirectional feature pyramid architecture, comprising a top-down upsampling path and a bottom-up downsampling path. The top-down path starts with the deepest 32x downsampled features, amplifies the feature size through a 2x upsampling operation, and adds element-wise with shallower 16x downsampled features to obtain a fused 16x feature. This fused 16x feature is then upsampled again by 2x and added element-wise with 8x downsampled features to obtain a fused 8x feature, completing the transfer of semantic information from deep to shallow layers. The bottom-up path starts with the fused 8x feature, performs a 2x downsampling operation through a convolutional layer with a stride of 2, and adds element-wise with the fused 16x feature to obtain an enhanced 16x feature. This enhanced 16x feature is then downsampled again by 2x and added element-wise with the 32x downsampled fused feature to obtain an enhanced 32x feature, completing the transfer of detailed information from shallow to deep layers. After bidirectional fusion, all three layers of features simultaneously contain detailed and semantic information, and the feature expression capabilities at different scales are improved, solving the problems of insufficient semantics in shallow features and lack of details in deep features in traditional single-path fusion.

[0055] A three-layer feature input converter encoder, which performs cross-scale fusion, independently encodes each layer of features. First, the two-dimensional feature map is flattened into a one-dimensional feature sequence based on spatial location, and a learnable positional code is added to each location. The positional code length is the same as the feature sequence length and is directly superimposed on the feature sequence, preserving the spatial location information of the features and avoiding the loss of spatial correlation after sequence flattening. Then, a multi-head self-attention mechanism is used for global context modeling. In this embodiment, the number of self-attention heads is set to 8, with each head focusing on a different feature subspace to capture different types of feature correlations. The outputs of all attention heads are concatenated and linearly transformed to obtain the global context features. The multi-head self-attention mechanism can capture the dependency relationship between any two locations in the feature map, and is particularly suitable for smoke targets with blurred edges and irregular shapes, effectively improving the feature recognition ability for large-scale diffuse smoke and small open flames.

[0056] In this embodiment, the converter encoder is set to 2 layers, the hidden dimension of the feedforward network is set to 1024, and a normalization structure of the front layer is used to ensure training stability. After the encoder finishes processing, it outputs the encoded features of the three corresponding scales, retains the original multi-scale hierarchical structure, and inputs them into the subsequent semantic vector extraction module.

[0057] In some embodiments, performing global average pooling calculation on each layer of encoded features to obtain semantic vectors corresponding to the three scales includes: performing global average pooling operation on each layer of two-dimensional feature map in the three layers of encoded features; calculating the average value of all pixels in each channel of a single-layer feature map; arranging the average values ​​of all channels in channel order to generate a one-dimensional semantic vector for the corresponding layer; and obtaining independent semantic vectors corresponding to the three scales.

[0058] This embodiment provides a detailed explanation of the specific implementation of global average pooling for extracting semantic vectors at various scales.

[0059] Global average pooling is performed on each of the three layers of coded features in a separate two-dimensional feature map. The operations are performed independently, and the processing of the three layers of features does not interfere with each other.

[0060] For a single-layer two-dimensional feature map, the feature map's dimensions include channel dimension and spatial dimension. The spatial dimension is the length and width of the feature map, and the channel dimension is the number of channels in the feature map. Global average pooling is performed along the spatial dimension: it iterates through each feature channel, extracts the corresponding two-dimensional spatial feature map, sums the values ​​of all pixels in the map, and divides this sum by the total number of pixels in that channel to obtain the average response value for that channel. This average value is the pooling result for the corresponding channel. The pooling results of all channels are arranged in channel order to form a one-dimensional vector, which is the semantic vector corresponding to that layer of features.

[0061] Taking the parameters of this embodiment as an example: the size of the encoded feature with 8x downsampling is 80×80, the number of channels is 128, and each channel contains 6400 pixels of 80×80. The average value of the 6400 pixels in each channel is taken to obtain 128 average values, which are arranged in channel order to form a one-dimensional semantic vector with a length of 128. Similarly, the encoded feature with 16x downsampling corresponds to a semantic vector with a length of 256, and the encoded feature with 32x downsampling corresponds to a semantic vector with a length of 512.

[0062] Each value in the semantic vector corresponds to the global average response of a feature channel, which can characterize the overall intensity of the feature corresponding to that channel in the whole image. Through global average pooling, the two-dimensional spatial features can be compressed into a one-dimensional global semantic representation, removing spatial location information and retaining only the overall semantic distribution, target proportion and scale attribute of the feature at that scale, providing a global semantic basis for subsequent query vector generation.

[0063] After performing the above operations on the three layers of encoded features, independent semantic vectors corresponding to the three scales are obtained. The three vectors have different dimensions and are input into the three subsequent independent fully connected network branches.

[0064] In some embodiments, the step of inputting semantic vectors of the three scales into three sets of independent two-layer fully connected network branches and mapping them to a query embedding space of a unified dimension, with each branch outputting a fixed number of query vectors, includes: inputting semantic vectors of the three scales into three sets of two-layer fully connected network branches with mutually exclusive network parameters; each branch first maps the semantic vectors to a hidden layer space of a preset dimension, performs nonlinear activation processing, and then maps them to a query embedding space of a unified dimension; each branch outputs a fixed number of query vectors, and all query vectors have the same dimension.

[0065] This embodiment provides a detailed explanation of the specific implementation method for generating query vectors using three sets of independent fully connected network branch mappings.

[0066] The semantic vectors of the three scales are respectively input into three independent two-layer fully connected network branches. The network parameters of the three branches are completely independent and do not share with each other. They are optimized for the semantic features of the corresponding scale, avoiding mutual interference from the distribution differences of features at different scales. Each branch specifically learns the mapping relationship from the semantics of the corresponding scale to the query space, which has higher mapping accuracy than sharing parameters.

[0067] Each fully connected network branch employs a structure of two linear transformation layers followed by one non-linear activation layer. Since the input dimensions of the three semantic vectors differ, the first fully connected layer of the three branches has input dimensions of 128, 256, and 512 respectively, with a unified output dimension of 256-dimensional hidden features. After the linear transformation of the hidden layer, a leaky rectified linear activation function is applied with a negative slope of 0.1, introducing non-linear transformation capability to enhance the upper limit of the network's feature representation and avoid the limitations of pure linear mapping. The second layer is the output linear transformation layer. The input dimension of all three branches is 256-dimensional, and the output dimension is unified at 25600, i.e., 100 query vectors multiplied by the 256-dimensional dimension of each vector. The output layer's result undergoes dimension reshaping to obtain 100 256-dimensional query vectors.

[0068] In this embodiment, each fully connected branch outputs 100 query vectors, and the three branches output a total of 300 query vectors. The 100 query vectors from each branch are adapted to the target detection requirements at the corresponding scale: the query vectors from the small-scale branch focus on matching fine-grained features of small fire spots and weak smoke, the medium-scale branch is adapted to regular-sized smoke and fire targets, and the large-scale branch is adapted to the semantic features of large-area open flames and large-scale smoke. All query vectors maintain a consistent 256-dimensional dimension to meet the dimensionality requirements of subsequent concatenation and decoder input.

[0069] In some embodiments, the step of concatenating the query vectors output by the three branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors includes: concatenating the query vectors output by the three fully connected network branches in the sequence dimension according to the order of their corresponding feature scales from smallest to largest; generating a dynamic query matrix containing all query vectors; and generating all parameters of the dynamic query matrix in real time by calculating the features of the current input image to replace the original fixed learnable query vectors in the detection network.

[0070] This embodiment provides a detailed explanation of the specific implementation method for generating a dynamic query matrix by concatenating query vectors.

[0071] The query vectors output by the three fully connected network branches are arranged in ascending order of their corresponding feature scales: the 100 query vectors from the small-scale branch are placed in positions 1-100, the 100 query vectors from the medium-scale branch are placed in positions 101-200, and the 100 query vectors from the large-scale branch are placed in positions 201-300. After arrangement, a concatenation operation is performed on the three sets of query vectors along the sequence dimension, merging the three independent query vector sequences into a single query vector sequence, forming a complete dynamic query matrix.

[0072] In this embodiment, the concatenated dynamic query matrix contains 300 query vectors, each with a dimension of 256. The overall dimension of the matrix is ​​completely consistent with that of the traditional fixed learnable query matrix. Therefore, it can directly replace the original fixed query input converter decoder without any modification to the decoder structure, and has strong compatibility.

[0073] All parameters of the dynamic query matrix are calculated and generated in real time based on the features of the current input image. There are no pre-trained fixed learnable parameters. Each forward propagation is recalculated based on the current input image, and it is entirely driven by the input features. When small targets account for a high proportion in the input image, the semantic vector of the small-scale branch generates query vectors that fit the features of small targets, improving the matching accuracy of small targets. When large targets account for a high proportion in the input image, the large-scale branch generates corresponding query vectors that fit the features, improving the detection effect of large targets. The dynamic query matrix can be adjusted in real time according to the target distribution of each frame of the input image, completely replacing the original fixed learnable query vectors in traditional detection networks, and solving the problem that fixed queries cannot adapt to the scale of large-span fireworks targets from the aerial perspective.

[0074] In some embodiments, the step of inputting the dynamic query matrix and the three-layer encoded features into the converter decoder to complete cross-attention interaction and output the smoke and fire point detection results on the image through the detection head includes: inputting the dynamic query matrix as the query end input and the three-layer encoded features as the key end input, and inputting them together into the multi-layer converter decoder; the decoder internally completes the information interaction between the query vector and the encoded features through a cross-attention mechanism to optimize the feature representation of the query vector; the optimized query vector is input into the dual-branch detection head, which outputs the category confidence and bounding box position parameters of the smoke and fire targets respectively, to obtain the smoke and fire point detection results on the image.

[0075] This embodiment provides a detailed explanation of the specific implementation methods for the interaction between the converter and decoder and the output of detection results by the detection head.

[0076] The dynamic query matrix is ​​used as the query input, and the feature sequence obtained by flattening and merging the three-layer encoded features is used as the key and value inputs, which are then input to the multi-layer converter decoder. In this embodiment, the converter decoder adopts a 6-layer stacked structure, with each layer having the same structure, sequentially including a self-attention module, layer normalization, cross-attention module, layer normalization, feedforward network module, and layer normalization. The modules are connected by a residual connection structure, and the input and output of each module are added before entering the next module, ensuring the stability of the training process.

[0077] The self-attention module performs self-attention calculations on the input query vector sequence, models the interrelationships between query vectors, optimizes the internal distribution of the query sequence, avoids multiple query vectors repeatedly focusing on the same target region, and reduces the generation of subsequent redundant detection boxes. The cross-attention module uses the query vector as the query and the flattened sequence of encoded features as the key and value, calculating the attention weights between the query vector and each position in the feature map, so that each query vector focuses on the corresponding target region in the feature map as needed, extracting the target's specific features. The feedforward network consists of two layers of linear transformation and activation functions, performing non-linear transformations on the features of each query vector to further enhance the expressive power of the features. In this embodiment, the hidden dimension of the feedforward network is set to 1024. After iterative optimization by a 6-layer decoder, each query vector completes a deep interaction with image features, resulting in an optimized query feature sequence.

[0078] The optimized query feature sequence is input into a dual-branch detector head. The two branches are independent of each other but share the input query features. The classification branch outputs the firework category confidence score for each query vector through a linear transformation layer. This score is then mapped to the 0-1 range using a sigmoid activation function, with higher values ​​indicating a greater probability that the corresponding region is a firework target. The regression branch outputs the bounding box parameters for each query vector through a linear transformation layer. These parameters include the x-coordinate of the bounding box center, the y-coordinate of the center, the width, and the height. All parameters are normalized values ​​relative to the width and height of the input image. Multiplying these values ​​by the 640-pixel size of the input image yields the absolute pixel coordinates.

[0079] The detection head ultimately outputs the detection results corresponding to all query vectors, which are the smoke and fire point detection results on the image, and can be directly used for subsequent result filtering and alarm judgment.

[0080] In some embodiments, the method further includes: acquiring flight altitude parameters collected in real time by the UAV, and matching a corresponding scale weight strategy according to the flight altitude; when the flight altitude is higher than a preset altitude threshold, increasing the confidence weight of the query vector output by the fully connected branch corresponding to the small scale and decreasing the confidence weight of the query vector output by the fully connected branch corresponding to the large scale; when the flight altitude is lower than the preset altitude threshold, balancing the confidence weights of the query vector output by the three fully connected branches; correcting the category confidence value output by the detection head based on the adjusted confidence weights, and then outputting the final fireworks fire point detection result.

[0081] This embodiment provides an adaptive adjustment scheme that combines the scale weight of the UAV's flight altitude, and details the dynamic optimization method for detection confidence.

[0082] During the detection process, the current flight altitude parameters are obtained through the real-time data interface of the UAV flight control system. The update frequency of the altitude parameters is consistent with the video frame rate, with the altitude parameters being updated synchronously for each detection frame. Based on the current flight altitude, a corresponding scale weighting strategy is matched, and the detection confidence of the outputs of different scale branches is dynamically weighted and adjusted.

[0083] In this embodiment, 100 meters is set as the preset height threshold. This threshold is selected based on the flight altitude range of conventional forest and grassland patrols. Above 100 meters, targets in aerial footage are generally smaller, while below 100 meters, the target scale range is larger. When the current flight altitude is detected to be above 100 meters, it is determined that the smoke and fire targets in the aerial footage are generally small, with a higher proportion of small targets. At this time, the confidence weight of the query vector output by the fully connected branch corresponding to the small scale is automatically increased to 1.2, the weight of the medium scale branch remains unchanged at 1.0, and the confidence weight of the query vector output by the fully connected branch corresponding to the large scale is appropriately reduced to 0.8 to enhance the detection capability of small targets and avoid missing small fire points during high-altitude patrols. When the current flight altitude is detected to be below or equal to 100 meters, it is determined that the scale range of the smoke and fire targets in the footage is larger, and targets of all sizes may appear. At this time, the confidence weights of the three fully connected branches are all set to 1.0 to balance the detection effect of targets of different sizes.

[0084] The specific method for weight adjustment is as follows: For each candidate detection box output by the detection head, first match the corresponding weight coefficient according to the source branch of its corresponding query vector, and then multiply the original confidence value by the corresponding weight coefficient to obtain the corrected category confidence value. The value range of the weight coefficient is controlled between 0.8 and 1.2 to avoid over-adjustment that may distort the detection results. The weight only affects the classification confidence and does not change the position of the regression box. The corrected confidence is used for subsequent threshold filtering.

[0085] This embodiment can dynamically adapt the detection strategy according to the actual patrol flight altitude, further improving the detection accuracy at different flight altitudes and adapting to the high-altitude operation scenarios in forest and grassland patrols.

[0086] In some embodiments, the method further includes: for the detected fire point targets, matching the source fully connected branch of the corresponding query vector to determine the scale level of the target; for fire targets determined to be of small scale level, extracting local detail features of the corresponding encoded feature region for secondary feature verification; if the secondary verification result corresponding to the secondary feature verification conforms to the fire feature distribution rules, then retaining the corresponding detection result; if the secondary verification result corresponding to the secondary feature verification does not conform to the fire feature distribution rules, then removing the corresponding detection box.

[0087] This embodiment provides a secondary feature verification scheme for small-scale targets, and details the implementation method for reducing false detections of small targets.

[0088] First, for all detected fire points, the source fully connected branch of the corresponding query vector is matched. Based on the scale level corresponding to the branch, the scale level of each detected target is determined: targets from the small-scale branch are classified as small-scale targets, targets from the medium-scale branch are classified as medium-scale targets, and targets from the large-scale branch are classified as large-scale targets. Only fire targets classified as small-scale targets undergo a secondary feature verification process to reduce the false detection probability of small targets without affecting the detection efficiency of medium and large targets.

[0089] The specific process of secondary feature verification is as follows: First, based on the normalized coordinates of the bounding box of the small-scale target, it is mapped onto the corresponding level of encoded feature map, and the local feature region surrounding the target is cropped out. The size of the cropped region is set to 1.2 times the size of the feature map corresponding to the detection box, including the context information around the target, to improve the accuracy of verification. Then, three types of detail features are extracted from this local region, namely texture complexity features, edge gradient distribution features, and channel mean distribution features.

[0090] The extracted local detail features are matched against pre-stored fireworks target feature distribution rules. These rules are derived from the statistical analysis of local features of thousands of positive and negative fireworks samples in the training set, and include numerical distribution thresholds for each of the three feature classes. If all three feature classes meet the threshold range, the fireworks feature distribution is considered to be consistent; otherwise, it is considered not consistent.

[0091] If the secondary verification result conforms to the distribution rules of fireworks features, it is determined to be a real fireworks target, and the corresponding detection result is retained; if the secondary verification result does not conform to the distribution rules of fireworks features, it is determined to be a false detection target, and the corresponding detection box is removed. Common small-scale false detection targets, such as red fallen leaves, small reflective objects, and scattered light spots, have texture and gradient features that are significantly different from real fireworks and can be effectively filtered out through secondary verification.

[0092] This embodiment addresses the issue of high false detection rates for small-scale targets by using secondary feature verification to further filter small-scale detection results. This approach can reduce the false detection rate of small targets while maintaining the detection rate of small targets, thereby improving the overall reliability of the detection results.

[0093] In some embodiments, the method further includes: obtaining the fire risk level of the current patrol area of ​​the UAV, configuring a query quantity matching strategy according to the risk level; increasing the number of query vector outputs for small-scale fully connected branches in high fire risk areas to enhance the detection capability of small fire points; reducing the total number of query vector outputs for all fully connected branches in low fire risk areas to improve the inference speed of the edge computing unit; generating a corresponding dynamic query matrix based on the adjusted number of query vectors, and executing subsequent decoding and detection processes.

[0094] This embodiment provides an adaptive adjustment scheme for the number of queries based on the fire risk level of the patrol area, and details the implementation method for balancing detection accuracy and inference speed.

[0095] Before detection, the fire risk level of the current patrol area is obtained. The fire risk level is comprehensively classified based on the vegetation type, season, recent precipitation, and historical fire data of the patrol area, and is pre-stored in the geographic information database of the onboard computing unit. The drone matches the current area's risk level according to real-time GPS coordinates and classifies it into three levels: high fire risk level, medium fire risk level, and low fire risk level. Coniferous forests and shrubland areas, during the spring and autumn fire prevention season, and areas with no precipitation for more than 7 consecutive days are judged as high risk; ordinary grasslands and mixed forests are judged as medium risk; wetlands and areas near water are judged as low risk.

[0096] Based on the risk level of the current region, a corresponding query quantity configuration strategy is matched, and the number of query vector outputs of the three fully connected branches is dynamically adjusted, thereby adjusting the total number of queries in the dynamic query matrix to balance detection accuracy and inference speed.

[0097] When drones patrol high-risk fire areas, priority is given to ensuring detection accuracy by increasing the number of query vectors output for small-scale targets corresponding to fully connected branches. In this embodiment, the high-risk configuration is as follows: the number of queries for small-scale branches is increased from 100 to 150, the number of queries for medium-scale branches remains unchanged at 100, and the number of queries for large-scale branches is adjusted to 50, with the total number of queries remaining at 300. By increasing the number of queries corresponding to small targets, the detection capability of small fire points is enhanced, avoiding missed detections in high-risk areas.

[0098] When the drone is patrolling in an area with a medium fire risk level, it balances accuracy and speed, with each of the three branches outputting 100 query vectors, for a total of 300 queries, thus balancing the detection effect of targets at different scales.

[0099] When the drone is patrolling in a low-fire-risk area, priority is given to ensuring inference speed, reducing the total number of query vectors output from all fully connected branches. In this low-risk configuration, the number of queries for the three branches is adjusted to 50, 50, and 50 respectively, for a total of 150 queries. By reducing the total number of queries, the computational load on the decoder is reduced, inference speed can be improved by approximately 30%, significantly reducing the power consumption of the onboard computing unit and extending the drone's endurance.

[0100] After adjusting the number of queries, a dynamic query matrix of corresponding size is generated based on the adjusted number of query vectors and input into the decoder to execute the subsequent decoding and detection process. The decoder can adapt to query sequences of different lengths without modifying its structure. This embodiment can dynamically adjust the computational load and detection accuracy of the detection model according to the fire risk level of the patrol area, ensuring the detection rate in high-risk areas and improving patrol efficiency in low-risk areas, thus adapting to the differentiated needs of wide-area forest and grassland patrols.

[0101] In some embodiments, this embodiment combines Figure 2 This paper provides a detailed explanation of the complete module composition and full-link data flow of the dynamic query adaptive detection architecture.

[0102] Figure 2 This diagram illustrates a dynamic query-based adaptive fireworks ignition detection architecture. The overall architecture arranges functional modules sequentially from left to right along the data flow direction. All modules are unidirectionally connected, and data flow is unidirectional, with no reverse loops. From left to right, the architecture comprises eight core units: a visible light image input module, a multi-scale appearance feature extraction module, a high-efficiency hybrid encoding module, a three-layer global average pooling unit group, three independent fully connected mapping branches, a query vector concatenation module, a converter decoding and detection module, and a fireworks ignition detection result output module. The specific functions and data flow of each unit are as follows: The first unit is the visible light image input module, located on the far left of the architecture. It serves as the input port for the entire detection architecture, responsible for receiving visible light images of forest and grassland areas collected by drones and outputting the original images to the multi-scale appearance feature extraction module as the input data source for feature extraction.

[0103] The second unit is a multi-scale appearance feature extraction module. It receives the original image from the visible light image input module, performs image preprocessing and multi-layer feature extraction operations, and extracts appearance features at three different scales through a backbone network. The module divides the three-layer multi-scale appearance features into two outputs: one is passed to the efficient hybrid coding module for subsequent encoding and dynamic query generation; the other is directly passed to the converter decoding and detection module as the key-value feature input in the decoding stage.

[0104] The third unit is an efficient hybrid coding module that receives the three-layer appearance features output by the multi-scale appearance feature extraction module. It first completes cross-scale feature fusion through the internal neck coding structure, and then completes global context modeling through the converter coding structure. It outputs the encoded three-layer multi-scale features and passes them to the three-layer global average pooling unit group.

[0105] The fourth unit is a three-layer global average pooling unit group, containing three parallel and independent pooling sub-units, each corresponding to one of the three scales of encoded features. Each sub-unit independently receives the encoded features of its corresponding layer, performs global average pooling on the single-layer two-dimensional features, and generates a one-dimensional semantic vector. The three pooling sub-units output three independent one-dimensional semantic vectors, which are then passed one-to-one to three sets of independent fully connected mapping branches.

[0106] The fifth unit consists of three independent fully connected mapping branches: the first, second, and third independent fully connected mapping branches. These three branches are arranged in parallel, and their network parameters are not shared. Each branch receives a semantic vector at the corresponding scale, passes it through two layers of fully connected non-linear mapping, and outputs a fixed number of query vectors. All query vectors output from the three branches are then passed to the query vector concatenation module.

[0107] The sixth unit is the query vector concatenation module. It receives query vectors from three independent fully connected mapping branches, performs sequential concatenation operations in the sequence dimension, and generates a complete dynamic query matrix. The module then passes the dynamic query matrix to the converter decoding and detection module as the query input for the decoding stage.

[0108] The seventh unit is the converter decoding and detection module, which receives two inputs simultaneously: one is the dynamic query matrix output by the query vector concatenation module, and the other is the multi-scale encoded features directly transmitted by the multi-scale appearance feature extraction module. Internally, the module completes cross-attention interaction through a multi-layer converter decoding structure, and then performs classification and regression inference through a dual-branch detection head, outputting the detection results for all candidate fireworks targets.

[0109] The eighth unit is the smoke and fire detection result output module, which receives the detection results output by the converter decoding detection module, organizes them into standardized smoke and fire detection data output, and provides them for subsequent alarm and handling processes.

[0110] The core innovation of the entire architecture lies in generating a dynamic query matrix in real time through multi-scale encoded features, completely replacing the traditional fixed learnable query and enabling the query mechanism to dynamically adapt to multi-scale smoke targets. The architecture adopts a modular design, with clear responsibilities for each module. The dynamic query generation module is pluggable, compatible with existing converter detection frameworks, and can be directly deployed on various edge computing units and server devices, adapting to the smoke detection needs of forestry and grassland drone patrols.

[0111] Please see Figure 3 As shown, Figure 3This is a schematic diagram of the structure of the forest and grassland fire prevention drone smoke and fire spot detection system 200 based on dynamic query adaptive method provided in this application embodiment. The system 200 is used to execute the steps of the forest and grassland fire prevention drone smoke and fire spot detection method based on dynamic query adaptive method shown in the above embodiments. The system 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, laptop computer, wearable device, or robot.

[0112] like Figure 3 As shown, the forest and grassland fire prevention drone smoke and fire spot detection system 200 based on dynamic query adaptive includes: The image acquisition unit 201 is used to acquire visible light images of forest and grassland areas collected by UAVs, perform size normalization and standardization processing on the images, and extract appearance features at three different scales. The appearance features at three different scales are input into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtain the encoded features at the three corresponding scales. The averaging unit 202 is used to perform global average pooling calculation on the encoded features of each layer to obtain the semantic vectors corresponding to the three scales respectively. The semantic vectors of the three scales are input into three sets of independent two-layer fully connected network branches and mapped to a query embedding space of the same dimension. Each branch outputs a fixed number of query vectors. The result detection unit 203 is used to concatenate the query vectors output by the three branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors; the dynamic query matrix is ​​input to the three-layer encoded feature converter decoder to complete cross-attention interaction, and the detection head outputs the smoke and fire point detection results on the image.

[0113] In some embodiments, the acquisition of visible light images of forest and grassland areas collected by a drone, the performance of image size normalization and standardization processing, and the extraction of appearance features at three different scales include: acquiring aerial images of forest and grassland areas collected by a drone equipped with a visible light imaging device; scaling the image to a preset input size and performing pixel value normalization and channel-specific standardization processing; inputting the processed image into a pre-trained hybrid backbone feature extraction network; and outputting three layers of appearance features corresponding to different downsampling ratios after multi-layer convolution operations and progressive downsampling processing.

[0114] In some embodiments, the step of inputting appearance features of three different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtaining coded features of three corresponding scales, includes: inputting appearance features of three different scales into an efficient hybrid encoder; completing cross-scale information interaction of features at different levels through a bidirectional multi-scale fusion path of the neck coding structure; completing global context information modeling through multi-head self-attention operation of the converter encoder; and outputting coded features of three corresponding scales.

[0115] In some embodiments, performing global average pooling calculation on each layer of encoded features to obtain semantic vectors corresponding to the three scales includes: performing global average pooling operation on each layer of two-dimensional feature map in the three layers of encoded features; calculating the average value of all pixels in each channel of a single-layer feature map; arranging the average values ​​of all channels in channel order to generate a one-dimensional semantic vector for the corresponding layer; and obtaining independent semantic vectors corresponding to the three scales.

[0116] In some embodiments, the step of inputting semantic vectors of the three scales into three sets of independent two-layer fully connected network branches and mapping them to a query embedding space of a unified dimension, with each branch outputting a fixed number of query vectors, includes: inputting semantic vectors of the three scales into three sets of two-layer fully connected network branches with mutually exclusive network parameters; each branch first maps the semantic vectors to a hidden layer space of a preset dimension, performs nonlinear activation processing, and then maps them to a query embedding space of a unified dimension; each branch outputs a fixed number of query vectors, and all query vectors have the same dimension.

[0117] In some embodiments, the step of concatenating the query vectors output by the three branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors includes: concatenating the query vectors output by the three fully connected network branches in the sequence dimension according to the order of their corresponding feature scales from smallest to largest; generating a dynamic query matrix containing all query vectors; and generating all parameters of the dynamic query matrix in real time by calculating the features of the current input image to replace the original fixed learnable query vectors in the detection network.

[0118] In some embodiments, the step of inputting the dynamic query matrix and the three-layer encoded features into the converter decoder to complete cross-attention interaction and output the smoke and fire point detection results on the image through the detection head includes: inputting the dynamic query matrix as the query end input and the three-layer encoded features as the key end input, and inputting them together into the multi-layer converter decoder; the decoder internally completes the information interaction between the query vector and the encoded features through a cross-attention mechanism to optimize the feature representation of the query vector; the optimized query vector is input into the dual-branch detection head, which outputs the category confidence and bounding box position parameters of the smoke and fire targets respectively, to obtain the smoke and fire point detection results on the image.

[0119] In some embodiments, the method further includes: acquiring flight altitude parameters collected in real time by the UAV, and matching a corresponding scale weight strategy according to the flight altitude; when the flight altitude is higher than a preset altitude threshold, increasing the confidence weight of the query vector output by the fully connected branch corresponding to the small scale and decreasing the confidence weight of the query vector output by the fully connected branch corresponding to the large scale; when the flight altitude is lower than the preset altitude threshold, balancing the confidence weights of the query vector output by the three fully connected branches; correcting the category confidence value output by the detection head based on the adjusted confidence weights, and then outputting the final fireworks fire point detection result.

[0120] In some embodiments, the method further includes: for the detected fire point targets, matching the source fully connected branch of the corresponding query vector to determine the scale level of the target; for fire targets determined to be of small scale level, extracting local detail features of the corresponding encoded feature region for secondary feature verification; if the secondary verification result corresponding to the secondary feature verification conforms to the fire feature distribution rules, then retaining the corresponding detection result; if the secondary verification result corresponding to the secondary feature verification does not conform to the fire feature distribution rules, then removing the corresponding detection box.

[0121] In some embodiments, the method further includes: obtaining the fire risk level of the current patrol area of ​​the UAV, configuring a query quantity matching strategy according to the risk level; increasing the number of query vector outputs for small-scale fully connected branches in high fire risk areas to enhance the detection capability of small fire points; reducing the total number of query vector outputs for all fully connected branches in low fire risk areas to improve the inference speed of the edge computing unit; generating a corresponding dynamic query matrix based on the adjusted number of query vectors, and executing subsequent decoding and detection processes.

[0122] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described dynamic query-adaptive forest and grassland fire prevention drone smoke and fire spot detection system and its modules can be referred to the corresponding content in the various embodiments of the above-described dynamic query-adaptive forest and grassland fire prevention drone smoke and fire spot detection method, and will not be repeated here.

[0123] The aforementioned method for detecting smoke and fire points using forest and grassland fire prevention drones based on dynamic query adaptation can be implemented as a computer program, which can be used in various ways, such as... Figure 3 It runs on the system shown.

[0124] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0125] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any dynamic query-based adaptive method for detecting smoke and fire points using forestry and grassland fire prevention drones.

[0126] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0127] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any dynamic query adaptive method for detecting smoke and fire points of forest and grassland fire prevention drones.

[0128] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0130] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Visible light images of forest and grassland areas collected by drones are acquired, and the images are subjected to size normalization and standardization processing to extract appearance features at three different scales. The appearance features at three different scales are then input into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, resulting in encoded features at the three corresponding scales. Global average pooling is performed on the encoded features of each layer to obtain semantic vectors corresponding to the three scales. The semantic vectors of the three scales are then input into three independent two-layer fully connected network branches and mapped to a query embedding space of the same dimension. Each branch outputs a fixed number of query vectors. The query vectors output from the three branches are concatenated along the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors. The dynamic query matrix is ​​then input into the three-layer encoded feature converter decoder to complete cross-attention interaction, and the smoke and fire point detection results are output by the detection head to the image.

[0131] In some embodiments, the acquisition of visible light images of forest and grassland areas collected by a drone, the performance of image size normalization and standardization processing, and the extraction of appearance features at three different scales include: acquiring aerial images of forest and grassland areas collected by a drone equipped with a visible light imaging device; scaling the image to a preset input size and performing pixel value normalization and channel-specific standardization processing; inputting the processed image into a pre-trained hybrid backbone feature extraction network; and outputting three layers of appearance features corresponding to different downsampling ratios after multi-layer convolution operations and progressive downsampling processing.

[0132] In some embodiments, the step of inputting appearance features of three different scales into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, and obtaining coded features of three corresponding scales, includes: inputting appearance features of three different scales into an efficient hybrid encoder; completing cross-scale information interaction of features at different levels through a bidirectional multi-scale fusion path of the neck coding structure; completing global context information modeling through multi-head self-attention operation of the converter encoder; and outputting coded features of three corresponding scales.

[0133] In some embodiments, performing global average pooling calculation on each layer of encoded features to obtain semantic vectors corresponding to the three scales includes: performing global average pooling operation on each layer of two-dimensional feature map in the three layers of encoded features; calculating the average value of all pixels in each channel of a single-layer feature map; arranging the average values ​​of all channels in channel order to generate a one-dimensional semantic vector for the corresponding layer; and obtaining independent semantic vectors corresponding to the three scales.

[0134] In some embodiments, the step of inputting semantic vectors of the three scales into three sets of independent two-layer fully connected network branches and mapping them to a query embedding space of a unified dimension, with each branch outputting a fixed number of query vectors, includes: inputting semantic vectors of the three scales into three sets of two-layer fully connected network branches with mutually exclusive network parameters; each branch first maps the semantic vectors to a hidden layer space of a preset dimension, performs nonlinear activation processing, and then maps them to a query embedding space of a unified dimension; each branch outputs a fixed number of query vectors, and all query vectors have the same dimension.

[0135] In some embodiments, the step of concatenating the query vectors output by the three branches in the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors includes: concatenating the query vectors output by the three fully connected network branches in the sequence dimension according to the order of their corresponding feature scales from smallest to largest; generating a dynamic query matrix containing all query vectors; and generating all parameters of the dynamic query matrix in real time by calculating the features of the current input image to replace the original fixed learnable query vectors in the detection network.

[0136] In some embodiments, the step of inputting the dynamic query matrix and the three-layer encoded features into the converter decoder to complete cross-attention interaction and output the smoke and fire point detection results on the image through the detection head includes: inputting the dynamic query matrix as the query end input and the three-layer encoded features as the key end input, and inputting them together into the multi-layer converter decoder; the decoder internally completes the information interaction between the query vector and the encoded features through a cross-attention mechanism to optimize the feature representation of the query vector; the optimized query vector is input into the dual-branch detection head, which outputs the category confidence and bounding box position parameters of the smoke and fire targets respectively, to obtain the smoke and fire point detection results on the image.

[0137] In some embodiments, the method further includes: acquiring flight altitude parameters collected in real time by the UAV, and matching a corresponding scale weight strategy according to the flight altitude; when the flight altitude is higher than a preset altitude threshold, increasing the confidence weight of the query vector output by the fully connected branch corresponding to the small scale and decreasing the confidence weight of the query vector output by the fully connected branch corresponding to the large scale; when the flight altitude is lower than the preset altitude threshold, balancing the confidence weights of the query vector output by the three fully connected branches; correcting the category confidence value output by the detection head based on the adjusted confidence weights, and then outputting the final fireworks fire point detection result.

[0138] In some embodiments, the method further includes: for the detected fire point targets, matching the source fully connected branch of the corresponding query vector to determine the scale level of the target; for fire targets determined to be of small scale level, extracting local detail features of the corresponding encoded feature region for secondary feature verification; if the secondary verification result corresponding to the secondary feature verification conforms to the fire feature distribution rules, then retaining the corresponding detection result; if the secondary verification result corresponding to the secondary feature verification does not conform to the fire feature distribution rules, then removing the corresponding detection box.

[0139] In some embodiments, the method further includes: obtaining the fire risk level of the current patrol area of ​​the UAV, configuring a query quantity matching strategy according to the risk level; increasing the number of query vector outputs for small-scale fully connected branches in high fire risk areas to enhance the detection capability of small fire points; reducing the total number of query vector outputs for all fully connected branches in low fire risk areas to improve the inference speed of the edge computing unit; generating a corresponding dynamic query matrix based on the adjusted number of query vectors, and executing subsequent decoding and detection processes.

[0140] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the dynamic query adaptive forest and grassland fire prevention drone smoke and fire spot detection method provided in any embodiment of this application.

[0141] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0142] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting smoke and fire points using forest and grassland fire prevention drones based on dynamic query adaptive methods, characterized in that, include: Acquire visible light images of forest and grassland areas collected by drones, perform size normalization and standardization processing on the images, and extract appearance features at three different scales; The appearance features at three different scales are input into an efficient hybrid encoder to complete cross-scale fusion and global context modeling, resulting in encoded features at the corresponding scales of the three layers. Global average pooling is performed on the encoded features of each layer to obtain semantic vectors corresponding to the three scales. The semantic vectors of the three scales are then input into three independent two-layer fully connected network branches and mapped to a query embedding space of the same dimension. Each branch outputs a fixed number of query vectors. The query vectors output from the three branches are concatenated along the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors. The dynamic query matrix is ​​then input into the three-layer encoded feature converter decoder to complete cross-attention interaction, and the smoke and fire point detection results are output by the detection head to the image.

2. The method according to claim 1, characterized in that, The process involves acquiring visible light images of forest and grassland areas collected by a drone, performing size normalization and standardization on the images, and extracting appearance features at three different scales, including: Acquire aerial images of forest and grassland areas taken by drones equipped with visible light imaging devices; The image is scaled to a preset input size, and pixel value normalization and channel-specific normalization are performed. The processed image is then input into a pre-trained hybrid backbone feature extraction network. After multi-layer convolution operations and progressive downsampling, the output consists of three layers of appearance features corresponding to different downsampling ratios.

3. The method according to claim 1, characterized in that, The process involves inputting appearance features from three different scales into a high-efficiency hybrid encoder to complete cross-scale fusion and global context modeling, resulting in encoded features at the corresponding scales of the three layers, including: Input the appearance features of three different scales into the high-efficiency hybrid encoder; A bidirectional, multi-scale fusion path through the neck coding structure enables cross-scale information interaction of features at different levels. The global context information modeling is then completed through multi-head self-attention computation of the converter encoder. Output the encoded features corresponding to the three scales.

4. The method according to claim 1, characterized in that, The process of performing global average pooling on each layer of encoded features yields semantic vectors corresponding to the three scales, including: Perform global average pooling on each of the two-dimensional feature maps in the three-layer coding features; For each channel of a single-layer feature map, calculate the average value of all pixels in that channel; Arrange the average values ​​of all channels in channel order to generate a one-dimensional semantic vector for the corresponding layer; We obtain independent semantic vectors corresponding to each of the three scales.

5. The method according to claim 1, characterized in that, The semantic vectors at three scales are input into three independent two-layer fully connected network branches, mapped to a query embedding space of a unified dimension. Each branch outputs a fixed number of query vectors, including: The semantic vectors at three scales are respectively input into three sets of two-layer fully connected network branches with mutually exclusive network parameters; Each branch first maps the semantic vector to a hidden layer space of a preset dimension, and then maps it to a query embedding space of a unified dimension after non-linear activation processing. Each branch outputs a fixed number of query vectors, and all query vectors maintain the same dimension.

6. The method according to claim 1, characterized in that, The step of concatenating the query vectors output from the three branches along the sequence dimension to generate a dynamic query matrix to replace the fixed learnable query vectors includes: The query vectors output by the three fully connected network branches are concatenated sequentially along the sequence dimension in order of ascending corresponding feature scales. Generate a dynamic query matrix containing all query vectors; All parameters of the dynamic query matrix are calculated and generated in real time from the features of the current input image, replacing the original fixed learnable query vector in the detection network.

7. The method according to claim 1, characterized in that, The process of inputting the dynamic query matrix and the three-layer encoded feature into the converter-decoder to complete cross-attention interaction, and outputting the smoke and fire point detection results on the image through the detection head, includes: The dynamic query matrix is ​​used as the query input, and the three-layer encoded features are used as the key input, both of which are input to the multilayer converter decoder. The decoder internally uses a cross-attention mechanism to facilitate information interaction between the query vector and the encoded features, thereby optimizing the feature representation of the query vector. The optimized query vector is input into the dual-branch detection head, which outputs the category confidence and bounding box position parameters of the smoke target, respectively, to obtain the smoke and fire point detection results on the image.

8. The method according to claim 1, characterized in that, The method further includes: Acquire the flight altitude parameters collected in real time by the UAV, and match the corresponding scale weight strategy according to the flight altitude; When the flight altitude is higher than the preset altitude threshold, the confidence weight of the query vector output by the fully connected branch corresponding to the small scale is increased, and the confidence weight of the query vector output by the fully connected branch corresponding to the large scale is decreased. When the flight altitude is below a preset altitude threshold, the confidence weights of the query vector output by the three fully connected branches are balanced. The category confidence value output by the detection head is corrected based on the adjusted confidence weight, and then the final fire point detection result is output.

9. The method according to claim 1, characterized in that, The method further includes: For the detected fire points, the source fully connected branch of the corresponding query vector is matched to determine the scale level of the target; For fireworks targets classified as small-scale, local detail features of the corresponding coded feature regions are extracted for secondary feature verification. If the secondary verification result corresponding to the secondary feature verification conforms to the firework feature distribution rules, then the corresponding detection result is retained; If the secondary verification result corresponding to the secondary feature verification does not conform to the firework feature distribution rules, the corresponding detection box will be removed.

10. The method according to claim 1, characterized in that, The method further includes: Obtain the fire risk level of the area currently being patrolled by the drone, and match the query quantity configuration strategy according to the risk level; In areas with high fire risk, the number of query vectors output corresponding to fully connected branches at small scales is increased to enhance the detection capability of small fire points; In low-fire-risk areas, the total number of query vectors output for all fully connected branches is reduced, thereby improving the inference speed of edge computing units; Based on the adjusted number of query vectors, a corresponding dynamic query matrix is ​​generated, and the subsequent decoding and detection process is executed.

Images