Forest fire smoke detection and analysis method, system, equipment and medium

Abstract

The invention provides a forest fire smoke detection and analysis method, system and device and a medium, and belongs to the technical field of image detection. The method comprises the following steps: collecting historical videos and pictures of combustion smoke as basic data, preprocessing the basic data, and constructing an image data set based on forest fire smoke; labeling the image data set based on the forest fire smoke by using a yolk labeling tool, generating labeling information, storing the labeling information in a txt label file, and generating a training test set based on the labeling information; building a forest fire smoke detection model based on a YOLOv10-S algorithm model; training the forest fire smoke detection model, and completing the optimization of the model; deploying the optimized forest fire smoke detection model to an edge server on the unmanned aerial vehicle; a video stream is collected in real time through an unmanned aerial vehicle, the video stream is detected through a forest fire smoke detection model, and a final target image of a forest fire smoke target is recognized and sent to a console.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and more specifically relates to a forest fire smoke detection and analysis method, system, equipment and medium. Background Art

[0002] As a highly destructive natural disaster, forest fires pose a great threat to people's lives and property, the natural environment, and the ecological environment due to their rapid spread and unpredictable characteristics. Therefore, early detection and effective treatment of forest fires have become an important issue that needs to be solved urgently. However, existing forest fire prevention and detection methods have certain limitations in practical applications, and a more efficient and accurate forest fire smoke detection and analysis method is urgently needed.

[0003] At present, manual patrol is a traditional means of forest fire detection, but this method is time-consuming, labor-intensive and costly, and is limited by manpower and field of view, making it difficult to achieve large-scale, efficient monitoring. At the same time, although fixed conventional and infrared camera monitoring has improved monitoring efficiency to a certain extent, its coverage is limited. For vast mountainous areas, deep forests and dense forests, a large number of monitoring equipment need to be deployed to achieve a certain degree of coverage, which undoubtedly increases costs. In addition, this method requires manual monitoring, and cannot achieve real-time monitoring, making it difficult to detect fires in a timely manner.

[0004] As a new monitoring method, drone patrol has the advantages of high flexibility and wide coverage, but most drone patrol methods do not carry detection algorithms. They only collect video streams and return them to the machine nest for post-analysis, or require manual synchronization to view the video stream images collected by the drone, which greatly reduces the real-time and accuracy of monitoring. In terms of fire risk detection, in most cases, the fire is only dealt with through manual alarm after it has burned to a certain range, which undoubtedly misses the best time to extinguish the fire.

[0005] In summary, the existing forest fire prevention and detection methods have certain defects in real-time, accuracy, coverage and cost. Therefore, inventing a method that can overcome the above defects and realize efficient, accurate and real-time forest fire smoke detection and analysis is of great significance for improving the early warning ability of forest fire disasters and reducing disaster losses. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a forest fire smoke detection and analysis method, system, equipment and medium. Based on the YOLOv10-S algorithm, real-time detection and analysis of forest fire smoke is performed through a drone, which can more quickly, accurately and objectively identify forest fire smoke and other information, and timely record, report and save the identification results, thereby realizing integrated detection of forest fire smoke and effectively controlling the disaster risk of forest fire smoke.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, an embodiment of the present application provides a forest fire smoke detection and analysis method, comprising: Collect historical videos and pictures of burning smoke as basic data, filter, expand and splice the basic data in turn, and build an image dataset based on forest fire smoke; Use the yolo annotation tool to annotate the image dataset based on forest fire smoke, generate annotation information and store it in a txt label file, and generate a training test set based on the annotation information; Build a forest fire smoke detection model based on the YOLOv10-S algorithm model; The training test set is input into the forest fire smoke detection model. Based on the control variable method, comparative training under different hyperparameters is completed, and the generalization effect of the algorithm is analyzed to select the optimal algorithm version to complete the model optimization. Deploy the optimized forest fire smoke detection model to the edge server on the drone; The video stream of the forest area is collected in real time by the drone, and the video stream is automatically detected through the forest fire smoke detection model; when the forest fire smoke target is detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database.

[0008] In an optional implementation, the basic data are screened, expanded and spliced ​​in sequence to construct an image dataset based on forest fire smoke, including: Based on the basic data, the image data is extracted by sampling frames according to different time, space and angle positions; Based on the image data, image recognition technology is used to filter out image data of non-forest fire scenes; The image data is expanded by using rotation, mirror flipping and target scaling methods; The image data is evenly divided into three groups of data. For each group of data, images with different frame numbers are merged into one frame to complete the image data splicing and generate an image dataset based on forest fire smoke.

[0009] In an optional embodiment, the image dataset based on forest fire smoke is annotated using the yolo annotation tool, annotation information is generated and stored in a txt tag file, and a training test set is generated based on the annotation information, including: Use the yolo annotation tool to annotate the image dataset based on forest fire smoke according to the preset label names and drawing rules; A txt label file is generated for each frame of image; the txt label file stores label information, including label type, horizontal axis coordinates of the center point of the marking frame, vertical axis coordinates of the center point of the marking frame, width of the marking frame, and length of the marking frame.

[0010] In an optional implementation, the forest fire smoke detection model is constructed based on the YOLOv10-S algorithm model, including: Based on the YOLOv10-S algorithm model, the forest fire smoke detection model was constructed using the CDAFN-FT training strategy and HE-ADMD model design method.

[0011] In an optional implementation, the training test set is input into the forest fire smoke detection model, and based on the control variable method, comparative training under different hyperparameters is completed, and the generalization effect of the algorithm is analyzed to select the optimal algorithm version to complete the optimization of the model, including: The training and test sets are divided into training set, validation set, and test set in a ratio of 5:1:1; Based on the training set, the model training is completed by calculating the loss function and updating the model weights through back propagation; During the training process, the validation set is used to evaluate the performance of the model and the parameters are adjusted based on the evaluation results; After training is complete, the final performance of the model is evaluated using the test set; The loss functions include: classification loss function BCE Loss, regression loss function CIoU Loss and data average fidelity loss function DFL Loss; Classification loss function BCE Loss, including:

[0012]

[0013] in, is the predicted value, is the label value, To input batches, For each category, Output probability for this class, is the truth value of the class; The regression loss function CIoU Loss includes:

[0014]

[0015]

[0016] in, is the size of the intersection-union ratio, is the distance between the center point of the real box and the predicted box, is the diagonal distance of the minimum rectangle containing the two boxes, is the weight, Used to measure the consistency of length and width, , are the width and height of the real frame respectively, , is the width and height of the prediction box; Data average fidelity loss function DFL Loss, including:

[0017] in, , for Two nearby predicted values, and .

[0018] In an optional embodiment, an edge server with a length of 103 mm, a width of 90.5 mm, and a height of 34 mm is provided on the belly of the drone.

[0019] In an optional embodiment, deploying the optimized forest fire smoke detection model to an edge server on the drone includes: Install the embedded AI computing platform in the edge server, configure the virtual environment, and use TensorRT tools to convert the optimized forest fire smoke detection model into the TensorRT engine; Stress test the drone and determine the video parameters of the drone's video input source.

[0020] In an optional embodiment, the video stream of the forest area is collected in real time by the drone, and the video stream is automatically detected by the forest fire smoke detection model; when the forest fire smoke target is detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database, including: The video stream of the forest area is collected in real time by drones, and one frame of image is extracted from the video stream every five frames, and sent to the forest fire smoke detection model; the forest fire smoke detection model performs image detection in a detection cycle of 10 seconds, and initializes the detection flag flag=0 and the target frame number num=0; after the detection is completed, the forest fire confidence or smoke confidence is generated; When the forest fire confidence is greater than the first confidence threshold or the smoke confidence is greater than the second confidence threshold, the corresponding image is a forest fire smoke target, and the detection flag flag=1 is set, and the target frame number num is accumulated and calculated; When the forest fire smoke detection model detects 20 frames of images as forest fire smoke targets in a detection cycle, the drone is controlled to automatically track the forest fire smoke target and set the detection flag flag=1 and the target frame number num=0; when the target frame number num in a subsequent detection cycle accumulates to 20, the noise reduction algorithm is automatically called to perform noise reduction on the image, and the detection flag flag=3 and the target frame number num=0 are set; when the target frame number num in a subsequent detection cycle accumulates to 15, the image of the 15th frame of the forest fire smoke target is sent to the console; The current drone's latitude and longitude, camera ID, target label, camera IP, and detection time records are saved in the database.

[0021] In a second aspect, the present application embodiment further provides a forest fire smoke detection and analysis system, including: The dataset construction module collects historical videos and pictures of burning smoke as basic data, and then performs data screening, data expansion and data splicing on the basic data to construct an image dataset based on forest fire smoke. The data annotation module is used to annotate the image dataset based on forest fire smoke using the yolo annotation tool, generate annotation information and store it in a txt tag file, and generate a training test set based on the annotation information; Model building module, used to build a forest fire smoke detection model based on the YOLOv10-S algorithm model; The model training module is used to input the training test set into the forest fire smoke detection model, complete comparative training under different hyperparameters based on the control variable method, analyze the generalization effect of the algorithm, and select the optimal algorithm version to complete the model optimization; A deployment module, used to deploy the optimized forest fire smoke detection model to the edge service detector on the drone; The flight detection and data storage module is used to collect video streams of forest areas in real time through drones, and automatically detect the video streams through forest fire smoke detection models; when forest fire smoke targets are detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database.

[0022] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the steps of the forest fire smoke detection and analysis method as described in any one of the above items are implemented.

[0023] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the forest fire smoke detection and analysis method as described in any one of the above items are implemented.

[0024] It can be seen from the above technical solutions that the present invention has the following advantages: In the forest fire smoke detection and analysis method provided by the present application, the algorithm server is integrated with the hardware of the drone, the drone data is collected, and the video frame is extracted to the YOLOv10-S algorithm, and the algorithm is used to detect and analyze whether there is fire point smoke information. If there is, the alarm information is transmitted to the console for analysis by relevant personnel. To a certain extent, the present invention solves the problem of integrated detection of forest fire smoke, effectively controls the disaster risks such as forest fire smoke, and also saves manpower and material resources of relevant departments.

[0025] The training and testing data for this application are derived from actual data sets of real forest fires, and the data expansion process meets the algorithm requirements and the degree of fit is the same as that of the on-site data, providing a basic guarantee for the stability of the algorithm detection results.

[0026] In terms of model accuracy and performance, this application uses the latest version of the YOLOv10-S lightweight algorithm in combination with the actual situation of drones. Based on its NMS-free design, the inference speed of each frame of image is saved by 5ms-5.5ms. Combined with the PSA structure design, important weights are preserved, which deepens its accuracy.

[0027] This application combines the drone with the NX board, avoiding the process of first collecting data locally and then processing it, and realizes processing on the side of border inspection, which is more timely.

[0028] This application can discover, detect, warn and save targets earlier, faster and better, effectively reducing the hazards after the occurrence of fire risks and providing a better solution for urban safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0030] Figure 1 A schematic diagram of the process flow of the forest fire smoke detection and analysis method provided in this application.

[0031] Figure 2 A schematic diagram of the structure of the forest fire smoke detection and analysis system provided in this application.

[0032] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0033] In the specific steps of the forest fire smoke detection and analysis method described in detail below, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0034] Hereinafter, the terms "include" or "may include" used in various embodiments of the present disclosure indicate the presence of disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. In addition, as used in various embodiments of the present disclosure, the terms "include", "have", and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components, or a combination of the foregoing, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components, or a combination of the foregoing or the possibility of adding one or more features, numbers, steps, operations, elements, components, or a combination of the foregoing.

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1 The figure is a flow chart of a method for forest fire smoke detection and analysis in a specific embodiment, the method comprising: S1: Collect historical videos and pictures of burning smoke as basic data, and then filter, expand and splice the basic data in turn to build an image dataset based on forest fire smoke.

[0037] In a specific implementation, firstly, the image data is extracted based on the basic data according to different time and space and different angle positions. Then, the image data is screened by image recognition technology to filter out the image data of non-forest fire scenes. At this time, the image data is expanded by rotation, mirror flipping and target scaling. Finally, the image data is evenly divided into three groups of data. For each group of data, the images with different frame numbers are merged into one frame of image to complete the splicing of the image data, and an image data set based on forest fire smoke is generated.

[0038] For example, we build an image dataset based on forest fire smoke. In terms of data collection, first, we use historical videos, pictures and other data provided by forest land as primary data; second, we collect public datasets, including videos and images. We use mobile phone data as basic data.

[0039] In terms of data optimization, first, the basic data obtained are analyzed, and image data are extracted by frame sampling according to their different time and space and different angle positions. Under the same video stream, the corresponding images are screened out based on the fact that there is at least an angle change of about 10° in any direction such as up, down, left, and right between images; secondly, the collected basic data are further screened to filter out non-forest fire scenes such as building burning and car burning, so as to select data that fits the actual application scenario as much as possible.

[0040] In terms of data expansion, firstly, based on the 360° monitoring characteristics of the drone and its flight altitude requirements, the pixel value size, location and other characteristics of the currently collected data are analyzed to expand the effective data volume in a targeted manner to increase its generalizability; secondly, in terms of data expansion methods, rotation (rotate the data to the left every 20°, for a total of 17 times), mirror flipping (flipping once), target scaling (if the target pixel is larger than 200*200, use 1.5 times and 0.5 times; if the target pixel is less than 75*75 and larger than 30*30, use 1.5 times and 2 times), and pixels smaller than 30*30 are filtered and deleted.

[0041] The processed smoke data was expanded by data splicing. Specifically, one-third of the smoke data was randomly extracted for data splicing, and the splicing method was to combine every 2 frames of images into 1 frame. Then, one-third of the forest fire data was randomly extracted for data splicing, and the splicing method was to combine every 3 frames of images into 1 frame. Finally, one-third of the forest fire and smoke data were randomly extracted for data splicing, and the splicing method was to combine every 4 frames of images into 1 frame. In this step, a total of 12,368 valid and available data were collected and expanded.

[0042] S2: Use the yolo annotation tool to annotate the forest fire smoke image dataset, generate annotation information and store it in a txt label file, and generate a training test set based on the annotation information.

[0043] In a specific implementation, the yolo annotation tool is first used to annotate the image dataset based on forest fire smoke according to preset label names and drawing rules; then a txt label file is generated for each frame of the image.

[0044] For example, the yolo annotation tool is used to annotate the image dataset based on forest fire smoke according to the preset label name and drawing rules, and the annotation results are stored in a txt folder. Each frame of the image corresponds to a label file, which stores the corresponding normalized annotation information, including five types of information: label type, horizontal axis coordinates of the center point of the marking box, vertical axis coordinates of the center point of the marking box, width of the marking box, and length of the marking box.

[0045] S3: Build a forest fire smoke detection model based on the YOLOv10-S algorithm model.

[0046] In a specific implementation, based on the YOLOv10-S algorithm model, the CDAFN-FT training strategy and the HE-ADMD model design method are used to construct a forest fire smoke detection model.

[0047] It should be noted that the common detection algorithms currently include two-stage algorithms such as RCNN and SSD series, cascade algorithms based on weak classifier integration, and end-to-end single-stage YOLO series. Based on the power consumption, computing power and other properties of the hardware equipment, and under the premise of ensuring that the detection accuracy meets the requirements, combined with the detection timeliness, this method chooses the end-to-end YOLO series algorithm. In the actual deployment stage, due to the heavy reliance on computing power performance, this method chooses the YOLOv10-S algorithm without NMS detection steps, adding a local self-attention module, and with fewer convolutional structures.

[0048] The core methods of the YOLOv10-S algorithm are CDAFN-FT (a consistent dual-assignment training strategy without NMS) and HE-ADMD (efficiency-accuracy driven model design). The details are as follows: 1. CDAFN-FT strategy. The specific principles are as follows: 1.1 Dual label assignments: One-to-one matching: A predicted box is assigned to a real object label, avoiding NMS, but resulting in weak supervision, resulting in poor model accuracy and affecting convergence speed.

[0049] One-to-many matching: This can solve the problem of one-to-one matching strategy. During the training process, multiple prediction boxes are assigned to one real object label. This strategy provides rich supervision signals and better optimization effect.

[0050] Dual-head architecture: The model uses two prediction heads during training, one using one-to-many assignment and the other using one-to-one assignment. With this architecture, the model can leverage the rich supervisory signals of one-to-many assignment during training and use the prediction results of one-to-one assignment during inference, thus achieving efficient inference without NMS.

[0051] 1.2 Consistent matching metric: In the actual allocation process, the one-to-one and one-to-many methods use indicators to quantitatively evaluate the consistency between predictions and instances, thereby achieving prediction-aware matching of the two branches. The unified matching metric is calculated as follows:

[0052] in, is the classification score, Denote the bounding boxes of predictions and instances, respectively. Represents the spatial prior, indicating whether the predicted anchor point is within the instance. are two hyperparameters that balance the impact of semantic prediction tasks and position regression tasks. The one-to-many and one-to-one indicators can be expressed as and , and their matching scores are and , IOU is used The supervision gap relationship between the two branches can be obtained by the 1-Wasserstein distance of different classification targets, as follows:

[0053] in, is the number of predictions selected, The score size is , is the set of all predicted target positive samples, k is Central Africa Collection Medium data; For one-to-one allocation The target value on For one-to-many assignment The target value on When the value increases, the gap between branches decreases, that is, exist rank higher in = When A reaches its minimum value, for The best positive sample in and , the consistent matching metric is satisfied.

[0054] 2. HE-ADMD model. The specific principles are as follows: 2.1 Efficiency-driven model design: Lightweight classification head: This part consists of two depth-wise separable convolutions with a kernel size of 3×3, followed by a 1×1 convolution, which reduces the computational overhead by simplifying the classification head architecture without affecting the performance.

[0055] Spatial-channel decoupled downsampling: During the downsampling process, the channel dimension is first modulated by point-wise convolution, and then deconvolution is used to perform spatial downsampling; decoupling the spatial downsampling and channel conversion operations maximizes the information retention rate during the downsampling process, thereby reducing latency.

[0056] 2.2 Precision-driven model design: Large-kernel deep convolution module: Use large-kernel convolution in the deep stage and increase the kernel size of the second 3×3 deep convolution in CIB to 7×7 to expand the receptive field and enhance model capabilities; use structure reparameterization technology to branch another 3×3 deep convolution to avoid inference overhead.

[0057] Partial Self-Attention (PSA) design: First, the channel features are evenly divided into two parts, one of which is processed by the NPSA module composed of MHSA and FFN and then fused with the other part; second, the dimensions of query and key are assigned to half of the MHSA median, and LayerNorm is replaced by BatchNorm to improve the inference speed; third, the PSA module is placed after stage four with the lowest resolution, effectively incorporating the global representation learning capability.

[0058] S4: Input the training test set into the forest fire smoke detection model, complete comparative training under different hyperparameters based on the control variable method, analyze the generalization effect of the algorithm, and select the optimal algorithm version to complete the model optimization.

[0059] In a specific implementation, the training test set is first divided into a training set, a validation set, and a test set in a ratio of 5:1:1; then, based on the training set, the model weights are updated by calculating the loss function and back-propagating to complete the model training. During the training process, the validation set is used to evaluate the performance of the model, and the parameters are adjusted according to the evaluation results; after the training is completed, the test set is used to evaluate the final performance of the model.

[0060] For example, first, divide the number of labeled data in the training test set into training set, validation set, and test set in a ratio of 5:1:1, and specify their relative positions in data.yaml for algorithm loading; secondly, select the GPU model, epoch number, batch, imgsz image size, loss function selection, etc. for initialization during training; thirdly, run the training algorithm based on the two RTX4090GPUs in the edge server on the drone, and calculate the loss function and back-propagate the updated weights through the backbone network structures such as downsampling, C2f, SCDown, C2fCIB, SPPF, PSA, and Upsample.

[0061] Among them, the loss functions include: classification loss function BCE Loss, regression loss function CIoU Loss and data average fidelity loss function DFL Loss; Classification loss function BCE Loss, including:

[0062]

[0063] in, is the predicted value, is the label value, To input batches, For each category, Output probability for this class, is the truth value of the class; The regression loss function CIoU Loss includes:

[0064]

[0065]

[0066] in, is the size of the intersection-union ratio, is the distance between the center point of the real box and the predicted box, is the diagonal distance of the minimum rectangle containing the two boxes, is the weight, Used to measure the consistency of length and width, , are the width and height of the real frame respectively, , is the width and height of the prediction box; Data average fidelity loss function DFL Loss, including:

[0067] in, , for Two nearby predicted values, and .

[0068] After comprehensive testing and comparison, this step selects epoch=150, batch-size=8, img-size=640*640, optimizer=BGD and other related basic designs.

[0069] S5: Deploy the optimized forest fire smoke detection model to the edge server on the drone.

[0070] In the specific implementation method, this method uses the integration of hardware detection equipment such as drone algorithm servers and the analysis of the characteristics of the mountain to be detected. First, according to the current drone load and hardware structure, an edge server with a length of 103mm, a width of 90.5mm, and a height of 34mm is selected and placed on the belly of the drone, and fastened with screws, connected to the power supply equipment, and its flight altitude and the clarity and frame rate under normal reading of the video stream are tested; secondly, in line with one mountain, one strategy, various situations such as mountain height, slope, fire risk probability point, current visual monitoring position, etc. are analyzed, and on the basis of standardization and checking for deficiencies and filling in the gaps, the relatively optimal solution for the drone flight route is determined.

[0071] When deploying the forest fire smoke detection model, first, configure the Ubuntu environment under NX to upgrade to version 20.04, install Minconda3, create a python3.10 version virtual environment, and install relevant plug-ins such as torch and top in the virtual environment; secondly, the forest fire smoke detection model is lightweighted by tensorRT and ported to the corresponding folder, and stress tests are performed on the flight stability of the drone, the frame rate clarity and stability of the video stream. While ensuring the detection accuracy, in order to meet the timeliness requirements, the drone video stream input source is set to 29fps at 1080p after comparative analysis.

[0072] S6: The video stream of the forest area is collected in real time by the drone, and the video stream is automatically detected by the forest fire smoke detection model; when the forest fire smoke target is detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database.

[0073] In a specific implementation, first, a video stream of the forest area is collected in real time by a drone, and one frame of image is extracted from the video stream every 5 frames and sent to a forest fire smoke detection model; the forest fire smoke detection model performs image detection with a detection cycle of every 10 seconds, and initializes the detection flag flag=0 and the target frame number num=0; after the detection is completed, a forest fire confidence or smoke confidence is generated.

[0074] When the forest fire confidence is greater than the first confidence threshold or the smoke confidence is greater than the second confidence threshold, the corresponding image is a forest fire smoke target, and the detection flag flag=1 is set, and the target frame number num is accumulated and calculated.

[0075] When the forest fire smoke detection model detects 20 frames of images as forest fire smoke targets within a detection cycle, the drone is controlled to automatically track the forest fire smoke target, and the detection flag flag=1 and the target frame number num=0 are set; when the target frame number num accumulates to 20 in a subsequent detection cycle, the noise reduction algorithm is automatically called to perform noise reduction on the image, and the detection flag flag=3 and the target frame number num=0 are set; when the target frame number num accumulates to 15 in a subsequent detection cycle, the image of the 15th frame of the forest fire smoke target is sent to the console.

[0076] Finally, the latitude and longitude of the current drone, camera ID, target label, camera IP, and detection time are recorded and saved in the database.

[0077] For example, during the initial drone flight altitude climbing to a certain height and flying around at a constant speed for detection, the video stream will extract 1 frame out of every 5 frames for forest fire smoke detection model detection. To avoid false detection, the algorithm will use 10 seconds as a detection cycle, and initialize the detection flag flag=0, the target frame number num=0, and the detection standard is that the forest fire confidence is greater than 0.93 and the smoke confidence is greater than 0.90.

[0078] When the forest fire smoke detection model detects a forest fire smoke target, the detection flag flag is set to 1, and the target frame number num is accumulated. When 20 frames of images are detected as forest fire smoke targets within 10 seconds, the drone will automatically track the target and move 5m-10m in the direction of the target. At this time, the detection flag flag is set to 2, and the target frame number num is set to 0; when the target frame number num accumulates to 20 within the 10-second detection cycle, the noise reduction algorithm is automatically called to avoid interference from factors such as clouds and fog. At this time, set flag to 3, and the target frame number num is set to 0.

[0079] In the next step, the video acquisition image must first be subjected to a noise reduction algorithm to remove interference. The denoised image enters the forest fire smoke detection model running the YOLOv10-S detection algorithm. When the target frame number num accumulates to 15 within a 10-second period, it can be considered that the drone has detected the forest fire smoke target. At the same time, on the one hand, the result of the 15th frame of the target image detected when flag=3, num=15 is transmitted to the console. On the other hand, the longitude and latitude of the drone, camera id, target label label, camera ip, detection time and other records are saved in the Hangao database for relevant personnel to check; at this time, the drone will stay at this location for 3 minutes by default, and will continue to push the target image and its results detected in the 20th frame every ten seconds to the database, and the staff will further decide the flight status of the drone. If the staff does not operate the drone within 3 minutes, the drone will return to the initial altitude and continue to patrol along the established route.

[0080] In this embodiment, through a series of scientific and rigorous steps, accurate detection and efficient response to forest fire smoke are achieved. First, by collecting and processing historical video and image data, an image dataset based on forest fire smoke is constructed, which provides a solid foundation for subsequent detection work. Then, the image dataset is finely annotated using the advanced YOLO annotation tool, and a forest fire smoke detection model is constructed based on the YOLOv10-S algorithm model. Through comparative training and algorithm generalization effect analysis, the optimal algorithm version is selected to further optimize the model performance. Finally, the optimized detection model is deployed in the edge server on the drone to achieve real-time video stream detection of the forest area. When a forest fire smoke target is detected, the drone can quickly execute the tracking and noise reduction detection process, collect and send the final target image to the console, and record the drone equipment information to the database, which provides strong support for subsequent analysis and management. This method not only improves the detection accuracy and response speed of forest fire smoke, but also helps to timely warn and respond to forest fires, which is of great significance to the protection of forest resources.

[0081] like Figure 2 As shown, the following is an embodiment of the forest fire smoke detection and analysis system provided by the embodiment of the present disclosure. The system and the forest fire smoke detection and analysis methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the forest fire smoke detection and analysis system, please refer to the embodiment of the above-mentioned forest fire smoke detection and analysis method.

[0082] A forest fire smoke detection and analysis system comprises: a data set construction module 1, a data annotation module 2, a model construction module 3, a model training module 4, a deployment module 5 and a flight detection and data storage module 6.

[0083] Dataset construction module 1 collects historical videos and pictures of burning smoke as basic data, and then screens, expands and splices the basic data in turn to build an image dataset based on forest fire smoke.

[0084] Data annotation module 2 is used to use the yolo annotation tool to annotate the image dataset based on forest fire smoke, generate annotation information and store it in a txt label file, and generate a training test set based on the annotation information.

[0085] Model building module 3 is used to build a forest fire smoke detection model based on the YOLOv10-S algorithm model.

[0086] Model training module 4 is used to input the training test set into the forest fire smoke detection model, complete comparative training under different hyperparameters based on the control variable method, analyze the generalization effect of the algorithm, and select the optimal algorithm version to complete the optimization of the model.

[0087] Deployment module 5 is used to deploy the optimized forest fire smoke detection model to the edge service detector on the drone.

[0088] The flight detection and data storage module 6 is used to collect the video stream of the forest area in real time through the UAV, and automatically detect the video stream through the forest fire smoke detection model; when the forest fire smoke target is detected, the UAV is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the UAV is stored in the database.

[0089] The forest fire smoke detection and analysis system provided in this embodiment constructs an image dataset based on historical data, optimizes the detection model using YOLO annotation and YOLOv10-S algorithm model, and is deployed on a drone edge server to achieve real-time detection and tracking. This effectively improves the accuracy and timeliness of forest fire smoke detection and provides strong support for rapid response and disposal of forest fires.

[0090] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0091] The forest fire smoke detection and analysis method provided in the embodiment of the present application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiment of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components. In the embodiment of the present invention, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0092] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, buttons, a camera, a display, and a SIM card interface, etc.

[0093] The processor may include one or more processing units, for example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors.

[0094] The processor can be the nerve center and command center of the electronic device. The controller can generate an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0095] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or is cyclically used. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves system efficiency.

[0096] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to implement data storage functions. For example, files such as music and videos can be saved in the external memory card.

[0097] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The internal memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0098] The wireless communication function of an electronic device can be realized through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0099] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0100] Electronic devices can implement audio functions, etc. through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0101] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0102] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0103] The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0104] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0105] The above-mentioned electronic device implements the forest fire smoke detection and analysis method of the present application by constructing a professional image data set, optimizing it using the YOLOv10-S algorithm model, and deploying it on a drone for real-time detection, thereby effectively improving the accuracy and efficiency of forest fire smoke detection, achieving rapid response and precise positioning, and having a beneficial effect on early warning and prevention and control of forest fires.

[0106] The storage medium provided in the present application stores a program product that can implement a forest fire smoke detection and analysis method.

[0107] The forest fire smoke detection and analysis method includes: collecting historical videos and pictures of burning smoke as basic data, screening, data expansion and data splicing the basic data in turn, and constructing an image data set based on forest fire smoke; using the yolo annotation tool to annotate the image data set based on forest fire smoke, generate annotation information and store it in a txt tag file, and generate a training test set based on the annotation information; constructing a forest fire smoke detection model based on the YOLOv10-S algorithm model; inputting the training test set into the forest fire smoke detection model, completing comparative training under different hyperparameters based on the control variable method, and analyzing the generalization effect of the algorithm, selecting the optimal algorithm version, and completing the optimization of the model; deploying the optimized forest fire smoke detection model to the edge server on the drone; collecting video streams of the forest area in real time through the drone, and automatically detecting the video streams through the forest fire smoke detection model; when a forest fire smoke target is detected, controlling the drone to execute the tracking and noise reduction detection process, collecting the final target image of the forest fire smoke target and sending it to the console, and storing the current device information of the drone in the database.

[0108] In some possible implementations, the building forest fire smoke detection and analysis method disclosed herein can be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above "Exemplary Method" section of this specification according to various exemplary implementations of the present disclosure.

[0109] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0110] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A forest fire smoke detection and analysis method, characterized in that: include: Collect historical videos and pictures of burning smoke as basic data, filter, expand and splice the basic data in turn, and build an image dataset based on forest fire smoke; Use the yolo annotation tool to annotate the image dataset based on forest fire smoke, generate annotation information and store it in a txt label file, and generate a training test set based on the annotation information; Construct a forest fire smoke detection model based on the YOLOv10-S algorithm model; The training test set is input into the forest fire smoke detection model. Based on the control variable method, comparative training under different hyperparameters is completed, and the generalization effect of the algorithm is analyzed to select the optimal algorithm version to complete the model optimization. Deploy the optimized forest fire smoke detection model to the edge server on the drone; Collecting video streams of forest areas in real time through drones, and automatically detecting the video streams through a forest fire smoke detection model; When a forest fire smoke target is detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database.

2. The forest fire smoke detection and analysis method according to claim 1, characterized in that: The basic data are screened, expanded and spliced ​​in sequence to construct an image dataset based on forest fire smoke, including: Based on the basic data, the image data is extracted by sampling frames according to different time, space and angle positions; Based on the image data, image recognition technology is used to filter out image data of non-forest fire scenes; The image data is expanded by using rotation, mirror flipping and target scaling methods; The image data is evenly divided into three groups of data. For each group of data, images with different frame numbers are merged into one frame to complete the image data splicing and generate an image dataset based on forest fire smoke.

3. The forest fire smoke detection and analysis method according to claim 2, characterized in that: The method uses the yolo annotation tool to annotate the image dataset based on forest fire smoke, generates annotation information and stores it in a txt tag file, and generates a training test set based on the annotation information, including: Use the yolo annotation tool to annotate the image dataset based on forest fire smoke according to the preset label names and drawing rules; A txt label file is generated for each frame of image; the txt label file stores label information, including label type, horizontal axis coordinates of the center point of the marking frame, vertical axis coordinates of the center point of the marking frame, width of the marking frame, and length of the marking frame.

4. The forest fire smoke detection and analysis method according to claim 1, characterized in that: The forest fire smoke detection model is constructed based on the YOLOv10-S algorithm model, including: Based on the YOLOv10-S algorithm model, the forest fire smoke detection model was constructed using the CDAFN-FT training strategy and HE-ADMD model design method.

5. The forest fire smoke detection and analysis method according to claim 3, characterized in that: The training test set is input into the forest fire smoke detection model, and based on the control variable method, comparative training under different hyperparameters is completed, and the generalization effect of the algorithm is analyzed to select the optimal algorithm version to complete the optimization of the model, including: The training and test sets are divided into training set, validation set, and test set in a ratio of 5:1:1; Based on the training set, the model training is completed by calculating the loss function and back-propagating to update the model weights; During the training process, the validation set is used to evaluate the performance of the model and the parameters are adjusted based on the evaluation results; After training is complete, the final performance of the model is evaluated using the test set; The loss functions include: classification loss function BCE Loss, regression loss function CIoU Loss and data average fidelity loss function DFL Loss; Classification loss function BCE Loss, including: in, is the predicted value, is the label value, To input batches, For each category, Output probability for this class, is the truth value of the class; The regression loss function CIoU Loss includes: in, is the size of the intersection-union ratio, is the distance between the center point of the real box and the predicted box, is the diagonal distance of the minimum rectangle containing the two boxes, is the weight, Used to measure the consistency of length and width, , are the width and height of the real frame respectively, , is the width and height of the prediction box; Data average fidelity loss function DFL Loss, including: in, , for Two nearby predicted values, and .

6. The forest fire smoke detection and analysis method according to claim 5, characterized in that: The belly of the drone is provided with an edge server with a length of 103 mm, a width of 90.5 mm, and a height of 34 mm; The optimized forest fire smoke detection model is deployed to the edge server on the drone, including: Install the embedded AI computing platform in the edge server, configure the virtual environment, and use TensorRT tools to convert the optimized forest fire smoke detection model into the TensorRT engine; Stress test the drone and determine the video parameters of the drone's video input source.

7. The forest fire smoke detection and analysis method according to claim 6, characterized in that: The video stream of the forest area is collected in real time by the drone, and the video stream is automatically detected by the forest fire smoke detection model; When a forest fire smoke target is detected, the drone is controlled to execute the tracking and noise reduction detection process, collect the final target image of the forest fire smoke target and send it to the console, and store the current device information of the drone in the database, including: The video stream of the forest area is collected in real time by drones, and one frame of image is extracted from the video stream every five frames, and sent to the forest fire smoke detection model; the forest fire smoke detection model performs image detection in a detection cycle of 10 seconds, and initializes the detection flag flag=0 and the target frame number num=0; after the detection is completed, the forest fire confidence or smoke confidence is generated; When the forest fire confidence is greater than the first confidence threshold or the smoke confidence is greater than the second confidence threshold, the corresponding image is a forest fire smoke target, and the detection flag flag=1 is set, and the target frame number num is accumulated and calculated; When the forest fire smoke detection model detects 20 frames of images as forest fire smoke targets in a detection cycle, the drone is controlled to automatically track the forest fire smoke target and set the detection flag flag=1 and the target frame number num=0; when the target frame number num in a subsequent detection cycle accumulates to 20, the noise reduction algorithm is automatically called to perform noise reduction on the image, and the detection flag flag=3 and the target frame number num=0 are set; when the target frame number num in a subsequent detection cycle accumulates to 15, the image of the 15th frame of the forest fire smoke target is sent to the console; The current drone's latitude and longitude, camera ID, target label, camera IP, and detection time records are saved in the database.

8. A forest fire smoke detection and analysis system, characterized in that: The system adopts the forest fire smoke detection and analysis method as claimed in any one of claims 1 to 7; The system comprises: The dataset construction module collects historical videos and pictures of burning smoke as basic data, and then performs data screening, data expansion and data splicing on the basic data to construct an image dataset based on forest fire smoke. The data annotation module is used to annotate the image dataset based on forest fire smoke using the yolo annotation tool, generate annotation information and store it in a txt tag file, and generate a training test set based on the annotation information; Model building module, used to build a forest fire smoke detection model based on the YOLOv10-S algorithm model; The model training module is used to input the training test set into the forest fire smoke detection model, complete comparative training under different hyperparameters based on the control variable method, analyze the generalization effect of the algorithm, and select the optimal algorithm version to complete the model optimization; A deployment module, used to deploy the optimized forest fire smoke detection model to the edge service detector on the drone; The flight detection and data storage module is used to collect video streams of forest areas in real time through drones, and automatically detect the video streams through forest fire smoke detection models; when forest fire smoke targets are detected, the drone is controlled to execute the tracking and noise reduction detection process, the final target image of the forest fire smoke target is collected and sent to the console, and the current device information of the drone is stored in the database.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the forest fire smoke detection and analysis method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the forest fire smoke detection and analysis method according to any one of claims 1 to 7 are implemented.

Images