Forest fire identification and positioning method and system based on fine-grained slices and depth estimation

Abstract

The invention belongs to the technical field of forest fire recognition, detection and positioning, and provides a forest fire recognition and positioning method and system based on fine-grained slices and depth estimation, and the method comprises the steps: preprocessing an input image, and obtaining a processed image; segmenting the processed image, and removing a sky region to obtain a segmented image; the sliding window divides the cut image into a plurality of slices; the YOLO target detection model identifies a forest fire sign of each slice and records an identification result; converting an identification result into an original image coordinate system; restoring the sky area of the cut image, and drawing an identification result to obtain a restored image; estimating a depth map of the processed image, and identifying a forest fire target coordinate to obtain a corresponding depth value; and reversely solving the longitude and latitude positioning of the forest fire according to the forest fire target coordinates. According to the invention, the detection capability of a small smoke target is enhanced, forest fire positioning is realized, the forest fire early warning detection precision is improved, the missing detection rate and the false detection rate are effectively reduced, and the manpower resource pressure is effectively reduced.

Description

Technical Field

[0001] This invention belongs to the field of wildfire identification, detection and location technology, specifically, it relates to a wildfire identification and location method and system based on fine-grained slicing and depth estimation. Background Technology

[0002] Traditional wildfire monitoring relies mainly on manual analysis of camera videos to determine the location of fires, but this method is difficult to cope with the frequent occurrence of fires.

[0003] Sensor-based detection methods determine the occurrence of wildfires by deploying sensor networks, such as temperature and smoke sensors, throughout the forest to monitor changes in environmental parameters. The advantage is the ability to monitor environmental parameters in real time, making it suitable for large-area monitoring. However, sensor deployment is costly, and these sensors are susceptible to environmental interference, leading to a high false alarm rate. In practical applications, the high cost and high false alarm rate of traditional sensor methods limit their widespread use in complex natural environments, especially in large-area monitoring areas such as forests, where the maintenance and replacement costs of sensors are significant.

[0004] AI-based visual detection methods utilize cameras to capture images and employ advanced target detection algorithms to identify wildfire signs within those images. Their rapid response and high accuracy make them particularly suitable for wildfire monitoring in complex environments. However, while computer vision-based wildfire detection methods are relatively mature in identifying visible flames, they still have significant shortcomings in early warning systems and exhibit clear limitations when handling complex backgrounds. For example, these methods often show high false detection rates when faced with interference from the sky and challenges arising from variations in shooting angle and lighting conditions. In data captured by outdoor cameras, tiny white clouds are highly similar to smoke, leading to a high false detection rate. When detecting small-target wildfires, the small size of the target and the complex background make accurate identification difficult with existing technologies, resulting in a significant decrease in accuracy. For instance, due to the distance between smoke and the ignition point and the small size of the target, existing detection systems often experience missed detections.

[0005] Furthermore, the existing system lacks ranging equipment such as lidar, which also limits its target localization capabilities. It cannot directly obtain the distance to the target or provide specific geographic coordinates of the wildfire. Obtaining geographic location is crucial for timely and effective firefighting operations, making this a critical deficiency in actual forest and wildfire prevention work. Fire source location still relies on manual estimation, which further reduces the accuracy and efficiency of monitoring.

[0006] Forest fires occurring inside forests or on the edges of forest areas have high vegetation density, resulting in relatively slow flame propagation, dense smoke, and a relatively stable background. Compared to wildfires, wildfires may involve various terrains and vegetation such as grasslands, shrubs, and woodlands. They spread rapidly, are subject to changeable wind directions, and often occur in complex backgrounds with open views and drastic changes in lighting (such as the sky, clouds, and rocks).

[0007] The patent document "A Fixed-Point Forest Fire Location Method" (CN112257575A) discloses a fixed-point automatic forest fire monitoring platform. This platform extracts minute moving targets through differential calculations between consecutive multi-frame images and background frames, detects flames using a YCbCr color model, and combines a rangefinder and BeiDou positioning module to achieve unmanned forest fire location and acquire latitude, longitude, and altitude data. However, the YCbCr color model used has a high false alarm rate under conditions of long distance, low light, and small size, and lacks processing for sky interference, making it difficult to use in forest fire detection. The "background frame modeling + background difference" method for extracting moving targets is also problematic in forest fire scenarios. Due to factors such as wind blowing grass, cloud shadows, and camera shake, the background changes drastically, easily generating a large number of false moving targets, leading to a sharp increase in the false detection rate.

[0008] In summary, there is a need for an innovative wildfire detection and localization method based on fine-grained slicing and depth estimation to reduce false positives and false negatives and to achieve wildfire localization. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for wildfire identification and location based on fine-grained slicing and depth estimation.

[0010] A wildfire identification and localization method based on fine-grained slicing and depth estimation, provided by the present invention, includes:

[0011] Step S1: Preprocess the input image to obtain the processed image;

[0012] Step S2: Segment the image, remove the sky region, and obtain the segmented image;

[0013] Step S3: Divide the image into multiple slices using a sliding window;

[0014] Step S4: The YOLO object detection model identifies volcanic features in each slice and records the identification results;

[0015] Step S5: Convert the recognition results to the original image coordinate system;

[0016] Step S6: Restore the sky region of the cut image and draw the converted recognition result to obtain the restored image;

[0017] Step S7: Depth estimation: Obtain a depth map from the input image, and locate the coordinates of the wildfire target and its corresponding depth value based on the reconstructed image;

[0018] Step S8: Solve the latitude and longitude of the wildfire based on the target coordinates.

[0019] Preferably, the preprocessing includes adjusting the resolution, normalizing pixel values, and changing the color channel order.

[0020] In step S2, the sky region of the image is segmented and removed using an image segmentation algorithm, and then filled with a set color.

[0021] In step S3, a sliding window mechanism is used to finely divide the cut image into multiple overlapping slices, and the position of each slice is determined according to the coordinates of the upper left corner:

[0022] x i =i·s

[0023] y j =j·s

[0024] Where i and j are unit vectors on the X and Y coordinate axes, respectively;

[0025] x i y j These represent the coordinates of the top-left corner of the slice along the X and Y axes in the original image, respectively.

[0026] s represents the step size.

[0027] Preferably, in step S4, each slice is input into a pre-trained target detection model to perform inference and obtain the recognition result:

[0028] B = {b1, b2, ..., b} n}

[0029] Where B represents the set of all bounding boxes in the recognition result;

[0030] n represents the total number of bounding boxes detected in the slice;

[0031] b represents the identified bounding box.

[0032] Each bounding box b contains:

[0033] b = (x min ,y min ,x max ,y max c)

[0034] Where c represents the confidence score of the bounding box;

[0035] x minx max These represent the minimum and maximum X-axis coordinates of the bounding box, respectively.

[0036] y min y max These represent the minimum and maximum Y-axis coordinates of the bounding box, respectively.

[0037] Sort the bounding boxes in descending order of confidence score, add the bounding box with the highest confidence score to the final result set, and determine the IoU between each bounding box and the bounding box with the highest confidence score.

[0038] If IoU is greater than the set threshold t iou If the IoU is less than or equal to the set threshold t, then remove it. iou If so, then retain it;

[0039] Iterate through all bounding boxes.

[0040] Preferably, in step S5, the corresponding position of the bounding box b in the original image coordinate system is:

[0041] x′ min =x min +x i

[0042] y′ min =y min +y j

[0043] x′ max =x max +x i

[0044] y′ max =y max +y j

[0045] Where, x′ min 、x′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively;

[0046] y′ min y′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively.

[0047] In step S6, the segmented sky region is added back to the processed image, and the bounding box of the recognition result is drawn on the corresponding position of the processed image according to the transformed coordinates.

[0048] In step S7, the coordinates (x, y) of the wildfire target are obtained on the restored image according to the target detection algorithm, and the depth map of the input image is obtained by using the monocular image depth estimation algorithm. The depth value corresponding to the coordinates of the wildfire target is found in the depth map as the distance d from the wildfire target to the camera.

[0049] Preferably, in step S8, based on the camera's intrinsic parameter matrix K and extrinsic parameter matrix, the wildfire target coordinates (x, y) are converted into a direction vector in a spatial coordinate system, and a normalized device coordinate vector is constructed:

[0050]

[0051] Among them, f x f y These represent different focal lengths;

[0052] c x c y These represent the X and Y coordinates of the principal point, respectively.

[0053] Based on the camera's extrinsic parameters, transform the device coordinate vector to the world coordinate system:

[0054] p w =R -1 (p ndc ·dT)

[0055] p w =(X w ,Y w Z w )

[0056] Calculate the azimuth angle θ and the elevation angle α:

[0057] θ = arctan2(Y) w ,X w )

[0058]

[0059] Calculate the projected distance on the horizontal plane:

[0060] d h =d / cos(α)

[0061] Calculate the latitude and longitude of wildfire targets using the Haversine formula:

[0062]

[0063] in, Indicates the camera's location;

[0064] X w Y w Zw These represent the X, Y, and Z axis values ​​of the wildfire target in the world coordinate system, respectively.

[0065] d represents the distance from the wildfire target;

[0066] θ represents the azimuth angle;

[0067] α represents the angle of elevation;

[0068] R 地 This represents the Earth's radius.

[0069] The present invention provides a wildfire identification and location system based on fine-grained slicing and depth estimation, comprising:

[0070] Module M1 preprocesses the input image to obtain the processed image;

[0071] Module M2 segments and processes the image, removing the sky region to obtain the cut image;

[0072] Module M3 uses a sliding window to divide and slice the image into multiple slices;

[0073] Module M4 and the YOLO object detection model identify volcanic features in each slice and record the identification results;

[0074] Module M5 converts the recognition results to the original image coordinate system;

[0075] Module M6 restores the sky region of the cropped image and draws the converted recognition result to obtain the restored image;

[0076] Module M7: Depth estimation. The input image is used to obtain a depth map, and the coordinates and corresponding depth values ​​of the wildfire target are located based on the reconstructed image.

[0077] Module M8: Solve the latitude and longitude of the wildfire from the target coordinates.

[0078] Preferably, the preprocessing includes adjusting the resolution, normalizing pixel values, and changing the color channel order.

[0079] In module M2, the sky region of the processed image is segmented and removed using an image segmentation algorithm, and then filled with a set color.

[0080] In module M3, a sliding window mechanism is used to finely divide the cut image into multiple overlapping slices, and the position of each slice is determined according to the coordinates of the upper left corner:

[0081] x i =i·s

[0082] y j =j·s

[0083] Where i and j are unit vectors on the X and Y coordinate axes, respectively;

[0084] x i y j These represent the coordinates of the top-left corner of the slice along the X and Y axes in the original image, respectively.

[0085] s represents the step size.

[0086] Preferably, in module M4, each slice is input into a pre-trained target detection model to perform inference and obtain the recognition result:

[0087] B = {b1, b2, ..., b} n}

[0088] Where B represents the set of all bounding boxes in the recognition result;

[0089] n represents the total number of bounding boxes detected in the slice;

[0090] b represents the identified bounding box.

[0091] Each bounding box b contains:

[0092] b = (x min ,y min ,x max ,y max c)

[0093] Where c represents the confidence score of the bounding box;

[0094] x min x max These represent the minimum and maximum X-axis coordinates of the bounding box, respectively.

[0095] y min y max These represent the minimum and maximum Y-axis coordinates of the bounding box, respectively.

[0096] Sort the bounding boxes in descending order of confidence score, add the bounding box with the highest confidence score to the final result set, and determine the IoU between each bounding box and the bounding box with the highest confidence score.

[0097] If IoU is greater than the set threshold t iou If the IoU is less than or equal to the set threshold t, then remove it. iou If so, then retain it;

[0098] Iterate through all bounding boxes.

[0099] Preferably, in module M5, the corresponding position of the bounding box b in the original image coordinate system is:

[0100] x′min =x min +x i

[0101] y′ min =y min +y j

[0102] x′ max =x max +x i

[0103] y′ max =y max +y j

[0104] Where, x′ min 、x′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively;

[0105] y′ min y′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively.

[0106] In module M6, the segmented sky region is added back to the processed image, and the bounding box of the recognition result is drawn on the corresponding position of the processed image according to the transformed coordinates.

[0107] In module M7, the coordinates (x, y) of the wildfire target are obtained on the restored image according to the target detection algorithm, and the depth map of the input image is obtained using the monocular image depth estimation algorithm. The depth value corresponding to the coordinates of the wildfire target is found in the depth map as the distance d from the wildfire target to the camera.

[0108] Preferably, in module M8, based on the camera's intrinsic parameter matrix K and extrinsic parameter matrix, the wildfire target coordinates (x, y) are converted into a direction vector in a spatial coordinate system, constructing a normalized device coordinate vector:

[0109]

[0110] Among them, f x f y These represent different focal lengths;

[0111] c x c y These represent the X and Y coordinates of the principal point, respectively.

[0112] Based on the camera's extrinsic parameters, transform the device coordinate vector to the world coordinate system:

[0113] p w =R -1(p ndc ·dT)

[0114] p w =(X w ,Y w Z w )

[0115] Calculate the azimuth angle θ and the elevation angle α:

[0116] θ = arctan2(Y) w ,X w )

[0117]

[0118] Calculate the projected distance on the horizontal plane:

[0119] d h =d / cos(α)

[0120] Calculate the latitude and longitude of wildfire targets using the Haversine formula:

[0121]

[0122] in, Indicates the camera's location;

[0123] X w Y w Z w These represent the X, Y, and Z axis values ​​of the wildfire target in the world coordinate system, respectively.

[0124] d represents the distance from the wildfire target;

[0125] θ represents the azimuth angle;

[0126] α represents the angle of elevation;

[0127] R 地 This represents the Earth's radius.

[0128] Compared with the prior art, the present invention has the following beneficial effects:

[0129] 1. This invention employs a multi-stage collaborative mechanism of "sky region removal + sliding window slicing + YOLO target detection + depth map fusion localization," combining advanced deep learning algorithms with fine-grained slicing technology to optimize existing AI vision algorithms. It particularly enhances the detection capability for small smoke targets, improves the accuracy of wildfire early warning detection, and effectively reduces the rate of missed detections and false detections.

[0130] 2. This invention utilizes a monocular image depth estimation algorithm to obtain distance information of wildfire targets and calculates their geographic coordinates using the Haversine formula, thereby enabling the location of wildfires and more effectively responding to frequent wildfire disasters.

[0131] 3. This invention can automatically determine the location of wildfires by detecting and identifying small targets, estimating depth, and solving latitude and longitude, thereby improving monitoring efficiency and accuracy, effectively reducing the pressure on human resources, and providing strong technical support for timely response to wildfire disasters.

[0132] 4. By clearly defining and eliminating the sky area, this invention fundamentally eliminates major dynamic interference sources such as clouds and cloud shadows, significantly reducing the false detection rate, enabling more accurate identification of actual moving targets, reducing unnecessary alarms, and improving detection stability. Attached Figure Description

[0133] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0134] Figure 1 This is a schematic diagram of the process for wildfire identification and localization based on fine-grained slicing and depth estimation.

[0135] Figure 2 This is a schematic diagram of a wildfire identification and localization system based on fine-grained slicing and depth estimation. Detailed Implementation

[0136] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0137] To reduce the impact of external factors on AI-based visual detection technology for wildfire detection and improve the accuracy of AI-based wildfire detection and localization methods, this invention proposes a wildfire identification and localization method based on fine-grained slicing and depth estimation. Through a multi-stage collaborative mechanism of "sky region removal + sliding window slicing + YOLO target detection + depth map fusion localization," and by using small target detection and identification, depth estimation, and latitude / longitude calculation, it reduces false positives and false negatives, and achieves wildfire localization. This effectively solves the three key problems of existing technologies: high false positive rate, false negatives of small targets, and inability to accurately locate wildfires, thus providing a more effective response to frequent wildfire disasters. Figure 1 For example, including:

[0138] Step S1: Input Image Preprocessing: First, input an RGB format image and preprocess it, including adjusting the resolution, normalizing pixel values, and converting the color channel order to generate an image that is compatible with the deep learning model.

[0139] Step S2, Sky Region Segmentation and Removal: Use image segmentation algorithms to segment the sky region and remove sky interference.

[0140] Specifically, the input image undergoes sky segmentation and removal. After training the segmentation model with a large number of images labeled with sky regions, it can efficiently identify and segment the sky portion of the image, effectively distinguishing the sky background from ground elements, and removing the sky background after obtaining the segmentation results.

[0141] The identified sky areas are filled with a specific color (e.g., black) from the original image, thus avoiding interference from elements such as clouds in the wildfire identification process. Clearly defining and removing the sky area fundamentally eliminates major dynamic interference sources such as clouds and cloud shadows, significantly reducing the false detection rate, improving detection stability, enabling more accurate identification of actual moving targets, and reducing unnecessary alarms.

[0142] Step S3, Image Slicing: The image obtained in step S2 is divided into fine-grained segments using a sliding window mechanism.

[0143] Specifically, in order to improve the detection accuracy of small targets in high-resolution images, the original image first needs to be segmented into multiple overlapping slices.

[0144] Let the original image size be W×H, and the slice size be w. s ×h s The step size is s. The number of slices in the horizontal direction is... The number of slices in the vertical direction is The position of each slice is determined by calculating the coordinates of its top-left corner, specifically:

[0145] x i =i·s

[0146] y j =j·s

[0147] Where i and j are unit vectors on the X and Y coordinate axes, respectively. i y j These represent the coordinates of the top left corner of the slice along the X and Y axes in the original image, respectively.

[0148] Step S4, Wildfire Target Detection: On each segment obtained, apply the YOLO target detection model to identify wildfire signs and record the target's location coordinates and category label.

[0149] A sliding window image slicing method combined with the YOLO object detection model was used to achieve refined local detection, enhancing the ability to perceive small fire points at a distance and effectively solving the problem of missed detection of small targets. Specifically, for each slice, it is input into a pre-trained object detection model to perform inference. The recognition result is a set of n bounding boxes:

[0150] B = {b1, b2, ..., b} n}

[0151] Where B represents the set of all bounding boxes in the recognition results, which is a mathematical set; n represents the total number of bounding boxes detected in the slice; and b represents any one bounding box in the set.

[0152] Each bounding box b contains the following location information:

[0153] b = (x min ,y min ,x max ,y max c)

[0154] Where b represents the identified bounding box, which is an element in the bounding box set B. χ min , χ max These represent the minimum and maximum X-axis coordinates of the bounding box, respectively; y min y max represents the minimum and maximum Y-axis coordinates of the bounding box, respectively. c represents the confidence score of the bounding box, with a value ranging from [0,1], reflecting the model's confidence in the existence of a target within the bounding box.

[0155] The YOLO target detection model based on deep learning is adopted to automatically learn the texture, shape and dynamic evolution features of flames and smoke. Through the deep learning model, it can not only identify the color features of flames, but also capture their complex texture and dynamic changes, which has stronger anti-interference ability and higher recognition accuracy, thereby improving the overall detection performance.

[0156] To remove redundant detection results, the Non-Maximum Suppression (NMS) algorithm is applied. Given an IoU threshold t... iou Sort all bounding boxes in descending order of confidence score, select the bounding box with the highest confidence score to add to the final result set, and remove the bounding box with an IoU greater than t. iou All other bounding boxes.

[0157] Repeat the process until all bounding boxes have been processed. By segmenting the image into blocks, the ability to capture detailed features is improved, ensuring accurate identification of small targets even against complex backgrounds.

[0158] Step S5: Map the recognition results of each slice back to the original image coordinate system through their relative positional relationships to achieve global target localization.

[0159] Specifically, the detection results on each slice are transformed back to the original image coordinate system. For any bounding box b = (x... min ,y min ,x max ,y max Its corresponding position in the original image can be calculated using the following formula:

[0160] x ′ min =x min +x i

[0161] y′ min =y min +y j

[0162] x′ max =x max +x i

[0163] y′ max =y max +y j

[0164] Where, x′ min 、x′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively; y′ min y′ max These represent the minimum and maximum Y-axis coordinates of the bounding box in the original image, respectively.

[0165] Step S6: Restore the sky region: Add the previously segmented sky region back into the original image and draw the detected target boxes in the corresponding positions to complete the labeling of wildfire signs.

[0166] Step S7: Estimate the depth map using a monocular image depth estimation algorithm. Based on the wildfire target coordinates (x, y) on the RGB image obtained in step S5, query the corresponding position on the depth map to obtain the distance value d of the target.

[0167] Visual depth maps are obtained based on monocular depth estimation. The three-dimensional coordinates and latitude and longitude of the target are then solved by combining the pose information of the camera. This enables non-contact three-dimensional positioning without the need for additional ranging equipment. This reduces costs and improves the flexibility and applicability of positioning, making it particularly suitable for large-scale deployments and applications in complex environments.

[0168] Specifically, the original input image is processed using a monocular image depth estimation algorithm to generate a corresponding depth map. This algorithm utilizes a deep learning model to predict the depth value of each pixel in the scene from a single RGB image. These depth values ​​constitute the depth map, where the grayscale or color value of each pixel represents the distance from that pixel to the camera.

[0169] Based on the wildfire target coordinates (x, y) obtained by the target detection algorithm, the depth value at the corresponding location is extracted from the depth map. Since each pixel in the depth map corresponds to a depth value, the corresponding depth value can be directly found in the depth map using the wildfire target's coordinates. This depth value is the distance d from the wildfire target to the camera.

[0170] Step S8: Solve the latitude and longitude of the wildfire using the Haversine formula.

[0171] Specifically, the azimuth and elevation angles of the wildfire are determined based on its location in the image and camera parameters. Combined with the camera's latitude and longitude coordinates, distance d, azimuth and elevation angles, the specific location (latitude and longitude) of the wildfire in the world coordinate system is calculated using geometric methods. This realizes the transformation from image recognition to geographic positioning, enabling rapid response to wildfire events.

[0172] To obtain the orientation of the wildfire target relative to the camera, it is necessary to convert the image coordinates (x, y) into a direction vector in the spatial coordinate system based on the camera's intrinsic parameter matrix K and extrinsic parameter matrices (including the rotation matrix R and translation vector T). The camera's intrinsic parameter matrix K is:

[0173]

[0174] Among them, f x f y These represent different focal lengths, c x c y Let x and y represent the principal point coordinates, respectively. For a given image coordinate (x, y), construct a normalized device coordinate (NDC) vector:

[0175]

[0176] Based on the camera's extrinsic parameters (rotation and translation), transform the device coordinate vector to the world coordinate system:

[0177] p w =R -1 (p ndc ·dT)

[0178] Point p in the world coordinate system obtained w =(X w ,Y w Zw ), calculate the azimuth angle θ and elevation angle α:

[0179] θ = arctan2(Y) w ,X w )

[0180]

[0181] Calculate the projected distance on the horizontal plane:

[0182] d h =d / cos(α)

[0183] The latitude and longitude of wildfire targets can be directly calculated using the Haversine formula.

[0184]

[0185] in, The position of the camera is indicated by d, the distance from the wildfire target is indicated by θ, the azimuth angle is indicated by α, and the elevation angle is indicated by R. 地 This represents the Earth's radius.

[0186] In more preferred examples, it supports pure vision solutions for target detection and spatial localization, is compatible with subsequent addition of sensors for multimodal fusion, has higher system scalability and low-cost deployment potential, not only simplifies system installation and maintenance, but also provides greater flexibility for future functional expansion.

[0187] Breaking away from the traditional sequential processing mode of "motion detection → flame recognition", a new route of "first suppressing interference → then fine detection → finally three-dimensional positioning" has been constructed. It is particularly suitable for field monitoring scenarios that require long-term unattended operation, such as along power lines, mountain watchtowers, and nature reserves. It has three major practical advantages: high robustness, low false alarm rate, and accurate positioning, and has broad application prospects and promotional value.

[0188] The present invention also provides a wildfire identification and location system based on fine-grained slicing and depth estimation. The wildfire identification and location system based on fine-grained slicing and depth estimation can be implemented by executing the process steps of the wildfire identification and location method based on fine-grained slicing and depth estimation. That is, those skilled in the art can understand the wildfire identification and location method based on fine-grained slicing and depth estimation as a preferred embodiment of the wildfire identification and location system based on fine-grained slicing and depth estimation.

[0189] This invention provides a wildfire identification and location system based on fine-grained slicing and depth estimation. Figure 2 For example, including:

[0190] Module M1 preprocesses the input image to obtain the processed image;

[0191] Module M2 segments and processes the image, removing the sky region to obtain the cut image;

[0192] Module M3 uses a sliding window to divide and slice the image into multiple slices;

[0193] Module M4 and the YOLO object detection model identify volcanic features in each slice and record the identification results;

[0194] Module M5 converts the recognition results to the original image coordinate system;

[0195] Module M6 restores the sky region of the cropped image and draws the converted recognition result to obtain the restored image;

[0196] Module M7: Depth estimation. The input image is used to obtain a depth map, and the coordinates and corresponding depth values ​​of the wildfire target are located based on the reconstructed image.

[0197] Module M8: Solve the latitude and longitude of the wildfire from the target coordinates.

[0198] In more preferred embodiments, the preprocessing includes adjusting the resolution, normalizing pixel values, and changing the color channel order.

[0199] In module M2, the sky region of the processed image is segmented and removed using an image segmentation algorithm, and then filled with a set color.

[0200] In module M3, a sliding window mechanism is used to finely divide the cut image into multiple overlapping slices, and the position of each slice is determined according to the coordinates of the upper left corner:

[0201] x i =i·s

[0202] y j =j·s

[0203] Where i and j are unit vectors on the X and Y coordinate axes, respectively;

[0204] x i y j These represent the coordinates of the top-left corner of the slice along the X and Y axes in the original image, respectively.

[0205] s represents the step size.

[0206] In more preferred embodiments, in module M4, each slice is input into a pre-trained object detection model to perform inference and obtain the recognition result:

[0207] B = {b1, b2, ..., b} n}

[0208] Where B represents the set of all bounding boxes in the recognition result;

[0209] n represents the total number of bounding boxes detected in the slice;

[0210] b represents the identified bounding box.

[0211] Each bounding box b contains:

[0212] b = (x min ,y min ,x max ,y max c)

[0213] Where c represents the confidence score of the bounding box;

[0214] x min x max These represent the minimum and maximum X-axis coordinates of the bounding box, respectively.

[0215] y min y max These represent the minimum and maximum Y-axis coordinates of the bounding box, respectively.

[0216] Sort the bounding boxes in descending order of confidence score, add the bounding box with the highest confidence score to the final result set, and determine the IoU between each bounding box and the bounding box with the highest confidence score.

[0217] If IoU is greater than the set threshold t iou If the IoU is less than or equal to the set threshold t, then remove it. iou If so, then retain it;

[0218] Iterate through all bounding boxes.

[0219] In more preferred embodiments, the corresponding position of the bounding box b in the original image coordinate system in module M5 is:

[0220] x′ min =x min +x i

[0221] y′ min =y min +y j

[0222] x′ max =x max +x i

[0223] y′ max =y max +y j

[0224] Where, x′ min 、x′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively;

[0225] y′ min y′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively.

[0226] In module M6, the segmented sky region is added back to the processed image, and the bounding box of the recognition result is drawn on the corresponding position of the processed image according to the transformed coordinates.

[0227] In module M7, the coordinates (x, y) of the wildfire target are obtained on the restored image according to the target detection algorithm, and the depth map of the input image is obtained using the monocular image depth estimation algorithm. The depth value corresponding to the coordinates of the wildfire target is found in the depth map as the distance d from the wildfire target to the camera.

[0228] In more preferred embodiments, in module M8, based on the camera's intrinsic and extrinsic parameter matrices, the wildfire target coordinates (x, y) are converted into a direction vector in a spatial coordinate system, constructing a normalized device coordinate vector:

[0229]

[0230] Among them, f x f y These represent different focal lengths;

[0231] c x c y These represent the X and Y coordinates of the principal point, respectively.

[0232] Based on the camera's extrinsic parameters, transform the device coordinate vector to the world coordinate system:

[0233] p w =R -1 (p ndc ·dT)

[0234] p w =(X w ,Y w Z w )

[0235] Calculate the azimuth angle θ and the elevation angle α:

[0236] θ = arctan2(Y) w ,X w )

[0237]

[0238] Calculate the projected distance on the horizontal plane:

[0239] d h =d / cos(α)

[0240] Calculate the latitude and longitude of wildfire targets using the Haversine formula:

[0241]

[0242] in, Indicates the camera's location;

[0243] d represents the distance from the wildfire target;

[0244] θ represents the azimuth angle;

[0245] α represents the angle of elevation;

[0246] R 地 This represents the Earth's radius.

[0247] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0248] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A wildfire identification and localization method based on fine-grained slicing and depth estimation, characterized in that, include: Step S1: Preprocess the input image to obtain the processed image; Step S2: Segment the image, remove the sky region, and obtain the segmented image; Step S3: Divide the image into multiple slices using a sliding window; Step S4: The YOLO object detection model identifies volcanic features in each slice and records the identification results; Step S5: Convert the recognition results to the original image coordinate system; Step S6: Restore the sky region of the cut image and draw the converted recognition result to obtain the restored image; Step S7: Depth estimation: Obtain a depth map from the input image, and locate the coordinates of the wildfire target and its corresponding depth value based on the reconstructed image; Step S8: Solve the latitude and longitude of the wildfire based on the target coordinates.

2. The wildfire identification and localization method based on fine-grained slicing and depth estimation according to claim 1, characterized in that, The preprocessing includes adjusting the resolution, normalizing pixel values, and converting the color channel order; In step S2, the sky region of the image is segmented and removed using an image segmentation algorithm, and then filled with a set color. In step S3, a sliding window mechanism is used to finely divide the cut image into multiple overlapping slices, and the position of each slice is determined according to the coordinates of the upper left corner: x i = i • s y j = j-s Where i and j are unit vectors on the X and Y coordinate axes, respectively; x i , y j respectively represent the coordinate values of the upper left corner of the slice in the original image along the X, Y axis direction; s represents the step size.

3. The wildfire identification and localization method based on fine-grained slicing and depth estimation according to claim 1, characterized in that, In step S4, each slice is input into a pre-trained target detection model to perform inference and obtain the recognition result: B = {b1, b2,..., b n} Where B represents the set of all bounding boxes in the recognition result; n represents the total number of bounding boxes detected in the slice; b represents the identified bounding box; Each bounding box b contains: b = (x min ,y min ,x max ,y max ,c) Where c represents the confidence score of the bounding box; x min , x max respectively represent the minimum and maximum X-axis coordinate values of the bounding box; y min , y max respectively represent the minimum and maximum Y-axis coordinate values of the bounding box; Sort the bounding boxes in descending order of confidence score, add the bounding box with the highest confidence score to the final result set, and determine the IoU between each bounding box and the bounding box with the highest confidence score. If the IoU is greater than a set threshold t iou , then remove, if the IoU is less than or equal to a set threshold t iou , then keep; Iterate through all bounding boxes.

4. The wildfire identification and localization method based on fine-grained slicing and depth estimation according to claim 1, characterized in that, In step S5, the corresponding position of bounding box b in the original image coordinate system is: x' min = x min + x i y' min = y min + y j x max = x max + x i y' max = y max + y j where x' min and x' max represent the minimum and maximum X-axis coordinate values of the bounding box in the original image, respectively. min , x' max respectively represent the minimum and maximum X-axis coordinate values of the bounding box in the original image. y' min , y' max respectively represent the minimum and maximum X-axis coordinate values of the bounding box in the original image; In step S6, the segmented sky region is added back to the processed image, and the bounding box of the recognition result is drawn on the corresponding position of the processed image according to the transformed coordinates. In step S7, the coordinates (x, y) of the wildfire target are obtained on the restored image according to the target detection algorithm, and the depth map of the input image is obtained by using the monocular image depth estimation algorithm. The depth value corresponding to the coordinates of the wildfire target is found in the depth map as the distance d from the wildfire target to the camera.

5. The wildfire identification and localization method based on fine-grained slicing and depth estimation according to claim 1, characterized in that, In step S8, based on the camera's intrinsic and extrinsic parameter matrices, the wildfire target coordinates (x, y) are converted into direction vectors in a spatial coordinate system, and a normalized device coordinate vector is constructed: wherein f x , f y represent different focal lengths, respectively; c x , c y respectively denote the principal point X, Y coordinates; Based on the camera's extrinsic parameters, transform the device coordinate vector to the world coordinate system: p w = R -1 (p ndc · d-T) p w = (X w , Y w , Z w ) Calculate the azimuth angle θ and the elevation angle α: θ = arctan2(Y w , X w ) Calculate the projected distance on the horizontal plane: d h = d / cos(a) Calculate the latitude and longitude of wildfire targets using the Haversine formula: in, Indicates the camera's location; X w , Y w , Z w respectively represent the X, Y, Z axis values of the forest fire target under the world coordinate system; d represents the distance from the wildfire target; θ represents the azimuth angle; α represents the angle of elevation; R 地 represents the radius of the earth.

6. A wildfire identification and localization system based on fine-grained slicing and depth estimation, characterized in that, include: Module M1 preprocesses the input image to obtain the processed image; Module M2 segments and processes the image, removing the sky region to obtain the cut image; Module M3 uses a sliding window to divide and slice the image into multiple slices; Module M4 and the YOLO object detection model identify volcanic features in each slice and record the identification results; Module M5 converts the recognition results to the original image coordinate system; Module M6 restores the sky region of the cropped image and draws the converted recognition result to obtain the restored image; Module M7: Depth estimation. The input image is used to obtain a depth map, and the coordinates and corresponding depth values ​​of the wildfire target are located based on the reconstructed image. Module M8: Solve the latitude and longitude of the wildfire from the target coordinates.

7. The wildfire identification and localization system based on fine-grained slicing and depth estimation according to claim 6, characterized in that, The preprocessing includes adjusting the resolution, normalizing pixel values, and converting the color channel order; In module M2, the sky region of the processed image is segmented and removed using an image segmentation algorithm, and then filled with a set color. In module M3, a sliding window mechanism is used to finely divide the cut image into multiple overlapping slices, and the position of each slice is determined according to the coordinates of the upper left corner: x i = i • s y j = j-s Where i and j are unit vectors on the X and Y coordinate axes, respectively; x i , y j respectively represent the coordinate values of the upper left corner of the slice in the original image along the X, Y axis direction; s represents the step size.

8. The wildfire identification and localization system based on fine-grained slicing and depth estimation according to claim 6, characterized in that, In module M4, each slice is input into a pre-trained object detection model to perform inference and obtain the recognition result: B = {b1, b2,..., b n} Where B represents the set of all bounding boxes in the recognition result; n represents the total number of bounding boxes detected in the slice; b represents the identified bounding box; Each bounding box b contains: b = (x min ,y min ,x max ,y max ,c) Where c represents the confidence score of the bounding box; x min , x max respectively represent the minimum and maximum X-axis coordinate values of the bounding box; y min , y max respectively represent the minimum, maximum Y-axis coordinate values of the bounding box; Sort the bounding boxes in descending order of confidence score, add the bounding box with the highest confidence score to the final result set, and determine the IoU between each bounding box and the bounding box with the highest confidence score. If the IoU is greater than a set threshold t iou , then remove, if the IoU is less than or equal to a set threshold t iou , then keep; Iterate through all bounding boxes.

9. The wildfire identification and localization system based on fine-grained slicing and depth estimation according to claim 6, characterized in that, In module M5, the corresponding position of bounding box b in the original image coordinate system is: x' min = x min + x i y' min = y min + y j x' max = x max + x i y' max = y max + y j Where, x′ min 、x′ max These represent the minimum and maximum X-axis coordinates of the bounding box in the original image, respectively; y' min , y' max respectively represent the minimum, maximum X-axis coordinate values of the bounding box in the original image; In module M6, the segmented sky region is added back to the processed image, and the bounding box of the recognition result is drawn on the corresponding position of the processed image according to the transformed coordinates. In module M7, the coordinates (x, y) of the wildfire target are obtained on the restored image according to the target detection algorithm. The depth map of the input image is obtained by using a monocular image depth estimation algorithm. The depth value corresponding to the coordinates of the wildfire target is found in the depth map as the distance d from the wildfire target to the camera.

10. The wildfire identification and localization system based on fine-grained slicing and depth estimation according to claim 6, characterized in that, In module M8, based on the camera's intrinsic and extrinsic parameter matrices, the wildfire target coordinates (x, y) are converted into direction vectors in a spatial coordinate system, constructing a normalized device coordinate vector: wherein f x , f y represent different focal lengths, respectively; c x , c y respectively denote the principal point X, Y coordinates; Based on the camera's extrinsic parameters, transform the device coordinate vector to the world coordinate system: p w = R -1 (p ndc • d-T) p w = (X w , Y w , Z w ) Calculate the azimuth angle θ and the elevation angle α: θ = arctan2(Y w ,X w ) Calculate the projected distance on the horizontal plane: d h =d / cos(α) Calculate the latitude and longitude of wildfire targets using the Haversine formula: in, Indicates the camera's location; X w Y w Z w These represent the X, Y, and Z axis values ​​of the wildfire target in the world coordinate system, respectively. d represents the distance from the wildfire target; θ represents the azimuth angle; α represents the angle of elevation; R 地 This represents the Earth's radius.

Images