A fire area rapid estimation method based on passive location

Abstract

A kind of fire area fast estimation method based on passive location.The present application relates to the field of image processing technology, in particular to a kind of method for calibrating laser optical axis deviation by target positioning in the air.The present application loads the digital map and digital elevation model file of target area actual measurement image, combines the pose data corresponding to single frame image to carry out orthographic transformation to image, and completes the work of foreground extraction and fire area estimation on this basis.The method comprises the following steps: preparing digital elevation model data set of the region to be measured;Taking the actual measurement image of the region to be measured;Solving the field center position;According to the field center position, find out the digital elevation model file corresponding to the actual measurement image;Orthographic transformation is carried out to obtain the image of the straight-down view angle;According to the image of the straight-down view angle, based on the digital elevation model file, the area of the region to be measured is solved;Extracting image foreground estimates the fire area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a fire area rapid estimation method based on passive positioning. BACKGROUND

[0002] Forest fire is one of the most serious disasters to the forest. Therefore, it has become an urgent need to quickly and accurately find and assess the forest fire. Using a photoelectric pod-equipped airplane for cruising is a common means of monitoring forest fires at present. The image processing system of the photoelectric pod can estimate the fire area. However, the current known fire area estimation methods generally have the problems of slow estimation speed and poor estimation accuracy. For example, the fire area cannot be quickly and accurately estimated, which will lead to the inability to reasonably respond to the current situation and the spreading trend of forest fires. Therefore, there is an urgent need for a method that can quickly and accurately estimate the fire area.

[0003] Chinese invention patent "Method for estimating irregularly shaped fire area using photoelectric pod" (CN115790458B) proposes a technical means of calculating the actual area represented by each fire point pixel using slant range and angle, and finally adding up to get the overall fire area. Although this method can estimate the area of any shaped fire (does not require the image center to align with the fire, as long as the fire enters the field of view, the estimation is accurate and rapid, and real-time estimation can be achieved), it ignores the influence of the earth's curvature at a long distance. Moreover, the method of calculating and adding the area represented by each pixel ignores the influence of the earth's curvature at a long distance. Moreover, the way of calculating and adding the area represented by each pixel will greatly increase the calculation amount in the case of high image resolution, which will lead to the calculation result of the area represented by the pixel corresponding to the fire point at the far end being more and more inaccurate (ignoring the earth's curvature), ultimately affecting the estimation accuracy of the overall fire area. When the image resolution is large, the calculation amount increases exponentially, which makes it difficult to guarantee real-time performance. SUMMARY

[0004] In view of the problems of insufficient estimation accuracy of the fire area and difficulty in guaranteeing the real-time performance of the calculation in the existing method, the purpose of the present application is to provide a fire area rapid estimation method based on passive positioning. By loading the digital map and digital elevation model file of the target area measured image, and combining the pose data corresponding to the single frame image to perform orthographic transformation on the image, the work of foreground extraction and fire area estimation is completed.

[0005] The method comprises the following steps:

[0006] S1, making a digital elevation model data set of the area to be measured;

[0007] S2. Take measured images of the area to be tested, and collect the position and attitude angle information of the inertial navigation and encoder respectively;

[0008] S3. Based on the position and attitude angle information collected in step S2, calculate the position of the center of the field of view using the passive positioning algorithm of the Earth ellipsoid model;

[0009] S4. Based on the center position of the field of view, find the digital elevation model file corresponding to the measured image in the dataset created in step S1;

[0010] S5. Preprocess the measured image and perform orthorectification on the preprocessed measured image to obtain the image of the downward viewing angle;

[0011] S6. Based on the image from the downward-looking perspective and the digital elevation model file, calculate the area of ​​the region to be measured. ;

[0012] S7. Based on the image viewed from a downward angle, extract the foreground of the image and obtain the number of pixels in the foreground. ;

[0013] S8, according to and Estimate the fire area .

[0014] Furthermore, the digital elevation model dataset of the area to be measured includes: 50×50 digital maps of the area to be measured and digital elevation model files of the area to be measured;

[0015] The resolution of the digital map of the area to be tested is 1280×960, and the latitude and longitude dimensions of each pixel are... ;

[0016] The resolution of the digital elevation model file for the area to be measured is 1280×960.

[0017] Furthermore, in step S2, during flight, an aerial camera is used to capture measured images of the area to be tested;

[0018] The position and attitude information of the inertial navigation and encoder meet the timing alignment requirements.

[0019] Furthermore, in step S4, based on the center position of the field of view, it is determined whether there is a digital map image in the dataset produced in step S1 whose latitude and longitude range meets the preset conditions. If so, it is determined to be the target digital map image corresponding to the measured image, and at the same time, the target digital elevation model file with the same latitude and longitude range as the target digital map image is found.

[0020] Furthermore, in step S5, the coordinates of the object point position corresponding to each pixel in the preprocessed measured image in the map coordinate system are calculated, and an orthorectification is performed based on the orthorectification matrix to obtain the image from the downward viewing angle.

[0021] The orthorectification matrix specifically refers to: transforming a point in the map coordinate system... and ideal image points on the image plane Establish the following relationship: ,in, Denotes any contraction factor. This represents the orientation element within the camera.

[0022] Furthermore, in step S6, the image from the downward-looking perspective is substituted into the digital elevation model file to obtain the specific altitude of the target area corresponding to the measured image. ;

[0023] Then, by combining a passive localization algorithm based on the Earth ellipsoid model, the side length above the target area is calculated. , the length of the side below the target area Distance between the top and bottom of the target area ;

[0024] pass , and Calculate the area of ​​the region to be measured Specifically: .

[0025] Furthermore, in step S7, histogram equalization is performed on the image from the downward viewing angle to obtain the histogram equalized image. ;

[0026] For images Significance detection is performed to obtain a saliency map;

[0027] Adaptive thresholds for the target and background are calculated using Gaussian filtering and the Otsu thresholding algorithm. ;

[0028] Iterate through the grayscale values ​​of the image; if the grayscale value of the image is greater than... If the value is 1, then the value of the corresponding pixel is changed to 1; otherwise, it is 0.

[0029] The image's grayscale values ​​have been traversed. At this point, the number of pixels with a value of 1 is equal to the number of foreground pixels in the image. equal.

[0030] Furthermore, the aforementioned The formula for calculation is: ,in, This represents the number of pixels in the measured image. ,in, Indicates the width of the image. Indicates the height of the image.

[0031] The beneficial effects of the method described in this invention are as follows:

[0032] (1) The method of the present invention fully solves the problem that the existing technology does not fully incorporate the curvature of the earth when observing from a distance, resulting in low evaluation accuracy. It breaks the limitation of the previous method of calculating the area by only using trigonometric function relationships, and fully considers the influence of the curvature of the earth when observing from a distance, thus greatly improving the estimation accuracy of the fire area.

[0033] (2) The method described in this invention, based on orthophoto transformation, combines local histogram equalization, FT saliency detection algorithm and Gaussian filtering and other techniques to extract the foreground image, and then combines Otsu threshold segmentation method to estimate the fire area, which greatly improves the calculation speed and fully meets the real-time requirements. Attached Figure Description

[0034] Figure 1 This is a flowchart of the method described in an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] This embodiment provides a method for rapid estimation of fire area based on passive location, and the flowchart of the method is as follows. Figure 1 As shown, the method includes the following steps:

[0037] S1. Create a digital elevation model dataset for the area to be measured;

[0038] In step S1, the digital elevation model dataset of the area to be measured includes: 50×50 digital maps of the area to be measured and digital elevation model files of the area to be measured;

[0039] The resolution of the digital map of the area to be tested is 1280×960, and the latitude and longitude dimensions of each pixel are... ;

[0040] At the same time, a corresponding digital elevation model file is created. Since the default resolution of the digital elevation model file is 3601×3601, it needs to be cropped and stitched to adapt to the resolution of the digital map image, resulting in a resolution of 1280×960 that is consistent with the latitude and longitude range of the digital map image.

[0041] Finally, the latitude and longitude ranges of each of the 50×50 digital elevation model files, which correspond one-to-one with each digital map image, are labeled and integrated into a single digital elevation model dataset.

[0042] S2. Take actual images of the area to be tested, and collect the position and attitude information of the inertial navigation and encoder respectively;

[0043] In step S2, during flight, an aerial camera (photoelectric platform) is used to capture measured images of the area to be tested. The measured images are aligned with the timing of the inertial navigation device and encoder (meeting the timing alignment requirements). Then, the inertial navigation device and encoder are used to collect the position and attitude angle information corresponding to the current measured image at that moment.

[0044] S3. Based on the position and attitude angle information collected in step S2, calculate the position of the center of the field of view using the passive positioning algorithm of the Earth ellipsoid model;

[0045] In step S3, based on the Earth ellipsoid model, the geographical location of the field of view center is initially calculated using the collected position and attitude angle information.

[0046] Projection point on the detector (center of the field of view) The coordinates in the camera coordinate system are:

[0047] in, Indicates the pixel size of the detector. This represents the offset of the image's horizontal axis relative to the center of the field of view. This represents the offset of the image's vertical axis relative to the center of the field of view. This indicates the focal length of the imaging system.

[0048] coordinates in the Earth's rectangular coordinate system Should meet

[0049]

[0050] in, and This indicates the transformation from the camera coordinate system to the aircraft coordinate system. , and This indicates the transformation from the aircraft coordinate system to the geographic coordinate system. , , and This indicates the conversion from the geographic coordinate system to the Earth's rectangular coordinate system.

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] in, Represents the coordinate system around the aircraft The rotation matrix of the axis. Indicates the platform's azimuth angle. Represents the coordinate system around the aircraft The rotation matrix of the axis. Indicates the platform's pitch angle. Represents the navigation coordinate system The rotation matrix of the axis. Indicates the aircraft's heading angle. Represents the navigation coordinate system The rotation matrix of the axis. Indicates the aircraft's pitch angle. This represents the rotation matrix about the x-axis of the aircraft coordinate system. Indicates the aircraft's roll angle. , , and These represent the four rotation matrices from the navigation coordinate system to the Earth's rectangular coordinate system. Indicates the aircraft's altitude. Indicates the aircraft's latitude. Indicates the longitude of the aircraft. This represents the radius of curvature of the Earth's ellipsoid, specifically the geoid. This represents the first eccentricity of the Earth's ellipsoid; , , , , , , and Data is collected via step S2.

[0061] Next, a passive localization algorithm based on the Earth ellipsoid model is used to calculate the geographical location of the field of view center. The basic principle is as follows: the coordinates of the camera origin in the Earth's rectangular coordinate system. Should meet

[0062]

[0063] For an ideal optical system, the object point, image point, and projection center are collinear. That is, the target point lies on the straight line connecting the target projection point and the origin of the camera coordinate system. Therefore, the coordinates of the target point in the Earth's Cartesian coordinate system are... Should meet

[0064]

[0065] in, This represents the coordinates of the projection point on the camera.

[0066] The average elevation of the area to be measured is given by the coordinates of the target in the Earth's rectangular coordinate system. Should meet

[0067]

[0068] in, , They represent The corresponding semi-major and semi-minor axes of the Earth's ellipsoid. Among them, The meter represents the semi-major axis of the Earth's ellipsoid. The equations are solved using a model of the Earth's ellipsoid. and This will give you the coordinates of the target in the Earth's rectangular coordinate system. Then transform it into a value in the geodetic coordinate system. That is, the geographical location of the center of the field of view.

[0069] S4. Based on the geographical location of the field of view center, find the digital elevation model file corresponding to the measured image in the dataset created in step S1.

[0070] In step S4, the geographical location of the field of view center obtained in step S3 is used. In the dataset constructed in step S1, the corresponding files are located. It is then determined whether the latitude and longitude ranges of the digital map images labeled in the dataset meet the following conditions. ,in, This represents the minimum latitude in a single digital map. This represents the maximum latitude in a single digital map. This represents the minimum longitude in a single map. This represents the maximum longitude in a single map image. If the above conditions are met, it is determined to be the target digital map image corresponding to the measured image. Simultaneously, according to the following formula, the target digital elevation model file with the same latitude and longitude range as the target digital map image is found:

[0071] in, This indicates the maximum altitude on a single map. This represents the minimum elevation in a single map. This represents the calculated target height. Indicates the platform's azimuth angle. Indicates the platform's pitch angle. Indicates the aircraft's heading angle. This represents the circumference of the Earth's equator;

[0072] S5. Preprocess the measured image and perform orthorectification on the preprocessed measured image to obtain the image of the downward viewing angle;

[0073] In step S5, the preprocessing includes: a dehazing algorithm and an enhancement algorithm;

[0074] Orthorectification specifically refers to transforming a point in a map coordinate system into a point in the map coordinate system. and ideal image points on the image plane Establish the following relationship

[0075]

[0076] in, Denotes any contraction factor. Represents the orientation element within the camera. ,in, and These represent the physical length and physical width of a single pixel, respectively, in units of... F represents the camera's focal length, in mm. These are the coordinates of the principal point of the image, in pixels. For camera exterior orientation elements, this represents the transformation relationship from the map coordinate system to the camera coordinate system:

[0077] in, This indicates the offset between the geographic coordinate system and the map coordinate system. , ;

[0078] First, calculate the offset between the aircraft's geographic coordinate system and the map coordinate system at that moment based on the GPS position corresponding to the captured measured image. Calculate the coordinates of the carrier aircraft in the Earth's rectangular coordinate system. :

[0079]

[0080] Among them, the semi-major axis of the ellipsoid Meters, semi-minor axis of the ellipsoid meters, the first eccentricity of the ellipsoid is e= ellipsoidal radius of curvature This represents the coordinates of the measured image location in the Earth's rectangular coordinate system. The coordinates of this location in the map coordinate system can be calculated using the following formula:

[0081]

[0082] in, This represents the coordinates of the map coordinate system origin in the Earth's rectangular coordinate system. This represents the rotation matrix from the Earth's Cartesian coordinate system to the map coordinate system.

[0083]

[0084] in, , Represents the latitude and longitude coordinates of the origin of the map coordinate system.

[0085] Through transformation, we can further obtain

[0086]

[0087] The matrix parameters in the above formula can be obtained by converting them using the following formula:

[0088]

[0089] Using the above formula, the coordinates of the object point position corresponding to each pixel in the preprocessed measured image in the map coordinate system can be calculated and orthorectified to obtain the image from the downward viewing angle.

[0090] S6. Based on the image from the downward-looking perspective and the digital elevation model file, calculate the area of ​​the region to be measured. ;

[0091] In step S6, the image from the downward-looking perspective is used to determine the specific altitude of the target area corresponding to the measured image using a digital elevation model file. Furthermore, Substituting the passive positioning algorithm based on the Earth ellipsoid model in step S3, the latitude, longitude, and altitude of the upper left, lower left, upper right, and lower right corners of the current downward-looking image are calculated. , , and Finally, substituting the latitude, longitude, and altitude of the four corners of the region into the following formula, we obtain the area of ​​the entire region. :

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102] in, , , and These represent the coordinates of the top-left, bottom-left, top-right, and bottom-right geographic points of the target area in a rectangular coordinate system. This indicates the side length above the target area. The side length below the target area, This indicates the coordinates of a geographic point directly above the target area in a rectangular coordinate system. This indicates the coordinates of a geographic point directly below the target area in a rectangular coordinate system. This indicates the vertical distance between the target area and the surrounding area.

[0103] S7. Based on the image viewed from a downward angle, extract the foreground of the image and obtain the number of pixels in the foreground. ;

[0104] Foreground Extraction: First, local histogram equalization is performed on the image (infrared image) from a downward-facing viewpoint. Because some images have low contrast, their grayscale histograms are concentrated in a small range along the grayscale axis, significantly hindering further processing. Therefore, the method described in this invention first uses histogram equalization to enhance the image, facilitating the next processing step. (Image from a downward-facing viewpoint) The histogram is used to reconstruct the gray-level distribution density function. (By readjusting the grayscale distribution of the image based on its histogram distribution), the histogram-equalized image can be obtained. , ,in, Represents the histogram equalization function, image At point grayscale value at It can be represented as: in, Points before histogram equalization grayscale value at that location , Represents the gray levels of an image. This represents the total number of pixels in the image. Indicates grayscale value The number of pixels.

[0105] The next step is to analyze the image. The high-brightness target areas in the image are subjected to saliency detection; that is, the contrast between the fire area and the surrounding background area is further improved by using the Fourier Transform (FT) saliency detection algorithm. This is achieved by the following formula: This yields the salient region. Representing an image Average characteristics, For image pixels in Features after Gaussian smoothing Representing an image The significance value, Represents the L2 paradigm. Image. After Gaussian smoothing and calculating the saliency value of each pixel using the above formula, a saliency map is obtained.

[0106] Next, Gaussian filtering is performed, combined with the Otsu thresholding algorithm to segment the image. First, the image (image...) is... To convert the image to grayscale, apply the following formula to perform Gaussian filtering and smoothing. The advantage of Gaussian smoothing is that it can smooth out the problem of prominent noise caused by subsequent Otsu segmentation (Otsu segmentation will highlight the noise in the image, while Gaussian smoothing can smooth the noise in the image in advance).

[0107]

[0108] in, Indicates pixel coordinates The grayscale value of the pixel, and These represent the horizontal and vertical coordinates of the pixel, respectively. This represents the mean square error coefficient of Gaussian smoothing. This represents the mean coefficient of Gaussian smoothing.

[0109] Regarding Otsu thresholding, let the image grayscale level be... grayscale value The number of pixels is , The total number of pixels is: The grayscale value is The probability of a point is calculated using the mean of the image's gray levels: , ,in, Indicates grayscale value The proportion of pixels to all pixels. This represents the average grayscale value of all pixels in the entire image.

[0110] Based on the grayscale characteristics of the image, thresholds are applied to the hypothetical target and background. Images are divided into two categories: target and background. and These represent the probabilities of the target and background occurring, respectively.

[0111] ,

[0112] The mean grayscale values ​​of the pixels in the target and background are:

[0113] ,

[0114] use This indicates that the threshold in the histogram is... The variance of is located as follows:

[0115]

[0116] The optimal threshold is the threshold corresponding to the maximum variance. Value, an adaptive threshold for the target and background. ;

[0117]

[0118] Combine the grayscale values ​​of the current image with Compare the values; if the grayscale value of the current image is greater than... If the value is 1, the corresponding pixel value is changed to 1; otherwise, it is 0. This allows us to count the number of foreground pixels in the image. .

[0119] S8, according to and Estimate the fire area.

[0120] The fire area can be calculated using the following formula.

[0121]

[0122] in, This represents the number of foreground pixels in the measured image. This represents the total number of pixels in the measured image. Indicates the width of the image. Indicates the height of the image.

Claims

1. A method for rapid estimation of fire area based on passive location, characterized in that, The method includes the following steps: S1. Create a digital elevation model dataset for the area to be measured; S2. Take measured images of the area to be tested, and collect the position and attitude angle information of the inertial navigation and encoder respectively; S3. Based on the position and attitude angle information collected in step S2, calculate the position of the center of the field of view using the passive positioning algorithm of the Earth ellipsoid model; S4. Based on the center position of the field of view, find the digital elevation model file corresponding to the measured image in the dataset created in step S1; S5. Preprocess the measured image and perform orthorectification on the preprocessed measured image to obtain the image of the downward viewing angle; S6. Based on the image from the downward-looking perspective and the digital elevation model file, calculate the area of ​​the region to be measured. ; In step S6, the image from the downward-looking perspective is substituted into the digital elevation model file to obtain the specific altitude of the target area corresponding to the measured image. ; Combining this with a passive positioning algorithm based on the Earth ellipsoid model, the latitude, longitude, and altitude of the four corners of the region are first calculated. Then, based on these corners, the coordinates of the top-left, bottom-left, top-right, and bottom-right geographic points of the target region in the Cartesian coordinate system are calculated. Finally, based on these coordinates, the length of the side above the target region is calculated. , the length of the side below the target area Distance between the top and bottom of the target area ; pass , and Calculate the area of ​​the region to be measured Specifically: ; The latitude, longitude, and altitude of the four corners of the region represent the latitude, longitude, and altitude of the upper left, lower left, upper right, and lower right corners of the current downward-facing viewpoint image; S7. Based on the image viewed from a downward angle, extract the foreground of the image and obtain the number of pixels in the foreground. ; In step S7, histogram equalization is performed on the image from the downward-facing viewpoint to obtain the histogram equalized image. ; For images Significance detection is performed to obtain a saliency map; Adaptive thresholds for the target and background are calculated using Gaussian filtering and the Otsu thresholding algorithm. ; Iterate through the grayscale values ​​of the image; if the grayscale value of the image is greater than... If the value is 1, then the value of the corresponding pixel is changed to 1; otherwise, it is 0. The image's grayscale values ​​have been traversed. At this point, the number of pixels with a value of 1 is equal to the number of foreground pixels in the image. equal; S8, according to and Estimate the fire area ; The The formula for calculation is: ,in, This represents the number of pixels in the measured image. ,in, Indicates the width of the image. Indicates the height of the image.

2. The method for rapid estimation of fire area based on passive positioning according to claim 1, characterized in that, The digital elevation model dataset of the area to be measured includes: 50×50 digital maps of the area to be measured and digital elevation model files of the area to be measured; The resolution of the digital map of the area to be tested is 1280×960, and the latitude and longitude dimensions of each pixel are... ; The resolution of the digital elevation model file for the area to be measured is 1280×960.

3. The method for rapid estimation of fire area based on passive positioning according to claim 2, characterized in that, In step S2, during flight, an aerial camera is used to capture measured images of the area to be tested. The position and attitude information of the inertial navigation and encoder meet the timing alignment requirements.

4. The method for rapid estimation of fire area based on passive positioning according to claim 3, characterized in that, In step S4, based on the center position of the field of view, it is determined whether there is a digital map image in the dataset created in step S1 whose latitude and longitude range meets the preset conditions. If so, it is determined to be the target digital map image corresponding to the measured image, and at the same time, the target digital elevation model file with the same latitude and longitude range as the target digital map image is found.

5. The method for rapid estimation of fire area based on passive positioning according to claim 4, characterized in that, In step S5, the coordinates of the object point position corresponding to each pixel in the preprocessed measured image in the map coordinate system are calculated, and an orthorectification is performed based on the orthorectification matrix to obtain the image from the downward viewing angle. The orthorectification matrix specifically refers to: transforming a point in the map coordinate system... and ideal image points on the image plane Establish the following relationship: ,in, Denotes any contraction factor. This represents the orientation element within the camera.

Images