Forest fire on-orbit detection method and device based on infrared multispectral image

Abstract

The present application relates to a kind of forest fire on-orbit detection method and device based on infrared multispectral image.The method comprises: obtaining the first infrared image and the preset quantity of second infrared image of target area, and each infrared image corresponds different spectral range and detection threshold value respectively;First target in first infrared image is extracted;Second infrared image is traversed in turn, and for each second infrared image traversed, it is executed: S1, whether the number of current remaining first target in first infrared image is 0 or the number of current second infrared image reaches the preset quantity is determined;If no, execute S2, if yes, execute S3;S2, second target in current second infrared image is extracted;The estimated position of current remaining first target in the second infrared image is estimated;Fire point matching is carried out based on the position of each first target, second target and estimated position;S3, output detection result.The present application method algorithm is simple, and the amount of calculation is small, and on-orbit real-time detection to forest fire can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of infrared remote sensing, and in particular to a forest fire on-orbit detection method and device based on infrared multispectral images. BACKGROUND

[0002] The forest fire detection sensor is carried on a satellite and has functions of infrared multispectral imaging and real-time forest fire detection. The working principle is that the infrared multispectral observation is performed on a ground target to obtain a multispectral image, and then the multispectral image is used to detect and identify the ground target to determine whether there is a fire point in the forest and the specific position of the fire point.

[0003] In the related art, high-precision pixel-level image registration and information fusion are usually performed by using a cross-correlation matching algorithm or image feature extraction and matching, pixel geographic information marking, and the like, and then the ground information is inferred by comprehensively using multiple spectral data. This method is logically complex and has a large amount of calculation, and a high-performance cluster computer is required to process the image. However, since the space and weight resources of the satellite given to the forest fire detection device are limited, it is not realistic to perform on-orbit real-time detection on the multispectral image.

[0004] At present, the infrared multispectral image obtained by the forest fire detection sensor is usually transmitted to the ground, and then a high-performance computer on the ground is used to identify the target. However, due to the influence of factors such as the ground equipment and the communication distance and signal of the satellite, the obtained image cannot be transmitted to the ground in real time, which leads to poor real-time performance of the fire detection and cannot discover the fire in time, thereby affecting the safety of the forest.

[0005] Therefore, there is an urgent need for a forest fire on-orbit detection method and device based on infrared multispectral images to solve the above problems. SUMMARY

[0006] The present application provides a forest fire on-orbit detection method and device based on infrared multispectral images, which has a simple algorithm and small amount of calculation, and can realize on-orbit real-time detection of forest fires.

[0007] In a first aspect, the present application provides a forest fire on-orbit detection method based on infrared multispectral images, comprising:

[0008] obtaining a first infrared image and a preset number of second infrared images of a target region, the first infrared image and each of the second infrared images corresponding to different spectral bands and detection thresholds, and the spectral bands and the detection thresholds being in a one-to-one correspondence;

[0009] extracting a first target in the first infrared image and recording the position of each first target, the first target being a target with a gray value greater than the detection threshold corresponding to the first infrared image, and the number of the first targets being at least one;

[0010] sequentially traversing the second infrared images, for each second infrared image traversed, performing:

[0011] S1, determining whether the number of current remaining first targets in the first infrared image is 0 or the number of current second infrared images reaches the preset number; if not, performing S2, if yes, performing S3;

[0012] S2, extracting the second targets in the current second infrared image and recording the position of each second target, the second target being a target with a gray value greater than the detection threshold corresponding to the second infrared image; estimating the estimated position of the current remaining first target in the first infrared image in the second infrared image; determining the first target to be matched based on each estimated position and the position of each second target; performing fire point matching on each first target to be matched and its corresponding second target, retaining the first target matched successfully and removing the first target matched unsuccessfully; taking the first target matched successfully as the current remaining first target of the first infrared image and adding 1 to the number of second infrared images, and returning to perform S1;

[0013] S3, ending the fire point detection and outputting the fire point detection result.

[0014] In a second aspect, the embodiment of the present application also provides a device for in-orbit detection of forest fire based on infrared multispectral images, comprising:

[0015] an acquisition module, configured to acquire a first infrared image of a target region and a preset number of second infrared images, the first infrared image and each second infrared image corresponding to different spectral bands and detection thresholds respectively, the spectral bands and the detection thresholds being in one-to-one correspondence;

[0016] an extraction module, configured to extract first targets in the first infrared image and record the position of each first target, the first target being a target with a gray value greater than the detection threshold corresponding to the first infrared image, and the number of first targets being at least one;

[0017] a detection module, configured to perform fire point detection in the following manner:

[0018] sequentially traversing the second infrared images, for each second infrared image traversed, performing:

[0019] S1, determining whether the number of current remaining first targets in the first infrared image is 0 or the number of current second infrared images reaches the preset number; if not, performing S2, if yes, performing S3;

[0020] S2, extract the second targets in the current second infrared image, and record the position of each second target, the second target being a target with a gray value greater than a detection threshold corresponding to the second infrared image; estimate the estimated position of the current remaining first target in the first infrared image in the second infrared image; determine the first target to be matched based on each estimated position and the position of each second target; perform fire point matching on each first target to be matched and its corresponding second target, retain the first target with successful matching, and remove the first target with failed matching; take the first target with successful matching as the current remaining first target of the first infrared image, add 1 to the number of second infrared images, and return to perform S1;

[0021] S3, end the fire point detection, and output the fire point detection result.

[0022] In a third aspect, an electronic device is provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the method described in any of the embodiments of the present specification when executing the computer program.

[0023] In a fourth aspect, a computer readable storage medium is provided, storing a computer program, and the computer program, when executed in a computer, causes the computer to perform the method described in any of the embodiments of the present specification.

[0024] The embodiments of the present application provide a forest fire on-orbit detection method based on infrared multispectral images. The method first acquires infrared multispectral images, including a first infrared image and a plurality of second infrared images. Since each infrared image corresponds to different spectral bands, each image can detect different targets. However, since fire points are high-temperature targets, they can exist in infrared images of various spectral bands. For low-temperature targets that are not fire points, they usually only exist in infrared images of certain spectral bands. Based on this principle, the present application extracts first targets and second targets for each infrared image based on their respective detection thresholds. Then, the first infrared image is sequentially registered with each second infrared image, and the position of the first target in the first infrared image in the second infrared image is estimated. Based on the estimated position, the second target position and the first target position are combined for fire point matching, and the points that exist in both the first infrared image and the second infrared image and match each other in position are selected as the final forest fire detection result. The detection method has simple algorithm and small calculation amount, and can realize on-orbit real-time detection of forest fires. BRIEF DESCRIPTION OF DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the structure of an on-orbit forest fire detection method based on infrared multispectral images provided in an embodiment of the present invention;

[0027] Figure 2 This is a hardware architecture diagram of an electronic device provided in an embodiment of the present invention;

[0028] Figure 3 This is a structural diagram of an on-orbit forest fire detection device based on infrared multispectral images, provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0030] Please refer to Figure 1 This invention provides an on-orbit forest fire detection method based on infrared multispectral images, the method comprising:

[0031] Step 100: Acquire a first infrared image of the target area and a preset number of second infrared images. The first infrared image and each second infrared image correspond to different spectral bands and detection thresholds, and the spectral bands and detection thresholds have a one-to-one correspondence.

[0032] Step 102: Extract the first target from the first infrared image and record the position of each first target. The first target is a target whose gray value is greater than the detection threshold corresponding to the first infrared image. The number of first targets is at least one.

[0033] Step 104: Iterate through the second infrared images sequentially. For each second infrared image encountered, perform the following:

[0034] S1, determine whether the number of remaining first targets in the first infrared image is 0, or whether the number of current second infrared images has reached a preset number; if not, execute S2, if yes, execute S3;

[0035] S2, extract the second target from the current second infrared image and record the position of each second target. The second target is the target whose gray value is greater than the detection threshold corresponding to the second infrared image; estimate the estimated position of the current remaining first target in the first infrared image in the second infrared image; based on each estimated position and the position of each second target, determine the first target to be matched; perform fire point matching on each first target to be matched and its corresponding second target, retain the first target that is successfully matched, and remove the first target that is not matched; take the first target that is successfully matched as the current remaining first target in the first infrared image, increment the number of the second infrared image by 1, and return to execute S1;

[0036] S3, end fire detection and output the fire detection result.

[0037] In this embodiment of the invention, firstly, multi-band infrared images are acquired, including a first infrared image and multiple second infrared images. Since each infrared image corresponds to a different spectral band, the targets that can be detected in each image are different. However, since fire points are high-temperature targets, they can exist in infrared images of various spectral bands, while low-temperature targets that are not fire points usually only exist in infrared images of certain spectral bands. Based on this principle, this invention extracts the first and second targets for each infrared image based on its respective detection threshold. Then, the first infrared image is sequentially registered with each second infrared image to estimate the position of the first target in the first infrared image in the second infrared image. Based on this estimated position, fire point matching is performed by combining the positions of the second and first targets, and points that exist simultaneously in both the first and second infrared images and whose positions match are selected as the final forest fire detection result. This detection method has a simple algorithm, low computational load, and can achieve real-time on-orbit detection of forest fires.

[0038] The following description Figure 1 The execution method for each step is shown.

[0039] First, regarding step 100:

[0040] The target area is the forest where fire detection is required. This can be the entire forest or a specific area within the forest prone to fire. The specific location and area of ​​the target area are set by the user as needed. The first and second infrared images can be obtained based on time-division imaging, aperture division, or focal plane method. When obtained based on time-division imaging, the first infrared image and a preset number of second infrared images form a time series, with the first infrared image preferably being the first image in this time series. When obtained based on aperture division or focal plane method, the first image can be any one of the acquired images.

[0041] Furthermore, each infrared image corresponds to a different spectral band and detection threshold. However, since the fire point is a high-temperature target, for any spectral band, the fire point can form a bright spot on its corresponding infrared image and be identified. For low-temperature targets, however, it may only exist in infrared images corresponding to certain spectral bands. Based on this, targets that only exist in certain infrared images can be considered false fire points, while targets that exist in all infrared images are considered true fire points. In addition, the detection threshold is determined by the user based on the spectral band and detection requirements; this application does not impose specific limitations.

[0042] Then, regarding step 102:

[0043] The location of the first target includes its shape and coordinates. After extracting the first target, intensity data for each first target can also be recorded. There must be at least one first target; otherwise, it is determined that no fire point has been detected.

[0044] Finally, regarding step 104:

[0045] In some implementations, when the first infrared image and the second infrared image are obtained by a forest fire detection sensor based on a time-division imaging method (the forest fire detection sensor is mounted on a satellite), the estimated position of the currently remaining first target in the first infrared image in the second infrared image is estimated using the following method:

[0046] Based on the satellite's orbital and motion information, the registration matrix from the first infrared image to the current second infrared image is estimated; based on the registration matrix, the estimated position of the remaining first target in the first infrared image in the second infrared image is estimated.

[0047] In some implementations, when the first infrared image and the second infrared image are obtained by the forest fire detection sensor based on the aperture splitting method or the focal plane splitting method, the estimated position of the currently remaining first target in the first infrared image in the second infrared image is estimated using the following method:

[0048] Based on the optical reference transformation matrix measured by ground equipment, the registration matrix from the first infrared image to the current second infrared image is calculated; based on the registration matrix, the estimated position of the remaining first target in the first infrared image in the second infrared image is estimated.

[0049] In some implementations, determining the first target to be matched based on each estimated location and the location of each second target includes:

[0050] For each estimated position, determine whether the Euclidean distance between the estimated position and the position of any second target is less than a preset value; if so, determine the first target corresponding to the estimated position as the first target to be matched; otherwise, remove the first target corresponding to the estimated position from the first infrared image.

[0051] In this embodiment, the preset value is determined based on the detection accuracy. The smaller the preset value, the higher the detection accuracy, but some fire points may be filtered out; conversely, the larger the preset value, the lower the detection accuracy, resulting in a larger number of targets to be detected and increasing the computational load. Therefore, users can determine the preset value according to their actual needs.

[0052] In some implementations, fire point matching is performed on each first target to be matched and its corresponding second target. Depending on the number of first targets to be matched, the matching can be divided into two cases. The specific matching process for each case is analyzed in detail below:

[0053] First scenario:

[0054] If there are one or two targets to be matched, single-fire point matching is used in this case. The specific matching process is as follows:

[0055] For each target to be matched, determine whether the following conditions are met simultaneously:

[0056] Or when P0>C2, P n -P0 <C4(h0+v0)

[0057] If yes, then the first target is determined to be a successful match with its corresponding second target, and the first target is retained; otherwise, the first target is determined to be a unsuccessful match with its corresponding second target, and the first target is removed.

[0058] In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the number of pixels of the first target, and P h Let h0 and v0 be the number of pixels of the second target corresponding to the first target, respectively, and h0 and v0 be the horizontal pixel width and vertical pixel height of the first target. n and v n These are the horizontal pixel width and vertical pixel height of the second target corresponding to the first target, respectively, n = 1, 2, ..., a preset number; C1 to C5 are respectively related to preset configurable parameters.

[0059] The second scenario:

[0060] If there are more than two high-temperature targets to be matched, single-fire-point matching is first used to remove the first target that failed to match, and the first target that matched successfully is retained. Then, joint fire-point matching is performed on the retained first target. The specific matching process is as follows:

[0061] For each target to be matched, determine whether the following conditions are met simultaneously:

[0062] Or when P0>C2, P n -P0 <C4(h0+v0)

[0063] If yes, then retain the first objective and the second objective corresponding to the first objective; otherwise, remove the first objective and the second objective corresponding to the first objective.

[0064] In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the number of pixels of the first target, and P n Let h0 and v0 be the number of pixels of the second target corresponding to the first target, respectively, and h0 and v0 be the horizontal pixel width and vertical pixel height of the first target. n and v n These are the horizontal pixel width and vertical pixel height of the second target corresponding to the first target, respectively, n = 1, 2, ... preset number; C1 to C5 are preset configurable parameters;

[0065] For both the retained primary and secondary objectives, execute:

[0066] Calculate the Euclidean distance error between the position of each second target and its corresponding estimated position;

[0067] Find the two second targets with the smallest Euclidean distance error, and denote them as F1 and F2 respectively. Also denote the two first targets corresponding to the two second targets with the smallest Euclidean distance error as F′1 and F′2 respectively.

[0068] For other second objectives besides F1 and F2, F i And other first objectives F′ besides F′1 and F′2 i Determine whether the following conditions are met simultaneously:

[0069]

[0070] If yes, then the first target is determined to be a successful match with its corresponding second target, and the first target is retained; otherwise, the first target is determined to be a unsuccessful match with its corresponding second target, and the first target is removed.

[0071] In the formula, L i For F i The length of the line segment connecting the centroid of F1 and the centroid of F1, Y i For F′ i The length of the line segment connecting the centroid of F' and the centroid of F′1; θ i For F iIn the triangle formed by the centroids of F1 and F2, the line segment <F1F i >and line segments <f1f2>the angle between the line segment and the line segment ; || represents taking absolute value; D1-D2 are preset configurable parameters respectively, i=3… the number of the first target reserved. i F′1 and F′2, the line segment <F′1F′2> is the line segment <F′1F′2>, and the line segment <F′1F′2> is the line segment <F′1F′2>. i F′1 and F′2, the line segment <F′1F′2> is the line segment <F′1F′2>, and the line segment <F′1F′2> is the line segment <F′1F′2>. i > with the line segment <F′1F′2>; || represents taking absolute value; D1-D2 are preset configurable parameters respectively, i=3… the number of the first target reserved.

[0072] In the above two cases, the initial values of C1-C5, D1 and D2 are based on satellite orbit information, speed, optical parameters, field of view size, pixel size, etc. The initial values are determined based on ground tests. Then, based on the known fire point, the simulation calculation method is used to adjust C1-C5, D1 and D2 to determine the specific values that meet the identification requirements. In addition, C1-C5, D1 and D2 are all configurable parameters, which can also be adjusted in real time according to the detection results during in-orbit detection.

[0073] It should be noted that the extraction of the first target in the first infrared image and the extraction of the second target in the current second infrared image are both achieved by the background filtering method. In the target extraction, based on the detection threshold corresponding to each infrared image, the area with a gray value lower than the detection threshold is filtered out, and the first target and the second target that meet the requirements are left.

[0074] Finally, for step S3:

[0075] After the detection is completed, if the number of the first target remaining in the first infrared image is 0, it is output that no fire point is detected; if the number of the first target remaining in the first infrared image is not 0, the information of each remaining first target, including the shape and the position, is output, which is the forest fire detection result.

[0076] Through the above detection method of the present application, the calculation amount can be reduced to realize in-orbit detection, and the detection accuracy can be improved and the false alarm rate can be reduced because the fire point detection is based on the infrared multispectral image and the detection results of multiple spectral bands are fused.

[0077] As shown in Figure 2 , Figure 3 , the present application embodiment provides a kind of forest fire in-orbit detection device based on infrared multispectral image. Device embodiment can be realized by software, it can also be realized by hardware or software and hardware combination mode. From hardware layer, as shown in Figure 2 , it is a kind of hardware architecture diagram of the electronic equipment where the forest fire in-orbit detection device based on infrared multispectral image provided by the present application embodiment, except Figure 2 In addition to the processor, the memory, the network interface, and the non-volatile memory shown, the electronic device in which the apparatus in the embodiment is located can also generally include other hardware, such as a forwarding chip responsible for processing packets, and the like. For example, in the case of software implementation, as shown in the figure, as a logically meaningful apparatus, it is formed by the CPU of the electronic device in which it is located reading the corresponding computer program in the non-volatile memory into the memory and running. Figure 3

[0078] The embodiment provides a forest fire on-orbit detection apparatus based on an infrared multispectral image, which comprises:

[0079] The acquisition module 300 is configured to acquire a first infrared image and a preset number of second infrared images of a target region, the first infrared image and each second infrared image correspond to different spectral bands and detection thresholds, and the spectral bands and the detection thresholds are in one-to-one correspondence.

[0080] The extraction module 302 is configured to extract a first target in the first infrared image and record the position of each first target, the first target is a target with a gray value greater than the detection threshold corresponding to the first infrared image, and the number of first targets is at least one.

[0081] The detection module 304 is configured to perform fire point detection in the following manner:

[0082] The second infrared images are sequentially traversed, and for each traversed second infrared image, the following is performed:

[0083] S1, determining whether the number of current remaining first targets in the first infrared image is 0 or the number of current second infrared images reaches the preset number; if not, performing S2, and if yes, performing S3.

[0084] S2, extracting a second target in the current second infrared image and recording the position of each second target, the second target being a target with a gray value greater than the detection threshold corresponding to the second infrared image; estimating the estimated position of the current remaining first target in the first infrared image in the second infrared image; determining a first target to be matched based on each estimated position and the position of each second target; performing fire point matching on each first target to be matched and the corresponding second target, retaining the first target that matches successfully and removing the first target that fails to match; taking the first target that matches successfully as the current remaining first target of the first infrared image, adding 1 to the number of second infrared images, and returning to perform S1.

[0085] S3, ending the fire point detection and outputting a fire point detection result.

[0086] ​In the embodiments of the present application, the acquisition module 300 can be configured to perform step 102 in the above-mentioned method embodiments, the extraction module 302 can be configured to perform step 102 in the above-mentioned method embodiments, and the detection module 304 can be configured to perform step 104 in the above-mentioned method embodiments.

[0087] In some embodiments, the first infrared image and the second infrared image are obtained by a forest fire detection sensor based on a time-sharing imaging method, and the forest fire detection sensor is carried on a satellite.

[0088] In the execution of estimating the estimated positions of the current remaining first targets in the first infrared image in the second infrared image, the detection module 304 includes:

[0089] Based on the orbit information and the motion information of the satellite, a registration matrix from the first infrared image to the current second infrared image is estimated, and based on the registration matrix, the estimated positions of the current remaining first targets in the first infrared image in the second infrared image are estimated.

[0090] In some embodiments, the first infrared image and the second infrared image are obtained by a forest fire detection sensor based on a split-aperture method or a split-focal-plane method.

[0091] In the execution of estimating the estimated positions of the current remaining first targets in the first infrared image in the second infrared image, the detection module 304 includes:

[0092] Based on an optical reference conversion matrix measured by a ground device, a registration matrix from the first infrared image to the current second infrared image is calculated, and based on the registration matrix, the estimated positions of the current remaining first targets in the first infrared image in the second infrared image are estimated.

[0093] In some embodiments, in the execution of determining the first targets to be matched based on the estimated positions and the positions of the second targets, the detection module 304 includes:

[0094] For each estimated position, it is determined whether the Euclidean distance between the estimated position and the position of any second target is less than a preset value; if yes, the first target corresponding to the estimated position is determined as the first target to be matched, and if no, the first target corresponding to the estimated position is removed from the first infrared image.

[0095] In some embodiments, the number of the first targets to be matched is one or two.

[0096] In the execution of performing fire point matching between each first target to be matched and the corresponding second target, retaining the first target with successful matching, and removing the first target with failed matching, the detection module 304 includes:

[0097] For each first target to be matched, it is determined whether the following conditions are simultaneously satisfied:

[0098] or when P0>C2, P n -P0

[0099] If yes, it is determined that the first target and the second target corresponding thereto are matched successfully, and the first target is reserved; if no, it is determined that the first target and the second target corresponding thereto are not matched successfully, and the first target is removed.

[0100] In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the pixel number of the first target, P n is the pixel number of the second target corresponding to the first target, h0 and v0 are the horizontal pixel width and the vertical pixel height of the first target respectively, h n and v n are the horizontal pixel width and the vertical pixel height of the second target corresponding to the first target respectively, n = 1, 2, …, a preset number; C1-C5 are preset configurable parameters.

[0101] In some embodiments, the number of the first targets to be matched is greater than 2;

[0102] When the detection module 304 performs: performing fire point matching on each first target to be matched and the second target corresponding thereto, reserving the first target matched successfully, and removing the first target matched unsuccessfully, the detection module 304 includes:

[0103] For each first target to be matched, it is determined whether the following conditions are met at the same time:

[0104] or when P0>C2, P n -P0

[0105] If yes, the first target and the second target corresponding thereto are reserved; if no, the first target and the second target corresponding thereto are removed.

[0106] In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the pixel number of the first target, P n is the pixel number of the second target corresponding to the first target, h0 and v0 are the horizontal pixel width and the vertical pixel height of the first target respectively, h n and v n are the horizontal pixel width and the vertical pixel height of the second target corresponding to the first target respectively, n = 1, 2, …, a preset number; C1-C5 are preset configurable parameters.

[0107] For both the retained primary and secondary objectives, execute:

[0108] Calculate the Euclidean distance error between the position of each second target and its corresponding estimated position;

[0109] Find the two second targets with the smallest Euclidean distance error, and denote them as F1 and F2 respectively. Also denote the two first targets corresponding to the two second targets with the smallest Euclidean distance error as F′1 and F′2 respectively.

[0110] For other second objectives besides F1 and F2, F i And other first objectives F′ besides F′1 and F′2 i Determine whether the following conditions are met simultaneously:

[0111]

[0112] If yes, then the first target is determined to be a successful match with its corresponding second target, and the first target is retained; otherwise, the first target is determined to be a unsuccessful match with its corresponding second target, and the first target is removed.

[0113] In the formula, L i For F i The length of the line segment connecting the centroid of F1 and the centroid of F1, Y i For F′ i The length of the line segment connecting the centroid of F' and the centroid of F′1; θ i For F i In the triangle formed by the centroids of F1 and F2, the line segment <F1F i The angle between the line segment <F1F2> and the line segment <F1F2>; δ i For F′ i In the triangle formed by the centroids of F′1 and F′2, the line segment <F′1F′ i >and line segments<F′1F′2> The included angle; || represents taking the absolute value; D1~D2 are preset configurable parameters, i=3……the number of first targets retained.

[0114] In some implementations, extracting the first target from the first infrared image and extracting the second target from the current second infrared image are both achieved through background filtering.

[0115] It can be understood that the structural schematic of the embodiments of the present application does not constitute a specific limitation on the device for detecting forest fire in orbit based on infrared multispectral images. In other embodiments of the present application, the device for detecting forest fire in orbit based on infrared multispectral images can include more or fewer components than the schematic, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0116] The information interaction, execution process and the like between the modules in the device are based on the same concept as the method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be described here.

[0117] The embodiments of the present application also provide an electronic device including a memory and a processor, the memory stores a computer program, and the processor implements the method for detecting forest fire in orbit based on infrared multispectral images in any of the embodiments of the present application when executing the computer program.

[0118] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program makes the processor execute the method for detecting forest fire in orbit based on infrared multispectral images in any of the embodiments of the present application when being executed by the processor.

[0119] Specifically, a system or device equipped with a storage medium can be provided, and the storage medium stores software program codes for realizing the functions of any of the above embodiments, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0120] In this case, the program codes read from the storage medium can realize the functions of any of the above embodiments, and thus the program codes and the storage medium storing the program codes constitute a part of the present application.

[0121] The storage medium for providing the program codes includes floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, nonvolatile memory cards and ROM. Alternatively, the program codes can be downloaded from a server computer through a communication network.

[0122] In addition, it should be clear that not only the program codes read by the computer can be executed, but also the operating system and the like operating on the computer can be completed by the instructions based on the program codes to realize the functions of any of the above embodiments.

[0123] Further, it is understood that the programs read out from the storage medium can be written to the storage device installed in the extension board inserted into the computer or the storage device installed in the extension module connected to the computer, and then the CPU or the like mounted on the extension board or the extension module is caused to perform part or all of the actual operation based on the instructions of the program codes, thereby realizing the functions of any of the above-described embodiments.

[0124] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0125] Finally, it should be noted that the above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the same; even though the present application has been described in detail with reference to the foregoing embodiments, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for on-orbit detection of forest fires based on infrared multispectral images, characterized in that, include: Acquire a first infrared image of the target area and a preset number of second infrared images. The first infrared image and each second infrared image correspond to different spectral bands and detection thresholds, and the spectral bands and detection thresholds have a one-to-one correspondence. Extract the first target from the first infrared image and record the position of each first target. The first target is a target whose gray value is greater than the detection threshold corresponding to the first infrared image. The number of the first targets is at least one. Iterate through the second infrared images sequentially, and for each second infrared image encountered, perform the following: S1, determine whether the number of remaining first targets in the first infrared image is 0, or whether the number of current second infrared images has reached a preset number; if not, execute S2, if yes, execute S3; S2, extract the second target in the current second infrared image and record the position of each second target. The second target is the target whose gray value is greater than the detection threshold corresponding to the second infrared image. Estimate the estimated position of the remaining first target in the first infrared image in the second infrared image; Based on each estimated location and the location of each second target, the first target to be matched is determined; For each first target to be matched, perform fire point matching with its corresponding second target, retain the first target that matches successfully, and remove the first target that fails to match; The first target that is successfully matched is taken as the current remaining first target in the first infrared image, and the number of the second infrared images is incremented by 1. Then, return to execute S1. S3, end fire detection and output the fire detection result.

2. The method according to claim 1, characterized in that, The first infrared image and the second infrared image were obtained by a forest fire detection sensor based on a time-division imaging method, and the forest fire detection sensor is mounted on a satellite; The estimation of the estimated position of the remaining first target in the first infrared image in the second infrared image includes: Based on the satellite's orbital and motion information, the registration matrix from the first infrared image to the current second infrared image is estimated; Based on the registration matrix, the estimated position of the remaining first target in the first infrared image is estimated in the second infrared image.

3. The method according to claim 1, characterized in that, The first infrared image and the second infrared image were obtained by the forest fire detection sensor based on the aperture division method or the focal plane division method; The estimation of the estimated position of the remaining first target in the first infrared image in the second infrared image includes: Based on the optical reference transformation matrix measured by ground equipment, calculate the registration matrix from the first infrared image to the current second infrared image; Based on the registration matrix, the estimated position of the remaining first target in the first infrared image is estimated in the second infrared image.

4. The method according to claim 1, characterized in that, The process of determining the first target to be matched based on each estimated location and the location of each second target includes: For each estimated position, determine whether the Euclidean distance between the estimated position and the position of any second target is less than a preset value; if so, determine the first target corresponding to the estimated position as the first target to be matched; otherwise, remove the first target corresponding to the estimated position from the first infrared image.

5. The method according to claim 4, characterized in that, The number of first targets to be matched is 1 or 2; The step of performing fire point matching on each first target to be matched and its corresponding second target, retaining the first targets that successfully match and removing the first targets that fail to match, includes: For each target to be matched, determine whether the following conditions are met simultaneously: Or when P0>C2, P n -P0 <C4(h0+v0) If yes, then the first target is determined to be a successful match with its corresponding second target, and the first target is retained; otherwise, the first target is determined to be a unsuccessful match with its corresponding second target, and the first target is removed. In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the number of pixels of the first target, and P n Let h0 and v0 be the number of pixels of the second target corresponding to the first target, respectively, and h0 and v0 be the horizontal pixel width and vertical pixel height of the first target. n and v n The horizontal pixel width and vertical pixel height of the second target corresponding to the first target are respectively, n = 1, 2, ... the preset number; C1 to C5 are respectively related to the preset configurable parameters.

6. The method according to claim 4, characterized in that, The number of high-temperature targets to be matched is greater than 2; The step of performing fire point matching on each first target to be matched and its corresponding second target, retaining the first targets that successfully match and removing the first targets that fail to match, includes: For each target to be matched, determine whether the following conditions are met simultaneously: Or when P0>C2, P n -P0 <C4(h0+v0) If yes, then retain the first objective and the second objective corresponding to the first objective; otherwise, remove the first objective and the second objective corresponding to the first objective. In the formula, d is the Euclidean distance between the centroid of the second target corresponding to the first target and the estimated position of the first target in the second infrared image, P0 is the number of pixels of the first target, and P n Let h0 and v0 be the number of pixels of the second target corresponding to the first target, respectively, and h0 and v0 be the horizontal pixel width and vertical pixel height of the first target. n and v n These are the horizontal pixel width and vertical pixel height of the second target corresponding to the first target, respectively, n = 1, 2, ... the preset number; C1 to C5 are preset configurable parameters; For both the retained primary and secondary objectives, execute: Calculate the Euclidean distance error between the position of each second target and its corresponding estimated position; Find the two second targets with the smallest Euclidean distance error, and denote them as F1 and F2 respectively. Also denote the two first targets corresponding to the two second targets with the smallest Euclidean distance error as F′1 and F′2 respectively. For other second objectives besides F1 and F2, F i And other first objectives F besides F′1 and F′2 i Each condition is checked to see if it simultaneously satisfies the following conditions: If yes, then the first target is determined to be a successful match with its corresponding second target, and the first target is retained; otherwise, the first target is determined to be a unsuccessful match with its corresponding second target, and the first target is removed. In the formula, L i For F i The length of the line segment connecting the centroid of F1 and the centroid of F1, Y i For F i The length of the line segment connecting the centroid of F′1 and the centroid of F′1; θ i For F i In the triangle formed by the centroids of F1 and F2, the line segment <F1 F i >and line segments<F1 F2> The included angle; δ i For F i In the triangle formed by the centroids of F′, F′1, and F′2, the line segment <F′1 F′ i >and line segments<F′1 F′2> The included angle; || represents taking the absolute value; D1~D2 are preset configurable parameters, i=3……the number of first targets retained.

7. The method according to claim 1, characterized in that, Extracting the first target from the first infrared image and extracting the second target from the current second infrared image are both achieved through background filtering.

8. A forest fire on-orbit detection device based on infrared multispectral images, characterized in that, include: The acquisition module is used to acquire a first infrared image of the target area and a preset number of second infrared images. The first infrared image and each second infrared image correspond to different spectral bands and detection thresholds, and the spectral bands and detection thresholds have a one-to-one correspondence. The extraction module is used to extract the first target in the first infrared image and record the position of each first target. The first target is a target whose gray value is greater than the detection threshold corresponding to the first infrared image. The number of the first targets is at least one. The detection module is used to detect fire points in the following manner: Iterate through the second infrared images sequentially, and for each second infrared image encountered, perform the following: S1, determine whether the number of remaining first targets in the first infrared image is 0, or whether the number of current second infrared images has reached a preset number; if not, execute S2, if yes, execute S3; S2, extract the second target in the current second infrared image and record the position of each second target. The second target is the target whose gray value is greater than the detection threshold corresponding to the second infrared image. Estimate the estimated position of the remaining first target in the first infrared image in the second infrared image; Based on each estimated location and the location of each second target, the first target to be matched is determined; For each first target to be matched, perform fire point matching with its corresponding second target, retain the first target that matches successfully, and remove the first target that fails to match; The first target that is successfully matched is taken as the current remaining first target in the first infrared image, and the number of the second infrared images is incremented by 1. Then, return to execute S1. S3, end fire detection and output the fire detection result.

9. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.

Images