Satellite-supported forest fire analysis
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- DEUTSCHES ZENTRUM FÜR LUFT UND RAUMFAHRT E V
- Filing Date
- 2024-08-07
- Publication Date
- 2026-06-17
Smart Images
Figure EP2024072299_13022025_PF_FP_ABST
Abstract
Description
[0001] Satellite-based forest fire analysis
[0002] The invention relates to a device for determining an area affected by fire in an open-area fire, as well as a computer-implemented method for determining an area affected by fire in an open-area fire.
[0003] Forest fires, particularly in large, contiguous forest areas and in forest areas in inaccessible and expansive regions, can often only be recorded with sufficient accuracy in terms of their extent and severity with considerable effort. While smaller areas can be recorded using observation towers or manned or unmanned aircraft, satellite-based analysis is suitable for larger areas.
[0004] When determining the size and / or intensity of the areas affected by a forest fire, analyses can be differentiated depending on their time horizon into:
[0005] - medium- to long-term data analysis: This analysis is not time-critical and its results are therefore particularly suitable for decisions on reforestation and similar measures;
[0006] - Short-term data analysis: This analysis is time-critical, especially if it is used as a data basis for targeted firefighting and assessment of a hazard situation;
[0007] Short-term data analysis is highly important for its applications, but is often hampered by disruptive factors such as smoke columns or cloud cover. This typically severely limits the evaluation of optical data in near-real time. Therefore, in practice, not only optical data is often used, but also additional data for support, which, in particular, indicate thermal anomalies in a particular area of the Earth's surface and can possibly be obtained from other satellite sensors or a service provider. Such anomalies can at least be used to identify active fires above a certain temperature and the associated thermal radiation emission, so-called hotspots.While an exclusively rule-based approach for more accurate, time-critical detection of burnt areas would require manual calibration of a model as well as consideration of and knowledge of predominant vegetation in the (potential) burnt area, machine learning methods would not require manual calibration, but would involve complex classification of training data and have the disadvantage that the analysis only leads to reliable results if the data available in the respective application case are sufficiently similar in terms of input variables to the training data - unknown vegetation types (i.e., deviating from the training data) would typically lead to an unacceptable number of false detections.
[0008] The current state of the art is the European Forest Fire Information System (EFFIS) of the Copernicus program. This data is not collected fully automatically; the analysis and quality assessment are largely performed manually. Results are therefore only generated during office hours and updated accordingly. This analysis method can also result in staff neglecting lower-priority areas in crisis situations, resulting in quality differences and variations in data coverage. Fully automated data analysis in near-real time upon arrival of new satellite data is not provided.
[0009] The object of the invention is therefore to provide a reliable method to determine in a timely manner (i.e., as explained above, in short-term data analysis) at least the extent of one or more individual forest fire areas.
[0010] The invention is based on the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims.
[0011] A first aspect of the invention relates to a device for determining an area affected by an open-area fire, comprising a first interface for providing satellite-captured and georeferenced image data of an area of the Earth's surface, a second interface for providing at least one first reference area in the area that is severely affected by fire and at least one second reference area in the area that is completely unaffected by fire, and comprising a computing unit configured to calibrate a respective classification between "affected by fire" and "not affected by fire" using a respective first and second reference area, to simplify the obtained image data so that a plurality of adjacent pixels of the image data that satisfy a similarity condition to one another are combined to form a respective pixel group,and starting from a selected first reference area or a pixel group located in the selected first reference area, using a division calibrated using the selected or another first reference area, to check pixel groups located in the path step by step along a respective path of adjacent pixel groups starting from the first reference area or from the pixel group located in the selected first reference area to determine whether they are affected by fire, and to abort the path if a pixel group on this path is classified as "not affected by fire", in order to identify, starting from the respective selected first reference area, all adjacent and adjacent pixel groups affected by fire as a respective part of an entire fire-affected area of an open-space fire.
[0012] An open-air fire can be a forest fire, a bush fire, a steppe fire, or similar; typically, vegetation is primarily or exclusively affected.
[0013] Georeferenced imagery is data whose content can be uniquely assigned to a specific coordinate on the Earth's surface. For example, each pixel of the image data has a reference coordinate in the WGS-84 coordinate system, and therefore has a longitude and a latitude coordinate with a specific number of decimal places.
[0014] The first interface is used to obtain reference areas. A first reference area severely affected by fire can indicate an area currently burning heavily or an area that has already been heavily burned. Initial reference areas are suitable for the initial selection and specification of respective pixel groups, from which the gradual verification of further adjacent pixel groups and of each other can be initiated. Such an initial reference area severely affected by fire is obtained, in particular, from an external source via the first interface and can be determined, in particular, by detecting a thermal anomaly.
[0015] Another reference surface obtained from the first interface indicates an area not affected by fire. Such a second reference surface can indicate an area that is not currently burning or an area that was not burned by a past fire. Such a reference surface is characterized in particular by a high reflectance value in the near-infrared range.
[0016] The term "affected by fire" can mean either a current fire or a burned area. The computing unit may be designed to determine only one of the alternatives, but advantageously it can determine both and make a distinction, particularly using the calibrated classification.
[0017] In other words, the computing unit's approach is a graph traversal approach, in which, starting from one (or more) initial pixel groups determined by a respective first reference area, further adjacent pixel groups are traversed step by step within a graph (a neighborhood graph). The initial pixel group is classified by definition as being affected by fire. Subsequently, all pixel groups (hereinafter referred to as the first pixel groups) adjacent to the initial pixel group are checked for fire affliction. Subsequently, second pixel groups adjacent to those first pixel groups for which a fire affliction check was positive are checked for fire affliction.
[0018] The test for fire vulnerability is carried out in particular depending on the spectral values of the pixel groups, whereby uniform spectral values are assumed across the pixel groups, for example by averaging the spectral values of the pixels contained in the pixel groups.
[0019] The area affected by the fire is classified as fire-affected or unaffected depending on the severity of the current fire according to the image data or the severity of the vegetation burning that has already occurred. The classification is carried out using the calibrated classification. By obtaining at least one georeferenced first reference area that is severely affected by the fire and at least one second reference area that is negligibly or not at all affected by the fire using external information via the second interface, a scale for the severity of the fire impact can be defined and, in particular, the classification can be used to define a boundary condition for whether a group of pixels is classified as fire-affected or not. The second reference area shows areas in the area that are not completely or equivalently almost not affected by fire, in other words, negligibly or slightly affected.The calibration of the classification refers in particular to the data values of the pixels or pixel groups, so that calculable quantities are assigned to the image data, which can be calculated in particular from the color values and / or brightness values of the pixels or pixel groups.
[0020] The classification is therefore dependent on the reference areas. Because the reference areas are located within the area being examined, this threshold value is in turn dependent on the area itself and is thus automatically calibrated to the area itself. Therefore, there is no need for manual support or input of information regarding which vegetation types are predominant in the observed, delimited area, and in particular, no need to train a machine learning model with area-dependent images.
[0021] By starting from a respective first reference area, and by defining a pixel group within it as definitely affected by fire, step-by-step checks can be carried out geometrically starting from the respective first reference area of the respective adjacent pixel groups. The step-by-step check is extended to further adjacent pixel groups of an already checked pixel group until a respective pixel group is classified as not affected by fire. This is a reliable approach, but it already omits the testing of pixel groups that are in all probability not affected by fire at all, since they neither lie in a first reference area nor border it, nor border a pixel group that has been classified as affected by fire.
[0022] The respective selected first reference area serves as the geometric starting point for checking further adjacent pixel groups for fire affliction on one or more paths leading away from it. If there is only a single first reference area in an area, only this single first reference area can be selected. If several first reference areas have been provided externally, the extent of the area affected by fire is determined, preferably sequentially or in parallel, starting from the first reference area selected from this set of first reference areas and using corresponding paths from adjacent pixel groups. In general, it is advisable to use all immediately adjacent pixel groups as the first step of a respective path, starting from a respective first reference area or a complete pixel group located in the first reference area.As long as no further pixel group concentrically surrounds the first reference area or the pixel group lying entirely within the first reference area, the pixel groups in several paths emanating from each selected first reference area must be checked.
[0023] The checking of individual pixel groups along a respective path around the selected first reference area for fire affliction is carried out using the calibrated classification, which was calibrated using a first reference area.Preferably, the division calibrated by means of the respectively selected first reference surface itself is used in each case, however, a division that was calibrated with a nearby first reference surface can also be used to check the pixel groups around a respective selected first reference surface - in general, it can be assumed that the use of the division calibrated with the respectively selected first reference surface is ideal for checking the pixel groups on the same respectively selected first reference surface and the use of a division that was calibrated by means of a first reference surface that is further away reduces the quality of the evaluations as the distance from the first reference surface increases.
[0024] One advantageous effect of the invention is that a large-scale analysis of the extent of forest fires can be carried out fully automatically and promptly. The rapid provision of comprehensive results allows for a correspondingly timely response, for example, for firefighting. Execution time tests have shown that, with currently available hardware, the results of the analysis are available just a few minutes after receiving the satellite data. The high degree of automation of the analysis method not only increases speed but also allows for frequent updates by repeating the analyses based on newly received satellite data, even from different sensors. There is no limitation on office hours when the analyses are carried out fully automatically.
[0025] Furthermore, any area of any size can be analyzed; there is no need to limit the analysis to risk areas. Another advantage of the analysis method is that it allows for the use of a wide variety of different sensors; the implementation of the analysis method is not geared to a specific sensor with specific characteristics. Optical satellite sensors can be used for a data basis, low- and high-resolution images can be applied, and a certain number of different spectra can be used, for example, in the optical range or multispectral sensors. Almost any resolution of the images from the satellite data can be processed, although higher resolutions can increase computing time, while too low resolutions may result in significant loss of spatial information.
[0026] The so-called single-scene approach is used, which, together with the low bandwidth requirements, makes the integration of new sensors into the processing significantly easier than would be possible with exclusive applications of artificial neural networks and other machine learning methods. This single-scene approach also makes it possible to avoid computationally intensive preprocessing steps such as co-registration.
[0027] A test method can also be implemented that applies a machine learning approach to node classification (so-called "node classification"). The algorithm learns from initial pixel groups and applies the learned rules to the remaining nodes. Neighborhood relationships and edge weights are included in the analysis.
[0028] According to an advantageous embodiment, the computing unit is designed to carry out the simplification of the image data by applying a superpixel algorithm to the image data, so that a respective pixel group forms a superpixel.
[0029] Superpixel techniques segment image data into regions by considering distance measures defined using perceptual features. A superpixel can therefore be defined as a group of pixels that share common features (e.g., pixel intensity, spectral information, etc.). By aggregating pixels into superpixels, the local redundancy of an image is reduced or even eliminated, while allowing negligible information loss and essentially preserving local structures. This significantly reduces the computational effort of subsequent steps and allows for easy implementation of the iterative approach from the initial superpixel to neighboring superpixels. Modeling the set of superpixels as an undirected neighborhood graph involves specifying the neighborhood relationships between the superpixels.These relationships, together with the superpixels, form the elementary input for subsequent evaluation, for example a rule-based evaluation or an evaluation using a machine learning model.
[0030] According to a further advantageous embodiment, the computing unit is designed to model the set of pixel groups as an undirected neighborhood graph by modeling the pixel groups as nodes of the neighborhood graph and neighborhood relationships between the pixel groups as edges of the neighborhood graph.
[0031] By simplifying the image data, for example, by applying the superpixel technique, direct neighborhood relationships between pixel groups can be lost, making the step-by-step examination, starting with the initial pixel group and then gradually progressing through adjacent pixel groups, more difficult or impossible, as information about the exact location and orientation of the adjacent pixel groups and the properties of the problem itself can be distorted or lost. By introducing a neighborhood relationship when modeling the set of pixel groups into an undirected neighborhood graph, such information can be restored and used.
[0032] According to a further advantageous embodiment, the computing unit is designed to provide the edges with a respective weighting, wherein the respective weighting indicates a probability for further fire spread.
[0033] According to a further advantageous embodiment, the image data comprises optically acquired data, in particular in a spectral band combination of red and near-infrared, or near-infrared and short-wave infrared.
[0034] According to a further advantageous embodiment, the computing unit is designed to make the decision to classify a group of pixels as affected by fire using a machine learning model.
[0035] According to a further advantageous embodiment, the computing unit is designed to make the decision to classify a pixel group as affected by fire in a rule-based manner, for example by comparing the information of a pixel group such as color and / or brightness with a predetermined comparison condition.
[0036] According to a further advantageous embodiment, the computing unit is designed to mark pixels and / or pixel groups as having no data. So-called "no-data" areas can therefore be excluded from the further examination of the fire areas.
[0037] According to a further advantageous embodiment, the computing unit is designed to repeatedly carry out the determination of the pixel groups affected by fire by means of current georeferenced image data of an area for a common set of reference areas in each case for the purpose of updating.
[0038] A further aspect of the invention relates to a computer-implemented method for determining an area affected by an open-area fire, wherein satellite-captured and georeferenced image data of an area of the Earth's surface, one or more first reference areas in the area that are severely affected by fire, and at least one second reference area in the area that is completely unaffected by fire are provided, wherein a respective classification between "affected by fire" and "not affected by fire" is calibrated using a respective first and a second reference area, and the obtained image data is simplified so that a plurality of adjacent pixels of the image data that satisfy a predetermined similarity condition to one another are combined to form a respective pixel group.and wherein, starting from a selected first reference area or a pixel group located in the selected first reference area, using a classification calibrated by means of the selected or another first reference area, adjacent pixel groups are checked step by step along a respective path to determine whether they are affected by fire, wherein the path is aborted if a pixel group on this path is classified as "not affected by fire", in order to identify, starting from the respectively selected first reference area, all adjacent and adjacent pixel groups affected by fire as a respective part of an entire fire-affected area of an open-space fire.
[0039] Advantages and preferred developments of the proposed method arise from an analogous and analogous application of the statements made above in connection with the proposed device. Further advantages, features, and details emerge from the following description, in which at least one embodiment is described in detail—where appropriate with reference to the drawings. Identical, similar, and / or functionally equivalent parts are provided with the same reference numerals.
[0040] They show:
[0041] Fig. 1 : Image data from a satellite with pixels according to an embodiment of the invention.
[0042] Fig. 2: A simplification of the image data of Fig. 1 .
[0043] Fig. 3: Reference fields in the simplified image data of Fig. 2.
[0044] Fig. 4: A calibrated division using the reference fields of Fig. 3.
[0045] Fig. 5: Schematic of an exemplary step-by-step test of superpixels.
[0046] Fig. 6: Areas affected by fire in the areas of the region assigned to the image data according to an embodiment of the invention.
[0047] Fig. 7 / 8: An exemplary practical result.
[0048] The representations in the figures are schematic and not to scale. Figures 1 to 6 show a schematic procedure for determining the area affected by an open-area fire.
[0049] A satellite in orbit around the Earth receives image data in the red / infrared range from a sensor on the satellite. This image data is composed of individual pixels 5. Fig. 1 shows an example section of this image data with the individual pixels 5, each of which has a specific composition of sensor channels, in particular a specific color value. All pixels 5 can be assigned a coordinate of the Earth's surface in order to be able to assign them to locations on a digital map. In Fig. 2, starting from Fig. 1, similar pixels 5 are grouped together to a degree that depends on the pixel density per area on the Earth's surface. This reduces the amount of data and facilitates the processing of the information in the image data in the following steps by reducing the computing effort.In the simplification step, as a preparatory measure, all pixels in a scene of interest are converted from the image data into the superpixel structure. Similar, neighboring pixels are combined into a homogeneous pixel group 7 (outlined in Fig. 2 by thickened, black, closed-curved border lines). This simplifies the subsequent classification and greatly increases the execution speed when checking areas of the scene for fire exposure. A superpixel algorithm is used here, which can process not only the red / green / blue channels, but also the corresponding number of input channels required by the image data from the remote sensing data. Furthermore, the superpixel algorithm is compatible with masked pixel values, i.e., pixels 5 that do not contain any image values but reflect so-called "no-data" areas.Such masking can also be applied when the Earth's surface is obscured by clouds and / or smoke to avoid false positive results in fire detection. Furthermore, the superpixel algorithm can be designed to consider a maximum value distance to maintain control over the algorithm's sensitivity. Figure 2 shows exemplary pixel groups 7 in the form of superpixels. These groups are naturally flat due to the grouping of similar pixels 5, but are simplified and only partially labeled as pixel groups 7.
[0050] In Fig. 3, further processing of the data takes place in the status shown in Fig. 2. Fire areas detected by thermal anomalies are transmitted georeferenced by an external service provider and incorporated into the area's image data as initial reference areas 1. These initial reference areas 1 are shown fully filled in Fig. 3. Superpixels that lie entirely within these reference areas 1 are marked accordingly in the simplified data set. In addition, at least one second reference area 3 is specified with a position assignment, which, according to the external service provider or the user's own manual definition, is not affected by fire at all.
[0051] In Fig. 4, a scale is calibrated using the first reference surface 1 and the second reference surface 3. The data of the superpixels within the respective reference surface 1, 3 are evaluated for their properties and defined as the extremes of the scale. This also makes it possible to define a boundary condition for the middle between the data values belonging to the respective first reference surface 1 and the respective second reference surface 3, with which the respective available data of the superpixels can be compared. Using the respective boundary condition defined in this way and related to a respective first reference surface 1, and starting from the situation in Fig. 3, a respective surrounding superpixel around the superpixels completely within a respective first reference surface 1 is first examined for exceeding or falling below the associated boundary condition in order to determine the maximum or minimum values for the superpixels according to the method shown in Fig.4 to determine whether a respective adjacent superpixel is classified as affected by the fire or not, see Fig. 5.
[0052] Fig. 5 schematically shows this step-by-step procedure. The examination for "affected by fire" is carried out for all superpixels surrounding the first reference area 1 (or the superpixel(s) located within it). If a surrounding superpixel is classified as not affected by fire, the examination of further adjacent superpixels ends at this point. If, however, such an adjacent superpixel is classified as affected by fire, the procedure is repeated and the superpixels adjacent to this superpixel as affected by fire are examined to determine whether they are affected by fire according to the calibrated classification, etc. This results in paths as shown by the arrows in Fig. 5; for the sake of simplicity, Fig.In Figure 5, only the superpixel corresponding to reference area 1 is shown as black, indicating that it is affected by fire, although the review of further fire-affected superpixels ends when a superpixel in this path is classified as not affected by fire. The result is a contiguous area of adjacent superpixels around the superpixel(s) located entirely within reference area 1, which is classified as affected by fire (see Figure 6).
[0053] Fig. 6 shows the result of the procedure in Fig. 5, with Fig. 3 representing the starting point. All superpixels completely filled with black were classified as fire-affected. All superpixels classified as fire-affected are arranged in a contiguous area around the first reference surface 1.
[0054] Practical results are shown in Fig. 7 and Fig. 8, which show the same visualization simply in different levels of brightness and contrast. Details in Fig. 7 and Fig. 8 are not relevant; rather, they serve to illustrate a practical result. In Fig. 7, the dark areas in the center of the image are shown as areas affected by a past fire. Figs. 7 to 8 do not show any device objects or process sequences within a respective figure. The recognition of details is therefore irrelevant. Figs. 7 to 8 merely serve to illustrate an exemplary embodiment of the device. This does not rule out the possibility that deviations from the exemplary method or the observation of other areas of the Earth's surface will lead to completely different appearances.
[0055] Although the invention has been illustrated and explained in detail by preferred embodiments, the invention is not limited by the disclosed examples, and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of variations exist. It is also clear that exemplary embodiments are truly only examples and should not be construed as limiting the scope, possible applications, or configuration of the invention in any way.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.
[0056] List of reference symbols
[0057] 1 first reference surface
[0058] 3 second reference surface
[0059] 5 pixels
[0060] 7 pixel group
Claims
Patent claims 1. A device for determining an area affected by an open-area fire, comprising a first interface for providing georeferenced image data of an area of the Earth's surface acquired by a satellite, a second interface for providing at least one first reference area (1) in the area that is severely affected by fire, and at least one second reference area (3) in the area that is completely unaffected by fire, and comprising a computing unit designed to calibrate a respective classification between "affected by fire" and "not affected by fire" using a respective first and second reference area (3), to simplify the obtained image data such that a plurality of adjacent pixels (5) of the image data that satisfy a similarity condition to one another are combined to form a respective pixel group (7),and starting from a selected first reference area (1) or a pixel group (7) located in the selected first reference area (1), using a division calibrated by means of the selected or another first reference area (1), to check pixel groups (7) located in the path step by step over a respective path of adjacent pixel groups (7) starting from the first reference area (1) or from the pixel group (7) located in the selected first reference area (1) to determine whether they are affected by fire, and to abort the path if a pixel group (7) on this path is classified as "not affected by fire", in order to identify, starting from the respectively selected first reference area (1), all adjacent pixel groups (7) affected by fire as a respective part of an entire fire-affected area of an open-area fire.
2. Device according to claim 1, wherein the computing unit is designed to carry out the simplification of the image data by applying a superpixel algorithm to the image data, so that a respective pixel group (7) forms a superpixel.
3. Device according to one of the preceding claims, wherein the computing unit is designed to model the set of pixel groups (7) as an undirected neighborhood graph by the pixel groups (7) as nodes of the neighborhood graph and neighborhood relationships between the pixel groups (7) are modeled as edges of the neighborhood graph.
4. Device according to claim 3, wherein the computing unit is designed to provide the edges with a respective weighting, wherein the respective weighting indicates a probability of further fire spread.
5. Device according to one of the preceding claims, wherein the image data comprises optically acquired data, in particular in a spectral band combination of red and near-infrared, or near-infrared and short-wave infrared.
6. Device according to one of claims 1 to 5, wherein the computing unit is designed to make the decision to classify a pixel group (7) as affected by fire using a machine learning model.
7. Device according to one of claims 1 to 5, wherein the computing unit is designed to make the decision to classify a pixel group (7) as affected by fire in a rule-based manner.
8. Device according to one of the preceding claims, wherein the computing unit is designed to mark pixels (5) and / or pixel groups (7) as dataless which have been manually designated for this purpose.
9. Device according to one of the preceding claims, wherein the computing unit is designed to repeatedly carry out the determination of the pixel groups (7) affected by fire by means of respectively current georeferenced image data of an area for updating purposes, while maintaining the reference areas (1, 3) and the classification(s) determined thereby.
10. Computer-implemented method for determining an area affected by a fire of an open-area fire, wherein image data of an area of the earth's surface acquired and georeferenced by a satellite, one or more first reference areas (1) in the area which are severely affected by fire and at least one second reference area (3) in the area which is completely unaffected by fire are provided, wherein a respective classification between "affected by fire" and "not affected by fire" is calibrated using a respective first and a second reference surface (3), and the obtained image data is simplified so that a plurality of adjacent pixels (5) of the image data that satisfy a predetermined similarity condition to one another are combined to form a respective pixel group (7), and wherein, starting from a selected first reference surface (1) or a pixel group (7) located in the selected first reference surface (1), using a classification calibrated by means of the selected or another first reference surface (1), adjacent pixel groups (7) are checked step by step over a respective path to determine whether they are affected by fire, wherein the path is aborted if a pixel group (7) on this path is classified as "not affected by fire",in order to identify, starting from the respectively selected first reference area (1), all adjacent and adjacent pixel groups (7) affected by fire as a respective part of an entire area affected by fire of an open-air fire.