Mining area fine land identification method based on multi-mode fusion and first inflection point constraint

Abstract

The invention discloses a mining area fine land identification method based on multi-mode fusion and first inflection point constraint. The method comprises the following steps: step A, downloading a research area satellite remote sensing image, constructing a strip mine area space range sample data set, training a DeepLabv3 + deep learning model, and predicting and delineating the maximum space range of a mining area in a research period; step B, acquiring mining area land utilization change data, constructing a time sequence change mode based on a land utilization type transfer relationship, and dividing the time sequence change mode into five types, namely a damaged area, a reclamation area, a reclamation degradation area, a building pressure occupation area and a water body pressure occupation area; and step C, preparing a mining area fine land utilization sample data set, establishing a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net), training the model to identify mining area annual fine land utilization types, and accurately tracking first transition time information of each type. Compared with the prior art, the mining area land utilization classification is further refined, the proposed MFFIC-Net model fuses long and short term memory and a feedback mechanism, and accurate identification of the land utilization type and the first change time is realized. A feedback mechanism enables the model to self-adjust a prediction result, the precision and the stability are improved, and multi-mode fusion enhances the adaptability of the model to a complex mining area environment. The mining area land utilization identification method provided by the invention is finer and more practical, can effectively identify key areas such as a damaged area, a reclamation area and an occupation area, and provides important technical support for ecological restoration and land management.

Description

Technical Field

[0001] The present invention relates to the fields of mining, remote sensing and geographic information, and in particular to a method for fine land recognition in mining areas based on multi-mode fusion and first turning point constraint. Background Art

[0002] The stable supply of coal is related to national energy security, and open-pit coal mines are an important guarantee for coal supply. However, open-pit mining inevitably damages the ecological environment of mining areas. Land use changes in open-pit mining areas, especially land use changes in damaged areas, reclamation areas, and occupied areas in mining areas, are directly related to the planning of ecological environmental restoration and sustainable development in mining areas. Therefore, how to accurately and quickly identify the types and distribution of damaged areas, reclamation areas, and occupied areas in mining areas has become a key issue in mining area management and ecological environmental governance.

[0003] Traditional land use monitoring methods in mining areas mainly rely on manual interpretation or traditional remote sensing image classification technology, which has problems such as high cost, low efficiency, and insufficient accuracy. With the rapid development of remote sensing technology and deep learning, mining land use identification technology based on remote sensing images has become a research hotspot. However, existing studies focus on the identification of areas such as spoil dumps, stopes, and transfer yards in mining areas. These land uses mainly reflect the mining process in mining areas and are difficult to meet the monitoring needs of key land use types such as damaged areas, reclaimed areas, and occupied areas in ecological and environmental supervision. Therefore, in view of the needs of ecological and environmental restoration and supervision in mining areas, accurate identification of these key land use types has become an important issue that needs to be solved urgently. At present, the identification methods of damaged areas and reclaimed areas in mining areas are usually based on long-term series spectral data or vegetation index for curve fitting and segmented analysis, such as LandTrendr algorithm, CCDC algorithm, Auto-VDR algorithm, etc. Although these methods can identify damaged areas and reclaimed areas to a certain extent, due to the single land use type and the algorithm's reliance on spline curve construction and multiple parameter settings, the algorithm complexity is high and the practicality is limited. Therefore, there is an urgent need for a more refined and practical method for identifying key land uses such as damaged areas, reclaimed areas and occupied areas in mining areas. Summary of the invention

[0004] The purpose of the present invention is to propose a method that can efficiently and quickly extract the spatial distribution of various land use types such as damaged areas, reclamation areas, building occupation areas, water body occupation areas and reclamation degradation areas of open-pit coal mines and their occurrence time information.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: a method for fine land identification in mining areas based on multi-mode fusion and first inflection point constraint, the method steps are as follows:

[0006] Step A: Download the satellite remote sensing images of the study area, construct a sample dataset of the spatial scope of the open-pit mining area, train the DeepLabv3+ deep learning model, predict the annual distribution of the spatial scope of the mining area during the study period through the trained model, and comprehensively analyze the data of all time nodes to delineate the maximum spatial scope of the mining area during the study period;

[0007] Step B: Obtain the land use change data within the maximum spatial scope of the mining area, construct a temporal change pattern of the land use types in the mining area based on the transfer relationship between land use types, and divide the change pattern into five categories: damaged area, reclaimed area, reclaimed degraded area, building occupied area, and water body occupied area;

[0008] Step C: Based on the constructed temporal change pattern of the land use types in the mining area, prepare a sample dataset of the fine land use types in the mining area, and establish a multi-mode fusion and first inflection constrain neural network model (MFFIC-Net, Multi-Mode Fusion and First Inflection Constrain Neural Network) for the identification of the fine land use types in the mining area. Use the sample dataset to train the model, and apply the trained MFFIC-Net to the identification of the annual fine land use types in the mining area, while accurately tracking the first transition time information of each type.

[0009] Preferably, in Step A, the method for delineating the spatial scope of the open-pit mining area is divided into the following three steps

[0010] First, download the high-resolution remote sensing images of the target area, combine the geographical information and mining area distribution data obtained through actual investigations, and identify the spatial scope of the open-pit coal mining area through visual interpretation to form vector sample label data;

[0011] Convert the vector sample labels into raster format, spatially align them with the remote sensing images, and divide the remote sensing images and their corresponding vector label data into small samples to form a sample dataset of the open-pit mining area corresponding to the images;

[0012] Second, train the DeepLabv3+ model based on the constructed sample dataset of the open-pit mining area. In the encoding stage, the model uses Xception as the basic feature extraction network and introduces an atrous spatial pyramid pooling (ASPP) module; in the decoder stage, the upsampling technique is used to restore and enhance the detailed information in the image;

[0013] Finally, use the model obtained through training to predict the remote sensing images of each year during the study period, extract the spatial scope of the open-pit mining area at each time point, and perform a union operation on the annual prediction results to obtain the maximum mining area spatial scope covering all years.

[0014] Preferably, in step B, the original land use data within the maximum spatial range of the mining area is obtained, and based on the transfer relationship between land use types, the temporal change pattern of land use types in the mining area is constructed. The change pattern is divided into five categories: damaged area, reclaimed area, reclamation degradation area, building occupation area, and water body occupation area, which are specifically divided into the following three steps.

[0015] First, based on the vector of the mining area spatial range obtained in step A, download the land use dataset in the study area from May to September every year.

[0016] Use the mode calculation method to obtain the annual land use classification data within the study area, and classify its land use types into four categories: vegetation, bare land, water body, and building.

[0017] Secondly, construct a transfer matrix between the inducted land use types (water body, vegetation, building, bare land).

[0018] Define the transformed target land use types as five types of fine land use types in the mining area: reclaimed area, damaged area, building occupation area, water body occupation area, and reclamation degradation area.

[0019] Finally, construct the temporal change pattern of the five types of fine land use types in the mining area, that is, the temporal change pattern of land use types in the reclaimed area, damaged area, building occupation area, water body occupation area, and reclamation degradation area.

[0020] Preferably, in step C, based on the constructed temporal change pattern of land use types in the mining area, prepare a sample dataset of fine land use types in the mining area, and establish a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net) for the identification of fine land use types in the mining area. Use the sample dataset to train the model, and apply the trained MFFIC-Net to the identification of annual fine land use types in the mining area, and at the same time accurately track the first change time information of each type, which is specifically divided into the following three steps.

[0021] First, use the ArcGIS tool, combine the temporal change pattern of the five types of fine land use types in the mining area with land data and field survey data, and determine the fine land use types of the surface coal mine and the time of the first change through visual interpretation, so as to form a sample dataset of fine land use types in the mining area that corresponds one by one with the original land use time series data.

[0022] Secondly, based on the temporal change pattern of the five types of fine land use types in the mining area, construct a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net) for the identification of fine land use types in the mining area.

[0023] The MFFIC-Net model consists of an input layer, a Long-Short Feedback Memory Network (LSFM) layer, a multi-modal fusion and first inflection point constraint layer, and an output layer;

[0024] The input layer, Long-Short Feedback Memory (LSFM) layer, multi-modal fusion and first inflection point constraint layer, and output layer are respectively as follows:

[0025] Input layer: It consists of the extracted long time-series land use change sequences of pixels. The annual land use types (X1, X2, ……, X t , where t represents the year sequence) of each pixel are used as the input data of the model and are sequentially input into the neural network for processing;

[0026] Long-Short Feedback Memory (LSFM) layer: It is used to judge the fine land use types of pixels. Based on the input data of adjacent time steps, LSFM judges the land use type change patterns at each time node, and then determines the target fine land use types at each time node. The output results include the short-term result (S t ) of this node and the long-term memory result (L t ) of the memory cells. When judging the change of the target fine land use type, the LSFM model not only needs to remember the information of the previous time node, but also the output result of the current time node will affect the result of the previous time node. Therefore, the LSFM network layer combines the long-term and short-term memory mechanisms and the feedback mechanism to effectively handle the long-term and short-term dependencies and feedback relationships;

[0027] Among them, the long-term and short-term memory mechanisms and the feedback mechanism of the LSFM network layer are realized by the gating mechanism. LSFM consists of a forget gate (F t ), an input gate (I t ), an output gate (O t ), and a feedback gate (B t ). At the same time, the information of the long-term memory is stored and updated by the memory cell (L t ). The input of LSFM: L t-1 , S t-1 , X t ; The output of LSFM: L t (long-term memory information), S t (short-term output result). The operating mechanism of LSFM: (1) Forget gate: At time t, the input data X t is first input through the input layer and is combined with the short-term result S t-1 of the previous time node.Combine and pass it to the forget gate. The forget gate controls the retention degree of the information passed from the previous moment through the sigmoid (σ) activation function. The output of the forget gate is the retention probability F of the information from the previous moment. t Specifically, the forget gate decides which information needs to be forgotten and which needs to be retained, and stores this decision in the form of a probability value F t , with a value range between 0 and 1. 1 means complete retention, and 0 means complete forgetting. The forget gate F t is calculated as shown in Equation (1). (2) Input gate: First, X t and S t-1 determine the preservation probability (I t ) of the current input information through the sigmoid function. Then, use the tanh function to weight X t and S t-1 to form the candidate memory unit of the input data Then multiply the preservation probability by the corresponding elements of the candidate memory unit to obtain the actually preserved input information, and the final output is I t and are calculated as shown in Equation (2) and Equation (3). (3) Long-term information memory cell: The output F t of the forget gate is multiplied by the corresponding elements of the long-term memory information L t-1 from the previous time node to obtain the effective information of this long-term memory. At the same time, add the effective information of this input obtained from the input gate Finally, obtain the final memory information L of this LSFM t . At the same time, L t will be used as the input of the t+1 time node. The calculation formula of L t is shown in Equation (4). (4) Output gate: First, determine the output probability O t of this information through the sigmoid function. Then, L t transmitted from the long-term memory cell undergoes non-linear weighted conversion through the tanh function and is multiplied by the corresponding elements of O t to obtain the output of the output gate , that is, the short-term output result not affected by the result feedback of the t+1 time node. The calculation formulas of O t and are shown in Equation (5) and Equation (6). (5) Feedback gate: The short-term output result S t+1 at the t+1 moment is used as the input of the feedback gate at the t moment. S t+1 calculates the feedback influence probability B t through the sigmoid function. Then, B t is multiplied by Perform element-wise multiplication to obtain the final output S at this time node t , which is the short-term output result after feedback adjustment. B t and S t The calculation formulas are shown in Equations (7) and (8);

[0028] F t = σ(W f · [S t-1 , X t ) + b f ) (1)

[0029] I t = σ(W i ·[S t-1 , X t ) + b i ) (2)

[0030] L t = tanh(W l · [S t-1 , X t ) + b l ) (3)

[0031] L t = F t × S t-1 + I t × L t ) (4)

[0032] O t = σ(W o · [S t-1 , X t ) + b o ) (5)

[0033] S t = tanh(L t ) × O t (6)

[0034] B t = σ(W b · [S t+1 , L t ) + b b ) (7)

[0035] S t = L t × B t (8)

[0036] In the formulas, W f , W i , W l , Wo , W b and b f , b i , b l , b o , b b are the weights and bias terms of the forget gate, input gate, memory cell, output gate, and feedback gate. It should be noted that in the present invention, all "×" represents element-wise multiplication (Hadamard product);

[0037] Multi-mode fusion and first inflection point constraint layer: The output results of the LSFM layer are respectively input into the multi-mode fusion (MF) and first inflection point constraint (FIC) function layers for processing. In the multi-mode fusion layer (MF), by fusing and calculating the temporal land use changes of different modes, a refined land use type (Label) is obtained. In the first inflection point constraint layer (FIC), the time information (Time) of the first occurrence of each mode is tracked and recorded, and the inflection point of the first change is used as the constraint condition for each mode, that is, only when the inflection point at the time of change satisfies the constraint condition of the first change, the inflection point is considered. This means that subsequent change points of the same land use type will no longer be regarded as new inflection points until the conditions of the first change are met. The final output is the refined land use type of each pixel at each time node and the time point of its first change. The calculation formulas of the MF and FIC layers are shown in Equations (9) and (10);

[0038] MF = [W1, W2, W3, ···, W t-1 , W t mf × mf([S1, S2, S3, ···, S t-1 , S t T ) (9)

[0039] FIC = [W1, W2, W3, ···, W t-1 , W t fic × ft([S1, S2, S3, ···, S t-1 , S t T ) (10)

[0040] In the formula, [W1, W2, W3, ···, W t-1 , W t mf and [W1, W2, W3, ···, W t-1 , W t fic respectively represent the weights of the multi-mode fusion (MF) and first inflection point constraint (FIC) functions; ​​​​​​

[0041] Output layer: It contains the detailed land use types of each pixel and the time information of the transformation of each type, which are stored by Label and Time respectively;

[0042] Finally, the MFFIC-Net model is trained using the detailed land use type sample dataset of the mining area. The cross-entropy loss function is used to measure the difference between the land use type predicted by the model and the actual label. The calculation formula is shown in Equation (11). The accuracy, precision, and recall are used to evaluate the performance of the model in identifying the detailed land use types of the mining area, and the trained model is used to predict the land use types of other mining areas;

[0043]

[0044] where, y t,c is the true label of the t-th time step and the c-th class, and y t,c is the predicted probability.

[0045] Compared with the prior art, the advantages of the present invention are as follows:

[0046] (1) First, different from the limitations of existing methods that usually only identify two types of damaged areas and reclaimed areas in the mining area or only focus on one type of land use in the spatial range of the mining area, the present invention further expands and refines the land use classification in the mining area by constructing a land use type transfer matrix in the mining area. The land use in the mining area is divided into five categories: reclaimed area, damaged area, building occupied area, water body occupied area, and reclaimed degraded area, realizing more accurate and perfect monitoring of the land use in the mining area.

[0047] (2) Second, the multi-mode fusion and first inflection point constrained neural network model (MFFIC-Net) proposed by the present invention combines the long short-term memory mechanism and the feedback mechanism, and can effectively fuse various change patterns of land use types, and accurately track and record the time of the first change of land use types. The advantage of this model is that it can retain historical information and dynamically update the model state during the process of processing long time series data, ensuring the accurate identification and prediction of land use change patterns. In addition, the feedback mechanism enables the model to self-adjust the previous prediction results, thereby improving the prediction accuracy and the stability of the model.

[0048] (3) The multi-mode fusion mechanism of the present invention enables the model to adapt to different types of land use changes. Especially in the mining area where the changes are frequent and the land use types are complex, the model can make full use of the multiple information in the time series data, improving the adaptability and robustness of the model in complex environments.

[0049] (4) In addition to identifying the spatial distribution of land use types, the present invention can accurately track the time when each land use type first changes through a model. This function has important application value in the fields of ecological restoration assessment, land use planning, resource management, etc., and can provide important time references for policymakers to help formulate more timely and targeted environmental protection or reclamation measures.

[0050] (5) Through the time series analysis based on the Dynamic World dataset, the present invention can provide monitoring results of land use changes in mining areas with high spatio-temporal resolution (10m). Compared with existing methods, the present invention can provide more detailed spatial distribution information and has a higher time frequency (such as monthly, quarterly), enabling a more comprehensive tracking and accurate description of the dynamic process of land use changes. Brief Description of the Drawings

[0051] Figure 1 is the technical flow chart of the present invention;

[0052] Figure 2 is the DeepLabv3+ model architecture used by the present invention to delineate the spatial scope of open-pit mining areas;

[0053] Figure 3 is a schematic diagram of the time series change pattern of land use types in the reclamation area constructed by the present invention;

[0054] Figure 4 is a schematic diagram of the time series change pattern of land use types in the damaged area constructed by the present invention;

[0055] Figure 5 is a schematic diagram of the time series change pattern of land use types in the area occupied by buildings constructed by the present invention;

[0056] Figure 6 is a schematic diagram of the time series change pattern of land use types in the area occupied by water bodies constructed by the present invention;

[0057] Figure 7 is a schematic diagram of the time series change pattern of land use types in the reclamation and degradation area constructed by the present invention;

[0058] Figure 8 is the structural diagram of the MFFIC-Net model proposed by the present invention

[0059] Figure 9 is the structural diagram of the long short-term feedback memory network (LSFM) proposed by the present invention

[0060] Figure 10 is a schematic diagram for determining the land use change pattern of typical pixels and identifying fine land use types and time information of the present invention;

[0061] Figure 11The identification results of the fine land use types in the mining area from 2016 to 2024 in the embodiment;

[0062] Figure 12 The time information results of the reclamation area, damaged area, building occupied area, water body occupied area, and reclamation degradation area in the embodiment; Embodiment

[0063] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0064] The present invention provides a rapid automatic identification method for open-pit mine damaged areas, reclamation areas, and occupied areas based on a multi-modal fusion and first inflection point constrained neural network model. The technical solution flow adopted by the present invention is as Figure 1 shown. The method steps are as follows:

[0065] 1. Step A, download the satellite remote sensing images of the study area, train the DeepLabv3+ deep learning model, predict the annual distribution of the spatial range of the mining area during the study period through the trained model, and comprehensively analyze the data of all time nodes to delineate the maximum spatial range of the mining area during the study period; The method for delineating the spatial range of the open-pit mining area is divided into the following three steps:

[0066] First, obtain the Sentinel high-resolution remote sensing images of the target area from 2016 to 2024 based on the GEE cloud platform;

[0067] Combine the geographical information and mining area distribution data obtained from actual investigations, and identify and label the spatial range of the open-pit coal mining area in 2024 through visual interpretation to form sample data of the mining area spatial range;

[0068] Secondly, based on the sample data of the spatial range of the open-pit mining area in 2024, use DeepLabv3+ as the model architecture for segmenting the spatial range of the mining area and train it. The specific structure of the DeepLabv3+ model is as Figure 2 shown.

[0069] Finally, use the trained model to predict the remote sensing images year by year during the study period. By predicting the Sentinel images of each year from 2016 to 2023, extract the spatial range of the open-pit mining area at each time point. In order to obtain the maximum mining area spatial range during the study period, the present invention performs a union operation on the annual prediction results and the results in 2024 to obtain the maximum mining area spatial range covering all years;

[0070] 2. Step B: Obtain the land use change data within the maximum spatial range of the mining area. Based on the transfer relationships between land use types, construct the temporal change pattern of land use types in the mining area, and divide the change pattern into five categories: damaged area, reclaimed area, reclamation degradation area, building occupation area, and water body occupation area. The construction method of the temporal change pattern of land use types in the mining area is divided into the following three steps:

[0071] (1) Download and induction of Dynamic World data

[0072] Through the GEE (Google Earth Engine) cloud platform, select the DynamicWorld dataset for the study area from May to September each year, and use the mode calculation method to obtain the land use classification data in the study area from 2016 to 2024. And merge the five land use types of trees, grasslands, flooded vegetation, crops, shrubs, and bushes into the "vegetation" category (see Table 1).

[0073] Table 1 Land use types and their induction of the Dynamic World dataset

[0074]

[0075] (2) Construct the land use type transfer matrix

[0076] Based on the inductive ground object types, a transfer matrix between four land use elements of water body, vegetation, building, and bare land is constructed (see Table 2), and the target land use type after transformation is clearly defined. The reclaimed area, damaged area, building occupation area, water body occupation area, and reclamation degradation area (see Table 3), these five land use types constitute the target land use types of the present invention, that is, the fine land use types in the mining area.

[0077] The present invention defines the area where bare land, buildings, and water bodies are converted into vegetation as the reclaimed area; the area where vegetation, bare land, buildings, and water bodies are converted into bare land as the damaged area; the area where vegetation, bare land, and buildings are converted into buildings as the building occupation area; the area where vegetation, bare land, and water bodies are converted into water bodies as the water body occupation area. Among them, the conversion of vegetation to vegetation means that there is no change in this area and there is no corresponding mining behavior, so there is no need to classify it. Although there is no change in bare land, buildings, and water bodies, they usually have corresponding mining behaviors, so they are respectively defined as the damaged area, building occupation area, and water body occupation area. In addition, there is no conversion relationship between water bodies and buildings, so the transfer relationship between them is empty. The reclamation degradation area refers to the area where land degradation occurs again after reclamation.

[0078] Table 2 Land use type transfer matrix

[0079]

[0080] Table 3 Definition and Description of Fine Land Use Types in Mining Areas

[0081]

[0082] (3) Construct the Temporal Change Patterns of Land Use Types in Mining Areas

[0083] Based on the defined 5 types of target land use types, the present invention constructs corresponding 5 temporal change patterns of land use types in mining areas, namely, the temporal change patterns of land use types in the reclamation area, damaged area, building occupied area, water body occupied area, and reclamation degradation area, as Figures 3 to 7 shown.

[0084] The specific change patterns of the 5 types of target land use types are respectively

[0085] a. Temporal Change Pattern of Land Use Type in Reclamation Area

[0086] Land use changes in the reclamation area: from bare land, buildings, and water bodies to vegetation. Therefore, in this pattern, there will be 3 change trends (see Figure 3 ). Change trend 1: from bare land to vegetation. In a certain year, the land use type changes from bare land to vegetation. The present invention reassigns the land use type in the year after this change to the reclamation area and marks the year when bare land first changes to vegetation as the first reclamation year. Change trend 2: from buildings to vegetation. In a certain year, the land use type changes from buildings to vegetation. The present invention reassigns the land use type in the year after this change to the reclamation area and marks the year when buildings first change to vegetation as the first reclamation year. Change trend 3: from water bodies to vegetation. In a certain year, the land use type changes from water bodies to vegetation. The present invention reassigns the land use type in the year after this change to the reclamation area and marks the year when water bodies first change to vegetation as the first reclamation year.

[0087] b. Temporal Change Pattern of Land Use Type in Damaged Area

[0088] Land use changes in the damaged area: from bare land, buildings, water bodies, and vegetation to bare land. Therefore, in this pattern, there will be 4 change trends (see Figure 4)。Trend 1: Bare land changes to bare land. The land use time series pattern of no change from bare land to bare land is usually due to land damage caused by mining activities before 2016 (the available time of the Dynamic World dataset). Therefore, for the areas within the time interval that conforms to this change pattern, they are all assigned as damaged areas. Trend 2: Buildings change to bare land. In a certain year, the land use type changes from buildings to bare land. In the present invention, the land use type in the years after this change is reassigned as a damaged area, and the year when the buildings first change to bare land is marked as the first damaged year. Trend 3: Water bodies change to bare land. In a certain year, the land use type changes from water bodies to bare land. In the present invention, the land use type in the years after this change is reassigned as a damaged area, and the year when the water bodies first change to bare land is marked as the first damaged year. Trend 4: Vegetation changes to bare land. In a certain year, the land use type changes from vegetation to bare land. In the present invention, the land use type in the years after this change is reassigned as a damaged area, and the year when the vegetation first changes to bare land is marked as the first damaged year.

[0089] c. Temporal change pattern of land use type in the area occupied by buildings

[0090] Land use changes in the area occupied by buildings: from bare land, buildings, and vegetation to buildings. Therefore, under this pattern, there are 3 trends (see Figure 5 ). Trend 1: Bare land changes to buildings. In a certain year, the land use type changes from bare land to buildings. In the present invention, the land use type in the years after this change is reassigned as the area occupied by buildings, and the year when the bare land first changes to buildings is marked as the first year of building occupation. Trend 2: Buildings change to buildings. The land use time series pattern of no change from buildings to buildings is usually due to the construction of surface facilities required for the mine production system and auxiliary production system before 2016 (the available time of the Dynamic World dataset), resulting in building occupation. Therefore, for the areas within the time interval that conforms to this change pattern, they are all assigned as the area occupied by buildings. Trend 3: Vegetation changes to buildings. In a certain year, the land use type changes from vegetation to buildings. In the present invention, the land use type in the years after this change is reassigned as the area occupied by buildings, and the year when the vegetation first changes to buildings is marked as the first year of building occupation.

[0091] d. Temporal change pattern of land use type in the area occupied by water bodies

[0092] Land use changes in the area occupied by water bodies: from bare land, water bodies, and vegetation to buildings. Therefore, under this pattern, there are 3 trends (see Figure 6)。Trend 1: Bare land is converted into water body. In a certain year, the land use type changes from bare land to water body. The present invention reassigns the land use type in the year after this change to the water body occupation area, and marks the year when the bare land is first converted into water body as the first water body occupation year. Trend 2: Change from water body to water body. The land use time series pattern with no change from water body to water body is usually caused by the construction of facilities such as groundwater recharge, wastewater sedimentation ponds, and irrigation reservoirs due to the damage of the groundwater layer during the open-pit mining process before 2016 (the available time of the Dynamic World dataset). Therefore, for the areas within the time interval that conforms to this change pattern, they are all assigned as water body occupation areas. Trend 3: Change from vegetation to water body. In a certain year, the land use type changes from vegetation to water body. The present invention reassigns the land use type in the year after this change to the water body occupation area, and marks the year when the vegetation is first converted into water body as the first water body occupation year.

[0093] e. Temporal change pattern of land use type in the reclaimed degraded area

[0094] Land use changes in the reclaimed degraded area: from the reclaimed area that has undergone reclamation to bare land, buildings, and water bodies. Therefore, in this pattern, there are 3 trends (see Figure 7 ). Trend 1: The reclaimed area is converted into bare land. In a certain year, the land use type changes from the reclaimed area to bare land. The present invention reassigns the land use type in the year after this change to the reclaimed degraded area, and marks the year when the reclaimed area is first converted into bare land as the first reclaimed degradation year. Trend 2: Change from the reclaimed area to buildings. In a certain year, the land use type changes from the reclaimed area to buildings. The present invention reassigns the land use type in the year after this change to the building occupation area. Trend 3: Change from the reclaimed area to water body. In a certain year, the land use type changes from the reclaimed area to water body. The present invention reassigns the land use type in the year after this change to the water body occupation area. It should be noted that: the change from the reclaimed area to buildings and water bodies does not mean land degradation. Therefore, for the areas that conform to this change pattern in the present invention, they are assigned as the building occupation area and the water body occupation area, rather than the reclaimed degraded area.

[0095] 3. Step C: Based on the constructed temporal change pattern of the land use types in the mining area, prepare a sample dataset of the fine land use types in the mining area, and establish a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net, Multi-Mode Fusion and First Inflection Constrain Neural Network) for identifying the fine land use types in the mining area. Use the sample dataset to train the model, and apply the trained MFFIC-Net to the identification of the annual fine land use types in the mining area, while accurately tracking the information of the first change time of each type. The method for identifying the fine land use types in the mining area is divided into the following three steps:

[0096] (1) Sample dataset of the fine land use types in the mining area

[0097] Based on the Dynamic World dataset, the present invention constructs a sample dataset including the original Dynamic World time series data, as well as the target fine land use types and the first change time. By combining the ArcGIS tool, field survey data and visual interpretation, the land use type of each pixel is labeled, and the first change time of each pixel at different time nodes is determined. This dataset provides sample data for the subsequent training of the MFFIC-Net model.

[0098] (2) Construction of the MFFIC-Net model

[0099] The present invention designs and constructs the MFFIC-Net model. This model is based on the multi-mode fusion and first inflection point constraint mechanism, adopts the long short-term memory mechanism to process time series data, and introduces a feedback mechanism for self-adjustment and optimization. The MFFIC-Net model consists of an input layer, a long short-term feedback memory network layer (Long-Short Feedback Memory Network, LSFM), a multi-mode fusion and first inflection point constraint layer, and an output layer. The structure is as Figure 8 shown. Among them, the internal structure of the LSFM network layer is as Figure 9 shown.

[0100] Input layer: It consists of the extracted long time series land use change sequences of pixels. The annual land use types (X1, X2, ……, X t , where t represents the year sequence) of each pixel are used as the input data of the model and are sequentially input into the neural network for processing.

[0101] Long-Short-Term Feedback Memory (LSFM) layer: used to determine the fine land use type of pixels. Based on the input data of adjacent time steps, LSFM judges the land use type change pattern at each time node, and then determines the target fine land use type at each time node. The output results include the short-term result (S t ) of this node and the long-term memory result (L t ) of the memory cell. When judging the change of the target fine land use type, the LSFM model not only needs to remember the information of the previous time node, but also the output result of the current time node will affect the result of the previous time node. Therefore, the LSFM network layer combines the long-term and short-term memory mechanisms and the feedback mechanism to effectively handle the long-term and short-term dependencies and feedback relationships.

[0102] The long-term and short-term memory mechanisms and the feedback mechanism of the LSFM network layer are realized by the gating mechanism. LSFM consists of a forget gate (F t ), an input gate (I t ), an output gate (O t ), and a feedback gate (B t ). At the same time, the information of the long-term memory is stored and updated by the memory cell (L t ). The input of LSFM: L t-1 , S t-1 , X t ; The output of LSFM: L t (long-term memory information), S t (short-term output result). The operation mechanism of LSFM: (1) Forget gate: At time t, the input data X t is first input through the input layer and combined with the short-term result S t-1 of the previous time node, and then passed to the forget gate. The forget gate controls the retention degree of the information passed from the previous moment through the sigmoid (σ) activation function. The output of the forget gate is the retention probability F t of the information at the previous moment. Specifically, the forget gate decides which information needs to be forgotten and which needs to be retained, and stores this decision in the form of a probability value F t , with a value range between 0 and 1. 1 means complete retention, and 0 means complete forgetting. The calculation formula of the forget gate F t is shown in Equation (1). (2) Input gate: First, X t and S t-1 determine the retention probability (I t ) of the current input information through the sigmoid function. Then, the tanh function is used to weight X t and S t-1 to form the candidate memory unit of the input data Then, the preservation probability is multiplied by the corresponding elements of the candidate memory unit to obtain the actually preserved input information, and the final output is I t and The calculation formulas of are shown in Equations (2) and (3). (3) Long-term information memory cell: The output F of the forget gate t is multiplied by the corresponding elements of the long-term memory information L at the previous time node t-1 to obtain the effective information of this long-term memory. At the same time, the effective information of this input obtained by the input gate is added Finally, the final memory information L of this LSFM is obtained t . At the same time, L t will be used as the input at the time node of t + 1, and the calculation formula of L t is shown in Equation (4). (4) Output gate: First, the output probability O of this information is determined through the sigmoid function t . Then, L passed from the long-term memory cell t is non-linearly weighted and transformed through the tanh function and multiplied by the corresponding elements of O t to obtain the output of the output gate That is, the short-term output result not affected by the result feedback at the time node of t + 1. The calculation formulas of O t and are shown in Equations (5) and (6). (5) Feedback gate: The short-term output result S at the time of t + 1 t+1 is used as the input of the feedback gate at time t. S t+1 is used to calculate the feedback influence probability B through the sigmoid function t . Then, B t is multiplied by the corresponding elements of to obtain the final output S of this time node t , that is, the short-term output result after feedback adjustment. The calculation formulas of B t and S t are shown in Equations (7) and (8).

[0103] F t = σ(W f · [S t-1 , X t ) + b f ) (1)

[0104] I t = σ(W i · [S t-1 , X t ) + b i ) (2)

[0105] L t= tanh(W l · [S t-1 ,X t ) + b l ) (3)

[0106] L t = F t × S t-1 + I t × L t ) (4)

[0107] O t = σ(W o · [S t-1 ,X t ) + b o ) (5)

[0108] S t = tanh(L t ) × O t (6)

[0109] B t = σ(W b · [S t+1 ,L t ) + b b ) (7)

[0110] S t = L t × B t (8)

[0111] In the formula, W f ,W i ,W l ,W o ,W b and b f ,b i ,b l ,b o ,b b are the weight and bias terms of the forget gate, input gate, memory cell, output gate, and feedback gate. It should be noted that in the present invention, all "×" represents element-wise multiplication (Hadamard product).

[0112] Multi - mode Fusion and First Inflection Point Constraint Layer: The output results of the LSFM layer are respectively input into the multi - mode fusion (MF) and first inflection point constraint (FIC) function layers for processing. In the multi - mode fusion layer (MF), by fusing and calculating the temporal land - use changes of different modes, a fine land - use type (Label) is obtained. In the first inflection point constraint layer (FIC), the time information (Time) of the first occurrence of each mode is tracked and recorded, and the inflection point of the first change is used as the constraint condition for each mode. That is, only when the inflection point at the time of change meets the constraint condition of the first change, this inflection point is considered. This means that subsequent change points of the same land - use type will no longer be regarded as new inflection points until the conditions for the first change are met. The final output is the fine land - use type of each pixel at each time node and the time point of its first change. The calculation formulas of the MF and FIC layers are shown in Equations (9) and (10).

[0113] MF = [W1,W2,W3,···,W t-1 ,W t mf ×mf([S1,S2,S3,···,S t-1 ,S t T ) (9)

[0114] FIC = [W1,W2,W3,···,W t-1 ,W t fic ×ft([S1,S2,S3,···,S t-1 ,S t T ) (10)

[0115] In the formulas, [W1,W2,W3,···,W t-1 ,W t mf and [W1,W2,W3,···,W t-1 ,W t fic respectively represent the weights of the multi - mode fusion (MF) and first inflection point constraint (FIC) functions.

[0116] Output Layer: It contains the fine land - use type of the mining area for each pixel and the time information of the transformation of each type, which are stored by Label and Time respectively;

[0117] ​​​​​​In the Python environment, the present invention develops an algorithm for identifying fine land use types in mining areas based on the MFFIC-Net model. This model combines the change patterns of land use types through a multi-modal fusion mechanism and simultaneously records the information on the first change time of land use types. To visually demonstrate the pattern determination of the land use change sequence and the corresponding identification process of fine land use types and time information in mining areas, a certain pixel is selected as an example for illustration. The specific process is as Figure 10 shown.

[0118] (3) Training and application of the MFFIC-Net model

[0119] In the model training stage, the present invention inputs the labeled time series data and the target fine land use type sample data set into the MFFIC-Net model for training. To ensure the generalization ability of the model, 80% of the data is used for training and 20% of the data is used for validation. During the training process, the cross-entropy loss function is used to measure the difference between the model prediction result and the actual label, and the Adam optimizer is used to optimize the model. After the training is completed, the validation set is used to evaluate the model. The main evaluation indicators include: Accuracy, which measures the overall classification ability of the model for land use types in mining areas; Precision, which evaluates the prediction accuracy of the model when identifying specific land use types (such as "damaged areas" or "reclaimed areas"); Recall, which measures the proportion that the model can correctly identify among all areas that actually belong to a certain land use type.

[0120] After the model training is completed and evaluated, the trained MFFIC-Net model is used to predict the new Dynamic World time series data to obtain the land use type of each pixel in the mining area and its corresponding first change time. Output the fine land use type distribution map and mark the first change time of each land use type. For example, the identification results of the fine land use types in the mining area from 2016 to 2024 are as Figure 11 shown, and the time information of the reclaimed area, damaged area, building occupied area, water body occupied area, and reclamation degradation area is as Figure 12 shown.

[0121] The above has introduced in detail a fine land identification method for mining areas based on multi-mode fusion and first inflection point constraint. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. It is possible to make changes and improvements to the present invention without exceeding the concept and scope defined by the appended claims. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A fine land recognition method for mining areas based on multi-mode fusion and first inflection point constraint, characterized in that The method steps are as follows: Step A: Obtain the satellite remote sensing images of the study area, construct a sample data set of the spatial scope of the open-pit mining area, train the DeepLabv3+ deep learning model, predict the annual distribution of the mining area's spatial scope during the study period through the trained model, and comprehensively analyze the data of all time nodes to delineate the maximum spatial scope of the mining area during the study period; Step B: Obtain the land use change data within the maximum spatial scope of the mining area, construct a temporal change pattern of the land use types in the mining area based on the transfer relationship between land use types, and divide the change pattern into 5 categories: damaged area, reclaimed area, reclaimed degraded area, building occupied area, and water body occupied area; Step C: Based on the constructed temporal change pattern of the land use types in the mining area, prepare a sample data set of the fine land use types in the mining area, and establish a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net, Multi-Mode Fusionand First Inflection Constrain Neural Network). Use the sample data set to train the model, and use the trained MFFIC-Net for the identification of the annual fine land use types in the mining area, and at the same time accurately track the first change time information of each type.

2. The fine land recognition method for mining areas based on multi-mode fusion and first inflection point constraint according to claim 1, wherein: In Step B, first, based on the vector of the mining area's spatial scope obtained in Step A, obtain the land use data set of each year in the study area, use the mode calculation method to obtain the annual land use classification data within the study area, and classify its land use types into four categories: vegetation, bare land, water body, and building; Secondly, construct a transfer matrix between the inducted land use types (water body, vegetation, building, bare land); finally, define the transformed land use types as 5 types of fine land use types in the mining area, namely reclaimed area, damaged area, building occupied area, water body occupied area, and reclaimed degraded area, so as to construct a temporal change pattern of the 5 types of fine land use types in the mining area.

3. The method for fine land identification in mining areas based on multi-modal fusion and first inflection point constraint according to claim 1, wherein: In Step C, the preparation of the sample data set of the fine land use types in the mining area: Based on the ArcGIS tool, combine the temporal change pattern of the 5 types of fine land use types in the mining area with the land data and field survey data, and determine the fine land use types of the open-pit coal mine and the time of the first change through visual interpretation, so as to form a sample data set of the fine land use types in the mining area that corresponds one by one to the original land use time series data.

4. The fine land recognition method for mining areas based on multi-modal fusion and first inflection point constraint according to claim 1, characterized in that: In Step C, establish a multi-mode fusion and first inflection point constraint neural network model (MFFIC-Net) for the identification of the fine land use types in the mining area. The MFFIC-Net model consists of an input layer, a long-short feedback memory network layer (Long-ShortFeedback Memory Network, LSFM), a multi-mode fusion and first inflection point constraint layer, and an output layer.

5. The method for fine land recognition in mining areas based on multi-modal fusion and first inflection point constraint according to claim 4, wherein: The long short-term feedback memory network (LSFM) layer is used to determine the fine land use type of pixels. Based on the input data of adjacent time steps, the LSFM determines the land use type change pattern at each time node, and then determines the target fine land use type at each time node. The output results include the short-term result (S t ) and the long-term memory result (L t ) of the memory cells. When judging the change of the target fine land use type, the LSFM model not only needs to remember the information of the previous time node, but also the output result of the current time node will affect the result of the previous time node. The LSFM network layer combines the long short-term memory mechanism and the feedback mechanism, and can effectively process the long short-term dependence relationship and the feedback relationship.

6. The fine land identification method for mining areas based on multi-modal fusion and first inflection point constraint according to claim 4, wherein: The LSFM network layer consists of a forget gate (F t ), an input gate (I t ), an output gate (O t ), and a feedback gate (B t ). Meanwhile, the information of long-term memory is stored and updated by memory cells (L t ). The operating mechanism of LSFM: (1) Forget gate: At time t, the input data X t is first input through the input layer and combined with the short-term result S t-1 from the previous time node, and then passed to the forget gate. The forget gate controls the retention degree of the information passed from the previous moment through the sigmoid (σ) activation function. The output of the forget gate is the retention probability F t of the information from the previous moment. t The calculation formula of the forget gate F (2) Input Gate: First, X t and S t-1 determine the preservation probability (I t ) of the current input information through the sigmoid function. Then, use the tanh function to perform weighted processing on X t and S t-1 to form the candidate memory unit of the input data Then, multiply the preservation probability by the corresponding elements of the candidate memory unit to obtain the actually preserved input information, and the final output is I t and The calculation formulas are shown in Equations (2) and (3). (3) Long-term information memory cells: Output F of the forgetting gate t Perform element-wise multiplication with the long-term memory information L at the previous time node t-1 to obtain the effective information of this long-term memory. At the same time, add the effective information of this input obtained by the input gate Finally, obtain the final memory information L of this LSFM t . At the same time, L t will be used as the input at the time node t + 1. The calculation formula of L t is shown in Equation (4). (4) Output gate: First, determine the output probability O of the current information through the sigmoid function. t . Then, L passed from the long-term memory cell t undergoes non-linear weighted conversion through the tanh function and is multiplied by the corresponding elements of O t to obtain the output of the output gate , that is, the short-term output result not affected by the result feedback at the t+1 time node. O t and are calculated as shown in Equations (5) and (6). (5) Feedback gate: The short-term output result S at time t+1 t+1 As the input of the feedback gate at time t, S t+1 Passes through the sigmoid function to calculate the feedback influence probability B t , and then B t Is multiplied element-wise with To obtain the final output S of this time node t , that is, the short-term output result after feedback adjustment. B t And S t The calculation formulas of are shown in Equations (7) and (8); F t = σ(W f · [S t-1 , X t ) + b f ) (1) I t = σ(W i · [S t-1 , X t ) + b i ) (2) L t = tanh(W l ·[S t-1 , X t ) + b l ) (3) L t = F t × S t-1 + I t × L t ) (4) O t = σ(W o · [S t-1 , X t ) + b o ) (5) S t =tanh(L t )×O t (6) B t = σ(W b · [S t+1 , L t ) + b b ) (7) S t = L t × B t (8) Where, W f , W i , W l , W o , W b and b f , b i , b l , b o , b b are the weights and bias terms of the forget gate, input gate, memory cell, output gate and feedback gate. "×" represents element-wise multiplication (Hadamard product).

7. The fine land identification method for mining areas based on multi-mode fusion and first inflection point constraint according to claim 4, wherein: The multi-modal fusion and first inflection point constraint layer inputs the output results of the LSFM layer into the multi-modal fusion (MF) and first inflection point constraint (FIC) function layers for processing. In the multi-modal fusion layer (MF), by fusing and calculating the temporal land use changes of different modes, a fine land use type (Label) is obtained. In the first inflection point constraint layer (FIC), the time information (Time) of the first occurrence of each mode is tracked and recorded, and the inflection point of the first change is used as the constraint condition for each mode. That is, only when the inflection point at the time of change satisfies the constraint condition of the first change, the inflection point is considered. This means that subsequent change points of the same land use type will no longer be regarded as new inflection points until the conditions of the first change are met. The final output is the fine land use type of each pixel at each time node and the time point of its first change. The calculation formulas of the MF and FIC layers are shown in Equations (9) and (10); MF = [W1, W2, W3, ···, W t-1 , W t mf × mf([S1, S2, S3, ···, S t-1 , S t T ) (9)​​ FIC = [W1, W2, W3, ···, W t-1 , W t fic × ft([S1, S2, S3, ···, S t-1 , S t T ) (10)​​ where, [W1, W2, W3, ···, W t-1 , W t mf and [W1, W2, W3, ···, W t-1 , W t fic represent the weights of the multi-modal fusion (MF) and the first inflection point constraint (FIC) functions, respectively.​​ 8. The fine land recognition method for mining areas based on multi-modal fusion and first inflection point constraint according to claim 4, characterized in that: After completing the construction of the multi-modal fusion and first inflection point constraint neural network model (MFFIC-Net), the fine land use type sample dataset of the mining area prepared in step C is input into the MFFIC-Net for training, and the cross-entropy loss function is used to measure the difference between the land use type predicted by the model and the actual label. The calculation formula is shown in Equation (11). The accuracy (Accuracy) is used to measure the overall classification ability of the model for the land use type of the mining area; the precision (Precision) evaluates the prediction accuracy of the model when identifying a specific land use type (such as "damaged area" or "reclaimed area"); the recall (Recall) measures the proportion that can be correctly identified in all areas that actually belong to a certain land use type; where y t,c is the true label for the t-th time step and the c-th class, and y t,c is the predicted probability.

9. The method for fine land identification in mining areas based on multi-modal fusion and first inflection point constraint according to claim 8, wherein: After completing the training of the multi-modal fusion and first inflection point constraint neural network model (MFFIC-Net), the model is applied to the identification of the fine land use type and the identification of the time of the first type transition at other research times.

Images