A landslide frequent secondary factor combination weighted mining method based on big data technology optimization

Abstract

A landslide frequent secondary factor combination weighted mining method based on big data technology optimization is provided in the application, specifically comprising: obtaining landslide disaster factor data and historical landslide disaster data of a target area, coupling each disaster factor with historical landslide disaster data to obtain a disaster factor and historical landslide correlation layer; taking the number of historical landslides, historical landslide area and historical landslide density as the basis for mining, respectively, using an optimized frequent pattern tree algorithm to mine landslide frequent secondary factor combinations, and obtaining strong correlation rules between landslide frequent secondary factor combinations and landslide disaster layers; according to the mining results of the optimized frequent pattern tree algorithm, superimposed analysis of the secondary factor combinations and the landslide disaster layer is performed to construct a combination weighting model, and the landslide frequency of the landslide frequent secondary factor combinations is evaluated; and the landslide frequent secondary factor combinations are verified by using the chi-square test. The application is aimed at the limitation that the association rule algorithm can only use the number of item sets as the basis for mining, and the optimized algorithm makes it applicable to research based on continuous variables as the basis for mining, and the influence mechanism of multiple disaster factors on the comprehensive action of landslides is explored by constructing a combination weighting model, thereby providing a basis for landslide disaster prevention.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of big data intelligent processing, in particular to a landslide frequent secondary factor combination weighted mining method based on big data optimization. BACKGROUND

[0002] At present, with the development of science and technology, the analysis and prediction technology of geological disasters gradually matures, but the research technology of the action mechanism of geological disasters is not mature enough. Among them, landslide disasters cause great damage to human life, property and production and life. In the current related research, the main research method of the action mechanism of landslide disasters is to use parameters such as landslide density or certainty factor as action indicators to evaluate the influence of a single disaster-causing factor on landslide disasters, and few studies have explored the influence mechanism of multiple factors on landslide disasters. At present, the research on factor combination mostly only considers the influence of fixed disaster-causing factor combination, rather than exploring the factor combination with the most significant influence degree by analyzing various factor combinations, so there is a lack of research on the influence mechanism of the comprehensive action of multiple disaster-causing factors of landslide.

[0003] Therefore, there is an urgent need for a landslide frequent secondary factor combination weighted mining method based on big data technology optimization to deeply explore the action mechanism of disaster-causing factors. SUMMARY

[0004] To solve the above-mentioned defects in the prior art, the purpose of the present application is to provide a landslide frequent secondary factor combination weighted mining method based on big data technology optimization, which aims to partially make up for the lack of in-depth exploration of the comprehensive action mechanism of landslide disaster-causing factors in the prior art.

[0005] According to a first aspect of the present application, a landslide frequent secondary factor combination weighted mining method based on big data technology optimization is provided, comprising:

[0006] Step 1: Obtain landslide disaster-causing factor data and historical landslide disaster data of a target area, couple each disaster-causing factor with historical landslide disaster data, and obtain a disaster-causing factor and historical landslide correlation layer.

[0007] Step 2: Take the number of historical landslides, the area of historical landslides and the density of historical landslides as the basis for mining, respectively, and use the optimized frequent pattern tree algorithm to mine landslide frequent secondary factor combinations, and obtain strong correlation rules between landslide frequent secondary factor combinations and landslide disaster layers.

[0008] Step 3: According to the mining results of the optimized frequent pattern tree algorithm, superimpose and analyze the secondary factor combination and the landslide disaster layer, construct a combination weighting model, and evaluate the landslide frequency of the landslide frequent secondary factor combination.

[0009] Step 4: Use the chi-square test to verify the combination of frequent secondary factors of landslides, and compare the evaluation results of the combined weighted model with the strong correlation rule of landslide disaster.

[0010] Furthermore, in the landslide frequent secondary factor combination weighted mining method optimized based on big data technology provided by the present invention, step 2 includes:

[0011] Step 2.1: Frequent pattern mining. Select the frequent parts from a series of candidate itemsets, that is, measure the frequency of the itemsets and find the frequent datasets whose support is greater than the set minimum support.

[0012] Step 2.2: Generate association rules. In the frequently discovered itemsets, determine strong association rules based on the frequent itemsets and the minimum confidence level.

[0013] Furthermore, the landslide frequent secondary factor combination weighted mining method optimized based on big data technology provided by this invention, wherein the strong association rule is an association rule that simultaneously satisfies the minimum support and minimum confidence of the itemset support; step 2 further includes:

[0014] In algorithms that use the number of historical landslides as the basis for data mining, the expression for support is Support(A, B) = P(A&B), and the expression for confidence is... Where P(A&B) is the probability that secondary factors A and B appear simultaneously; P(B|A) is the probability that B appears given that secondary factor A has appeared; and P(A) is the probability that secondary factor A appears.

[0015] In algorithms that use historical landslide area as the basis for data extraction, the expression for the support is: The expression for confidence level is: Where Area(A&B) is the historical landslide area where secondary factors A and B coexist; ∑Area is the total historical landslide area; Area(A) is the historical landslide area where secondary factor A is located.

[0016] In algorithms that use historical landslide density as the basis for data mining, the expression for the support is: The expression for confidence level is: Wherein, LD(A&B) is the historical landslide density where secondary factors A and B coexist; ∑LD is the total historical landslide density of the target area; and LD(A) is the historical landslide density where secondary factor A is located.

[0017] Furthermore, the landslide frequent secondary factor combination weighted mining method based on big data technology optimization provided by this invention includes the following optimized frequent pattern tree algorithm:

[0018] Scan the database itemset, set a minimum support, remove items with less than the minimum support, and sort the original dataset items in descending order of frequency.

[0019] Scan the itemset again and create an item header table and a frequent pattern tree. The root node of the tree is empty, and each item in the transaction is a child node. Branches are formed from the root node to the child nodes in descending order of support. Frequent itemsets are gradually added to the tree structure.

[0020] Find the pattern base for each condition from bottom to top, recursively call the tree structure, remove items with less than the minimum support, calculate the confidence of frequent itemsets, and find strong association rules that meet the conditions.

[0021] Furthermore, in the landslide frequent secondary factor combination weighted mining method optimized based on big data technology provided by the present invention, step 3 includes:

[0022] The actual landslide area of ​​the statistical second-level factor combination in For mining method j, the second-order factor combination F is mined. i The landslide area in the region;

[0023] Calculate the theoretical landslide area based on the combination of second-order factors. Among them, A L The historical landslide area of ​​the target region; For the second-order factor combination F i Area of ​​the region; A is the area of ​​the target region.

[0024] The combined precision weight W = E·P, where E = (1, 1, ..., 1) 1×n P is the combined precision matrix, P = (p ij ) n×m ,in, n represents the number of second-level factor combinations mined, and m represents the number of mining methods.

[0025] Evaluation factor combination F i frequency of landslides Among them, w i Let W be an element of matrix W, i.e., W = (w i ) 1×m c i =(c ij ) m×1 c ij For mining the factor combination F in method j i The confidence level.

[0026] Furthermore, in the landslide frequent secondary factor combination weighted mining method optimized based on big data technology provided by the present invention, step 4 includes:

[0027] The expression for the chi-square test is: Among them, X i χ² is the actual observed value, T is the theoretical predicted value, and χ² is the theoretical value. 2 This represents the chi-square test value for the combination of secondary factors and landslide disaster.

[0028] According to a second aspect of the present invention, a computer device is provided, characterized in that it comprises:

[0029] A memory for storing instructions; and a processor for invoking the instructions stored in the memory to execute the method of the first aspect.

[0030] According to a third aspect of the present invention, a computer-readable storage medium is provided, characterized in that it stores instructions which, when executed by a processor, perform the method of the first aspect.

[0031] Compared with the prior art, the above-mentioned technical solution conceived by the present invention has at least the following beneficial effects: Through the solution of this application, the original association rule algorithm is limited to mining based only on the number of itemsets. Based on historical landslide data, the association rule mining algorithm is optimized to make it applicable to research based on historical landslide data (continuous variables). By constructing a combined weighted model, the influence mechanism of multiple disaster-causing factors on the comprehensive effect of landslides is explored, providing a basis for landslide disaster prevention and control.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0034] Figure 1 A flowchart illustrating a weighted mining method for frequent landslide secondary factors based on big data technology, provided for this application;

[0035] Figure 2 This application provides a distribution map of landslide hazard-causing factors;

[0036] Figure 3 This application provides a map showing the ratio of secondary factors to landslide disaster area.

[0037] Figure 4 This application provides a structure diagram of a frequent pattern tree algorithm. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0039] See Figure 1 This is a flowchart illustrating a weighted mining method for frequent landslide secondary factors based on big data technology, as provided in this application. Figure 1 As shown, the method mainly includes the following steps:

[0040] Step 1: Obtain landslide disaster causative factor data and historical landslide disaster data for the target area, and couple each causative factor with the historical landslide disaster data to obtain a causative factor-historical landslide association layer.

[0041] For example, when the target area is Liangshan Yi Autonomous Prefecture, eight influencing factors can be obtained, including lithology, elevation, slope, surface relief, topography, topographic humidity index, average annual rainfall, and distance from geological structure lines, as well as historical landslide disaster data for Liangshan Yi Autonomous Prefecture. Among the eight disaster-causing factors, there are two with fixed categories: lithology and topography, and the distance from geological structure lines, which has a limited influence range on the study area. The remaining five disaster-causing factors can be divided into secondary factor intervals based on the natural discontinuity method, and their distribution map is shown below. Figure 2 As shown in the figure. The correlation between the secondary factors of the eight disaster-causing factors and landslide disasters is obtained by coupling each of the secondary factor zoning maps with the historical landslide disaster data, as shown in the area percentage map. Figure 3 As shown.

[0042] Encode the secondary factor attributes of each research unit, and use the combination of the codes to characterize the combination of secondary factor attributes of the research unit;

[0043] For example, "random landslide" is coded as "12"; "elevation 2393-2990m" is coded as "23"; and "slope 23.98-32.09°" is coded as "34". Therefore, the code combination "12, 23, 34" represents the secondary factor combination "random landslide, elevation 2393-2990m, slope 23.98-32.09°". The secondary factor combination attribute yp of each research unit can be represented using its corresponding code combination. The secondary factor coding table for disaster-causing factors is shown below.

[0044] Table 1. Secondary Factor Coding Table for Disaster-Causing Factors

[0045]

[0046]

[0047] Step 2: Using the number of historical landslides, the area of ​​historical landslides, and the density of historical landslides as the basis for data mining, the optimized frequent pattern tree algorithm is used to mine the frequent secondary factor combinations of landslides, and the strong correlation rules between the frequent secondary factor combinations of landslides and the landslide disaster layer are obtained.

[0048] Based on the above embodiments, step 2, which uses the number of historical landslides, the area of ​​historical landslides, and the density of historical landslides as the mining criteria, and employs the optimized frequent pattern tree algorithm to mine combinations of frequent secondary factors of landslides, includes:

[0049] Frequent pattern mining selects frequent itemsets from a series of candidate itemsets, measures the frequency of the itemsets, and finds frequent datasets whose support is greater than the set minimum support.

[0050] Generate association rules. In the frequently discovered itemsets, determine strong association rules based on the frequent itemsets and the minimum confidence level.

[0051] Furthermore, the strong association rule is an association rule that simultaneously satisfies the minimum support and minimum confidence of the itemset support and confidence.

[0052] In algorithms that use the number of historical landslides as the basis for data mining, the expression for support is Support(A, B) = P(A&B), and the expression for confidence is... Where P(A&B) is the probability that secondary factors A and B occur simultaneously; P(B|A) is the probability that B occurs given the occurrence of secondary factor A; and P(A) is the probability that secondary factor A occurs.

[0053] In algorithms that use historical landslide area as the basis for data extraction, the expression for the support is: The expression for confidence level is: Where Area(A&B) is the historical landslide area where secondary factors A and B coexist; ∑Area is the total historical landslide area; Area(A) is the historical landslide area where secondary factor A is located;

[0054] In algorithms that use historical landslide density as the basis for data mining, the expression for the support is: The expression for confidence level is: Wherein, LD(A&B) is the historical landslide density where secondary factors A and B coexist; ∑LD is the total historical landslide density of the target area; and LD(A) is the historical landslide density where secondary factor A is located.

[0055] Meanwhile, the construction process of the frequent pattern tree algorithm is as follows: Figure 4As shown, the specific process is as follows: 1) Scan the database itemset, set the minimum support, remove items with less than the minimum support, and sort the original dataset items in descending order of frequency; 2) Scan the itemset again and create an item header table and a frequent pattern tree. The root node of the tree is empty, and each transaction item is a child node. Branches are formed from the root node to the child nodes in descending order of support, and frequent itemsets are gradually added to the tree structure; 3) Find the conditional pattern base from bottom to top, recursively call the tree structure, remove items with less than the minimum support, calculate the confidence of frequent itemsets, and find strong association rules that meet the conditions.

[0056] The frequent pattern tree algorithm was optimized based on the Python platform. It was used to mine frequent secondary factor combinations of landslides based on the number of historical landslides, the area of ​​historical landslides, and the density of historical landslides, and to obtain strong correlation rules between each secondary factor combination and landslide disasters.

[0057] Step 3: Based on the results of the optimized frequent pattern tree algorithm, overlay the analysis of the secondary factor combination and the landslide hazard layer to construct a combined weighted model and evaluate the landslide frequency of the secondary factor combination.

[0058] The construction process of the combined weighted model is as follows: 1) Statistical analysis of the actual landslide area of ​​the combination of secondary factors. in, For mining method j, the second-order factor combination F is mined. i 1) Calculate the landslide area of ​​the area; 2) Calculate the theoretical landslide area of ​​the combination of secondary factors. Among them, A L The historical landslide area of ​​the target region; For the second-order factor combination F i Area of ​​the region; A is the total area of ​​the target region; 3) Combined precision weight W = E·P, where E = (1, 1, ..., 1) 1×n P is the combined precision matrix, P = ( pij ) n×m ,in, n is the number of secondary factor combinations mined, and m is the number of mining methods; 4) Evaluate factor combination F i frequency of landslides Among them, w i Let W be an element of matrix W, i.e., W = (w i ) 1×m c i =(c ij ) m×1 c ij For mining the factor combination F in method j i The confidence level.

[0059] In practice, a combined weighted model was constructed for the secondary factor combinations based on the three mining methods to obtain the landslide frequency of each factor combination. The actual landslide area of ​​each factor combination was statistically analyzed on the GIS platform. The landslide frequency of each factor combination was evaluated by combining the total area of ​​the target area, the landslide area, and the area of ​​the factor combination's region, and then ranked.

[0060] The frequency of landslides for each combination of secondary factors was thus obtained, and after normalization, they were arranged from largest to smallest as shown in the table below.

[0061] Table 2. Normalized values ​​of landslide frequency from the mined secondary factor combinations.

[0062]

[0063] The landslide frequency of each secondary factor combination was assessed and ranked based on the combined weighted model. The secondary factor combination with the landslide frequency normalized value greater than 60% was determined. The secondary factor combinations (21, 54), (21, 42), (22, 54) and (12, 54) were obtained as the secondary factor combinations with the landslide frequency of the target area.

[0064] Step 4: Use the chi-square test to verify the combination of frequent secondary factors of landslides, and compare the evaluation results of the combined weighted model with the strong correlation rule of landslide disaster.

[0065] The expression for the chi-square test is:

[0066]

[0067] In the formula, X i χ² is the actual observed value, T is the theoretical predicted value, and χ² is the theoretical value. 2 This represents the chi-square test value for the combination of secondary factors and landslide disaster.

[0068] In practice, a chi-square test is performed on areas where landslides have occurred and areas where landslides have not occurred for frequently occurring secondary factor combinations to determine the correlation between the secondary factor combinations and landslide disasters. The area of ​​each factor combination and the area where landslides have occurred are statistically analyzed on the GIS platform. The actual chi-square test value is calculated based on the area of ​​the factor combination and whether or not landslides have occurred. The landslide area ratio of the target area is multiplied by the area of ​​each factor combination to obtain the landslide area of ​​that combination, and the corresponding theoretical chi-square test value is calculated. The chi-square test value for each secondary factor combination is calculated using equation (1) and compared with the critical value of the chi-square test. If the chi-square test value is greater than the critical value, it indicates a significant correlation between the secondary factor combination and landslide disasters. The larger the chi-square test value, the more significant the correlation between the secondary factor combination and landslide disasters.

[0069] The chi-square test values ​​of the mined second-order factor combinations are shown in the table below.

[0070] Table 3. Chi-square test values ​​of the mined second-order factor combinations

[0071]

[0072] The chi-square test value of the secondary factor combination shows a certain positive correlation with the landslide frequency based on the combined weighted model. The chi-square test values ​​of the secondary factor combinations for landslide frequency are all greater than the critical value of 3.841, indicating that the correlation between the secondary factor combinations obtained based on the combined weighted model and landslide disasters is very significant. This combined weighted model is suitable for mining the secondary factor combinations for landslide frequency in target areas, exploring the influence mechanism of multiple disaster-causing factors on the comprehensive effect of landslide disasters, and providing a scientific basis for the rational prevention and control of landslide disasters.

[0073] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof.

[0074] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A landslide frequent secondary factor combination weighted mining method based on big data technology optimization, characterized in that, The method comprises the following steps: Step 1: obtaining landslide disaster factor data and historical landslide disaster data of a target area, coupling each disaster factor with the historical landslide disaster data to obtain a disaster factor and historical landslide association layer; Step 2: taking the number of historical landslides, the area of historical landslides and the density of historical landslides as the basis for mining, respectively, and using an optimized frequent pattern tree algorithm to mine landslide frequent secondary factor combinations and obtain strong association rules of the landslide frequent secondary factor combinations and the landslide disaster layer; Step 3: according to the mining results of the optimized frequent pattern tree algorithm, superimposed analysis of the secondary factor combinations and the landslide disaster layer is performed to construct a combination weighting model and evaluate the landslide frequency of the landslide frequent secondary factor combinations; Step 4: using chi-square test to verify the landslide frequent secondary factor combinations and comparing the evaluation results of the combination weighting model with the strong association rules of the landslide disaster.

2. The method of claim 1, wherein, The step 2 comprises: Step 2.1: frequent pattern mining, selecting frequent parts from a series of candidate sets, i.e. measuring the frequency of item sets, and finding frequent data sets with support greater than the set minimum support; Step 2.2: generating association rules, in the discovered frequent item sets, determining strong association rules according to the frequent item sets and the minimum confidence. 3.The landslide frequent secondary factor combination weighted mining method based on big data technology optimization of claim 2, wherein, The strong association rules are association rules in which the item set support and the confidence meet the minimum support and the minimum confidence; the step 2 further comprises: In the algorithm based on the historical landslide number, the expression of the support is Support(A, B) = P(A&B), and the expression of the confidence is Wherein, P(A&B) is the probability of the simultaneous appearance of the secondary factor A and B; P(B|A) is the probability of the appearance of B on the basis of the appearance of the secondary factor A; and P(A) is the appearance probability of the secondary factor A. In the algorithm based on historical landslide area, the expression of the support is The expression of the confidence is Wherein, Area(A&B) is the historical landslide area where the secondary factors A and B coexist; ∑Area is the total historical landslide area; Area(A) is the historical landslide area where the secondary factor A is located. In the algorithm based on historical landslide density, the expression of the support is The expression of the confidence is Wherein, LD(A&B) is the historical landslide density of the coexistence of the two factors A and B; ∑LD is the total historical landslide density of the target area; and LD(A) is the historical landslide density where the two factors A is located. 4.The landslide frequent secondary factor combination weighted mining method based on big data technology optimization of claim 3, wherein, The optimized frequent pattern tree algorithm comprises: Scanning the database item set, setting the minimum support, removing items smaller than the minimum support, and arranging the original data set entries in descending order of frequency; Scanning the item set again and creating an item header table and a frequent pattern tree, the root node of the tree is empty, the transaction items are child nodes, and branches from the root node to the child nodes are formed in descending order of support, and the frequent item sets are gradually added to the tree structure; From bottom to top, find the conditional pattern base of each item, recursively call the tree structure, remove items smaller than the minimum support, calculate the confidence of the frequent item set, and find strong association rules that meet the conditions. 5.The landslide frequent secondary factor combination weighted mining method based on big data technology optimization of claim 1, wherein, The step 3 comprises: Statistical secondary factor combination of actual landslide area wherein, Secondary factor combination F mined for mining method j i Landslide area of the region where the user is located; Theoretical landslide area calculated by combination of secondary factors wherein A L is the historical landslide area of the target region; is the area of the region where the combination of secondary factors F i is the area of the target region; Combination precision weight W = E·P, wherein, E = (1, 1,..., 1) 1×n , P is a combination precision matrix, P = (p ij ) n×m Wherein, n is the number of excavated secondary factor combinations, and m is the number of excavation methods Evaluation of factor combination F i the frequency of landslides where w i is an element of the matrix W, i.e. W = (w i ) 1×m , C i = (c ij ) m×1 , c ij is the confidence of factor combination F i in mining method j. 6.The landslide frequent secondary factor combination weighted mining method based on big data technology optimization of claim 1, wherein, The step 4 comprises: The expression of chi-square test is Wherein, x i is the actual observation value, T is the theoretical value, χ 2 is the chi-square test value of the secondary factor combination and landslide disaster.

7. A computer device, comprising: The method comprises the following steps: A memory for storing instructions; A processor for calling the instructions stored in the memory to execute the method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, Instructions stored in the memory, which are executed by the processor to execute the method according to any one of claims 1-6.

Images