Drought disaster risk assessment method and system based on big data
By constructing the distance measurement matrix and defining the approximation set, refine the sample classification, and designing the fitness function to update the cluster center position, the problem of rough sample classification and slow clustering processing in drought disaster risk assessment is solved, and the accuracy of the assessment is improved and stability.
Patent Information
- Application Number
- CN202411232898.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-09-04
AI Technical Summary
The existing drought disaster risk assessment methods have too rough sample classification, poor data interpretation, poor evaluation accuracy due to improper selection of data set features, and inappropriate selection of initial cluster centers, resulting in slow convergence speed and single data point allocation, resulting in inaccurate risk assessment results. The problem.
By constructing a distance measurement matrix, refine the sample classification, define the approximation set to obtain the positive domain, boundary domain and negative domain, dynamic feature selection is performed, and the fitness function is designed to update the cluster center position, and the optimal cluster center set is obtained based on the iterative convergence mechanism. Divide data points into core areas and edge areas for allocation.
It improves the accuracy and robustness of drought risk assessment, enhances the stability and accuracy of clustering, so that each data point can be reasonably allocated to the appropriate cluster, and improves the accuracy of the final drought disaster risk assessment.
Smart Images

Figure CN118735284B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of risk assessment, and in particular to a drought disaster risk assessment method and system based on big data. Background Art
[0002] Drought disaster risk assessment method is a systematic method used to evaluate and predict the impact of drought disasters on specific areas, thereby helping decision makers to develop effective drought management and response strategies. However, the general drought disaster risk assessment method has the problems of too rough sample classification, poor data interpretation, and improper selection of data set features, resulting in poor final assessment accuracy; the general drought disaster risk assessment method has the problem of improper selection of initial cluster centers during data set clustering, resulting in slow convergence, and the simplification of data point distribution, resulting in inaccurate risk assessment results. Summary of the invention
[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a drought disaster risk assessment method and system based on big data. In view of the problems that the general drought disaster risk assessment method has too rough classification of samples, poor data interpretation, and improper selection of data set features leading to poor final assessment accuracy, the present invention constructs a neighborhood rough set based on the distance metric matrix, refines the classification of samples, and makes the drought risk assessment more accurate; by defining the approximate set to obtain the positive domain, boundary domain and negative domain, the data label is corrected twice; dynamic feature selection is performed through significance calculation, making the processing of drought disaster risk assessment data more flexible and efficient; thereby improving the adaptability and robustness of the drought disaster risk assessment method; in view of the general drought Disaster risk assessment methods have problems such as slow convergence due to improper selection of initial cluster centers when clustering data sets, and inaccurate risk assessment results caused by the simplification of data point allocation. This scheme updates the cluster center positions by designing a fitness function, and obtains the optimal cluster center set based on an iterative convergence mechanism; thereby improving the stability and accuracy of clustering; by dividing data points into core areas and edge areas, different types of data are allocated to data points based on the relative distance set and minimum distance of the data points; the division method can provide more detailed clustering results; so that each data point can be reasonably allocated to the appropriate cluster according to its relative position, thereby improving the classification accuracy of the cluster; and thus making the accuracy of the final drought disaster risk assessment better.
[0004] The technical solution adopted by the present invention is as follows: The drought disaster risk assessment method based on big data provided by the present invention comprises the following steps:
[0005] Step S1: data collection;
[0006] Step S2: data preprocessing;
[0007] Step S3: data optimization;
[0008] Step S4: clustering processing;
[0009] Step S5: Drought disaster risk assessment.
[0010] Furthermore, in step S1, the data collection is to collect historical drought assessment data and real-time drought assessment data; the historical drought assessment data and the real-time drought assessment data both include meteorological data, hydrological data, soil data and agricultural data; the historical drought assessment data also includes drought assessment level; the drought assessment level is used as a data label, and the data label does not participate in clustering operations.
[0011] Furthermore, in step S2, the data preprocessing is to perform data cleaning, data conversion and standardization on the collected historical drought assessment data and real-time drought assessment data; the data cleaning is to process missing values, duplicate values and outliers; the data conversion is to convert the data into vector form; and the standardization is to standardize the data based on the maximum and minimum normalization method.
[0012] Furthermore, in step S3, the data optimization is to process the data set obtained after the preprocessing in step S2; specifically, the following steps are included:
[0013] Step S31: construct a distance metric matrix; for sample data x and y, construct a distance metric matrix and calculate the distance function ; The formula used is as follows:
[0014] ;
[0015] Where M is a positive definite symmetric matrix; T is the matrix transpose;
[0016] Step S32: Calculate neighborhood rough sets; generate neighborhood relations and neighborhood granularity through distance metric learning; define neighborhood relations based on distance metric learning for ; Define neighborhood granularity for ; Where U is the sample set; δ is the neighborhood threshold;
[0017] Step S33: Calculate the approximation set; calculate the lower approximation and upper approximation sets of the historical drought assessment data; the approximation set is defined as follows:
[0018] ;
[0019] ;
[0020] In the formula, and They are the lower approximation and upper approximation sets of the data set D respectively; and They are logical AND operation and logical OR operation respectively; is the attribute subset; h is the total number of subsets, k is the subset index; is x for the subset The neighborhood granularity;
[0021] Step S34: Drought risk area division: According to the approximation set, the positive domain, boundary domain and negative domain of drought risk are divided; the positive domain is regarded as a high-risk area; the boundary domain is regarded as an uncertain risk area; the negative domain is regarded as a low-risk area; the label type is revised twice based on the drought risk area; the drought risk area is defined as follows:
[0022] ;
[0023] ;
[0024] ;
[0025] In the formula, It is the positive domain of drought risk; It is the boundary domain of drought risk; It is the positive domain of drought risk;
[0026] Step S35: Calculate significance; for historical drought assessment data and real-time drought assessment data, start from an empty attribute subset, and select the features with the highest significance value from all features in turn until the maximum number of features is reached; after the selection is completed, update the data set; the significance is defined as follows:
[0027] ;
[0028] ;
[0029] In the formula, Add attributes to attribute subset B After that, the significant changes of dataset D; and In attribute subset B and attribute subset Next, the approximation degree of the data set D; is the number of data points.
[0030] Furthermore, in step S4, the clustering process specifically includes the following steps:
[0031] Step S41: Initialization: randomly selecting K data points from the data set as initial cluster centers;
[0032] Step S42: fitness function definition; specifically including:
[0033] Step S421: define the cohesion within the cluster; the formula used is as follows:
[0034] ;
[0035] In the formula, It is a cluster The cohesion degree; is a data point; is the Euclidean distance;
[0036] Step S422: define the inter-cluster separation metric; the formula used is as follows:
[0037] ;
[0038] In the formula, is the inter-cluster separation measure; CS is the cluster set; c i3 and c j3 are two cluster centers;
[0039] Step S423: define the fitness function; the formula used is as follows:
[0040] ;
[0041] ;
[0042] In the formula, is the cohesion of the entire cluster set; is the fitness value function;
[0043] Step S43: Optimization; for each cluster center, calculate the fitness value of the location, and update the access level and location; take the location with the maximum fitness value as the optimal cluster center; expressed as follows:
[0044] ;
[0045] ;
[0046] In the formula, is the optimal cluster center set; and They are the position after updating and the position before updating respectively; is the update coefficient; is the location of the optimal cluster center;
[0047] Step S44: iteration; for each cluster center, if the updated fitness value is higher than the fitness value before the update, the position is updated; otherwise, the position is not updated; the position update is iterated repeatedly until convergence;
[0048] Step S45: clustering processing; pre-perform k-means clustering on the data set using the converged initial cluster centers; the subsequent clustering processing specifically includes:
[0049] Step S451: Region division: Divide the core region into ; ; ; Divide the edge area, expressed as ;in ; Define the upper approximation set of the cluster as ;in, is the core area of the kth cluster; x is a data point; and There are two types of data point sets; is the upper approximation set of the kth cluster; is an expansion set; is the number of data points in the expansion set; is the optimal cluster center of the kth cluster; is the updated cluster center; β is the core area threshold; is the distance between cluster centers before and after the update; , and is the extended data point; is the upper approximation set of the jth cluster; K is the total number of clusters; is the edge region of the kth cluster;
[0050] Step S452: Data point allocation; for the data points in the core area, define a relative distance set, expressed as ; Classify data points into the upper approximate set of clusters, expressed as ; For data points in the edge area, the minimum distance is defined as , assigning data to the cluster with the smallest distance; where, is the relative distance set of x; is the cluster center of the jth cluster; is the nearest cluster center; α is the decision threshold;
[0051] Step S453: clustering judgment; when clustering reaches convergence or the maximum number of iterations, clustering ends; and clustering results of historical drought assessment data and real-time drought assessment data are obtained.
[0052] Further, in step S5, the drought disaster risk assessment is based on the clustering results of historical drought assessment data and real-time drought assessment data; the label type with the largest number of historical drought assessment data is used as the cluster label; the cluster label type corresponding to the real-time drought assessment data is used as the assessment result of the real-time drought assessment data to realize drought disaster risk assessment.
[0053] The drought disaster risk assessment system based on big data provided by the present invention comprises a data acquisition module, a data preprocessing module, a data optimization module, a clustering processing module and a drought disaster risk assessment module;
[0054] The data acquisition module acquires historical drought assessment data and real-time drought assessment data, and sends the data to the data preprocessing module;
[0055] The data preprocessing module performs data cleaning, data conversion and standardization processing on the collected historical drought assessment data and real-time drought assessment data, and sends the data to the data optimization module;
[0056] The data optimization module constructs a neighborhood rough set based on a distance metric matrix; obtains a positive domain, a boundary domain, and a negative domain by defining an approximation set; performs dynamic feature selection by saliency calculation; thereby achieving data optimization; and sends the data to a clustering processing module;
[0057] The cluster processing module updates the cluster center position by designing a fitness function, obtains the optimal cluster center set based on an iterative convergence mechanism, allocates different types of data to the data points based on the relative distance set and the minimum distance of the data points, obtains the clustering results, and sends the data to the drought disaster risk assessment module;
[0058] The drought disaster risk assessment module performs risk assessment on the real-time drought assessment data based on the clustering results.
[0059] The beneficial effects achieved by the present invention using the above scheme are as follows:
[0060] (1) In view of the problems of rough sample classification, poor data interpretation and inappropriate feature selection of data sets in general drought disaster risk assessment methods, which lead to poor final assessment accuracy, this scheme constructs a neighborhood rough set based on the distance metric matrix to refine the sample classification and make the drought risk assessment more accurate. The positive domain, boundary domain and negative domain are obtained by defining the approximate set, and the data labels are corrected twice. Dynamic feature selection is performed through significance calculation, making the processing of drought disaster risk assessment data more flexible and efficient. This improves the adaptability and robustness of the drought disaster risk assessment method.
[0061] (2) In view of the problems of slow convergence speed caused by improper selection of initial cluster centers when clustering data sets in general drought disaster risk assessment methods, and inaccurate risk assessment results caused by the uniform distribution of data points, this scheme updates the cluster center positions by designing a fitness function and obtains the optimal cluster center set based on an iterative convergence mechanism, thereby improving the stability and accuracy of clustering; by dividing data points into core areas and edge areas, different types of data are allocated to data points based on the relative distance set and minimum distance of data points; the division method can provide more detailed clustering results; so that each data point can be reasonably allocated to the appropriate cluster according to its relative position, thereby improving the classification accuracy of the cluster; and thus making the accuracy of the final drought disaster risk assessment better. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 A schematic diagram of the process of drought disaster risk assessment method based on big data provided by the present invention;
[0063] Figure 2 A schematic diagram of a drought disaster risk assessment system based on big data provided by the present invention;
[0064] Figure 3 is a schematic diagram of the process of step S3;
[0065] Figure 4 It is a schematic diagram of the process of step S4.
[0066] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION
[0067] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0068] In the description of the present invention, it is necessary to understand that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred system or element must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.
[0069] Example 1, see Figure 1The present invention provides a drought disaster risk assessment method based on big data, which comprises the following steps:
[0070] Step S1: data collection: collecting historical drought assessment data and real-time drought assessment data;
[0071] Step S2: data preprocessing: data cleaning, data conversion and standardization of the collected historical drought assessment data and real-time drought assessment data;
[0072] Step S3: data optimization; constructing a neighborhood rough set based on the distance metric matrix; obtaining the positive domain, boundary domain and negative domain by defining the approximation set; performing dynamic feature selection by saliency calculation; thereby achieving data optimization;
[0073] Step S4: clustering processing; updating the cluster center position by designing a fitness function, obtaining the optimal cluster center set based on an iterative convergence mechanism; performing different types of data allocation for the data points based on the relative distance set and the minimum distance of the data points; and obtaining the clustering result;
[0074] Step S5: Drought disaster risk assessment: Perform risk assessment on the real-time drought assessment data based on the clustering results.
[0075] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, both the historical drought assessment data and the real-time drought assessment data include meteorological data, hydrological data, soil data and agricultural data; the historical drought assessment data also includes a drought assessment level; the drought assessment level is used as a data label, and the data label does not participate in the clustering operation.
[0076] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S2, data cleaning is to process missing values, duplicate values and outliers; data conversion is to convert data into vector form; and standardization is to standardize data based on the maximum and minimum normalization method.
[0077] Example 4, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S3, data optimization is to process the data set obtained after preprocessing in step S2. Specifically, the following steps are included:
[0078] Step S31: construct a distance metric matrix; for sample data x and y, construct a distance metric matrix and calculate the distance function ; The formula used is as follows:
[0079] ;
[0080] Where M is a positive definite symmetric matrix; T is the matrix transpose;
[0081] Step S32: Calculate neighborhood rough sets; generate neighborhood relations and neighborhood granularity through distance metric learning; define neighborhood relations based on distance metric learning for ; Define neighborhood granularity for ; Where U is the sample set; δ is the neighborhood threshold;
[0082] Step S33: Calculate the approximation set; calculate the lower approximation and upper approximation sets of the historical drought assessment data; the approximation set is defined as follows:
[0083] ;
[0084] ;
[0085] In the formula, and They are the lower approximation and upper approximation sets of the data set D respectively; and They are logical AND operation and logical OR operation respectively; is the attribute subset; h is the total number of subsets, k is the subset index; is x for the subset The neighborhood granularity;
[0086] Step S34: Drought risk area division: According to the approximation set, the positive domain, boundary domain and negative domain of drought risk are divided; the positive domain is regarded as a high-risk area; the boundary domain is regarded as an uncertain risk area; the negative domain is regarded as a low-risk area; the label type is revised twice based on the drought risk area; the drought risk area is defined as follows:
[0087] ;
[0088] ;
[0089] ;
[0090] In the formula, It is the positive domain of drought risk; It is the boundary domain of drought risk; It is the positive domain of drought risk;
[0091] Step S35: Calculate significance; for historical drought assessment data and real-time drought assessment data, start from an empty attribute subset, and select the features with the highest significance value from all features in turn until the maximum number of features is reached; after the selection is completed, update the data set; the significance is defined as follows:
[0092] ;
[0093] ;
[0094] In the formula, Add attributes to attribute subset B After that, the significant changes of dataset D; and In attribute subset B and attribute subset Next, the approximation degree of the data set D; is the number of data points.
[0095] By performing the above operations, in view of the problems that the general drought disaster risk assessment method has too rough classification of samples, poor data interpretation, and improper feature selection of data sets leading to poor final assessment accuracy, this scheme constructs a neighborhood rough set based on the distance metric matrix to refine the classification of samples, making the assessment of drought risk more accurate; by defining the approximation set to obtain the positive domain, boundary domain and negative domain, the data label is corrected twice; dynamic feature selection is performed through significance calculation, making the processing of drought disaster risk assessment data more flexible and efficient; thereby improving the adaptability and robustness of the drought disaster risk assessment method.
[0096] Example 5, see Figure 1 and Figure 4 This embodiment is based on the above embodiment. In step S4, the clustering process specifically includes the following steps:
[0097] Step S41: Initialization: randomly selecting K data points from the data set as initial cluster centers;
[0098] Step S42: fitness function definition; specifically including:
[0099] Step S421: define the cohesion within the cluster; the formula used is as follows:
[0100] ;
[0101] In the formula, It is a cluster The cohesion degree; is a data point; is the Euclidean distance;
[0102] Step S422: define the inter-cluster separation metric; the formula used is as follows:
[0103] ;
[0104] In the formula, is the inter-cluster separation measure; CS is the cluster set; c i3 and c j3are two cluster centers;
[0105] Step S423: define the fitness function; the formula used is as follows:
[0106] ;
[0107] ;
[0108] In the formula, is the cohesion of the entire cluster set; is the fitness value function;
[0109] Step S43: Optimization; for each cluster center, calculate the fitness value of the location, and update the access level and location; take the location with the maximum fitness value as the optimal cluster center; expressed as follows:
[0110] ;
[0111] ;
[0112] In the formula, is the optimal cluster center set; and They are the position after updating and the position before updating respectively; is the update coefficient; is the location of the optimal cluster center;
[0113] Step S44: iteration; for each cluster center, if the updated fitness value is higher than the fitness value before the update, the position is updated; otherwise, the position is not updated; the position update is iterated repeatedly until convergence;
[0114] Step S45: clustering processing; pre-perform k-means clustering on the data set using the converged initial cluster centers; the subsequent clustering processing specifically includes:
[0115] Step S451: Region division: Divide the core region into ; ; ; Divide the edge area, expressed as ;in ; Define the upper approximation set of the cluster as ;in, is the core area of the kth cluster; x is a data point; and There are two types of data point sets; is the upper approximation set of the kth cluster; is an expansion set; is the number of data points in the expansion set; is the optimal cluster center of the kth cluster; is the updated cluster center; β is the core area threshold; is the distance between cluster centers before and after the update; , and is the extended data point; is the upper approximation set of the jth cluster; K is the total number of clusters; is the edge region of the kth cluster;
[0116] Step S452: Data point allocation; for the data points in the core area, define a relative distance set, expressed as ; Classify data points into the upper approximate set of clusters, expressed as ; For data points in the edge area, the minimum distance is defined as , assigning data to the cluster with the smallest distance; where, is the relative distance set of x; is the cluster center of the jth cluster; is the nearest cluster center; is the decision threshold;
[0117] Step S453: clustering judgment; when clustering reaches convergence or the maximum number of iterations, clustering ends; and clustering results of historical drought assessment data and real-time drought assessment data are obtained.
[0118] By performing the above operations, in view of the problems that the general drought disaster risk assessment method has the problems of slow convergence speed due to improper selection of initial cluster centers when clustering the data set, and inaccurate risk assessment results caused by the simplification of data point allocation, this scheme updates the cluster center positions by designing a fitness function, and obtains the optimal cluster center set based on the iterative convergence mechanism; thereby improving the stability and accuracy of clustering; by dividing the data points into core areas and edge areas, different types of data are allocated to the data points based on the relative distance set and the minimum distance of the data points; the division method can provide more detailed clustering results; so that each data point can be reasonably allocated to the appropriate cluster according to its relative position, thereby improving the classification accuracy of the cluster; and thus making the accuracy of the final drought disaster risk assessment better.
[0119] Example 6, see Figure 1 This embodiment is based on the above embodiment. In step S5, drought disaster risk assessment is based on the clustering results of historical drought assessment data and real-time drought assessment data; the label type with the largest number of historical drought assessment data is used as the cluster label; the cluster label type corresponding to the real-time drought assessment data is used as the assessment result of the real-time drought assessment data to realize drought disaster risk assessment.
[0120] Embodiment 7, see Figure 2, this embodiment is based on the above embodiment, and the drought disaster risk assessment system based on big data provided by the present invention includes a data acquisition module, a data preprocessing module, a data optimization module, a clustering processing module and a drought disaster risk assessment module;
[0121] The data acquisition module acquires historical drought assessment data and real-time drought assessment data, and sends the data to the data preprocessing module;
[0122] The data preprocessing module performs data cleaning, data conversion and standardization processing on the collected historical drought assessment data and real-time drought assessment data, and sends the data to the data optimization module;
[0123] The data optimization module constructs a neighborhood rough set based on a distance metric matrix; obtains a positive domain, a boundary domain, and a negative domain by defining an approximation set; performs dynamic feature selection by saliency calculation; thereby achieving data optimization; and sends the data to a clustering processing module;
[0124] The cluster processing module updates the cluster center position by designing a fitness function, obtains the optimal cluster center set based on an iterative convergence mechanism, allocates different types of data to the data points based on the relative distance set and the minimum distance of the data points, obtains the clustering results, and sends the data to the drought disaster risk assessment module;
[0125] The drought disaster risk assessment module performs risk assessment on the real-time drought assessment data based on the clustering results.
[0126] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0127] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.
[0128] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.
Claims
1. A drought disaster risk assessment method based on big data, characterized by: The method comprises the following steps: Step S1: data collection; Step S2: data preprocessing; Step S3: data optimization; constructing a neighborhood rough set based on the distance metric matrix; obtaining the positive domain, boundary domain and negative domain by defining the approximation set; performing dynamic feature selection by saliency calculation; thereby achieving data optimization; Step S4: clustering processing; updating the cluster center position by designing a fitness function, obtaining the optimal cluster center set based on an iterative convergence mechanism; performing different types of data allocation for the data points based on the relative distance set and the minimum distance of the data points; and obtaining the clustering result; Step S5: Drought disaster risk assessment; Step S3: data optimization includes step S33: calculating an approximation set; calculating a lower approximation and an upper approximation set of historical drought assessment data; the approximation set is defined as follows: ; ; In the formula, and They are the lower approximation and upper approximation sets of the data set D respectively; and They are logical AND operation and logical OR operation respectively; is the attribute subset; h is the total number of subsets, k is the subset index; is x for the subset Neighborhood granularity; x is sample data; U is the sample set.
2. The drought disaster risk assessment method based on big data according to claim 1 is characterized by: In step S4, the clustering process specifically includes the following steps: Step S41: Initialization: randomly selecting K data points from the data set as initial cluster centers; Step S42: fitness function definition; specifically including: Step S421: define the cohesion within the cluster; the formula used is as follows: ; In the formula, It is a cluster The cohesion degree; is a data point; is the Euclidean distance; Step S422: define the inter-cluster separation metric; the formula used is as follows: ; In the formula, is the inter-cluster separation measure; CS is the cluster set; c i3 and c j3 are two cluster centers; Step S423: define the fitness function; the formula used is as follows: ; ; In the formula, is the cohesion of the entire cluster set; is the fitness value function; Step S43: Optimization; for each cluster center, calculate the fitness value of the location, and update the access level and location; take the location with the maximum fitness value as the optimal cluster center; expressed as follows: ; ; In the formula, is the optimal cluster center set; and They are the position after updating and the position before updating respectively; is the update coefficient; is the location of the optimal cluster center; Step S44: iteration; for each cluster center, if the updated fitness value is higher than the fitness value before the update, the position is updated; otherwise, the position is not updated; the position update is iterated repeatedly until convergence; Step S45: Clustering processing.
3. The drought disaster risk assessment method based on big data according to claim 2 is characterized in that: In step S45, the clustering process is to perform a k-means clustering on the data set using the converged initial cluster centers in advance; the subsequent clustering process specifically includes: Step S451: Region division: Divide the core region into ; ; ; Divide the edge area, expressed as ;in ; Define the upper approximation set of the cluster as ;in, is the core area of the kth cluster; x is a data point; and There are two types of data point sets; is the upper approximation set of the kth cluster; is an expansion set; is the number of data points in the expansion set; is the optimal cluster center of the kth cluster; is the updated cluster center; β is the core area threshold; is the distance between cluster centers before and after the update; , and is the extended data point; is the upper approximation set of the jth cluster; K is the total number of clusters; is the edge region of the kth cluster; Step S452: Data point allocation; for the data points in the core area; define a relative distance set, expressed as ; Classify data points into the upper approximate set of clusters, expressed as ; For data points in the edge area, the minimum distance is defined as , assigning data to the cluster with the smallest distance; where, is the relative distance set of x; is the cluster center of the jth cluster; is the nearest cluster center; is the decision threshold; Step S453: clustering judgment; when clustering reaches convergence or the maximum number of iterations, clustering ends; and clustering results of historical drought assessment data and real-time drought assessment data are obtained.
4. The drought disaster risk assessment method based on big data according to claim 1 is characterized in that: In step S3, the data optimization is to process the data set obtained after the preprocessing in step S2; specifically, the following steps are included: Step S31: construct a distance metric matrix; for sample data x and y, construct a distance metric matrix and calculate the distance function ; The formula used is as follows: ; Where M is a positive definite symmetric matrix; T is the matrix transpose; Step S32: Calculate neighborhood rough sets; generate neighborhood relations and neighborhood granularity through distance metric learning; define neighborhood relations based on distance metric learning for ; Define neighborhood granularity for ; Where U is the sample set; δ is the neighborhood threshold; Step S33: Calculate the approximation set; Step S34: Drought risk area division: According to the approximation set, the positive domain, boundary domain and negative domain of drought risk are divided; the positive domain is regarded as a high-risk area; the boundary domain is regarded as an uncertain risk area; the negative domain is regarded as a low-risk area; the label type is revised twice based on the drought risk area; the drought risk area is defined as follows: ; ; ; In the formula, It is the positive domain of drought risk; It is the boundary domain of drought risk; It is the positive domain of drought risk; Step S35: Calculate significance; for historical drought assessment data and real-time drought assessment data, start from an empty attribute subset, and select the features with the highest significance value from all features in turn until the maximum number of features is reached; after the selection is completed, update the data set; the significance is defined as follows: ; ; In the formula, Add attributes to attribute subset B After that, the significant changes of dataset D; and In attribute subset B and attribute subset Next, the approximation degree of the data set D; is the number of data points.
5. The drought disaster risk assessment method based on big data according to claim 1, characterized in that: In step S1, the data collection is to collect historical drought assessment data and real-time drought assessment data; the historical drought assessment data and the real-time drought assessment data both include meteorological data, hydrological data, soil data and agricultural data; the historical drought assessment data also includes drought assessment level; the drought assessment level is used as a data label, and the data label does not participate in clustering operations.
6. The drought disaster risk assessment method based on big data according to claim 1 is characterized by: In step S5, the drought disaster risk assessment is based on the clustering results of the historical drought assessment data and the real-time drought assessment data; the label type with the largest number of historical drought assessment data is used as the cluster label; The cluster label type corresponding to the real-time drought assessment data is used as the assessment result of the real-time drought assessment data to realize drought disaster risk assessment.
7. The drought disaster risk assessment method based on big data according to claim 1 is characterized in that: In step S2, the data preprocessing is to perform data cleaning, data conversion and standardization on the collected historical drought assessment data and real-time drought assessment data; the data cleaning is to process missing values, duplicate values and outliers; the data conversion is to convert the data into vector form; the standardization is to standardize the data based on the maximum and minimum normalization method.
8. A drought disaster risk assessment system based on big data, used to implement the drought disaster risk assessment method based on big data as described in any one of claims 1 to 7, characterized in that: It includes data acquisition module, data preprocessing module, data optimization module, cluster processing module and drought disaster risk assessment module; The data acquisition module acquires historical drought assessment data and real-time drought assessment data, and sends the data to the data preprocessing module; The data preprocessing module performs data cleaning, data conversion and standardization processing on the collected historical drought assessment data and real-time drought assessment data, and sends the data to the data optimization module; The data optimization module constructs a neighborhood rough set based on a distance metric matrix; By defining the approximation set, the positive domain, the boundary domain and the negative domain are obtained; dynamic feature selection is performed through saliency calculation; thereby data optimization is achieved; and the data is sent to the clustering processing module; The cluster processing module updates the cluster center position by designing a fitness function, obtains the optimal cluster center set based on an iterative convergence mechanism, and performs different types of data allocation for the data points based on the relative distance set and the minimum distance of the data points, thereby obtaining the clustering result. And send the data to the drought disaster risk assessment module; The drought disaster risk assessment module performs risk assessment on the real-time drought assessment data based on the clustering results.
Citation Information
Cited By
Grassland ecosystem comprehensive drought risk assessment system based on multi-source data
CN121564416A
