Black land cultivated land quality evaluation method

Abstract

The invention relates to a black land cultivated land quality evaluation method, which relates to the technical field of agricultural engineering, and comprises the step of carrying out series processing on a cultivated land quality data set, specifically comprising missing value deletion, filling, qualitative data standardization processing and the like. The cultivated land quality evaluation indexes are more and belong to high-dimensional feature data, errors are reduced through data dimension reduction, and the precision is improved. And under the condition that the evaluation indexes are the same, clustering is carried out on cultivated land quality data participating in evaluation by adopting multiple unsupervised clustering algorithms, and a contour coefficient is introduced to verify the cohesion and the separation degree of clustering. And calculating the Bhattacharyya distance and the average yield of each cluster, carrying out credibility and separability test on the clusters through the Bhattacharyya distance and variance analysis, and fusing cultivated land pattern spot clusters without separability. And finally, generating a cultivated land quality evaluation result according to the average yield of each cluster.

Description

Technical Field

[0001] The invention relates to the technical field of agricultural engineering, and in particular to a method for evaluating the quality of black soil arable land. Background Art

[0002] The black soil in Northeast China is known as the "giant panda" among my country's cultivated land. It is an important grain production area and commercial grain production base in my country. It is the "stabilizer" and "ballast stone" of China's food security, providing important guarantees for my country's food security. However, due to the long-term lack of the concept of "sustainable development", in the development and utilization of black soil, people have over-cultivated, over-utilized, abused fertilizers and pesticides, and the influence of natural factors, which has led to a rapid decline in the quality of black soil cultivated land and is facing a series of problems and challenges.

[0003] As a special kind of geographic data, cultivated land quality data has important significance for its visualization research. Hang Tianwen and others developed a national cultivated land quality big data platform, but the platform failed to treat cultivated land data in different regions differently; Chen Yibing and others built a black soil (cultivated land) quality big data platform in Jilin Province based on cloud computing, which mainly focuses on the storage, analysis and visualization of cultivated land quality data of black soil in Jilin Province. Existing cultivated land quality evaluation software generally shows great limitations in responding to the needs of modern cultivated land big data visualization. In the existing technology, these software often find it difficult to adapt to multi-source heterogeneous data types and the requirements of real-time dynamic updates, resulting in incomplete and inaccurate evaluation results. Summary of the invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art that it is difficult to adapt to the requirements of multi-source heterogeneous data types and real-time dynamic updates, resulting in inaccurate and ineffective evaluation results, the main purpose of the present invention is to provide a method for evaluating the quality of black soil arable land. In order to accurately protect the black soil in Northeast China and improve the quality of black soil arable land, it is necessary to adopt scientific, objective and solid methods and technologies to comprehensively evaluate and analyze the quality of black soil arable land.

[0005] To achieve the above object, the present invention adopts the following technical scheme, a method for evaluating the quality of black soil arable land, comprising the following steps:

[0006] Obtain black soil arable land quality data, perform preprocessing and data dimension reduction, and obtain low-dimensional black soil arable land quality data after preprocessing;

[0007] A variety of unsupervised clustering algorithms are used to cluster low-dimensional black soil arable land quality data to obtain a variety of clustering results, and the silhouette coefficient is introduced to evaluate the various clustering results to obtain the best clustering result.

[0008] Obtain the Bhattacharyya distance of each cluster in the best clustering result, and perform credibility and separability tests on the cluster of the best clustering result by combining Bhattacharyya distance with variance analysis, and merge the clusters that are not separable until multiple clusters with the best separability and credibility are obtained;

[0009] The average yield of multiple clusters is obtained, and the black soil arable land is classified according to the average yield to obtain the classification results of the black soil arable land, that is, to generate the arable land quality evaluation results.

[0010] The black soil arable land quality data includes 16 evaluation indicators and yield attribute data.

[0011] The method of obtaining black soil arable land quality data, performing preprocessing and data dimension reduction comprises the following steps:

[0012] The quality data of black soil arable land is proofread by the nearest value assignment method to obtain the proofread quality data of black soil arable land;

[0013] The membership value is used to quantify the quality data of black soil cultivated land after calibration, and the quantified quality data of black soil cultivated land is obtained;

[0014] The quantified black soil arable land quality data were checked for missing data, and the missing data were filled by the mode interpolation method to obtain the standardized black soil arable land quality data;

[0015] The standardized black soil arable land quality data were processed by t-SNE data dimensionality reduction method to obtain low-dimensional black soil arable land quality data.

[0016] The method of performing data dimensionality reduction processing on the standardized black soil arable land quality data by using the t-SNE data dimensionality reduction method to obtain low-dimensional black soil arable land quality data includes the following steps:

[0017] The similarity between the standardized black soil arable land quality data points in high-dimensional space is calculated through Gaussian distribution, and a symmetric similarity matrix is ​​constructed to quantify the similarity between the standardized black soil arable land quality data points in high-dimensional space.

[0018] Randomly initialize the standardized black soil arable land quality data points in low-dimensional space;

[0019] Taking the symmetric similarity matrix as the optimization target, the similarity of the standardized black soil arable land quality data points in low-dimensional space was calculated using t distribution.

[0020] The position of the midpoint in the low-dimensional space is optimized by KL divergence. Combined with the symmetric similarity matrix, the gradient descent method is used to minimize the KL divergence, update the position of the midpoint in the low-dimensional space, and obtain the low-dimensional black soil arable land quality data.

[0021] The multiple clustering algorithms include: K-means, BIRCH, Gaussian mixture model GMM, spectral clustering and K-medoids.

[0022] The method of calculating the Bhattacharyya distance and average yield of each cluster center and testing the reliability and separability of the clusters by Bhattacharyya distance and variance analysis includes the following steps:

[0023] Randomly select a cluster and use it as the center point. Use Bhattacharyya distance to calculate the similarity between the cluster and other clusters, and arrange the clusters from near to far.

[0024] Perform variance analysis on adjacent clusters, merge two clusters with P values ​​less than the set value to obtain new clusters;

[0025] After completing the pairwise ANOVA, further ANOVA was conducted on all clusters, and the average yield of each cluster was obtained. The cultivated land was classified and graded according to the average yield.

[0026] A system for evaluating the quality of black soil arable land, comprising:

[0027] A data processing module is used to obtain black soil arable land quality data, perform preprocessing and data dimension reduction, and obtain low-dimensional black soil arable land quality data after preprocessing;

[0028] The data clustering analysis module uses a variety of unsupervised clustering algorithms to cluster low-dimensional black soil arable land quality data, obtains a variety of clustering results, and introduces the silhouette coefficient to evaluate the various clustering results to obtain the best clustering result; obtains the Bhattacharyya distance of each cluster in the best clustering result, and performs credibility and separability tests on the clusters of the best clustering result through Bhattacharyya distance combined with variance analysis, and merges the indivisible clusters until multiple clusters with the best separability and credibility are obtained;

[0029] Evaluation and analysis module, used to obtain the average yield of multiple clusters,

[0030] According to the average yield, the black soil arable land is graded to obtain the black soil arable land classification results, that is, the arable land quality evaluation results are generated.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: the black soil arable land quality evaluation process based on the hierarchical analysis method and the clustering method provides method support for the analysis and evaluation of the quality of black soil arable land, and can scientifically and reasonably analyze and evaluate the quality of black soil arable land. In the hierarchical analysis method, the quality of black soil arable land is divided into 10 levels through a series of processes such as dividing the evaluation unit, selecting the evaluation index, establishing the evaluation index system, determining the evaluation index weight and membership, etc., to provide a scientific basis for judging the quality of black soil arable land. In view of the disadvantage of poor objectivity of the hierarchical analysis method, a black soil arable land quality evaluation method based on clustering is proposed. In this method, 5 different clustering algorithms are deeply analyzed, and the rationality of the clustering results is verified by using methods such as the silhouette coefficient, the Bhattacharyya distance, and the variance analysis. Finally, the arable land is classified and graded according to the average yield, which provides strong support for the rational utilization and optimal allocation of arable land resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0033] Figure 1 It is a schematic diagram of the process structure of the present invention;

[0034] Figure 2 is a schematic diagram of spectral clustering of the present invention; wherein, Figure 2 (a) is the spectral clustering diagram; Figure 2 (b) is the spectral clustering cut graph. DETAILED DESCRIPTION

[0035] As a key geographic information resource, the visualization of cultivated land quality data has far-reaching significance for agricultural development and land resource management. Although the national cultivated land quality big data platform developed by Hang Tianwen's team has made contributions in data integration and display, it has failed to fully consider the differences between regions when processing cultivated land data in different regions, which may affect the accuracy of the evaluation.

[0036] At the same time, Chen Yibing and others built a big data platform focusing on the quality of black soil in Jilin Province based on cloud computing technology. The platform is dedicated to data storage, in-depth analysis and intuitive visualization of black soil in Jilin Province, providing strong support for local agricultural decision-making. However, existing arable land quality evaluation software generally has certain limitations when responding to the needs of modern arable land big data visualization.

[0037] These software often seem to be unable to cope with multi-source heterogeneous data types and real-time dynamic update requirements, which leads to unsatisfactory comprehensiveness and accuracy of evaluation results. Therefore, in order to improve the efficiency of cultivated land quality evaluation, it is urgent to develop advanced visualization technology that can adapt to complex data environments and dynamic update requirements.

[0038] In order to overcome the shortcomings of the above-mentioned prior art that it is difficult to adapt to the requirements of multi-source heterogeneous data types and real-time dynamic updates, resulting in inaccurate and ineffective evaluation results, the main purpose of the present invention is to provide a method for evaluating the quality of black soil arable land. In order to accurately protect the black soil in Northeast China and improve the quality of black soil arable land, it is necessary to adopt scientific, objective and solid methods and technologies to comprehensively evaluate and analyze the quality of black soil arable land.

[0039] To achieve the above object, the present invention adopts the following technical scheme, a method for evaluating the quality of black soil arable land, comprising the following steps:

[0040] Obtain black soil arable land quality data, perform preprocessing and data dimension reduction, and obtain low-dimensional black soil arable land quality data after preprocessing;

[0041] A variety of unsupervised clustering algorithms are used to cluster low-dimensional black soil arable land quality data to obtain a variety of clustering results, and the silhouette coefficient is introduced to evaluate the various clustering results to obtain the best clustering result.

[0042] Obtain the Bhattacharyya distance of each cluster in the best clustering result, and perform credibility and separability tests on the cluster of the best clustering result by combining Bhattacharyya distance with variance analysis, and merge the clusters that are not separable until multiple clusters with the best separability and credibility are obtained;

[0043] The average yield of multiple clusters is obtained, and the black soil arable land is classified according to the average yield to obtain the classification results of the black soil arable land, that is, to generate the arable land quality evaluation results.

[0044] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0045] Example:

[0046] As an unsupervised classification method, clustering method is completely data-driven and can minimize the interference of subjective factors and ensure the objectivity of the evaluation results. Figure 1 shown.

[0047] First, data processing. The cultivated land quality data set is processed, including deletion and filling of missing values, and standardization of qualitative data. Second, data dimension reduction. The cultivated land quality data involves many evaluation indicators and belongs to high-dimensional feature data. There will be errors when using machine learning for prediction and classification. Data dimension reduction can reduce errors and improve accuracy. Then, when all factors are the same, a variety of clustering algorithms are used for clustering, and the silhouette coefficient is introduced to verify the cohesion and separation of the clusters. Furthermore, the Bhattacharyya distance and average yield of each cluster center are calculated, and the credibility and separability of the clusters are tested by Bhattacharyya distance and variance analysis. Finally, the cultivated land quality evaluation results are generated by the average yield.

[0048] The quality data of black soil arable land is composed of 16 evaluation indicators and yield attribute data, including conceptual indicator data and numerical indicator data. First, the data was proofread for the first time. For data values ​​with unclear meanings, missing values, data errors, etc., the nearest value assignment method was used for proofreading; secondly, the membership value was used to represent the unique meaning of the conceptual indicator data, and the numerical indicator data was guaranteed to be of the same type, thus completing the data quantification. Furthermore, with the help of third-party libraries (pandas, numpy, scikit-learn), the quantified arable land quality data was checked for data types and missing conditions as a whole, and the majority interpolation method was used to fill in the missing values.

[0049] As a kind of high-dimensional characteristic data, black soil arable land quality data is difficult to accurately analyze and visualize. In order to reduce errors, improve data processing accuracy, and facilitate subsequent visualization, this paper uses the t-SNE data dimensionality reduction method.

[0050] As a data dimensionality reduction method of nonlinear transformation, t-SNE not only performs well in processing high-dimensional data, but also takes into account the needs of data dimensionality reduction and visualization. The core idea of ​​t-SNE is to construct a probability distribution between data points in high-dimensional space, which can reflect the similarity between each point. Then, in low-dimensional space, t-SNE creates a similar probability distribution. By minimizing the difference between the two distributions (KL divergence), the mapping of high-dimensional data to low-dimensional space is completed.

[0051] The specific process is as follows:

[0052] (1) The similarity between data points in high-dimensional space is calculated through Gaussian distribution, and a symmetric similarity matrix is ​​constructed to quantify the similarity between data points in high-dimensional space. The similarity between points is obtained through formula (1), and the symmetric similarity matrix is ​​constructed through formula (2).

[0053]

[0054] In formula (1), the numerator is x i and x j The square of the Euclidean distance between them is converted into probability through Gaussian distribution. The denominator is the normalization factor to ensure that p j|i The sum of is 1. i Represents the variance of the Gaussian distribution, determines the range of the nearest neighbors, and is usually measured by "perplexity".

[0055]

[0056] (2) Randomly initialize data points in the low-dimensional space.

[0057] (3) The similarity of low-dimensional space data points is calculated using t distribution. It is obtained by formula (3).

[0058]

[0059] In formula (3), the numerator is y i and j The square of the Euclidean distance between them is converted into a probability using the Cauchy distribution. The denominator is a normalization factor to ensure that all q ij The sum is 1.

[0060] (4) Optimize the position of the points in the low-dimensional space and finally complete the data dimensionality reduction. First, optimize the position formula of the point in the low-dimensional space through KL divergence (4). Second, use the gradient descent method to minimize the KL divergence and update the position formula of the point in the low-dimensional space (5).

[0061]

[0062] Cluster analysis specifically uses five clustering algorithms, including K-means, BIRCH, Gaussian mixture model (GMM), spectral clustering, and K-medoids, to cluster the cultivated land patch data in Heilongjiang Province, and introduces the silhouette coefficient to verify the cohesion and separation of the clustering results.

[0063] (1) K-means clustering model analysis

[0064] K-means is a partition-based clustering algorithm. Its core goal is to divide a given data set into K clusters and give the center point corresponding to each sample data. It iterates continuously until the final cluster center and K clusters are output. It is widely used in different scenarios because of its easy operation and strong interpretability of results. However, the algorithm occasionally falls into a local minimum. The K-means clustering algorithm process is as follows:

[0065] The first step is to randomly select K centers, denoted as

[0066] The second step is to take each sample x in the dataset. i Calculate the distances to the K cluster centers and divide them into the clusters corresponding to the cluster centers with the smallest distances. And update the “cluster centers” according to the newly divided clusters.

[0067] The third step is to repeat the second step until the cluster center no longer changes.

[0068] The fourth step is to output the final cluster center and K clusters.

[0069] (2) BIRCH clustering model analysis

[0070] BIRCH is a clustering algorithm for large data sets, especially for large-scale data sets with noise. The core idea of ​​the BIRCH algorithm is to quickly complete clustering by constructing a clustering feature tree (CF Tree). CF Tree is a balanced tree structure used to store clustering features (CF). Each node in the CF Tree (including leaf nodes) has several CFs, and the CF of the internal node points to the pointer of the child node. All leaf nodes are linked with a bidirectional linked list. Therefore, each node in the tree corresponds to a cluster, and the cluster corresponding to the child node is the subcluster of the cluster corresponding to the parent node. Finally, the clustering result is obtained.

[0071] The BIRCH algorithm can build a CF Tree by just scanning the training set once. It has the characteristics of small space occupation, fast clustering speed, noise point identification and preprocessing. However, since the CF Tree has a limit on the number of CFs for each node, the clustering result may be different from the actual category distribution. It is not effective in clustering data with high-dimensional features. If the distribution cluster of the data set is not similar to a hypersphere, the clustering effect is not good. The BIRCH clustering algorithm process is as follows:

[0072] The first step is to build the CF Tree, read the data points, and update the CF Tree. If the new data point can be merged into the existing cluster without violating the definition of the tree, then merge it; otherwise, create a new leaf node.

[0073] In the second step, the CF Tree established for the first time is screened to remove abnormal CF nodes.

[0074] The third step is to use other clustering algorithms (such as K-Means) to cluster all CF tuples to obtain a better CF Tree.

[0075] In the fourth step, the centroid of all CF nodes of the CF Tree generated in the third step is used as the initial centroid point to cluster all sample points according to distance.

[0076] (3) Gaussian Mixture Model (GMM)

[0077] The Gaussian Mixture Model (GMM) has something in common with the K-means algorithm, both of which require the K value to be specified. The difference between it and K-means is that K-means can only assign each point to one center, and only one. Its limitation is that it does not provide a measure of the probability or possibility of a data point to the center of other clusters. GMM can give the probability that a sample corresponds to all categories. GMM is an algorithm that combines Gaussian distribution with a mixture model. It is a non-spherical cluster. The parameter mean and covariance in the Gaussian distribution can be regarded as describing the center and flatness of this non-spherical cluster. The focus of GMM is how to determine the parameters of GMM. The expectation maximization (EM) is used to iteratively solve the parameters. GMM works better in processing multidimensional data and data of different dimensions because there are two values, standard deviation and variance, which makes the model more accurate. However, the model needs to be iterated continuously, so the computational complexity is high.

[0078] The process of EM clustering method using GMM is as follows:

[0079] The first step is to build a Gaussian model. Assume that the data is composed of several Gaussian distributions. Each Gaussian distribution is a probability distribution function corresponding to one parameter. Therefore, the Gaussian model can be used to describe the distribution of the data.

[0080] The second step is to determine the parameters of each Gaussian distribution. The mean and variance of each Gaussian distribution are determined, and these parameters can be used to calculate the probability distribution of each data point.

[0081] The third step is to determine which Gaussian distribution each data point belongs to. The maximum likelihood method is used to determine which Gaussian distribution each data point belongs to.

[0082] The fourth step is to use the expectation maximization (EM) algorithm to solve the parameters of GMM. In the EM algorithm, it is necessary to iterate continuously until the parameters of the model converge.

[0083] The fifth step is to calculate the responsivity of all data points. The responsivity represents the probability that a data point belongs to each Gaussian distribution and can be calculated using the probability distribution function in the Gaussian model.

[0084] Step 6: Update the parameters of each Gaussian distribution. Update the mean and variance of each Gaussian distribution, using the responsiveness of all data points to calculate.

[0085] The seventh step is to calculate the likelihood function of the model. The likelihood function indicates the fit of the model to the data, and is used to determine whether the model parameters have converged. The model parameters are continuously updated iteratively until the model parameters converge.

[0086] (4) Spectral Clustering

[0087] Spectral Clustering is a clustering method based on graph evolution, that is, each data point is regarded as a vertex on the graph. Therefore, the basis for spectral clustering to determine whether two points belong to the same cluster is to determine whether the two points are connected by an edge on the graph, which can be directly or indirectly connected. The edge weight value between two points that are far away is low, while the edge weight value between two points that are close is high. By cutting the graph composed of all data points, the sum of the edge weights between different subgraphs after cutting is as low as possible, and the sum of the edge weights within the subgraph is as high as possible, so as to achieve the purpose of clustering. Therefore, spectral clustering is essentially a problem of cutting the graph. Spectral clustering mainly relies on the similarity matrix constructed between data, so it is very effective in processing sparse matrices. However, because spectral clustering relies too much on the similarity matrix, the final effects obtained by different similarity matrices are also very different. Therefore, the low-dimensional level is more conducive to matrix operations. Spectral clustering also uses dimensionality reduction methods, which is faster and more convenient when clustering high-dimensional data. If the dimension of the final clustering is very high, the running speed of spectral clustering and the final clustering effect will be greatly reduced due to insufficient dimensionality reduction.

[0088] The clustering process of spectral clustering can be roughly divided into two steps, namely, graph construction and graph cutting. The first step is graph construction. The sampling point data is constructed into a network graph, represented as G(V,E), where V represents the points in the graph and E represents the edges between the points. Figure 2 The second step is to cut the graph, that is, to cut the graph constructed in the first step into different graphs according to certain cutting criteria, and different subgraphs are clustering results. Figure 2 shown.

[0089] Spectral clustering algorithm process:

[0090] The first step is to construct the sample similarity matrix S according to the generation method of the input similarity matrix.

[0091] The second step is to construct the adjacency matrix W and the degree matrix D based on the similarity matrix S.

[0092] The third step is to calculate the Laplace matrix L, L = DW.

[0093] The fourth step is to construct the standardized Laplace matrix.

[0094] Step 5: Calculate The eigenvector f corresponding to each of the smallest k1 eigenvalues.

[0095] The sixth step is to normalize the matrix composed of the corresponding eigenvectors f by row, and finally form an n×k1-dimensional feature matrix F.

[0096] In the seventh step, each row in F is treated as a k1-dimensional sample, with a total of n samples, and clustered using the input clustering method with a clustering dimension of k2.

[0097] Step 8: Get the clustering partition C (c1, c2, ..., c k2 ).

[0098] (5) K-medoids clustering

[0099] The K-medoids clustering algorithm is similar to the K-means algorithm, both of which use distance as the criterion for similarity. K-means uses the average value of each cluster as the new centroid, and iterates repeatedly until the distribution of objects in the cluster no longer changes, which can easily make the algorithm sensitive to extreme values. The K-medoids clustering algorithm no longer selects the average value, but instead uses the object at the center of the cluster as the reference point. Therefore, the K-medoids clustering algorithm is fast and has a fixed number of convergence times. However, the first center point selection of the algorithm is randomly selected based on the actual data points, so there may be errors in each clustering result.

[0100] K-medoids clustering algorithm process:

[0101] The first step is to input the expected number of clusters K. Randomly select k points from n data objects as the initial center set.

[0102] In the second step, the distance between each non-central object and each central point is calculated and assigned to the cluster with a closer distance.

[0103] In the third step, for each non-central object, replace one of the centers with the current point in turn, and calculate the cost function generated by the replacement. If it is a negative value, replace it, otherwise do not replace it and restore the center point.

[0104] The fourth step is to obtain a final set of k optimal center points, and re-divide all objects into the cluster closest to them according to the minimum distance principle.

[0105] Evaluation of clustering results based on silhouette coefficient. Clustering algorithm is an unsupervised learning method without any prior knowledge and knows nothing about the given data structure. Therefore, no matter which clustering algorithm is used, the rationality of its results requires subsequent rigorous evaluation. In practice, the silhouette coefficient is usually used to quantify the cohesion and separation of clustering results, so as to achieve a scientific evaluation of the clustering effect. Therefore, this method introduces the silhouette coefficient to evaluate the clustering results. For a certain clustering of a data set, its silhouette coefficient S i The definition is as follows:

[0106]

[0107] Among them, i represents a sample to be evaluated, and n represents the number of samples to be evaluated.

[0108] a i It represents the average distance between sample i and other samples belonging to the same cluster, and measures the compactness within the cluster where sample i is located. i The smaller it is, the closer sample i is to other samples in the same cluster.

[0109] b i It represents the minimum value of the average distance between sample i and other clusters, and measures the degree of separation between sample i and other clusters. i The larger it is, the more separated sample i is from other clusters.

[0110] S i Represents the silhouette coefficient of sample i, which is used to comprehensively evaluate the separation and cohesion of sample i. i The value range of S is in [1,-1]. i When the value of is close to 1, it indicates that the clustering result of sample i has a high accuracy; if S i The value of is close to -1, indicating that sample i should be classified into other clusters; if S i When the value of is approximately 0, it means that the sample is on the boundary of two clusters.

[0111] S k is the average silhouette coefficient, usually S k Perform clustering validity analysis.

[0112] The cohesion and separation of clustering results are verified by the silhouette coefficient. The high or low silhouette coefficient indicates the rationality of the clustering results. However, there is still uncertainty in the accurate classification of cultivated land. In this regard, the Bhattacharyya distance combined with variance analysis is used to verify the credibility and separability of clustering.

[0113] ANOVA is a statistical analysis method used to check whether there are differences between experimental results, that is, to test whether there are differences between and within groups. After obtaining clusters, variance analysis is used to compare the differences between different categories to verify the credibility of clustering results. This paper verifies the correlation between yield and other indicators through the Pearson correlation coefficient, which is significantly correlated. Therefore, the cultivated land is graded by obtaining the average yield.

[0114] First, the Bhattacharyya distance is used to verify the separability of clusters. A cluster is randomly selected and used as the center point. The Bhattacharyya distance is used to calculate the similarity between the cluster and other clusters, and the clusters are arranged from near to far. In this experimental data, each value in the cluster is a discrete value. Therefore, the Bhattacharyya distance for discrete distribution can be defined as:

[0115] D b (p,q)=-ln(BC(p,q)) (8)

[0116] Where p and q represent two discrete values ​​in the same domain X. BC(p,q) is the Bhattacharyya coefficient, which is an approximate measure of the overlap between two statistical samples and can be used to determine the relative closeness of the two samples under consideration. The Bhattacharyya coefficient can be defined under discrete distribution as:

[0117]

[0118] Secondly, the reliability of clustering was verified by analysis of variance (ANOVA). ANOVA was performed on adjacent clusters. If the P-vluae between the two groups was <0.05 or <0.01, it indicated that there were significant differences within and between clusters, indicating that the clustering effect was good. Otherwise, they should be merged. After completing the pairwise ANOVA, all clusters were further analyzed for variance, and the average yield of each cluster was obtained. The cultivated land was classified according to the average yield. The credibility and separability test of the clustering results were completed by using the Bhattacharyya distance and variance analysis, indicating that the clustering results were acceptable.

[0119] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0120] The above embodiments are merely examples of the present invention and do not limit the protection scope of the present invention. All designs that are the same or similar to the present invention fall within the protection scope of the present invention.

Claims

1. A method for evaluating the quality of black soil arable land, characterized in that: The following steps are involved: Obtain black soil arable land quality data, perform preprocessing and data dimension reduction, and obtain low-dimensional black soil arable land quality data after preprocessing; A variety of unsupervised clustering algorithms are used to cluster low-dimensional black soil arable land quality data to obtain a variety of clustering results, and the silhouette coefficient is introduced to evaluate the various clustering results to obtain the best clustering result. Obtain the Bhattacharyya distance of each cluster in the best clustering result, and perform credibility and separability tests on the cluster of the best clustering result by combining Bhattacharyya distance with variance analysis, and merge the clusters that are not separable until multiple clusters with the best separability and credibility are obtained; The average yield of multiple clusters is obtained, and the black soil arable land is graded according to the average yield and the set grade standard to obtain the black soil arable land classification result, that is, to generate the arable land quality evaluation result.

2. The method for evaluating the quality of black soil arable land according to claim 1, characterized in that: The method of obtaining the pre-processed low-dimensional black soil arable land quality data comprises the following steps: The quality data of black soil arable land is proofread by the nearest value assignment method to obtain the proofread quality data of black soil arable land; The membership value is used to quantify the quality data of black soil cultivated land after calibration, and the quantified quality data of black soil cultivated land is obtained; The quantified black soil arable land quality data were checked for missing data, and the missing data were filled by the mode interpolation method to obtain the standardized black soil arable land quality data; The standardized black soil arable land quality data were processed by t-SNE data dimensionality reduction method to obtain low-dimensional black soil arable land quality data.

3. The method for evaluating the quality of black soil arable land according to claim 2, characterized in that: The method of obtaining low-dimensional black soil arable land quality data comprises the following steps: The similarity between the standardized black soil arable land quality data points in high-dimensional space is calculated through Gaussian distribution, the similarity between the standardized black soil arable land quality data points in high-dimensional space is quantified, and a symmetric similarity matrix is ​​constructed; Randomly initialize the standardized black soil arable land quality data points in low-dimensional space; Taking the symmetric similarity matrix as the optimization target, the similarity of the standardized black soil arable land quality data points in low-dimensional space was calculated using t distribution. The position of the midpoint in the low-dimensional space is optimized by KL divergence. Combined with the symmetric similarity matrix, the gradient descent method is used to minimize the KL divergence, update the position of the midpoint in the low-dimensional space, and obtain the low-dimensional black soil arable land quality data.

4. The method for evaluating the quality of black soil arable land according to claim 1, characterized in that: The multiple clustering algorithms include: K-means, BIRCH, Gaussian mixture model GMM, spectral clustering and K-medoids.

5. The method for evaluating the quality of black soil arable land according to claim 1, characterized in that: The reliability and separability test of the clusters of the optimal clustering results by combining Bhattacharyya distance with variance analysis includes the following steps: Randomly select a cluster from the best clustering results, take it as the center point, use Bhattacharyya distance to calculate the similarity between this cluster and other clusters, and arrange the clusters from near to far; Perform variance analysis on adjacent clusters, merge two clusters with P values ​​less than the set value to obtain new clusters; After completing the pairwise ANOVA, further ANOVA is performed on all clusters, and the average yield of each cluster is obtained. The cultivated land is classified and graded according to the average yield and the set grade standards.

6. The method for evaluating the quality of black soil arable land according to claim 1, characterized in that: The black soil arable land quality data includes 16 evaluation indicators and yield attribute data.

7. A system for evaluating the quality of black soil arable land, characterized in that: include: A data processing module is used to obtain black soil arable land quality data, perform preprocessing and data dimension reduction, and obtain low-dimensional black soil arable land quality data after preprocessing; The data clustering analysis module uses a variety of unsupervised clustering algorithms to cluster low-dimensional black soil arable land quality data, obtains a variety of clustering results, and introduces the silhouette coefficient to evaluate the various clustering results to obtain the best clustering result; Obtain the Bhattacharyya distance of each cluster in the best clustering result, and perform credibility and separability tests on the cluster of the best clustering result by combining Bhattacharyya distance with variance analysis, and merge the clusters that are not separable until multiple clusters with the best separability and credibility are obtained; The evaluation and analysis module is used to obtain the average yield of multiple clusters, classify the black soil arable land according to the average yield, obtain the black soil arable land classification results, and generate the arable land quality evaluation results.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images