Remote sensing interpretation data post-processing method based on clustering and correlation analysis

Abstract

The application relates to a remote sensing interpretation data post-processing method based on clustering and correlation analysis, relates to the field of data processing, and solves the problem that existing image parameters and other dimensional historical data are not fully utilized, resulting in deviation between remote sensing interpretation data and real object statistical data, etc.The method comprises the following steps: obtaining remote sensing interpretation data and real statistical data, fusing the remote sensing interpretation data and the real statistical data, and calculating the difference between the remote sensing interpretation data and the real data; obtaining a fusion vector; performing error clustering analysis, remote sensing image parameter and error category correlation analysis, and establishing an error correction model through classification, etc.The method has the characteristics of low CPU resource occupation, small memory burden and high time efficiency, and fully utilizes historical data and shooting parameters, can greatly improve the accuracy of remote sensing interpretation data in some fields, and can be used in series with an error correction method based on imageology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a remote sensing interpretation data post-processing method based on clustering and association analysis. Through the method, multi-period remote sensing interpretation data from multiple satellite models can be reasonably classified, error correction models can be respectively constructed, data errors can be reduced, and data usability and the accuracy of prediction models can be improved. BACKGROUND

[0002] In the optimization strategy of remote sensing interpretation data, the existing technology mainly uses remote sensing image preprocessing methods or optimizes remote sensing interpretation models, including geometric correction, image finishing, dehazing, super-resolution, increasing training data, using complex neural network models, etc. The existing methods for optimizing remote sensing interpretation data belong to the optimization of pre-processing technology and the interpretation model itself. The optimization ability of this method is close to the bottleneck, and the image parameters and other dimensional historical data are not fully utilized. The post-processing method of clustering and association analysis is adopted in the present application. The image parameters are classified first, then the categories and error types are associated, and the multi-dimensional data is used to fit and correct the interpretation data. The post-processing method provides a new strategy for the optimization of interpretation data.

[0003] In data processing, the process of dividing a data set based on data similarity, i.e. dividing similar data sets into the same category, is called clustering. In the cluster generated by clustering, the data objects in the same cluster are similar to each other and different from the objects in other clusters.

[0004] The association between the occurrence of a certain transaction and the occurrence of other transactions is called association. Association analysis is the process of mining the association rules and laws of object sets in information carriers. Association rules refer to implications such as X=>Y, where X and Y are called the antecedent and consequent of the association rule, respectively. The implication is that the occurrence of X items is associated with the occurrence of Y items. The significance of association rule mining for remote sensing interpretation data optimization is that it can quantitatively analyze the relationship between object clusters after clustering analysis and errors, determine transactions with clear association rules, establish error correction models based on different transactions, and improve the accuracy of error correction and the robustness of the system.

[0005] Improving the alignment between remote sensing interpretation data and ground feature statistics is a crucial requirement for remote sensing services in agriculture, finance, and statistics. Traditional methods primarily rely on image preprocessing and optimization strategies based on the interpretation model itself. This invention employs a post-processing approach, proposing the integration of image acquisition parameters (including satellite model and side-swing angle) with historical ground feature data for post-processing optimization of interpretation data. First, interpretation errors are clustered and categorized into multiple levels, such as: too high, reasonable, and too low. Then, transaction vectors are constructed using cluster labels and image parameters. Association analysis is used to quantitatively select association rules that reach the lift threshold, i.e., the relationship between "satellite model and side-swing angle" and "interpretation error type." Regression analysis models are established for transaction vectors under different association rules, enabling targeted optimization of interpretation data under different scenarios. This represents a novel method for optimizing remote sensing interpretation data. Summary of the Invention

[0006] This invention addresses the problem that existing methods for preprocessing remote sensing images, which rely on optimizing the preprocessing methods and interpretation models, fail to fully utilize historical data from image parameters and other dimensions, leading to discrepancies between remote sensing interpretation data and actual ground feature statistics. The invention provides a post-processing method for remote sensing interpretation data based on clustering and association analysis to reduce this discrepancy.

[0007] A post-processing method for remote sensing interpretation data based on clustering and association analysis, which is implemented by the following steps:

[0008] Step 1: Obtain remote sensing interpretation data and actual statistical data, fuse the remote sensing interpretation data and actual statistical data, and calculate the difference between the remote sensing interpretation data and the actual data; obtain the fusion vector C;

[0009] Step 2: Error clustering analysis;

[0010] The K-Means algorithm is used to perform vector point clustering analysis to divide the dataset;

[0011] Step 3: Correlation analysis between remote sensing image parameters and error categories;

[0012] By introducing imaging parameters into the fusion dataset of interpreted and statistical data, an association analysis model is used to explore whether there are association rules between the dataset labels and the remote sensing data imaging parameters;

[0013] Step 4: Establish error correction models by category;

[0014] For rules that elevation index reaches a threshold, correlation rules are established. Regression analysis is performed on the fused dataset under these rules to establish error correction models. Error correction models under each correlation rule are used to correct errors in remote sensing image interpretation data with different shooting parameters.

[0015] The method of the present application uses remote sensing interpretation data combined with multi-dimensional data analysis strategy, uses clustering and correlation analysis on image parameters and errors, respectively, error regression modeling of different types of remote sensing interpretation data and true data, and constructs a targeted error correction model.

[0016] In the present application, by combining remote sensing interpretation data with true statistical data, the interpretation error is classified and evaluated, and a clustering algorithm is used for statistical classification.

[0017] Using a series combination, the classification results of the error are used as the input items of the correlation analysis, and the original image parameters and errors of the interpretation data are analyzed for correlation, excluding transaction models without obvious correlation, and preventing overfitting of error optimization.

[0018] Quantitative determination of the relationship between image parameters and errors, respectively, for each case to establish a regression correction model, with good optimization effect and robustness.

[0019] Compared with the error correction strategy based on imaging, the present method has the characteristics of less CPU resource occupation, small memory burden and high time efficiency, and makes full use of historical data and shooting parameters, which can greatly improve the accuracy of remote sensing interpretation data in some fields, and can be used in series with the error correction method based on imaging. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The flowchart of the remote sensing interpretation data post-processing method based on clustering and correlation analysis described in the present application;

[0021] Figure 2 The flowchart of the clustering analysis in the present application method. DETAILED DESCRIPTION

[0022] The technical solutions of the present application will be described clearly and completely below, and the described content will be illustrated with specific examples. The examples in the description are a specific example of the technical solutions described in the present application, and other personnel in the art can obtain all other examples without creative labor, and of course, other examples obtained based on the present method belong to the scope of protection of the present application.

[0023] In combination Figure 1 And Figure 2 To illustrate the present embodiment, the remote sensing interpretation data post-processing method based on clustering and correlation analysis is realized by the following steps:

[0024] Step 1: Fusion of remote sensing interpretation data and true statistical data;

[0025] First, define the remote sensing interpretation data object as R(d r ,ir ,pr1,pr2,pr3,......prn), where R contains the image capture date d. r Remote sensing interpreted ground feature scalar data i r , where i r The physical meaning varies depending on the scenario, such as: the inventory of crude oil in a storage area, the area of ​​a construction site, or the area of ​​a certain crop; it also includes the raw image parameters p from remote sensing interpretation data. r1 -p rn Examples include: lateral tilt angle, cloud cover, and satellite model.

[0026] Define a real statistical data object S(d) s i s S contains statistical data on real land features. s and date d s Data is generally obtained from the National Bureau of Statistics, industry associations, and local survey agencies.

[0027] The remote sensing interpretation data object R and the statistical data object S are set to have the same date, i.e., d r =d s This constitutes the fusion vector C(d,i) r i s ,r,pr1,p r2 ,p r3 ,......prn), where d=d r =d s r is the difference between remote sensing interpreted data and statistical data, i.e., r = i r -i s .

[0028] It should be noted that for cases where the lengths of objects R and S do not match or the dates do not correspond, methods such as multinomial fitting and SVM models can be used to fill in the missing values ​​and achieve the requirements of length matching and date correspondence.

[0029] To enhance the clarity of the description of the specific implementation methods, a case study of remote sensing interpretation and analysis of crude oil storage capacity in Cushing is used as an illustration:

[0030] Suppose that 100 sets of interpretation data are interpreted from the remote sensing image of the Cushing area in 100 periods, forming the interpretation object R = [(‘2019 / 1 / 1’, 5000, 20, 10, ‘JL01A’), (‘2019 / 1 / 11’, 5600, 2, 20, ‘JL01B’), …… (‘2021 / 11 / 1’, 5000, 20, 30, ‘JL01A’)], wherein each field in each row of data respectively represents: date, interpreted oil storage (unit: ten thousand barrels), satellite side swing angle (unit: degree), cloud cover percentage and satellite model. The corresponding real statistical data S in this period is [(‘2019 / 1 / 1’, 5300), (‘2019 / 1 / 10’, 5100) …… (‘2021 / 11 / 10’, 5100)].

[0031] For the case that the data length in R and S does not correspond or part of the date does not match, polynomial fitting is used to fill in the intermediate empty data, so as to meet the requirements of length matching and date correspondence.

[0032] The matched R and S form the fusion vector C = [(‘2019 / 1 / 1’, 5000, 5300, -300, 20, 10, ‘JL01A’), (‘2019 / 1 / 11’, 5600, 5100, 500, 2, 20, ‘JL01B’), …… (‘2021 / 11 / 1’, 5000, 5100, -100, 20, 30, ‘JL01A’)], wherein the 7 dimensions in C correspond to d, i r ,i s ,r,p r1 ,p r2 ,p r3 in the above formula, respectively representing: date, interpreted data, statistical data, error, side swing angle, cloud cover and satellite model.

[0033] Step two: error clustering analysis

[0034] For the fusion vector C generated in the previous step, take its error dimension data C’ = C(r) to do clustering analysis.

[0035] The clustering algorithm is generally for two-dimensional vectors, so C’(r) is expanded to increase the constant term 0, that is, it is expanded to C’(r, c), wherein C’(c) = [0, 0, 0, ……, 0].

[0036] Use the K-Means algorithm to divide C’(r, c), and the division principle is:

[0037] According to the distance between samples, divide the sample set into K clusters. Make the points in the cluster as close as possible, and make the distance between clusters as large as possible.

[0038] Generally, the principle of minimizing the square error E is adopted to divide the data set into clusters (C1, C2,...C k ):

[0039]

[0040] where u i is the mean vector of cluster C i , and the expression is:

[0041]

[0042] A point is randomly selected from the input data point set as the first cluster center μ1, and then a heuristic iterative method is used to divide all the fusion vector data. The classified cluster identification is added to the fusion vector C, i.e. C = (d, i r ,i s ,r,p r1 ,p r2 ,p r3 ...p rn ,t), where t is the cluster identification of the cluster division.

[0043] In the scenario of measuring the reserves of crude oil in the Kuznetsk region, t is divided into three categories: t1 (the mean error is obviously negative, i.e. the scalar value of the interpreted data is obviously smaller than the true value), t2 (the mean error tends to 0, i.e. the scalar value of the interpreted data is close to the true value), and t3 (the mean error is obviously positive, i.e. the scalar value of the interpreted data is obviously larger than the true value).

[0044] Step three: association analysis of remote sensing image parameters and error categories

[0045] Next, the image parameter items p r1 -p rn and the cluster identification items t in C are taken out to form the input items of the association analysis: the transaction vector T = (p r1 ,p r2 ,p r3 ...p rn ,t), which means whether the occurrence of the item set p r1 ,p r2 ,p r3 ...p rn is associated with the cluster category t to which the error item belongs. A quantitative evaluation method is used to exclude weakly associated rules and construct error correction models for strongly associated rules.

[0046] In the scenario of measuring the reserves of crude oil in the Kuznetsk region, the core of the association algorithm is a recursive algorithm based on the two-stage frequent item set idea, which can mine rules and calculate quantitative information such as the support and lift of each rule.

[0047] The key concepts include:

[0048] Association rule: implication expression in the form of A=>B, where A and B are item sets that are not comparable;

[0049] Support: the ratio of the number of transactions containing a specific item set to the total number of transactions. The support of association rule A=>B can be identified as:

[0050]

[0051] In the formula, σ(A=>B) represents the number of transactions in the association rule A=>B, and ||D|| represents the total number of all transactions;

[0052] Lift: the lift of association rule A=>B, which represents the degree to which the occurrence of item set A affects the probability of the occurrence of item set B, and can be represented as:

[0053]

[0054] In the calculation scenario of the oil reserves in the Khushen area, for the association rule A=>B, A is p r1 , p r2 , p r3 , that is, various combinations of image shooting parameters; B is t, that is, error type. Using the association algorithm, the association rule table is as follows:

[0055]

[0056]

[0057] The first row in the table means: for the image with the parameter set {side swing angle > 20 degrees, cloud cover < 10 degrees, satellite model 'JL01A'}, the interpretation data will result in a significant negative mean error, that is, the interpretation data scalar value is significantly smaller than the true value. The support of this rule is 0.2, and the lift is 1.6.

[0058] Generally, rules with support > 0.1 and lift > 1.5 are selected as strong association rules. For the first two rows and the last row in the above table: {‘>20’, ‘<10’, ‘JL101A’}=>{t1}, {”JL1KF01A”}=>{t2} meet the requirements of strong association rules and can be used as classification basis to establish corresponding error correction models.

[0059] Step four: classification and establishment of error correction model

[0060] For the transaction set that meets the requirements of strong association rules, error correction models are constructed according to error categories t.

[0061] As in the scenario of the oil reserves estimation in the Cushing area, for the error category t1, there is a transaction set: {‘>20’, ‘<10’, ‘JL101A’}={t1}, {‘JL1GF105B’}={t1}, {‘>10”, ‘JL1GF02B’}={t1}, which means that for the image data of the following shooting parameters: {‘>20’, ‘<10’, ‘JL101A’}, {‘JL1GF105B’}, {‘>10”, ‘JL1GF02B’}, the interpretation data is obviously less than the true value, the error parameter mean is negative, and after correlation analysis, this clustering method has statistical significance.

[0062] Therefore, the image interpretation data set under this parameter is modeled separately, and the error linear regression equation is constructed:

[0063] ε t1 = α t1 + β t1 i r

[0064] In the above formula, ε t1 is the estimated error, i r is the current interpreted oil reserves, and parameters α t1 and β t1 can be obtained using the least squares estimation method, and of course other parameter estimation methods can also be used to obtain the regression parameters.

[0065] The corrected interpretation data value is:

[0066] i revise = i r - ε t1

[0067] or

[0068] i revise = (1- β t1 ) i r - α t1

[0069] Thus, the corrected data under the error classification t1 data set can be obtained, and the corrected data under other classifications can be obtained in the same way, achieving the purpose of error correction of remote sensing interpretation data.

[0070] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0071] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A remote sensing interpretation data post-processing method based on clustering and correlation analysis, characterized in that: The method is realized by the following steps: Step one, obtaining remote sensing interpretation data and real statistical data, fusing the remote sensing interpretation data and real statistical data, and calculating the difference between the remote sensing interpretation data and the real data; Obtain the fusion vector C; The specific process of step one is: Define the remote sensing interpretation data object as R(d) r i r ,p r1 ,p r2 ,p r3 ,......p rn ), d r For the date the image was taken, i r p is the scalar data of ground features interpreted by remote sensing. r1 -p rn The raw shooting parameters for remote sensing interpretation data; Define the actual statistical data object S(d) s i s ), i s Statistical data of real land features d s The statistical date for actual geographical features; The remote sensing interpretation data object R and the statistical data object S are taken as the condition of the same date, i.e. d r =d s , to constitute a fusion vector C(d, i r ,i s ,r , p r1 ,p r2 ,p r3 ,......p rn ), wherein d=d r =d s , and r is the difference between the remote sensing interpretation data and the statistical data, i.e. r=i r -i s ; Step two, error clustering analysis; The K-Means algorithm is used to cluster the error dimension data C of the fusion vector C ’ =C(r) to divide the data set, and the classified cluster identification is added to the fusion vector C, that is, C= (d, i r ,i s ,r,p r1 ,p r2 ,p r3 ...p rn ,t), wherein t is the cluster identification of the cluster division. Step three, remote sensing image parameter and error category association analysis; Using the association analysis model to mine whether the data set label and the remote sensing data shooting parameter have an association rule; Step four, classification to establish an error correction model; For the rules whose promotion index reaches the threshold value, the association rules are established, the fusion data set under the rules is subjected to regression analysis, and error correction models are respectively established; the error correction models under each association rule are used to correct the error of the remote sensing image interpretation data of different shooting parameters.

2. The method of claim 1, wherein the method comprises: For the object R and the object S, if the length is not matched or the date is not corresponding, then the polynomial fitting or SVM model method is used to fill in the intermediate null value, so as to meet the requirements of length matching and date correspondence.

3. The method of claim 1, wherein the method further comprises: The specific process of step two is: To the fusion vector C generated in step one, take its pair error dimension data C ’ = C(r) to do clustering analysis, because the clustering algorithm is generally for two-dimensional vector, to C ’ (r) expansion, increase the constant term 0, that is, expand to C ’ (r, c), where C ’ (c) = [0, 0, 0, …, 0]; C is divided using K-Means algorithm ’ (r,c) is divided according to the principle that According to the distance between samples, the sample set is divided into k clusters; The data set is divided into clusters (C1, C2,... Cn) using the principle of minimizing the square error E: k ): ； wherein is the mean vector of the cluster C i is the mean vector of the cluster C ； A point is randomly selected from the input data point set as the first cluster center μ1, and then a heuristic iterative method is used to divide all the fusion vector data.

4. The method of claim 1, wherein the method further comprises: In step three, different shooting parameter items p r1 -p rn and cluster identification items t in the fusion vector C are taken out to form the input items of the correlation analysis: transaction vector T=(p r1 ,p r2 ,p r3 ...p rn ,t), which means whether the occurrence of the item set p r1 ,p r2 ,p r3 ...p rn is related to the cluster class t to which the error item belongs. Using a quantitative evaluation method, the rules of weak correlation are excluded, and the error correction model of strong correlation is constructed respectively.

5. The method of claim 4, wherein the method further comprises: Select the rules with support>0.1 and promotion>1.5 as strong association rules.

6. The method of claim 5, wherein the method further comprises: For the transaction set meeting the strong association rule requirement, error correction models are respectively constructed according to the error category t, that is, regression analysis is performed on the fusion data set under the strong association rule, error correction models are respectively established, and finally the correction data under different error classifications are obtained.

Images