A method for rapid detection and evaluation of defects in land use sample sets

Abstract

The present invention provides a method for quickly discovering and evaluating defects in a land use sample set, including a sample design defect discovery part and a sample production defect discovery part; the sample design defect discovery part includes: inputting the data in the sample set into a semantic proximity defect detection model, obtaining a semantic proximity distribution map, and finding the semantic defect category; the sample production defect discovery part: inputting sample data of the semantic defect category into a sample production defect detection model, the sample production defect detection model is obtained by an improved PBLC model, converting the average probability value Cp into an equivalent positive sample labeled ratio Cn; according to the obtained semantic proximity distribution map and the positive sample labeled ratio, the semantic design defects and the in-class sample defects and the out-of-class sample defects are obtained. The beneficial effect of the present invention is that the defect category and the number of defects in the sample set can be accurately obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of land use samples, and in particular to a method for quickly discovering and evaluating defects in a land use sample set. Background Art

[0002] The core of land use sample system design is the definition of category semantics. The lower the similarity of semantic features, the more reasonable the system design. At present, there has been a mature study on semantic proximity in China. Jia Xiaobin et al., based on the semantic features of land use images, constructed a multi-detail hierarchical expression model of semantic features of land use images based on the content of "Land Use Status Classification" and the ownership characteristics of land use types. Then, according to the idea of ​​feature matching, the matching relationship of each category of images in 11 aspects such as ownership, cover cause, cover type, purpose, utilization status, vegetation type, attachment properties, and utilization method was calculated to obtain the specific relationship value of semantic proximity of land use images, realizing the whole process of land use images from semantic feature modeling to semantic proximity calculation; Luo Fang et al. proposed a method for triple semantic accuracy quantitative evaluation. First, the semantic hierarchy tree and semantic distance were used to establish a semantic membership calculation model, and then the accuracy evaluation model was established from the local to the whole, from the single map, the single coverage type and the whole map. These methods are of great help in the design of the type system of sample sets, but the current design is for manual land management and is not entirely suitable for the construction of land use sample sets. This is because people's understanding of utilization is much higher than that of machines, and machines require a more specific and detailed classification system. Therefore, the methods of predecessors can be used to quantitatively evaluate the design quality of the existing system samples of the sample set by analyzing the connotation distance between different types of samples.

[0003] Foreign scholars have conducted in-depth research on the impact of sample errors on deep learning benchmarks and believe that defect errors have little impact on training, but have a greater impact on evaluation and improvement of deep learning methods. Northcutt et al. [3] found that machine learning test sets do contain errors and that these errors can undermine the stability of machine learning benchmarks, because researchers rely on benchmark test data sets to evaluate and measure the latest progress and verify theoretical findings. If labeling errors occur in large numbers, it may undermine the framework for measuring machine learning progress and cause researchers to draw incorrect conclusions about which models perform best in practice. However, this study did not involve the research and evaluation of sample defect detection methods, and foreign studies did not explore in depth the discovery and causes of errors in large land use sample sets widely used in the remote sensing field. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for quickly discovering and evaluating defects in a land use sample set, which mainly includes: a sample design defect discovery part and a sample production defect discovery part;

[0005] The sample design defect discovery section includes:

[0006] Input the data in the sample set into the semantic proximity defect detection model to obtain the semantic proximity distribution map and semantic defect categories;

[0007] Sample production defect discovery section:

[0008] The semantic defect category data is input into the sample production defect detection model, which is obtained by the improved PBLC model, and the average probability value Cp is converted into an equivalent positive sample labeled proportion Cn;

[0009] According to the obtained semantic proximity distribution map and the proportion of positive samples marked, semantic design defects, in-class sample defects, and out-of-class sample defects are obtained, that is, defects are discovered. In-class sample defects (InError) are the number of erroneous samples from other classes mixed into this class; out-of-class sample defects (OutError) are the number of erroneous samples from this class mixed into other classes. When the sample size is large enough, it can be considered that the in-class sample defects are equal to the out-of-class sample defects. After obtaining the semantic design defects, the defect category is obtained. After the defects of the entire sample set are determined to a specific category, the sum of the in-class sample defects and out-of-class sample defects is the specific number of defects in this sample set.

[0010] Furthermore, the semantic proximity defect detection model includes a multi-detailed hierarchical expression model of land use data semantics, a semantic feature expression module and a semantic proximity calculation module. The data in the sample set is first input into the multi-detailed hierarchical expression model of land use data semantics, and the semantic feature items are described for each category in the sample set. Then the data enters the semantic feature expression module, and finally enters the semantic proximity calculation module to finally obtain a semantic proximity distribution map.

[0011] Furthermore, the semantic proximity S(a, b) between categories is calculated according to formula (1), and a semantic proximity distribution map is obtained by calculating multiple semantic proximities, thereby finding the easily confused categories of design defects;

[0012]

[0013] Among them, G() is a density function used to characterize the hierarchical set of semantic feature items; parameters α and β are correction coefficients, and A and B represent two different categories.

[0014] Furthermore, the sample production defect detection model includes a module for estimating the probability of positive sample being marked (i.e., the traditional PBLC model) and a probability frequency graph module. The data in the sample set is first input into the module for estimating the probability of positive sample being marked to obtain the estimated probability of positive sample being marked, and then enters the marked positive sample module and the probability frequency graph module to finally obtain the positive sample marked proportion Cn.

[0015] Furthermore, the calculation formula of the positive sample labeled ratio Cn is as follows:

[0016]

[0017] Among them, g(x) is the probability of positive samples, f(x) is the probability of sample labeling, L is the labeled positive sample identified by the classifier, P is the positive sample identified by the classifier, and T is the total number of positive samples.

[0018] Furthermore, this method can also obtain the number of defects LabelError through the following formula:

[0019]

[0020] Among them, Cn represents the proportion of positive samples that are labeled, TrueN represents the set of labeled positive samples, InError represents the sample defect within the class, and OutError represents the sample defect outside the class.

[0021] The beneficial effect of the technical solution provided by the present invention is that the defect category and defect number of the sample set can be accurately obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0023] Figure 1 The present invention is a flowchart of a method for quickly discovering and evaluating defects in a land use sample set in an embodiment of the present invention.

[0024] Figure 2 It is a schematic diagram of a specific process of designing defects of a land use sample set in an embodiment of the present invention.

[0025] Figure 3 Schematic diagram of a multi-level-of-detail expression model of land use data semantics in an embodiment of the present invention.

[0026] Figure 4 It is a semantic proximity distribution diagram obtained by taking the AID sample set as an example in the embodiment of the present invention.

[0027] Figure 5 It is a flow chart of the PBLC improved model in an embodiment of the present invention.

[0028] Figure 61 is a partial frequency graph obtained by taking the AID sample set as an example in the embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0030] An embodiment of the present invention provides a method for quickly discovering and evaluating defects in a land use sample set.

[0031] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for quickly discovering and evaluating defects in a land use sample set in an embodiment of the present invention, which specifically includes two parts: discovering defects in sample design and discovering defects in sample production:

[0032] 1. Sample Design Defect Discovery Method

[0033] The discovery of sample design defects depends on the semantic proximity of samples, which is the similarity of semantic features between sample sets and categories. The value range is between 0 and 1. The larger the value, the higher the similarity, the higher the overlap between categories, the smaller the distance between classes, and the worse the separability. On the contrary, the smaller the value, the lower the similarity, the lower the overlap between categories, the larger the distance between classes, and the better the separability. The specific process of sample design defects is as follows: Figure 2 shown.

[0034] According to the established multi-level-of-detail expression model of land use data semantics, such as Figure 3 As shown, the semantic feature items are described for each category of the data set. Classification is an existing technology. Figure 3 Each category is described with features and recorded as a feature set. Then, for each feature set of two categories, the semantic proximity between the two categories is calculated according to formula (1). Then, a semantic proximity distribution map is obtained to find the easily confused categories of design defects.

[0035]

[0036] Wherein, G is a density function, representing a hierarchical set of semantic feature items; parameters α and β are correction coefficients, and A and B represent two different categories, such as farmland and residential. For the sake of simplicity in calculation, in this embodiment, α and β are both set to a constant of 1.

[0037] Taking the AID sample set as an example, the semantic proximity of the sample set is calculated, and the following is obtained: Figure 4 The semantic proximity distribution diagram shown in the figure is used to find the category pairs with a semantic proximity of 1, and exclude the category pairs whose sample features are completely unrelated, and find the three pairs of defect categories shown in the figure, for a total of seven categories.

[0038] 2. Sample production defect discovery method

[0039] Sample production defect discovery is based on the PBLC (positive and background learning with constraints) model to discover the number of sample defects. The PBLC model can estimate the probability of positive samples being labeled (Cp), which represents the average probability of the classifier obtaining all labeled positive samples. Specifically, the classifier calculates the probability (similarity) of each labeled positive sample in a category on the feature dimension of the class in the feature space of the sample set, and then takes the average of all probabilities of this class, recorded as a constant Cp. When there are sample defects, the traditional PBLC model is used. Some samples of the labeled positive samples do not actually belong to the "label", and the probability of the classifier mapping to the label is reduced, which reduces Cp and increases the dispersion of the features of this category. Sample defects are usually characterized by number. In order to directly link the dispersion with the number of defects, the present invention improves the PBLC model and converts the average probability value Cp into an equivalent proportion Cn of positive samples being labeled. According to Figure 5 (a) shows the traditional PBLC model:

[0040] have Therefore, the derivation formula of Cn can be obtained:

[0041]

[0042] Among them, g(x) is the probability of positive samples, f(x) is the probability of sample labeling, L is the labeled positive sample identified by the classifier, P is the positive sample identified by the classifier, T is the total number of positive samples, and the output of the improved model is the proportion of positive samples that are labeled Cn. The PBLC improved model process is as follows Figure 5 (b) as shown.

[0043] For each class of samples in the sample set, the set of labeled positive samples is the labeled class samples themselves, denoted as TrueN. Due to labeling errors, the set of positive samples that should be labeled as this class but are not labeled is denoted as OutError. Therefore, the labeled proportion Cn of positive samples can be expressed as follows:

[0044]

[0045] Therefore, OutError can be expressed as:

[0046]

[0047] At the same time, when the sample size is large enough, InError = OutError, so:

[0048]

[0049] The labeled proportion of positive samples Cn can be obtained through the improved PBLC model. Substituting it into formula (5), we can get the number of errors (i.e., number of defects) of this type of samples as LabelError, TrueN represents the labeled positive sample set, InError represents the sample defects within the class, and OutError represents the positive sample set that should be labeled as this class but is not labeled (i.e., out-of-class defects).

[0050] Taking the AID sample set as an example, seven defect categories were input into the sample to make the defect detection model. The experiment was repeated five times and the following results were obtained: Figure 6 The partial frequency diagram shown in Figure 1 and the number of out-of-class defects shown in Table 1.

[0051] Table 1. Calculation results of defects of out-of-class samples in AID sample set

[0052]

[0053]

[0054]

[0055] Therefore, we can accurately obtain the number of sample defects in each defect category in the AID dataset: airport (10 images), viaduct (2 images), church (6 images), business district (8 images), densely populated residential area (5 images), school (10 images), and medium-sized residential area (6 images).

[0056] The key technical points of the present invention are:

[0057] 1. The PBLC algorithm is improved to be suitable for sample defect discovery in actual application scenarios, saving manual time, accurately counting defects, and avoiding the decrease in accuracy caused by too many defective samples in the sample set. At the same time, a conversion process from the probability of positive sample being marked to the proportion of positive sample being marked is proposed to fit the defect discovery process and make the calculation smoother.

[0058] 2. The semantic defect method was introduced into the sample set evaluation method. Considering the sample set as a whole, systematic suggestions were put forward for the improvement of the sample set.

[0059] The beneficial effect of the present invention is that the defect category and defect number of the sample set can be accurately obtained.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for quickly discovering and evaluating defects in a land use sample set, characterized by: It includes the sample design defect discovery part and the sample production defect discovery part; The sample design defect discovery section includes: Input the data in the sample set into the semantic proximity defect detection model to obtain the semantic proximity distribution map and semantic defect categories; Sample production defect discovery section: The data of the semantic defect category is input into the sample production defect detection model, which is obtained by the improved PBLC model, and the average probability value Cp is converted into an equivalent positive sample labeled ratio Cn; the sample production defect detection model includes a positive sample labeled probability estimation module, a labeled positive sample module and a probability frequency graph module. The data in the sample set is first input into the positive sample labeled probability estimation module to obtain the estimated positive sample labeled probability, and then enters the labeled positive sample module and the probability frequency graph module to finally obtain the positive sample labeled ratio Cn; The calculation formula of the positive sample labeled ratio Cn is as follows: Among them, g(x) is the probability of positive samples, f(x) is the probability of sample labeling, L is the labeled positive sample identified by the classifier, P is the positive sample identified by the classifier, and T is the total number of positive samples; According to the obtained semantic proximity distribution map and the proportion of positive samples labeled, the semantic design defects, in-class sample defects and out-of-class sample defects are obtained.

2. The method for quickly discovering and evaluating defects in a land use sample set according to claim 1, characterized in that: The semantic proximity defect detection model includes a multi-detailed hierarchical expression model of land use data semantics, a semantic feature expression module and a semantic proximity calculation module. The data in the sample set are first input into the multi-detailed hierarchical expression model of land use data semantics, and the semantic feature items are described for each category in the sample set. Then the data enters the semantic feature expression module, and finally enters the semantic proximity calculation module to finally obtain a semantic proximity distribution map.

3. The method for quickly discovering and evaluating defects in a land use sample set according to claim 2, characterized in that: According to formula (1), the semantic proximity S(a, b) between two categories is calculated. By calculating the multiple semantic proximities, a semantic proximity distribution map is obtained, and then the easily confused categories of design defects are found. Among them, G() is a density function used to characterize the hierarchical set of semantic feature items; parameters α and β are correction coefficients, and A and B represent two different categories.

4. The method for quickly discovering and evaluating defects in a land use sample set according to claim 1, characterized in that: This method also obtains the number of defects LabelError through the following formula: Among them, Cn represents the proportion of positive samples that are labeled, TrueN represents the set of labeled positive samples, InError represents the sample defect within the class, and OutError represents the sample defect outside the class.

Images