Natural resource field large model evaluation set generation method

Abstract

The invention provides a method for generating a large model evaluation set in the field of natural resources, and the method comprises the steps: obtaining the annotation quality distribution of a data set through statistical inference based on a data distribution model and an annotation model; setting the number of kernel functions for kernel density estimation as the number of data sources in the field of natural resources, performing expansion approximation on annotated quality distribution, and determining the center and bandwidth of each kernel function; and taking each kernel function center as an unbiased estimation value, optimizing data fusion through a weighted average fusion algorithm, and obtaining a fusion estimation value of the data set labeling quality. According to the method, the accuracy and reliability of data set labeling quality can be improved, and high-quality data support is provided for generation of a large model evaluation set.

Description

Technical Field

[0001] The present invention relates to the field of natural resources, and more specifically, to a method for generating a large model evaluation set in the field of natural resources. Background Art

[0002] In the field of natural resources, with the development of big data and artificial intelligence technology, building a high-quality large model evaluation set has become a key link in promoting model performance improvement. Existing data annotation methods are usually based on simple annotation models, assuming that the data distribution is uniform and the annotation noise is standard. However, data in the field of natural resources is complex, the data distribution is often non-uniform, and the annotation noise may also present non-standard characteristics. This situation makes it difficult for existing methods to accurately reflect the true annotation quality distribution of data when processing data in the field of natural resources, which in turn affects the quality of the evaluation set and the performance of the model.

[0003] In terms of technical principles, traditional annotation quality assessment methods mainly rely on a single annotation model and lack in-depth analysis and modeling of data distribution characteristics. Kernel density estimation, as a non-parametric estimation method, has certain advantages in data distribution modeling, but in the application of natural resources, the correspondence between the number of data sources and the number of kernel functions has not been fully considered, and how to further improve the fusion estimation accuracy of annotation quality through optimization algorithms.

[0004] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing annotation methods cannot effectively handle the non-uniform distribution and non-standard annotation noise of data in the field of natural resources, resulting in inaccurate estimation of the annotation quality distribution; at the same time, there is a lack of an effective fusion optimization mechanism to integrate the estimation results of multiple kernel functions to further improve the reliability of annotation quality. These problems limit the generation of high-quality evaluation sets, which in turn affects the performance and application effect of large models in the field of natural resources. Summary of the invention

[0005] The present invention provides a method for generating a large model evaluation set in the field of natural resources.

[0006] In a first aspect of the present invention, a method for generating a large model evaluation set in the field of natural resources is provided, comprising: S1. Obtaining the annotation quality distribution of the data set by statistical inference based on the data distribution model and the annotation model, wherein the data distribution model includes non-uniform data distribution, and the annotation model includes non-standard annotation noise; S2. Setting the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and performing an expansion approximation on the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; S3. The centers of the kernel functions are taken as unbiased estimates, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set. Data is selected based on the annotation quality to form a data set to form an evaluation set.

[0007] Furthermore, in step S1, obtaining the annotation quality distribution of the data set by statistical inference based on the data distribution model and the annotation model includes: S11. Establish data distribution model and annotation model; S12, obtaining the annotation quality distribution according to statistical inference based on the data distribution model and the annotation model; S13, performing weighted approximation on the non-uniform data distribution and the non-standard annotation noise by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise respectively; S14, obtaining an approximate representation of the annotation quality distribution based on the approximate representations of the non-uniform distributions and the non-standard annotation noises and the relationship between the non-uniform distributions and the non-standard annotation noises and the annotation quality distribution.

[0008] Furthermore, in step S12, obtaining the annotation quality distribution according to statistical inference based on the data distribution model and the annotation model includes: Based on the annotation quality distribution and data distribution model of the previous sample, the prior distribution of the current sample is obtained through prediction; Based on the labeling model, the prior distribution of the current sample is updated to obtain the labeling quality distribution of the current sample, and the calculation method is: the product of the labeling value distribution of sample k based on the data features through the labeling model and the prior distribution based on the labeling value of the previous sample is divided by the prior distribution based on the labeling value of the previous sample.

[0009] Furthermore, in step S13, weighted approximation is performed on the non-uniform data distribution and the non-standard annotation noise respectively by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise, including: Construct a kernel density estimation model with m kernel functions; A weighted sum is performed on each kernel function of the kernel density estimation model to obtain an approximation of the non-uniform data distribution and the non-standard annotation noise.

[0010] Further, in step S14, based on the approximate representations of the non-uniform distributions and the non-standard annotation noises and the relationship between the non-uniform distributions and the non-standard annotation noises and the annotation quality distribution, an approximate representation of the annotation quality distribution is obtained, including: Approximating the labeled quality distribution of the previous sample by kernel density estimation; Substituting an approximate representation of the non-uniform data distribution into the data distribution model to approximate the data distribution model; Substituting an approximate representation of the non-standard annotation noise into the annotation model to obtain an approximation of the distribution of annotation values ​​obtained by the annotation model; Substituting the approximation of the data distribution model and the labeled quality distribution of the previous sample k - 1 into the prior distribution to obtain an approximation of the prior distribution; An approximation of the prior distribution and an approximation of the label value distribution are substituted into the posterior distribution to obtain an approximate representation of the label quality distribution.

[0011] Furthermore, the centers of the kernel functions are taken as unbiased estimates, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set, including: The fused estimated value is set to be the sum of the product of the fusion weight corresponding to each unbiased estimated value and the unbiased estimated value, and the sum of the fusion weights is 1; Determining the optimization objective function to minimize the mean square error of the fused estimate, and obtaining the initial mean square error based on the covariance matrix or cross-covariance matrix between the initial fusion weights and each unbiased estimate; Calculating the gradient of the objective function with respect to each of the fusion weights; Updating the fusion weight based on the gradient; Calculate a new mean square error based on the updated fusion weights. If the new mean square error is smaller than the mean square error before this iteration, update the mean square error to the mean square error obtained in this iteration. Otherwise, retain the mean square error and fusion weights before the update. Iteratively updating the fusion weight until a preset condition is met to obtain a final fusion weight; A final fusion estimate of the data set annotation quality is obtained based on the final fusion weight and the corresponding unbiased estimate.

[0012] Furthermore, the objective function is equivalently determined as: in, represents the objective function, represents the fusion weight of the i-th unbiased estimate, represents the variance of the ith unbiased estimate, represents the covariance between the ith unbiased estimator and the jth unbiased estimator.

[0013] In a second aspect of the present invention, a device for generating a large model evaluation set in the field of natural resources is provided, comprising: An inference module, configured to obtain a label quality distribution of a data set by statistical inference based on a data distribution model and a labeling model, wherein the data distribution model includes a non-uniform data distribution and the labeling model includes non-standard labeling noise; An approximation module is used to set the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and to approximate the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; The fusion module is used to use the centers of the kernel functions as unbiased estimates, and perform data fusion optimization through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set.

[0014] In a third aspect of the present invention, an electronic device is provided, comprising: at least one processor, a memory and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute any one of the methods described in the first aspect.

[0015] In a fourth aspect of the present invention, a computer-readable storage medium is provided, which includes instructions, and when the instructions are executed on a computer, the computer executes any one of the methods in the first aspect.

[0016] The above-mentioned embodiments of the present invention have at least the following beneficial effects: the present invention can effectively improve the annotation quality of large model evaluation sets in the field of natural resources. By establishing a data distribution model and an annotation model, combined with statistical inference methods, the non-uniform distribution and non-standard annotation noise characteristics of the data can be accurately characterized, thereby providing a more reliable basis for estimating the annotation quality distribution. In addition, the accuracy and flexibility of the annotation quality distribution estimation can be further improved by using kernel density estimation to expand and approximate the annotation quality distribution, and matching the number of kernel functions with the number of data sources in the field of natural resources. At the same time, based on the weighted average fusion algorithm to optimize the unbiased estimate of each kernel function, it is possible to combine the advantages of multiple estimation results, reduce errors, and ultimately obtain a high-quality fusion estimate of the data set annotation quality.

[0017] The present invention can also provide higher quality data support for the evaluation of large models in the field of natural resources. By optimizing the estimation process of the annotation quality distribution, the errors caused by uneven data distribution and annotation noise can be effectively reduced, thereby improving the accuracy and reliability of the evaluation set. This is of great significance for improving the application effect and performance of large models in the field of natural resources, and can better meet the model evaluation needs in the complex data environment of the field of natural resources, and promote the development and application of related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which: Figure 1 A schematic diagram of a flow chart of a method for generating a large model evaluation set in the field of natural resources provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a large model evaluation set generating device in the field of natural resources provided by an embodiment of the present invention; Figure 3 The schematic diagram schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0020] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0021] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0022] Reference below Figure 1 , Figure 1 The following is a flow chart of a method for generating a large model evaluation set in the field of natural resources provided by an embodiment of the present invention. Figure 1 As shown, a method 100 for generating a large model evaluation set in the field of natural resources includes: S1. Obtaining the annotation quality distribution of the data set by statistical inference based on the data distribution model and the annotation model, wherein the data distribution model includes non-uniform data distribution, and the annotation model includes non-standard annotation noise; S2. Setting the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and performing an expansion approximation on the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; S3. Taking the centers of the kernel functions as unbiased estimates, respectively, data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set.

[0023] It should be noted that the present invention relates to a method for generating a large model evaluation set in the field of natural resources, which aims to improve the accuracy and reliability of the evaluation set by optimizing the data annotation quality distribution. The method includes three main steps: first, based on the data distribution model and the annotation model, the annotation quality distribution of the data set is obtained by statistical inference. The data distribution model is used to describe the non-uniform distribution characteristics of the data, while the annotation model is used to describe the non-standard noise in the data annotation process. Through statistical inference, the annotation quality distribution of the data set can be accurately estimated.

[0024] Specifically, the establishment of data distribution model and annotation model is the key step of this method. The data distribution model can include various non-uniform data distribution forms, such as normal distribution, Poisson distribution, etc., while the annotation model can include different types of annotation noise, such as Gaussian noise, uniform noise, etc. Kernel density estimation is used to expand and approximate the annotation quality distribution, that is, to expand the approximate representation of the annotation quality distribution. The number of kernel functions is set to the number of data sources in the natural resources field to ensure the accuracy and flexibility of the estimation. The kernel function in the kernel density estimation can select Gaussian kernel, triangular kernel, etc., and the bandwidth parameter is adjusted according to the data characteristics.

[0025] More specifically, data fusion optimization is achieved through a weighted average fusion algorithm. The center of each kernel function is used as an unbiased estimate, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the quality of the data set annotation. The optimization goal is to minimize the mean square error of the fusion estimate, and finally obtain the optimal fusion weight by iteratively updating the fusion weight. Preferably, the gradient descent method can be used to calculate the gradient of the objective function for each fusion weight, and the fusion weight is updated according to the gradient to ensure the convergence and stability of the fusion process.

[0026] In some embodiments, in step S1, obtaining the annotation quality distribution of the data set by statistical inference based on the data distribution model and the annotation model includes: S11. Establish a data distribution model and a labeling model, expressed as: in, represents the data sample number, Representation sample The data characteristics of Representation sample The data distribution model, represents non-uniform data distribution, Representation sample The annotation value of Representation sample The annotation model, represents non-standard annotation noise; S12, obtaining the annotation quality distribution according to statistical inference based on the data distribution model and the annotation model; S13, performing weighted approximation on the non-uniform data distribution and the non-standard annotation noise by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise respectively; S14, obtaining an approximate representation of the annotation quality distribution based on the approximate representations of the non-uniform distributions and the non-standard annotation noises and the relationship between the non-uniform distributions and the non-standard annotation noises and the annotation quality distribution.

[0027] It should be noted that, based on the establishment of data distribution model and annotation model, the present invention obtains the annotation quality distribution process through statistical inference, and further refines the model construction and inference method. The data distribution model is expressed as ,in Representation sample The data characteristics of Representation sample The data distribution model, represents non-uniform data distribution. The annotation model is represented as ,in Representation sample The annotation value of Representation sample The annotation model, Represents non-standard annotation noise. By weighted approximation of non-uniform data distribution and non-standard annotation noise through kernel density estimation, we can obtain their respective approximate representations, thus providing a basis for estimating the annotation quality distribution.

[0028] Specifically, the parameter setting of the data distribution model and the annotation model is the key to implementing this method. In the data distribution model, It can be a linear or nonlinear function to describe the dynamic change process of data. For example, polynomial functions or neural networks can be used to approximate complex data distribution. Non-uniform data distribution It can be a random variable with a specific probability distribution, such as Gaussian distribution or Laplace distribution, whose parameters (such as mean and variance) can be adjusted according to the actual data characteristics. It can be a simple linear mapping or a complex nonlinear mapping, which is used to describe the relationship between data features and annotation values. It can be Gaussian noise, uniform noise or other types of noise, and its parameters (such as noise intensity) can also be set according to the actual error in the annotation process. To match the number of data sources in the field of natural resources, the kernel function can be selected from Gaussian kernel, uniform kernel or triangular kernel, etc., and the bandwidth parameter The optimization selection is performed based on the distribution characteristics of the data.

[0029] Preferably, in order to further improve the accuracy of the annotation quality distribution estimation, the operation steps can be refined in the following ways: First, for the selection of kernel function in kernel density estimation, the Gaussian kernel function can be used because it has good smoothness and locality and can better adapt to the characteristics of non-uniform data distribution. Secondly, the bandwidth parameter The cross-validation method can be used to optimize the selection to ensure the accuracy and stability of the kernel density estimate. In addition, for the approximate representation of non-uniform data distribution and non-standard annotation noise, an adaptive weighting method can be used to dynamically adjust the weights according to the local characteristics of the data to improve the accuracy of the approximation. Finally, the approximation of the annotation quality distribution can be further optimized by Bayesian inference methods, combining the prior distribution and observed data to obtain a more accurate estimate of the posterior distribution.

[0030] In some embodiments, in step S12, obtaining the annotation quality distribution according to statistical inference based on the data distribution model and the annotation model includes: Based on the annotation quality distribution of the previous sample and the data distribution model, the prior distribution of the current sample is predicted, which is expressed as: in, represents the prior distribution of the labeled value based on the previous sample, represents the data feature distribution of sample k obtained through the data distribution model, Represents the annotation quality distribution of the previous sample k-1; Based on the annotation model, the prior distribution of the current sample is updated to obtain the annotation quality distribution of the current sample, which is expressed as: in, It represents the distribution of the label value of sample k based on the data features through the labeling model. Represents the annotation quality distribution of the current sample k.

[0031] It should be noted that in the statistical inference process of the annotation quality distribution, the present invention further refines the calculation method of the prior distribution and the posterior distribution by combining the data distribution model and the annotation model. Specifically, based on the annotation quality distribution and data distribution model of the previous sample, the prior distribution of the current sample is predicted, which is expressed as .in, Represents the prior distribution based on the previous sample label value, represents the data feature distribution of sample k obtained through the data distribution model, Indicates the previous sample The annotation quality distribution of the current sample is obtained by updating the prior distribution through the annotation model. This process utilizes the idea of ​​Bayesian inference, combining prior knowledge and observed data to dynamically update the estimate of the annotation quality distribution.

[0032] Specifically, the calculation of prior distribution and posterior distribution is the core of label quality distribution estimation. In the calculation of prior distribution, Data distribution model and non-uniform data distribution To describe the sample For example, if the data distribution model is a linear dynamic system, can be a linear function, It can be Gaussian noise. In the update of the posterior distribution, the label model Describes the relationship between data features and annotation values, non-standard annotation noise It can be Gaussian noise or other types of noise. ,Combining the prior distribution and the observed data, the estimation of the annotation quality distribution can be dynamically updated.,This process not only takes into account the dynamic changes of the data, but also combines the noise characteristics of the annotation process, thereby improving the accuracy of the annotation quality distribution estimation.

[0033] Preferably, in order to further optimize the estimation process of the annotation quality distribution, the calculation of the prior distribution and the posterior distribution can be refined. For example, in the calculation of the prior distribution, time series analysis methods such as Kalman filtering or particle filtering can be introduced to better handle the dynamic changes of the data. ,An adaptive noise modeling method can be used to dynamically adjust the noise parameters according to the statistical characteristics of the labeled data.

[0034] Furthermore, in the update of the posterior distribution, the Monte Carlo method can be used for numerical calculation to improve the calculation efficiency and accuracy. In the alternative, for the parameter estimation of the data distribution model and the annotation model, the maximum likelihood estimation or Bayesian estimation method can be used, combined with actual data for optimization, so as to further improve the accuracy and reliability of the annotation quality distribution estimation.

[0035] In some embodiments, in step S13, weighted approximation is performed on the non-uniform data distribution and the non-standard annotation noise by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise, respectively, including: Construct a kernel density estimation model with m kernel functions, expressed as: in, represents the kernel density estimation model, represents the kernel function, represents the center of the i-th kernel function, represents the bandwidth of the i-th kernel function; Performing weighted summation on the kernel functions of the kernel density estimation model to obtain respective approximations of the non-uniform data distribution and the non-standard annotation noise; The approximate representation of the non-uniform data distribution is: in, is an approximate representation of the non-uniform data distribution, For approximation The number of kernel functions, is the weight corresponding to the i-th kernel function in the approximate representation of non-uniform data distribution; The approximate expression of the non-standard annotation noise is: in, is an approximate representation of non-standard annotation noise, For approximation The number of kernel functions, Represents the weight corresponding to the i-th kernel function in the approximate representation of non-standard annotation noise.

[0036] It should be noted that in the process of approximating the annotation quality distribution, the present invention uses kernel density estimation to perform weighted approximation on non-uniform data distribution and non-standard annotation noise, further refining the approximation method of annotation quality distribution. Kernel density estimation is a non-parametric estimation method that approximates complex data distribution through the weighted sum of multiple kernel functions. In the present invention, the kernel density estimation model is expressed as ,in represents the kernel density estimation model, represents the kernel function, Indicates The center of the kernel function, Indicates Through kernel density estimation, we can obtain approximate representations of non-uniform data distribution and non-standard annotation noise, respectively, thus providing a more accurate basis for estimating the annotation quality distribution.

[0037] Specifically, the kernel function selection and parameter setting in kernel density estimation are the key points to achieve the approximate distribution of annotation quality. You can choose Gaussian kernel, uniform kernel, triangular kernel or Kernel functions have different smoothness and locality. For example, the Gaussian kernel function is suitable for approximating most data distributions due to its good smoothness and locality. and bandwidth is the key parameter of kernel density estimation. The bandwidth can be determined by the local feature points of the data. It determines the smoothness of the kernel function and is usually adjusted according to the distribution characteristics and noise level of the data. and approximate representation of non-standard annotation noise , by choosing the appropriate number of kernel functions and weight To achieve this. Number of kernel functions The weights can be set according to the number and complexity of data sources. It can be determined by optimization methods such as minimizing the mean square error.

[0038] Preferably, in order to further optimize the kernel density estimation process, the selection of kernel functions and parameter settings can be refined. For example, in the selection of kernel functions, a mixed kernel function can be selected in combination with the distribution characteristics of the data, such as a combination of a Gaussian kernel and a triangular kernel, to better adapt to complex data distribution. , an adaptive bandwidth method can be used to dynamically adjust the bandwidth according to the local data density, thereby improving the accuracy of kernel density estimation. In addition, the weight of the kernel function Adjustments can be made through cross-validation or Bayesian optimization methods to ensure the stability and accuracy of the kernel density estimate. Alternatively, regularization terms such as L1 or L2 regularization can be introduced to prevent overfitting and further improve the reliability of the approximation of the annotation quality distribution.

[0039] In some embodiments, in step S14, obtaining an approximate representation of the annotation quality distribution based on the approximate representation of each non-uniform distribution and non-standard annotation noise and the relationship between each non-uniform distribution and non-standard annotation noise and the annotation quality distribution includes: The label quality distribution of the previous sample is approximated by kernel density estimation, which is expressed as: in, represents the label quality distribution of the previous sample k-1, is the weight corresponding to the i-th kernel function in the approximate representation of the labeled quality distribution of the previous sample; Substituting the approximate representation of the non-uniform data distribution into the data distribution model to approximate the data distribution model is expressed as: Substitute the approximate representation of the non-standard annotation noise into the annotation model to obtain an approximation of the annotation value distribution obtained by the annotation model, expressed as: Substitute the approximation of the data distribution model and the labeled quality distribution of the previous sample k - 1 into the prior distribution to obtain the approximation of the prior distribution, expressed as: Substitute the approximation of the prior distribution and the approximation of the label value distribution into the posterior distribution to obtain an approximate representation of the label quality distribution, expressed as: It should be noted that the present invention further refines the application of kernel density estimation in the process of approximate representation of annotation quality distribution, by applying kernel density estimation to the annotation quality distribution of the previous sample and the approximation of the data distribution model and the annotation model, thereby obtaining an approximate representation of the annotation quality distribution. This process utilizes the flexibility and non-parametric characteristics of kernel density estimation, and can better adapt to the complexity of non-uniform data distribution and non-standard annotation noise. Specifically, kernel density estimation approximates the annotation quality distribution of the previous sample, combines the approximate results of the data distribution model and the annotation model, and finally obtains an approximate representation of the annotation quality distribution of the current sample, providing a basis for subsequent data fusion optimization.

[0040] Specifically, the application of kernel density estimation in the approximation of annotation quality distribution involves several key steps. First, for the annotation quality distribution of the previous sample , approximated by kernel density estimation, expressed as ,in is the weight, is the kernel function center, is the bandwidth. Secondly, the approximation of the data distribution model The approximate result of the non-uniform data distribution can be substituted into the model, expressed as Similarly, the distribution of annotation values ​​of the annotation model is approximately The approximate result can be expressed by substituting the non-standard annotation noise into These approximate results are combined with the update formula of the prior distribution and the posterior distribution to finally obtain an approximate representation of the label quality distribution. In terms of parameter setting, the number of kernel functions You can choose according to the complexity of the data source, weight The bandwidth can be determined by optimization methods (such as minimizing the mean square error). Adjustment can be done through cross-validation or heuristic methods.

[0041] Preferably, in order to further optimize the approximate process of the annotation quality distribution, the parameter selection and calculation method of the kernel density estimation can be refined. For example, in the selection of the kernel function, the Gaussian kernel function can be combined with the local adaptive bandwidth method to better adapt to the local changes in the data distribution. , can be dynamically adjusted through the Bayesian optimization method to ensure the accuracy and stability of the kernel density estimation.

[0042] Furthermore, regularization terms can be introduced to prevent overfitting, especially when dealing with high-dimensional data. As an alternative, for the approximate representation of the annotation quality distribution, a hierarchical kernel density estimation method can be used to divide the data into multiple levels for modeling, thereby improving the fitting ability of complex data distribution. This method can further improve the accuracy of annotation quality distribution estimation while maintaining computational efficiency.

[0043] In some embodiments, the center of each kernel function is used as an unbiased estimate, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set, including: The fused estimated value is set to be the sum of the product of the fusion weight corresponding to each unbiased estimated value and the unbiased estimated value, and the sum of the fusion weights is 1; Determining the optimization objective function to minimize the mean square error of the fused estimate, and obtaining the initial mean square error based on the covariance matrix or cross-covariance matrix between the initial fusion weights and each unbiased estimate; Calculating the gradient of the objective function with respect to each of the fusion weights; Updating the fusion weight based on the gradient; Calculate a new mean square error based on the updated fusion weights. If the new mean square error is smaller than the mean square error before this iteration, update the mean square error to the mean square error obtained in this iteration. Otherwise, retain the mean square error and fusion weights before the update. Iteratively updating the fusion weight until a preset condition is met to obtain a final fusion weight; A final fusion estimate of the data set annotation quality is obtained based on the final fusion weight and the corresponding unbiased estimate.

[0044] It should be noted that in the fusion estimation process of the annotation quality distribution, the present invention optimizes the unbiased estimate obtained by the kernel density estimation through the weighted average fusion algorithm to obtain the fusion estimate of the annotation quality of the data set. The core of this process is to set the fusion estimate as the optimization result of the objective function, where the sum of the fusion weights is 1, and the objective function is to minimize the mean square error of the fusion estimate. By iteratively updating the fusion weights, the optimal fusion estimate is finally obtained, thereby improving the reliability and accuracy of the annotation quality.

[0045] Specifically, the implementation of the weighted average fusion algorithm involves multiple key steps and parameter settings. First, the calculation formula of the fusion estimate is: ,in It is The fusion weight of the unbiased estimate, It is unbiased estimates. The objective function is defined as minimizing the mean square error of the fused estimates, i.e. ,in It is The variance of an unbiased estimate, It is and The initial fusion weights can be set by uniform distribution or based on prior knowledge, and then the gradient of the objective function with respect to each fusion weight is calculated. To update the weights. In each iteration, if the new mean square error is less than the current value, the fusion weights and mean square error are updated, otherwise the current values ​​are retained. This process is iteratively optimized to eventually obtain the optimal fusion weights and fusion estimates.

[0046] Preferably, in order to further optimize the fusion estimation process, the definition of the objective function and the optimization method can be refined. For example, the objective function can introduce a regularization term, such as L2 regularization, to prevent overfitting and improve the stability of the fusion estimation. In terms of optimization methods, in addition to the gradient descent method, more efficient optimization algorithms, such as Newton's method or conjugate gradient method, can also be used to speed up convergence and improve computational efficiency. In addition, for the initial setting of the fusion weight, adaptive initialization can be performed based on the variance and covariance matrices of the unbiased estimate instead of a simple uniform distribution. In an alternative, it is possible to consider introducing a dynamic weight adjustment mechanism to dynamically adjust the weights according to the error feedback of each iteration, so as to better adapt to the differences in annotation quality of different data sources. This method can further improve the robustness and adaptability of the annotation quality distribution estimation while maintaining the accuracy of the fusion estimation.

[0047] In some embodiments, the objective function is equivalently determined as: in, represents the objective function, Indicates The fusion weight of the unbiased estimate, Indicates The variance of an unbiased estimate, Indicates unbiased estimates and The covariance between the two unbiased estimates.

[0048] It should be noted that in the fusion estimation process of the annotation quality distribution, the present invention further clarifies the equivalent form of the objective function to optimize the calculation of the fusion weight. The objective function is expressed as ,in It is The fusion weight of the unbiased estimate, It is The variance of an unbiased estimate, It is and This form decomposes the mean square error of the fused estimate into a combination of variance and covariance, providing a clearer mathematical basis for optimizing the fusion weights, thereby further improving the accuracy and reliability of the annotation quality distribution estimation.

[0049] Specifically, the equivalent form of the objective function involves the calculation of fusion weights, variance, and covariance. are the parameters that need to be adjusted during the optimization process, and their sum is 1, that is, .variance Represents the uncertainty of each unbiased estimate, which can be obtained by statistical analysis of the output of each kernel function. Describes the correlation between different unbiased estimates and reflects the degree of information sharing between them. During the optimization process, the initial fusion weight can be assigned according to the variance of the unbiased estimate. For example, the estimate with a smaller variance can be assigned a larger weight. The optimization goal is to minimize the objective function , by calculating the gradient of the objective function to the fusion weight The weights are updated until the preset convergence conditions are met, such as the mean square error no longer decreases significantly or the number of iterations reaches the upper limit. This process ensures that the fusion estimate has the minimum uncertainty in a statistical sense by dynamically adjusting the fusion weights.

[0050] Preferably, in order to further optimize the fusion estimation process, the optimization strategy and parameter setting of the objective function can be refined. For example, constraints can be introduced, such as limiting the range of fusion weights (e.g., 0 ≤ ≤1), to avoid some weights being too large or too small, thereby improving the stability of the fusion results. In the selection of optimization algorithms, in addition to the gradient descent method, more advanced optimization algorithms can also be used, such as genetic algorithms or particle swarm optimization algorithms, which can better handle complex nonlinear optimization problems, especially when there are multiple local optimal solutions.

[0051] Furthermore, for the calculation of variance and covariance, a dynamic adjustment mechanism can be introduced to update these parameters according to the error feedback of each iteration, so as to better adapt to the changes in data. As an alternative, a multi-objective optimization framework can be considered to simultaneously minimize the mean square error and maximize the diversity of the fused estimate to further improve the performance of the annotation quality distribution estimation.

[0052] The above-mentioned embodiments of the present invention have the following beneficial effects: the present invention can effectively improve the annotation quality of the large model evaluation set in the field of natural resources through statistical inference and kernel density estimation methods. Specifically, based on the data distribution model and the annotation model, it can accurately capture the non-uniform distribution and non-standard annotation noise characteristics of the data, thereby providing a reliable basis for estimating the annotation quality distribution. By expanding and approximating the annotation quality distribution through kernel density estimation and reasonably setting the number of kernel functions, the accuracy and flexibility of the annotation quality distribution estimation can be improved, ensuring the high quality of the evaluation set.

[0053] In addition, the present invention can optimize the unbiased estimation value of each kernel function through a weighted average fusion algorithm, combine the advantages of multiple estimation results, reduce errors, and obtain high-quality data set annotation quality fusion estimation values. This method can not only improve the accuracy and reliability of annotation quality, but also provide higher quality data support for the evaluation of large models in the field of natural resources, meet the model evaluation needs in complex data environments, and promote the development and application of related technologies in the field of natural resources.

[0054] like Figure 2 As shown, in some embodiments, a large model evaluation set generation device 200 in the field of natural resources includes: An inference module 201 is used to obtain the annotation quality distribution of the data set through statistical inference based on the data distribution model and the annotation model, wherein the data distribution model includes non-uniform data distribution and the annotation model includes non-standard annotation noise; An approximation module 202 is used to set the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and to perform an expansion approximation on the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; The fusion module 203 is used to use the centers of the kernel functions as unbiased estimates, and perform data fusion optimization through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set.

[0055] It can be understood that the modules recorded in the natural resources field large model evaluation set generation device 200 and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method for generating a large model evaluation set in the field of natural resources are also applicable to the device for generating a large model evaluation set in the field of natural resources 200 and the modules contained therein, and will not be repeated here.

[0056] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0057] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0058] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0059] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0060] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A method for generating a large model evaluation set in the field of natural resources, characterized in that: include: S1. Obtaining the annotation quality distribution of the data set by statistical inference based on the data distribution model and the annotation model, wherein the data distribution model includes non-uniform data distribution, and the annotation model includes non-standard annotation noise; S2. Setting the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and performing an expansion approximation on the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; S3. The centers of the kernel functions are taken as unbiased estimates, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set. Data is selected based on the annotation quality to form a data set to form an evaluation set.

2. The method for generating a large model evaluation set in the field of natural resources according to claim 1, characterized in that: In step S1, obtaining the annotation quality distribution of the data set through statistical inference based on the data distribution model and the annotation model includes: S11. Establish data distribution model and annotation model; S12, obtaining the annotation quality distribution according to statistical inference based on the data distribution model and the annotation model; S13, performing weighted approximation on the non-uniform data distribution and the non-standard annotation noise by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise respectively; S14, obtaining an approximate representation of the annotation quality distribution based on the approximate representations of the non-uniform distributions and the non-standard annotation noises and the relationship between the non-uniform distributions and the non-standard annotation noises and the annotation quality distribution.

3. The method for generating a large model evaluation set in the field of natural resources according to claim 2, characterized in that: In step S12, the annotation quality distribution is obtained according to statistical inference based on the data distribution model and the annotation model; including: Based on the annotation quality distribution and data distribution model of the previous sample, the prior distribution of the current sample is obtained through prediction; Based on the labeling model, the prior distribution of the current sample is updated to obtain the labeling quality distribution of the current sample, and the calculation method is: the product of the labeling value distribution of sample k based on the data features through the labeling model and the prior distribution based on the labeling value of the previous sample is divided by the prior distribution based on the labeling value of the previous sample.

4. The method for generating a large model evaluation set in the field of natural resources according to claim 3, characterized in that: In step S13, weighted approximation is performed on the non-uniform data distribution and the non-standard annotation noise by kernel density estimation to obtain approximate representations of the non-uniform data distribution and the non-standard annotation noise, respectively, including: Construct a kernel density estimation model with m kernel functions; A weighted sum is performed on each kernel function of the kernel density estimation model to obtain an approximation of the non-uniform data distribution and the non-standard annotation noise.

5. The method for generating a large model evaluation set in the field of natural resources according to claim 4, characterized in that: In step S14, based on the approximate representations of the non-uniform distributions and the non-standard annotation noises and the relationship between the non-uniform distributions and the non-standard annotation noises and the annotation quality distribution, an approximate representation of the annotation quality distribution is obtained, including: Approximating the labeled quality distribution of the previous sample by kernel density estimation; Substituting an approximate representation of the non-uniform data distribution into the data distribution model to approximate the data distribution model; Substituting an approximate representation of the non-standard annotation noise into the annotation model to obtain an approximation of the distribution of annotation values ​​obtained by the annotation model; Substituting the approximation of the data distribution model and the labeled quality distribution of the previous sample k - 1 into the prior distribution to obtain an approximation of the prior distribution; An approximation of the prior distribution and an approximation of the label value distribution are substituted into the posterior distribution to obtain an approximate representation of the label quality distribution.

6. The method for generating a large model evaluation set in the field of natural resources according to claim 5, characterized in that: The center of each kernel function is used as an unbiased estimate, and data fusion optimization is performed through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set, including: The fused estimated value is set to be the sum of the product of the fusion weight corresponding to each unbiased estimated value and the unbiased estimated value, and the sum of the fusion weights is 1; Determining the optimization objective function to minimize the mean square error of the fused estimate, and obtaining the initial mean square error based on the covariance matrix or cross-covariance matrix between the initial fusion weights and each unbiased estimate; Calculating the gradient of the objective function with respect to each of the fusion weights; Updating the fusion weight based on the gradient; A new mean square error is calculated based on the updated fusion weights. If the new mean square error is smaller than the mean square error before this iteration, the mean square error is updated to the mean square error obtained in this iteration. Otherwise, the mean square error and fusion weights before the update are retained. Iteratively updating the fusion weight until a preset condition is met to obtain a final fusion weight; A final fusion estimate of the data set annotation quality is obtained based on the final fusion weight and the corresponding unbiased estimate.

7. The method for generating a large model evaluation set in the field of natural resources according to claim 6, characterized in that: The objective function is equivalently determined as: in, represents the objective function, represents the fusion weight of the i-th unbiased estimate, represents the variance of the ith unbiased estimate, represents the covariance between the ith unbiased estimator and the jth unbiased estimator.

8. A device for generating a large model evaluation set in the field of natural resources, characterized in that: include: An inference module, configured to obtain a label quality distribution of a data set by statistical inference based on a data distribution model and a labeling model, wherein the data distribution model includes a non-uniform data distribution and the labeling model includes non-standard labeling noise; An approximation module is used to set the number of kernel functions of kernel density estimation to the number of data sources in the field of natural resources, and to approximate the annotation quality distribution through kernel density estimation to determine the center and bandwidth of each kernel function; The fusion module is used to use the centers of the kernel functions as unbiased estimates, and perform data fusion optimization through a weighted average fusion algorithm to obtain a fusion estimate of the annotation quality of the data set.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, it implements the method for generating a large model evaluation set in the natural resources field as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for generating a large model evaluation set in the field of natural resources as described in any one of claims 1 to 7 is implemented.

Images