Application performance evaluation model construction method, application performance evaluation method and device

Abstract

The present invention provides a method for constructing an application effectiveness evaluation model, an application effectiveness evaluation method and a device, which relate to the technical field of remote sensing satellite application effectiveness evaluation, including: obtaining a sample data set; modeling a training sample data set by presetting a Gaussian process regression model, and performing hyperparameter learning processing on the Gaussian process regression model after modeling to determine the target hyperparameter; determining the application effectiveness prediction value of the remote sensing satellite based on the Gaussian process regression model after modeling and the target hyperparameter, and performing confidence evaluation processing on the application effectiveness prediction value to determine the confidence evaluation result, and determining the Gaussian process regression model after modeling and the target hyperparameter as the target application effectiveness evaluation model when the actual value of the application effectiveness is within the range of the confidence evaluation result of the application effectiveness prediction value. The present invention can adapt to the nonlinear characteristics of remote sensing satellite application effectiveness evaluation and significantly improve the accuracy of remote sensing satellite application effectiveness evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing satellite application effectiveness evaluation, and in particular to a method for constructing an application effectiveness evaluation model, an application effectiveness evaluation method and a device. Background Art

[0002] With the advancement of science and technology, the composition and functions of remote sensing satellites are becoming more and more complex, and the uncertainty of completing remote sensing tasks is also increasing. Under the influence of multiple factors, performance evaluation has nonlinear and dynamic changes, which makes it difficult to evaluate. At present, relevant technologies have proposed that performance evaluation can be carried out through analytical methods and comprehensive evaluation methods.

[0003] However, the above scheme requires prior knowledge of the clear relationship between performance indicators and performance or the degree of impact on performance, and cannot adapt to the nonlinear and dynamic changes in remote sensing satellite performance. If performance evaluation is based on the intelligent model method, there is a problem that the confidence level of the performance evaluation cannot be directly obtained, and the degree of impact of each performance indicator on the remote sensing satellite performance cannot be quantitatively analyzed. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a remote sensing satellite application effectiveness evaluation method, device and server, which can adapt to the nonlinear characteristics of remote sensing satellite application effectiveness evaluation and significantly improve the accuracy of remote sensing satellite application effectiveness evaluation.

[0005] In a first aspect, an embodiment of the present invention provides a method for constructing an application effectiveness evaluation model, which is applied to a remote sensing satellite experiment, and the method includes: obtaining a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, and the training sample data set and the test sample data set include: a performance index system of the remote sensing satellite, and an application effectiveness evaluation value of the remote sensing satellite corresponding to each performance index in the performance index system; modeling the training sample data set by presetting a Gaussian process regression model to convert the effectiveness evaluation problem of the remote sensing satellite into a Gaussian process regression problem, and performing hyperparameter learning processing on the modeled Gaussian process regression model to determine a target hyperparameter; based on the modeled Gaussian process regression model and the target hyperparameter, predicting the effectiveness index in the test sample data set to determine the application effectiveness prediction value of the remote sensing satellite, and performing confidence evaluation processing on the application effectiveness prediction value to determine a confidence evaluation result, so that when the test sample data sets are all within the range of the confidence evaluation result of the application effectiveness prediction value, the modeled Gaussian process regression model and the target hyperparameter are determined as the target application effectiveness evaluation model.

[0006] In one embodiment, the step of modeling the training sample data set includes: setting the various performance indicators of the remote sensing satellite in the training sample data set as one-dimensional training samples of a preset Gaussian process regression model, and setting the application performance evaluation value as the label of the multidimensional training sample; using the multidimensional training samples and their corresponding labels to perform modeling processing to regress the nonlinear dynamic relationship between the performance indicators and the application performance evaluation value, so that the sample data of the nonlinear relationship is mapped to the feature space and then converted into sample data of the linear relationship, and hyperparameter learning processing is performed.

[0007] In one embodiment, a hyperparameter learning process is performed on the modeled Gaussian process regression model to determine the target hyperparameters, including: modeling the relationship between any two training samples through a squared exponential covariance function to determine the covariance matrix; based on the covariance matrix, the hyperparameters of the modeled Gaussian process regression model are learned by maximizing the pseudo-log marginal likelihood, and the pseudo-log marginal likelihood is maximized using a preset gradient descent model, and the partial differential of the log marginal likelihood with respect to each hyperparameter is calculated to determine the target hyperparameters.

[0008] In one embodiment, after the step of determining the target hyperparameters, the step includes: obtaining the scale factors corresponding to each dimension in the target hyperparameters, and performing application performance evaluation processing on the performance indicators according to the scale factors, performing quantitative analysis on the performance indicators, and determining the degree of influence of each performance indicator on the application performance. The smaller the scale factor, the greater the influence of the corresponding performance indicator on the application performance.

[0009] In one embodiment, the confidence assessment process is performed on the application effectiveness prediction value, and the step of determining the confidence assessment result includes: determining the mean and variance of the remote sensing satellite application effectiveness evaluation based on the application effectiveness prediction value through the modeled Gaussian process regression model and target hyperparameters; and performing confidence assessment process based on the mean and variance of the remote sensing satellite application effectiveness evaluation through a preset confidence calculation model to determine the confidence assessment result.

[0010] In a second aspect, an embodiment of the present invention further provides an application performance evaluation method, the method comprising: obtaining the application performance to be evaluated of the remote sensing satellite; inputting the performance indicator corresponding to the application performance to be evaluated into a pre-constructed target application performance evaluation model, determining the application performance evaluation result of the remote sensing satellite mission, and feeding back the application performance evaluation result to the user end; wherein, the target application performance evaluation model is constructed based on the construction method of the application performance evaluation model of any one of the first aspects.

[0011] In a third aspect, an embodiment of the present invention further provides a device for constructing an application performance evaluation model, which is applied to a remote sensing satellite experiment, and the device includes: a sample data acquisition module, which obtains a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, and the training sample data set and the test sample data set include: a performance index system of the remote sensing satellite, and an application performance evaluation value of the remote sensing satellite corresponding to each performance index in the performance index system; a hyperparameter learning module, which models the training sample data set through a preset Gaussian process regression model to convert the performance evaluation value of the remote sensing satellite into a real-time performance evaluation value. The problem is converted into a Gaussian process regression problem, and the hyperparameter learning processing is performed on the Gaussian process regression model after modeling to determine the target hyperparameters; the model training module predicts the performance indicators in the test sample data set based on the Gaussian process regression model after modeling and the target hyperparameters, determines the application performance prediction value of the remote sensing satellite, and performs confidence assessment processing on the application performance prediction value to determine the confidence assessment result, so that when the test sample data sets are all within the range of the confidence assessment result of the application performance prediction value, the Gaussian process regression model after modeling and the target hyperparameters are determined as the target application performance evaluation model.

[0012] In a fourth aspect, an embodiment of the present invention further provides an application performance evaluation device, the device comprising: a data acquisition module, for acquiring the application performance to be evaluated of the remote sensing satellite; a performance evaluation module, for inputting the performance indicator corresponding to the application performance to be evaluated into a pre-constructed target application performance evaluation model, determining the application performance evaluation result of the remote sensing satellite mission, and feeding back the application performance evaluation result to the user end; wherein, the target application performance evaluation model is constructed based on the construction method of the application performance evaluation model of any one of the first aspects.

[0013] In a fifth aspect, an embodiment of the present invention further provides a server, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0014] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0015] The embodiments of the present invention bring the following beneficial effects:

[0016] The embodiment of the present invention provides a remote sensing satellite application efficiency evaluation method, device and server. After obtaining a sample data set, the method performs modeling processing on the training sample data set by presetting a Gaussian process regression model to convert the remote sensing satellite efficiency evaluation problem into a Gaussian process regression problem, and performs hyperparameter learning processing on the Gaussian process regression model after modeling to determine the target hyperparameter. Finally, based on the Gaussian process regression model after modeling and the target hyperparameter, the efficiency index is predicted to determine the application efficiency prediction value of the remote sensing satellite, and the confidence evaluation processing is performed on the application efficiency prediction value to determine the confidence evaluation result. When the actual value of the application efficiency is within the range of the confidence evaluation result of the application efficiency prediction value, the Gaussian process regression model after modeling and the target hyperparameter are determined as the target application efficiency evaluation model. The embodiment of the present invention can well adapt to the nonlinear and dynamic characteristics of efficiency evaluation, and can give the confidence of efficiency evaluation when accurately evaluating the remote sensing satellite efficiency. At the same time, it can also quantitatively analyze the importance of each efficiency index, thereby significantly improving the accuracy of remote sensing satellite application efficiency evaluation.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a method for constructing an application effectiveness evaluation model provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a remote sensing satellite application efficiency index system provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of another remote sensing satellite application efficiency indicator system provided by an embodiment of the present invention;

[0023] Figure 4A schematic diagram of a flow chart of an application performance evaluation method provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of a device for constructing an application effectiveness evaluation model provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of the structure of an application effectiveness evaluation device provided by an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] At present, with the advancement of science and technology, the composition and functions of remote sensing satellites are becoming more and more complex, and the uncertainty of completing remote sensing tasks is also increasing. Under the influence of multiple factors, performance evaluation has nonlinear and dynamic changes, and it is difficult to evaluate it. At present, performance evaluation methods can be roughly divided into two categories: analytical method, comprehensive evaluation method and data-driven intelligent model method.

[0029] The analytical method is a quantitative method, which mainly calculates system performance based on the clear mathematical relationship between performance indicators and performance. The most commonly used method is the ADC method, which decomposes the system performance evaluation into the product of the availability vector, the credibility matrix and the capability vector from an analytical perspective.

[0030] The comprehensive evaluation method is a combination of qualitative and quantitative methods. It mainly conducts a comprehensive evaluation of performance based on the multi-attribute description of system complexity, including hierarchical analysis method, fuzzy comprehensive evaluation method, etc.

[0031] In recent years, artificial intelligence technology has gradually become a research hotspot for effectiveness evaluation. Intelligent model methods represented by structural equation models, combat loops, BP neural networks, support vector regression machines, and Bayesian networks have been gradually applied to remote sensing satellite effectiveness evaluation. These methods are generally data-driven, and use machine learning and other methods to obtain model parameter construction, which is then used for remote sensing satellite application effectiveness evaluation with high accuracy.

[0032] However, analytical methods represented by ADC need to construct a clear mathematical model between performance indicators and capabilities based on prior information, and then calculate the performance of remote sensing satellites in performing typical observation tasks based on the ADC model; hierarchical analysis method and fuzzy comprehensive evaluation method need to invite experts to compare the importance of each performance indicator based on experience to obtain the weight of each performance indicator in the linear model. The above methods require prior knowledge of the clear relationship between performance indicators and performance or the degree of influence on performance, and cannot adapt to the nonlinear and dynamic changes in remote sensing satellite performance; performance evaluation methods based on intelligent models, represented by structural equation models, combat loops, BP neural networks, support vector regression machines, and Bayesian networks, are data-driven. Although they can also solve the above problems well, they cannot directly derive the confidence level of performance evaluation, nor can they quantitatively analyze the degree of influence of each performance indicator on the performance of remote sensing satellites.

[0033] Based on this, the remote sensing satellite application performance evaluation method, device and server provided by the present invention can accurately evaluate the remote sensing satellite application performance, and can also provide the confidence interval of the remote sensing satellite application performance evaluation. In addition, the present invention is data-driven, and by learning the scale factor of the Gaussian process regression hyperparameter, it can quantitatively analyze the importance of each performance indicator on the remote sensing satellite application performance. And significantly improve the accuracy of remote sensing satellite application performance evaluation.

[0034] See also Figure 1 The flowchart of a method for constructing an application effectiveness evaluation model is shown, and the method mainly includes the following steps S102 to S106:

[0035] Step S102, obtaining a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, the training sample data set and the test sample data set include: a performance index system of a remote sensing satellite, and an application performance evaluation value of the remote sensing satellite corresponding to each performance index in the performance index system. In one embodiment, the application performance evaluation value of the remote sensing satellite directly depends on the test value of each performance index, wherein the test value range of each performance index is normalized to [0, 1], and the value range of the application performance is [0, 1]. In particular, the solution of the present invention is to directly obtain the sample data set as known data, so as to perform subsequent model training and construct an application performance evaluation model. The sample data set can be an expert evaluation value or data corresponding to historical remote sensing satellite performance indicators.

[0036] Step S104, by presetting the Gaussian process regression model, the training sample data set is modeled to convert the remote sensing satellite performance evaluation problem into a Gaussian process regression problem, and the modeled Gaussian process regression model is subjected to hyperparameter learning to determine the target hyperparameters. In one embodiment, the Gaussian process regression model can be used to model the nonlinear and dynamic relationship between the remote sensing satellite application performance and performance indicators. The Gaussian process is a combination of a set of random variables. Any finite number of random variables has a joint Gaussian distribution. It is a continuous Bayesian model without parameters, which provides a set of strong and robust bases for representing interrelated and uncertain data.

[0037] Step S106, based on the modeled Gaussian process regression model and the target hyperparameters, predict the performance indicators in the test sample data set, determine the application performance prediction value of the remote sensing satellite, and perform confidence assessment processing on the application performance prediction value to determine the confidence assessment result. When the test sample data sets are all within the range of the confidence assessment result of the application performance prediction value, the modeled Gaussian process regression model and the target hyperparameters are determined as the target application performance evaluation model. In one embodiment, the remote sensing satellite application performance predicted by the Gaussian process regression method can be compared with the actual application performance. If the comparison result shows that for each test sample, the marked true value of the remote sensing satellite application performance is always within 3 of the predicted mean If the confidence level is within 99.6%, it means that the confidence level of using Gaussian process regression to evaluate the application effectiveness of remote sensing satellites is high and the model training is completed.

[0038] The method for constructing the above-mentioned application effectiveness evaluation model provided by the embodiment of the present invention can adapt to the nonlinear characteristics of remote sensing satellite application effectiveness evaluation by constructing a target application effectiveness evaluation model, and significantly improve the accuracy of remote sensing satellite application effectiveness evaluation.

[0039] The present invention also provides an implementation method of a method for constructing a sample data set, which is described in detail in (1) to (2) below:

[0040] (1) Constructing an efficiency indicator system, see Figure 2 The schematic diagram of a remote sensing satellite application effectiveness index system is shown in FIG. For a remote sensing satellite to perform a certain observation mission, combined with the mission profile and the key application problems to be solved, starting from the needs of users, directly related indicators are selected to construct an application effectiveness index system. Figure 3 The schematic diagram of another remote sensing satellite application effectiveness indicator system shown is an example of any remote sensing satellite application effectiveness indicator system.

[0041] (2) Domain experts mark evaluation samples. Domain experts are invited to mark the remote sensing satellite application effectiveness evaluation values ​​according to the evaluation values ​​of each performance indicator, so as to obtain a complete sample. In order to avoid loss of generality, it is assumed that the remote sensing satellite application effectiveness indicator system contains d performance indicators.

[0042] Each domain expert labels n samples. For performance index i, a uniform distribution is used to randomly select A number with a value between [0, 1] , forming a column vector :

[0043]

[0044] The column vectors of d performance indicators randomly selected form the matrix E:

[0045]

[0046] Among them, the kth row of matrix E Represents a combination of d performance index test values, as a training sample. At this time, the domain experts give the evaluation value of the remote sensing satellite application efficiency at this time based on this combination and experience. , the application performance evaluation values ​​of all n training sample labels form a column vector c:

[0047]

[0048] The embodiment of the present invention also provides an implementation method of remote sensing satellite application effectiveness evaluation, see the following (1) to (3) for details:

[0049] (1) When learning the hyperparameters of the Gaussian process regression model, it is first necessary to construct a Gaussian process regression model. The Gaussian process regression model sets the various performance indicators of the remote sensing satellite in the training sample data set as one-dimensional training samples of the preset Gaussian process regression model, and sets the application performance evaluation value as the label of the multidimensional training sample. The multidimensional training samples and their corresponding labels are used for modeling processing to regress the nonlinear dynamic relationship between the performance indicator and the application performance evaluation value, so that the sample data of the nonlinear relationship is mapped to the feature space and then converted into the sample data of the linear relationship, and the hyperparameter learning processing is performed. Specifically, the Gaussian process is completely composed of its mean function and covariance function definition:

[0050]

[0051] The Gaussian process can be defined as:

[0052] ~

[0053] Given a training set consisting of n observation samples:

[0054]

[0055] in, , is the corresponding observation value, i.e., label. According to the definition of Gaussian process, the training set output and contains Test set of test points Output on Obey the joint Gaussian distribution and satisfy:

[0056]

[0057] in, for A positive definite symmetric matrix of order , any element of which The training samples are measured and relevance; express The covariance matrix of order, each element of which is used to describe the spatial relationship between each training sample and test sample; Since the spatial relationship between the test samples is described, A positive definite symmetric matrix of order .

[0058] At a given test point Under the conditions of and training set D, according to the Bayesian posterior probability formula:

[0059] ~

[0060] in:

[0061]

[0062] Since the Gaussian process covariance function is positive definite on a finite input set and is a symmetric function that satisfies the Mercer condition, the covariance function is equivalent to the kernel function. The test point The mean formula of the predicted values ​​at is rewritten as:

[0063]

[0064] in, , test point The predicted values ​​at are all kernel functions The linear combination of nonlinear relationships can be mapped to feature space and then converted into linear relationships, so that complex nonlinear problems can be converted into linear problems.

[0065] For two samples , covariance function To model the relationship between two samples, the present invention uses a squared exponential covariance function:

[0066]

[0067] in, It is the scale factor of each dimension. The smaller its value is, the greater the correlation between the input and output of this dimension is, that is, the greater the impact on the output is. Represent the signal variance and the noise variance respectively. The signal variance Used to control the degree of local correlation, is the Kronecker symbol.

[0068] (2) The relationship between any two training samples is modeled through the squared exponential covariance function to determine the covariance matrix. Based on the covariance matrix, the hyperparameters of the modeled Gaussian process regression model are learned by maximizing the pseudo-log marginal likelihood. The pseudo-log marginal likelihood is maximized using the preset gradient descent model, and the partial differential of the log marginal likelihood with respect to each hyperparameter is calculated to determine the target hyperparameters.

[0069] In one embodiment, given a training set , all training samples marked by the mth domain expert can be expressed as ,in , , represents the number of samples marked by the mth domain expert, , the corresponding application effectiveness of expert marking is Since the training samples given by different experts are independent, the hyperparameters are learned by maximizing the pseudo-log marginal likelihood.

[0070]

[0071] in, Represents the covariance matrix of the noisy observations (application performance evaluation values) labeled by the mth expert, and each element can be calculated by the squared exponential covariance function.

[0072] Gradient descent is used to maximize the pseudo-log marginal likelihood, calculating the pseudo-log marginal likelihood with respect to each hyperparameter The partial differential of is as follows:

[0073]

[0074]

[0075] in, represents the trace of the matrix, ,and Represents the partial differential of each element in the covariance matrix relative to the hyperparameters. The gradient-based optimization algorithm can solve the above optimization problem and obtain all the hyperparameters , and bring in the squared exponential covariance function for subsequent remote sensing satellite application effectiveness evaluation.

[0076] By learning the scale factor of the Gaussian process regression hyperparameter through the above formula, the importance of each performance indicator on the application performance of remote sensing satellites can be quantitatively analyzed. Among them, the smaller the scale factor corresponding to the performance indicator, the greater the impact of the indicator on the application performance of remote sensing satellites.

[0077] (3) Evaluate application performance and calculate confidence, obtain the scale factor corresponding to each dimension in the target hyperparameter, and perform application performance evaluation on the performance indicator according to the scale factor, perform quantitative analysis on the performance indicator, and determine the degree of influence of each performance indicator on the application performance. The smaller the scale factor, the greater the influence of the corresponding performance indicator on the application performance. The mean and variance of the remote sensing satellite application performance evaluation are determined based on the application performance prediction value through the modeled Gaussian process regression model and the target hyperparameter. The confidence evaluation is performed based on the mean and variance of the remote sensing satellite application performance evaluation through the preset confidence calculation model to determine the confidence evaluation result. In one embodiment, after the actual observation mission of the remote sensing satellite, the normalized evaluation results of the d indicators obtained from the actual observation mission are obtained. Based on the hyperparameters obtained through learning, the Gaussian process regression model can be used to calculate the mean c and variance of the remote sensing satellite application performance evaluation. :

[0078]

[0079] in, .

[0080] In obtaining the mean c and variance of remote sensing satellite application performance evaluation After that, the application performance evaluation value is in the range The confidence level can be calculated by the following formula:

[0081]

[0082] Generally speaking, the application efficiency of remote sensing satellites is The confidence level between The confidence level between The confidence level is 99.6%. The above method can not only accurately evaluate the application effectiveness of remote sensing satellites, but also provide a confidence interval for the evaluation of the application effectiveness of remote sensing satellites.

[0083] See also Figure 4 The flowchart of an application performance evaluation method shown in FIG. 4 mainly includes the following steps S402 to S404:

[0084] Step S402, obtaining the application performance to be evaluated of the remote sensing satellite.

[0085] Step S404, input the performance indicators corresponding to the application performance to be evaluated into a pre-constructed target application performance evaluation model, determine the application performance evaluation results of the remote sensing satellite mission, and feed back the application performance evaluation results to the user end; wherein, the target application performance evaluation model is constructed based on any of the above-mentioned application performance evaluation model construction methods.

[0086] The application effectiveness evaluation method provided by the embodiment of the present invention adapts to the nonlinear characteristics of remote sensing satellite application effectiveness evaluation and significantly improves the accuracy of remote sensing satellite application effectiveness evaluation.

[0087] In actual applications, when evaluating the basic data for performance, if a remote sensing satellite conducts application performance evaluation, an application performance indicator system containing 9 performance indicators is constructed, that is, the training samples include 9 dimensions. The project team invited 4 front-line experts to mark the training samples. Each expert scored the application performance of the remote sensing satellite based on 50 different values ​​of the 9 performance indicators (normalized to between 0 and 1). Finally, 200 different samples were formed. Due to the different professional backgrounds of each expert, the four experts had different perspectives and scales for evaluating the application performance of remote sensing satellites, so there were certain differences between the samples in the groups.

[0088] Furthermore, 45 samples were randomly selected from the 50 samples given by each expert as training samples, and the remaining 5 samples were used as test samples, with a total of 180 training samples and 20 test samples, so as to conduct an importance analysis of performance indicators. The hyperparameters of Gaussian regression were learned by maximizing pseudo-log marginal likelihood, as follows:

[0089]

[0090] Judging from the scale factors in the hyperparameters obtained through learning, the application effectiveness of remote sensing satellites is less affected by changes in the 1st, 2nd and 5th performance indicators, but is greatly affected by changes in the 6th and 7th performance indicators. Among the 9 performance indicators, the 7th indicator has the greatest impact on the application effectiveness of remote sensing satellites, and the 1st indicator has the smallest impact. When evaluating the application effectiveness of remote sensing satellites, special attention should be paid to the testing of the 6th and 7th performance indicators. The test values ​​of these two performance indicators change slightly, which has a great impact on the evaluation value of the application effectiveness of remote sensing satellites.

[0091] Afterwards, the remote sensing satellite application effectiveness predicted by the Gaussian process regression method was compared with the actual application effectiveness marked by the experts for 4 experts, each with 5 test samples. The application effectiveness marked by the experts was the true value, which was compared with the mean and 3 values ​​of the application effectiveness of remote sensing satellites evaluated by Gaussian process regression. The comparison results show that for each test sample, the true value of the remote sensing satellite application efficiency is always within 3 of the predicted mean. (99.6% confidence level), which means that the confidence level of using Gaussian process regression to evaluate the effectiveness of remote sensing satellite applications is very high.

[0092] Finally, in order to more accurately reflect the accuracy of using Gaussian process regression for remote sensing satellite application effectiveness evaluation, the present invention compares the evaluation results with the evaluation results obtained by four commonly used algorithms, namely, BP neural network, IGWO-SVR (support vector regression based on improved grey wolf optimization algorithm), Mesh-SVR (support vector regression based on mesh method), and SVR: 5 test samples are randomly selected, and the IGWO-SVR algorithm is used, and the mean square error MSE is 0.10909; the BP neural network algorithm is used, and the mean square error MSE is 0.08309; the SVR algorithm is used, and the mean square error MSE is 0.054602; the Mesh-SVR algorithm is used, and the mean square error MSE is 0.04836; the Gaussian process regression is used; the root mean square error is 0.012738. Therefore, it can be seen that the Gaussian process regression is used for equipment application effectiveness evaluation with the smallest error.

[0093] In summary, the present invention first constructs an efficiency index system for remote sensing satellites to perform typical observation tasks; then, the application efficiency evaluation problem is modeled as a Gaussian process regression problem, each efficiency index is used as one dimension of a training sample, and the application efficiency evaluation value is used as a label of the corresponding sample; before the remote sensing satellite performs the observation task, domain experts give labels (application efficiency evaluation values) of multiple training samples (each efficiency index randomly takes a value within a value range) based on their experience, and learn the hyperparameters of the Gaussian process regression model; then, according to the scale factor in the hyperparameter, the influence of each efficiency index on the application efficiency is quantitatively analyzed; finally, through actual observation tasks, after obtaining the test value of each efficiency index, the Gaussian process regression model is used to predict the mean and standard deviation of the remote sensing satellite application efficiency, which is used to evaluate the application efficiency and its confidence. Therefore, the present invention can adapt to the nonlinear characteristics of remote sensing satellite application efficiency evaluation and significantly improve the accuracy of remote sensing satellite application efficiency evaluation.

[0094] For the method for constructing the application effectiveness evaluation model provided in the above-mentioned embodiment, an embodiment of the present invention provides a device for constructing the application effectiveness evaluation model, which is applied to remote sensing satellite experiments, see Figure 5 The schematic diagram of the structure of a device for constructing an application effectiveness evaluation model is shown, and the device includes the following parts:

[0095] The sample data collection module 502 acquires a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, and the training sample data set and the test sample data set include: an efficiency index system of a remote sensing satellite, and an application efficiency evaluation value of the remote sensing satellite corresponding to each efficiency index in the efficiency index system;

[0096] The hyperparameter learning module 504 performs modeling processing on the training sample data set by presetting the Gaussian process regression model, so as to convert the remote sensing satellite effectiveness evaluation problem into a Gaussian process regression problem, and performs hyperparameter learning processing on the Gaussian process regression model after modeling to determine the target hyperparameter;

[0097] The model training module 506 predicts the performance indicators in the test sample data set based on the modeled Gaussian process regression model and target hyperparameters, determines the application performance prediction value of the remote sensing satellite, and performs confidence assessment processing on the application performance prediction value to determine the confidence assessment result. When the test sample data sets are all within the range of the confidence assessment result of the application performance prediction value, the modeled Gaussian process regression model and target hyperparameters are determined as the target application performance evaluation model.

[0098] The device for constructing the above-mentioned application performance evaluation model provided in the embodiment of the present application can adapt to the nonlinear characteristics of remote sensing satellite application performance evaluation by constructing a target application performance evaluation model, and significantly improve the accuracy of remote sensing satellite application performance evaluation.

[0099] In one embodiment, when performing the step of modeling the training sample data set, the above-mentioned hyperparameter learning module 504 is also used to: set the various performance indicators of the remote sensing satellite in the training sample data set as one-dimensional training samples of a preset Gaussian process regression model, and set the application performance evaluation value as the label of the multidimensional training sample; use the multidimensional training samples and their corresponding labels to perform modeling processing to regress the nonlinear dynamic relationship between the performance indicators and the application performance evaluation value, so that the sample data of the nonlinear relationship is mapped to the feature space and then converted into sample data of the linear relationship, and perform hyperparameter learning processing.

[0100] In one embodiment, when performing hyperparameter learning processing on the modeled Gaussian process regression model and determining the target hyperparameters, the above-mentioned hyperparameter learning module 504 is also used to: model the relationship between any two training samples through the square exponential covariance function to determine the covariance matrix; based on the covariance matrix, learn the hyperparameters of the modeled Gaussian process regression model by maximizing the pseudo-log marginal likelihood, and use the preset gradient descent model to maximize the pseudo-log marginal likelihood, calculate the partial differential of the log marginal likelihood with respect to each hyperparameter, and determine the target hyperparameters.

[0101] In one embodiment, after the step of determining the target hyperparameters, the hyperparameter learning module 504 is further used to: obtain the scale factors corresponding to each dimension in the target hyperparameters, and perform application performance evaluation processing on the performance indicators according to the scale factors, perform quantitative analysis on the performance indicators, and determine the degree of influence of each performance indicator on the application performance. The smaller the scale factor, the greater the influence of the corresponding performance indicator on the application performance.

[0102] In one embodiment, when performing confidence assessment processing on the application effectiveness prediction value and determining the confidence assessment result, the above-mentioned model training module 506 is also used to: determine the mean and variance of the remote sensing satellite application effectiveness evaluation based on the application effectiveness prediction value through the modeled Gaussian process regression model and target hyperparameters; perform confidence assessment processing based on the mean and variance of the remote sensing satellite application effectiveness evaluation through a preset confidence calculation model to determine the confidence assessment result.

[0103] For the application performance evaluation method provided in the above embodiment, the present invention provides an application performance evaluation device, see Figure 6 The schematic diagram of the structure of an application performance evaluation device shown in FIG. 1 includes the following parts:

[0104] The data acquisition module 602 acquires the application performance to be evaluated of the remote sensing satellite;

[0105] The performance evaluation module 604 inputs the performance indicators corresponding to the application performance to be evaluated into a pre-constructed target application performance evaluation model, determines the application performance evaluation results of the remote sensing satellite mission, and feeds back the application performance evaluation results to the user end; wherein, the target application performance evaluation model is constructed based on any of the above-mentioned application performance evaluation model construction methods.

[0106] The above-mentioned application performance evaluation device provided in the embodiment of the present application can utilize the target application performance evaluation model to adapt to the nonlinear characteristics of remote sensing satellite application performance evaluation and significantly improve the accuracy of remote sensing satellite application performance evaluation.

[0107] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0108] An embodiment of the present invention provides a server. Specifically, the server includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.

[0109] Figure 7 A structural diagram of a server provided in an embodiment of the present invention, the server 100 includes: a processor 70, a memory 71, a bus 72 and a communication interface 73, wherein the processor 70, the communication interface 73 and the memory 71 are connected via the bus 72; the processor 70 is used to execute an executable module stored in the memory 71, such as a computer program.

[0110] The memory 71 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0111] The bus 72 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0112] Among them, the memory 71 is used to store programs, and the processor 70 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 70 or implemented by the processor 70.

[0113] The processor 70 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 70. The above processor 70 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 71, and the processor 70 reads the information in the memory 71 and completes the steps of the above method in combination with its hardware.

[0114] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0115] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0116] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for constructing an application effectiveness evaluation model, characterized in that: The method is applied to remote sensing satellite experiments, and the method comprises: Acquire a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, and the training sample data set and the test sample data set include: an efficiency index system of a remote sensing satellite, and an application efficiency evaluation value of the remote sensing satellite corresponding to each efficiency index in the efficiency index system; By presetting a Gaussian process regression model, the training sample data set is modeled to convert the remote sensing satellite effectiveness evaluation problem into a Gaussian process regression problem, and a hyperparameter learning process is performed on the modeled Gaussian process regression model to determine the target hyperparameter; Based on the modeled Gaussian process regression model and the target hyperparameters, the performance indicators in the test sample data set are predicted to determine the application performance prediction value of the remote sensing satellite, and the application performance prediction value is subjected to confidence assessment processing to determine the confidence assessment result, so that when the test sample data set is within the range of the confidence assessment result of the application performance prediction value, the modeled Gaussian process regression model and the target hyperparameters are determined as the target application performance evaluation model; Among them, the step of modeling the training sample data set includes: setting each of the performance indicators of the remote sensing satellite in the training sample data set as a one-dimensional training sample of a preset Gaussian process regression model, and setting the application performance evaluation value as a label of a multidimensional training sample; using the multidimensional training samples and their corresponding labels to perform modeling processing, so as to regress the nonlinear dynamic relationship between the performance indicator and the application performance evaluation value, so that the sample data of the nonlinear relationship is mapped to the feature space and then converted into the sample data of the linear relationship, and perform hyperparameter learning processing; The Gaussian process is represented by its mean function and covariance function definition: The Gaussian process is defined as: ~ Among them, given a training set consisting of n observation samples: in, , is the corresponding observation value. According to the definition of Gaussian process, the training set output and contains Test set of test points Output on , the output obeys the joint Gaussian distribution and satisfies: in, for A positive definite symmetric matrix of order , any element of which The training samples are measured and relevance; express The covariance matrix of order, each element of which is used to describe the spatial relationship between each training sample and test sample; It is used to describe the spatial relationship between test samples. A positive definite symmetric matrix of order; Among them, at a given test point Under the conditions of and training set D, according to the Bayesian posterior probability formula: ~ in: Among them, the covariance function is equivalent to the kernel function, and the test point The mean formula of the predicted values ​​at is rewritten as: in, , test point The predicted values ​​at are all kernel functions The linear combination of nonlinear relationship is mapped to the feature space and then converted into linear relationship, so that complex nonlinear problems can be converted into linear problems; Among them, for two samples , through the covariance function Model the relationship between two samples: in, It is the scale factor of each dimension. The smaller its value is, the greater the correlation between the input and output of this dimension is. Represent the signal variance and the noise variance respectively. The signal variance Used to control the degree of local correlation, is the Kronecker symbol; Among them, the step of performing hyperparameter learning processing on the Gaussian process regression model after modeling and determining the target hyperparameters includes: modeling the relationship between any two training samples through a square exponential covariance function to determine the covariance matrix; based on the covariance matrix, learning the hyperparameters of the Gaussian process regression model after modeling by maximizing the pseudo-log marginal likelihood, and maximizing the pseudo-log marginal likelihood using a preset gradient descent model, calculating the partial differential of the log marginal likelihood with respect to each hyperparameter, and determining the target hyperparameters; Among them, given the training set , all training samples of the mth label can be expressed as ,in, , , represents the number of samples of the mth label, , indicating that the application efficiency of the mth mark is , the hyperparameters are learned by maximizing the pseudo-log marginal likelihood: in, represents the covariance matrix of the mth labeled noisy observations; Among them, the gradient descent method is used to maximize the pseudo-log marginal likelihood, and the pseudo-log marginal likelihood is calculated relative to each hyperparameter The partial derivative of : in, represents the trace of the matrix, ,and Represents the partial differential of each element in the covariance matrix relative to the hyperparameters, and solves it based on the gradient optimization algorithm to obtain all the hyperparameters , in order to bring the hyperparameters into the squared exponential covariance function for subsequent remote sensing satellite application performance evaluation.

2. The method for constructing an application effectiveness evaluation model according to claim 1, characterized in that: After the step of determining the target hyperparameter, the method further comprises: The scale factor corresponding to each dimension in the target hyperparameter is obtained, and the application performance evaluation processing is performed on the performance indicator according to the scale factor, and the performance indicator is quantitatively analyzed to determine the influence of each performance indicator on the application performance. The smaller the scale factor, the greater the influence of the corresponding performance indicator on the application performance.

3. The method for constructing an application effectiveness evaluation model according to claim 1, characterized in that: The step of performing confidence assessment processing on the application effectiveness prediction value to determine the confidence assessment result includes: Determine the mean and variance of the remote sensing satellite application effectiveness evaluation based on the application effectiveness prediction value by using the modeled Gaussian process regression model and the target hyperparameters; By presetting a confidence calculation model, confidence assessment processing is performed based on the mean and the variance of the remote sensing satellite application effectiveness assessment to determine the confidence assessment result.

4. An application performance evaluation method, characterized in that: The method comprises: Obtain the application performance of remote sensing satellites to be evaluated; Inputting the performance index corresponding to the application performance to be evaluated into a pre-built target application performance evaluation model, determining the application performance evaluation result of the remote sensing satellite mission, and feeding back the application performance evaluation result to the user end; Wherein, the target application effectiveness evaluation model is constructed based on the method for constructing an application effectiveness evaluation model according to any one of claims 1 to 3.

5. A device for constructing an application effectiveness evaluation model, characterized in that: The device is applied to remote sensing satellite experiments, and comprises: A sample data acquisition module is used to obtain a sample data set, wherein the sample data set includes: a training sample data set and a test sample data set, and the training sample data set and the test sample data set include: an efficiency index system of a remote sensing satellite, and an application efficiency evaluation value of the remote sensing satellite corresponding to each efficiency index in the efficiency index system; A hyperparameter learning module performs modeling processing on the training sample data set by presetting a Gaussian process regression model, so as to convert the remote sensing satellite effectiveness evaluation problem into a Gaussian process regression problem, and performs hyperparameter learning processing on the Gaussian process regression model after modeling to determine the target hyperparameter; A model training module predicts the performance index in the test sample data set based on the modeled Gaussian process regression model and the target hyperparameters, determines the application performance prediction value of the remote sensing satellite, and performs confidence assessment processing on the application performance prediction value to determine the confidence assessment result, so that when the test sample data set is within the range of the confidence assessment result of the application performance prediction value, the modeled Gaussian process regression model and the target hyperparameters are determined as the target application performance evaluation model; Among them, the step of modeling the training sample data set includes: setting each of the performance indicators of the remote sensing satellite in the training sample data set as a one-dimensional training sample of a preset Gaussian process regression model, and setting the application performance evaluation value as a label of a multidimensional training sample; using the multidimensional training samples and their corresponding labels to perform modeling processing, so as to regress the nonlinear dynamic relationship between the performance indicator and the application performance evaluation value, so that the sample data of the nonlinear relationship is mapped to the feature space and then converted into the sample data of the linear relationship, and perform hyperparameter learning processing; The Gaussian process is represented by its mean function and covariance function definition: The Gaussian process is defined as: ~ Among them, given a training set consisting of n observation samples: in, , is the corresponding observation value. According to the definition of Gaussian process, the training set output and contains Test set of test points Output on , the output obeys the joint Gaussian distribution and satisfies: in, for A positive definite symmetric matrix of order , any element of which The training samples are measured and relevance; express The covariance matrix of order, each element of which is used to describe the spatial relationship between each training sample and test sample; It is used to describe the spatial relationship between test samples. A positive definite symmetric matrix of order; Among them, at a given test point Under the conditions of and training set D, according to the Bayesian posterior probability formula: ~ in: Among them, the covariance function is equivalent to the kernel function, and the test point The mean formula of the predicted values ​​at is rewritten as: in, , test point The predicted values ​​at are all kernel functions The linear combination of nonlinear relationship is mapped to the feature space and then converted into linear relationship, so that complex nonlinear problems can be converted into linear problems; Among them, for two samples , through the covariance function Model the relationship between two samples: in, It is the scale factor of each dimension. The smaller its value is, the greater the correlation between the input and output of this dimension is. Represent the signal variance and the noise variance respectively. The signal variance Used to control the degree of local correlation, is the Kronecker symbol; Among them, the step of performing hyperparameter learning processing on the Gaussian process regression model after modeling and determining the target hyperparameters includes: modeling the relationship between any two training samples through a square exponential covariance function to determine the covariance matrix; based on the covariance matrix, learning the hyperparameters of the Gaussian process regression model after modeling by maximizing the pseudo-log marginal likelihood, and maximizing the pseudo-log marginal likelihood using a preset gradient descent model, calculating the partial differential of the log marginal likelihood with respect to each hyperparameter, and determining the target hyperparameters; Among them, given the training set , all training samples of the mth label can be expressed as ,in, , , represents the number of samples of the mth label, , indicating that the application efficiency of the mth mark is , the hyperparameters are learned by maximizing the pseudo-log marginal likelihood: in, represents the covariance matrix of the mth labeled noisy observations; Among them, the gradient descent method is used to maximize the pseudo-log marginal likelihood, and the pseudo-log marginal likelihood is calculated relative to each hyperparameter The partial derivative of : in, represents the trace of the matrix, ,and Represents the partial differential of each element in the covariance matrix relative to the hyperparameters, and the gradient-based optimization algorithm is used to obtain all the hyperparameters , in order to bring the hyperparameters into the squared exponential covariance function for subsequent remote sensing satellite application performance evaluation.

6. An application performance evaluation device, characterized in that: The device comprises: Data acquisition module, which obtains the application performance to be evaluated of remote sensing satellites; The performance evaluation module inputs the performance index corresponding to the application performance to be evaluated into a pre-built target application performance evaluation model, determines the application performance evaluation result of the remote sensing satellite mission, and feeds back the application performance evaluation result to the user end; Wherein, the target application effectiveness evaluation model is constructed based on the method for constructing an application effectiveness evaluation model according to any one of claims 1 to 3.

7. A server, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 3.

Images