A method, device and equipment for evaluating the adequacy of test samples

Abstract

The present application relates to a test sample adequacy evaluation method, device and equipment. The method includes: calculating the current best index value; randomly extracting multiple sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated from the unmanned aerial vehicle equipment test space, replacing a part of them with the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and adding the remaining part to the unmanned aerial vehicle equipment test sample set to be evaluated, calculating the current adequacy unmanned aerial vehicle equipment evaluation index value of the new unmanned aerial vehicle equipment test sample set, using the current adequacy unmanned aerial vehicle equipment evaluation index value and the adequacy unmanned aerial vehicle equipment evaluation index value of the unmanned aerial vehicle equipment test sample set to be evaluated to calculate the index improvement ratio, and save the corresponding improved unmanned aerial vehicle equipment test sample set. The use of this method can improve the accuracy of unmanned aerial vehicle equipment performance testing.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicle equipment testing, and in particular to a test sample adequacy evaluation method, device and equipment. Background Art

[0002] The purpose of the UAV equipment test sample adequacy evaluation is to answer whether the established UAV equipment test sample set can achieve the test purpose. In UAV equipment testing, accuracy is crucial. For example, when testing the flight stability of a UAV, multiple factors need to be considered, including different meteorological conditions (such as breeze, strong wind, wind direction change, etc.) and different load conditions (such as carrying equipment of different weights). If the sample is insufficient, the flight stability in calm weather and light load may only be tested, while the situation in complex weather and heavy load is ignored. Through sample adequacy evaluation, it is necessary to ensure that the sample set can cover various factors that may affect flight stability, so as to accurately draw conclusions about the flight stability of the UAV in actual use scenarios to achieve the purpose of the test. Sample adequacy evaluation is to ensure that the sample set contains enough test samples of different flight modes and different geographical environments (such as plains, mountains, sea, etc.), so that the endurance data obtained can represent the actual endurance of the UAV in various actual application scenarios, so as to achieve the test purpose of accurately evaluating the endurance of the UAV. The purpose of sample adequacy evaluation is to ensure that the sample set contains a variety of situations to fully verify the reliability and safety of the obstacle avoidance system. Only in this way can we determine whether the obstacle avoidance system of the drone can work effectively when encountering various potentially dangerous situations, thereby achieving the test purpose of evaluating the safety performance of the drone.

[0003] In the process of UAV equipment testing, according to the timing of the evaluation, it can be divided into pre-test adequacy evaluation and post-test adequacy evaluation; according to the size of the UAV equipment test space, it can be divided into conditional fixed test adequacy evaluation and conditional non-fixed test adequacy evaluation. For conditional fixed tests, the sample adequacy evaluation problem is equivalent to the sample size determination problem, and there are a lot of studies at home and abroad; it is mainly based on the accuracy of statistical inference and resource constraints, such as determining the sample size based on the accuracy of parameter estimation, or determining the sample size based on the two types of risks of hypothesis testing. For conditional non-fixed tests, the sample size and the distribution of sample points jointly determine the adequacy of the test. Existing methods generally split the sample size determination and sample point selection into two problems: first, experts in related fields determine the test sample size based on test resources, and then statisticians use specific test design methods to provide a sample set for UAV equipment tests.

[0004] Existing methods for determining sample size based on statistical inference accuracy all assume that the test data comes from a single population, which is obviously not suitable for the adequacy evaluation of tests with unfixed conditions. The stopping criteria of sequential tests are usually based on the response model established based on the existing test results. However, there is no test data before the test, and the response model cannot be established. Therefore, it is not suitable for the adequacy evaluation before the test. In the two-step method of first determining the sample size and then determining the sample points, the sample size and sample points are optimized separately, thus ignoring the coupling effect between the two, which reduces the accuracy of the UAV equipment performance test. Summary of the invention

[0005] Based on this, it is necessary to provide a test sample adequacy evaluation method, device and equipment that can be applicable to both fixed-condition tests and non-fixed-condition tests, both post-test evaluation and pre-test evaluation, to improve the accuracy of UAV equipment performance testing, in response to the above-mentioned technical problems.

[0006] A method for evaluating the adequacy of a test sample, the method comprising:

[0007] Given the UAV equipment test space and the UAV equipment test sample set to be evaluated, calculate the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated, and record the value as the current best index value;

[0008] A plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space; a portion of the plurality of sample points are replaced with the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set;

[0009] Calculate the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set, judge the current adequacy UAV equipment evaluation index value according to the recorded current best index value, if the current adequacy UAV equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, record the current adequacy UAV equipment evaluation index value as the best index value, use the current adequacy UAV equipment evaluation index value and the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, return the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set.

[0010] In one embodiment, the test sample set of the unmanned aerial vehicle equipment to be evaluated is , Indicates samples.

[0011] In one embodiment, a plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space, a portion of the plurality of sample points are used to replace the same number of samples randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set, including:

[0012] Randomly select from the UAV equipment test space those that do not belong to the UAV equipment test sample set to be evaluated. Different sample points , generating values ​​at Random number From the test sample set of unmanned aerial vehicle equipment to be evaluated Randomly select samples, replaced by , and Added to the UAV equipment test sample set, and obtained new UAV equipment test sample set of test sample points .

[0013] In one embodiment, calculating the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set includes:

[0014] The test factors in the UAV equipment test space are judged by type, the values ​​of the quantitative factors are normalized, and the normalized results are used to calculate the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set.

[0015] In one embodiment, Indicates the number of Any value of the experimental factor, express After normalization, if If the experimental factors are continuous numerical types, the method for normalizing them is

[0016] ;

[0017] in, and Respectively represent the first The lower and upper bounds of the range of values ​​of the factors; if the If the experimental factors are discrete values, the method to normalize them is

[0018] ;

[0019] in, and Respectively represent the first The maximum and minimum values ​​of the factors, Indicates the number of Half the smallest interval between all values ​​of a factor.

[0020] In one embodiment, the normalized result is used to calculate the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set, including:

[0021] Calculate the new UAV equipment test sample set The current adequacy evaluation index of UAV equipment is

[0022] ;

[0023] in, It represents the UAV equipment test space.

[0024] ;

[0025] Represents sample points and The distance between represents the number of quantitative factors, represents the total number of factors, and Represents sample points and No. The normalized value of the factor is is the Dirac function, when is equal to 1 when , and equal to 0 otherwise.

[0026] In one embodiment, the indicator improvement ratio is calculated using the current adequacy UAV equipment evaluation index value and the adequacy UAV equipment evaluation index value of the test sample set of the UAV equipment to be evaluated, including:

[0027] The improvement ratio of the index is calculated by using the current adequacy evaluation index value of UAV equipment and the adequacy evaluation index value of the UAV equipment test sample set to be evaluated:

[0028] ;

[0029] in, The evaluation index value of the UAV equipment representing the adequacy of the UAV equipment test sample set to be evaluated, Indicates the current adequacy of the UAV equipment evaluation index value, represents the test cost corresponding to the current adequacy UAV equipment evaluation index value, It represents the test cost corresponding to the evaluation index value of the adequacy of the test sample set of the UAV equipment to be evaluated.

[0030] A test sample adequacy evaluation device, the device comprising:

[0031] A new UAV equipment test sample set construction module is used to calculate the adequacy evaluation index value of the UAV equipment test sample set to be evaluated, given the UAV equipment test space and the UAV equipment test sample set to be evaluated, and record the value as the current best index value;

[0032] A plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space; a portion of the plurality of sample points are replaced with the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set;

[0033] The sufficiency evaluation module is used to calculate the current sufficiency UAV equipment evaluation index value of the new UAV equipment test sample set, and judge the current sufficiency UAV equipment evaluation index value according to the recorded current best index value. If the current sufficiency UAV equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, the current sufficiency UAV equipment evaluation index is recorded as the best index value, and the current sufficiency UAV equipment evaluation index and the sufficiency UAV equipment evaluation index of the UAV equipment test sample set to be evaluated are used to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set are returned.

[0034] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Given the UAV equipment test space and the UAV equipment test sample set to be evaluated, calculate the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated, and record the value as the current best index value;

[0036] A plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space; a portion of the plurality of sample points are replaced with the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set;

[0037] Calculate the current sufficiency UAV equipment evaluation index of the new UAV equipment test sample set, judge the current sufficiency UAV equipment evaluation index according to the recorded current best index value, if the current sufficiency UAV equipment evaluation index is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, record the current sufficiency UAV equipment evaluation index as the best index value, use the current sufficiency UAV equipment evaluation index and the sufficiency UAV equipment evaluation index of the UAV equipment test sample set to be evaluated to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, return the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set.

[0038] The above-mentioned test sample adequacy evaluation method, device and equipment first randomly extracts multiple sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated from the unmanned aerial vehicle equipment test space, replaces a corresponding number of randomly selected sample points in the sample set to be evaluated with a part of them, and adds the other part to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set; then calculates the current adequacy unmanned aerial vehicle equipment evaluation index of the new unmanned aerial vehicle equipment test sample set, and judges the current adequacy unmanned aerial vehicle equipment evaluation index according to the current best index value. If the current adequacy unmanned aerial vehicle equipment evaluation index is better than the best index value and the current evaluation times is not less than the preset evaluation times threshold, the current adequacy unmanned aerial vehicle equipment evaluation index is recorded as the best index value, and the current adequacy unmanned aerial vehicle equipment evaluation index and the adequacy unmanned aerial vehicle equipment evaluation index of the unmanned aerial vehicle equipment test sample set to be evaluated are used to calculate the index improvement ratio, and the corresponding improved unmanned aerial vehicle equipment test sample set is recorded. The method uses random replacement or addition to improve the UAV equipment test sample set to be evaluated multiple times, and returns the best improvement ratio of the index value and the corresponding best improved sample set; it realizes the separation of sufficiency evaluation indicators and sufficiency evaluation technology, and the integration of sufficiency evaluation and optimality evaluation of the UAV equipment test sample set, so that users can flexibly construct evaluation indicators according to the purpose of the test, and decide whether the UAV equipment test sample set to be evaluated is sufficient and whether a new UAV equipment test sample set is needed according to the improvement ratio; the method is applicable to both conditional fixed tests and conditional non-fixed tests, and to both post-test evaluation and pre-test evaluation, which improves the accuracy of UAV equipment performance testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a flow chart of a test sample adequacy evaluation method in an embodiment;

[0040] Figure 2 A flowchart of adequacy evaluation in one embodiment;

[0041] Figure 3 is a structural block diagram of a test sample adequacy evaluation device in one embodiment;

[0042] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In one embodiment, Figure 1 As shown, a method for evaluating the adequacy of a test sample is provided, comprising the following steps:

[0045] Step 102, given the UAV equipment test space and the UAV equipment test sample set to be evaluated, calculate the adequacy UAV equipment evaluation index of the UAV equipment test sample set to be evaluated, and record the value as the current best index value; randomly extract multiple sample points that do not belong to the UAV equipment test sample set to be evaluated from the UAV equipment test space; replace the same number of sample points randomly selected from the UAV equipment test sample set to be evaluated with a part of the multiple sample points, and add the remaining part to the UAV equipment test sample set to be evaluated, to obtain a new UAV equipment test sample set.

[0046] The UAV equipment test space is a space described by the value range (or level) of all test factors. The test factors include factors related to the physical characteristics of the UAV itself, factors related to flight performance, factors related to environmental factors, and factors related to mission load and function. Among them, factors related to the physical characteristics of the UAV itself include fuselage size, fuselage weight, and material characteristics; factors related to flight performance include flight speed, flight altitude, and endurance; factors related to environmental factors include meteorological conditions and geographical environment; and factors related to mission load and function include mission load weight, sensor type and performance, and communication system parameters. In the algorithm implementation, the data format shown in the example in Table 1 can be used to describe the UAV equipment test space. Each row represents a factor, the first column is the factor name, the second column is the factor type, and the third column is the value range or level of the factor. Factor names are named according to the physical characteristics of the factors and are described by strings. Factor types are limited to continuous values, discrete values, ordered values, and categorical values, which are described by four strings: "Continuous", "Discrete", "Ordinary", and "Categorical". If the factor is continuous, the third column gives its value range in the form of intervals. If the factor is discrete, the third column gives all its values ​​in the form of sets. If the factor is ordered, the third column gives all its possible levels in order from small to large. If the factor is categorical, the third column lists all its levels in any order. It is recommended to use non-negative integers to encode the levels of ordered and categorical factors.

[0047] Table 1

[0048]

[0049] The points in the UAV equipment test space are called sample points, the set of sample points (usually a finite set) is called the UAV equipment test sample set, and the number of sample points in the UAV equipment test sample set is called the sample size. In the algorithm implementation, the data format shown in Table 2 can be used to describe the UAV equipment test sample set. Each column represents the value of a factor in each sample. The first row is the factor name, and each of the remaining rows represents a sample point.

[0050] Table 2

[0051]

[0052] Suppose there are a total of factors, respectively ; If If a factor is not continuous, then assume that it has Level ;by Indicates the value range of each factor. represents the UAV equipment test space, assuming ; to collect represents the UAV equipment test sample set to be evaluated, Indicates the first Sample points.

[0053] The adequacy of UAV equipment evaluation index is a quantitative basis for measuring the adequacy of UAV equipment test sample set. Represents the sufficiency evaluation index of UAV equipment, which is a function defined on the UAV equipment test space and its finite subsets. "Sufficiency" is a relative concept. The present invention evaluates the sufficiency of the UAV equipment test sample set from the perspectives of "quality" and "quantity". "Quality" refers to the optimality of the UAV equipment test sample set, that is, for a certain UAV equipment evaluation index, whether there is a UAV equipment test sample set with the same sample size that is better than the UAV equipment test sample set. The UAV equipment evaluation index includes flight performance index, mission execution capability index, and reliability and safety index; "quantity" refers to the sufficiency of the UAV equipment test sample set, which is measured by the marginal benefit of adding one sample. Figure 2 As shown, firstly, the evaluation index of UAV equipment is , UAV equipment test space , Test sample set of unmanned aerial vehicle equipment to be evaluated , Evaluation level And the number of evaluations is the input. Evaluation order Indicates the number of changes or new sample points in the UAV equipment test sample set in each evaluation, and the number of evaluations is the number of iterations of the algorithm.

[0054] Figure 2 Assume that the evaluation index of UAV equipment The smaller the number, the larger the number is. The greater the sufficiency, the better. If the constructed index is the larger the better, it can be converted to the smaller the better by taking the opposite number. It can be defined by the user according to the usage scenario. If there are multiple sufficient UAV equipment evaluation indicators, they can also be combined into a comprehensive indicator by weighted average, geometric mean, maximum (or minimum) and other methods. In order to ensure the significance of adequacy evaluation, the evaluation indicators of UAV equipment should meet the following seven basic properties:

[0055] (1) Validity. The indicator can truly reflect the adequacy of the UAV equipment test sample set, be sensitive to the changes of sample points in the UAV equipment test sample set, be easy to understand, and have clear statistical or geometric meaning.

[0056] (2) Computability. The index value of the predetermined accuracy can be calculated in polynomial time.

[0057] (3) Boundedness. As the sample set approaches the UAV equipment test space, the UAV equipment evaluation index converges to a finite optimal value.

[0058] (4) Consistency. The change in the index value of adding sample points to the UAV equipment test sample set is monotonic, and the direction of change is consistent with the optimization direction of the index. That is, if adding sample points will lead to an increase in the index value, the larger the index, the better the adequacy of the UAV equipment test sample set; if adding sample points leads to a decrease in the index value, the smaller the index, the better the adequacy of the UAV equipment test sample set.

[0059] (5) Design the matrix to be invariant for row-column-row permutation. When regarded as a function on the UAV equipment test sample set, changing the order of rows and columns of the design matrix of the UAV equipment test sample set does not affect the value of the adequacy UAV equipment evaluation index.

[0060] (6) Invariance of permutation of classification factor levels. The level encoding of classification factors does not affect the value of the evaluation index of adequacy of UAV equipment. In other words, For categorical factors, or To encode, you can also replace any two levels of encoding numbers.

[0061] (7) Invariance of quantitative factors under linear transformation. In order to maintain linear invariance, the levels of all quantitative factors can be normalized first when defining the indicators.

[0062] As an example of an indicator, define the drone equipment test sample set Test space for drone equipment The minimum coverage radius is

[0063]

[0064] Obviously, the smaller the minimum coverage radius, the better the adequacy of the UAV equipment test sample set. The minimum coverage radius can be used as a pre-test evaluation, and it does not require the support of test results. The minimum coverage radius depends on the distance in the UAV equipment test space. If the UAV equipment test space contains both qualitative and quantitative factors, it is advisable to set quantitative factors and A qualitative factor, assuming The first factor is a quantitative factor, and the The factors are qualitative factors. Define any two sample points and The distance between

[0065]

[0066] Formula (2) assumes that the distances between any two different levels of each qualitative factor are equal. If there is expert knowledge to define different distances between different levels of a qualitative factor, Formula (2) can be modified, but it is necessary to ensure that the defined distances satisfy the non-negativity, symmetry and triangle inequality of the distance.

[0067] In order to maintain the invariance of the linear transformation of the quantitative factors, it is necessary to normalize the quantitative factor values ​​in the UAV equipment test space before calculating the adequacy of the UAV equipment evaluation index. If the type of a factor is "Continuous", then and They represent the first The lower and upper bounds of the range of factors can be arbitrarily set by the following linear transformation Change to between:

[0068]

[0069] If If the factor type is "Discrete", then

[0070] ;

[0071] They represent the maximum and minimum values ​​of this factor in the UAV equipment test space respectively.

[0072] ;

[0073] It represents half of the minimum interval between all possible values ​​of this factor in the UAV equipment test space. Any value of this factor can be transformed as follows Change to between:

[0074]

[0075] For a qualitative factor of type "Ordinary", in order to utilize the order relationship between the levels, the above two formulas can also be used to transform all its levels to between.

[0076] In one embodiment, the test sample set of the unmanned aerial vehicle equipment to be evaluated is , Indicates samples.

[0077] In one embodiment, a plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space, a portion of the plurality of sample points are used to replace the same number of samples randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set, including:

[0078] Randomly select the test sample set of the unmanned aerial vehicle equipment that does not belong to the unmanned aerial vehicle equipment to be evaluated from the unmanned aerial vehicle equipment test space of Different samples , generating values ​​at Random number from Randomly select sample points, replaced by , and Add to the UAV equipment test sample set to obtain a new UAV equipment test sample set This step requires the difference set Randomly select different sample points. When the drone is equipped with a test space is an infinite set, the difference set is still an infinite set. At this time, the following rejection sampling method can be used: Randomly select a sample point from (can be selected component by component), if the sample point does not belong to Accept, otherwise reject; until you get Different sample points .

[0079] Step 104, calculate the current sufficiency UAV equipment evaluation index value of the new UAV equipment test sample set, judge the current sufficiency UAV equipment evaluation index value according to the recorded current best index value, if the current sufficiency UAV equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, record the current sufficiency UAV equipment evaluation index value as the best index value, use the current sufficiency UAV equipment evaluation index value and the sufficiency UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, return the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set.

[0080] In one embodiment, calculating the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set includes:

[0081] The types of test factors in the UAV equipment test space are judged, the values ​​of quantitative factors are normalized according to the test factor types, and the normalized results are used to calculate the current adequacy UAV equipment evaluation index values ​​of the new UAV equipment test sample set.

[0082] In one embodiment, the test factors in the UAV equipment test space are judged by type. If the experimental factors are continuous values, then any value The normalization method is

[0083]

[0084] in, and They represent the first The lower and upper bounds of the range of values ​​of the factors, Indicates value The normalized value. If the experimental factors are discrete values, then any value The normalization method is

[0085]

[0086] in, and They represent the first The maximum and minimum values ​​of the factors, Indicates the number of drone equipment test spaces half of the smallest interval between all values ​​of a factor, Indicates value Normalized value.

[0087] In one embodiment, the normalized result is used to calculate the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set, including:

[0088] Calculate the new UAV equipment test sample set The current adequacy evaluation index of UAV equipment is

[0089] ;

[0090] in, It represents the UAV equipment test space.

[0091] ;

[0092] Represents sample points and The distance between represents the number of quantitative factors, represents the total number of factors, and Represents sample points and No. The normalized value of the factor is is the Dirac function, when is equal to 1 when , and equal to 0 otherwise.

[0093] In one embodiment, the indicator improvement ratio is calculated using the current adequacy UAV equipment evaluation index value and the adequacy UAV equipment evaluation index value of the test sample set of the UAV equipment to be evaluated, including:

[0094] The improvement ratio of the index is calculated by using the current adequacy evaluation index value of UAV equipment and the adequacy evaluation index value of the UAV equipment test sample set to be evaluated:

[0095] ;

[0096] in, The evaluation index value of the UAV equipment representing the adequacy of the UAV equipment test sample set to be evaluated, Indicates the current adequacy of the UAV equipment evaluation index value, represents the test cost corresponding to the current adequacy UAV equipment evaluation index value, It represents the test cost corresponding to the evaluation index value of the adequacy of the test sample set of the UAV equipment to be evaluated.

[0097] The above-mentioned test sample adequacy evaluation method first calculates the adequacy drone equipment evaluation index value of the drone equipment test sample set to be evaluated, and records it as the current best index value; then randomly extracts multiple sample points that do not belong to the drone equipment test sample set to be evaluated from the drone equipment test space, replaces the same number of sample points randomly selected from the sample set with a part of them, and adds the other part to the drone equipment test sample set to be evaluated; finally, calculates the current adequacy drone equipment evaluation index value of the new drone equipment test sample set, if the current adequacy drone equipment evaluation index value is better than the best index value and the current evaluation times is not less than the preset evaluation times threshold, records the current adequacy drone equipment evaluation index value as the best index value, and uses the current adequacy drone equipment evaluation index value and the adequacy drone equipment evaluation index value of the drone equipment test sample set to be evaluated to calculate the index improvement ratio and the corresponding improved drone equipment test sample set.

[0098] The sufficiency evaluation index is separated from the sufficiency evaluation technology, and a sufficiency evaluation method applicable to any evaluation index is provided, so that users can flexibly construct evaluation indicators according to the purpose of the test, and then use the method provided in this application to evaluate the sufficiency. This application gives the basic properties that the evaluation index should meet, and gives an example of the evaluation index (minimum coverage radius). Through the combination of random replacement and random addition of sample points, a comprehensive evaluation of the sufficiency and optimality of the test samples is achieved. The evaluation result is the improved UAV equipment test sample set and the improvement ratio. Users can decide whether the UAV equipment test sample set to be evaluated is sufficient and whether a new UAV equipment test sample set needs to be accepted based on the improvement ratio. For example, the user can set the improvement ratio threshold to , so if the improvement ratio returned by Algorithm 1 does not exceed 5%, it can be considered that the UAV equipment test sample set is optimal. The process of Algorithm 1 is as follows:

[0099]

[0100] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0101] In one embodiment, Figure 3 As shown, a test sample adequacy evaluation device is provided, including: a new UAV equipment test sample set construction module 302 and an adequacy evaluation module 304, wherein:

[0102] The new UAV equipment test sample set construction module 302 is used to calculate the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated given the UAV equipment test space and the UAV equipment test sample set to be evaluated, and record the value as the current best index value; randomly extract multiple sample points that do not belong to the UAV equipment test sample set to be evaluated from the UAV equipment test space; replace the same number of sample points randomly selected from the UAV equipment test sample set to be evaluated with a part of the multiple sample points, and add the remaining part to the UAV equipment test sample set to be evaluated, so as to obtain a new UAV equipment test sample set;

[0103] The sufficiency evaluation module 304 is used to calculate the current sufficiency drone equipment evaluation index value of the new drone equipment test sample set, and judge the current sufficiency drone equipment evaluation index value according to the recorded current best index value. If the current sufficiency drone equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, the current sufficiency drone equipment evaluation index value is recorded as the best index value, and the current sufficiency drone equipment evaluation index value and the sufficiency drone equipment evaluation index value of the drone equipment test sample set to be evaluated are used to calculate the index improvement ratio, and save the corresponding improved drone equipment test sample set; until the preset evaluation times are reached, the best improvement ratio of the index value and the corresponding best improved drone equipment test sample set are returned.

[0104] For the specific definition of a test sample adequacy evaluation device, please refer to the definition of a test sample adequacy evaluation method mentioned above, which will not be repeated here. Each module in the above-mentioned test sample adequacy evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0105] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a test sample adequacy evaluation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0106] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for evaluating the adequacy of a test sample, characterized in that: The method comprises: Given a UAV equipment test space and a UAV equipment test sample set to be evaluated, calculate the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated, and record the value as the current best index value; Randomly extract multiple sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated from the unmanned aerial vehicle equipment test space; replace the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated with a part of the multiple sample points, and add the remaining part to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set; Calculate the current sufficiency UAV equipment evaluation index value of the new UAV equipment test sample set, judge the current sufficiency UAV equipment evaluation index value according to the recorded current best index value, if the current sufficiency UAV equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, record the current sufficiency UAV equipment evaluation index value as the best index value, use the current sufficiency UAV equipment evaluation index value and the sufficiency UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, return the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set.

2. The method according to claim 1, characterized in that The test sample set of the unmanned aerial vehicle equipment to be evaluated is , Indicates samples.

3. The method according to claim 1, characterized in that A plurality of sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated are randomly selected from the unmanned aerial vehicle equipment test space, a portion of the plurality of sample points are used to replace the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated, and the remaining portion is added to the unmanned aerial vehicle equipment test sample set to be evaluated, to obtain a new unmanned aerial vehicle equipment test sample set, including: Randomly select from the UAV equipment test space the samples that do not belong to the UAV equipment test sample set to be evaluated. Different sample points , generating values ​​at Random number From the test sample set of unmanned aerial vehicle equipment to be evaluated Randomly select sample points, replaced by , and Added to the UAV equipment test sample set, and obtained new UAV equipment test sample set of sample points .

4. The method according to claim 1, characterized in that: Calculating the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set, including: The test factors in the test space are judged by type, the values ​​of the quantitative factors are normalized according to the types of the test factors, and the current adequacy UAV equipment evaluation index value of the new UAV equipment test sample set is calculated using the results of the normalization of the test factors.

5. The method according to claim 4, characterized in that The method further comprises: by Indicates the number of Any value of the experimental factor, express After normalization, if If the experimental factors are continuous numerical types, the method for normalizing them is in, and Respectively represent the first The lower and upper bounds of the range of values ​​of the factors; if the If the experimental factors are discrete values, the method to normalize them is in, and Respectively represent the first The maximum and minimum values ​​of the factors, Indicates the number of Half the smallest interval between all values ​​of a factor.

6. The method according to claim 4, characterized in that The method further comprises: Calculate the new UAV equipment test sample set The evaluation index of adequacy of UAV equipment is in, It represents the UAV equipment test space. Represents sample points and The distance between represents the number of quantitative factors, represents the total number of factors, and Represents sample points and No. The normalized value of the factor is is the Dirac function, when is equal to 1 when , otherwise it is equal to 0.

7. The method according to claim 1, characterized in that The indicator improvement ratio is calculated using the current adequacy UAV equipment evaluation index value and the adequacy UAV equipment evaluation index value of the test sample set of the UAV equipment to be evaluated, including: The improvement ratio of the index is calculated by using the current adequacy evaluation index value of UAV equipment and the adequacy evaluation index value of the UAV equipment test sample set to be evaluated: ； in, The evaluation index value of the UAV equipment representing the adequacy of the UAV equipment test sample set to be evaluated, Indicates the current adequacy of the UAV equipment evaluation index value, represents the test cost corresponding to the current adequacy UAV equipment evaluation index value, It represents the test cost corresponding to the evaluation index value of the adequacy of the test sample set of the UAV equipment to be evaluated.

8. A test sample adequacy evaluation device, characterized in that: The device comprises: A new UAV equipment test sample set construction module is used to calculate the adequacy UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated given the UAV equipment test space and the UAV equipment test sample set to be evaluated, and record the value as the current best index value; Randomly extract multiple sample points that do not belong to the unmanned aerial vehicle equipment test sample set to be evaluated from the unmanned aerial vehicle equipment test space; replace the same number of sample points randomly selected from the unmanned aerial vehicle equipment test sample set to be evaluated with a part of the multiple sample points, and add the remaining part to the unmanned aerial vehicle equipment test sample set to be evaluated, so as to obtain a new unmanned aerial vehicle equipment test sample set; The sufficiency evaluation module is used to calculate the current sufficiency UAV equipment evaluation index value of the new UAV equipment test sample set, and judge the current sufficiency UAV equipment evaluation index value according to the recorded current best index value. If the current sufficiency UAV equipment evaluation index value is better than the current best index value and the current evaluation times are not less than the preset evaluation times threshold, the current sufficiency UAV equipment evaluation index value is recorded as the best index value, and the current sufficiency UAV equipment evaluation index value and the sufficiency UAV equipment evaluation index value of the UAV equipment test sample set to be evaluated are used to calculate the index improvement ratio, and save the corresponding improved UAV equipment test sample set; until the preset evaluation times are reached, the best improvement ratio of the index value and the corresponding best improved UAV equipment test sample set are returned.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images