Beidou satellite navigation product performance detection method and system

Abstract

The invention provides a Beidou satellite navigation product performance detection method and system, and the method comprises the steps: obtaining a complete monitoring value sequence of a to-be-detected Beidou satellite navigation product in a plurality of performance dimensions, the complete monitoring numerical value sequence of each performance dimension comprises a monitoring numerical value at the current moment and a historical monitoring numerical value sequence before the current moment; for each performance dimension, calculating an abnormal deviation feature of the monitoring value of the performance dimension at the current moment relative to the complete monitoring value sequence; constructing an original feature vector of the Beidou satellite navigation product to be tested based on the abnormal deviation feature of each performance dimension; and inputting the original feature vector into a pre-trained scoring model to obtain a performance score, and determining a performance detection result of the Beidou satellite navigation product to be detected according to the size of the performance score. According to the method, the accuracy of Beidou satellite navigation product performance detection is improved based on extraction and analysis of the multi-dimensional abnormal deviation features.

Description

Technical Field

[0001] This invention relates to the field of product testing technology, and more specifically, to a method and system for testing the performance of BeiDou satellite navigation products. Background Technology

[0002] In the field of performance testing for BeiDou satellite navigation products, traditional methods typically rely on single performance indicators or simple threshold judgments, which are insufficient to comprehensively reflect the true performance status of the product. While existing technologies employ multi-dimensional monitoring methods, they often only perform simple statistical analysis on the raw monitoring data, ignoring the local fluctuation characteristics and extreme event information contained within the data, thus reducing the accuracy and reliability of the testing results. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for testing the performance of BeiDou satellite navigation products, so as to improve the above-mentioned problems.

[0004] To achieve the above objectives, this application provides the following technical solution: On the one hand, embodiments of this application provide a method for testing the performance of BeiDou satellite navigation products, the method comprising: Obtain complete monitoring value sequences for the BeiDou satellite navigation product under test across multiple performance dimensions. The complete monitoring value sequence for each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. For each performance dimension, calculate the abnormal deviation characteristics of the current monitoring value of that performance dimension relative to the complete monitoring value sequence; based on the abnormal deviation characteristics of each performance dimension, construct the original feature vector of the Beidou satellite navigation product under test. The original feature vector is input into a pre-trained scoring model to obtain a performance score. The performance test result of the BeiDou satellite navigation product under test is determined based on the magnitude of the performance score.

[0005] Secondly, this application provides a BeiDou satellite navigation product performance testing system, the system comprising: The acquisition module is used to acquire the complete monitoring value sequence of the Beidou satellite navigation product under test in multiple performance dimensions. The complete monitoring value sequence of each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. The module is used to calculate the abnormal deviation characteristics of the current monitoring value of each performance dimension relative to the complete monitoring value sequence for each performance dimension; and to construct the original feature vector of the Beidou satellite navigation product under test based on the abnormal deviation characteristics of each performance dimension. The scoring module is used to input the original feature vector into a pre-trained scoring model to obtain a performance score, and to determine the performance test result of the Beidou satellite navigation product under test based on the size of the performance score.

[0006] Thirdly, this application provides a performance testing device for BeiDou satellite navigation products, the device comprising a memory and a processor. The memory stores a computer program; the processor executes the computer program to implement the steps of the aforementioned BeiDou satellite navigation product performance testing method.

[0007] Fourthly, this application provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described BeiDou satellite navigation product performance testing method.

[0008] The beneficial effects of this invention are as follows: 1. This invention achieves a comprehensive characterization of the performance status of BeiDou satellite navigation products by constructing a multi-dimensional anomaly deviation feature system that includes a first deviation measure, a second deviation measure, a third deviation measure, and a fourth deviation measure. The four measures complement each other, effectively covering various anomaly modes from minor degradation to extreme failures, thus improving the comprehensiveness and accuracy of detection.

[0009] 2. The manual review and judgment process of the present invention improves the review efficiency while ensuring the accuracy of the judgment results by combining weighted descending sorting with weighted traversal with termination conditions.

[0010] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the performance testing method for Beidou satellite navigation products described in this embodiment of the invention; Figure 2 This is a schematic diagram of the structure of the Beidou satellite navigation product performance testing system described in this embodiment of the invention; Figure 3This is a schematic diagram of the structure of the Beidou satellite navigation product performance testing equipment described in this embodiment of the invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0014] It should be noted that similar reference numerals or letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0015] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for testing the performance of Beidou satellite navigation products, which includes steps S1, S2 and S3.

[0016] Step S1: Obtain the complete monitoring value sequence of the Beidou satellite navigation product under test in multiple performance dimensions. The complete monitoring value sequence of each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. In this step, the BeiDou satellite navigation product under test is put into operation, and then a complete monitoring value sequence is obtained; performance dimensions may include, for example, horizontal positioning error, vertical positioning error, etc.

[0017] For each performance dimension, a fixed-length time window is constructed (e.g., it can be set to include data from the most recent 2 hours). The data stored within this window constitutes the complete monitoring value sequence for that performance dimension. This complete monitoring value sequence allows the system to simultaneously monitor the latest real-time state changes and historical performance over a past period.

[0018] Step S2: For each performance dimension, calculate the abnormal deviation characteristics of the current monitoring value of that performance dimension relative to the complete monitoring value sequence; based on the abnormal deviation characteristics of each performance dimension, construct the original feature vector of the Beidou satellite navigation product under test. In this step, the specific implementation steps for calculating the abnormal deviation characteristics of the current monitoring value of this performance dimension relative to the complete monitoring value sequence include step S21; Step S21: Calculate the standard deviation and interquartile range of the historical monitoring numerical sequence, then determine the bandwidth parameter for kernel density estimation according to the Silverman rule, use the bandwidth parameter to perform kernel density estimation on the historical monitoring numerical sequence, fit the first statistical distribution function, substitute the monitoring value at the current moment into the first statistical distribution function, calculate the cumulative probability value, perform a logarithmic transformation on the cumulative probability value to obtain the first deviation measure; calculate the moving average sequence for the complete monitoring numerical sequence, calculate the absolute difference between the complete monitoring numerical sequence and the moving average sequence to obtain the residual sequence, calculate the mean and standard deviation of the residual sequence, subtract the mean from the residual at each moment in the residual sequence and divide by the standard deviation to obtain the residual deviation sequence; extract the residual deviation corresponding to the current moment from the residual deviation sequence as the second deviation measure; calculate the third and fourth deviation measures based on the residual deviation sequence, and use the first, second, third, and fourth deviation measures as abnormal deviation features.

[0019] In this step, the specific calculation steps for the first deviation metric are as follows: calculate the standard deviation σ and interquartile range (IQR) of the historical monitoring numerical series; determine the bandwidth parameter for kernel density estimation according to the Silverman rule, i.e., through the formula... Calculate the bandwidth parameter, where n represents the sample size of the historical monitoring numerical sequence; use the calculated bandwidth parameter to estimate the kernel density of the historical monitoring numerical sequence and fit it to obtain the first statistical distribution function; substitute the current monitoring value into the first statistical distribution function to calculate its cumulative probability value; perform a logarithmic transformation on the cumulative probability value to obtain the first deviation measure, the formula for which is: , This represents the cumulative probability value.

[0020] The specific calculation steps of the second deviation metric are as follows: First, calculate the moving average sequence for the complete monitoring value sequence. The complete monitoring value sequence is smoothed using a centered moving average, and the window length can be set to k = 10. For the middle time t that can fill the complete window (i.e., when 5 < t ≤ L - 5, where L is the total length of the sequence), take the monitoring values of the first 5 times and the last 4 times, plus the monitoring value at the current time itself, and calculate the arithmetic mean of the monitoring values of 10 consecutive times as the moving average value at this time; for the boundary time t at the beginning of the sequence (i.e., when 1 ≤ t ≤ 5), when t = 1, use 6 points from the 1st to the 6th time, when t = 2, use 7 points from the 1st to the 7th time, when t = 3, use 8 points from the 1st to the 8th time, when t = 4, use 9 points from the 1st to the 9th time, and when t = 5, use 10 points from the 1st to the 10th time. For the boundary time t at the end of the sequence (i.e., when L - 4 ≤ t ≤ L), when t = L - 4, use 10 points from the (L - 9)th to the Lth time, when t = L - 3, use 9 points from the (L - 8)th to the Lth time, when t = L - 2, use 8 points from the (L - 7)th to the Lth time, when t = L - 1, use 7 points from the (L - 6)th to the Lth time, and when t = L, use 6 points from the (L - 5)th to the Lth time. Through this boundary processing method, it is ensured that each time in the entire original monitoring sequence has a corresponding moving average value, and the calculation at the boundary transitions smoothly without data loss or mutation due to window truncation.

[0021] Calculate the absolute difference between the complete monitoring value sequence and the moving average sequence to obtain the residual sequence; calculate the residual deviation degree sequence based on the residual sequence, and the calculation formula is , where is the mean of the residual sequence, is the standard deviation of the residual sequence; is the residual deviation degree at time t; finally, extract the residual deviation degree corresponding to the current time from the residual deviation degree sequence as the second deviation metric; In this step, the specific implementation steps for calculating the third deviation metric and the fourth deviation metric based on the residual deviation degree sequence include step S211 and step S212; Step S211: Exclude the residual deviation degree at the current time from the residual deviation degree sequence to obtain the historical residual deviation degree sequence; use the historical residual deviation degree sequence to fit the second statistical distribution function, substitute the residual deviation degree at the current time into the second statistical distribution function, calculate the cumulative probability value, and perform a logarithmic transformation on the cumulative probability value to obtain the third deviation metric; In this step, the specific calculation steps of the third deviation metric are as follows: First, calculate the standard deviation σ and the interquartile range IQR of the historical residual deviation degree sequence; determine the bandwidth parameter of the kernel density estimation according to the Silverman rule, that is, through the formula Calculate the bandwidth parameter, where n represents the number of samples in the historical residual deviation sequence; use the calculated bandwidth parameter to perform kernel density estimation on the historical residual deviation sequence, and fit it to obtain the second statistical distribution function; substitute the residual deviation at the current time into the second statistical distribution function to calculate its cumulative probability value; perform a logarithmic transformation on the cumulative probability value to obtain the third deviation measure, the formula for which is... , This represents the cumulative probability value.

[0022] Step S212: Calculate the mean and standard deviation of the historical residual deviation sequence. Add three times the standard deviation to the mean and set it as the upper threshold. Subtract three times the standard deviation from the mean and set it as the lower threshold. Mark all values ​​in the historical residual deviation sequence that are greater than the upper threshold as upper extreme values ​​and sort them in chronological order to form the upper extreme value sequence. Mark all values ​​in the historical residual deviation sequence that are less than the lower threshold as lower extreme values ​​and sort them in chronological order to form the lower extreme value sequence. Calculate the fourth deviation based on the upper extreme value sequence and the lower extreme value sequence.

[0023] In this step, the specific implementation steps for calculating the fourth deviation based on the upper extreme value sequence and the lower extreme value sequence include step S2121; Step S2121: Construct upper and lower extreme value distribution functions using the upper and lower extreme value sequences respectively. Based on the relationship between the current residual deviation and the mean of the historical residual deviation sequence, select the corresponding extreme value distribution function. If the current residual deviation is greater than the mean, substitute the current residual deviation into the upper extreme value distribution function to calculate the cumulative probability value; if the current residual deviation is less than the mean, substitute the current residual deviation into the lower extreme value distribution function to calculate the cumulative probability value; perform a logarithmic transformation on the cumulative probability value to obtain the fourth deviation metric; if the current residual deviation is equal to the mean, the fourth deviation metric is set to zero.

[0024] The specific calculation steps for the fourth deviation metric are as follows: First, calculate the standard deviation σ and interquartile range (IQR) of the obtained upper extreme value sequence; then, determine the bandwidth parameter for kernel density estimation based on the Silverman rule, i.e., using the formula... Calculate the bandwidth parameter, where n represents the number of samples in the upper extreme value sequence; use the calculated bandwidth parameter to estimate the kernel density of the upper extreme value sequence and fit it to obtain the upper extreme value distribution function; similarly, calculate the lower extreme value distribution function. If the residual deviation at the current time step is greater than the mean, then substitute the residual deviation at the current time step into the upper extreme value distribution function to calculate the cumulative probability value; if the residual deviation at the current time step is less than the mean, then substitute the residual deviation at the current time step into the lower extreme value distribution function to calculate the cumulative probability value; perform a logarithmic transformation on the cumulative probability value to obtain the fourth deviation measure, the logarithmic transformation formula is as follows: , This is the cumulative probability value; if the residual deviation at the current time is equal to the mean, then the fourth deviation measure is zero.

[0025] These four deviation metrics together constitute a multi-dimensional evaluation system for the performance status of BeiDou satellite navigation products. The first deviation metric, based on overall statistical distribution, measures the position of the current value within the historical data distribution, effectively identifying anomalies such as gradual drift or slow deterioration. The second deviation metric, based on moving average residual analysis, quantifies the strength of the current value's deviation from local fluctuation trends, and is particularly sensitive to sudden jumps or instantaneous interference. The third deviation metric performs secondary statistical distribution verification based on the residual deviation sequence, judging the significance of the current anomaly from the perspective of overall fluctuation levels, effectively avoiding false alarms caused by single-point misjudgments. The fourth deviation metric focuses on historical extreme events, using extreme value distribution analysis to assess whether the current anomaly has exceeded the range of previous extreme fluctuations, specifically designed to capture severe anomalies exceeding historical extremes. These four metrics complement each other and progress layer by layer, covering different types of anomaly patterns while effectively reducing the risk of misjudgment that may arise from a single indicator through multi-angle verification, providing reliable feature input for subsequent comprehensive evaluation.

[0026] After calculating the first deviation metric, second deviation metric, third deviation metric, and fourth deviation metric, the specific implementation method for constructing the original feature vector of the BeiDou satellite navigation product under test based on the abnormal deviation characteristics of each performance dimension is as follows: For each performance dimension, four deviation metrics (first deviation metric, second deviation metric, third deviation metric, and fourth deviation metric) are calculated through the aforementioned steps. Assuming there are a total of M performance dimensions, each performance dimension corresponds to a four-dimensional feature vector. These M four-dimensional feature vectors are concatenated in a fixed dimensional order to form a one-dimensional vector of length M×4, which is the original feature vector of the BeiDou satellite navigation product under test.

[0027] In addition, the specific implementation steps for calculating the fourth deviation based on the upper and lower extreme value sequences may also include: Step S21211: Perform a first filtering operation on the upper extreme value sequence to obtain a preliminary upper extreme value sequence: calculate the change range of each upper extreme value in the upper extreme value sequence. For the first upper extreme value in the upper extreme value sequence, only calculate the first absolute difference with the upper extreme value at the next time step; for the last upper extreme value in the upper extreme value sequence, only calculate the second absolute difference with the upper extreme value at the previous time step; for other upper extreme values, calculate the third absolute difference with the upper extreme value at the previous time step and the fourth absolute difference with the upper extreme value at the next time step respectively; set a continuity threshold. If the first absolute difference is greater than the continuity threshold, delete the corresponding upper extreme value. If the second absolute difference is greater than the continuity threshold, delete the corresponding upper extreme value. If one of the third absolute difference and the fourth absolute difference is greater than the continuity threshold or both are greater than the continuity threshold, delete the corresponding upper extreme value. This step can be understood as follows: Calculate the first absolute difference between the first upper extreme value and the upper extreme value at the next time step in the upper extreme value sequence. If the first absolute difference is greater than the preset continuity threshold, then delete the first upper extreme value from the upper extreme value sequence. Calculate the second absolute difference between the last upper extreme value and the upper extreme value at the previous time step in the upper extreme value sequence. If the second absolute difference is greater than the continuity threshold, then delete the second upper extreme value from the upper extreme value sequence. For other upper extreme values ​​in the upper extreme value sequence, calculate the third absolute difference between the upper extreme value and the upper extreme value at the previous time step, and the fourth absolute difference between the upper extreme value and the upper extreme value at the next time step. If one of the third absolute difference and the fourth absolute difference is greater than the continuity threshold, or both are greater than the continuity threshold, then delete the upper extreme value from the upper extreme value sequence.

[0028] Step S21212: Perform the second filtering operation on the preliminary upper extreme value sequence to obtain the final upper extreme value sequence: For each upper extreme value in the preliminary upper extreme value sequence, construct the neighborhood of the corresponding time of the upper extreme value in the historical residual deviation sequence, determine whether the upper extreme value is the maximum value in the neighborhood, if not, delete the upper extreme value from the preliminary upper extreme value sequence to obtain the final upper extreme value sequence. This step can be understood as follows: preset the width of the neighborhood window to be w, where w is an odd number, and define the half-window width as m = (w-1) / 2; For the upper extreme value corresponding to time t in the initial upper extreme value sequence, its neighborhood N(t) in the historical residual deviation sequence is determined as follows: If t satisfies m+1≤t≤Lm, where L is the total length of the historical residual deviation sequence, then t is in the middle of the sequence, and the neighborhood takes a complete window: N(t)=[tm, t+m]; If t satisfies 1≤t≤m, then t is at the starting boundary of the historical residual deviation sequence, and the neighborhood is taken from the starting point of the historical residual deviation sequence to time t+m: N(t)=[1, t+m]; If t satisfies L-m+1≤t≤L, then t is at the end boundary of the historical residual deviation sequence, and the neighborhood is taken from tm to the end time of the historical residual deviation sequence: N(t)=[tm, L]; Within a defined neighborhood N(t), if the upper extreme value at time t in the initial upper extreme value sequence is the maximum value among the residual deviations in the neighborhood N(t), then the upper extreme value is retained; otherwise, it is deleted from the initial upper extreme value sequence. After performing the above judgment on all upper extreme values ​​in the initial upper extreme value sequence in turn, the retained upper extreme values ​​constitute the final upper extreme value sequence.

[0029] Step S21213: Perform the first filtering operation on the lower extreme value sequence to obtain a preliminary lower extreme value sequence: calculate the change range of each lower extreme value in the lower extreme value sequence. For the first lower extreme value in the lower extreme value sequence, only calculate the fifth absolute difference with the lower extreme value at the next time step; for the last lower extreme value in the lower extreme value sequence, only calculate the sixth absolute difference with the lower extreme value at the previous time step; for other lower extreme values, calculate the seventh absolute difference with the lower extreme value at the previous time step and the eighth absolute difference with the lower extreme value at the next time step respectively; set a continuity threshold. If the fifth absolute difference is greater than the continuity threshold, delete the corresponding lower extreme value. If the sixth absolute difference is greater than the continuity threshold, delete the corresponding lower extreme value. If one of the seventh absolute difference and the eighth absolute difference is greater than the continuity threshold or both are greater than the continuity threshold, delete the corresponding lower extreme value. Step S21214: Perform the second filtering operation on the preliminary lower extreme value sequence to obtain the final lower extreme value sequence: For each lower extreme value in the preliminary lower extreme value sequence, construct the neighborhood of the corresponding time of the lower extreme value in the historical residual deviation sequence, determine whether the lower extreme value is the minimum value in the neighborhood, if not, delete the lower extreme value from the preliminary lower extreme value sequence to obtain the final lower extreme value sequence; This step can be understood as follows: preset the width of the neighborhood window to be w, where w is an odd number, and define the half-window width as m = (w-1) / 2; For the lower extreme value corresponding to time t in the initial lower extreme value sequence, its neighborhood N(t) in the historical residual deviation sequence is determined as follows: If t satisfies m+1≤t≤Lm, where L is the total length of the historical residual deviation sequence, then t is in the middle of the sequence, and the neighborhood takes a complete window: N(t)=[tm, t+m]; If t satisfies 1≤t≤m, then t is at the starting boundary of the historical residual deviation sequence, and the neighborhood is taken from the starting point of the historical residual deviation sequence to time t+m: N(t)=[1, t+m]; If t satisfies L-m+1≤t≤L, then t is at the end boundary of the historical residual deviation sequence, and the neighborhood is taken from tm to the end time of the historical residual deviation sequence: N(t)=[tm, L]; Within a defined neighborhood N(t), if the lower extreme value at time t in the initial lower extreme value sequence is the minimum value among the residual deviations within the neighborhood N(t), then the lower extreme value is retained; otherwise, it is deleted from the initial lower extreme value sequence. After performing the above judgment on all lower extreme values ​​in the initial lower extreme value sequence, the retained lower extreme values ​​constitute the final lower extreme value sequence.

[0030] In the above steps, the upper and lower extreme value sequences are filtered twice in sequence. The advantage of this is that it can effectively remove isolated outliers caused by instantaneous noise and remove non-peak redundant samples that appear consecutively in the same extreme event, thereby improving the reliability of the data finally used to fit the extreme value distribution.

[0031] Step S21215: Determine whether the number of samples in both the final upper extreme value sequence and the final lower extreme value sequence is greater than the preset minimum number of samples. If so, construct the upper extreme value distribution function and the lower extreme value distribution function using the final upper extreme value sequence and the final lower extreme value sequence, respectively; select the corresponding extreme value distribution function based on the relationship between the residual deviation at the current time and the mean of the historical residual deviation sequence. If the residual deviation at the current time is greater than the mean, substitute the residual deviation at the current time into the upper extreme value distribution function to calculate the cumulative probability value; if the residual deviation at the current time is less than the mean, substitute the residual deviation at the current time into the lower extreme value distribution function to calculate the cumulative probability value; perform a logarithmic transformation on the cumulative probability value to obtain the fourth deviation metric; if the residual deviation at the current time is equal to the mean, the fourth deviation metric is set to zero.

[0032] In this step, the calculation method is the same as in step S2121; If not, calculate the 95th quantile A and 5th quantile B of the historical residual deviation sequence, and record the maximum value C and minimum value D of the historical residual deviation sequence; determine the relationship between the current residual deviation E and the mean u of the historical residual deviation sequence. If E is greater than u, then the cumulative probability value... If E is less than u, the cumulative probability value If E equals u, then the cumulative probability is 0.5; performing a logarithmic transformation on the cumulative probability yields the fourth deviation.

[0033] In this step, when the sample size is sufficient, the fourth deviation metric is calculated using the same method as in step S2121; when the sample size is insufficient, the tail features of historical data are used for estimation, thus avoiding the problem of being unable to calculate or the distribution being distorted due to insufficient samples.

[0034] Step S3: Input the original feature vector into the pre-trained scoring model to obtain the performance score, and determine the performance test result of the Beidou satellite navigation product under test based on the performance score.

[0035] In this step, the scoring model is used to represent the one-to-one correspondence between the original feature vectors and the performance scores. A large number of historical original feature vectors can be labeled with performance scores. After labeling, the historical original feature vectors are used as input and the performance scores are used as output to train the convolutional neural network model to obtain the scoring model. In this step, the performance test results of the BeiDou satellite navigation product under test are determined based on the performance score, including: Step S31: Obtain the first judgment threshold and the second judgment threshold, wherein the second judgment threshold is greater than the first judgment threshold; compare the performance score with the first judgment threshold and the second judgment threshold. If the performance score is less than the first judgment threshold, the performance status of the Beidou satellite navigation product under test is determined to be normal; if the performance score is greater than the second judgment threshold, the performance status of the Beidou satellite navigation product under test is determined to be abnormal; if the performance score is greater than or equal to the first judgment threshold and less than or equal to the second judgment threshold, it is determined that the current condition is in the uncertain range, and the manual review process is initiated to determine the performance test result of the Beidou satellite navigation product under test.

[0036] In this step, the specific implementation steps for initiating the manual review process to determine the performance test results of the BeiDou satellite navigation product under test include steps S311 and S312; Step S311: Obtain the abnormal scores and weight coefficients of each performance dimension given by the testing personnel. The sum of the weight coefficients of all performance dimensions is 1. Sort the abnormal scores in descending order according to the size of the weight coefficients to obtain the abnormal score sequence. In this step, the anomaly scores for each performance dimension obtained by the testing personnel after evaluating the Beidou satellite navigation product under test, and the weighting coefficients are also given by the testing personnel; Step S312: Following the order of the anomaly score sequence, iterate through each anomaly score sequentially. When iterating through each performance dimension, perform the following operations: Based on all the abnormal scores already traversed and their corresponding weight coefficients, calculate the current weighted average; then, sequentially determine the following termination conditions: First termination condition: If the abnormal score currently traversed reaches the preset direct anomaly detection threshold for this performance dimension, the performance of the Beidou satellite navigation product under test is immediately determined to be abnormal, and the entire traversal process is terminated. Second termination condition: If the current weighted average value is greater than or equal to the preset anomaly judgment threshold, the performance of the Beidou satellite navigation product under test is immediately judged to be abnormal, and the entire traversal process is terminated. If no termination condition is triggered after traversing all performance dimensions, the performance test result of the BeiDou satellite navigation product under test is determined based on the final weighted average. If the final weighted average is less than or equal to the preset normal judgment threshold, the performance of the Beidou satellite navigation product under test is judged to be normal. If the final weighted average value is between the normal judgment threshold and the abnormal judgment threshold, the performance test result of the Beidou satellite navigation product under test is determined according to the preset interval in which the weighted average value is located. Each interval has a corresponding performance test result.

[0037] In this step, the weighted average is calculated by summing the products of each anomaly score and its corresponding weight coefficient, and dividing by the sum of the weight coefficients of all anomaly scores already traversed. Additionally, the anomaly detection threshold is greater than the normal detection threshold. In this step, determining the performance test result of the BeiDou satellite navigation product under test based on the preset interval in which the weighted average value falls can be understood as follows: the (normal judgment threshold, abnormal judgment threshold) can be divided into several continuous intervals, each interval corresponding to a performance test result. The performance test result of the BeiDou satellite navigation product under test is the interval in which the weighted average value falls. For example, if the (normal judgment threshold, abnormal judgment threshold) is (2.0, 4.0), then (2.0, 3.0) can be set as: slight degradation, [3.0, 4.0) as: moderate abnormality; if the weighted average value is 2.5, then the performance test result of the BeiDou satellite navigation product under test is slight degradation.

[0038] In this step, on the one hand, a two-level termination mechanism is set up, namely a direct anomaly detection threshold and an anomaly detection threshold, which can terminate the traversal in advance when serious anomalies occur in key dimensions or the comprehensive score has reached an abnormal level, thus avoiding redundant calculations. On the other hand, for samples that have not triggered the termination condition, a refined performance degradation level is output by determining the interval level. Through this setting, a good balance can be achieved between efficiency and accuracy.

[0039] Example 2 like Figure 2 As shown in the figure, this embodiment provides a BeiDou satellite navigation product performance testing system, which includes an acquisition module 1, a construction module 2, and a scoring module 3.

[0040] Module 1 is used to acquire the complete monitoring value sequence of the Beidou satellite navigation product under test in multiple performance dimensions. The complete monitoring value sequence of each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. Module 2 is used to calculate the abnormal deviation characteristics of the current monitoring value of each performance dimension relative to the complete monitoring value sequence for each performance dimension; and to construct the original feature vector of the Beidou satellite navigation product under test based on the abnormal deviation characteristics of each performance dimension. The scoring module 3 is used to input the original feature vector into the pre-trained scoring model to obtain a performance score, and to determine the performance test result of the Beidou satellite navigation product under test based on the size of the performance score.

[0041] In one specific embodiment of this disclosure, the construction module 2 further includes a first computing unit 21.

[0042] The first calculation unit 21 is used to calculate the standard deviation and interquartile range of the historical monitoring numerical sequence, then determine the bandwidth parameter for kernel density estimation according to the Silverman rule, use the bandwidth parameter to perform kernel density estimation on the historical monitoring numerical sequence, fit to obtain the first statistical distribution function, substitute the monitoring value at the current moment into the first statistical distribution function, calculate the cumulative probability value, perform a logarithmic transformation on the cumulative probability value to obtain the first deviation measure; calculate the moving average sequence for the complete monitoring numerical sequence, calculate the absolute difference between the complete monitoring numerical sequence and the moving average sequence to obtain the residual sequence, calculate the mean and standard deviation of the residual sequence, subtract the mean from the residual at each moment in the residual sequence and divide by the standard deviation to obtain the residual deviation sequence; extract the residual deviation corresponding to the current moment from the residual deviation sequence as the second deviation measure; calculate the third and fourth deviation measures based on the residual deviation sequence, and use the first, second, third, and fourth deviation measures as abnormal deviation features.

[0043] In one specific embodiment of this disclosure, the first computing unit 21 further includes a second computing unit 211 and a third computing unit 212.

[0044] The second calculation unit 211 is used to exclude the residual deviation at the current time from the residual deviation sequence to obtain the historical residual deviation sequence; to fit the historical residual deviation sequence to obtain the second statistical distribution function; to substitute the residual deviation at the current time into the second statistical distribution function to calculate the cumulative probability value; and to perform a logarithmic transformation on the cumulative probability value to obtain the third deviation measure. The third calculation unit 212 is used to calculate the mean and standard deviation of the historical residual deviation sequence. The mean plus three times the standard deviation is recorded as the upper threshold, and the mean minus three times the standard deviation is recorded as the lower threshold. All values ​​in the historical residual deviation sequence that are greater than the upper threshold are marked as upper extreme values ​​and sorted in chronological order to form an upper extreme value sequence. All values ​​in the historical residual deviation sequence that are less than the lower threshold are marked as lower extreme values ​​and sorted in chronological order to form a lower extreme value sequence. The fourth deviation is calculated based on the upper extreme value sequence and the lower extreme value sequence.

[0045] In one specific embodiment of this disclosure, the third computing unit 212 further includes a fourth computing unit 2121.

[0046] The fourth calculation unit 2121 is used to construct upper and lower extreme value distribution functions using the upper and lower extreme value sequences, respectively. Based on the relationship between the current residual deviation and the mean of the historical residual deviation sequence, the corresponding extreme value distribution function is selected. Specifically, if the current residual deviation is greater than the mean, it is substituted into the upper extreme value distribution function to calculate the cumulative probability value; if the current residual deviation is less than the mean, it is substituted into the lower extreme value distribution function to calculate the cumulative probability value. A logarithmic transformation is performed on the cumulative probability value to obtain the fourth deviation metric; if the current residual deviation is equal to the mean, the fourth deviation metric is set to zero.

[0047] In one specific embodiment of this disclosure, the scoring module 3 further includes a determination unit 31.

[0048] The determination unit 31 is used to obtain a first determination threshold and a second determination threshold, wherein the second determination threshold is greater than the first determination threshold; compare the performance score with the first determination threshold and the second determination threshold; if the performance score is less than the first determination threshold, the performance status of the Beidou satellite navigation product under test is determined to be normal; if the performance score is greater than the second determination threshold, the performance status of the Beidou satellite navigation product under test is determined to be abnormal; if the performance score is greater than or equal to the first determination threshold and less than or equal to the second determination threshold, it is determined that the current condition is in an uncertain range, and the manual review process is initiated to determine the performance test result of the Beidou satellite navigation product under test.

[0049] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0050] Example 3 Corresponding to the above method embodiments, this disclosure also provides a BeiDou satellite navigation product performance testing device. The BeiDou satellite navigation product performance testing device described below and the BeiDou satellite navigation product performance testing method described above can be referred to each other.

[0051] Figure 3 This is a block diagram illustrating a BeiDou satellite navigation product performance testing device 300 according to an exemplary embodiment. For example... Figure 3 As shown, the BeiDou satellite navigation product performance testing device 300 may include: a processor 301 and a memory 302. The BeiDou satellite navigation product performance testing device 300 may also include one or more of the following: a multimedia component 303, an I / O interface 304, and a communication component 305.

[0052] The processor 301 controls the overall operation of the BeiDou satellite navigation product performance testing equipment 300 to complete all or part of the steps in the aforementioned BeiDou satellite navigation product performance testing method. The memory 302 stores various types of data to support the operation of the BeiDou satellite navigation product performance testing equipment 300. This data may include, for example, instructions for any application or method operating on the BeiDou satellite navigation product performance testing equipment 300, as well as application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 302 or transmitted via the communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 305 is used for wired or wireless communication between the Beidou satellite navigation product performance testing equipment 300 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 305 may include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0053] In an exemplary embodiment, the BeiDou satellite navigation product performance testing device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the aforementioned BeiDou satellite navigation product performance testing method.

[0054] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the BeiDou satellite navigation product performance testing method described above. For example, the computer-readable storage medium may be the memory 302 including program instructions, which may be executed by the processor 301 of the BeiDou satellite navigation product performance testing device 300 to complete the BeiDou satellite navigation product performance testing method described above.

[0055] Example 4 Corresponding to the above method embodiments, this disclosure also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the BeiDou satellite navigation product performance testing method described above.

[0056] A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the BeiDou satellite navigation product performance testing method described in the above method embodiments.

[0057] Specifically, the readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0058] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for testing the performance of BeiDou satellite navigation products, characterized in that, include: Obtain complete monitoring value sequences for the BeiDou satellite navigation product under test across multiple performance dimensions. The complete monitoring value sequence for each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. For each performance dimension, calculate the abnormal deviation characteristics of the current monitoring value of that performance dimension relative to the complete monitoring value sequence; Based on the abnormal deviation characteristics of each performance dimension, the original feature vector of the Beidou satellite navigation product under test is constructed. The original feature vector is input into a pre-trained scoring model to obtain a performance score. The performance test result of the BeiDou satellite navigation product under test is determined based on the magnitude of the performance score.

2. The method for testing the performance of BeiDou satellite navigation products according to claim 1, characterized in that, Calculate the abnormal deviation characteristics of the current monitoring value of this performance dimension relative to the complete monitoring value sequence, including: The standard deviation and interquartile range of the historical monitoring numerical sequence are calculated. Then, the bandwidth parameter for kernel density estimation is determined according to the Silverman rule. Kernel density estimation is performed on the historical monitoring numerical sequence using the bandwidth parameter, and a first statistical distribution function is obtained by fitting. The monitoring value at the current moment is substituted into the first statistical distribution function to calculate the cumulative probability value. The cumulative probability value is logarithmically transformed to obtain the first deviation measure. A moving average sequence is calculated for the complete monitoring numerical sequence. The absolute difference between the complete monitoring numerical sequence and the moving average sequence is calculated to obtain the residual sequence. The mean and standard deviation of the residual sequence are calculated. The residual at each moment is subtracted from the mean and divided by the standard deviation to obtain the residual deviation sequence. The residual deviation corresponding to the current moment is extracted from the residual deviation sequence as the second deviation measure. The third and fourth deviation measures are calculated based on the residual deviation sequence. The first, second, third, and fourth deviation measures are used as abnormal deviation features.

3. The method for testing the performance of BeiDou satellite navigation products according to claim 2, characterized in that, The third and fourth deviation measures are calculated based on the residual deviation sequence, including: The residual deviation at the current time is excluded from the residual deviation sequence to obtain the historical residual deviation sequence; the second statistical distribution function is obtained by fitting the historical residual deviation sequence; the residual deviation at the current time is substituted into the second statistical distribution function to calculate the cumulative probability value; and the third deviation measure is obtained by performing a logarithmic transformation on the cumulative probability value. Calculate the mean and standard deviation of the historical residual deviation sequence. Add three times the standard deviation to the mean and set it as the upper threshold. Subtract three times the standard deviation from the mean and set it as the lower threshold. Mark all values ​​in the historical residual deviation sequence that are greater than the upper threshold as upper extreme values ​​and sort them in chronological order to form the upper extreme value sequence. Mark all values ​​in the historical residual deviation sequence that are less than the lower threshold as lower extreme values ​​and sort them in chronological order to form the lower extreme value sequence. Calculate the fourth deviation based on the upper and lower extreme value sequences.

4. The method for testing the performance of BeiDou satellite navigation products according to claim 3, characterized in that, The fourth deviation is calculated based on the upper and lower extreme value sequences, including: An upper extreme value distribution function and a lower extreme value distribution function are constructed using the upper and lower extreme value sequences, respectively. Based on the relationship between the current residual deviation and the mean of the historical residual deviation sequences, the corresponding extreme value distribution function is selected. Specifically, if the current residual deviation is greater than the mean, it is substituted into the upper extreme value distribution function to calculate the cumulative probability value; if the current residual deviation is less than the mean, it is substituted into the lower extreme value distribution function to calculate the cumulative probability value. A logarithmic transformation is performed on the cumulative probability value to obtain the fourth deviation metric; if the current residual deviation is equal to the mean, the fourth deviation metric is set to zero.

5. The method for testing the performance of BeiDou satellite navigation products according to claim 1, characterized in that, The performance test results of the BeiDou satellite navigation product under test are determined based on the performance score, including: Obtain a first judgment threshold and a second judgment threshold, wherein the second judgment threshold is greater than the first judgment threshold; compare the performance score with the first judgment threshold and the second judgment threshold. If the performance score is less than the first judgment threshold, the performance status of the Beidou satellite navigation product under test is determined to be normal; if the performance score is greater than the second judgment threshold, the performance status of the Beidou satellite navigation product under test is determined to be abnormal; if the performance score is greater than or equal to the first judgment threshold and less than or equal to the second judgment threshold, it is determined that the current situation is in the uncertain range, and the manual review process is initiated to determine the performance test result of the Beidou satellite navigation product under test.

6. A performance testing system for BeiDou satellite navigation products, characterized in that, include: The acquisition module is used to acquire the complete monitoring value sequence of the Beidou satellite navigation product under test in multiple performance dimensions. The complete monitoring value sequence of each performance dimension includes the monitoring value at the current moment and the historical monitoring value sequence before the current moment. The module is used to calculate the abnormal deviation characteristics of the current monitoring value of each performance dimension relative to the complete monitoring value sequence for each performance dimension. Based on the abnormal deviation characteristics of each performance dimension, the original feature vector of the Beidou satellite navigation product under test is constructed. The scoring module is used to input the original feature vector into a pre-trained scoring model to obtain a performance score, and to determine the performance test result of the Beidou satellite navigation product under test based on the size of the performance score.

7. The BeiDou satellite navigation product performance testing system according to claim 6, characterized in that, Build modules, including: The first calculation unit is used to calculate the standard deviation and interquartile range of the historical monitoring numerical sequence, then determine the bandwidth parameter for kernel density estimation according to the Silverman rule, and use the bandwidth parameter to perform kernel density estimation on the historical monitoring numerical sequence to obtain the first statistical distribution function. The monitoring value at the current moment is substituted into the first statistical distribution function to calculate the cumulative probability value. The cumulative probability value is logarithmically transformed to obtain the first deviation measure. The moving average sequence is calculated for the complete monitoring numerical sequence, and the absolute difference between the complete monitoring numerical sequence and the moving average sequence is calculated to obtain the residual sequence. The mean and standard deviation of the residual sequence are calculated, and the residual at each moment is subtracted from the mean and divided by the standard deviation to obtain the residual deviation sequence. The residual deviation corresponding to the current moment is extracted from the residual deviation sequence as the second deviation measure. The third and fourth deviation measures are calculated based on the residual deviation sequence, and the first, second, third, and fourth deviation measures are used as abnormal deviation features.

8. The BeiDou satellite navigation product performance testing system according to claim 7, characterized in that, The first computing unit includes: The second calculation unit is used to exclude the residual deviation at the current time from the residual deviation sequence to obtain the historical residual deviation sequence; the second statistical distribution function is obtained by fitting the historical residual deviation sequence; the residual deviation at the current time is substituted into the second statistical distribution function to calculate the cumulative probability value; and the third deviation measure is obtained by performing a logarithmic transformation on the cumulative probability value. The third calculation unit is used to calculate the mean and standard deviation of the historical residual deviation sequence. The mean plus three times the standard deviation is recorded as the upper threshold, and the mean minus three times the standard deviation is recorded as the lower threshold. All values ​​in the historical residual deviation sequence that are greater than the upper threshold are marked as upper extreme values ​​and sorted in chronological order to form an upper extreme value sequence. All values ​​in the historical residual deviation sequence that are less than the lower threshold are marked as lower extreme values ​​and sorted in chronological order to form a lower extreme value sequence. The fourth deviation is calculated based on the upper extreme value sequence and the lower extreme value sequence.

9. The BeiDou satellite navigation product performance testing system according to claim 8, characterized in that, The third calculation unit includes: The fourth calculation unit is used to construct upper and lower extreme value distribution functions using the upper and lower extreme value sequences, respectively. Based on the relationship between the current residual deviation and the mean of the historical residual deviation sequence, the corresponding extreme value distribution function is selected. Specifically, if the current residual deviation is greater than the mean, it is substituted into the upper extreme value distribution function to calculate the cumulative probability value; if the current residual deviation is less than the mean, it is substituted into the lower extreme value distribution function to calculate the cumulative probability value. A logarithmic transformation is performed on the cumulative probability value to obtain the fourth deviation metric; if the current residual deviation is equal to the mean, the fourth deviation metric is set to zero.

10. The BeiDou satellite navigation product performance testing system according to claim 6, characterized in that, The rating module includes: The judgment unit acquires a first judgment threshold and a second judgment threshold, wherein the second judgment threshold is greater than the first judgment threshold; it compares the performance score with the first judgment threshold and the second judgment threshold. If the performance score is less than the first judgment threshold, the performance status of the Beidou satellite navigation product under test is judged to be normal; if the performance score is greater than the second judgment threshold, the performance status of the Beidou satellite navigation product under test is judged to be abnormal; if the performance score is greater than or equal to the first judgment threshold and less than or equal to the second judgment threshold, it is judged to be currently in an uncertain range, and the manual review process is initiated to determine the performance test result of the Beidou satellite navigation product under test.

Images