Method and System for Supporting Quality Analysis and Evaluation of Space Science and Application Data

Abstract

The present invention provides a method and system for supporting the quality analysis and evaluation of space science and application data. Based on the data type of the data included in any file, the target evaluation index type is determined. The evaluation index corresponding to the target evaluation index type is determined as the target evaluation index, and the actual evaluation range of the data included in any file under each target evaluation index is obtained. Based on the fact that the actual evaluation range of any file under any target evaluation index does not completely fall within the preset evaluation range of any target evaluation index, it is determined that any target evaluation index corresponding to any file is abnormal. Based on the ratio of the number of abnormal target evaluation indexes corresponding to any file to the total number of target evaluation indexes being greater than the abnormal ratio threshold, it is determined that any file is abnormal. The present invention can realize the quality analysis and evaluation of space science and application data.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing for manned spaceflight engineering, and particularly to a method and system for supporting the quality analysis and evaluation of space science and application data. Background Art

[0002] Space science and application data refers to data related to space science experiments, the cosmic space, the Earth, etc. obtained by on-orbit scientific experiment payloads on the space station during scientific experiments, including information of multiple types (such as images, videos, parameters, etc.) and multiple disciplines (such as materials, biology, astrophysics, astronomy, etc.). It is a very important information resource that can be used for scientific research, resource development, environmental monitoring, etc., and plays an important role in applications in multiple fields such as life science, materials science, microgravity physics, etc., and has important significance and value.

[0003] Data quality analysis and evaluation refers to the systematic analysis and evaluation of data, including a series of steps such as data collection, cleaning, format interpretation, integration, management, etc., aiming to ensure the accuracy, integrity, consistency, reliability, etc. of the data, and is an important part of data processing production, management, and application.

[0004] Due to the large scale of space science and application data and the complexity of related fields, extremely high requirements are imposed on the security and reliability of each stage of downlink data transmission, reception, interpretation, storage, etc. Therefore, it is necessary to conduct quality analysis and evaluation on the data involved to master the quality of the data, timely discover problems and effectively solve them, and ensure the accuracy and integrity of the data.

[0005] It can be seen that how to conduct quality analysis and evaluation on space science and application data is an urgent problem to be solved. Summary of the Invention

[0006] The present invention provides a method and system for supporting the quality analysis and evaluation of space science and application data, aiming to solve the technical problems described in the above background art.

[0007] The technical solution of the present invention for solving the above technical problems is as follows:

[0008] In a first aspect, the present invention provides a method for supporting the quality analysis and evaluation of space science and application data. In this method, for any one of at least one file to be analyzed, based on the data type of the data included in any one file, a target evaluation index type is determined from at least one evaluation index type. Among them, different evaluation index types correspond to different evaluation indexes, and the evaluation index type is used to characterize the preset evaluation range of the data corresponding to the data type. The evaluation index corresponding to the target evaluation index type is determined as the target evaluation index, and further, the actual evaluation range of the data included in any one file under each target evaluation index is obtained. For any one target evaluation index, based on the fact that the actual evaluation range of any one file under any one target evaluation index does not completely fall within the preset evaluation range of any one target evaluation index, it is determined that any one target evaluation index corresponding to any one file is abnormal. Based on the ratio of the number of abnormal target evaluation indexes corresponding to any one file to the total number of target evaluation indexes being greater than the abnormal ratio threshold, it is determined that any one file is abnormal.

[0009] On the basis of the above technical solution, the present invention can also be improved as follows.

[0010] Further, at least one evaluation index type includes a parameter - type data evaluation index type, an image - type data evaluation index type, and a video - type data evaluation index type. In the case where the data included in any one file is of the parameter - type data type or the binary - type data type, the parameter - type data evaluation index type is determined as the target evaluation index type. In the case where the data included in any one file is of the image - type data type or the spectral - type data type, the image - type data evaluation index type is determined as the target evaluation index type. In the case where the data included in any one file is of the video - type data type, the video - type data evaluation index type is determined as the target evaluation index type.

[0011] Further, the evaluation indicators corresponding to the parameter class data evaluation indicator type include at least one of the data mean, data median, data standard deviation, and reasonable data ratio. Among them, the reasonable data ratio is the ratio of the first value to the second value. The first value is the number of sampled data that fall within the preset minimum reasonable value range and the preset maximum reasonable value range, and the second value is the total number of all sampled data. The evaluation indicators corresponding to the image class data evaluation indicator type include at least one of the image signal-to-noise ratio, image clarity, image contrast, image information entropy, the range of gray values of the pixel points included in the image, image brightness, image resolution, image pixels, and the occlusion rate of the experimental target in the image. The evaluation indicators corresponding to the video class data evaluation indicator type include at least one of the video signal-to-noise ratio, video clarity, video contrast, video information entropy, the range of gray values of the pixel points included in the video, video brightness, video resolution, video pixels, video frame rate, video continuity, and video distortion. Among them, the video continuity is associated with the similarity between adjacent frame images in the video.

[0012] Further, every preset period, in at least one file to be analyzed, any file in the unselected state is determined as any file, and any file is marked as the selected state.

[0013] Further, after determining that any file is abnormal, any file is marked as the unselected state.

[0014] Further, receive the data analysis instruction input by the user. In response to the data analysis instruction, determine any file in at least one file to be analyzed.

[0015] Further, display in the warning interface the information associated with the abnormality of any target evaluation indicator corresponding to any file. After determining that any file is abnormal, display in the warning interface the information associated with the abnormality of any file. Among them, the information associated with the abnormality of any target evaluation indicator corresponding to any file and the information associated with the abnormality of any file are displayed in the warning interface in at least one of the forms of text, table, and image.

[0016] Further, display a configuration page, which is used to support the user to set at least one evaluation indicator type. In response to the user's selection operation on the first evaluation indicator type among at least one evaluation indicator type, display a list including the evaluation indicators corresponding to the first evaluation indicator type. In response to the user's selection operation on the first evaluation indicator among the evaluation indicators included in the list, display an input box for adjusting the preset evaluation range of the first evaluation indicator. In response to the user inputting the target preset evaluation range in the input box, adjust the preset evaluation range of the first evaluation indicator to the target preset evaluation range.

[0017] In a second aspect, the present invention provides a system for supporting the quality analysis and evaluation of space science and application data, and the system includes a quality analysis and evaluation management module. Among them, the quality analysis and evaluation management module is used for:

[0018] For any one of at least one file to be analyzed, based on the data type of the data included in any one file, determine a target evaluation index type from at least one evaluation index type. Among them, different evaluation index types correspond to different evaluation indexes, and the evaluation index type is used to characterize the preset evaluation range of the data corresponding to the data type. Determine the evaluation index corresponding to the target evaluation index type as the target evaluation index, and obtain the actual evaluation range of the data included in any one file under each target evaluation index. For any one target evaluation index, based on the fact that the actual evaluation range of any one file under any one target evaluation index does not completely fall within the preset evaluation range of any one target evaluation index, determine that any one target evaluation index corresponding to any one file is abnormal. Based on the ratio of the number of abnormal target evaluation indexes corresponding to any one file to the total number of target evaluation indexes being greater than the abnormal ratio threshold, determine that any one file is abnormal.

[0019] Based on the above technical solution, the present invention can also be improved as follows.

[0020] Furthermore, the system provided by the present invention further includes a quality evaluation abnormality and analysis module, and a result and report management module. Among them, the quality evaluation abnormality and analysis module is used for:

[0021] Display a configuration page, and the configuration page is used to support the user to set at least one evaluation index type. In response to a selection operation of the user on a first evaluation index type among at least one evaluation index type, display a list including each evaluation index corresponding to the first evaluation index type. In response to a selection operation of the user on a first evaluation index among each evaluation index included in the list, display an input box for adjusting the preset evaluation range of the first evaluation index. In response to the user inputting a target preset evaluation range in the input box, adjust the preset evaluation range of the first evaluation index to the target preset evaluation range.

[0022] The result and report management module is used for:

[0023] Display a warning interface, and the warning interface includes information associated with any abnormal target evaluation index corresponding to any one file, and information associated with any abnormal file. Among them, the information associated with any abnormal target evaluation index corresponding to any one file and the information associated with any abnormal file are displayed in the warning interface in at least one of the forms of text, table, and image.

[0024] In a third aspect, the present invention provides an electronic device, including: a memory and one or more processors; the memory and the processors are coupled; wherein, computer program code is stored in the memory, and the computer program code includes computer instructions, when the computer instructions are executed by the processors, the electronic device is caused to execute the method for supporting the analysis and evaluation of the quality of space science and application data described in any one of the above first aspects.

[0025] In a fourth aspect, a computer-readable storage medium is provided, including computer instructions, when the computer instructions are run on an electronic device, the electronic device is caused to execute the method for supporting the analysis and evaluation of the quality of space science and application data described in any one of the above first aspects.

[0026] In a fifth aspect, a computer program product is provided, when the computer program product is run on a computer, the computer is caused to execute the method for supporting the analysis and evaluation of the quality of space science and application data described in any one of the above first aspects.

[0027] The beneficial effects of the present invention are as follows: Based on different data types, different types of evaluation indicators and the evaluation indicators corresponding to each type of evaluation indicator are designed, which can ensure the accuracy of the evaluation of different types of data products. By constructing a data product evaluation process and executing the quality evaluation task in two ways, namely, a scheduled task and a manual initiation, the integrity of the data product quality evaluation can be ensured. Detecting and tracking data products with quality anomalies in real time can timely master the changes in the downlink data and product quality, and correct and optimize the downlink data processing models and algorithms. Displaying the evaluation results can assist in scientific management and decision-making, and improve the capabilities of operation and maintenance and decision-making of the space science and application data platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic flowchart of the method for supporting the analysis and evaluation of the quality of space science and application data provided by the present invention;

[0029] Figure 2 is a schematic flowchart of the manual working mode provided by the present invention;

[0030] Figure 3 is a schematic flowchart of the automatic working mode provided by the present invention;

[0031] Figure 4 is a schematic flowchart of the 0-level data custom task provided by the present invention;

[0032] Figure 5 is a schematic flowchart of the data quality analysis task provided by the present invention;

[0033] Figure 6 is a schematic diagram of the quality evaluation process monitoring page provided by the present invention;

[0034] Figure 7 A schematic diagram of a judgment process provided by the present invention;

[0035] Figure 8 A schematic diagram of querying abnormal results provided by the present invention;

[0036] Figure 9 A schematic diagram of a warning interface provided by the present invention;

[0037] Figure 10 A schematic diagram of a report query interface provided by the present invention;

[0038] Figure 11 A schematic diagram of previewing report query provided by the present invention;

[0039] Figure 12 A schematic diagram of the structure of a system for supporting data quality analysis and evaluation of space science and applications provided by the present invention;

[0040] Figure 13 A schematic diagram of the process of configuring an algorithm program provided by the present invention;

[0041] Figure 14 A schematic diagram of a data quality analysis task configuration page provided by the present invention;

[0042] Figure 15 A schematic diagram of an abnormal index configuration page provided by the present invention. Detailed implementation manners

[0043] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, " / " means that the objects associated before and after are in an "or" relationship. For example, A / B may represent A or B; "and / or" in the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. These three situations, where A and B can be singular or plural. Also, in the description of the present application, unless otherwise specified, "a plurality of" means two or more than two. "At least one (item)" or its similar expression below refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c may represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple. In addition, in order to clearly describe the technical solutions in the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit to be different. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions.

[0044] The present invention provides a method and system for supporting the quality analysis and evaluation of space science and application data, which can be applied to any scenario of quality analysis and evaluation of space science and application data. Through the method of the present invention, based on different data types, different types of evaluation indicators and evaluation parameters corresponding to each evaluation indicator type are designed, which can ensure the accuracy of the evaluation of different types of data products; by constructing a data product evaluation process and executing quality evaluation tasks in two ways: scheduled tasks and manual initiation, the integrity of the data product quality evaluation can be ensured; real-time detection and tracking of data products with quality anomalies can timely master the changes in the quality of downlink data and products, and correct and optimize the downlink data processing models and algorithms; the result statistical chart and analysis report can assist in scientific management and decision-making, and improve the operation and maintenance and decision-making capabilities of the space science and application data platform.

[0045] The following will further illustrate the solution of the present invention with specific embodiments.

[0046] See Figure 1, a method for supporting the analysis and evaluation of space science and application data quality provided by the present invention, includes the following steps S101 - S104:

[0047] S101: For any one of at least one file to be analyzed, based on the data type of the data included in any one file, determine the target evaluation index type among at least one evaluation index type.

[0048] Among them, different evaluation index types correspond to different evaluation indexes, and the evaluation indexes are used to characterize the preset evaluation range of the data of the corresponding data type.

[0049] In some embodiments, according to the data format type of the space science and application data quality analysis and evaluation system, it can be divided into binary data, parameter data, image data, video data, and spectral data. Among them, binary data can be regarded as a special type of parameter data, and spectral data can be regarded as a special type of image data. Based on this, for different data types, the present invention can set different evaluation index types, and evaluate the data of different data types based on the evaluation indexes corresponding to different evaluation index types.

[0050] Specifically, at least one evaluation index type in the present invention includes a parameter data evaluation index type, an image data evaluation index type, and a video data evaluation index type. When the data in any one file includes data of the parameter data type or the binary data type, the parameter data evaluation index type can be determined as the target evaluation index type. When the data in any one file includes data of the image data type or the spectral data type, the image data evaluation index type can be determined as the target evaluation index type. When the data in any one file includes data of the video data type, the video data evaluation index type can be determined as the target evaluation index type.

[0051] In some embodiments, the evaluation indexes corresponding to the parameter data evaluation index type include at least one of the data mean, data median, data maximum value, data minimum value, data standard deviation, and reasonable data ratio. Among them, the reasonable data ratio is the ratio of the first value to the second value. The first value is the number of sampled data that fall within the preset minimum reasonable value range and the preset maximum reasonable value range, and the second value is the total number of all sampled data.

[0052] In some embodiments, the evaluation indexes corresponding to the image data evaluation index type include at least one of the image signal-to-noise ratio, image clarity, image contrast, image information entropy, the range of gray values of the pixel points included in the image, image brightness, image resolution, image pixels, and the occlusion rate of the experimental target in the image.

[0053] Specifically, the image signal-to-noise ratio is an objective standard for evaluating image quality based on the error between corresponding pixel points. It is the logarithm of the ratio of the image mean to the image standard deviation, with the unit of dB. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140*R + 0.5870*G + 0.2989*B. When the input is a grayscale image, the specific calculation formula is as follows:

[0054]

[0055]

[0056] Among them, height and width respectively represent the height and width of the original image. The image is evenly divided into several sub-block regions with a side length of blk. According to experience, blk is generally taken as 4. For each sub-block region with a side length of blk, its pixel mean mean(i,j) and pixel standard deviation std(i,j) are obtained. After traversing the entire image, the means and standard deviations of all sub-block regions are averaged, and finally the mean Mean(img) and standard deviation STD(img) of the entire image are obtained. The corresponding image signal-to-noise ratio PSNR is the logarithm of the ratio of the image mean to the image standard deviation.

[0057] Image sharpness is an index describing the clarity of an image. The clearer the image, the higher the quality. The greater the sharpness, the less clear (more blurred) the image, and the lower the quality, the smaller the sharpness. The specific algorithm is as follows:

[0058]

[0059] Among them, img(i,j) represents the image pixel value at the coordinate (i,j). For each pixel point, the ratio of the pixel difference between it and the surrounding 8 pixel points to the distance is obtained. The entire image is traversed, and the sharpness clarity(i,j) at each pixel point is summed and averaged, and finally the sharpness description Clarity(img) of the entire image is obtained.

[0060] Image sharpness can also be determined based on the average gradient. The average gradient refers to the average value of the gray change rate and is used to represent image sharpness, reflecting the rate of change of the contrast of small details in the image, that is, the rate of change of the density in multiple dimensions of the image, characterizing the relative clarity of the image. The average gradient is the image sharpness, reflecting the ability of the image to express detail contrast. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140*R + 0.5870*G + 0.2989*B. When the input is a grayscale image, the calculation formula is expressed as:

[0061]

[0062] Among them, M×N represents the size of the image, represents the gradient in the horizontal direction, represents the gradient in the vertical direction.

[0063] Image contrast refers to the measurement of the different brightness levels between the brightest white and the darkest black in the light and dark areas of an image, that is, the size of the gray-scale contrast of an image. The larger the difference range, the greater the contrast; the smaller the difference range, the smaller the contrast. Usually, the gray-level co-occurrence matrix is used to describe the contrast.

[0064] The size of the image information entropy reflects the amount of information carried by the image. Usually, the larger the information entropy of the image, the more abundant the information and the better the quality. The information entropy can be used to compare the differences in the amount of information of different images, but the image information entropy cannot be used as the only standard to measure the quality of the image. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140*R + 0.5870*G + 0.2989*B. When the input is a grayscale image, the specific algorithm process of the information entropy is as follows:

[0065] Statistical gray-scale probability: Assume that the maximum gray value of the picture is M (M = 255 for 8-bit images, M = 65535 for 16-bit images, etc.). Then first count the number of pixels histogram[i] corresponding to each pixel value i from 0 to M, and initialize the information entropy H = 0;

[0066] Calculate the ratio of the number of pixels corresponding to each pixel value to the total number of pixels, ratio(i) = histogram[i] / (H×W);

[0067] Information entropy:

[0068] The range of the gray values of the pixel points included in the image is to count the pixel gray value of each point and see the range of the gray levels of the pixel points in this image. If an image has a very wide dynamic range, its contrast will be higher and the visual effect will be clearer. If two adjacent gray levels are very close, it is not easy for the human eye to distinguish clearly. That is to say, the larger the image dynamic range and the larger the gray-level span, the clearer the image resolution. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140*R + 0.5870*G + 0.2989*B. When the input is a grayscale image, the specific algorithm calculation process is as follows:

[0069] Statistical gray value of each pixel: Find the pixel maximum value max and the pixel minimum value min;

[0070] The range of the gray values of the pixel points included in the image is |max - min|.

[0071] The image brightness is determined based on the mean value of the image. The larger the mean value, the brighter the image; conversely, the smaller the mean value. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140 * R + 0.5870 * G + 0.2989 * B. When the input is a grayscale image, the specific algorithm definition is as follows: Given an image img with height H and width W, the image mean value:

[0072]

[0073] The image pixels are determined based on the image standard deviation. The image standard deviation reflects the degree of dispersion between the image pixel values and the mean value. The larger the standard deviation, the higher the pixels of the image. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140 * R + 0.5870 * G + 0.2989 * B. When the input is a grayscale image, the specific algorithm definition is as follows:

[0074] Given an image img with height H and width W, calculate the image mean value:

[0075]

[0076] The image standard deviation is expressed as:

[0077] The image resolution is the image size (or image dimensions) set when the digital camera takes a picture. Generally, a digital image with a certain size is called the image resolution. For the same actual physical scene, the larger the set image size, the more pixels the captured image has and the clearer the image. Therefore, this indicator can measure the quality of the image.

[0078] The occlusion of the experimental target in the image refers to the phenomenon that due to foreign object occlusion formed by external factors on the front-end camera, part or all of the field of view in the main area of the video or image screen is blocked, resulting in serious lack of picture monitoring information. When the input is an RGB three-channel image, it is converted to a grayscale image using Gray = 0.1140 * R + 0.5870 * G + 0.2989 * B. When the input is a grayscale image, the specific algorithm process definition is as follows:

[0079] Perform binarization on the input image img with size H×W. The darker part is the background, and the other part is the foreground. First, calculate the grayscale average value of the entire image Traverse the entire image. If the pixel value is greater than the average value, set it to 255; if the pixel value is less than the average value, set it to 0;

[0080] Using the binarized image, traverse each pixel i and perform connected component detection on the background area. Determine if there are any points in the leftmost and uppermost positions among the four-neighborhood of pixel i. If there are no points, it indicates the start of a new area. If there is a point in the leftmost position but no point in the uppermost position among the four-neighborhood of this point, mark this point with the value of the leftmost point; if there is no point in the leftmost position but a point in the uppermost position among the four-neighborhood of this point, mark this point with the value of the uppermost point. If there are points in both the leftmost and uppermost positions among the four-neighborhood of this point, mark this point with the smaller of the two marked points and modify the larger mark to the smaller mark;

[0081] Filter out the connected components that meet this condition: there are at least 2 or more pixel coordinates within the connected component at the upper, lower, left, and right boundaries of the image. If not, the experimental target is not occluded, the occlusion rate is 0, and the algorithm ends;

[0082] Find the largest connected component C among the filtered connected components. Determine the average pixel value of region C in the original image. If it is 0 (all black), it is an occluded region; otherwise, the experimental target is not occluded, the occlusion rate is 0, and the algorithm ends;

[0083] Obtain the total number of pixels M within the largest connected component C. The ratio of M to the total number of pixels H×W of the entire image, i.e., M / (H×W), is the occlusion rate;

[0084] It should be noted that assume the area of the largest connected component among all the connected components obtained in the second step is S. Generally speaking, when the occlusion area is large, the largest connected component should be the occluded region. Therefore, when M≥S, we believe that there is a high possibility of occlusion in this image and further tracking and analysis are required.

[0085] In some embodiments, the evaluation metrics corresponding to the video data evaluation metric type include at least one of video signal-to-noise ratio, video clarity, video contrast, video information entropy, the range of gray values of pixel points included in the video, video brightness, video resolution, video pixels, video frame rate, video continuity, and video distortion.

[0086] Specifically, the video frame rate is defined as the number of frames presented per second in a dynamic picture, which is used to measure the frequency of continuous occurrence of video signals, with the unit of frames per second (fps). The video frame rate is finally obtained by reading using the opencv library. First, obtain the total number of frames N of the video and the total duration T of the video. Then the frame rate is defined as: N / T (fps).

[0087] Video continuity refers to the similarity between all adjacent two frames of a video. The higher the similarity, the smoother the video and the better the continuity. Analyzing a video from its content, it may contain multiple video segments with different contents. Therefore, first, find the change boundaries of the video scenes. For the images X and Y of two consecutive frames of the video, take the central region R of size n×n (n = 16) of the current frame image Y as the reference block, traverse and search the previous frame image X, find the most similar block region and calculate the similarity with this region.

[0088] The evaluation of video distortion models and statistically analyzes the spatio-temporal natural scenes of the video, comprehensively evaluating the degree of distortion in terms of time and space. The specific algorithm process is as follows:

[0089] For a video input sequence containing M frames, subtract two consecutive adjacent frames to obtain M - 1 difference frames. Each difference frame is divided into multiple image blocks of size n×n. Apply two-dimensional DCT to each image block of size n×n. The DCT coefficients of each image block from each difference frame are modeled as following a generalized Gaussian probability distribution. Assume the spatial size of each video frame is H×W, then each frame will have (H×W) / (n×n) DCT blocks, each DCT block contains n×n frequency coefficients, and each coefficient will appear (H×W) / (n×n) times in each difference frame. Here n = 5;

[0090] Next, use the parametric density function to fit the histogram of each frequency coefficient in each difference frame.

[0091]

[0092] where μ is the mean, γ is the shape parameter, and α and β are the normalization and scale parameters:

[0093]

[0094] where σ is the standard deviation, and Γ represents the original gamma function

[0095]

[0096] Next, obtain the γ coefficient matrix of size 5×5 corresponding to each difference frame by fitting the DCT coefficients on each difference frame.

[0097]

[0098] Determine the estimated value of the corrected mean of the absolute value:

[0099]

[0100] Calculate the ratio:

[0101]

[0102] The finally obtained optimal shape parameters, where:

[0103]

[0104] Up to here, a 5×5-sized γ matrix corresponding to each difference frame is obtained through parameter fitting. The γ matrix is divided into shape parameters in three frequency bands: low, medium, and high. The geometric mean of the shape parameters in each frequency band is calculated as:

[0105]

[0106] where f ∈ {low, mid, high}. In each 5×5-sized γ matrix,

[0107] {γ 12 , γ 13 , γ 21 , γ 22 , γ 23 , γ 31 , γ 32 , γ 33} is the low frequency,

[0108] {γ 14 , γ 15 , γ 24 , γ 33 , γ 42 , γ 43 , γ 51 , γ 52} is the medium frequency,

[0109] {γ 25 , γ 34 , γ 35 , γ 44 , γ 45 , γ 53 , γ 54 , γ 55} is the high frequency;

[0110] Finally, the following spectral ratios can be calculated:

[0111]

[0112] Finally, the spectral ratios of the shape parameters obtained above are input into the SVR model trained on the video dataset for quality score prediction.

[0113] In some embodiments, evaluation algorithms corresponding to each type of evaluation index can be generated based on the evaluation indexes corresponding to each type of evaluation index, and the evaluation algorithms can be encapsulated and integrated so that users can call them at any time. Specifically, evaluation algorithms corresponding to the parameter class data evaluation index type, the image class data evaluation index type, and the video class data evaluation index type can be written, and different evaluation algorithms can be converted into Singularity images through a Docker image repository, and the administrator is provided with permissions to import and store them in the corresponding database. For example, import and store them in the algorithm image library of the quality analysis and evaluation system.

[0114] It can be seen that the present invention can evaluate the quality of data more precisely and accurately. By comprehensively considering multiple dimensions of evaluation index types and the evaluation indexes corresponding to each type of evaluation index, the product quality can be evaluated more comprehensively, and the credibility and reliability of data quality evaluation can be improved. Encapsulating the evaluation algorithms corresponding to each type of evaluation index as Docker images and converting and uploading them through Singularity images realizes the standardization and modularization of the evaluation algorithms corresponding to each type of evaluation index, and improves the maintainability and scalability of the evaluation algorithms corresponding to each type of evaluation index.

[0115] In some embodiments, the data quality evaluation task can be executed in a manual / automatic timing manner.

[0116] Specifically, if the data quality evaluation task is executed in an automatic timing manner, a preset period can be set, and every time the preset period elapses, any file in at least one file to be analyzed that is in an unselected state is determined as any file, and any file is marked as the selected state. Then, steps S101-S104 can be continued to be executed. Among them, after determining that any file is abnormal, any file can be marked as unselected again.

[0117] If the data quality evaluation task is executed manually, a data analysis instruction input by the user can be received. In response to the data analysis instruction, any file is determined in at least one file to be analyzed. Then, steps S101-S104 can be continued to be executed.

[0118] The execution of the data quality evaluation task in a manual / automatic timing manner will be described in detail below with specific examples.

[0119] S11, adopt different working modes according to different requirements of data quality analysis and evaluation, including automatic timing tasks and manual tasks. The administrator can configure and start automatic tasks, and the administrator and the operator can configure and start manual tasks, which are divided into automatic working modes and manual working modes.

[0120] Among them, the quality analysis and evaluation modes corresponding to different data types are shown in Table 1 below:

[0121] Table 1

[0122]

[0123]

[0124] It should be noted that the manual working mode is also called the manual task-driven mode. According to the spacecraft identification, payload type, data level, data name (Chinese description, non-data file name), data acquisition start and end times, etc., select the data set to be evaluated, select the data quality evaluation process and start the quality evaluation workflow. The quality analysis and evaluation system collects the process information and algorithm results during the workflow execution and displays them on the user interface. The process of the manual working mode is as Figure 2 shown.

[0125] The automatic working mode is also called automatic task-driven. The process of the automatic working mode is as Figure 3 shown, including two modes: data quality extraction mode and data quality analysis mode, which respectively correspond to the extraction and calculation of 0-level data quality information for science and application and the sampling inspection of science and application high-level data products and the analysis of their quality information through algorithms. Among them, the 0-level data is to uniformly deformat the downlink all raw data to generate 0-level data products of various scientific data and engineering data.

[0126] S12. For all 0-level science and application data products, based on the automatic mode, configure the extraction and calculation of different types of 0-level data quality information, set the execution time and execution cycle (by day, by week, by month, by year), regularly parse the quality evaluation information in the file, save it to the data quality evaluation result table, and display it on the user interface. The 0-level data custom task process is as Figure 4 shown.

[0127] S13. For all 1-level and 2-level science and application data products and 0-level science and application data products of the multi-functional optical cabin, configure the quality analysis and evaluation processes for different levels and different types of data, select the data product types and levels to be sampled, set the sampling ratio, execution time and execution cycle (by day, by week, by month, by year), select the data quality evaluation process, and regularly trigger the quality evaluation process according to the user settings. Collect the data information (data list, data file storage address) within the execution cycle according to the sampling ratio, regularly start the quality evaluation process, and after the process execution is completed, save the quality evaluation result to the data quality evaluation result table and display it on the user interface. Among them, the data quality analysis task process is as Figure 5 shown.

[0128] It should be noted that the level-1 data is obtained by further extracting the payload observation / measurement data from each level-0 data product according to the data protocol format of each payload. Based on the scientific objectives of the payload, the types and characteristics of the observations / measurements carried out, the configured observation / measurement instruments, and the association and continuity between the observations / measurements, metadata is extracted, and the data and metadata are encapsulated into files in a standard format and output to provide users with high-quality level-1 data products for each payload. The level-2 data is obtained by further processing and encapsulating the level-1 data.

[0129] In some embodiments, a monitoring function can also be provided for all automatic and manual task-driven quality evaluation processes. The algorithm progress and status of the quality evaluation process can be viewed through a visual interface, and information such as spacecraft identification, payload type, data level, data type, start and end times of data acquisition, process name, algorithm metrics, process status (not started, completed), process start time, and process end time is displayed in a list form. Among them, the schematic diagram of the quality evaluation process monitoring page is as Figure 6 shown.

[0130] The beneficial effects of adopting the above further solution are as follows: the automatic execution of tasks is realized, ensuring the high efficiency of the quality evaluation task; the administrator can configure specific parameters of the task according to needs, such as execution cycle, sampling ratio, etc., realizing the refined management and control of the data quality analysis and evaluation work; for data products that have not been quality-evaluated or have been quality-evaluated, users can quickly locate the data set to be evaluated through manual initiation and initiate the process, ensuring the flexibility of the quality evaluation work.

[0131] S102: Determine the evaluation indicators corresponding to the target evaluation indicator type as the target evaluation indicators, and obtain the actual evaluation range of the data included in any file under each target evaluation indicator.

[0132] Among them, the target evaluation indicator type corresponds to multiple evaluation indicators, and each evaluation indicator corresponding to the target evaluation indicator type can be determined as the target evaluation indicator. Then, based on the data included in any file, the actual evaluation range of the data included in any file under each target evaluation indicator can be calculated.

[0133] S103: For any target evaluation indicator, determine that any file corresponding to any target evaluation indicator is abnormal based on the fact that the actual evaluation range of any file under any target evaluation indicator does not completely fall within the preset evaluation range of any target evaluation indicator.

[0134] In some embodiments, information associated with the abnormality of any target evaluation index corresponding to any file is displayed in the warning interface. Among them, the information associated with the abnormality of any target evaluation index corresponding to any file can be displayed in the warning interface in at least one of the forms of text, table, and image.

[0135] S104: Determine that any file is abnormal based on the ratio of the number of abnormal target evaluation indexes corresponding to any file to the total number of target evaluation indexes being greater than the abnormal ratio threshold.

[0136] Combining S103 and S104, it can be seen that this application can configure the abnormal indexes of data products and monitor and track the data with abnormal corresponding evaluation indexes. For example, a configuration page can be displayed, and the configuration page is used to support the user to set at least one evaluation index type. In response to the user's selection operation on the first evaluation index type among the at least one evaluation index type, a list including each evaluation index corresponding to the first evaluation index type is displayed. In response to the user's selection operation on the first evaluation index among the evaluation indexes included in the list, an input box for adjusting the preset evaluation range of the first evaluation index is displayed. In response to the user inputting the target preset evaluation range in the input box, the preset evaluation range of the first evaluation index is adjusted to the target preset evaluation range.

[0137] Furthermore, the following combines specific examples to elaborate in detail on configuring the abnormal indexes of data products in the present invention and monitoring and tracking the data with abnormal corresponding evaluation indexes.

[0138] The present invention can determine whether the current data product file is abnormal based on the evaluation result, and it is necessary to configure the data product abnormal indexes. The data product abnormal indexes are divided into index abnormal configuration and file abnormal configuration, and support two methods: automatic configuration and manual configuration.

[0139] Among them, the index abnormal configuration is the configuration for determining whether a certain index of a data product is abnormal. The content that needs to be configured includes the data product, file type, index classification, index name, and the normal value range of the index. When the index value calculated by the algorithm exceeds the normal value range, it is considered that the current index of the data product is abnormal. The automatic configuration of index abnormality is a method of obtaining the normal value range of the index through scientific calculation. By means of a timed task, the historical quality analysis and evaluation results of the current data product are scanned regularly, and then the normal value range is calculated. This method can effectively improve the configuration efficiency. At the same time, a visual interface is provided to support manual adjustment of the abnormal configuration of the index.

[0140] The automatic calculation method of index abnormality is, taking the clarity index of image type files as an example. First, sum the N clarity parameters x of the image type files under the current product i and calculate the average value of clarity According to the average value and the clarity parameter x i obtain the standard deviation s of the clarity of the image file of the current product N ; According to the normal distribution, the clarity parameters within three standard deviations account for 99% are normal, and the rest are abnormal parameters. The calculation formula is as follows:

[0141]

[0142] The file exception configuration is the configuration for determining whether the current data product file is abnormal. A single indicator exception does not necessarily mean that the file is abnormal. The content that needs to be configured includes the data product, file type, and abnormal indicator ratio. When the number of abnormal indicators / total indicators is greater than the abnormal indicator ratio value, it is considered that the current data product file is abnormal. For newly added data product types, the default abnormal ratio is 1 / 3, and a visual interface is provided to support adjusting the file exception configuration.

[0143] In some embodiments, the user can configure the file exception indicators of the data product through the visual interface, and needs to add the product name, file classification, abnormal indicator ratio, etc., and save them to the exception configuration table for abnormal analysis of the data quality evaluation results, supporting the configuration of different file types for a class of products. The specific configuration steps are as follows:

[0144] 1. First, select the data product and product classification;

[0145] 2. Enter the value of the abnormal indicator ratio;

[0146] Perform file configuration for all products under the selected section. The indicator classification includes evaluation types such as data verification, parameters, images, videos, and spectra. The value range of the abnormal indicator ratio is between 0 - 100%.

[0147] The user can also configure according to the indicator exceptions of the product file. The content that needs to be added includes the product name, file type, indicator classification, indicator name, indicator normal range, etc., and the data is saved to the indicator exception configuration table.

[0148] Support the configuration of different parameters for the same type of file. The specific steps are as follows:

[0149] 1. First, select the data product;

[0150] 2. For parameter - type data, the parameter indicators need to be selected. It is possible to perform a fuzzy search for the parameter indicator name and support selecting multiple items simultaneously;

[0151] 3. Click the data type drop - down box and select the corresponding data type;

[0152] 4. Click the drop-down box of indicator classification and select the indicator classification (including image evaluation indicator type, video evaluation indicator type, spectral evaluation indicator type, etc.);

[0153] 5. Click the drop-down box of evaluation indicator type classification, select the indicator names (supporting multiple selections at the same time), and for each selected indicator, set the preset evaluation range;

[0154] As Figure 7 shown, the result of the data product after the algorithm process will be compared with the normal range of the indicators. The result within the normal indicator range is a normal result, and the result outside the normal indicator range is an abnormal result; then, according to the abnormal indicator ratio of the product file = the number of abnormal indicators / the total number of indicators, if the abnormal indicator ratio exceeds the abnormal indicator ratio data in the abnormal configuration table, it is determined as an abnormal file.

[0155] It can be seen that the present invention can perform detailed abnormal configuration for multiple indicators of the data product, thereby improving the accuracy and comprehensiveness of the abnormal analysis of the data quality evaluation result; users can quickly understand the overall situation and specific problems of the data quality based on key information such as abnormal data types, abnormal generation time, evaluation indicators, etc., so as to take timely and effective countermeasures.

[0156] In some embodiments, information associated with any file abnormality can be displayed in the warning interface. Among them, the information associated with any file abnormality can be displayed in the warning interface in at least one of the forms of text, table, and image.

[0157] Specifically, according to the data quality evaluation analysis result, a quality analysis evaluation statistical chart and an analysis report can be generated and displayed in the warning interface.

[0158] Among them, the warning interface will default to collect the abnormal information of each analysis result involved in the quality analysis and evaluation here and display it to the user in the form of a list. The displayed data information includes abnormal data types, abnormal generation time, evaluation indicator types (parameter class data evaluation indicator type, image class data evaluation indicator type, video class data evaluation indicator type, etc.), abnormal descriptions, etc., and supports conditional filtering of file, data type, completion time, etc. of the abnormal information; at the same time, it also provides a statistical bar chart of abnormal information at the product file level and the indicator level, which can be filtered according to time period, product, file type, etc., to facilitate quick positioning to abnormal files and abnormal indicators. The schematic diagram of abnormal result query is as Figure 8 shown.

[0159] See Figure 9, the warning interface can also collect the results of the analysis and evaluation of all data. The result query and statistics are presented in various advanced two / three-dimensional data visualization forms such as tables, line charts, bar charts, pie charts, radar charts, and funnel charts. The results of the quality analysis and evaluation of various scientific experiment data are statistically displayed according to experimental payloads, data types, data result statuses (completed, in progress, not started, and failed), index classifications, result statistics, and evaluation indicators.

[0160] In some embodiments, quality anomaly tracking analysis, quality statistical analysis, and quality evaluation task summary analysis can also be performed based on the quality evaluation results, respectively forming the following three types of reports: Quality Analysis and Evaluation Anomaly Analysis Report (referred to as the anomaly report), Quality Analysis and Evaluation Statistical Report (referred to as the statistical report), and Quality Analysis and Evaluation Summary Report (referred to as the summary report). The present invention can also provide Figure 10 a page for querying and retrieving quality analysis and evaluation reports as shown, and Figure 11 a page that supports previewing quality analysis and evaluation reports and downloading selected quality analysis and evaluation reports to the local area as shown.

[0161] It can be seen that the present invention can use intuitive data visualization to make complex data quality analysis intuitive and easy to understand; assist in scientific management and decision-making, and improve the platform operation and maintenance capabilities.

[0162] The present invention also provides a system for supporting the quality analysis and evaluation of space science and application data. Refer to Figure 12 , the system includes a user authentication and log management module, a quality analysis and evaluation management module, a quality evaluation anomaly and analysis module, and a result and report management module.

[0163] Among them, the user authentication and log management module is used for user permission authentication of the quality analysis and evaluation system and for providing functions such as log query and log export for users.

[0164] The data quality analysis and evaluation module is used for the encapsulation and integration of the foregoing evaluation algorithms, constructing a quality evaluation process and starting it, abnormal data analysis, result statistics, and report generation, and is the main module of the system for supporting the quality analysis and evaluation of space science and application data.

[0165] The quality analysis and evaluation management module is used for the encapsulation and integration of the foregoing evaluation algorithms, constructing a quality evaluation process, automatic task configuration, manual initiation of the quality evaluation process, and monitoring and querying of the quality evaluation process.

[0166] Specifically, the quality analysis and evaluation management module is used to determine a target evaluation index type from at least one evaluation index type for any one of at least one file to be analyzed, based on the data type of the data included in any one of the files. Among them, different evaluation index types correspond to different evaluation indexes, and the evaluation index type is used to characterize the preset evaluation range of the data of the corresponding data type. Determine the evaluation index corresponding to the target evaluation index type as the target evaluation index, and obtain the actual evaluation range of the data included in any one of the files under each target evaluation index. For any one of the target evaluation indexes, determine that any one of the target evaluation indexes corresponding to any one of the files is abnormal based on the fact that the actual evaluation range of any one of the files under any one of the target evaluation indexes does not completely fall into the preset evaluation range of any one of the target evaluation indexes. Determine that any one of the files is abnormal based on the ratio of the number of abnormal target evaluation indexes corresponding to any one of the files to the total number of target evaluation indexes being greater than the abnormal ratio threshold.

[0167] The quality evaluation anomaly and analysis module is used to configure data product anomaly indicators, query the details of data products with abnormal indicators, and trace the source.

[0168] Specifically, the quality evaluation anomaly and analysis module is used to display a configuration page, and the configuration page is used to support the user to set at least one evaluation index type. In response to the user's selection operation on the first evaluation index type among at least one evaluation index type, display a list including each evaluation index corresponding to the first evaluation index type. In response to the user's selection operation on the first evaluation index among each evaluation index included in the list, display an input box for adjusting the preset evaluation range of the first evaluation index. In response to the user inputting a target preset evaluation range in the input box, adjust the preset evaluation range of the first evaluation index to the target preset evaluation range.

[0169] The result and report management module is used for quality evaluation result query and statistics, report configuration generation, and report query.

[0170] Specifically, the result and report management module is used to display a warning interface, and the warning interface includes information associated with any abnormal target evaluation index corresponding to any one of the files, and information associated with any abnormal file. Among them, the information associated with any abnormal target evaluation index corresponding to any one of the files and the information associated with any abnormal file are displayed in the warning interface in at least one of the forms of text, table, and image.

[0171] In some embodiments, data information (data list, data file storage address) within the execution cycle can also be collected according to the sampling inspection task, and the quality evaluation process can be periodically initiated. Specifically, by reading data from various space science experiment payloads, the data information (data list, data file storage address) and the process template can be sent to the Kafka message queue. The system provided by the present invention can simultaneously monitor the Kafka message, obtain data product information, allocate different quality analysis and evaluation processes, and initiate the processes. After the quality evaluation process is completed, the algorithm program needs to send the calculation results to the Kafka message queue. The system provided by the present invention can obtain the information in the message queue, parse the data to obtain the quality evaluation results and store them in the database. For the data files that fail in the quality evaluation during the process, the information of the failed data files and their result status can be recorded in the database, and this failed data can be selected again to initiate the process.

[0172] The system provided by the embodiments of the present application can also be connected to the space science experiment payload data archiving and management module. Among them, the space science experiment payload data archiving and management module is used to store and manage scientific and application data. The system provided by the embodiments of the present application can obtain the data directory, data address, data details, etc. in the space science experiment payload data archiving and management module.

[0173] It can be seen that the present invention improves the stability of the quality analysis and evaluation system; records the data files that fail in the quality evaluation during the process, allowing users or administrators to select these failed data again to initiate the process in subsequent operations, improving the reliability and integrity of the data quality evaluation.

[0174] To better illustrate and understand the principle of the method provided by the present invention, the solution of the present invention will be described below in conjunction with an optional specific embodiment. It should be noted that the specific implementation manners of each step in this specific embodiment should not be construed as a limitation to the solution of the present invention. Based on the principle of the solution provided by the present invention, other implementation manners that can be thought of by those skilled in the art should also be regarded as within the protection scope of the present invention.

[0175] In this embodiment, taking the scenario of analyzing and evaluating data products of space science and application level 1 parameter types as a specific embodiment, the operation is implemented according to the following steps in this embodiment:

[0176] S1, Write and encapsulate the parameter evaluation algorithm program.

[0177] The parameter evaluation algorithm archives the data files (csv files) in the received data archiving software, retrieves the data, extracts all different types of parameters recorded in the data files, including temperature, humidity, liquid / gas composition, speed, microgravity, light intensity, etc., calculates for each type of parameter the proportion of reasonable values, maximum and minimum values, mean, median, standard deviation, etc., and saves the calculation results in the quality evaluation result table.

[0178] The parameter evaluation algorithm is a python program that needs to be encapsulated in a Docker image, configure the algorithm program, package and upload the configuration file and the algorithm image file, as Figure 13 shown. The algorithm program needs to send three types of messages, namely control, progress, and details, to the Kafka of the quality analysis and evaluation system to remind the start of the next step. After the algorithm program finishes execution, it puts the algorithm results into the detailed message and sends it to the Kafka message. The quality analysis and evaluation system obtains the algorithm results and stores them in the database.

[0179] S2. Configure the data quality analysis task for the data products of the first-level parameter types.

[0180] The user selects the data products of the first-level parameter types to be sampled in the quality analysis and evaluation system, sets the sampling ratio, execution time, and execution cycle (daily, weekly, monthly, yearly), and selects the data quality evaluation process. The schematic diagram of the data quality analysis task configuration interface is as Figure 14 shown.

[0181] The data quality analysis task extraction rule: Use the time period set by the automatic task as the starting point for extracting data, and randomly and evenly extract based on this time period for different data types. For example, by reading the data of the space science experiment payload, it is obtained that there are 100 pieces of data in the time period from January 26th, 2022 to January 29th, 2022 for the data products of the first-level parameter types. Randomly and evenly extract 20 pieces of data from these 100 pieces of data according to a 20% ratio. The sampling quantity = extraction percentage * total number of pieces in the data directory of this type.

[0182] Based on the sampling task configuration, the quality analysis and evaluation system will automatically create corresponding scheduled tasks. The system will regularly trigger the quality evaluation process of the first-level parameter data products according to the user settings, collect the data information (data list, data file storage address) within the execution cycle according to the sampling ratio, and regularly send the data information and the process template to the message queue of Kafka. The quality analysis and evaluation system listens to the Kafka message and starts the quality evaluation process.

[0183] S3. Configure the abnormal index range for the data products of the first-level parameter types.

[0184] Parameter type data products (CSV files) may contain different types of parameters, such as temperature, humidity, liquid / gas composition, speed, microgravity, light intensity, etc. The calculation of each type of parameter includes six types of index data: reasonable value proportion, maximum and minimum values, mean value, median value, and standard deviation. For parameter type data, the reasonable value proportion index is mainly concerned. Users select the level-1 parameter type data products to be sampled in the quality analysis and evaluation system and configure the normal value ranges of the reasonable value proportion indexes for each parameter. The schematic diagram of abnormal index configuration is as shown in Figure 15 shown. At the same time, to improve the operation efficiency of administrators, the quality analysis and evaluation system also supports the batch abnormal setting function.

[0185] S4, Quality evaluation process monitoring.

[0186] Through quality evaluation process monitoring, the algorithm progress and status of the quality evaluation process, the name of the quality evaluation process, algorithm indicators, process status (not started, completed), etc. can be viewed. The schematic diagram of the quality evaluation process monitoring page is as shown in Figure 7 shown.

[0187] S5, Quality evaluation result generation.

[0188] After the quality evaluation process is executed, the algorithm results will be placed in the detailed report and sent to the Kafka message. The quality analysis and evaluation system obtains the algorithm results by listening to the Kafka message and parses the index values of the parameters. The obtained reasonable value proportion index value is compared with the normal range of the index. If the index value exceeds the normal range of the index, it proves that the current parameter is abnormal; if the index value is within the normal range, it proves that the current parameter is normal. Calculate the proportion of abnormal parameter indexes of the product: Proportion of abnormal parameter indexes = Number of abnormal parameters / Total number of parameters. Compare the calculated proportion of abnormal parameters with the corresponding abnormal index proportion in the abnormal index configuration table. If it exceeds, the current file is abnormal.

[0189] After obtaining the quality evaluation results of the product file, it is stored in the database for preservation, based on the MySQL database.

[0190] S6, Quality evaluation result statistics and report generation.

[0191] The quality evaluation result statistics interface, in the form of line charts, bar charts, pie charts, etc., shows the quality evaluation quantity, completion ratio of the current level-1 parameter type data products, and the normal, abnormal, and evaluation failure situations of product quality over time.

[0192] Supports generating abnormal analysis reports and quality analysis and evaluation statistical reports for the current data product. There are two types of reports. One is the incremental report, which generates a statistical report within a certain time range; the other is the cumulative mode, which generates a statistical report by summarizing all quality evaluation results.

[0193] Among them, the above-mentioned system for supporting the quality analysis and evaluation of space science and application data can be a computer program (including program code) running on a computer device. For example, the system for supporting the quality analysis and evaluation of space science and application data is an application software; this system can be used to execute the corresponding steps in the method provided by the embodiments of the present invention.

[0194] In some solutions, multiple embodiments of the present application can be combined and the combined solution can be implemented. Optionally, some operations in the processes of the method embodiments are optionally combined, and / or the order of some operations is optionally changed. Moreover, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps. The steps can also be in other execution orders. It is not intended to indicate that the described execution order is the only order in which these operations can be performed. Those of ordinary skill in the art will think of various ways to reorder the operations described herein. In addition, it should be noted that the process details involved in a certain embodiment herein are equally applicable to other embodiments in a similar manner, or different embodiments can be combined and used.

[0195] In addition, some steps in the method embodiments can be equivalently replaced with other possible steps. Or, some steps in the method embodiments can be optional and can be deleted in some usage scenarios. Or, other possible steps can be added to the method embodiments. Moreover, the method embodiments can be implemented separately or in combination.

[0196] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to complete all or part of the functions described above.

[0197] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the system or unit can be in electrical, mechanical or other forms.

[0198] In addition, in each embodiment of the present application, each functional unit may be integrated into a processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0199] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that makes a contribution, or all or part of the technical solution, may be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which may be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0200] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for supporting data quality analysis and evaluation for space science and applications, characterized in that: include: For any file in at least one file to be analyzed, based on the data type of data included in the any one file, determine a target evaluation indicator type in at least one evaluation indicator type; wherein the at least one evaluation indicator type includes a parameter data evaluation indicator type, an image data evaluation indicator type, and a video data evaluation indicator type; different evaluation indicator types correspond to different evaluation indicators; the evaluation indicator is used to characterize a preset evaluation range of data of the corresponding data type; the evaluation indicator corresponding to the parameter data evaluation indicator type at least includes a data mean, a data median, a data standard deviation, and a reasonable data ratio; wherein the reasonable data ratio is a ratio of a first value to a second value; the first value is the number of sampled data falling within a preset minimum reasonable value range and a preset maximum reasonable value range; the second value is the total number of all sampled data; Determine the evaluation indicator corresponding to the target evaluation indicator type as the target evaluation indicator, and obtain the actual evaluation range of the data included in any one of the files under each of the target evaluation indicators; For any target evaluation indicator, based on the fact that the actual evaluation range of any file under any target evaluation indicator does not completely fall within the preset evaluation range of any target evaluation indicator, determining that any target evaluation indicator corresponding to any file is abnormal; Based on the fact that the ratio of the number of abnormal target evaluation indicators corresponding to the any file to the total number of the target evaluation indicators is greater than an abnormal proportion threshold, it is determined that the any file is abnormal.

2. The method according to claim 1, characterized in that The step of determining a target evaluation indicator type from at least one evaluation indicator type based on the data type of the data included in any one of the files comprises: In the case where any of the files includes data of a parameter type or a binary type, determining the parameter type of data evaluation index type as the target evaluation index type; In the case that any of the files includes data of an image data type or a spectral data type, determining the image data evaluation index type as the target evaluation index type; In the case that any of the files includes data of a video data type, the video data evaluation index type is determined as the target evaluation index type.

3. The method according to claim 2, characterized in that The evaluation index corresponding to the image data evaluation index type includes at least one of image signal-to-noise ratio, image clarity, image contrast, image information entropy, a range of grayscale values ​​of pixels included in the image, image brightness, image resolution, image pixels, and an occlusion rate of an experimental target in the image; The evaluation indicators corresponding to the video data evaluation indicator type include at least one of video signal-to-noise ratio, video clarity, video contrast, video information entropy, the range of grayscale values ​​of pixels included in the video, video brightness, video resolution, video pixels, video frame rate, video continuity, and video distortion; wherein the video continuity is associated with the similarity between adjacent frame images in the video.

4. The method according to claim 2, characterized in that: Based on the data type of the data included in any one of the files, before determining the target evaluation indicator type in at least one evaluation indicator type, the method further includes: At every preset period, any file in the at least one file to be analyzed that is in an unselected state is determined as the any file, and the any file is marked as being in a selected state.

5. The method according to claim 4, characterized in that After determining that any one of the files is abnormal, the method further includes: Mark any of the files as unselected.

6. The method according to claim 2, characterized in that Based on the data type of the data included in any one of the files, before determining the target evaluation indicator type in at least one evaluation indicator type, the method further includes: Receiving data analysis instructions input by a user; In response to the data analysis instruction, the any one file is determined in the at least one file to be analyzed.

7. The method according to claim 1, characterized in that After determining that any target evaluation indicator corresponding to any file is abnormal, the method further includes: Displaying information associated with the abnormality of any target evaluation indicator corresponding to any file in the early warning interface; After determining that any one of the files is abnormal, the method further includes: Displaying information associated with any of the file anomalies in the warning interface; Among them, the information associated with the abnormality of any target evaluation indicator corresponding to any file and the information associated with the abnormality of any file are displayed in the early warning interface in at least one form of text, table, and image.

8. The method according to claim 1, characterized in that Also includes: Displaying a configuration page; the configuration page is used to support the user to set the at least one evaluation indicator type; In response to a user input selecting a first evaluation indicator type among the at least one evaluation indicator type, displaying a list of evaluation indicators corresponding to the first evaluation indicator type; In response to a user input of selecting a first evaluation indicator among the evaluation indicators included in the list, an input box for adjusting a preset evaluation range of the first evaluation indicator is displayed; In response to the user inputting a target preset evaluation range in the input box, the preset evaluation range of the first evaluation indicator is adjusted to the target preset evaluation range.

9. A system for supporting data quality analysis and evaluation for space science and applications, characterized in that: include: Quality analysis and evaluation management module; the quality analysis and evaluation management module is used to: For any file in at least one file to be analyzed, based on the data type of data included in the any one file, determine a target evaluation indicator type in at least one evaluation indicator type; wherein the at least one evaluation indicator type includes a parameter data evaluation indicator type, an image data evaluation indicator type, and a video data evaluation indicator type; different evaluation indicator types correspond to different evaluation indicators; the evaluation indicator is used to characterize a preset evaluation range of data of the corresponding data type; the evaluation indicator corresponding to the parameter data evaluation indicator type at least includes a data mean, a data median, a data standard deviation, and a reasonable data ratio; wherein the reasonable data ratio is a ratio of a first value to a second value; the first value is the number of sampled data falling within a preset minimum reasonable value range and a preset maximum reasonable value range; the second value is the total number of all sampled data; Determine the evaluation indicator corresponding to the target evaluation indicator type as the target evaluation indicator, and obtain the actual evaluation range of the data included in any one of the files under each of the target evaluation indicators; For any target evaluation indicator, based on the fact that the actual evaluation range of any file under any target evaluation indicator does not completely fall within the preset evaluation range of any target evaluation indicator, determining that any target evaluation indicator corresponding to any file is abnormal; Based on the fact that the ratio of the number of abnormal target evaluation indicators corresponding to the any file to the total number of the target evaluation indicators is greater than an abnormal proportion threshold, it is determined that the any file is abnormal.

10. The system according to claim 9, characterized in that Also includes: Quality assessment anomaly and analysis module, result and report management module; among them, The quality assessment anomaly and analysis module is used to: Displaying a configuration page; the configuration page is used to support the user to set the at least one evaluation indicator type; In response to a user input selecting a first evaluation indicator type among the at least one evaluation indicator type, displaying a list of evaluation indicators corresponding to the first evaluation indicator type; In response to a user input of selecting a first evaluation indicator among the evaluation indicators included in the list, an input box for adjusting a preset evaluation range of the first evaluation indicator is displayed; In response to a user inputting a target preset evaluation range in the input box, adjusting the preset evaluation range of the first evaluation indicator to the target preset evaluation range; The result and report management module is used to: Display a warning interface; the warning interface includes information associated with the abnormality of any target evaluation indicator corresponding to any file, and information associated with the abnormality of any file; wherein the information associated with the abnormality of any target evaluation indicator corresponding to any file and the information associated with the abnormality of any file are displayed in the warning interface in at least one form of text, table, and image.

Images