Data evaluation method, device, electronic device and storage medium
By standardizing the synthetic data set and analytical model evaluation, cumulative variance interpretation indicators and Jensen-Shannon divergence are determined, and evaluation models are constructed to evaluate the synthesis quality of synthetic data, which solves the credibility and reliability of synthetic data and achieves more efficient data evaluation and improvement.
Patent Information
- Application Number
- CN202411455392.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-10-15
- Filing Date
- 2024-10-18
- Publication Date
- 2025-04-01
AI Technical Summary
How to ensure the credibility, rationality, effectiveness and reliability of synthetic data, so as to expand the knowledge boundaries of large-scale models and promote large-scale model reasoning and generalization capabilities.
The datasets containing the original and synthetic datasets are obtained, and the datasets are standardized and entered into a pre-established analytical model to determine the cumulative variance interpretation metrics and Jensen-Shannon divergence. These metrics are then weighted averaged to build an evaluation model and evaluate the synthesis quality of the synthetic dataset.
It improves the credibility, rationality, effectiveness and reliability of synthetic data. By comprehensively evaluating the authenticity and credibility of synthetic data, it provides application suggestions for improving data sets, such as model training, model alignment and data interpolation.
Smart Images
Figure CN119739991A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of generative artificial intelligence technology, and specifically to a data evaluation method, device, electronic device and storage medium. Background Art
[0002] With technological advancements, particularly breakthroughs in deep learning and natural language processing, the demand for large amounts of high-quality data continues to increase. Real-world data is often difficult or prohibitively expensive to obtain, and can raise privacy and security concerns. The development of synthetic data generation technology has opened up new avenues for efficient data utilization and intelligent analysis.
[0003] Among related technologies, the development of deep learning techniques such as variational autoencoders (VAEs) and generative adversarial networks (GANs) has enabled the quality of synthetic data to become increasingly high and indistinguishable from real data. Furthermore, the success of large language models (LLMs) such as Generative Prediction (GPT) in natural language processing has demonstrated the enormous potential of synthetic data for text generation. These technological developments have not only advanced research in synthetic data generation but also provided new solutions for data processing and analysis in practical applications.
[0004] Although there are currently a variety of solutions for generating synthetic data, how to ensure the credibility, rationality, validity, and reliability of synthetic data in order to expand the knowledge boundaries of large models and promote the reasoning and generalization capabilities of large models is an urgent issue that needs to be addressed. Summary of the Invention
[0005] In view of the above problems, the present application provides a data evaluation method, device, electronic device and storage medium to at least solve the problems existing in the related art.
[0006] In a first aspect, an embodiment of the present application provides a data evaluation method, the data evaluation method comprising:
[0007] Acquire a data set, wherein the data set includes: an original data set and a synthetic data set generated by using a model based on the original data set;
[0008] performing standardization processing on the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set;
[0009] Inputting the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index and a Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set;
[0010] The cumulative variance explained index and the Jensen-Shannon divergence are weighted averaged to construct an evaluation model, and the synthesis quality of the synthetic dataset is evaluated according to the evaluation model.
[0011] In some embodiments, inputting the standardized data set into a pre-established analysis model to determine a cumulative variance explained index comprises:
[0012] Calculating the covariance matrix of the standardized original data set and the standardized synthetic data set;
[0013] Performing eigendecomposition processing based on the covariance matrix to obtain corresponding eigendecomposition results, wherein the eigendecomposition results include: eigenvalues and eigenvectors;
[0014] The cumulative variance explained index is determined based on the eigendecomposition result.
[0015] In some embodiments, determining the cumulative variance explained index based on the eigendecomposition result includes:
[0016] Calculate a first variance explanation ratio of the standardized original data set and a second variance explanation ratio of the standardized synthetic data set based on the eigenvalues and eigenvectors respectively;
[0017] Setting a first threshold and calculating a first cumulative variance explained ratio of the standardized original data set according to the first threshold and the first variance explained ratio;
[0018] setting a second threshold and calculating a second cumulative variance explained proportion of the standardized synthetic data set according to the second threshold and the second variance explained proportion;
[0019] The cumulative variance explained index is determined according to a ratio of the first cumulative variance explained ratio to the second cumulative variance explained ratio.
[0020] In some embodiments, inputting the standardized data set into a pre-established analysis model to determine the Jensen-Shannon divergence comprises:
[0021] Calculating the probability distribution of the normalized original data set and the normalized synthetic data set;
[0022] Calculating based on the probability distribution to obtain an average distribution;
[0023] The Jensen-Shannon divergence is calculated based on the probability distribution and the mean distribution.
[0024] In some embodiments, performing weighted averaging on the cumulative variance explained index and the Jensen-Shannon divergence to construct an evaluation model includes:
[0025] Different weight coefficients are set for the cumulative variance explanation index and the Jensen-Shannon divergence according to data evaluation requirements;
[0026] An evaluation model is constructed by summing the weight coefficient and the product of the cumulative variance explained index and the Jensen-Shannon divergence.
[0027] In some embodiments, normalizing the data set to obtain a standardized data set includes:
[0028] The standardized data set is obtained by applying a minimum-maximum normalization process or a score normalization process to the data set.
[0029] In some embodiments, evaluating the synthesis quality of the synthetic dataset according to the evaluation model includes:
[0030] Evaluate the analysis results of the cumulative variance explanation index and Jensen-Shannon divergence corresponding to the processed synthetic data set according to the evaluation model;
[0031] An application suggestion for improving the synthesis quality of the synthetic data set is given according to the evaluation model evaluation, wherein the application suggestion includes: model training, model alignment and data interpolation.
[0032] In a second aspect, an embodiment of the present application provides an over-temperature protection device, comprising:
[0033] The acquisition module is used to acquire a data set, wherein the data set includes: an original data set and a synthetic data set generated according to the original data set using a model.
[0034] The processing module is configured to perform standardization processing on the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set.
[0035] A determination module is used to input the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index and a Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set.
[0036] An evaluation module is used to perform weighted averaging processing on the cumulative variance explained index and the Jensen-Shannon divergence to construct an evaluation model, and evaluate the synthesis quality of the synthetic data set according to the evaluation model.
[0037] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores program code that can be run on the processor, and when the program code is executed by the processor, the data evaluation method described in any embodiment of the first aspect is implemented.
[0038] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing one or more programs, which can be executed by the electronic device as described in the third aspect to implement the data evaluation method as described in any embodiment of the first aspect.
[0039] The embodiments of the present application provide a data evaluation method, device, electronic device and storage medium, which obtain a data set, wherein the data set includes: an original data set and a synthetic data set generated by using a model based on the original data set, and standardizes the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set, and inputs the standardized data set into a pre-established analysis model to determine the cumulative variance explanation index and the Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the probability distribution similarity of the standardized original data set and the standardized synthetic data set, and performs weighted averaging on the cumulative variance explanation index and the Jensen-Shannon divergence to construct an evaluation model, and evaluates the synthesis quality of the synthetic data set based on the evaluation model. The present application adopts a distribution similarity method and a cumulative variance explanation method, and effectively combines them, and introduces a weight coefficient, which can comprehensively evaluate the authenticity and credibility of the synthetic data.
[0040] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Hereinafter, the present application will be described in more detail based on embodiments with reference to the accompanying drawings.
[0042] Figure 1 A schematic diagram of a data evaluation method process proposed in one embodiment of the present application is shown;
[0043] Figure 2 The following is a structural block diagram of a data evaluation device proposed in one embodiment of the present application;
[0044] Figure 3 A structural block diagram of an electronic device for executing a data evaluation method according to an embodiment of the present application is shown;
[0045] Figure 4A computer-readable storage medium proposed in an embodiment of the present application for storing or carrying a data evaluation method according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0046] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.
[0047] Although there are currently a variety of solutions for generating synthetic data, there is relatively little research in the field of how to ensure the credibility, rationality, validity, and reliability of synthetic data to expand the knowledge boundaries of large models and promote the reasoning and generalization capabilities of large models.
[0048] In the study of related technologies, it was found that the source data of industrial product defects is usually used to train the data generation model and obtain synthetic defect generation data, which is then input into the comprehensive performance evaluation index model to calculate the performance parameter values of the synthetic defect generation data, which is then input into the defect substitutability evaluation model to calculate the final substitutability evaluation value.
[0049] After analyzing the above technology, the applicant found that the problem is that it only considers the evaluation index of the defective data substitutability (rationality), but does not consider the reliability and credibility dimension index of the defective data.
[0050] Based on this, applicants score the authenticity of synthetic data, that is, how to determine whether the synthetic data can effectively represent the characteristics and patterns of the original data source in the target field. This involves in-depth analysis and understanding of data distribution, as well as data sampling and adjustment for specific tasks. The credibility of synthetic data is mainly based on whether the original data source can be traced or verified for comprehensive improvement.
[0051] In response to the above problem description, the applicant has proposed a data evaluation method, device, electronic device, and storage medium provided in the embodiments of the present application. Specifically, the method can be performed by obtaining a data set including an original data set and a synthetic data set generated by using a model based on the original data set, normalizing the data set to obtain a standardized data set, and inputting the standardized data set into a pre-established analysis model to determine the cumulative variance explained index and the Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set. The cumulative variance explained index and the Jensen-Shannon divergence are weighted averaged to construct an evaluation model, and the synthesis quality of the synthetic data set is evaluated based on the evaluation model to improve the credibility, rationality, validity, and reliability evaluation of the synthetic data. The data evaluation method is described in detail in the subsequent embodiments.
[0052] The following describes the application scenarios of the data evaluation method provided in the embodiments of the present application:
[0053] See also Figure 1 , Figure 1 This is a flow chart of a data evaluation method provided in an embodiment of the present application. In this embodiment, the data evaluation method can be applied to Figure 2 The data evaluation device 300 shown is Figure 3 In the electronic device 200 shown, the electronic device may include one or more electronic devices, and information may be transmitted between the multiple electronic devices in a wireless and / or wired manner. The multiple electronic devices may collaborate to complete the data evaluation method. For example, the electronic device may include a mobile terminal, a computer, a tablet, etc., which is not limited in this application. Figure 1 The process shown is described in detail, and the data evaluation method may include S110 to S140.
[0054] S110: Acquire a data set, where the data set includes: an original data set and a synthetic data set generated by using a model based on the original data set.
[0055] In the embodiment of the present application, the original data set X: {x1, x2, ....., x m}, which represents a representative sample of real data, the synthetic dataset Y{y1,y2,....,y n}, which can generate data based on the characteristics of the original dataset through generative models (such as GAN, VAE, etc.).
[0056] It should be noted that datasets can include continuous and discrete data sets. Continuous data is an unrestricted, infinite data type, such as temperature. Discrete data is a finite data type with a fixed range of values and no continuous relationship between values, such as height, which ranges from 10 cm to 250 cm. A discrete dataset is a dataset consisting of discrete data. This application is preferably applied to discrete datasets.
[0057] S120: performing normalization processing on the data set to obtain a normalized data set, wherein the normalized data set includes: a normalized original data set and a normalized synthetic data set.
[0058] In the embodiment of the present application, for a set of original data sets X consisting of m original data samples: {x1, x2, ..., x m}(including original data samples x1, x2, ....., x m ) and a set of synthetic data sets Y{y1,y2,....,yn Standardization is performed to ensure data comparability.
[0059] In some embodiments, step S120 may include S121.
[0060] S121: Obtain a standardized data set by applying minimum-maximum normalization processing or score normalization processing to the data set.
[0061] In some embodiments, the minimum-maximum normalization method is used for normalization to obtain the normalized raw data samples. And the standardized synthetic data samples after normalization
[0062] Among them, the standardization processing formula of the original data set is:
[0063]
[0064] Where, i is the i-th original data sample, is the i-th original data sample after standardization, min(·) is the minimum value in the original data set X, and max(x) is the maximum value in the original data set X.
[0065] The normalization formula for the synthetic data set is:
[0066]
[0067] Where y i is the i-th synthetic data sample, is the i-th synthetic data sample after standardization, min(y) is the minimum value in the synthetic data set Y, and max(y) is the maximum value in the synthetic data set Y.
[0068] In other embodiments, z-score normalization is used to obtain a normalized dataset.
[0069] Among them, the standardization processing formula of the original data set is:
[0070]
[0071] Where x i is the i-th original data sample, is the i-th original data sample after standardization, μ is the mean of the original data set X, and σ is the standard deviation of the original data set X.
[0072] The normalization formula for the synthetic data set is:
[0073]
[0074] Where y i is the i-th synthetic data sample, is the i-th synthetic data sample after standardization, is the mean of the synthetic dataset Y, is the standard deviation of the synthetic dataset Y.
[0075] S130: Input the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index and a Jason-Shannon divergence, wherein the Jason-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set.
[0076] In this embodiment, the standardized original data set X norm ={x i ,.....,x inorm ,....,x mnorm} and standardized synthetic dataset Y norm ={y 1norm ,.....,y inorm ,....y nnorm} Dimensionality reduction is performed and principal component analysis is used to extract the main features.
[0077] In some embodiments, step S130 of inputting the standardized data set into a pre-established analysis model to determine the cumulative variance explained index may include S131 to S133.
[0078] S131: Calculate the covariance matrix of the standardized original data set and the standardized synthetic data set.
[0079] In the embodiment of the present application, the covariance matrix calculation formula of the standardized original data set is:
[0080]
[0081] Where ∑original is the covariance matrix of the standardized original data set, m is the number of samples in the original data set, X norm To standardize the original dataset, is the transposed matrix of the normalized original dataset.
[0082] The covariance matrix of the standardized synthetic dataset is calculated as:
[0083]
[0084] Where ∑synthesize is the covariance matrix of the standardized synthetic data set, n is the number of samples in the standardized synthetic data set, and Y norm To standardize the synthetic dataset, is the transposed matrix of the normalized synthetic dataset.
[0085] S132: Perform eigendecomposition processing based on the covariance matrix to obtain corresponding eigendecomposition results, where the eigendecomposition results include: eigenvalues and eigenvectors.
[0086] The eigenvalues and corresponding eigenvectors of the standardized original dataset and the standardized synthetic dataset are obtained by solving the characteristic polynomials of the covariance matrices of the standardized original dataset and the standardized synthetic dataset.
[0087] In the embodiment of the present application, the formula for solving the eigenvalues and corresponding eigenvectors of the normalized original data set is:
[0088] ∑Original v 原始 =λ 原始 v 原始 ;
[0089] Where ∑ original is the covariance matrix of the standardized original data set, v 原始 is a set of feature vectors of the standardized original data set, which can be further expressed as v 原始 ={ω1,...,ω i ,...},λ 原始 Represents a set of eigenvalues of the normalized original dataset, denoted as λ 原始 ={λ 1，原始 ,...,λ i，原始 ,...}.
[0090] The formula for solving the eigenvalues and corresponding eigenvectors of the standardized synthetic data set is:
[0091] ∑ synthesis v 合成 =λ 合成 v 合成 ;
[0092] Where ∑synthesize is the covariance matrix of the standardized synthetic dataset, v 合成 is a set of feature vectors of the standardized synthetic dataset, v 合成 Further expressed as λ 合成 is a set of feature values of the standardized synthetic dataset, which can be further expressed as λ 合成 ={λ 1，合成 ,...,λ i，合成 ,...}.
[0093] S133: Determine the cumulative variance explanation index based on the eigendecomposition results.
[0094] The variance explanation ratio and cumulative variance explanation ratio of the standardized original data set and the standardized synthetic data set are calculated respectively, and the cumulative variance explanation ratio of the standardized original data set and the variance explanation ratio of the standardized synthetic data set are compared. The closer the ratio is to 1, the higher the synthesis quality of the synthetic data set.
[0095] In the embodiment of the present application, step S133 may include S1331 to S1334.
[0096] S1331: Calculate the first variance explanation ratio of the standardized original data set and the second variance explanation ratio of the standardized synthetic data set based on the eigenvalues and eigenvectors.
[0097] In the embodiment of the present application, the first variance explanation ratio calculation formula is:
[0098]
[0099] Where p i,原始 is the variance explained proportion of the first eigenvalue of the standardized original data set, λ i,原始 is the i-th eigenvalue of the standardized original data set, is the sum of all eigenvalues of the normalized original dataset.
[0100] Among them, the calculation formula for the second variance explanation ratio is:
[0101]
[0102] Where p i,合成 is the variance explained proportion of the first eigenvalue of the standardized synthetic dataset, λ i,合成 is the i-th eigenvalue of the normalized synthetic dataset, is the sum of all eigenvalues of the normalized synthetic dataset.
[0103] S1332: Setting a first threshold and calculating a first cumulative variance explanation ratio of the standardized original data set according to the first threshold and the first variance explanation ratio.
[0104] In the embodiment of the present application, a threshold value (such as 85%, 90% or 95%) is set, and the number of principal components that reaches the threshold value is selected as r. The calculation formula for calculating the cumulative variance explained ratio of the standardized original data set is:
[0105]
[0106] Where λ j , 原始 is the eigenvalue of the jth principal component of the standardized original data set, C 原始 is the proportion of cumulative variance explained in the standardized original data set, is the sum of all eigenvalues of the normalized original dataset.
[0107] S1333: Set a second threshold and calculate a second cumulative variance explained ratio of the standardized synthetic data set based on the second threshold and the second variance explained ratio.
[0108] In the embodiment of the present application, a threshold value (such as 85%, 90% or 95%) is set, and the number of principal components that reaches the threshold value is selected as θ. The formula for calculating the cumulative variance explained ratio of the standardized synthetic data set is:
[0109]
[0110] Where λ j , 合成 is the eigenvalue of the jth principal component of the normalized synthetic dataset, C 合成 is the proportion of cumulative variance explained in the standardized synthetic data set, is the sum of all eigenvalues of the normalized synthetic dataset.
[0111] S1334: Determine a cumulative variance explanation index according to the ratio of the first cumulative variance explanation ratio to the second cumulative variance explanation ratio.
[0112] In the embodiment of the present application, the calculation formula of the cumulative variance explained index is:
[0113]
[0114] Where L is the cumulative variance explanation index, C 合成 is the cumulative variance explained ratio in the standardized synthetic data set, C 原始 is the proportion of cumulative variance explained in the standardized original data set, Indicates taking the absolute value.
[0115] Compare the size relationship between L and 0. If L is close to 0, it means that the synthetic data generated by the generative model based on the original dataset is more accurate and stable.
[0116] In some embodiments, step S130 of inputting the normalized data set into a pre-established analysis model to determine the Jason-Shannon divergence may include steps S134 to S136 .
[0117] In this embodiment, Jensen-Shannon divergence is used to measure the similarity of the probability distribution of the standardized original data set and the standardized synthetic data set as an authenticity evaluation indicator for the synthetic data.
[0118] S134: Calculate the probability distribution of the normalized original data set and the normalized synthetic data set.
[0119] In the embodiment of the present application, based on the above X norm ={x i ,.....,x inorm ,....,x mnorm} and Y norm ={y 1norm ,.....,y inorm ,....y nnorm}, calculate the Jensen-Shannon divergence.
[0120] S135: Calculate the average distribution based on the probability distribution.
[0121] In the embodiment of the present application, X is calculated respectively. norm and Y norm The probability distribution of .
[0122] For discrete data sets, the probability distribution P of the standardized original data set is calculated as:
[0123]
[0124] Where m is the number of samples in the normalized original dataset.
[0125] For discrete data sets, the probability distribution Q of the standardized synthetic data set is calculated as:
[0126]
[0127] Where n is the number of samples in the standardized synthetic dataset.
[0128] S136: Calculate the Jensen-Shannon divergence based on the probability distribution and the mean distribution.
[0129] In the embodiment of the present application, the KL divergence and Jensen-Shannon divergence are calculated as follows:
[0130]
[0131] Where D kL is the Kullback-Leibler divergence.
[0132] In this embodiment, the Jensen-Shannon divergence ranges from 0 to 1. A lower Jensen-Shannon divergence indicates that the synthesized dataset is more similar to the original dataset.
[0133] S140: Perform weighted averaging on the cumulative variance explanation index and the Jensen-Shannon divergence to construct an evaluation model, and evaluate the synthesis quality of the synthetic dataset based on the evaluation model.
[0134] In some embodiments, step S140 performs weighted averaging on the cumulative variance explained index and the Jensen-Shannon divergence to construct an evaluation model, including: S141 to S142.
[0135] S141: Set different weight coefficients for the cumulative variance explained index and the Jensen-Shannon divergence according to data evaluation requirements.
[0136] In an embodiment of the present application, the comprehensive evaluation can be performed on the synthetic data set by combining the variance explanation of PCA and the similarity metric of Jensen-Shannon divergence.
[0137] S142: The evaluation model is constructed by summing the weight coefficient with the product of the cumulative variance explained index and the Jensen-Shannon divergence respectively.
[0138] In the embodiment of the present application, the calculation formula of the evaluation model is:
[0139] Quality Assessment=ω1×L+ω2×JSD(P,Q);
[0140] Where ω1 and ω2 are weight coefficients, ω1+ω2=1.
[0141] The weight coefficient is used to adjust the PCA variance explanation and its importance in the comprehensive evaluation of Jensen-Shannon divergence. This application takes a weighted average of the cumulative variance explanation ratio and efficiency score.
[0142] In this embodiment, the choice of weighting coefficients ω1 and ω2 depends on the specific objectives and context of the assessment. For example, if the explanatory power of the data is of greater concern, the variance explained by PCA can be given a higher weight. If the authenticity is of greater concern, the Jensen-Shannon divergence can be given a higher weight. The final comprehensive quality assessment score can be used to compare the performance of different datasets or models. The lower the score, the better the dataset or model performs in terms of explanatory power and authenticity.
[0143] In some embodiments, step S140 , evaluating the synthesis quality of the synthesized data set according to the evaluation model, includes: S143 to S144 .
[0144] S143: Evaluate the cumulative variance explanation index and Jensen-Shannon divergence analysis results of the processed synthetic dataset according to the evaluation model.
[0145] S144: Providing application suggestions for improving the synthesis quality of the synthetic dataset based on the evaluation of the evaluation model, wherein the application suggestions include: model training, model alignment, and data interpolation.
[0146] In specific use, the model can be based on the input original data set and the synthetic data set generated by the model based on the original data set. Especially for generative models, the model can evaluate the effect of data synthesis based on the input, and provide suggestions and analysis on the generation defects of the production model.
[0147] In this application, the synthesis quality of the synthesized data is evaluated by comparing the efficiency scores and statistical tests of the original data and the synthesized data. If the efficiency score of the synthesized data is similar to that of the original data, it means that the synthesis quality of the synthesized data is high; if there is a significant difference, the synthesis quality may be low. Combining the dimensionality reduction ability of PCA and the distribution consistency ability of Jensen-Shannon divergence, a solution is provided for the synthesis quality and authenticity evaluation of synthetic data.
[0148] See also Figure 2 , Figure 2 This is a structural block diagram of a data evaluation device provided by the present application. The data evaluation device 300 includes: an acquisition module 310, a processing module 320, a determination module 330, and an evaluation module 340, wherein:
[0149] The acquisition module 310 is configured to acquire a data set, wherein the data set includes an original data set and a synthetic data set generated using a model based on the original data set.
[0150] The processing module 320 is configured to perform standardization processing on the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set.
[0151] The determination module 330 is used to input the standardized data set into a pre-established analysis model to determine the cumulative variance explained index and the Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set.
[0152] The evaluation module 340 is used to perform weighted averaging processing on the cumulative variance explanation index and the Jensen-Shannon divergence to construct an evaluation model, and evaluate the synthesis quality of the synthetic data set according to the evaluation model.
[0153] The device embodiment in this application may also include other modules, which specifically correspond to part of the content of the above method.
[0154] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the contents of the aforementioned method embodiments and will not be repeated here.
[0155] In several embodiments provided in this embodiment, the coupling between modules may be electrical, mechanical or other forms of coupling.
[0156] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.
[0157] See also Figure 3 , Figure 3 The present invention provides a structural block diagram of an electronic device 200 that can execute the above-mentioned data evaluation method, and the electronic device 200 can be a smart phone, tablet computer, computer or portable computer.
[0158] The electronic device 200 further includes a processor 202 and a memory 204 . The memory 204 stores a program that can execute the contents of the aforementioned embodiments, and the processor 202 can execute the program stored in the memory 204 .
[0159] The processor 202 may include one or more cores for processing data and a message matrix unit. The processor 202 utilizes various interfaces and circuits to connect various components within the electronic device 200. It executes instructions, programs, code sets, or instruction sets stored in the memory 204, and accesses data stored in the memory 204 to perform various functions and process data within the electronic device 200. Optionally, the processor 202 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 202 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem (decoder). The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem (decoder) may not be integrated into the processor and may be implemented separately via a communications chip.
[0160] The memory 204 may include a random access memory (RAM) or a read-only memory (ROM). The memory 204 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 204 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., instructions for a user to obtain a random number), instructions for implementing the various method embodiments described below, and the like. The data storage area may also store data (e.g., random numbers) created by the terminal during use.
[0161] The electronic device 200 may also include a network module and a screen. The network module is used to receive and transmit electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices, such as communicating with an audio playback device. The network module may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a user identity module (SIM) card, a memory, etc. The network module can communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other devices via a wireless network. The above-mentioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. The screen can display interface content and perform data interaction.
[0162] Please refer to Figure 4 , Figure 4 The computer-readable storage medium 400 stores program code 410, which can be called by a processor to execute the method described in the above method embodiment.
[0163] Computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Alternatively, the computer-readable storage medium includes a non-transitory computer-readable storage medium. Computer-readable storage medium 400 has storage space for program code 410 for executing any of the method steps in the above method. These program codes 410 can be read from or written to one or more computer program products. Program code 410 can be compressed, for example, in a suitable form.
[0164] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data evaluation method described in the various optional implementations described above.
[0165] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A data evaluation method, characterized in that: The method comprises: Acquire a data set, wherein the data set includes: an original data set and a synthetic data set generated by using a model based on the original data set; Performing standardization processing on the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set; Inputting the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index and a Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of the probability distributions of the standardized original data set and the standardized synthetic data set; The cumulative variance explanation index and the Jensen-Shannon divergence are weighted averaged to construct an evaluation model, and the synthesis quality of the synthetic data set is evaluated according to the evaluation model.
2. The data evaluation method according to claim 1, characterized in that: The step of inputting the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index comprises: Calculating the covariance matrix of the standardized original data set and the standardized synthetic data set; Performing eigendecomposition processing based on the covariance matrix to obtain corresponding eigendecomposition results, wherein the eigendecomposition results include: eigenvalues and eigenvectors; The cumulative variance explanation index is determined based on the eigendecomposition result.
3. The data evaluation method according to claim 2, characterized in that: The step of determining the cumulative variance explanation index based on the characteristic decomposition result comprises: Calculate a first variance explanation ratio of the standardized original data set and a second variance explanation ratio of the standardized synthetic data set based on the eigenvalues and the eigenvectors respectively; Setting a first threshold and calculating a first cumulative variance explanation ratio of the standardized original data set according to the first threshold and the first variance explanation ratio; Setting a second threshold and calculating a second cumulative variance explained ratio of the standardized synthetic data set according to the second threshold and the second variance explained ratio; The cumulative variance explained index is determined according to the ratio of the first cumulative variance explained ratio to the second cumulative variance explained ratio.
4. The data evaluation method according to claim 2, characterized in that: The step of inputting the standardized data set into a pre-established analysis model to determine the Jensen-Shannon divergence comprises: Calculating the probability distribution of the standardized original data set and the standardized synthetic data set; Calculating based on the probability distribution to obtain an average distribution; The Jensen-Shannon divergence is calculated based on the probability distribution and the mean distribution.
5. The data evaluation method according to claim 1, characterized in that: The weighted average processing of the cumulative variance explanation index and the Jensen-Shannon divergence to construct an evaluation model includes: Different weight coefficients are set for the cumulative variance explanation index and Jason-Shannon divergence according to data evaluation requirements; The evaluation model is constructed by summing the weight coefficient and the product of the cumulative variance explanation index and the Jensen-Shannon divergence respectively.
6. The data evaluation method according to claim 1, characterized in that: The step of normalizing the data set to obtain a standardized data set includes: The standardized data set is obtained by applying a minimum-maximum normalization process or a score normalization process to the data set.
7. The data evaluation method according to claim 1, characterized in that: The step of evaluating the synthesis quality of the synthetic data set according to the evaluation model comprises: Evaluate the analysis results of the cumulative variance explanation index and the Jensen-Shannon divergence corresponding to the processed synthetic data set according to the evaluation model; Based on the evaluation of the evaluation model, application suggestions are given to improve the synthesis quality of the synthetic data set, wherein the application suggestions include: model training, model alignment and data interpolation.
8. A data evaluation device, characterized in that: The device comprises: An acquisition module, used to acquire a data set, wherein the data set includes: an original data set and a synthetic data set generated by using a model based on the original data set; A processing module, configured to perform standardization processing on the data set to obtain a standardized data set, wherein the standardized data set includes: a standardized original data set and a standardized synthetic data set; A determination module, configured to input the standardized data set into a pre-established analysis model to determine a cumulative variance explanation index and a Jensen-Shannon divergence, wherein the Jensen-Shannon divergence is determined based on the similarity of probability distributions of the standardized original data set and the standardized synthetic data set; An evaluation module is used to perform weighted average processing on the cumulative variance explanation index and the Jensen-Shannon divergence to construct an evaluation model, and evaluate the synthesis quality of the synthetic data set according to the evaluation model.
9. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein the memory stores program codes that can be executed on the processor, and when the program codes are executed by the processor, the data evaluation method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by one or more processors to execute the data evaluation method according to any one of claims 1 to 7.