Data valuation method, model and system based on two-factor fusion and dynamic weighting and storage medium
By generating adversarial networks to detect and repair abnormal data points, and combining cost and market dual-factor models to dynamically adjust weights, the shortcomings of traditional data valuation methods in complex real-time scenarios are solved, achieving efficient and accurate data valuation.
Patent Information
- Application Number
- CN202510925116.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-04
- Publication Date
- 2025-11-21
AI Technical Summary
Traditional data valuation methods struggle to handle non-Gaussian distributed data streams in complex real-time scenarios, and fixed-weight fusion valuation results are prone to deviating from reality. They are unable to effectively cope with high-dimensional, heterogeneous, and noisy data, and the changes in cost and market factor contribution caused by market fluctuations are not effectively captured.
We employ anomaly detection and repair based on generative adversarial networks, combined with a cost and market dual-factor model. We generate data valuation results through dynamic weighting, extract features using Transformer and graph neural networks, and dynamically adjust weights to cope with market fluctuations.
It significantly improves the robustness of data valuation in complex real-time scenarios, generates high-quality preprocessed data streams, and generates real-time data valuation results through dynamic weight fusion, thereby improving the accuracy and adaptability of the valuation.
Smart Images
Figure CN120996838A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of data processing and artificial intelligence technology, and in particular to a data valuation method, model, system and storage medium based on two-factor fusion and dynamic weighting. Background Technology
[0002] With the development of information technology, real-time data valuation is playing an increasingly important role in fields such as finance, supply chain management, and e-commerce. However, as data valuation is applied in complex real-time scenarios such as high-frequency trading and real-time risk control, traditional data valuation methods have several shortcomings when dealing with the large amounts of data generated in these complex real-time scenarios:
[0003] In complex real-time scenarios, non-Gaussian distributed data streams (such as high-dimensional, heterogeneous, and noisy data) are difficult to clean and detect anomalies during data preprocessing. Traditional time series analysis models with a single architecture perform poorly when dealing with high-dimensional data and complex market dynamics in complex real-time scenarios. In addition, occasional market fluctuations in complex real-time scenarios can change the contribution of costs and market factors, which can lead to traditional fixed-weight fusion valuation results that are prone to deviating from reality when performing data valuation.
[0004] In view of this, this application aims to propose a new data valuation method to overcome the problems caused by the above-mentioned defects. Summary of the Invention
[0005] The main purpose of this application is to provide a data valuation method based on two-factor fusion and dynamic weighting, which aims to solve the problem of how to value data streams in complex real-time scenarios.
[0006] To achieve the above objectives, this application provides a data estimation method based on two-factor fusion and dynamic weighting, the method comprising:
[0007] Identify the abnormal and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset.
[0008] The preprocessed dataset is input into a first model and a second model respectively, so as to calculate the cost estimate in the preprocessed dataset through the first model and predict the market estimate in the preprocessed dataset through the second model;
[0009] Obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value;
[0010] The data valuation result of the data stream to be valued is determined based on the current first weight, the current second weight, the cost valuation, and the market valuation.
[0011] Optionally, the steps of determining the abnormal data points and normal data points in the input data stream to be valued, repairing the abnormal data points, and performing noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset include:
[0012] The skewness, kurtosis, mean, and variance of the input data stream to be valued are extracted to form a feature vector;
[0013] The feature vector is input into the pre-trained generative adversarial model;
[0014] The discriminator of the generative adversarial model evaluates the anomaly score of each data point in the feature vector, and determines the target data point whose anomaly score is greater than a preset score threshold as an anomaly data point, otherwise it is determined as a normal data point; the generator of the generative adversarial model generates a synthetic data point corresponding to the anomaly data point, and uses the synthetic data point to replace the anomaly data point as the repaired anomaly data point.
[0015] The data stream output by the generative adversarial model is denoised by using a moving average filter, and the dimensions of the data points in the denoised data stream are scaled to the target range by normalization to obtain the preprocessed dataset.
[0016] Optionally, the first model includes the following steps in predicting cost estimates in the preprocessed dataset:
[0017] Extract the total production cost data from the preprocessed dataset, wherein the total production cost data includes data acquisition cost, data processing cost, data storage cost, and data maintenance cost;
[0018] Obtain the input expected profit margin, and determine the cost estimate based on the expected profit margin and the total production cost characteristics:
[0019]
[0020] In the formula, For cost estimation, The total production cost of the data. This represents the expected profit margin.
[0021] Optionally, the second model includes a Transformer module and a graph neural network module. In predicting market valuations in the preprocessed dataset, the second model includes the following steps:
[0022] The Transformer module extracts time-series data from the preprocessed dataset to obtain output time-series features. ;as well as,
[0023] The graph neural network module extracts graph structure data from the preprocessed dataset to obtain output graph structure features. ;
[0024] Temporal features output by Transformer Sum graph structural features Concatenation generates combined feature vectors :
[0025]
[0026] In the formula, R represents a real number. Representing time series features Dimensions Representing graph structural features The dimension;
[0027] The combined feature vector is processed through a fully connected layer. Mapped to market valuation :
[0028]
[0029] In the formula, For the weight vector, This is a bias term.
[0030] Optionally, the step of calculating the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value includes:
[0031] Determine whether the target market volatility value is greater than a preset volatility threshold;
[0032] If it is greater than, then the current first weight is greater than the current second weight;
[0033] Otherwise, the current first weight is less than the current second weight.
[0034] Optionally, the target market volatility value is calculated based on the historical average price standard deviation of the data stream to be valued in the market.
[0035] Optionally, the step of determining the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation includes:
[0036] Calculate the first product of the current first weight and the cost valuation, and the second product of the current second weight and the market valuation;
[0037] The sum of the first product value and the second product value is determined as the data estimation result.
[0038] Furthermore, to achieve the above objectives, this application also provides a data valuation model, which includes:
[0039] The streaming data collaborative preprocessing module is used to identify abnormal data points and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset.
[0040] The parallel two-factor valuation module is used to input the preprocessed dataset into a first model and a second model respectively, so as to calculate the cost valuation in the preprocessed dataset through the first model and predict the market valuation in the preprocessed dataset through the second model.
[0041] The dynamic weighting module is used to obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value.
[0042] The fusion valuation module is used to determine the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation.
[0043] In addition, to achieve the above objectives, this application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data estimation method based on two-factor fusion and dynamic weighting as described in any of the preceding claims.
[0044] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the data estimation method based on two-factor fusion and dynamic weighting as described in any of the preceding claims.
[0045] This application has at least the following beneficial effects:
[0046] High-quality preprocessed data streams are generated by filtering anomalies in streaming data. Then, a cost-market dual-factor joint model is used to generate valuation indicators in two dimensions. The fusion weights are dynamically adjusted according to market fluctuations. The valuation results of the two factor models are generated to produce real-time data valuation results, which significantly improves the valuation robustness in complex real-time scenarios. Attached Figure Description
[0047] Figure 1 This is a flowchart illustrating the first embodiment of the data valuation method based on two-factor fusion and dynamic weighting in this application;
[0048] Figure 2 This is a schematic diagram of the architecture of the data valuation model involved in the embodiments of this application;
[0049] Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.
[0050] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0051] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.
[0052] First Embodiment
[0053] Reference Figure 1 This embodiment provides a data estimation method based on two-factor fusion and dynamic weighting, the method including the following steps:
[0054] Step S10: Determine the abnormal data points and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset.
[0055] In this embodiment, the input data stream to be valued is first preprocessed. This preprocessing includes two parts: anomaly detection and repair, and data cleaning.
[0056] The data stream to be valued refers to the streaming data input by the user that needs to be valued. This data contains complex non-Gaussian distributed data and has the characteristic of single access, so it needs to be processed in real time or it will be lost.
[0057] In some alternative implementations, the distinction between outlier and normal data points is achieved using a generative adversarial model (GAN):
[0058] First, from the raw data stream Extract statistical features from each, where each It is A data set consists of dimensional data points, including skewness, kurtosis, mean, and variance. Skewness measures the asymmetry of the data distribution, with positive values indicating right skewness and negative values indicating left skewness; kurtosis reflects the sharpness of the data distribution; the mean measures abrupt changes in the distribution; and variance monitors the cumulative error in the data.
[0059] Next, skewness, kurtosis, mean, and variance are integrated into a feature vector F. A discriminator based on a generative adversarial model is then used to evaluate the anomaly score for each data point in this feature vector. In some alternative implementations, the anomaly scoring criteria for each indicator may refer to the following rules:
[0060] If the rate of change of skewness within a window exceeds a preset skewness threshold, the data points within that window are marked as having an abnormal probability.
[0061] If the kurtosis of N consecutive windows is greater than a preset kurtosis threshold, the data points within those N windows are marked as having an anomaly probability.
[0062] Generate a normal distribution reference interval for the mean μ and variance σ²: [μ-3σ, μ+3σ]. Data points exceeding this interval are marked as having an outlier probability.
[0063] The anomaly probability of each metric being labeled is multiplied by the preset weight corresponding to that metric to obtain the sub-anomaly score for that data point under that metric. The anomaly score for the data point is obtained by summing the sub-anomaly scores for each metric. The entire process is implemented by the discriminator of the generative adversarial model.
[0064] For example, the discriminator evaluates each data point Abnormal scores Set threshold If abnormal scores ,but It was identified as an anomalous data point.
[0065] In some alternative implementations, the repair of anomalous data points is achieved by a generator of a generative adversarial model. The generator generates synthetic data points corresponding to the anomalous data points and replaces the anomalous data points with the synthetic data points as the repaired anomalous data points.
[0066] For example, the generator draws from the noise distribution Sampling noise vector in (standard normal distribution) Generate synthetic data points that are similar to real data. , use replace The repaired data stream is obtained:
[0067]
[0068] In some alternative implementations, data cleaning is achieved through two operations: noise filtering and normalization. First, a moving average filter is used to clean the data stream output by the generative adversarial model. Noise reduction is performed, and then normalization is applied to the noise-reduced data stream. The dimensions of the data points are scaled to the target dimension range to obtain the preprocessed dataset. Optionally, the target dimension range can be... .
[0069] Step S20: Input the preprocessed dataset into the first model and the second model respectively, so as to calculate the cost estimate in the preprocessed dataset through the first model and predict the market estimate in the preprocessed dataset through the second model;
[0070] In this embodiment, after obtaining the preprocessed dataset through the aforementioned preprocessing, the preprocessed dataset is input into the first and second models obtained after training and debugging, and the two models perform cost and market valuation of the preprocessed dataset in parallel.
[0071] The first model refers to the mathematical model used to calculate cost factors; the second model refers to the deep model used to value market factors.
[0072] It should be noted that cost valuation reflects the estimated cost of the data stream to be valued, while market valuation reflects the estimated market value of the data stream. In this embodiment, the cost value and market value of the data stream are calculated / predicted in parallel through two channels, and the results are then merged to serve as the data valuation result.
[0073] In some alternative implementations, in the first model, a cost-based estimate is calculated considering data generation costs and expected profits. The calculation formula is as follows:
[0074]
[0075] in, For cost estimation, This represents the total production cost of the data. The expected profit margin reflects the profit return that a data provider hopes to obtain from a data transaction.
[0076] Total production cost of data It can be broken down into the following parts:
[0077]
[0078] in, For data acquisition costs (such as purchasing datasets, sensor acquisition fees, etc.). Data processing costs (such as computing resource costs and data engineer labor costs). Data storage costs (such as cloud storage fees). For data maintenance costs (such as data updates and quality assurance fees).
[0079] It is worth noting that V_cost here should be understood as "data valuation based on cost factors", that is, how much these data should be worth in terms of cost.
[0080] Secondly, C_prod represents the total production cost of the data, which is an objective fact. It consists of five parts: data acquisition cost C_acq, data processing cost C_proc, data storage cost C_store, and data maintenance cost C_maint.
[0081] C_prod = C_acq+C_proc+C_store+C_main
[0082] Then, P represents the expected profit margin, reflecting the profit return that the data provider hopes to obtain from the data transaction. For example, if the government wants to purchase some data from an institution, the institution, as the data provider, can independently determine the size of P, thereby deciding how much added value the data should add on top of its cost, that is, how much profit it should make from selling this data to the government.
[0083] Therefore, the final formula is:
[0084]
[0085] We obtained a cost-based valuation based on the data.
[0086] In some alternative implementations, the second model includes a Transformer module, a graph neural network module, and a feature fusion module.
[0087] The Transformer module is used to extract time series data from the preprocessed dataset to obtain the output time series features. ;
[0088] The graph neural network module is used to extract graph structure data from the preprocessed dataset to obtain the output graph structure features. ;
[0089] Temporal features output by Transformer Sum graph structural features Concatenation generates combined feature vectors :
[0090]
[0091] In the formula, R represents a real number. Representing time series features Dimensions Representing graph structural features The dimension;
[0092] The combined feature vector is processed through a fully connected layer. Mapped to market valuation :
[0093]
[0094] In the formula, For the weight vector, This is a bias term.
[0095] Furthermore, in some alternative implementations, in order to minimize the forecast market valuation Compared with actual market data The error between the two models is calculated using the mean squared error (MSE) as the loss function for the second model.
[0096]
[0097] in It is the number of training samples. and They are the first The predicted and actual values of each sample.
[0098] Step S30: Obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value;
[0099] Step S40: Determine the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation.
[0100] In this embodiment, the obtained cost valuation and market valuation are weighted and fused together, and the weights of the cost valuation and market valuation are dynamically adjusted according to market volatility to finally generate real-time data valuation results.
[0101] Market volatility, as a quantitative measure of market volatility, is calculated from the actual market price of the data stream to be valued. In some alternative implementations, the standard deviation of the data stream over the past 30 days in market prices is used as the target market volatility for that type of data stream.
[0102] In some alternative implementations, the target market volatility value is not directly used as a factor in calculating the first and second weights, but it will affect the relationship between the first and second weights:
[0103] If the target market volatility value is greater than the preset volatility threshold, it indicates that the market is unstable, and the current first weight is greater than the current second weight.
[0104] If the target market volatility value is less than or equal to the preset volatility threshold, it indicates that the market is stable, and the current first weight is less than the current second weight.
[0105] For example, a preset volatility threshold φ is set, and when the target market volatility φ > φ, the current first weight is determined. If the value is 0.7, then the current second weight is... It is 0.3; when When ≤ 𝜃, then determine the current first weight. It is 0.3, currently the second weight. It is 0.7.
[0106] Furthermore, in some optional implementations, the calculation of the data valuation result for the data stream to be valued specifically includes: calculating the first product of the current first weight and the cost valuation, and the second product of the current second weight and the market valuation; and determining the sum of the first product and the second product as the data valuation result.
[0107] For example, the calculation expression for the data estimation result can be:
[0108]
[0109] In the formula, For cost estimation, For market valuation, As the current top weight, It is currently the second weight.
[0110] In the technical solution provided in this embodiment, a high-quality preprocessed data stream is generated by filtering streaming data anomalies. Then, a cost-market dual-factor joint model is used to generate valuation indicators in two dimensions. The fusion weights are dynamically adjusted according to market fluctuations. The valuation of the two factor models is combined to generate real-time indicators, which significantly improves the valuation robustness in complex real-time scenarios.
[0111] Second Embodiment
[0112] Based on the first embodiment, this embodiment takes a real-time data valuation system of a specific data trading platform as an example for further explanation. The data stream input to this platform includes financial market data and Internet of Things sensor data, as detailed below:
[0113] Assume the input data is a stock price data stream from the financial market, flowing into the system once per second. The real-time price sequence of a particular stock is... (Unit: Yuan); Taking the most recent minute (60 data points) as an example, calculate the average. = 100.8, variance =0.25, skewness, kurtosis, etc., form the feature vector F.
[0114] For real-time data points Anomaly scores are calculated using a discriminator. Assuming =0.2, then = 0.8. Set threshold. =0.5, because Therefore, 105.0 is determined to be an outlier.
[0115] For the outlier 105.0, sample from the noise distribution. Use a generator to generate repair values Assuming =100.9, then replace 105.0 with 100.9 to obtain the repaired data stream. .
[0116] Next, the preprocessed data stream Smoothing and noise reduction are performed to obtain
[0117] After preprocessing, the data streams are merged. Input the subsequent parallel pricing model.
[0118] In the first model, it is assumed that the production cost of the stock price data includes: data acquisition costs. =1000 yuan, data processing cost =500 yuan, data storage cost =200 yuan, data maintenance cost =300 yuan, then the total production cost is... for:
[0119] Yuan
[0120] Set expected profit margin = 20%, reflecting the supplier's expected revenue, then the cost valuation is calculated. for:
[0121] Yuan
[0122] In the second model, the Transformer model is used to process the daily closing prices of the past 30 days as a stock price series. A self-attention mechanism is employed to capture the temporal dependencies of prices, outputting a temporal feature vector. The dimension is assumed to be 64.
[0123] Graph Neural Networks (GNNs) are used to process transaction network data. Assuming nodes represent stocks and edges represent transaction relationships, the GNN aggregates neighbor information and outputs a graph structure feature vector. The dimension is assumed to be 32.
[0124] Then feature fusion is performed to combine the temporal feature vectors. Graph structure eigenvectors Concatenate into a combined feature vector :
[0125]
[0126] in, The dimensions are 64 + 32 = 96.
[0127] Mapping to market valuation via linear layer Post-training optimization and Predicted = 2500 yuan.
[0128] Furthermore, assuming the target market volatility... =2.5, threshold =2.0. Because This indicates significant market volatility, therefore the current top weighting will be adjusted. =0.7, the current second weight is =0.3.
[0129] Finally, the data is aggregated and calculated to obtain a final valuation:
[0130] Yuan
[0131] In addition, as an implementation scheme, refer to Figure 2 This embodiment also proposes a data valuation model, which includes:
[0132] The data stream collaborative preprocessing module 100 is used to determine abnormal data points and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset.
[0133] The parallel two-factor valuation module 200 is used to input the preprocessed dataset into a first model and a second model respectively, so as to calculate the cost valuation in the preprocessed dataset through the first model and predict the market valuation in the preprocessed dataset through the second model.
[0134] The dynamic weighting module 300 is used to obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value.
[0135] The fusion valuation module 400 is used to determine the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation.
[0136] Furthermore, as an implementation scheme, Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.
[0137] like Figure 3 As shown, the computer system may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0138] Those skilled in the art will understand that Figure 3 The computer system architecture shown does not constitute a limitation on the computer system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0139] like Figure 3As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a program that manages and controls the hardware and software resources of the computer system, as well as the operation of the computer programs and other software or programs.
[0140] exist Figure 3 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used to communicate data with the backend server; and the processor 1001 can be used to call the computer program stored in the memory 1005.
[0141] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:
[0142] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0143] Identify the abnormal and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset.
[0144] The preprocessed dataset is input into a first model and a second model respectively, so as to calculate the cost estimate in the preprocessed dataset through the first model and predict the market estimate in the preprocessed dataset through the second model;
[0145] Obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value;
[0146] The data valuation result of the data stream to be valued is determined based on the current first weight, the current second weight, the cost valuation, and the market valuation.
[0147] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0148] The skewness, kurtosis, mean, and variance of the input data stream to be valued are extracted to form a feature vector;
[0149] The feature vector is input into the pre-trained generative adversarial model;
[0150] The discriminator of the generative adversarial model evaluates the anomaly score of each data point in the feature vector, and determines the target data point whose anomaly score is greater than a preset score threshold as an anomaly data point, otherwise it is determined as a normal data point; the generator of the generative adversarial model generates a synthetic data point corresponding to the anomaly data point, and uses the synthetic data point to replace the anomaly data point as the repaired anomaly data point.
[0151] The data stream output by the generative adversarial model is denoised by using a moving average filter, and the dimensions of the data points in the denoised data stream are scaled to the target range by normalization to obtain the preprocessed dataset.
[0152] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0153] Extract the total production cost data from the preprocessed dataset, wherein the total production cost data includes data acquisition cost, data processing cost, data storage cost, and data maintenance cost;
[0154] Obtain the input expected profit margin, and determine the cost estimate based on the expected profit margin and the total production cost characteristics:
[0155] In the formula, represents the cost estimate, represents the total production cost of the data, and represents the expected profit margin.
[0156] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0157] The Transformer module extracts time-series data from the preprocessed dataset to obtain output time-series features. ;as well as,
[0158] The graph neural network module extracts graph structure data from the preprocessed dataset to obtain output graph structure features. ;
[0159] Temporal features output by Transformer Sum graph structural features Concatenation generates combined feature vectors :
[0160]
[0161] In the formula, R represents a real number. Representing time series features Dimensions Representing graph structural features The dimension;
[0162] The combined feature vector is processed through a fully connected layer. Mapped to market valuation :
[0163]
[0164] In the formula, For the weight vector, This is a bias term.
[0165] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0166] Determine whether the target market volatility value is greater than a preset volatility threshold;
[0167] If it is greater than, then the current first weight is greater than the current second weight;
[0168] Otherwise, the current first weight is less than the current second weight.
[0169] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0170] Calculate the first product of the current first weight and the cost valuation, and the second product of the current second weight and the market valuation;
[0171] The sum of the first product value and the second product value is determined as the data estimation result.
[0172] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.
[0173] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the data estimation method based on two-factor fusion and dynamic weighting as described in the above embodiments.
[0174] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.
[0175] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.
[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0180] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. This application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0181] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0182] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A data valuation method based on two-factor fusion and dynamic weighting, characterized in that, The method includes the following steps: Identify the abnormal and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset. The preprocessed dataset is input into a first model and a second model respectively, so as to calculate the cost estimate in the preprocessed dataset through the first model and predict the market estimate in the preprocessed dataset through the second model; Obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value; The data valuation result of the data stream to be valued is determined based on the current first weight, the current second weight, the cost valuation, and the market valuation.
2. The method as described in claim 1, characterized in that, The steps of determining the abnormal and normal data points in the input data stream to be valued, repairing the abnormal data points, and performing noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain the preprocessed dataset include: The skewness, kurtosis, mean, and variance of the input data stream to be valued are extracted to form a feature vector; The feature vector is input into the pre-trained generative adversarial model; The discriminator of the generative adversarial model evaluates the anomaly score of each data point in the feature vector, and determines the target data point whose anomaly score is greater than a preset score threshold as an anomaly data point, otherwise it is determined as a normal data point; the generator of the generative adversarial model generates a synthetic data point corresponding to the anomaly data point, and uses the synthetic data point to replace the anomaly data point as the repaired anomaly data point. The data stream output by the generative adversarial model is denoised by using a moving average filter, and the dimensions of the data points in the denoised data stream are scaled to the target range by normalization to obtain the preprocessed dataset.
3. The method as described in claim 1, characterized in that, The first model includes the following steps in calculating the cost estimate in the preprocessed dataset: Extract the total production cost data from the preprocessed dataset, wherein the total production cost data includes data acquisition cost, data processing cost, data storage cost, and data maintenance cost; Obtain the input expected profit margin, and determine the cost estimate based on the expected profit margin and the total production cost characteristics: ； In the formula, For cost estimation, The total production cost of the data. This represents the expected profit margin.
4. The method as described in claim 1, characterized in that, The second model includes a Transformer module and a graph neural network module. In predicting market valuations in the preprocessed dataset, the second model includes the following steps: The Transformer module extracts time-series data from the preprocessed dataset to obtain output time-series features. ;as well as, The graph neural network module extracts graph structure data from the preprocessed dataset to obtain output graph structure features. ; Temporal features output by Transformer Sum graph structural features Concatenation generates combined feature vectors : ； In the formula, R represents a real number. Representing time series features Dimensions Representing graph structural features The dimension; The combined feature vector is processed through a fully connected layer. Mapped to market valuation : ； In the formula, For the weight vector, This is a bias term.
5. The method as described in claim 1, characterized in that, The step of calculating the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility includes: Determine whether the target market volatility value is greater than a preset volatility threshold; If it is greater than, then the current first weight is greater than the current second weight; Otherwise, the current first weight is less than the current second weight.
6. The method as described in claim 1 or 5, characterized in that, The target market volatility value is calculated based on the historical average price standard deviation of the data stream to be valued in the market.
7. The method as described in claim 1, characterized in that, The step of determining the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation includes: Calculate the first product of the current first weight and the cost valuation, and the second product of the current second weight and the market valuation; The sum of the first product value and the second product value is determined as the data estimation result.
8. A data valuation model, characterized in that, The data valuation model includes: The streaming data collaborative preprocessing module is used to identify abnormal data points and normal data points in the input data stream to be valued, repair the abnormal data points, and perform noise filtering and standardization on the data stream composed of the repaired abnormal data points and the normal data points to obtain a preprocessed dataset. The parallel two-factor valuation module is used to input the preprocessed dataset into a first model and a second model respectively, so as to calculate the cost valuation in the preprocessed dataset through the first model and predict the market valuation in the preprocessed dataset through the second model. The dynamic weighting module is used to obtain the target market volatility value corresponding to the data stream to be valued, and calculate the current first weight associated with the cost valuation and the current second weight associated with the market valuation based on the target market volatility value. The fusion valuation module is used to determine the data valuation result of the data stream to be valued based on the current first weight, the current second weight, the cost valuation, and the market valuation.
9. A computer system, characterized in that, The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the data estimation method based on two-factor fusion and dynamic weighting as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data estimation method based on two-factor fusion and dynamic weighting as described in any one of claims 1 to 7.