Object price monitoring method and system based on big data, computing device and storage medium

By collecting data from multiple sources, extracting key features and combining deep learning and time series analysis, a price prediction model is built, and the problem of inaccurate price prediction in the existing technology is solved, and more efficient price prediction and real-time feedback are achieved, improving Market insights and decision-making support.

CN118941313AInactive Publication Date: 2024-11-12BEIJING ALL VIEW CLOUD DATA TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410936242.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The price prediction results in the prior art are inaccurate and it is difficult to capture the complex and changing nature of the market. Especially when faced with sudden events, seasonal fluctuations or long-term trends, the prediction accuracy and real-timeness of traditional methods are insufficient.

Method used

By collecting historical price-related data of target objects from multiple sources, extracting key features, combining deep learning and time series analysis, building prediction models, predicting trend information of target prices, and feeding them back to users in real time so that relevant measures can be taken.

Benefits of technology

It significantly improves the accuracy and real-timeness of price prediction, provides more comprehensive market insights, helping users to grasp price trends in a timely manner, optimize decisions, and improve economic benefits and social value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941313A_ABST
    Figure CN118941313A_ABST
Patent Text Reader

Abstract

The invention provides an object price monitoring method and system based on big data, computing equipment and a storage medium. The method comprises the following steps: collecting data related to historical prices of a target object from a plurality of sources; extracting a plurality of key features from the data; predicting a target price according to the historical price of the target object and the plurality of key features; performing time sequence analysis on the historical price of the target object to predict trend information of the target price; and feeding back the target price and the trend information of the target price to a user as a prediction result, so that the user can take related measures based on the prediction result. According to the technical scheme provided by the invention, the precision and real-time performance of price prediction can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a method, system, computing device and storage medium for monitoring object prices based on big data. Background Art

[0002] With the advent of the digital age, big data has become a key factor in promoting the development of all industries, especially in the fields of finance, retail and real estate, where the prediction of price fluctuations has become increasingly important. Traditional price prediction methods often rely on a single data source and simple statistical models, which are difficult to capture the complex and changing nature of the market, especially in the face of sudden events, seasonal fluctuations or long-term trends. The prediction accuracy and real-time performance of traditional methods are often greatly reduced.

[0003] In recent years, with the development of big data technology, more and more research has begun to focus on how to use massive and diverse data resources to improve the accuracy of price forecasting. Big data not only includes traditional structured data, but also covers unstructured information such as social media, news reports, and macroeconomic indicators. These data contain rich signals of market sentiment, consumer behavior, and changes in the economic environment, which can provide a more comprehensive and in-depth perspective for price forecasting.

[0004] However, applying big data to price forecasting often results in inaccurate price forecasts due to insufficient data sources and improper data processing. Summary of the invention

[0005] The embodiments of the present application provide a method, system, computing device and storage medium for monitoring object prices based on big data, so as to solve the problem of inaccurate price prediction results in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for monitoring object prices based on big data, comprising:

[0007] Gather data related to the historical prices of the target object from multiple sources;

[0008] extracting a plurality of key features from the data;

[0009] predicting a target price based on the historical price of the target object and the multiple key features;

[0010] By performing time series analysis on the historical prices of the target object, trend information of the target price is predicted;

[0011] The target price and the trend information of the target price are fed back to the user as the prediction result, so that the user can take relevant measures based on the prediction result.

[0012] Optionally, predicting the target price according to the historical price of the target object and the key feature includes:

[0013] Determining the influence of the multiple key features on the historical price of the target object to obtain an adaptability weight coefficient corresponding to each key feature;

[0014] Constructing a key feature set based on each key feature and an adaptive weight coefficient corresponding to each key feature, so as to train a price prediction model through the key feature set;

[0015] The historical price of the target object and the key features are input into the price prediction model to predict the target price through the price prediction model.

[0016] Optionally, the performing time series analysis on the historical prices of the target object to predict the trend information of the target price includes:

[0017] Constructing a time series according to the historical prices, wherein the time series is arranged in chronological order;

[0018] Extracting time series characteristics of the time series, the time series characteristics at least including: whether the time series has periodic fluctuations, and whether the time series identity has an upward or downward trend;

[0019] According to the time series characteristics of the time series, the trend information of the target price is predicted.

[0020] Optionally, predicting the trend information of the target price according to the time series characteristics of the time series includes:

[0021] Selecting a matching trend prediction model according to the time series characteristics of the time series;

[0022] A training set and a test set are constructed according to the time series of the historical prices to train the trend prediction model, and the time series of the target price is input into the trend prediction model to predict the trend information of the target price through the trend prediction model.

[0023] Optionally, it also includes:

[0024] Detecting whether the target price fluctuates abnormally;

[0025] If so, an abnormality detection result is generated and fed back to the user.

[0026] Optionally, before extracting a plurality of key features from the data, the method further includes:

[0027] The data is preprocessed, and the preprocessing at least includes: removing duplicate data, filling missing values, detecting and correcting outliers, and data standardization.

[0028] Optionally, determining the influence of the multiple key features on the historical price of the target object to obtain an adaptability weight coefficient corresponding to each key feature includes:

[0029] By formula: Determine the adaptability weight coefficient corresponding to each key feature;

[0030] Among them, AWC i (t) is represented as the adaptive weight coefficient of the i-th key feature at time point t, and the weight coefficient is used to indicate the influence of the i-th key feature at time point t. α is represented as an adjustable parameter used to control the adjustment speed of the weight coefficient. i (tj) represents the value of the i-th key feature at time point (tj), Y(tj) represents the historical price at time point (tj), Corr(X i (tj), Y(tj)) represents the immediate correlation between the i-th key feature and the historical price Y at the time point (tj), j represents the lag parameter, which indicates the step back in the time series, n represents the data length of the key feature, m represents the total number of all key features, k represents the index variable used to traverse all key features, X k (tk) represents the value of the kth key feature at the time point (tk), and Y(tk) represents the historical price at the time point (tk).

[0031] In a second aspect, an embodiment of the present application provides an object price monitoring system based on big data, including:

[0032] A collection module for collecting data related to the historical prices of target objects from multiple sources;

[0033] An extraction module, used for extracting a plurality of key features from the data;

[0034] A prediction module, configured to predict a target price based on the historical price of the target object and the multiple key features; and predict the trend information of the target price by performing a time series analysis on the historical price of the target object;

[0035] The feedback module is used to feed back the target price and the trend information of the target price as the prediction result to the user, so that the user can take relevant measures based on the prediction result.

[0036] In a third aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a big data-based object price monitoring method as described in the first aspect above.

[0037] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements an object price monitoring method based on big data as described in the first aspect above.

[0038] In an embodiment of the present application, data related to the historical price of a target object are collected from multiple sources; multiple key features are extracted from the data; the target price is predicted based on the historical price of the target object and the multiple key features; time series analysis is performed on the historical price of the target object to predict trend information of the target price; the target price and the trend information of the target price are fed back to a user as a prediction result so that the user can take relevant measures based on the prediction result.

[0039] The beneficial effects of this method are reflected in the following aspects:

[0040] Multi-source data integration and key feature extraction: By collecting data related to the historical price of the target object from multiple sources, this method can cover a wider range of information sources and improve the comprehensiveness and reliability of the forecast. At the same time, the extraction of key features can effectively reduce the data dimension and focus on the factors that really affect the price change, thereby improving the efficiency and accuracy of the forecast model.

[0041] Combination of deep learning and time series analysis: This method uses deep learning technology to build a forecasting model, which can automatically learn complex patterns and nonlinear relationships in the data, improving the model's fitting and generalization capabilities. Combined with time series analysis, it can not only capture short-term price fluctuations, but also predict long-term trends, providing users with more comprehensive market insights.

[0042] Real-time feedback and decision support: By providing users with real-time feedback on forecast results, this method can help users keep abreast of price trends and take relevant measures based on the forecast results, such as adjusting inventory, optimizing pricing strategies or investment decisions, thereby gaining an advantage in a highly competitive market environment.

[0043] In summary, the method proposed in this application significantly improves the accuracy and real-time performance of price prediction by integrating multi-source big data, intelligent feature selection and advanced prediction models, provides users with powerful decision-making support, and has significant economic benefits and social value.

[0044] These and other aspects of the present application will become more apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of a method for monitoring object prices based on big data provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of an object price monitoring device based on big data provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0050] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0051] The inventors found that in recent years, with the development of big data technology, more and more studies have begun to focus on how to use massive and diverse data resources to improve the accuracy of price forecasting. Big data not only includes traditional structured data, but also covers unstructured information such as social media, news reports, and macroeconomic indicators. These data contain rich signals of market sentiment, consumer behavior, and changes in the economic environment, which can provide a more comprehensive and in-depth perspective for price forecasting.

[0052] However, it is not easy to apply big data to price prediction. First, it is a challenge to effectively screen out the key features that have a significant impact on prices from massive data. Second, due to the heterogeneity and high dimensionality of the data, it is also difficult to build a prediction model that can accurately reflect the dynamic changes in prices. Finally, short-term price fluctuations and long-term trends are often intertwined, and how to distinguish and predict these two types of trends is also a difficulty in research.

[0053] In view of this, the present application proposes an object price monitoring method based on big data, which aims to overcome the above challenges and achieve more accurate and real-time price prediction. The method specifically includes: collecting data related to the historical prices of the target object from multiple sources; extracting multiple key features from the data; predicting the target price based on the historical prices of the target object and the multiple key features; predicting the trend information of the target price by performing time series analysis on the historical prices of the target object; and feeding back the target price and the trend information of the target price as prediction results to the user, so that the user can take relevant measures based on the prediction results.

[0054] The technical solution of the embodiment of the present application collects data related to the historical price of the target object from multiple sources to cover a wider range of information sources, thereby improving the comprehensiveness and reliability of the prediction. At the same time, the extraction of key features can effectively reduce the data dimension and focus on the factors that really affect price changes, thereby improving the efficiency and accuracy of the prediction model. Combined with time series analysis, it can not only capture short-term price fluctuations, but also predict long-term trends, providing users with more comprehensive market insights. By feeding back the prediction results to users in real time, it can help users keep abreast of price trends.

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0056] Figure 1 A flowchart of a method for monitoring object prices based on big data is provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0057] 101. Collect data related to the historical prices of the target object from multiple sources;

[0058] In this step, the system collects historical price data about the target object (for example, a certain type of goods, assets or services) from multiple public online resources through automated means. These online resources (i.e. sources) may include preset databases, industry reports, public market data, news media, etc. The data related to historical prices can cover multiple dimensions such as time span, geographic location, and transaction volume.

[0059] In an embodiment of the present application, the system automatically captures market price data of specific crops (such as wheat) in a certain region over the past ten years from the public APIs of major weather stations around the world, and simultaneously collects environmental data such as rainfall, temperature, soil moisture, and export volume and policy change information of major producing countries in the world. Among them, environmental data, export volume, and policy change information are all data related to historical prices.

[0060] 102. Extracting a plurality of key features from the data;

[0061] In this step, the system uses feature engineering methods to identify and extract key features that have a significant impact on price changes from the collected raw data set. This includes using statistical and machine learning techniques to screen out the most predictive variables.

[0062] In the embodiment of the present application, the system uses principal component analysis (PCA) and mutual information method to select the top ten key features most closely related to wheat price fluctuations from environmental data and market information, such as rainfall, average temperature, global export volume index, etc.

[0063] 103. Predicting a target price based on the historical price of the target object and the multiple key features;

[0064] In this step, it involves building and training a price prediction model, which predicts the future price of the target object (i.e., the target price) based on historical price data and key features. Commonly used models include linear regression, support vector machine, random forest, deep learning network, etc.

[0065] In the embodiment of the present application, the system uses a gradient boosting tree model, inputs wheat price data and key features for the past five years, and trains the model to predict the average monthly price for the next six months. During the model training process, the system adjusts the hyperparameters through cross-validation to optimize the model's prediction performance.

[0066] 104. Predicting trend information of the target price by performing time series analysis on the historical price of the target object;

[0067] In this step, the system applies time series analysis techniques, such as autoregression (AR), moving average (MA), autoregressive integrated moving average (ARIMA), etc., to analyze the long-term trends and seasonal patterns of historical price data and predict future price trends.

[0068] In the embodiment of the present application, the system adopts the seasonal decomposition of time series analysis method (Seasonal Decomposition of Time Series, STL) to decompose the wheat price data of the past ten years, identify obvious seasonal fluctuations and long-term growth trends, and predict seasonal price fluctuations and overall trends in the next year.

[0069] 105. Feedback the target price and the trend information of the target price as a prediction result to the user, so that the user can take relevant measures based on the prediction result.

[0070] In this step, the system will feed back the predicted future price and trend information to the user in the form of charts, reports or notifications to help the user understand and use this information to make decisions, such as adjusting inventory, making purchasing plans, optimizing resource allocation, etc.

[0071] In the embodiment of the present application, the system sends a report containing wheat price forecasts and seasonal trend analysis for the next six months to agricultural cooperatives and food processing companies via email or a customized web interface. The report contains intuitive charts and text instructions to guide users to adjust inventory levels and plan procurement time according to the forecast results to reduce cost risks caused by market fluctuations.

[0072] Optionally, in the embodiment of the present application, step 103 may specifically include the following steps:

[0073] 1031. Determine the influence of the multiple key features on the historical price of the target object to obtain an adaptability weight coefficient corresponding to each key feature;

[0074] Alternatively, by the formula: Determine the adaptability weight coefficient corresponding to each key feature;

[0075] Among them, AWC i (t) is represented as the adaptive weight coefficient of the i-th key feature at time point t, and the weight coefficient is used to indicate the influence of the i-th key feature at time point t. α is represented as an adjustable parameter used to control the adjustment speed of the weight coefficient. i (tj) represents the value of the i-th key feature at time point (tj), Y(tj) represents the historical price at time point (tj), Corr(X i(tj), Y(tj)) represents the immediate correlation between the i-th key feature and the historical price Y at the time point (tj), j represents the lag parameter, which indicates the step back in the time series, n represents the data length of the key feature, m represents the total number of all key features, k represents the index variable used to traverse all key features, X k (tk) represents the value of the kth key feature at the time point (tk), and Y(tk) represents the historical price at the time point (tk).

[0076] In this step, the system quantifies the impact of each key feature on the price by calculating the correlation between each key feature and the historical price, and further obtains the adaptive weight coefficient. This coefficient is used to adjust the importance of the key feature in the prediction model so that the model can more accurately reflect the intensity of each factor in the real world.

[0077] In the embodiment of the present application, we assume that we have three key features: X1 (5-day moving average), X2 (10-day moving average), and X3 (market sentiment index). We consider the data of the past 30 days, that is, n=30, and set α=0.8. Assume that at time point t, the correlations between X1, X2, X3 and the historical price Y in the past 30 days are: 0.6, 0.4, and 0.3 respectively. The total correlation between all key features and historical prices is 1.3. Then AWC1(t), AWC2(t), and AWC3(t) are:

[0078] AWC1(t)=0.8×(0.6 / 1.3)=0.3846;

[0079] AWC2(t)=0.8×(0.4 / 1.3)=0.2308;

[0080] AWC3(t)=0.8×(0.3 / 1.3)=0.1846.

[0081] That is, the adaptability weight coefficients corresponding to the above three key features are obtained.

[0082] 1032. Construct a key feature set based on each key feature and an adaptive weight coefficient corresponding to each key feature, so as to train a price prediction model through the key feature set;

[0083] In this step, the system weights the key features according to the calculated adaptive weight coefficients to form a weighted key feature set. This feature set is then used to train the price prediction model to ensure that the model can better reflect the actual impact of key features on prices.

[0084] Continuing to use the example data from the previous step, we get the adaptive weight coefficient of each key feature. Next, we build a weighted key feature set for training the price prediction model. For example, we can use a linear regression model with weights, where the weights of X1, X2, and X3 are 0.3846, 0.2308, and 0.1846, respectively.

[0085] 1033. Input the historical price of the target object and the key features into the price prediction model to predict the target price through the price prediction model.

[0086] In this step, the system inputs the historical prices and the weighted key feature set into the trained price prediction model to predict the target price. This step realizes the final application of the model, converting historical data into predictions of future prices.

[0087] In the embodiment of the present application, using the trained model, we input the latest historical price and weighted key feature set and the corresponding adaptive weight coefficient to output the price forecast value for the next day through the price prediction model.

[0088] For example, suppose the price prediction model is a simple linear regression model of the following form:

[0089] Y^=b0+w1*X1+w2*X2+w3*X3;

[0090] Among them, Y^ represents the predicted price, b0 represents the intercept term of the model, w1, w2, w3 represent the weights of X1, X2, X3 of the weighted key features respectively, and X1, X2, X3 represent the values ​​of the weighted key features respectively.

[0091] Now we use the latest set of key feature values ​​for prediction, assuming X1 = 100, X2 = 95, X3 = 0.7 (market sentiment index, assuming its value range is between 0 and 1), and substitute this set of key feature values ​​into the price prediction model, and we get:

[0092] Y^=100+0.3846*100+0.2308*95+0.1846*0.7≈160.51522;

[0093] Therefore, the target price is 160.51522.

[0094] It should be noted that the above-mentioned price prediction model is a linear regression model which is only an example. The core of this application lies in how to determine the adaptive weight coefficient corresponding to each key feature to construct a key feature set and train an adaptation model based on the key feature set.

[0095] Optionally, in the embodiment of the present application, step 104 may specifically include the following steps:

[0096] 1041. Construct a time series according to the historical prices, wherein the time series is arranged in chronological order;

[0097] In this step, the system organizes the collected historical price data about the target object into a time series format. A time series is a collection of data arranged in chronological order, where each data point is associated with a timestamp. For price data, a time series usually contains a date or time point and the corresponding market price.

[0098] In this embodiment of the application, assume that we have daily closing price data of a company's stock over the past year, from January 1, 2022 to December 31, 2022. Each record contains a date and a price, for example:

[0099] 2022-01-01: $170.50

[0100] 2022-01-02: $171.25 ...

[0102] 2022-12-31: $165.75

[0103] We sort this data by date to form a time series where each element is a price value, arranged in ascending order of time.

[0104] 1042. Extracting time series characteristics of the time series, the time series characteristics at least including: whether the time series has periodic fluctuations, and whether the time series identity has an upward or downward trend;

[0105] This step involves analyzing the time series data to identify its inherent patterns and characteristics. The main time series characteristics include, but are not limited to, periodicity (such as seasonal fluctuations), trend (long-term rise or fall), and randomness. These characteristics are crucial in choosing an appropriate time series forecasting model.

[0106] Continuing with the time series of daily closing prices of a company's stock, we can analyze its characteristics by plotting the time series and calculating the autocorrelation function (ACF) and partial autocorrelation function (PACF). Suppose we find that there is a clear annual periodicity in the data (perhaps due to the impact of the holiday sales season) and a slight long-term upward trend (the company's performance continues to grow).

[0107] 1043. Predict trend information of the target price based on the time series characteristics of the time series.

[0108] In this step, after identifying the time series characteristics of the time series, this step involves selecting and training one or more forecasting models to predict the future trend of the target price. The choice of model should be based on the time series characteristics. For example, for time series with obvious periodicity, seasonal ARIMA model can be used; while for time series with trend, it may be necessary to use difference or trend decomposition methods.

[0109] Specifically, the specific execution process of "selecting a matching trend prediction model according to the time series characteristics of the time series" in step 1043 may include: constructing a training set and a test set according to the time series of the historical prices to train the trend prediction model, and inputting the time series of the target price into the trend prediction model to predict the trend information of the target price through the trend prediction model.

[0110] In the embodiment of the present application, it is assumed that our analysis shows that the stock price time series of a company has an annual periodicity and a slight upward trend. We decided to use the seasonal ARIMA model for prediction. First, we divide the data into a training set and a test set. For example, the first 80% of the data (i.e., from January 1, 2022 to September 30, 2022) is used as the training set, and the remaining 20% ​​of the data (i.e., from October 1, 2022 to December 31, 2022) is used as the test set.

[0111] Next, we use the training set data to train the seasonal ARIMA model. The specific parameters may be (p, d, q)(P, D, Q)_s, where p, d, q are the parameters of the non-seasonal part, P, D, Q are the parameters of the seasonal part, and s is the seasonal period (365 days in this case). Suppose we find the optimal parameter combination (1, 1, 1)(1, 0, 1)_365 through grid search and AIC criterion.

[0112] Finally, we input the test set data into the trained model to evaluate the model's predictive performance. Assuming that the model predicts that the stock price of a company on December 31, 2022 is 165.20, and the actual price is 165.20, and the actual price is 165.75, we can evaluate the accuracy of the model by calculating the prediction error (such as absolute error, mean square error, etc.).

[0113] After completing the model evaluation, we can retrain the model using the complete dataset (including training and test sets) to obtain the final prediction model for predicting future price trends. For example, we may predict that the average price of a company's stock in the first quarter of 2024 will be about 2% higher than that in the fourth quarter of 2023, taking into account the impact of annual cyclicality and long-term upward trends.

[0114] Optionally, in the embodiment of the present application, the method further includes:

[0115] Detecting whether the target price fluctuates abnormally;

[0116] If so, an abnormality detection result is generated and fed back to the user.

[0117] In this step, anomaly detection refers to the process of identifying observations in a data stream that significantly deviate from normal behavior or patterns. In the financial and economic fields, abnormal fluctuations may refer to a sudden and unusually large increase or decrease in prices, which may be caused by a variety of factors, such as market sentiment, policy changes, emergencies, etc. Detecting abnormal fluctuations is crucial for timely response to market changes, risk management, and decision making.

[0118] In the embodiment of the present application, consider a price time series of a commodity futures contract, and we want to detect abnormal fluctuations in it. To this end, the Z-score method in statistics can be used to quantify the degree to which the price deviates from its historical mean. Z-score is defined as: Z = σ (X - μ);

[0119] Among them, X is the current price, μ is the average of historical prices, and σ is the standard deviation of historical prices. If the absolute value of the Z-score is greater than a certain threshold (for example, 3), it is considered that the price fluctuates abnormally.

[0120] Suppose we have a set of daily closing price data for a commodity futures contract, the average daily closing price for the last 30 days is 500, the standard deviation is 500, and the standard deviation is 15. One day, the closing price of the contract suddenly jumped to $545. We calculate the Z-score as follows: Z = 15 (545-500) = 3;

[0121] Since the Z-score is equal to 3, which means that the price deviates from its historical mean by 3 standard deviations, we can conclude that there was an abnormal price fluctuation on this day.

[0122] Once an abnormal fluctuation is detected, the system will generate anomaly detection results, which usually include the time when the anomaly occurred, the size of the anomaly value, and possible causes. The system will then feed this information back to the user so that the user can quickly understand market dynamics and make corresponding adjustments or decisions.

[0123] Optionally, in an embodiment of the present application, the method further includes, before step 102: preprocessing the data, wherein the preprocessing includes at least: removing duplicate data, filling missing values, detecting and correcting outliers, and standardizing data.

[0124] In this step, duplicate data is removed: Duplicate records often appear in the data set, which may be caused by errors in the data collection process or the characteristics of the data source. Duplicate data will affect the accuracy of the analysis results, especially in statistical analysis and machine learning model training. Removing duplicate data can ensure that each sample is only calculated or learned once.

[0125] Missing value filling: Some feature values ​​in the data set may be missing for various reasons. The existence of missing values ​​will affect the integrity of data analysis and may even distort the analysis results. Common processing methods include filling with the mean, median, mode, or predicting missing values ​​based on other features.

[0126] Outlier detection and correction: Outliers are observations that are significantly different from the majority of the data. They may be caused by measurement errors, data entry errors, or other abnormal situations. Outliers may seriously distort the results of data analysis, so they need to be detected and appropriately handled, such as deletion, correction, or labeling.

[0127] Data standardization: Data standardization is a preprocessing technique used to convert features of different scales to the same scale, thereby eliminating the dimensionality effect between features and improving the effectiveness of subsequent analysis and model training. Common standardization methods include Min-Max Scaling and z-score standardization.

[0128] In the present application example, assume that we have a data set containing student test score data, where each record contains a student ID, math score, English score, and physics score. The following is a specific implementation example for the above data preprocessing steps:

[0129] Remove duplicate data: Check if there are identical records in the dataset, i.e., the student ID, math score, English score, and physics score are all the same. If duplicate records are found, keep one record and delete the rest.

[0130] Missing value filling: Check if there are any missing grades in the dataset. For missing math grades, fill them with the average score of all students in the subject. Example: If the average score of math grades is 75, fill all missing math grades with 75.

[0131] Outlier detection and correction: Use the IQR (interquartile range) method to detect outliers in physics scores. Calculate the first quartile (Q1) and the third quartile (Q3), and then calculate IQR = Q3-Q1. Determine the lower bound (Q1-1.5*IQR) and upper bound (Q3+1.5*IQR), and any value below the lower bound or above the upper bound is considered an outlier.

[0132] Example: If Q1 = 60, Q3 = 85, then IQR = 25, the lower bound is 32.5, and the upper bound is 117.5. If a physics score is 120, it is considered an outlier. For detected outliers, you can choose to delete or modify (such as replacing them with upper or lower bound values).

[0133] Data standardization: The math scores were standardized using the z-score standardization method. First, the mean μ and standard deviation σ of the math scores were calculated.

[0134] Example: If μ = 75, σ = 10, then z-score normalization is performed on each math score x, and the calculation formula is z = σx-μ. The normalized z-score is used as the new math score.

[0135] Through the above steps, the data set has been effectively preprocessed, which improves the data quality and the accuracy of subsequent analysis.

[0136] Figure 2 A schematic diagram of the structure of an object price monitoring device based on big data provided in an embodiment of the present application;

[0137] A collection module 21, for collecting data related to the historical price of the target object from multiple sources;

[0138] An extraction module 22, used to extract a plurality of key features from the data;

[0139] A prediction module 23, configured to predict a target price based on the historical price of the target object and the multiple key features; and predict the trend information of the target price by performing a time series analysis on the historical price of the target object;

[0140] The feedback module 24 is used to feed back the target price and the trend information of the target price as the prediction result to the user, so that the user can take relevant measures based on the prediction result.

[0141] Optionally, in an embodiment of the present application, the prediction module 23 is specifically used to determine the degree of influence of the multiple key features on the historical price of the target object to obtain an adaptive weight coefficient corresponding to each key feature; construct a key feature set based on each key feature and the adaptive weight coefficient corresponding to each key feature to train a price prediction model through the key feature set; input the historical price of the target object and the key features into the price prediction model to predict the target price through the price prediction model.

[0142] Optionally, in an embodiment of the present application, the prediction module 23 is specifically used to construct a time series based on the historical prices, and the time series is arranged in chronological order; extract the time series characteristics of the time series, and the time series characteristics at least include: whether the time series has periodic fluctuations, and whether the time series identity has an upward or downward trend; based on the time series characteristics of the time series, predict the trend information of the target price.

[0143] Optionally, in an embodiment of the present application, the prediction module 23 is also used to select a matching trend prediction model based on the time series characteristics of the time series; construct a training set and a test set based on the time series of the historical prices to train the trend prediction model, and input the time series of the target price into the trend prediction model to predict the trend information of the target price through the trend prediction model.

[0144] Optionally, in the embodiment of the present application, the device further includes: a detection module 25;

[0145] The detection module 25 is used to detect whether the target price has abnormal fluctuations;

[0146] If the detection module 25 detects that the target price has abnormal fluctuations, it generates an abnormal detection result and feeds back the abnormal detection result to the user through the feedback module 24 .

[0147] Optionally, in an embodiment of the present application, the extraction module 22 is also used to preprocess the data, and the preprocessing at least includes: removing duplicate data, filling missing values, detecting and correcting outliers, and data standardization.

[0148] Optionally, in the embodiment of the present application, the prediction module 23 is further configured to use the formula: Determine the adaptability weight coefficient corresponding to each key feature;

[0149] Among them, AWC i (t) is represented as the adaptive weight coefficient of the i-th key feature at time point t, and the weight coefficient is used to indicate the influence of the i-th key feature at time point t. α is represented as an adjustable parameter used to control the adjustment speed of the weight coefficient. i (tj) represents the value of the i-th key feature at time point (tj), Y(tj) represents the historical price at time point (tj), Corr(X i(tj), Y(tj)) represents the immediate correlation between the i-th key feature and the historical price Y at the time point (tj), j represents the lag parameter, which indicates the step back in the time series, n represents the data length of the key feature, m represents the total number of all key features, k represents the index variable used to traverse all key features, X k (tk) represents the value of the kth key feature at the time point (tk), and Y(tk) represents the historical price at the time point (tk).

[0150] Figure 2 The object price monitoring device based on big data can perform Figure 1 The implementation principle and technical effect of the object price monitoring method based on big data described in the illustrated embodiment will not be repeated. The specific manner in which each module and unit performs operations in the object price monitoring device based on big data in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0151] In one possible design, Figure 2 The object price monitoring device based on big data in the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0152] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0153] The processing component 32 is used to: collect data related to the historical price of the target object from multiple sources; extract multiple key features from the data; predict the target price based on the historical price of the target object and the multiple key features; predict the trend information of the target price by performing time series analysis on the historical price of the target object; and feed back the target price and the trend information of the target price as prediction results to the user so that the user can take relevant measures based on the prediction results.

[0154] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0155] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0156] Of course, a computing device may also include other components, such as input / output interfaces, communication components, etc.

[0157] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0158] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0159] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0160] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The object price monitoring method based on big data of the illustrated embodiment.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for monitoring object prices based on big data, characterized in that: include: Gather data related to the historical prices of the target object from multiple sources; extracting a plurality of key features from the data; predicting a target price based on the historical price of the target object and the multiple key features; By performing time series analysis on the historical prices of the target object, trend information of the target price is predicted; The target price and the trend information of the target price are fed back to the user as the prediction result, so that the user can take relevant measures based on the prediction result.

2. The method according to claim 1, characterized in that The predicting the target price according to the historical price of the target object and the key features includes: Determining the influence of the multiple key features on the historical price of the target object to obtain an adaptability weight coefficient corresponding to each key feature; Constructing a key feature set based on each key feature and an adaptive weight coefficient corresponding to each key feature, so as to train a price prediction model through the key feature set; The historical price of the target object and the key features are input into the price prediction model to predict the target price through the price prediction model.

3. The method according to claim 1, characterized in that The step of performing time series analysis on the historical prices of the target object to predict the trend information of the target price includes: Constructing a time series according to the historical prices, wherein the time series is arranged in chronological order; Extracting time series characteristics of the time series, the time series characteristics at least including: whether the time series has periodic fluctuations, and whether the time series identity has an upward or downward trend; According to the time series characteristics of the time series, the trend information of the target price is predicted.

4. The method according to claim 3, characterized in that The step of predicting the trend information of the target price according to the time series characteristics of the time series includes: Selecting a matching trend prediction model according to the time series characteristics of the time series; A training set and a test set are constructed according to the time series of the historical prices to train the trend prediction model, and the time series of the target price is input into the trend prediction model to predict the trend information of the target price through the trend prediction model.

5. The method according to claim 1, characterized in that Also includes: Detecting whether the target price fluctuates abnormally; If so, an abnormality detection result is generated and fed back to the user.

6. The method according to claim 1, characterized in that Before extracting a plurality of key features from the data, the method further includes: The data is preprocessed, and the preprocessing at least includes: removing duplicate data, filling missing values, detecting and correcting outliers, and data standardization.

7. The method according to claim 2, characterized in that: Determining the influence of the multiple key features on the historical price of the target object to obtain an adaptability weight coefficient corresponding to each key feature includes: By formula: Determine the adaptability weight coefficient corresponding to each key feature; Among them, AWC i (t) is represented as the adaptive weight coefficient of the i-th key feature at time point t, and the weight coefficient is used to indicate the influence of the i-th key feature at time point t. α is represented as an adjustable parameter used to control the adjustment speed of the weight coefficient. i (tj) represents the value of the i-th key feature at time point (tj), Y(tj) represents the historical price at time point (tj), Corr(X i (tj), Y(tj)) represents the immediate correlation between the i-th key feature and the historical price Y at the time point (tj), j represents the lag parameter, which indicates the step back in the time series, n represents the data length of the key feature, m represents the total number of all key features, k represents the index variable used to traverse all key features, X k (tk) represents the value of the kth key feature at the time point (tk), and Y(tk) represents the historical price at the time point (tk).

8. An object price monitoring system based on big data, characterized in that: include: A collection module for collecting data related to the historical prices of target objects from multiple sources; An extraction module, used for extracting a plurality of key features from the data; A prediction module, configured to predict a target price based on the historical price of the target object and the multiple key features; and predict the trend information of the target price by performing a time series analysis on the historical price of the target object; The feedback module is used to feed back the target price and the trend information of the target price as the prediction result to the user, so that the user can take relevant measures based on the prediction result.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an object price monitoring method based on big data as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer executes the computer program, the object price monitoring method based on big data as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Agricultural product price prediction method and system

    CN112053181A

  • Prediction method and device for digital currency price change trend

    CN112418833A

  • Investment-reinsurance decision-making method and system

    CN112925830A