Product price index analysis method, system and device based on data processing

Abstract

The invention discloses a product price index analysis method, system and device based on data processing, and the method comprises the steps: obtaining and preprocessing a product related data set, and obtaining a preprocessed product data set; extracting comprehensive features of the preprocessed product data to obtain a product comprehensive feature set; the index weight analysis pre-training model analyzes and predicts the comprehensive characteristics of the product to obtain a price index weight; performing analysis based on the price index weight and the product price data to obtain product average price data; according to the time period and the corresponding product quantity data, obtaining initial price index data through a chain type pull formula; and analyzing the historical product price data and the historical fluctuation price factor based on the price fluctuation pre-training model to obtain a fluctuation price factor, and correcting the initial price index data to obtain price index data. According to the invention, through multi-dimensional data processing and multi-level feature extraction, the prediction precision of the product price index is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a product price index analysis method, system and device based on data processing. Background Art

[0002] The product price index is an economic indicator that reflects the changes in the price level of products or services in different periods, and is used to measure the degree and trend of price changes. At present, most of the calculation methods of the product price index rely on weighting the average price, and use the weighted result as a price index to express the fluctuation of the price index. Such methods are essentially still an expression of the trend of the average price of a product, not a price index of the product. In addition, the current product price index is only applicable to the city level, and is jointly formulated by one or more companies through analysis of limited data. The product price index has the problem of insufficient data and is incomplete. Therefore, even if the current product price index can reflect the trend of product price changes to a certain extent, it has the problems of low prediction accuracy and slow response to emergencies, and cannot effectively process large amounts of data.

[0003] At the same time, with the rapid changes in market data, the existing product price index cannot accurately reflect the real-time price dynamics of the product. Therefore, a more comprehensive, accurate and real-time price index calculation method is needed to ensure calculation efficiency while improving the prediction accuracy of the product price index and adapt to the rapidly changing product market environment. Summary of the invention

[0004] The present invention aims at the shortcomings of the prior art and provides a product price index analysis method, system and device based on data processing.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions: A product price index analysis method based on data processing comprises the following steps: Acquire and preprocess product-related data sets to obtain preprocessed product data sets, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; Extract comprehensive features from pre-processed product data to form a comprehensive product feature set, where comprehensive product features include production and sales features, supply and demand features, regional features, fluctuation features, and trend features; Establish an index weight analysis pre-training model. The index weight analysis pre-training model uses a price learner to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, where the price index weight includes the regional index weight, category index weight and grade index weight; The product price data is used to obtain the category price data and grade price data. The category index weight and grade index weight are combined to obtain the price data of different regions. The product average price data is obtained by combining the price data of different regions and the corresponding regional index weights. Preset the time period and obtain the corresponding product quantity data, combine it with the product average price data, and analyze it through the chain-pull formula to obtain the initial price index data; Construct a volatility price pre-training model, analyze historical product price data and historical volatility price factors, obtain the product price relationship equation, and then obtain the volatility price factor. Based on the volatility price factor, correct the initial price index data to obtain the price index data.

[0006] As an implementable method, obtaining a product-related data set and preprocessing it to obtain a preprocessed product data set includes the following steps: Clean and denoise product-related data to obtain initial preprocessed data; Perform missing value analysis and outlier analysis on the initial preprocessed data to obtain missing data and outlier data; Preset data thresholds, take missing data and abnormal data as the center, and obtain product-related data within the data threshold range to obtain the first adjacent related data; Preset a time threshold, and obtain product-related data corresponding to missing data and abnormal data within the time threshold, i.e., second adjacent related data; Based on the first adjacent related data and the second adjacent related data, the missing data and the abnormal data are supplemented by bilinear difference to correct the initial preprocessed data, thereby obtaining the preprocessed product data.

[0007] As an implementable method, the price learner is used to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, which includes the following steps: By analyzing the comprehensive characteristics of the product, the corresponding marginal contribution data is obtained, which is expressed as follows:

[0008] Based on the marginal contribution data corresponding to the comprehensive characteristics of the product, the learner weight of the price learner is obtained. Based on the learner weight and the price learner, the comprehensive characteristics of the product are analyzed to obtain the initial price data, which is expressed as follows:

[0009] Based on the initial price data and the real product price data, a price loss function is constructed, which is expressed as follows:

[0010] Based on the price loss function, the initial price data is subjected to gradient analysis, and a price learner is established based on the gradient analysis results. The gradient analysis results and the price learner are expressed as follows:

[0011]

[0012] Through iterative analysis of the price learner and the price loss function, an index weight analysis model is obtained. Based on the index weight analysis model, the comprehensive characteristics of the product to be analyzed are inferred to obtain the price index weight. in, Represents feature prediction contribution data, Indicates that it does not contain A subset of the comprehensive characteristics of a product, represents the size of the feature subset, represents the comprehensive feature set of the product, Indicates that it contains The model prediction value of the feature subset of the comprehensive features of the product, represents the factorial of the total number of comprehensive characteristics of the product, represents the initial price data, represents the number of price learners, represents the learner weight, Indicates A price learner, represents the price loss function, Represents the real product price data, Represents the results of gradient analysis.

[0013] As an implementable method, the method of obtaining category price data and grade price data through product price data, combining category index weight and grade index weight to obtain price data of different regions, combining price data of different regions and corresponding regional index weights to obtain product average price data includes the following steps: By analyzing the category price data, category index weight, grade price data and grade index weight, we can get the price data of different regions, which are shown as follows:

[0014] By analyzing the price data of different regions and the corresponding regional index weights, the average price data of the products is obtained, which is expressed as follows:

[0015] in, Indicates price data for different regions. represents the category index weight, Represents category price data, represents the grade index weight, Indicates grade price data, Indicates the average price of the product. represents the regional index weight, Represents price data of different regions.

[0016] As an implementation method, the preset time period and obtaining the corresponding product quantity data, combined with the product average price data, are analyzed by a chain pull formula to obtain the initial price index data, including the following steps: Get product quantity data corresponding to different time periods; The initial price index data is obtained by analyzing the product quantity data and the product average price data using the chain pull formula, where the chain pull formula is expressed as follows:

[0017] in, Indicates The initial price index data corresponding to the time period, Indicates The initial price index data corresponding to the time period, , Indicates , The average product price data corresponding to the time period, Indicates The product quantity data corresponding to each time period, Indicates the number of time periods.

[0018] As an implementable method, the fluctuation price factor is obtained by the following steps: The historical product price data and historical price fluctuation factors are analyzed through the price fluctuation pre-training model to obtain the product price relationship equation, which is expressed as follows:

[0019] The product error function is constructed based on the product price relationship equation, and the product price relationship equation is solved based on the least squares method to obtain the price regression factor. The specific solution process is as follows:

[0020]

[0021] Based on the price regression factor and product price data, the initial volatility price factor is obtained; Construct a volatility loss function for the volatility price pre-training model, train the volatility price pre-training model, obtain the volatility price model, perform inference analysis based on product price data, and obtain the volatility price factor, where the volatility loss function is expressed as follows:

[0022] in, represents the volatility loss function, represents the price loss function, represents the regularization coefficient, represents the initial volatility price factor, represents the number of volatility price factors, represents the historical volatility price factor, represents the intercept term, , represents the price regression factor, , Represents historical product price data, represents the error term, represents the product error function, Indicates Historical volatility price factors, Indicates The predicted value of the historical volatility price factor, Represents the number of historical volatility price factors.

[0023] As an implementable method, it also includes optimizing the efficiency of the index weight analysis pre-training model and the volatility price pre-training model through data partitioning and parallel computing, wherein the model processing time after efficiency optimization is expressed as follows:

[0024] in, represents the model processing time after efficiency optimization, represents the processing time before efficiency optimization, Indicates the number of cores or parallelism before efficiency optimization. Indicates the number of cores or degree of parallelism after efficiency optimization.

[0025] A product price index analysis system based on data processing, comprising a data preprocessing module, a comprehensive feature extraction module, an index weight calculation module, a product average price calculation module, a product price analysis module and a price index correction module; The data preprocessing module acquires and preprocesses a product-related data set to obtain a preprocessed product data set, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; The comprehensive feature extraction module extracts comprehensive features from the pre-processed product data to form a product comprehensive feature set, wherein the product comprehensive features include production and sales features, supply and demand features, regional features, fluctuation features and trend features; The index weight calculation module establishes an index weight analysis pre-training model. The index weight analysis pre-training model analyzes and predicts the comprehensive characteristics of the product through a price learner to obtain a price index weight, wherein the price index weight includes a regional index weight, a category index weight, and a grade index weight; The product average price calculation module obtains category price data and grade price data through product price data, obtains price data of different regions by combining category index weight and grade index weight, and obtains product average price data by combining price data of different regions and corresponding regional index weight; The product price analysis module presets a time period and obtains the corresponding product quantity data, combines the product average price data, and analyzes through a chain-pull formula to obtain initial price index data; The price index correction module constructs a fluctuation price pre-training model, analyzes historical product price data and historical fluctuation price factors, obtains a product price relationship equation, and then obtains a fluctuation price factor, and corrects the initial price index data based on the fluctuation price factor to obtain price index data.

[0026] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following method is implemented: Acquire and preprocess product-related data sets to obtain preprocessed product data sets, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; Extract comprehensive features from pre-processed product data to form a comprehensive product feature set, where comprehensive product features include production and sales features, supply and demand features, regional features, fluctuation features, and trend features; Establish an index weight analysis pre-training model. The index weight analysis pre-training model uses a price learner to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, where the price index weight includes the regional index weight, category index weight and grade index weight; The product price data is used to obtain the category price data and grade price data. The category index weight and grade index weight are combined to obtain the price data of different regions. The product average price data is obtained by combining the price data of different regions and the corresponding regional index weights. Preset the time period and obtain the corresponding product quantity data, combine it with the product average price data, and analyze it through the chain-pull formula to obtain the initial price index data; Construct a volatility price pre-training model, analyze historical product price data and historical volatility price factors, obtain the product price relationship equation, and then obtain the volatility price factor. Based on the volatility price factor, correct the initial price index data to obtain the price index data.

[0027] A product price index analysis device based on data processing includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following method when executing the computer program: Acquire and preprocess product-related data sets to obtain preprocessed product data sets, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; Extract comprehensive features from pre-processed product data to form a comprehensive product feature set, where comprehensive product features include production and sales features, supply and demand features, regional features, fluctuation features, and trend features; Establish an index weight analysis pre-training model. The index weight analysis pre-training model uses a price learner to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, where the price index weight includes the regional index weight, category index weight and grade index weight; The product price data is used to obtain the category price data and grade price data. The category index weight and grade index weight are combined to obtain the price data of different regions. The product average price data is obtained by combining the price data of different regions and the corresponding regional index weights. Preset the time period and obtain the corresponding product quantity data, combine it with the product average price data, and analyze it through the chain-pull formula to obtain the initial price index data; Construct a volatility price pre-training model, analyze historical product price data and historical volatility price factors, obtain the product price relationship equation, and then obtain the volatility price factor. Based on the volatility price factor, correct the initial price index data to obtain the price index data.

[0028] The present invention has significant technical effects due to the adoption of the above technical solution: The present invention collects product-related data sets and performs preprocessing, and then obtains a product comprehensive feature set through comprehensive feature extraction, analyzes and predicts the product comprehensive feature set to obtain a price index weight, and obtains product average price data through calculation, constructs a fluctuating price pre-training model and obtains a fluctuating price factor through analysis, and then obtains price index data through correction. The method of the present invention analyzes and extracts features of multi-dimensional product data, effectively improves the prediction accuracy of the price index, improves the calculation efficiency, reduces the cumulative error in the price index data calculation process, and provides a more timely and accurate market trend forecast. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0030] Figure 1 It is a schematic flow diagram of the method of the present invention; Figure 2 It is an overall schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below in conjunction with embodiments. The following embodiments are for explanation of the present invention but the present invention is not limited to the following embodiments.

[0032] Embodiment 1: A product price index analysis method based on data processing, such as Figure 1 As shown, the following steps are included: S100, obtaining a product-related data set and preprocessing it to obtain a preprocessed product data set, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; S200, extracting comprehensive features from the pre-processed product data to form a product comprehensive feature set, wherein the product comprehensive features include production and marketing features, supply and demand features, regional features, fluctuation features, and trend features; S300, establishing an index weight analysis pre-training model, wherein the index weight analysis pre-training model analyzes and predicts the comprehensive characteristics of the product through a price learner to obtain a price index weight, wherein the price index weight includes a regional index weight, a category index weight, and a grade index weight; S400, obtaining category price data and grade price data through product price data, combining category index weight and grade index weight to obtain price data of different regions, combining price data of different regions and corresponding regional index weights to obtain product average price data; S500, presetting a time period and obtaining corresponding product quantity data, combining the product average price data, analyzing through a chain-pull formula, and obtaining initial price index data; S600, construct a price fluctuation pre-training model, analyze historical product price data and historical price fluctuation factors, obtain a product price relationship equation, and then obtain a price fluctuation factor, correct the initial price index data based on the price fluctuation factor, and obtain price index data.

[0033] The present invention obtains a product comprehensive feature set by preprocessing and extracting comprehensive features of product-related data, and analyzes and predicts the product comprehensive feature set to obtain a price index weight, analyzes the price index weight and product price data, obtains product average price data, and then obtains initial price index data through a chain pull formula, and corrects the initial price index data through a fluctuating price factor to obtain price index data. The method of the present invention performs multidimensional analysis on product-related data, effectively improves the prediction accuracy of price index data, and improves the calculation efficiency of price index data through parallel processing. It solves the problem that the price index data currently obtained by one or more joint companies based on limited data is incomplete and insufficient in consideration of factors, avoids only reflecting the situation in a specific region, improves the response speed to emergencies and processes a large amount of data in a timely and effective manner, meets the rapid changes in large-scale market data, accurately reflects real-time price dynamics, and adapts to the dynamically changing market environment.

[0034] In this embodiment, glass products are taken as an example. Ways to obtain product-related data include: obtaining glass sheet transaction order data from all over the country from the glass sheet transaction platform, obtaining glass downstream product demand data, production capacity data and sensor data from the proprietary glass production management platform, and obtaining glass sheet production and inventory data from all over the country from the National Bureau of Statistics. Data analysis is performed on historical prices, market demand, transaction order data and other information of glass sheet transaction data to obtain product-related data sets. In this embodiment, product-related data include product price data, market demand data, product inventory data and product production data.

[0035] Preprocess the product-related data set to ensure the quality and availability of the product-related data. In this embodiment, the preprocessing first cleans and denoises the product-related data to obtain initial preprocessed data, and then fills the missing values ​​of the initial preprocessed data, including the following steps: Step 1: Traverse the initial preprocessed data and perform missing value analysis and outlier analysis to obtain missing data and outlier data; Step 2: In this embodiment, based on the fact that the price of glass fluctuates less within a week, the preset data threshold is 3, that is, taking the missing data and abnormal data as the center, the product-related data of the previous and next 3 days are selected to obtain the first adjacent related data; Step 3: In this embodiment, since the time interval between years is relatively far, selecting one year can take into account the impact of seasonality on glass prices, and there are fewer irrelevant factors. The preset time threshold is 1, that is, the product-related data of the same day in the previous and next years are selected to obtain the second adjacent related data; Step 4: In this embodiment, the idea of ​​bilinear difference is adopted, and based on the first adjacent related data and the second adjacent related data, a 9-square grid is constructed with the missing data and the abnormal data as the center, and the missing data and the abnormal data are supplemented and corrected to obtain a preprocessed product data set.

[0036] Comprehensive feature extraction is performed on the pre-processed product data. In this embodiment, comprehensive feature extraction is performed based on product output data and product inventory data to obtain production and marketing features; comprehensive feature extraction is performed based on product output data and market demand data to obtain supply and demand features; market demand data from all over the country are analyzed to obtain differences in product demand regions to obtain regional features; fluctuations in historical product price data are comprehensively analyzed to obtain fluctuation features; and comprehensive feature analysis of market production and marketing trends across the country based on product output data and market demand data is performed to obtain trend features. A comprehensive feature set of products is formed through production and marketing features, supply and demand features, regional features, fluctuation features, and trend features.

[0037] An index weight analysis pre-training model is established. In this embodiment, the index weight analysis pre-training model is a gradient boosting decision tree algorithm. The comprehensive characteristics of the product are analyzed and predicted by the index weight analysis pre-training model to obtain the influence weights of glass products in different regions, different categories, and different grades on the price, that is, the regional index weight, category index weight, and grade index weight. Specifically, the following steps are included: Step 1: Perform SHAP value analysis on the comprehensive characteristics of the products to allocate the contribution of each comprehensive characteristic of the products to the prediction results of the index weight analysis pre-training model, and obtain the marginal contribution data of the corresponding prediction results, which is expressed as follows:

[0038] Step 2: Based on the marginal contribution data of the corresponding prediction results, the learning weight corresponding to the price learner in the exponential weight analysis pre-training model is obtained, and the comprehensive characteristics of the product are analyzed through the learner weight and the price learner to obtain the initial price data, which is expressed as follows:

[0039] Step 3: Based on the initial price data and the corresponding real product price data, this embodiment constructs a price loss function through the mean square error, which is expressed as follows:

[0040] Step 4: Perform negative gradient analysis on the initial price data based on the price loss function, and establish a new price learner based on the gradient analysis results. The gradient analysis results and price learner are expressed as follows:

[0041]

[0042] Step 5: Perform iterative analysis through the price learner and the price loss function until convergence to obtain an index weight analysis model. Based on the index weight analysis model, analyze and predict the data set related to the product to be analyzed to obtain the price index weight; in, Represents feature prediction contribution data, Indicates that it does not contain A subset of the comprehensive characteristics of a product, represents the size of the feature subset, represents the comprehensive feature set of the product, Indicates that it contains The model prediction value of the feature subset of the comprehensive features of the product, represents the factorial of the total number of comprehensive characteristics of the product, represents the initial price data, represents the number of price learners, represents the learner weight, Indicates A price learner, represents the price loss function, Represents the real product price data, Represents the results of gradient analysis.

[0043] The category price data and grade price data are obtained through the product price data. In this embodiment, taking glass products as an example, the category price data represents the price of a certain category of glass (such as white glass, ultra-white glass), and the grade price data represents the price of a certain grade of glass (such as superior product, first-class product). According to the category price data, category index weight, grade price data and grade index weight, the price data of different regions are obtained, which are expressed as follows:

[0044] Through the price data of different regions and the corresponding regional index weights, the average price data of the product is obtained, which is expressed as follows:

[0045] in, Indicates price data for different regions. represents the category index weight, Represents category price data, represents the grade index weight, Indicates grade price data, Indicates the average price of the product. represents the regional index weight, Represents price data of different regions.

[0046] After obtaining the product average price data, in this embodiment, a chain pull formula is used to set a time period, and the initial price index data is obtained by the product quantity data and the product average price data corresponding to the time period. The chain pull formula is specifically expressed as follows:

[0047] in, Indicates The initial price index data corresponding to the time period, Indicates The initial price index data corresponding to the time period, , Indicates , The average price data of products corresponding to the time period, Indicates The product quantity data corresponding to each time period, Indicates the number of time periods.

[0048] The initial price index data is corrected for cumulative errors by constructing a fluctuating price pre-training model. In this embodiment, the fluctuating price pre-training model is a regression model. Analysis is performed based on the fluctuating price pre-training model to obtain a fluctuating price factor. The initial price index data is corrected by the fluctuating price factor to obtain price index data, including the following steps: Step 1: Obtain historical product price data and historical price fluctuation factors, analyze them through the price fluctuation pre-training model, and construct the product price relationship equation, which is expressed as follows:

[0049] Step 2: Construct a product error function based on the product price relationship equation, and solve the price regression factor in the product price relationship equation based on the least squares method, as shown below:

[0050]

[0051] Step 3: Through the product price relationship equation, based on the price regression factor and product price data, analyze to obtain the initial fluctuation price factor; Step 4: Based on the price loss function and the initial volatility price factor, a volatility loss function of the volatility price pre-training model is constructed, and the volatility price pre-training model is trained by the volatility loss function to obtain a volatility price model, and reasoning analysis is performed based on the product price data to obtain the volatility price factor, where the volatility loss function is expressed as follows:

[0052] Step 5: Correct the initial price index data by using the fluctuating price factor to eliminate the impact of the cumulative error and obtain the price index data; in, represents the volatility loss function, represents the price loss function, represents the regularization coefficient, represents the initial volatility price factor, represents the number of volatility price factors, represents the historical volatility price factor, represents the intercept term, , represents the price regression factor, , Represents historical product price data, represents the error term, represents the product error function, Indicates Historical volatility price factors, Indicates The predicted value of the historical volatility price factor, Represents the number of historical volatility price factors.

[0053] At the same time, in order to improve the model calculation efficiency and real-time performance when facing a large number of data sets, this embodiment introduces the parallel data calculation framework spark, and optimizes the efficiency of the index weight analysis pre-training model and the volatility price pre-training model through data partitioning and parallel calculation. The model processing time after efficiency optimization is expressed as follows:

[0054] in, represents the model processing time after efficiency optimization, represents the processing time before efficiency optimization, Indicates the number of cores or parallelism before efficiency optimization. Indicates the number of cores or degree of parallelism after efficiency optimization.

[0055] Through the above steps, this method performs multi-dimensional analysis and processing on product-related data sets, eliminates the cumulative errors of price index data, and improves the accuracy of price index data. At the same time, this method improves the real-time calculation of price index data, ensures calculation efficiency, tracks market price changes of products in real time, provides more timely and accurate market trend forecasts, and effectively improves the forecast accuracy of price index data.

[0056] Embodiment 2: A product price index analysis system based on data processing, such as Figure 2 As shown, it includes a data preprocessing module 100, a comprehensive feature extraction module 200, an index weight calculation module 300, a product average price calculation module 400, a product price analysis module 500 and a price index correction module 600; The data preprocessing module 100 acquires and preprocesses a product-related data set to obtain a preprocessed product data set, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; The comprehensive feature extraction module 200 extracts comprehensive features from the pre-processed product data to form a product comprehensive feature set, wherein the product comprehensive features include production and sales features, supply and demand features, regional features, fluctuation features and trend features; The index weight calculation module 300 establishes an index weight analysis pre-training model. The index weight analysis pre-training model analyzes and predicts the comprehensive characteristics of the product through a price learner to obtain a price index weight, wherein the price index weight includes a regional index weight, a category index weight, and a grade index weight; The product average price calculation module 400 obtains category price data and grade price data through product price data, obtains price data of different regions by combining category index weight and grade index weight, and obtains product average price data by combining price data of different regions and corresponding regional index weight; The product price analysis module 500 presets a time period and obtains corresponding product quantity data, combines the product average price data, and analyzes through a chain-pull formula to obtain initial price index data; The price index correction module 600 constructs a fluctuation price pre-training model, analyzes historical product price data and historical fluctuation price factors, obtains a product price relationship equation, and then obtains a fluctuation price factor, and corrects the initial price index data based on the fluctuation price factor to obtain price index data.

[0057] Various changes and modifications can be made without departing from the spirit and scope of the present invention, and all equivalent technical solutions also belong to the scope of the present invention.

[0058] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0059] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0061] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0063] It should be noted that: The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0064] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A product price index analysis method based on data processing, characterized in that: The following steps are involved: Acquire and preprocess product-related data sets to obtain preprocessed product data sets, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; Extract comprehensive features from pre-processed product data to form a comprehensive product feature set, where comprehensive product features include production and sales features, supply and demand features, regional features, fluctuation features, and trend features; Establish an index weight analysis pre-training model. The index weight analysis pre-training model uses a price learner to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, where the price index weight includes the regional index weight, category index weight and grade index weight; The product price data is used to obtain the category price data and grade price data. The category index weight and grade index weight are combined to obtain the price data of different regions. The product average price data is obtained by combining the price data of different regions and the corresponding regional index weights. Preset the time period and obtain the corresponding product quantity data, combine it with the product average price data, and analyze it through the chain-pull formula to obtain the initial price index data; Construct a volatility price pre-training model, analyze historical product price data and historical volatility price factors, obtain the product price relationship equation, and then obtain the volatility price factor. Based on the volatility price factor, correct the initial price index data to obtain the price index data.

2. The product price index analysis method based on data processing according to claim 1 is characterized in that: The step of obtaining a product-related data set and preprocessing it to obtain a preprocessed product data set includes the following steps: Clean and denoise product-related data to obtain initial preprocessed data; Perform missing value analysis and outlier analysis on the initial preprocessed data to obtain missing data and outlier data; Preset data thresholds, take missing data and abnormal data as the center, and obtain product-related data within the data threshold range to obtain the first adjacent related data; Preset a time threshold, and obtain product-related data corresponding to missing data and abnormal data within the time threshold, i.e., second adjacent related data; Based on the first adjacent related data and the second adjacent related data, the missing data and the abnormal data are supplemented by bilinear difference to correct the initial preprocessed data, thereby obtaining the preprocessed product data.

3. The product price index analysis method based on data processing according to claim 1 is characterized in that: The price learner is used to analyze and predict the comprehensive characteristics of the product to obtain the price index weight, including the following steps: By analyzing the comprehensive characteristics of the product, the corresponding marginal contribution data is obtained, which is expressed as follows: Based on the marginal contribution data corresponding to the comprehensive characteristics of the product, the learner weight of the price learner is obtained. Based on the learner weight and the price learner, the comprehensive characteristics of the product are analyzed to obtain the initial price data, which is expressed as follows: Based on the initial price data and the real product price data, a price loss function is constructed, which is expressed as follows: Based on the price loss function, the initial price data is subjected to gradient analysis, and a price learner is established based on the gradient analysis results. The gradient analysis results and the price learner are expressed as follows: Through iterative analysis of the price learner and the price loss function, an index weight analysis model is obtained. Based on the index weight analysis model, the comprehensive characteristics of the product to be analyzed are inferred to obtain the price index weight. in, represents the feature prediction contribution data, Indicates that it does not contain A subset of the comprehensive characteristics of a product, represents the size of the feature subset, represents the comprehensive feature set of the product, Indicates that it contains The model prediction value of the feature subset of the comprehensive features of the product, represents the factorial of the total number of comprehensive characteristics of the product, represents the initial price data, represents the number of price learners, represents the learner weight, Indicates A price learner, represents the price loss function, Represents the real product price data, Represents the results of gradient analysis.

4. The product price index analysis method based on data processing according to claim 1 is characterized in that: The method of obtaining category price data and grade price data through product price data, combining category index weights and grade index weights to obtain price data of different regions, and combining price data of different regions and corresponding regional index weights to obtain product average price data includes the following steps: By analyzing the category price data, category index weight, grade price data and grade index weight, we can get the price data of different regions, which are shown as follows: By analyzing the price data of different regions and the corresponding regional index weights, the average price data of the products is obtained, which is expressed as follows: in, Indicates price data for different regions. represents the category index weight, Represents category price data, represents the grade index weight, Indicates grade price data, Indicates the average price of the product. represents the regional index weight, Represents price data of different regions.

5. The product price index analysis method based on data processing according to claim 1 is characterized in that: The preset time period and obtaining the corresponding product quantity data, combined with the product average price data, are analyzed through a chain pull formula to obtain the initial price index data, including the following steps: Get product quantity data corresponding to different time periods; The initial price index data is obtained by analyzing the product quantity data and the product average price data using the chain pull formula, where the chain pull formula is expressed as follows: in, Indicates The initial price index data corresponding to the time period, Indicates The initial price index data corresponding to the time period, , Indicates , The average price data of products corresponding to the time period, Indicates The product quantity data corresponding to each time period, Indicates the number of time periods.

6. The product price index analysis method based on data processing according to claim 1 is characterized in that: The fluctuation price factor is obtained by the following steps: The historical product price data and historical price fluctuation factors are analyzed through the price fluctuation pre-training model to obtain the product price relationship equation, which is expressed as follows: The product error function is constructed based on the product price relationship equation, and the product price relationship equation is solved based on the least squares method to obtain the price regression factor. The specific solution process is as follows: Based on the price regression factor and product price data, the initial volatility price factor is obtained; Construct a volatility loss function for the volatility price pre-training model, train the volatility price pre-training model, obtain the volatility price model, perform inference analysis based on product price data, and obtain the volatility price factor, where the volatility loss function is expressed as follows: in, represents the volatility loss function, represents the price loss function, represents the regularization coefficient, represents the initial volatility price factor, represents the number of volatility price factors, represents the historical volatility price factor, represents the intercept term, , represents the price regression factor, , Represents historical product price data, represents the error term, represents the product error function, Indicates Historical volatility price factors, Indicates The predicted value of the historical volatility price factor, Represents the number of historical volatility price factors.

7. The product price index analysis method based on data processing according to claim 1 is characterized in that: It also includes optimizing the efficiency of the index weight analysis pre-training model and the volatility price pre-training model through data partitioning and parallel computing. The model processing time after efficiency optimization is expressed as follows: in, represents the model processing time after efficiency optimization, represents the processing time before efficiency optimization, Indicates the number of cores or parallelism before efficiency optimization. Indicates the number of cores or degree of parallelism after efficiency optimization.

8. A product price index analysis system based on data processing, characterized in that: It includes data preprocessing module, comprehensive feature extraction module, index weight calculation module, product average price calculation module, product price analysis module and price index correction module; The data preprocessing module acquires and preprocesses a product-related data set to obtain a preprocessed product data set, wherein the product-related data includes product price data, market demand data, product inventory data, and product output data; The comprehensive feature extraction module extracts comprehensive features from the pre-processed product data to form a product comprehensive feature set, wherein the product comprehensive features include production and sales features, supply and demand features, regional features, fluctuation features and trend features; The index weight calculation module establishes an index weight analysis pre-training model. The index weight analysis pre-training model analyzes and predicts the comprehensive characteristics of the product through a price learner to obtain a price index weight, wherein the price index weight includes a regional index weight, a category index weight, and a grade index weight; The product average price calculation module obtains category price data and grade price data through product price data, obtains price data of different regions by combining category index weight and grade index weight, and obtains product average price data by combining price data of different regions and corresponding regional index weight; The product price analysis module presets a time period and obtains the corresponding product quantity data, combines the product average price data, and analyzes through a chain-pull formula to obtain initial price index data; The price index correction module constructs a fluctuation price pre-training model, analyzes historical product price data and historical fluctuation price factors, obtains a product price relationship equation, and then obtains a fluctuation price factor, and corrects the initial price index data based on the fluctuation price factor to obtain price index data.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A product price index analysis device based on data processing, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images