Data processing method, device, equipment and computer readable storage medium
By training neural network models in the community to predict material demand, the problem that the existing technology cannot accurately predict community material demand is solved, and the efficiency of resource allocation and operational costs are achieved.
Patent Information
- Application Number
- CN202010642357.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-07-06
AI Technical Summary
The existing technology cannot accurately predict community material demand, resulting in increased cross-infection probability and low resource allocation efficiency when residents purchase.
By training a neural network model, the target category is determined based on the order data in the target area, the training data is obtained and the material demand is predicted, and the data foundation is provided to support product recommendations and resource allocation.
Accurate prediction of community material demand has been achieved, resource allocation efficiency has been improved, operating costs have been reduced, and platform trust and usage rate have been improved.
Smart Images

Figure CN113781077B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a data processing method, apparatus, device, and computer-readable storage medium. Background Art
[0002] During the epidemic of infectious diseases, residents going out to purchase daily necessities will increase the probability of cross-infection, and residents' purchasing needs cannot be met in time. The operation mode of unified procurement and distribution for the community based on the community material demand forecast has gradually become a trend. Community material demand forecast is to specify the forecast of daily necessities urgently needed by community residents, providing a data basis for the rational allocation of resources and meeting the living needs of residents.
[0003] In the process of realizing the present invention, the inventors found that the demand forecasting methods in the prior art are all aimed at predicting user demand, and currently cannot accurately predict community material demand. Summary of the invention
[0004] Embodiments of the present invention provide a data processing method, apparatus, device, and computer-readable storage medium for training a prediction model that can accurately predict the demand for materials of a target category in a target area.
[0005] In a first aspect, an embodiment of the present invention provides a data processing method, including:
[0006] According to the area range of the target area, acquiring order data within the target area;
[0007] Determining target categories to be predicted and training data for each of the target categories based on the order data in the target area;
[0008] The neural network model is trained according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
[0009] In a second aspect, an embodiment of the present invention provides a data processing method, which is applied to a service platform including a data collector and a processor, wherein the service platform is used to train a prediction model corresponding to each target category in the target area based on historical order data in the target area, and the method includes:
[0010] According to the area range of the target area, acquiring order data within the target area;
[0011] Determining target categories to be predicted and training data for each of the target categories based on the order data in the target area;
[0012] The neural network model is trained according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
[0013] In a third aspect, an embodiment of the present invention provides a data processing device, including:
[0014] A data collection module, used for acquiring order data in a target area according to the area range of the target area;
[0015] A data processing module, used to determine the target category to be predicted and the training data of each target category according to the order data in the target area;
[0016] The model training module is used to train the neural network model according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
[0017] In a fourth aspect, an embodiment of the present invention provides a data processing device, including:
[0018] A processor, a memory, and a computer program stored on the memory and executable on the processor;
[0019] Wherein, when the processor runs the computer program, the data processing method described in the first aspect above is implemented.
[0020] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data processing method described in the first aspect is implemented.
[0021] The data processing method, device, equipment and computer-readable storage medium provided by the embodiments of the present invention define multiple target areas, and for each target area, obtain the order data in the target area; determine the target category to be predicted and the training data of each target category based on the order data in the target area; use the training data of each target category to train the neural network model to obtain the prediction model corresponding to the target area and the target category, and the prediction model can accurately predict the demand for materials of the target category in the target area, thereby providing a data basis for product recommendation, allocation, and unified distribution. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A flow chart of a method for data processing provided in Embodiment 1 of the present invention;
[0023] Figure 2 A flow chart of a method for data processing provided in Embodiment 2 of the present invention;
[0024] Figure 3 A schematic diagram of the sensitivity of various categories of materials to seasonal and periodic influences provided in the second embodiment of the present invention;
[0025] Figure 4 A framework diagram of a neural network model provided in Embodiment 2 of the present invention;
[0026] Figure 5 A pseudo code example of automatic hyperparameter selection provided in the second embodiment of the present invention;
[0027] Figure 6 A schematic diagram of a regional material category demand forecasting process provided in the second embodiment of the present invention;
[0028] Figure 7 This is an example diagram of demand forecasting results provided in the second embodiment of the present invention;
[0029] Figure 8 A schematic diagram of the structure of a data processing device provided in Embodiment 3 of the present invention;
[0030] Fig. 9 A schematic diagram of the structure of a data processing device provided in Embodiment 4 of the present invention;
[0031] Fig.10 This is a schematic diagram of the structure of a data processing device provided in Embodiment 5 of the present invention.
[0032] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0034] The terms "first", "second", etc. involved in the embodiments of the present invention are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
[0035] During an epidemic of infectious diseases, residents going out to purchase daily necessities will increase the probability of cross-infection, and residents' purchasing needs cannot be met in a timely manner. The operation model of unified procurement and distribution for the community based on community material demand forecasts has gradually become a trend. Its main advantages are: (1) From the perspective of community residents, accurate demand forecasts can meet residents' life needs to the greatest extent, enhance their trust in the platform, and thus increase the platform usage rate; (2) From the perspective of operators, how to select products and make recommendations is a problem that plagues operators. Experience-based product selection is time-consuming and labor-intensive. Accurate demand forecasts can provide important references for product selection, greatly reduce operating costs, and improve operational efficiency. Community material demand forecasts specify forecasts of daily necessities urgently needed by community residents, and provide a data basis for the rational allocation of resources to meet residents' life needs. In the process of realizing the present invention, the inventors found that the demand forecasting methods in the prior art are all for user demand forecasts, and currently cannot accurately predict community material demand.
[0036] In response to the above technical problems, embodiments of the present invention provide a data processing method, apparatus, device and computer-readable storage medium, which can train a prediction model corresponding to each commonly used material category in a region based on the historical order data in the region. The prediction model can be used to predict the demand for materials of the target category in the target region, thereby providing a data basis for product recommendation, allocation, unified distribution, etc.
[0037] The method provided in the embodiment of the present invention is applied to the service platforms of various e-commerce companies, operators, physical stores, physical warehouses, etc. The executor of the method can be the server of the service platform. In other embodiments, the method can also be applied to other devices. This embodiment is schematically illustrated using the server as an example.
[0038] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.
[0039] Figure 1 This is a flow chart of a method for data processing provided in Embodiment 1 of the present invention. Figure 1 As shown, the specific steps of this method are as follows:
[0040] Step S101: Acquire order data within a target area according to the area range of the target area.
[0041] The target area refers to any area where the demand for user materials is to be predicted, and the area range of the target area refers to the range defined by the geographical boundaries of the target area. The geographical boundaries can be defined according to the needs of the actual application scenario, and this embodiment does not make specific limitations here. For example, the target area can be a cell, community, village, etc.
[0042] For example, the geographical boundary of the target area for which the material demand is to be predicted may be pre-delineated, or the area range of the pre-delineated target area input by the user may be accepted.
[0043] In actual applications, there may be multiple target areas that a user is concerned about, and the data processing process for each target area is the same. This embodiment takes the data processing process for one target area as an example for illustrative description.
[0044] In the embodiment of the present invention, online consumption orders and / or offline consumption orders in the target area within the preset historical duration can be obtained to obtain order data in the target area. The preset historical duration can be set and adjusted according to the actual application scenario, and this embodiment is not specifically limited here. For example, the preset historical duration can be 28 days, one month, several months, etc.
[0045] The order data includes the delivery address, the purchased materials and their corresponding categories, the order time, etc. The target area where the delivery address is located is determined based on the delivery address in the order, thereby determining which target area the order belongs to.
[0046] Step S102: Determine the target category to be predicted and the training data for each target category according to the order data in the target area.
[0047] The target category to be predicted refers to the material category for which user demand prediction is required. In the embodiment of the present invention, for a target area, a prediction model is trained for each target category to predict the demand for materials of the target category in the target area.
[0048] Since there are many categories of goods, for example, in the category system of an e-commerce company, there are more than 10,000 fine-grained categories and more than 100 coarse-grained categories. If a prediction model is trained for each category, the efficiency of model training will be very low and the hardware cost will be very high.
[0049] In an embodiment of the present invention, after the order data is obtained, the material categories whose sales volume meets certain conditions can be determined as target categories to be predicted based on the order data and the sales volume of each material category. This allows key and commonly used material categories to be used as target categories, thereby improving efficiency and reducing costs while meeting demand.
[0050] In this embodiment, for each target category in the target area, a set of training data is determined for training a prediction model corresponding to the target category in the target area.
[0051] Step S103: Train the neural network model according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category. The prediction model is used to predict the demand for materials of the target category in the target area.
[0052] The neural network model is trained using the training data of each target category in the target area to obtain the prediction model corresponding to the target area and target category. The prediction model can be used to predict the demand of the target area and target category in the future period, that is, to predict the sales volume of the target area and target category in the future period.
[0053] The embodiment of the present invention defines multiple target areas, and for each target area, obtains order data within the target area; determines the target category to be predicted and the training data of each target category based on the order data within the target area; uses the training data of each target category to train a neural network model to obtain a prediction model corresponding to the target area and the target category. The prediction model can accurately predict the demand for materials of the target category in the target area, thereby providing a data basis for product recommendation, allocation, and unified distribution.
[0054] Figure 2 A flow chart of a method for data processing provided in the second embodiment of the present invention. Based on the above-mentioned first embodiment, in this embodiment, the training data of the target category includes sample data of the target category in multiple historical time periods, and the sample data of the target category in each historical time period includes discrete features and time series features, wherein the discrete features at least include the week and month corresponding to the historical time period, and the time series features at least include the sales volume and number of users of the target category in the historical time period. Figure 2 As shown, the specific steps of this method are as follows:
[0055] Step S201: Acquire order data within a target area according to the area range of the target area.
[0056] The target area refers to any area where the demand for user materials is to be predicted, and the area range of the target area refers to the range defined by the geographical boundaries of the target area. The geographical boundaries can be defined according to the needs of the actual application scenario, and this embodiment does not make specific limitations here. For example, the target area can be a cell, community, village, etc.
[0057] For example, the geographical boundary of the target area for which the material demand is to be predicted may be pre-delineated, or the area range of the pre-delineated target area input by the user may be accepted.
[0058] In actual applications, there may be multiple target areas that a user is concerned about, and the data processing process for each target area is the same. This embodiment takes the data processing process for one target area as an example for illustrative description.
[0059] In the embodiment of the present invention, online consumption orders and / or offline consumption orders in the target area within the preset historical duration can be obtained to obtain order data in the target area. The preset historical duration can be set and adjusted according to the actual application scenario, and this embodiment is not specifically limited here.
[0060] The order data includes the delivery address, the purchased materials and their corresponding categories, the order time, etc. The target area where the delivery address is located is determined based on the delivery address in the order, thereby determining which target area the order belongs to.
[0061] Step S202: Update the category information of the materials in the order data according to the mapping relationship between the materials and the material categories.
[0062] The target category to be predicted refers to the material category for which user demand prediction is required. In the embodiment of the present invention, for a target area, a prediction model is trained for each target category to predict the demand for materials of the target category in the target area.
[0063] For example, in actual applications, the corresponding categories of materials in the order information are seriously missing, and the category information of materials in the order data is updated, which at least includes: filling in the missing material categories in the order data, improving the reliability of the order data, and thus improving the prediction accuracy of the prediction model trained based on the order data.
[0064] Exemplarily, since the categories of materials may change over time, the material categories corresponding to the same material may be different at different times. The category information of the materials in the order data is updated, and the categories of the materials in the order data may be corrected so that the categories corresponding to the same material are the same, thereby improving the reliability of the order data and thus improving the prediction accuracy of the prediction model trained based on the order data.
[0065] Step S203: Determine the sales volume and number of users of each material category in each historical period based on the updated order data.
[0066] In this embodiment, determining the sales volume of each material category in each historical period and determining the number of users of each material category in each historical period may be two relatively independent data processing processes, and the execution order of the two processes is not limited here.
[0067] In addition, in order to improve the efficiency of data processing, the sales volume of each material category in each historical period and the number of users of each material category in each historical period can be determined together. By scanning the order data once, the sales volume and the number of users can be counted at the same time.
[0068] Among them, each historical period determines the time granularity of the forecasting model to predict the demand, and the forecasting model is used to predict the demand in at least one future period, where the length of the future period is consistent with the length of the historical period.
[0069] In addition, the division method of the historical period can be set and adjusted according to the actual application scenario, and this embodiment does not make specific restrictions here. For example, one historical period can correspond to one natural day.
[0070] Step S204: Determine at least one target category based on the sales volume of each material category in each historical period.
[0071] The target category to be predicted refers to the material category for which user demand prediction is required. In the embodiment of the present invention, for a target area, a prediction model is trained for each target category to predict the demand for materials of the target category in the target area.
[0072] In actual applications, there are many categories of goods. For example, in the category system of an e-commerce company, there are more than 10,000 fine-grained categories and more than 100 coarse-grained categories. If a prediction model is trained for each category, the efficiency of model training will be very low and the hardware cost will be very high.
[0073] In an embodiment of the present invention, after the order data is obtained, the material categories whose sales volume meets certain conditions can be determined as target categories to be predicted based on the order data and the sales volume of each material category. While meeting the demand, the categories with very low sales volume can be filtered out based on the sales volume of each material category, and the key and commonly used commodity categories can be retained, which can improve efficiency and reduce costs.
[0074] For example, according to the sales volume of each material category in each historical period, at least one target category is determined, which can be achieved by any of the following methods:
[0075] The material category whose total sales volume in each historical period is greater than the first threshold is determined as the target category; or, the material category whose average sales volume in each historical period is greater than the second threshold is determined as the target category; or, the material category whose sales volume in at least one historical period is greater than the third threshold is determined as the target category.
[0076] Among them, the first threshold, the second threshold and the third threshold can all be set and adjusted according to the actual application scenario, and this embodiment does not make any specific limitations here.
[0077] In addition, the material categories may be screened in other ways according to the sales volume of each material category in each historical period to determine at least one target category, which is not limited in this embodiment.
[0078] Step S205: Generate a piece of sample data according to the sales volume and the number of users of each target category in each historical period. The sample data of each target category in each historical period constitutes the training data of the target category.
[0079] The sample data includes: the sales volume and number of users of the target category in the historical period, and the week and month corresponding to the historical period.
[0080] In this embodiment, the training data of the target category includes sample data of the target category in multiple historical periods, and the sample data of the target category in each historical period includes discrete features and time series features. The discrete features include at least the week and month corresponding to the historical period, and the time series features include at least the sales volume and number of users of the target category in the historical period.
[0081] Exemplarily, the week and month corresponding to the historical period are in the form of one-hot encoding.
[0082] In practical applications, since the target area (such as a community) is small, the extreme behavior of individual users has a greater impact on the demand forecast of the entire target area. For example, for a community with a gym, the gym suddenly purchased a large number of sports bras in a certain week, causing a sharp increase in the sales of sports bras in the entire community, but this does not mean that the residents of the entire community will purchase a large number of sports bras in the future. Therefore, the extreme behavior of individual users will affect the stability of the prediction model. For example, if a person suddenly buys a lot of decorative calligraphy and paintings, it does not mean that the entire community will need a large number of decorative calligraphy and paintings in the future.
[0083] In addition, different categories of materials have different sensitivities to seasonal and cyclical effects. Figure 3 As shown, the trend of digital products is relatively stable, the upward trend of fresh products is obvious, and the cyclical fluctuations are large.
[0084] In the embodiment of the present application, at least the following two types of data are integrated: the first type is time series data, including the sales volume of the target category in each historical period and the number of users who purchased such target materials; the second type is factors that affect the model, such as holidays, weeks, months, etc. The prediction model characterizes and fuses features for the two forms of data, and finally performs time series prediction, outputs the prediction error of the model and the demand forecast for the future period. By adding the number of users who purchase the category, the influence of individual extreme behaviors can be prevented. Based on multi-source data fusion, the stability and accuracy of the prediction model can be improved.
[0085] Step S206: Train the neural network model according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category.
[0086] Among them, the forecasting model is used to predict the demand for materials of the target category in the target area.
[0087] For example, Figure 4 As shown, the neural network model includes: a representation layer and a time series prediction layer, and the time series prediction layer includes two layers of long short-term memory (LSTM) neural networks.
[0088] The representation layer is used to reduce the dimension of the discrete features in each input sample data and fuse them with the time series features. The obtained feature data is used as the input of the time series prediction layer.
[0089] The time series prediction layer is used to predict the demand for target category materials in the target area based on the input feature data.
[0090] Specifically, Figure 4 As shown in the figure, the input of the neural network model includes four aspects of feature data, among which the time series feature includes the number of users of the target category in the historical period (which can be expressed by x u The sales volume of the target category in the historical period (can be represented by x s The discrete features include the week corresponding to the historical period (which can be represented by x d indicates that the value is between 0 and 6); and the month (which can be represented by x m Indicates that the value is between 0 and 11), and all discrete features can be represented in the form of one-hot representation.
[0091] like Figure 4 As shown in the figure, the prediction model is mainly divided into the following two parts: (1) the representation layer, which is mainly used to learn the representation of input data; (2) the time series prediction layer, which is mainly used to predict the needs of various purposes in the future based on historical data. The following will explain the two parts in detail:
[0092] The representation layer first transforms the discrete features (xd and x m ) is used for dimensionality reduction and then combined with the time series features (x u and x s ) for fusion.
[0093] Specifically, we first use a layer of representation network to learn the representation of the day of the week and month. The calculation formula is as follows:
[0094] e d =x d *w d , e m =x m *w m ,
[0095] in, Among them, d0 and d1 are adjustable parameters, representing the representation dimension of the week and the representation dimension of the month respectively, w d and w m They represent the weights of the weekly data representation network and the monthly data representation network respectively. Then, the four types of representations are fused to obtain the fused representation as the input of the time series prediction layer. The calculation formula of the fused representation is as follows:
[0096] e=[x u , x s , e d , e m ].
[0097] For example, when reducing the dimension of discrete features, the discrete features can be reduced to a preset dimension. The preset dimension can be set according to an empirical value in combination with the actual application scenario, for example, it can be 2 or 3; or it can be determined as a hyperparameter in the model training, which is not specifically limited in this embodiment.
[0098] The time series prediction layer is a stacked two-layer LSTM, divided into two structures, the left side represents the learning of historical states, and the right side represents the speculation of future multi-step states. The input of the time series prediction layer is {e1,...,e i , ..., e t},e i It represents the input data representation at time i, and t represents the size of the window, which is an adjustable hyperparameter. They represent the hidden layer states of the first LSTM layer and the hidden layer states of the second LSTM layer, respectively. It represents the demand forecast result for the i-th time period in the future, and k represents the step size of predicting the future time.
[0099] For example, the data window size can be set to 28 and the step size of future prediction can be set to 7, which means that based on the training data of the past 28 days (4 weeks), the demand for the target category per day in the next 7 days (a week) is estimated.
[0100] In actual applications, since different material categories have different sensitivity to time, the sales trends are also different. Therefore, using the same model parameters for all categories cannot meet business needs, which will result in accurate predictions for some material categories, while large errors will occur for other categories.
[0101] In an embodiment of the present invention, when training a neural network model, automatic hyperparameter selection can be implemented through an automatic hyperparameter selection module, and a hyperparameter combination can be automatically selected for model training based on a predefined hyperparameter search space and a preset search strategy.
[0102] Specifically, the automatic hyperparameter selection module can select the most suitable parameters for each material category to improve the overall prediction accuracy of the model. The process of the hyperparameter selection module includes: according to the pre-defined hyperparameter search space, a set of hyperparameters is extracted from the search space each time the search is performed, the model is trained, and then according to the difference in evaluation indicators returned by the model, inappropriate parameters are removed to narrow the scope of the search space until the stopping condition is reached.
[0103] For example, automatic hyperparameter selection can be done using Figure 5 Pseudocode implementation shown.
[0104] Specifically, Figure 5 As shown in Figure 1, the input of the automatic hyperparameter selection module can be a predefined hyperparameter search space, which mainly includes the set of learning rates lrs and the set of network hidden layer dimensions dims. In the process of learning hyperparameters, the outer layer index m is initialized first. o and inner index m i is an infinite value, and then traverses each set of hyperparameter combinations in the search space, trains the model and obtains the model evaluation index m. If the model error of the current parameter is the smallest, the model parameters are updated; if the model error of the current parameter is not the smallest, the current search is jumped out, the current hyperparameter combination is judged, and the search space is narrowed; and so on, until all possible hyperparameter combinations are traversed.
[0105] In addition, automatic hyperparameter selection can also be implemented using similar methods in the prior art, which will not be described in detail in this embodiment.
[0106] Furthermore, during the model training process, the optimization strategy of the model is to optimize the parameters of the network according to the square error loss. The calculation formula of the square error loss (MSE) is as follows:
[0107]
[0108] For example, the mean absolute percentage error (MAPE) value may be used to measure the prediction error of the future period, and the calculation formula is as follows:
[0109]
[0110] where y i is the actual demand value, is the predicted demand value.
[0111] Optionally, other error calculation methods may be used to measure the prediction error of the future time period according to the actual application scenario, such as root mean square error, mean absolute error, etc., which are not specifically limited in this embodiment.
[0112] By performing the processing of steps S201 to S206 on each target area, the prediction model corresponding to each target category in each target area can be determined.
[0113] Figure 6 A schematic diagram of the regional material category demand forecasting process is provided, such as Figure 6 As shown in the figure, first determine the target area according to the circled area; extract the order data of the target area, and perform data preprocessing to fill in the missing values; filter the material categories according to the order data, and retain the key and commonly used target categories; train the model through the automatic hyperparameter selection module to obtain the prediction model corresponding to each target category in each target area; and predict the demand of the target category according to the prediction model. Among them, the automatic hyperparameter selection module selects a parameter combination from the predefined search space according to the pre-set selection strategy, determines the evaluation index of the prediction model with the current parameter combination as the model parameter, removes inappropriate parameters according to the feedback strategy, and narrows the scope of the search space until the stop condition is reached.
[0114] For example, the training is based on the order data of an e-commerce platform from 2016 to 2019, and the data of 2020 is tested. The time granularity is 1 day, the window size is 28, and the prediction step is 7. Taking the Yushu Linfeng Community in Jimo District, Qingdao as an example, the prediction error of 58% of the categories is below 0.2, and the prediction error of 70% of the categories is below 0.3.
[0115] For example, Figure 7 The demand forecast results of multiple material categories in three target areas (communities or communities) are given. Figure 7 It can be seen from the forecast results that there is a relatively large demand for condiments and snacks in different communities, and the forecast error for categories with a shelf life such as milk and dairy products is within an acceptable range.
[0116] Step S207: predict the demand for materials of the target category in the target area according to the prediction model.
[0117] After obtaining the prediction model corresponding to the target category in the target area, according to the size of the data window in the prediction model, the historical order data in the target area within a recent period of time (consistent with the size of the data window) can be obtained, and the historical order data is processed by steps S201-S203 and step S205 to obtain multiple input data, each input data including: the sales volume and number of users of the target category in the corresponding historical period, and the week and month corresponding to the historical period.
[0118] By feeding the input data into the forecasting model, the demand for target categories of materials in the target area in the future period can be accurately predicted.
[0119] Step S208: Allocate materials to the target area according to the demand for materials of the target category in the target area.
[0120] After predicting the demand for materials of the target category in the target area in the future time period, materials can be allocated to the target area according to the demand for materials of the target category in the target area in the future time period.
[0121] For example, based on the demand for target category materials in the target area in the future period, the target area is assigned with recommended goods, and the information of the recommended goods is pushed to the corresponding users. Accurate demand forecasting greatly reduces the cost of product selection and improves the accuracy of product recommendations.
[0122] For example, according to the demand for materials of a target category in the target area in a future period, the materials to be delivered are allocated to the target area, so that the materials to be delivered are uniformly delivered to the sites in the target area.
[0123] In the embodiment of the present invention, the training data incorporates at least the following two types of data: the first type is time series data, including the sales volume of the target category in each historical period and the number of users who purchased such target materials; the second type is factors that affect the model. The prediction model characterizes and fuses the features of the two forms of training data, and finally performs time series prediction, outputs the prediction error of the model and the demand forecast for the future period, and based on multi-source data fusion, can prevent the influence of individual extreme behaviors and improve the stability and accuracy of the prediction model.
[0124] Figure 8 The structure diagram of the data processing device provided in the third embodiment of the present invention is shown in FIG. The data processing device provided in the embodiment of the present invention can execute the processing flow provided in the data processing method embodiment. Figure 8As shown, the data processing device 30 includes: a data acquisition module 301, a data processing module 302 and a model training module 303.
[0125] Specifically, the data collection module 301 is used to obtain order data in the target area according to the area range of the target area.
[0126] The data processing module 302 is used to determine the target category to be predicted and the training data of each target category according to the order data in the target area.
[0127] The model training module 303 is used to train the neural network model according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category. The prediction model is used to predict the demand for materials of the target category in the target area.
[0128] The device provided in the embodiment of the present invention can be specifically used to execute the method embodiment provided in the above-mentioned embodiment 1, and the specific functions will not be repeated here.
[0129] The embodiment of the present invention defines multiple target areas, and for each target area, obtains order data within the target area; determines the target category to be predicted and the training data of each target category based on the order data within the target area; uses the training data of each target category to train a neural network model to obtain a prediction model corresponding to the target area and the target category. The prediction model can accurately predict the demand for materials of the target category in the target area, thereby providing a data basis for product recommendation, allocation, and unified distribution.
[0130] Fig. 9 A schematic diagram of the structure of a data processing device provided for Embodiment 4 of the present invention. Based on the above Embodiment 3, in this embodiment, the training data of the target category includes sample data of the target category in multiple historical time periods, and the sample data of the target category in each historical time period includes discrete features and time series features, wherein the discrete features at least include the week and month corresponding to the historical time period, and the time series features at least include the sales volume and number of users of the target category in the historical time period.
[0131] In a possible implementation manner, the data processing module is further used for:
[0132] According to the mapping relationship between materials and material categories, the category information of materials in the order data is updated; according to the updated order data, the sales volume of each material category in each historical period is determined; according to the sales volume of each material category in each historical period, at least one target category is determined.
[0133] In a possible implementation manner, the data processing module is further used for:
[0134] Fill in the missing material categories in the order data.
[0135] In a possible implementation manner, the data processing module is further used for:
[0136] The material category whose total sales volume in each historical period is greater than the first threshold is determined as the target category; or, the material category whose average sales volume in each historical period is greater than the second threshold is determined as the target category; or, the material category whose sales volume in at least one historical period is greater than the third threshold is determined as the target category.
[0137] In a possible implementation manner, the data processing module is further used for:
[0138] Based on the updated order data, determine the number of users of each material category in each historical period; based on the sales volume and number of users of each target category in each historical period, generate a sample data, the sample data includes: the sales volume and number of users of the target category in the historical period, and the week and month corresponding to the historical period; the sample data of each target category in each historical period constitutes the training data of the target category.
[0139] In a possible implementation, in the sample data, the week and month corresponding to the historical period are encoded in a one-hot encoding format.
[0140] In a possible implementation, Fig. 9 As shown, the data processing device 30 further includes: a prediction module 304 .
[0141] The prediction module 304 is used to: predict the demand for materials of the target category in the target area according to the prediction model; and allocate materials to the target area according to the demand for materials of the target category in the target area.
[0142] In a possible implementation, the neural network model includes: a representation layer and a time series prediction layer, and the time series prediction layer includes two layers of long short-term memory neural networks.
[0143] In a possible implementation, the representation layer is used to reduce the dimension of discrete features in each input sample data and then fuse them with time series features, and the obtained feature data is used as the input of the time series prediction layer.
[0144] In a possible implementation, the time series prediction layer is used to predict the demand for materials of a target category in a target area based on input feature data.
[0145] In a possible implementation, the model training module is further used to:
[0146] When training a neural network model, a hyperparameter combination is automatically selected for model training based on a predefined hyperparameter search space and a preset search strategy.
[0147] The device provided in the embodiment of the present invention can be specifically used to execute the method embodiment provided in the above-mentioned embodiment 2, and the specific functions will not be repeated here.
[0148] In the embodiment of the present invention, the training data incorporates at least the following two types of data: the first type is time series data, including the sales volume of the target category in each historical period and the number of users who purchased such target materials; the second type is factors that affect the model. The prediction model characterizes and fuses the features of the two forms of training data, and finally performs time series prediction, outputs the prediction error of the model and the demand forecast for the future period, and based on multi-source data fusion, can prevent the influence of individual extreme behaviors and improve the stability and accuracy of the prediction model.
[0149] Fig.10 This is a schematic diagram of the structure of a data processing device provided in Embodiment 5 of the present invention. Fig.10 As shown, the device 100 includes: a processor 1001 , a memory 1002 , and a computer program stored in the memory 1002 and executable on the processor 1001 .
[0150] The processor 1001 implements the data processing method provided by any of the above method embodiments when running the computer program.
[0151] The embodiment of the present invention defines multiple target areas, and for each target area, obtains order data within the target area; determines the target category to be predicted and the training data of each target category based on the order data within the target area; uses the training data of each target category to train a neural network model to obtain a prediction model corresponding to the target area and the target category. The prediction model can accurately predict the demand for materials of the target category in the target area, thereby providing a data basis for product recommendation, allocation, and unified distribution.
[0152] In addition, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the data processing method provided by any of the above method embodiments is implemented.
[0153] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0154] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention is indicated by the following claims.
[0155] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for data processing, characterized in that: include: According to the area range of the target area, order data in the target area is obtained; wherein the order data at least includes categories corresponding to the purchased materials; according to the order data in the target area, target categories to be predicted and training data of each target category are determined; wherein, according to the order data in the target area, the target categories to be predicted are determined, including: according to the sales volume of each material category in the order data, the material category whose sales volume meets the conditions is determined as the target category to be predicted; The neural network model is trained according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
2. The method according to claim 1, characterized in that The step of determining the target category to be predicted and the training data of each target category according to the order data in the target area includes: According to the mapping relationship between materials and material categories, the category information of the materials in the order data is updated; Based on the updated order data, determine the sales volume of each material category in each historical period; Determine at least one target category based on the sales volume of each material category in each historical period.
3. The method according to claim 2, characterized in that The updating process of the category information of the materials in the order data according to the mapping relationship between the materials and the material categories includes: Fill in the missing material categories in the order data.
4. The method according to claim 2, characterized in that: Determining at least one target category according to the sales volume of each material category in each historical period includes: The material categories whose total sales volume in each historical period is greater than the first threshold are determined as target categories; or, The material categories whose average sales volume in each historical period is greater than the second threshold are determined as target categories; or, A material category whose sales volume in at least one of the historical periods is greater than a third threshold is determined as a target category.
5. The method according to claim 2, characterized in that: The step of determining the target category to be predicted and the training data of each target category according to the order data in the target area further includes: Determine the number of users of each material category in each historical period according to the updated order data; Generate a piece of sample data according to the sales volume and the number of users of each target category in each historical period, the sample data including: the sales volume and the number of users of the target category in the historical period, and the week and month corresponding to the historical period; The sample data of each target category in each historical period constitutes the training data of the target category.
6. The method according to claim 5, characterized in that In the sample data, the week and month corresponding to the historical period are encoded in the form of one-hot encoding.
7. The method according to claim 1, characterized in that After training the neural network model according to the training data of each target category to obtain the prediction model corresponding to the target area and the target category, the method further includes: According to the prediction model, predict the demand for materials of the target category in the target area; Allocate materials to the target area according to the demand for materials of the target category in the target area.
8. The method according to any one of claims 1 to 7, characterized in that: The training data of the target category includes sample data of the target category in multiple historical time periods, and the sample data of the target category in each historical time period includes discrete features and time series features, wherein the discrete features include at least the week and month corresponding to the historical time period, and the time series features include at least the sales volume and the number of users of the target category in the historical time period.
9. The method according to claim 8, characterized in that: The neural network model includes: a representation layer and a time series prediction layer, and the time series prediction layer includes two layers of long short-term memory neural network.
10. The method according to claim 9, characterized in that: The representation layer is used to reduce the dimension of the discrete features in each input sample data and fuse them with the time series features, and the obtained feature data is used as the input of the time series prediction layer.
11. The method according to claim 10, characterized in that: The time series prediction layer is used to predict the demand for materials of the target category in the target area according to the input feature data.
12. The method according to any one of claims 1 to 7, characterized in that: When training a neural network model, a hyperparameter combination is automatically selected for model training based on a predefined hyperparameter search space and a preset search strategy.
13. A method of data processing, characterized in that: Applied to a service platform including a data collector and a processor, the service platform is used to train a prediction model corresponding to each target category in the target area based on historical order data in the target area, the method comprising: According to the area range of the target area, order data in the target area is obtained; wherein the order data at least includes categories corresponding to the purchased materials; according to the order data in the target area, target categories to be predicted and training data of each target category are determined; wherein, according to the order data in the target area, the target categories to be predicted are determined, including: according to the sales volume of each material category in the order data, the material category whose sales volume meets the conditions is determined as the target category to be predicted; The neural network model is trained according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
14. A data processing device, characterized in that: include: A data collection module, used to obtain order data in the target area according to the area range of the target area; wherein the order data at least includes the category corresponding to the purchased materials; A data processing module, used to determine the target category to be predicted and the training data of each target category according to the order data in the target area; The data processing module is specifically used to determine the material categories whose sales volume meets the conditions as the target categories to be predicted according to the sales volume of each material category in the order data; The model training module is used to train the neural network model according to the training data of each target category to obtain a prediction model corresponding to the target area and the target category, and the prediction model is used to predict the demand for materials of the target category in the target area.
15. A data processing device, characterized in that: include: A processor, a memory, and a computer program stored on the memory and executable on the processor; Wherein, when the processor runs the computer program, the method according to any one of claims 1 to 12 is implemented.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.
17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.
Citation Information
Patent Citations
Order forecasting method and device
CN106127329A
Index prediction method and apparatus, and index prediction model training method and apparatus
CN108960431A
Order allocation method and device
CN109426885A
Order prediction method and device, equipment and computer readable storage medium
CN110400024A
Order processing method and device, server and nonvolatile storage medium
CN111127154A
