Data collection method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a data collection method, device, electronic device and storage medium. The data collection method includes: constructing samples according to the historical demand characteristic data of the items in the target warehouse to obtain an original sample set; classifying the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set; obtaining a sample collection strategy set for each type of sample set obtained by classification; collecting samples from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set. The target sample set is used for model training, which can improve the model training efficiency and improve the prediction accuracy of the trained model.

Description

Technical Field

[0001] The present invention relates to computer technology, and in particular to a data acquisition method, device, electronic equipment and storage medium. Background Art

[0002] In recent years, the scale of the logistics market has continued to expand with economic growth, and the logistics network has also entered a large-scale, high-growth stage. The logistics network involves a large number of warehouses, and the forecast of warehouse item demand plays an important role in the refined operation of the warehouse.

[0003] The existing forecast of warehouse item demand is usually achieved by training a machine learning model. In the process of realizing the present invention, the inventors found that in the prior art, when training a machine learning model for predicting the warehouse item demand, it is usually based on all the historical data of the warehouse. The model training is based on all the historical data, and all historical data are treated equally. The different impacts of historical data generated at different times on the demand forecast are not considered, resulting in low training efficiency and insufficient prediction accuracy of the trained model. Summary of the invention

[0004] The embodiments of the present invention provide a data collection method, device, electronic device and storage medium. The collected data is used for model training, which can improve the model training efficiency and improve the prediction accuracy of the trained model.

[0005] In a first aspect, an embodiment of the present invention provides a data collection method, including:

[0006] Construct samples based on the historical demand characteristic data of items in the target warehouse to obtain an original sample set;

[0007] Classifying the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set;

[0008] Obtain the sample collection strategy set for each type of sample set obtained by classification;

[0009] Samples are collected from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

[0010] In a second aspect, an embodiment of the present invention provides a data acquisition device, the device comprising:

[0011] A sample construction module is used to construct samples according to the historical demand characteristic data of items in the target warehouse to obtain an original sample set;

[0012] A sample classification module, used to classify the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set;

[0013] A strategy acquisition module is used to acquire a sample collection strategy set for each type of sample set obtained by classification;

[0014] The sample collection module is used to collect samples from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a data acquisition method as described in any one of the embodiments of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data collection method as described in any one of the embodiments of the present invention.

[0017] In the embodiment of the present invention, samples can be constructed based on the historical demand feature data of the items in the target warehouse to obtain an original sample set; the samples in the original sample set are classified according to the data generation time corresponding to each sample in the original sample set; the sample collection strategy set for each type of sample set obtained by classification is obtained; samples are collected from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set. That is, in the embodiment of the present invention, the target sample set can be used for model training. The target sample set used for model training is equivalent to the sample collected from the original sample set. The target sample set is a subset of the original sample set, thereby reducing the number of samples used for model training and improving the model training efficiency; in addition, when collecting samples, the samples in the original sample set will be classified according to the data generation time corresponding to each sample. For the sample sets of different categories obtained by classification, different sample collection strategies are used to collect samples, taking into account the different effects of historical data generated at different times on demand prediction, thereby improving the prediction accuracy of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flow chart of the data collection method provided by an embodiment of the present invention.

[0020] Figure 2 It is another flowchart of the data collection method provided by an embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the effect of sample clustering provided by an embodiment of the present invention.

[0022] Figure 4 It is a structural schematic diagram of a data acquisition device provided by an embodiment of the present invention.

[0023] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0025] Figure 1 The present invention provides a flow chart of a data acquisition method according to an embodiment of the present invention. The method can be executed by a data acquisition device according to an embodiment of the present invention. The device can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a server, a computer, etc. The following embodiments will be described by taking the device integrated into an electronic device as an example. Figure 1 , the method may specifically include the following steps:

[0026] Step 101, constructing samples according to the historical demand characteristic data of items in the target warehouse to obtain an original sample set.

[0027] For example, the target warehouse can be any warehouse in the logistics network. The items in the target warehouse can include one or more types. Different types of items can be distinguished by item identification information. The item identification information can be, for example, a stock keeping unit (SKU). Specifically, the items can include, but are not limited to: electrical appliances, clothing, food, books, etc. The historical demand characteristic data of the items in the target warehouse can include all the item demand characteristic data generated by the target warehouse in the past (such as all the item demand characteristic data generated since the establishment of the warehouse), and can also include the item demand characteristic data generated by the target warehouse in a certain period of time in the past (such as the item demand characteristic data generated in the past 3 years or the past 5 years). The item demand characteristic data can be order quantity data, sales data, etc.

[0028] In a specific implementation, the demand characteristic data of the target warehouse in the past days can be sorted out from the historical demand characteristic data of the items in the target warehouse. After sorting, the demand characteristic data of the target warehouse in the past days can be preprocessed, and the preprocessing can include cleaning abnormal data, missing data, etc., and then a sample set is constructed based on the preprocessed data. Specifically, a sample can be constructed based on the demand characteristic data of the target warehouse in the past day, and the demand characteristic data of the target warehouse in the past days can be traversed to obtain a sample set. The sample set is the original sample set, and the original sample set usually includes a large number of samples. In a specific embodiment, each sample in the original sample set can include the following data: data generation time (for example, it can be in the format of year, month, and day), item identification information (such as SKU), order quantity (it can also be sales volume), etc.

[0029] Step 102: Classify the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set.

[0030] For example, the samples in the original sample set can be classified according to the different effects of the data generation time on the demand forecast. As the types and categories of items stored in the target warehouse may change over time, these changes directly affect the future demand for items in the target warehouse. Generally speaking, data with a relatively recent generation time should have a greater impact on the future demand forecast for items in the target warehouse than data with a relatively long generation time.

[0031] In a specific embodiment, the samples in the original sample set can be divided into two categories as follows: the samples in the original sample set whose corresponding data generation time belongs to a preset time set are classified into a first sample set, and the samples in the original sample set except the first sample set are classified into a second sample set.

[0032] Among them, the preset time set may include a time set consisting of special times such as statutory holidays (such as Dragon Boat Festival, Labor Day, Mid-Autumn Festival, etc.), promotion days, anniversaries (such as opening anniversary), that is, the samples consisting of data generated at special times can be classified as the first sample set, and the samples consisting of data generated at normal times can be classified as the second sample set. Both the first sample set and the second sample set can include a large number of samples. Relatively speaking, compared with the samples consisting of data generated at normal times, the samples consisting of data generated at special times have a greater impact on demand forecasting. Of course, in actual applications, other methods can be used to classify the samples in the original sample set according to the different impacts of the time when the data is generated on the demand forecast, which is not specifically limited here.

[0033] Step 103: Obtain a sample collection strategy set for each type of sample set obtained by classification.

[0034] In the specific implementation, different sample collection strategies can be set for different types of sample sets to achieve differentiated sampling and improve sampling quality. For example, a sample collection strategy based on a clustering algorithm and a sample collection strategy based on a weight decay algorithm can be set. The so-called sample collection strategy based on a clustering algorithm can be to find samples that can form clusters through clustering to filter out low-quality samples such as noise point samples and isolated point samples, thereby obtaining high-quality samples. The so-called sample collection strategy based on a weight decay algorithm can be to calculate the sampling weights of samples generated at different data generation times according to a certain decay algorithm, and collect samples according to the sampling weights; wherein the sampling weight can be understood as the probability of the sample being collected; for example, the weight decay algorithm can be designed so that the closer the data generation time is to the current time, the greater the sampling weight, and the farther the data generation time is from the current time, the smaller the sampling weight; because compared with samples with a relatively long data generation time, the samples generated recently should have a greater impact on the demand forecast, and the samples generated at different times can be treated differently by setting the weights, thereby improving the quality of the collected samples.

[0035] For example, the clustering algorithms mentioned in the embodiments of the present invention may include a density-based spatial clustering of applications with noise (DBSCAN) algorithm, a K-means (K-Means) clustering algorithm, an expectation maximum (EM) clustering algorithm, etc.; the weight decay algorithm mentioned in the embodiments of the present invention may include an exponential decay algorithm, a linear decay algorithm, etc.

[0036] Step 104: collect samples from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

[0037] That is, samples can be collected from the sample sets of the corresponding categories according to the sample collection strategy corresponding to each category of sample sets, and the collected samples can be merged to obtain the target sample set.

[0038] For example, when the original sample set is classified into a first sample set and a second sample set, and the sample collection strategy set for the first sample set is the first sample collection strategy, and the sample collection strategy set for the second sample set is the second sample collection strategy, samples can be collected from the first sample set according to the first sample collection strategy to obtain a first sample subset, and samples can be collected from the second sample set according to the second sample collection strategy to obtain a second sample subset, and the first sample subset and the second sample subset are merged to obtain a target sample set.

[0039] In the embodiment of the present invention, samples can be constructed based on the historical demand feature data of the items in the target warehouse to obtain an original sample set; the samples in the original sample set are classified according to the data generation time corresponding to each sample in the original sample set; the sample collection strategy set for each type of sample set obtained by classification is obtained; samples are collected from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set. That is, in the embodiment of the present invention, the target sample set can be used for model training. The target sample set used for model training is equivalent to the sample collected from the original sample set. The target sample set is a subset of the original sample set, thereby reducing the number of samples used for model training and improving the model training efficiency; in addition, when collecting samples, the samples in the original sample set will be classified according to the data generation time corresponding to each sample. For the sample sets of different categories obtained by classification, different sample collection strategies are used to collect samples, taking into account the different effects of historical data generated at different times on demand prediction, thereby improving the prediction accuracy of the trained model.

[0040] The data collection method provided by the embodiment of the present invention is further described below. This embodiment is described by taking the example of dividing the samples in the original sample set into two categories and setting two sample collection strategies to collect samples to construct the target sample set. Figure 2 As shown, the following steps may be specifically included:

[0041] Step 201, constructing samples according to the historical demand characteristic data of items in the target warehouse to obtain an original sample set.

[0042] Specifically, the daily demand feature data of the target warehouse in the past can be sorted out from the historical demand feature data of the items in the target warehouse. After sorting, the daily demand feature data of the target warehouse in the past can be preprocessed, and the preprocessing can include cleaning abnormal data, missing data, etc., and then a sample set is constructed based on the preprocessed data. Specifically, a sample can be constructed based on the daily demand feature data of the target warehouse in the past day, and the daily demand feature data of the target warehouse in the past day can be traversed to obtain a sample set. The sample set is the original sample set, and the original sample set usually includes a large number of samples. In a specific embodiment, each sample in the original sample set can include the following data: data generation time (for example, it can be in the format of year, month, and day), item identification information (such as SKU), order quantity (it can also be sales volume), etc.

[0043] Step 202: Classify the samples in the original sample set whose corresponding data generation time belongs to a preset time set as a first sample set, and classify the samples in the original sample set except the first sample set as a second sample set.

[0044] By way of example, the preset time set may include a time set consisting of special times such as statutory holidays (such as Dragon Boat Festival, Labor Day, Mid-Autumn Festival, etc.), promotion days, anniversaries (such as business anniversary), etc., that is, samples consisting of data generated at special times can be classified as the first sample set, and samples consisting of data generated at normal times can be classified as the second sample set.

[0045] For the first sample set, steps 203 and 204 may be performed to obtain a first sample subset; for the second sample set, steps 205, 206, and 207 may be performed to obtain a second sample subset.

[0046] Step 203: Obtain the set minimum values ​​of cluster radius and cluster density.

[0047] Specifically, each sample in the first sample set can be regarded as a point, thereby converting the sample clustering problem into a point density clustering problem. In the point density clustering, two pre-set parameters are required: the clustering radius E and the minimum clustering density MinPts. The minimum clustering density MinPts refers to the minimum number of points (number of samples) required to be included in the cluster. If the distance between two points is less than or equal to the clustering radius E, then the two points are of the same type and will be clustered into the same cluster. The clustering radius E and the minimum clustering density MinPts can be set according to actual needs or experimental data.

[0048] Step 204: cluster the first sample set based on the cluster radius and the minimum cluster density to obtain sample clusters, and use the sample clusters as the first sample subsets.

[0049] Density clustering involves the following definitions:

[0050] Ε neighborhood: The area within a radius of Ε of a given object is called the Ε neighborhood of the object;

[0051] Core object: If the number of sample points in the neighborhood of a given object Ε is greater than or equal to MinPts, then the object is called a core object;

[0052] Direct density reachability: For a sample set D, if sample point q is in the E neighborhood of p and p is a core object, then object q is directly density reachable from object p;

[0053] Density reachable: For a sample set D, given a string of sample points p1, p2…p n , p=p1,q=p n , if the object p i From p i-1 Direct density reachable, then object q is density reachable from object p;

[0054] Density-connected: Given a point o in the sample set D, if object o is density-reachable from object p and object q, then p and q are density-connected.

[0055] It can be found that density-reachability is the transitive closure of direct density-reachability, and this relationship is asymmetric, while density-connectedness is a symmetric relationship. The purpose of clustering is to find the largest set of density-connected objects. Specifically, in the embodiments of the present invention, it is to find the points that can form clusters, thereby filtering out noise points. The specific clustering method can be as follows:

[0056] Start with an unvisited point, and find all nearby points whose distance from it is within the clustering radius Ε (including E).

[0057] If the number of nearby points ≥ MinPts, the current point and its nearby points form a cluster, and the starting point is marked as visited. Then recursively, process all unmarked points within the cluster in the same way to expand the cluster.

[0058] If the number of nearby points < MinPts, the point is temporarily marked as a noise point.

[0059] If the cluster is fully expanded, that is, all points within the cluster are marked as visited, then use the same algorithm to process unvisited points until all points are assigned to a cluster or marked as noise points. The samples corresponding to the points assigned to the cluster are classified into the first sample subset. In practical applications, after clustering the points corresponding to the first sample set, multiple clusters may be formed. All clusters or some clusters can be selected according to actual needs, and the samples corresponding to the points in the selected clusters are classified into the first sample subset.

[0060] In a specific embodiment, for example, the clustering result of the points corresponding to the first sample set can be as Figure 3 shown. Figure 3 The points within the circle can be considered as the points assigned to the cluster, and the points outside the circle can be considered as noise points. Then, the samples corresponding to the points within the circle can be classified into the first sample subset.

[0061] The above-described clustering method is the clustering method based on the DBSCAN algorithm. Compared with other clustering algorithms, the DBSCAN algorithm has the following advantages: It does not need to know in advance the number of clusters to be formed, can find clusters of any shape, and can identify noise points. Therefore, in specific implementations, the clustering method based on the DBSCAN algorithm can be preferably used. Of course, in practical applications, other clustering algorithms can also be selected, which are not specifically limited here.

[0062] Step 205 : input the data generation time corresponding to each sample in the second sample set into an exponential decay model to obtain a sampling weight decay coefficient of each sample in the second sample set.

[0063] Specifically, the time distance between the data generation time and the current time corresponding to each sample in the second sample set can be calculated, and the time distance can be input into the exponential decay model to obtain the sampling weight decay coefficient of each sample in the second sample set. The larger the time distance, the larger the sampling weight decay coefficient of the corresponding sample.

[0064] For example, the exponential decay model may be as follows:

[0065] N(t)=N0e -γt

[0066] Among them, N(t) represents the sampling weight attenuation coefficient of the sample, N0 is the first preset parameter, γ is the second preset parameter, N0 and γ can be fitted using the maximum likelihood estimation algorithm, and t is the data generation time corresponding to the sample (which can be expressed by time distance).

[0067] Step 206: Determine the sampling weight of each sample in the second sample set according to the sampling weight attenuation coefficient of each sample in the second sample set.

[0068] For example, the sampling weight of each sample in the second sample set may be determined according to the default sampling weight of each sample in the second sample set and the corresponding sampling weight attenuation coefficient, and the default sampling weight of each sample in the second sample set may be the same, for example, all may be 1. For example, the default sampling weight of each sample in the second sample set may be multiplied by the corresponding sampling weight attenuation coefficient to obtain the sampling weight of each sample in the second sample set.

[0069] Step 207: Collect samples from the second sample set according to the sampling weight of each sample in the second sample set to obtain a second sample subset.

[0070] Step 208: merge the first sample subset and the second sample subset to obtain a target sample set.

[0071] For example, the target sample set can be used to train a feature-based time series prediction algorithm model, which is used to predict the future demand for items in the target warehouse. The future demand can be the order volume or sales volume of items in the target warehouse in a certain period of time in the future. The time series prediction algorithm is a regression prediction method and belongs to quantitative prediction. Its basic principle is: on the one hand, it recognizes the continuity of the development of things, uses past time series data for statistical analysis, and infers the development trend of things; on the other hand, it fully considers the randomness caused by accidental factors. In order to eliminate the impact of random fluctuations, it uses historical data for statistical analysis and appropriately processes the data to make trend predictions.

[0072] In an embodiment of the present invention, a target sample set can be used for model training. The target sample set used for model training is equivalent to samples collected from the original sample set. The target sample set is a subset of the original sample set, thereby reducing the number of samples used for model training and improving the efficiency of model training. In addition, when collecting samples, the samples in the original sample set will be classified according to the data generation time corresponding to each sample. Different sample collection strategies are used to collect samples for sample sets of different categories obtained by classification, taking into account the different impacts of historical data generated at different times on demand forecasting, thereby improving the prediction accuracy of the trained model.

[0073] It should be noted that this embodiment only takes the example of dividing the samples in the original sample set into two categories and setting two sample collection strategies to collect samples to construct the target sample set. In actual applications, the samples in the original sample set can also be divided into more categories and more sample collection strategies can be set to collect samples to construct the target sample set. No specific limitation is made here.

[0074] Figure 4 FIG. 1 is a structural diagram of a data acquisition device provided by an embodiment of the present invention, and the device is suitable for executing the data acquisition method provided by an embodiment of the present invention. Figure 4 As shown, the device may specifically include:

[0075] The sample construction module 401 is used to construct samples according to the historical demand characteristic data of the items in the target warehouse to obtain an original sample set;

[0076] A sample classification module 402, configured to classify samples in the original sample set according to the data generation time corresponding to each sample in the original sample set;

[0077] A strategy acquisition module 403 is used to acquire a sample collection strategy set for each type of sample set obtained by classification;

[0078] The sample collection module 404 is used to collect samples from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

[0079] In one embodiment, the target sample set is used to train a feature-based time series prediction algorithm model, and the time series prediction algorithm model is used to predict the future demand for items in the target warehouse.

[0080] In one embodiment, the sample classification module 402 is specifically used for:

[0081] Classify the samples in the original sample set whose corresponding data generation time belongs to a preset time set as a first sample set;

[0082] The samples in the original sample set except the first sample set are classified into a second sample set.

[0083] In one embodiment, the policy acquisition module 403 is specifically used to:

[0084] Acquire a first sample collection strategy set for the first sample set, where the first sample collection strategy is a sample collection strategy based on a density clustering algorithm;

[0085] A second sample collection strategy set for the second sample set is acquired, where the second sample collection strategy is a sample collection strategy based on an exponential decay algorithm.

[0086] In one embodiment, the sample collection module 404 is specifically used for:

[0087] Collect samples from the first sample set according to the first sample collection strategy to obtain a first sample subset;

[0088] Collect samples from the second sample set according to the second sample collection strategy to obtain a second sample subset;

[0089] The first sample subset and the second sample subset are combined to obtain the target sample set.

[0090] In one embodiment, the sample collection module 404 collects samples from the first sample set according to the first sample collection strategy to obtain a first sample subset, including:

[0091] Get the set minimum cluster radius and cluster density;

[0092] The first sample set is clustered based on the cluster radius and the minimum cluster density to obtain a sample cluster, and the sample cluster is used as the first sample subset.

[0093] In one embodiment, the sample collection module 404 collects samples from the second sample set according to the second sample collection strategy to obtain a second sample subset, including:

[0094] Inputting the data generation time corresponding to each sample in the second sample set into an exponential decay model to obtain a sampling weight decay coefficient of each sample in the second sample set;

[0095] Determine a sampling weight for each sample in the second sample set according to a sampling weight attenuation coefficient of each sample in the second sample set;

[0096] Samples are collected from the second sample set according to the sampling weight of each sample in the second sample set to obtain the second sample subset.

[0097] In one embodiment, the exponential decay model is as follows:

[0098] N(t)=N0e -γt

[0099] Wherein, N(t) represents the sampling weight attenuation coefficient of the sample, N0 is the first preset parameter, γ is the second preset parameter, and t is the data generation time corresponding to the sample.

[0100] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0101] The device of the embodiment of the present invention can construct samples according to the historical demand feature data of the items in the target warehouse to obtain the original sample set; classify the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set; obtain the sample collection strategy set for each type of sample set obtained by classification; collect samples from the sample set of the corresponding category according to the sample collection strategy to obtain the target sample set. That is, in the embodiment of the present invention, the target sample set can be used for model training. The target sample set used for model training is equivalent to the sample collected from the original sample set. The target sample set is a subset of the original sample set, thereby reducing the number of samples used for model training and improving the model training efficiency; in addition, when collecting samples, the samples in the original sample set will be classified according to the data generation time corresponding to each sample. For the sample sets of different categories obtained by classification, different sample collection strategies are used to collect samples, taking into account the different effects of historical data generated at different times on demand prediction, thereby improving the prediction accuracy of the trained model.

[0102] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data acquisition method provided in any of the above embodiments is implemented.

[0103] An embodiment of the present invention further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the data collection method provided by any of the above embodiments is implemented.

[0104] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of an electronic device suitable for implementing an embodiment of the present invention. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0105] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0106] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0107] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0108] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0109] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The modules and / or units involved in the embodiments of the present invention may be implemented in software or hardware. The modules and / or units described may also be set in a processor. For example, they may be described as: a processor includes a sample construction module, a sample classification module, a strategy acquisition module, and a sample collection module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0111] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: constructing samples according to the historical demand characteristic data of the items in the target warehouse to obtain an original sample set; classifying the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set; obtaining the sample collection strategy set for each type of sample set obtained by classification; collecting samples from the sample set of the corresponding category according to the sample collection strategy to obtain the target sample set.

[0112] According to the technical solution of the embodiment of the present invention, samples can be constructed according to the historical demand feature data of the items in the target warehouse to obtain the original sample set; the samples in the original sample set can be classified according to the data generation time corresponding to each sample in the original sample set; the sample collection strategy set for each type of sample set obtained by classification is obtained; samples are collected from the sample set of the corresponding category according to the sample collection strategy to obtain the target sample set for training the demand prediction model, and the demand prediction model is used to predict the future demand of the items in the target warehouse. That is, in the embodiment of the present invention, the target sample set used for model training is equivalent to the sample collected from the original sample set, and the target sample set is a subset of the original sample set, thereby reducing the number of samples used for model training and improving the model training efficiency; in addition, when collecting samples, the samples in the original sample set will be classified according to the data generation time corresponding to each sample, and different sample collection strategies are used to collect samples for the sample sets of different categories obtained by classification, taking into account the different effects of historical data generated at different times on demand prediction, thereby improving the prediction accuracy of the trained model.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data collection method, It is characterized in that include: Construct samples based on the historical demand characteristic data of items in the target warehouse to obtain an original sample set; Classifying the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set; Obtaining a sample collection strategy set for each type of sample set obtained by classification; wherein the sample collection strategy includes a sample collection strategy based on a clustering algorithm and a sample collection strategy based on a weight decay algorithm; Samples are collected from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

2. The data collection method according to claim 1, It is characterized in that The target sample set is used to train a feature-based time series prediction algorithm model, and the time series prediction algorithm model is used to predict the future demand for items in the target warehouse.

3. The data collection method according to claim 1, It is characterized in that The classifying the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set includes: Classify the samples in the original sample set whose corresponding data generation time belongs to a preset time set as a first sample set; The samples in the original sample set except the first sample set are classified into a second sample set.

4. The data collection method according to claim 3, It is characterized in that The obtaining of a sample collection strategy set for each type of sample set obtained by classification includes: Acquire a first sample collection strategy set for the first sample set, where the first sample collection strategy is a sample collection strategy based on a density clustering algorithm; A second sample collection strategy set for the second sample set is acquired, where the second sample collection strategy is a sample collection strategy based on an exponential decay algorithm.

5. The data collection method according to claim 4, It is characterized in that The step of collecting samples from a sample set of a corresponding category according to the sample collection strategy to obtain a target sample set includes: Collect samples from the first sample set according to the first sample collection strategy to obtain a first sample subset; Collect samples from the second sample set according to the second sample collection strategy to obtain a second sample subset; The first sample subset and the second sample subset are combined to obtain the target sample set.

6. The data collection method according to claim 5, It is characterized in that The collecting samples from the first sample set according to the first sample collection strategy to obtain a first sample subset includes: Get the set minimum cluster radius and cluster density; The first sample set is clustered based on the cluster radius and the minimum cluster density to obtain a sample cluster, and the sample cluster is used as the first sample subset.

7. The data collection method according to claim 5, It is characterized in that The collecting samples from the second sample set according to the second sample collection strategy to obtain a second sample subset includes: Inputting the data generation time corresponding to each sample in the second sample set into an exponential decay model to obtain a sampling weight decay coefficient of each sample in the second sample set; Determine a sampling weight for each sample in the second sample set according to a sampling weight attenuation coefficient of each sample in the second sample set; Samples are collected from the second sample set according to the sampling weight of each sample in the second sample set to obtain the second sample subset.

8. The data collection method according to claim 7, It is characterized in that The exponential decay model is as follows: N(t)=N 0 e -γt Among them, N(t) represents the sampling weight attenuation coefficient of the sample, N 0 is the first preset parameter, γ is the second preset parameter, and t is the data generation time corresponding to the sample.

9. A data acquisition device, It is characterized in that include: A sample construction module is used to construct samples according to the historical demand characteristic data of items in the target warehouse to obtain an original sample set; A sample classification module, used to classify the samples in the original sample set according to the data generation time corresponding to each sample in the original sample set; A strategy acquisition module, used to acquire a sample collection strategy set for each type of sample set obtained by classification; wherein the sample collection strategy includes a sample collection strategy based on a clustering algorithm and a sample collection strategy based on a weight decay algorithm; The sample collection module is used to collect samples from the sample set of the corresponding category according to the sample collection strategy to obtain a target sample set.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the data collection method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the data collection method according to any one of claims 1 to 8 is implemented.

Images