Item replenishment method, device, computer equipment and storage medium

Abstract

The embodiment of the present application belongs to the field of artificial intelligence, and relates to a method for replenishing items, including obtaining the inventory quantity, demand quantity, and order quantity of historical items at different stages, and calculating the historical state variables according to the inventory quantity, demand quantity, and order quantity; obtaining the reward function and the target constraint capacity, and constructing a basic deep neural network according to the target constraint capacity and the reward function; training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining the target deep neural network; obtaining the current state variable of the target item, and calculating the predicted reward value of the target item under different replenishment quantities according to the target deep neural network; selecting the replenishment quantity corresponding to the predicted reward value with the maximum value as the target replenishment quantity. The present application also provides an item replenishment device, a computer device, and a storage medium. The present application realizes the accurate prediction of the replenishment quantity of items.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an item replenishment method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of the offline retail market, major offline retail stores usually take a series of measures to attract customers to the store, including promotions, fresh food, etc., to increase the flow of customers in the store through these traffic diversion strategies. However, most of the goods in the store are mainly for room temperature demand goods. These goods will be affected by promotions, other brands, the amount of goods placed, natural climate, etc., but for most goods, without the influence of promotions and natural weather, the main influencing factor is still the amount of goods placed on the shelves.

[0003] At present, in practical applications, there are two main ways to determine item replenishment strategies: one is to set a display safety stock and add the display safety stock to the replenishment strategy; the other is to use market population experiments to determine the relationship between group purchasing motivation and display quantity to provide a reference for subsequent replenishment strategies. The above two replenishment strategies often have the problem of too much or too little item replenishment. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, computer device and storage medium for replenishing items to solve the technical problem that the quantity of item replenishment is not accurate enough.

[0005] In order to solve the above technical problems, the present application embodiment provides a method for replenishing goods, which adopts the following technical solution:

[0006] Obtaining the inventory quantity, demand quantity and order quantity of the historical item at different stages, and calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity;

[0007] Obtaining a preset reward function and target constraint capacity, and constructing a basic deep neural network according to the target constraint capacity and the reward function;

[0008] Training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining a target deep neural network;

[0009] Obtaining a current state variable of a target item, inputting the current state variable into the target deep neural network, and calculating a predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0010] The replenishment quantity corresponding to the predicted reward value with the maximum value is selected as the target replenishment quantity of the target item.

[0011] Furthermore, the step of obtaining a preset reward function includes:

[0012] Obtaining the minimum display quantity, out-of-stock rate index, display relative distance, and display proportion coefficient of the historical item;

[0013] The reward function is calculated according to the minimum display quantity, the out-of-stock rate index, the display relative distance, the display proportion coefficient and the inventory quantity.

[0014] Furthermore, the step of obtaining the minimum display quantity of the historical items includes:

[0015] Calculating the difference in the placement amount of two adjacent groups of items in the historical items;

[0016] When the placement quantity difference is less than or equal to a preset placement threshold, the sub-placement quantity corresponding to the placement quantity difference is determined to be the minimum display quantity.

[0017] Furthermore, the step of calculating the difference in the placement amount between two adjacent groups of items in the historical items includes:

[0018] Obtaining the daily placement amount of the historical objects on different dates, calculating the placement significance value of the historical objects according to the daily placement amount, and determining whether the placement significance value is less than a preset significance value;

[0019] When the placement significance value is less than the preset significance value, the sub-placement quantities of two adjacent groups of items of the historical items on the same shelf are obtained, and the placement quantity difference is calculated according to the sub-placement quantities.

[0020] Furthermore, the step of calculating the placement significance value of the historical items according to the daily placement quantity includes:

[0021] Obtaining a preset maximum capacity value and a preset time interval of the historical item;

[0022] Calculating the placement control number of the historical items according to the maximum capacity value and the preset time interval;

[0023] Randomly generating a placement quantity sequence based on the placement control number, and determining the daily placement quantity of the historical items according to the placement quantity sequence;

[0024] A one-way analysis of variance is performed on the daily placement quantity to obtain the placement significance value of the historical items.

[0025] Furthermore, the step of constructing a basic deep neural network according to the target constraint capacity and the reward function includes:

[0026] Obtaining the observation value and the stage number of the historical item, and calculating the number of nodes of the basic deep neural network according to the observation value and the stage number;

[0027] According to the target constraint capacity and the number of nodes, the hidden layer parameters of the basic deep neural network are determined, and the basic deep neural network is constructed based on the hidden layer parameters.

[0028] Furthermore, the step of calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity includes:

[0029] Determine the initial state variables of the historical items at different stages according to the inventory quantity, the demand quantity and the order quantity;

[0030] The observation period number of the historical item is obtained, and the historical state variable is calculated according to the initial state variable and the observation period number.

[0031] In order to solve the above technical problems, the embodiment of the present application further provides an item replenishment device, which adopts the following technical solution:

[0032] An acquisition module, used to acquire the inventory quantity, demand quantity and order quantity of historical items at different stages, and calculate the historical state variable of the historical items according to the inventory quantity, demand quantity and order quantity;

[0033] A construction module, used to obtain a preset reward function and a target constraint capacity, and construct a basic deep neural network according to the target constraint capacity and the reward function;

[0034] A training module, used to train the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determine that the training of the basic deep neural network is completed, and obtain a target deep neural network;

[0035] A prediction module, used to obtain a current state variable of a target item, input the current state variable into the target deep neural network, and calculate a predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0036] A selection module is used to select the replenishment quantity corresponding to the maximum predicted reward value as the target replenishment quantity of the target item.

[0037] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0038] Obtaining the inventory quantity, demand quantity and order quantity of the historical item at different stages, and calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity;

[0039] Obtaining a preset reward function and target constraint capacity, and constructing a basic deep neural network according to the target constraint capacity and the reward function;

[0040] Training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining a target deep neural network;

[0041] Obtaining a current state variable of a target item, inputting the current state variable into the target deep neural network, and calculating a predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0042] The replenishment quantity corresponding to the predicted reward value with the maximum value is selected as the target replenishment quantity of the target item.

[0043] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0044] Obtaining the inventory quantity, demand quantity and order quantity of the historical item at different stages, and calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity;

[0045] Obtaining a preset reward function and target constraint capacity, and constructing a basic deep neural network according to the target constraint capacity and the reward function;

[0046] Training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining a target deep neural network;

[0047] Obtaining a current state variable of a target item, inputting the current state variable into the target deep neural network, and calculating a predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0048] The replenishment quantity corresponding to the predicted reward value with the maximum value is selected as the target replenishment quantity of the target item.

[0049] The present application obtains the inventory quantity, demand quantity and order quantity of historical items at different stages, and calculates the historical state variables according to the inventory quantity, demand quantity and order quantity; then, obtains the reward function and the target constraint capacity, and constructs a basic deep neural network according to the target constraint capacity and the reward function; then, the basic deep neural network is trained based on the historical state variables, and when the loss function of the basic deep neural network converges, it is determined that the training of the basic deep neural network is completed, and the target deep neural network is obtained, and the replenishment quantity of the item can be accurately and efficiently obtained according to the target deep neural network; finally, the current state variable of the target item is obtained, and the predicted reward value of the target item under different replenishment quantities is calculated according to the target deep neural network; the replenishment quantity corresponding to the predicted reward value with the maximum value is selected as the target replenishment quantity of the target item, thereby realizing the accurate prediction of the replenishment quantity of the target item, improving the replenishment efficiency of the target item, and further enabling the target item to be efficiently and accurately replenished according to the replenishment quantity, thereby avoiding the waste of item resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0052] Figure 2 A flowchart of an embodiment of a method for replenishing items according to the present application;

[0053] Figure 3 is a structural schematic diagram of an embodiment of an item replenishment device according to the present application;

[0054] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application.

[0055] Reference numerals: item replenishment device 300 , acquisition module 301 , construction module 302 , training module 303 , prediction module 304 , and selection module 305 . DETAILED DESCRIPTION

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0057] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0060] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0061] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0062] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0063] It should be noted that the item replenishment method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the item replenishment device is generally set in the server / terminal device.

[0064] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0065] Continue to refer Figure 2 , showing a flow chart of an embodiment of a method for replenishing items according to the present application. The method for replenishing items comprises the following steps:

[0066] Step S201, obtaining the inventory quantity, demand quantity and order quantity of historical items at different stages, and calculating the historical state variable of the historical items according to the inventory quantity, demand quantity and order quantity;

[0067] In this embodiment, the historical item is a selected reference item, and the inventory quantity, demand quantity, and order quantity of the historical item in a preset period are obtained. Specifically, the preset period is divided into different stages, and the inventory quantity, demand quantity, and order quantity of the historical item in different stages are obtained respectively. Among them, the inventory quantity is the inventory level of the historical item at the beginning of different stages; the demand quantity corresponding to the inventory quantity of the historical item in the same period is the demand quantity corresponding to the previous stage during this stage; the order quantity of the historical item in the same period is the order quantity to be received during this stage. The inventory quantity, demand quantity, and order quantity of multiple groups of historical items at different stages are obtained, and the historical state variables of the historical items at different stages are calculated according to the inventory quantity, demand quantity, and order quantity. The historical state variables are the state variables corresponding to the historical items at different stages.

[0068] Step S202, obtaining a preset reward function and target constraint capacity, and constructing a basic deep neural network according to the target constraint capacity and the reward function;

[0069] In this embodiment, the reward function is a preset incentive function, and the target constraint capacity is the preset constraint capacity of the historical item, and the target constraint capacity is within the range of the set upper and lower constraints. Among them, the upper constraint is the maximum shelf capacity of the historical item, and the lower constraint is the minimum order quantity provided by the supplier corresponding to the historical item. When the preset reward function and target constraint capacity are obtained, the reward function is used as the reward function of the basic deep neural network, and the target constraint capacity is used as the spatial constraint of the basic deep neural network, and the action space under any state is limited to the range of the target constraint capacity. The basic deep neural network is DQN (Deep Q Network), and the reward function is used as the reward after the basic deep neural network simulates the execution of the replenishment action, and the target constraint capacity constrains the action of replenishing items; the basic architecture of the basic deep neural network is obtained according to the reward function and the target constraint capacity.

[0070] Step S203, training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining a target deep neural network;

[0071] In this embodiment, when the basic deep neural network is obtained, the basic deep neural network is trained according to the historical state variables of historical objects at different stages. Specifically, the historical state variables of different stages are input into the basic deep neural network, and the historical state variables of each stage are calculated according to the basic deep neural network to obtain the target state variables. Afterwards, the loss function of the basic deep neural network is calculated according to the obtained historical state variables and the target state variables; the network parameters of the basic deep neural network are adjusted by the loss function until the loss function calculated according to the basic deep neural network after the network parameters are adjusted converges, and it is determined that the training of the basic deep neural network is completed, and the target deep neural network is obtained.

[0072] Step S204, obtaining the current state variable of the target item, inputting the current state variable into the target deep neural network, and calculating the predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0073] In this embodiment, the target item is an item that currently needs to be estimated for replenishment, and the current state variable is the state variable of the target item at the current stage. Specifically, the inventory quantity of the target item at the beginning of the current stage, the demand quantity of the target item in the current stage corresponding to the previous stage, and the number of orders that the target item will receive in the current stage are obtained; the current state variable of the target item is calculated based on the inventory quantity of the target item, the demand quantity of the target item, and the order quantity of the target item. When the current state variable is calculated, the current state variable is input into the target deep neural network; the current state variable is calculated based on the target deep neural network to obtain the predicted reward value of the target item under different replenishment quantities, and the predicted reward value is the Q value of the target item.

[0074] Step S205 , selecting the replenishment quantity corresponding to the maximum predicted reward value as the target replenishment quantity of the target item.

[0075] In this embodiment, when the predicted reward values ​​of the target item under different replenishment quantities are calculated, the replenishment quantities are sorted from high to low according to the predicted reward values, and the replenishment quantity corresponding to the maximum predicted reward value is selected as the target replenishment quantity of the target item.

[0076] This embodiment achieves accurate prediction of the replenishment quantity of the target item, improves the replenishment efficiency of the target item, and further enables the target item to be replenished efficiently and accurately according to the replenishment quantity, thereby avoiding waste of item resources.

[0077] In some optional implementations of this embodiment, the step of obtaining a preset reward function includes:

[0078] Obtaining the minimum display quantity, out-of-stock rate index, display relative distance, and display proportion coefficient of the historical item;

[0079] The reward function is calculated according to the minimum display quantity, the out-of-stock rate index, the display relative distance, the display proportion coefficient and the inventory quantity.

[0080] In this embodiment, the minimum display quantity is the minimum display quantity of historical items; the out-of-stock rate index is the service level of historical items in each stage, for example, the proportion of non-zero inventory of historical items in the number of days included in stage i; the display relative distance is the relative display distance between items, which is calculated based on the inventory and the minimum display quantity of the historical items; the display proportion coefficient is a preset proportion coefficient. The minimum display quantity, out-of-stock rate index, display relative distance and display proportion coefficient of historical items are obtained, and the reward function is calculated based on the minimum display quantity, out-of-stock rate index, display relative distance, display proportion coefficient and inventory. The calculation formula of the reward function is as follows:

[0081]

[0082] Among them, SL i is the out-of-stock rate indicator, α is the display ratio coefficient, BAIL i is the inventory quantity, MIND is the minimum display quantity, ed i To display the relative distance.

[0083] This embodiment determines the reward function through parameters such as the minimum display quantity of items and the out-of-stock rate index, so that the item replenishment quantity obtained through the reward function can better meet the actual demand, avoid the risk of item inventory backlog due to improving service levels, improve the accuracy of replenishment quantity acquisition, and avoid waste of resources.

[0084] In some optional implementations of this embodiment, the step of obtaining the minimum display quantity of the historical objects includes:

[0085] Calculating the difference in the placement amount of two adjacent groups of items in the historical items;

[0086] When the placement quantity difference is less than or equal to a preset placement threshold, the sub-placement quantity corresponding to the placement quantity difference is determined to be the minimum display quantity.

[0087] In this embodiment, when calculating the minimum display quantity of historical items, the minimum display quantity can be calculated by the difference in the placement quantity of the historical items. Specifically, the placement quantity difference is the difference in the sub-placement quantities of two adjacent groups of items on the same shelf. If, as the sub-placement quantities of the two groups of items increase, the placement quantity difference between the two groups of items is less than or equal to a preset placement threshold, then the sub-placement quantity corresponding to the placement quantity difference is determined to be the minimum display quantity; if, as the sub-placement quantities of the two groups of items increase, the placement quantity difference between the two groups of items is still greater than the preset placement threshold, then the sub-placement quantities of the two groups of items are adjusted until the placement quantity difference between the two groups of items is less than or equal to the preset placement threshold.

[0088] This embodiment determines the minimum display quantity of items by the placement quantity difference, so that the accuracy of the reward function calculation is further improved through the minimum display quantity, avoiding the problem that the final calculated item replenishment quantity is too high or too low and does not conform to the actual situation.

[0089] In some optional implementations of this embodiment, the above-mentioned calculating the difference in the placement amount of two adjacent groups of items in the historical items includes:

[0090] Obtaining the daily placement amount of the historical objects on different dates, calculating the placement significance value of the historical objects according to the daily placement amount, and determining whether the placement significance value is less than a preset significance value;

[0091] When the placement significance value is less than the preset significance value, the sub-placement quantities of two adjacent groups of items of the historical items on the same shelf are obtained, and the placement quantity difference is calculated according to the sub-placement quantities.

[0092] In this embodiment, when calculating the placement volume difference, it is necessary to determine whether the placement volume of the historical item affects the demand. If it is determined that the placement volume of the historical item will not affect the demand, there is no need to obtain the placement volume difference, that is, there is no need to obtain the minimum display volume of the historical item; if it is determined that the placement volume of the historical item will affect the demand, then the placement volume difference of the historical item is obtained. Specifically, the daily placement volume of the historical item on different dates is obtained, and the placement significance value of the historical item is calculated based on the daily placement volume. If the placement significance value is greater than or equal to the preset significance value (such as 0.05), it is determined that the placement volume of the historical item will not affect the demand; if the placement significance value is less than the preset significance value, it is determined that the placement volume of the historical item will affect the demand. At this time, the sub-placement volume of two adjacent groups of items on the same shelf of the historical item is obtained, and the placement volume difference is calculated based on the sub-placement volume. The difference between the sub-placement volumes of the two adjacent groups of items is the placement volume difference of the historical item.

[0093] This embodiment determines whether the placement quantity will affect the demand for the item, thereby achieving accurate determination of the minimum placement quantity of historical items, and further improving the calculation efficiency and accuracy of the item replenishment quantity.

[0094] In some optional implementations of this embodiment, the above-mentioned calculating the placement significance value of the historical object according to the daily placement quantity includes:

[0095] Obtaining a preset maximum capacity value and a preset time interval of the historical item;

[0096] Calculating the placement control number of the historical items according to the maximum capacity value and the preset time interval;

[0097] Randomly generating a placement quantity sequence based on the placement control number, and determining the daily placement quantity of the historical items according to the placement quantity sequence;

[0098] A one-way analysis of variance is performed on the daily placement quantity to obtain the placement significance value of the historical items.

[0099] In this embodiment, the maximum capacity value is the maximum value of the capacity of the historical item on the shelf. The maximum capacity value and the preset time interval of the historical item are obtained, and the placement quantity control number is calculated based on the maximum capacity value and the preset time interval. Specifically, the placement quantity control number is the number of placement quantity control levels corresponding to the historical item, and the placement quantity control number can be obtained by calculating the ratio of the maximum capacity value and the time interval; then, at the beginning of each stage of the historical item, a placement quantity sequence composed of the placement quantity control levels of the placement quantity control number is randomly generated, and the daily placement quantity of the historical item on different dates is determined based on the placement quantity sequence. Then, a single-factor variance analysis is performed on the placement quantity change data of the historical items on different dates generated based on the daily placement quantity through statistical tools such as SPSS (Statistical Product and Service Solutions) to obtain the placement significance value of the historical item.

[0100] This embodiment finally obtains the placement significance value of the historical items through the maximum capacity value and the preset time interval, thereby achieving accurate calculation of the placement significance value of the historical items and further improving the accuracy of calculating the item replenishment quantity.

[0101] In some optional implementations of this embodiment, constructing a basic deep neural network according to the target constraint capacity and the reward function includes:

[0102] Obtaining the observation value and the stage number of the historical item, and calculating the number of nodes of the basic deep neural network according to the observation value and the stage number;

[0103] According to the target constraint capacity and the number of nodes, the hidden layer parameters of the basic deep neural network are determined, and the basic deep neural network is constructed based on the hidden layer parameters.

[0104] In this embodiment, the observation value is the dimension contained in the historical state variable corresponding to the historical item at the beginning of each stage, and the number of stages is the number of stages obtained by dividing the historical item according to the preset time period. The observation value and the number of stages of the historical item are obtained, and the product of the observation value and the number of stages is calculated to obtain the number of nodes of the basic deep neural network. Since the output of the basic deep neural network is the Q value corresponding to all actions of the historical item in a certain state, the basic deep neural network is constrained by the upper and lower limits according to the target constraint capacity, and it can be determined that the action space of the historical item in any state is within the range of the target constraint capacity. According to the target constraint capacity and the number of nodes, the hidden layer parameters of the basic deep neural network are determined, and the basic deep neural network is constructed based on the hidden layer parameters.

[0105] For example, the historical state variables corresponding to the historical item at the beginning of each stage include variables of three dimensions: inventory quantity, demand quantity, and order quantity. The observed value of the historical item is 3; the number of stages obtained by dividing the historical item according to the preset time period is m, and the number of nodes of the basic deep neural network calculated based on the observed value and the number of stages is 3m. At the same time, the upper limit constraint of the target constraint capacity of the historical item is a u , the lower limit constraint is a l , the target constraint capacity is a∈[a l ,a u ]∪{0}and a∈Z, then the action space of the historical item in any state is [a l ,a u ]∪{0}. l = 0, the number of output nodes of the basic deep neural network is a u -a l +1; in a l ≠0, the number of output nodes of the basic deep neural network is a u -a l +2. The final construction is that the hidden layer includes 3 layers, and the network structure is 3m, 6m+1, 3m+1, 2m, a u -a l +1 or a u -a l +2 for basic deep neural networks.

[0106] This embodiment constructs a basic deep neural network through the target constraint capacity, observation value and number of stages of historical items, so that the replenishment quantity of items can be accurately calculated through the basic deep neural network, and the calculation efficiency of the replenishment quantity of items is further improved.

[0107] In some optional implementations of this embodiment, the above-mentioned calculation of the historical state variable of the historical item based on the inventory quantity, the demand quantity and the order quantity includes:

[0108] Determine the initial state variables of the historical items at different stages according to the inventory quantity, the demand quantity and the order quantity;

[0109] The observation period number of the historical item is obtained, and the historical state variable is calculated according to the initial state variable and the observation period number.

[0110] In this embodiment, the replenishment quantity of the item will not only affect the current state variable, but also the state variable of the later period. Therefore, when the inventory quantity, demand quantity and order quantity of the historical item at different stages are obtained, the initial state variable of the historical item at the beginning of each stage is determined according to the inventory quantity, demand quantity and order quantity. Specifically, the calculation formula of the initial state variable is as follows:

[0111] o i ={IL i , LD i , A.S. i}

[0112] Among them, i is the current stage number, o i is the initial state variable, IL i is the inventory quantity, LD i is the demand quantity, AS i is the order quantity.

[0113] When the initial state variable is obtained, the preset observation period number of the historical item is obtained, and the observation period number is the preset total observation period number corresponding to the historical item observation value. The historical state variable of the historical item is determined according to the observation period number and the initial state variable. The calculation formula of the historical state variable is as follows:

[0114]

[0115] Among them, m is the number of observation periods, s i is the historical state variable.

[0116] This embodiment achieves accurate determination of the item status by calculating the historical status variables, further improving the accuracy of calculating the item replenishment quantity.

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0118] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0119] Further references Figure 3 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of an item replenishment device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0120] like Figure 3 As shown, the item replenishment device 300 described in this embodiment includes: an acquisition module 301, a construction module 302, a training module 303, a prediction module 304 and a selection module 305. Among them:

[0121] An acquisition module 301 is used to acquire the inventory quantity, demand quantity and order quantity of historical items at different stages, and calculate the historical state variable of the historical items according to the inventory quantity, demand quantity and order quantity;

[0122] In some optional implementations of this embodiment, the acquisition module 301 includes:

[0123] A first confirmation unit, used to determine the initial state variables of the historical item at different stages according to the inventory quantity, the demand quantity and the order quantity;

[0124] The first calculation unit is used to obtain the observation period number of the historical item, and calculate the historical state variable according to the initial state variable and the observation period number.

[0125] In this embodiment, the historical item is a selected reference item, and the inventory quantity, demand quantity, and order quantity of the historical item in a preset period are obtained. Specifically, the preset period is divided into different stages, and the inventory quantity, demand quantity, and order quantity of the historical item in different stages are obtained respectively. Among them, the inventory quantity is the inventory level of the historical item at the beginning of different stages; the demand quantity corresponding to the inventory quantity of the historical item in the same period is the demand quantity corresponding to the previous stage during this stage; the order quantity of the historical item in the same period is the order quantity to be received during this stage. The inventory quantity, demand quantity, and order quantity of multiple groups of historical items at different stages are obtained, and the historical state variables of the historical items at different stages are calculated according to the inventory quantity, demand quantity, and order quantity. The historical state variables are the state variables corresponding to the historical items at different stages.

[0126] A construction module 302 is used to obtain a preset reward function and a target constraint capacity, and construct a basic deep neural network according to the target constraint capacity and the reward function;

[0127] In some optional implementations of this embodiment, the building module 302 includes:

[0128] An acquisition unit, used to acquire the minimum display quantity, out-of-stock rate index, display relative distance and display proportion coefficient of the historical item;

[0129] The second calculation unit is used to calculate the reward function according to the minimum display quantity, the out-of-stock rate index, the display relative distance, the display proportion coefficient and the inventory quantity.

[0130] In some optional implementations of this embodiment, the acquisition unit includes:

[0131] A first calculation subunit is used to calculate the placement quantity difference between two adjacent groups of items in the historical items;

[0132] The first confirmation subunit is used to determine that the sub-placement quantity corresponding to the placement quantity difference is the minimum display quantity when the placement quantity difference is less than or equal to a preset placement threshold.

[0133] In some optional implementations of this embodiment, the first computing subunit includes:

[0134] A second confirmation subunit is used to obtain the daily placement volume of the historical objects on different dates, calculate the placement significance value of the historical objects according to the daily placement volume, and determine whether the placement significance value is less than a preset significance value;

[0135] The first acquisition subunit is used to obtain the sub-placement quantities of two adjacent groups of items of the historical items on the same shelf when the placement significance value is less than the preset significance value, and calculate the placement quantity difference according to the sub-placement quantities.

[0136] In some optional implementations of this embodiment, the second confirmation subunit includes:

[0137] A second acquisition subunit is used to acquire a preset maximum capacity value and a preset time interval of the historical item;

[0138] A second calculation subunit, configured to calculate the placement control number of the historical items according to the maximum capacity value and the preset time interval;

[0139] A third confirmation subunit is used to randomly generate a placement quantity sequence based on the placement control number, and determine the daily placement quantity of the historical items according to the placement quantity sequence;

[0140] The analysis subunit is used to perform a one-way variance analysis on the daily placement quantity to obtain the placement significance value of the historical items.

[0141] In this embodiment, the reward function is a preset incentive function, and the target constraint capacity is the preset constraint capacity of the historical item, and the target constraint capacity is within the range of the set upper and lower constraints. Among them, the upper constraint is the maximum shelf capacity of the historical item, and the lower constraint is the minimum order quantity provided by the supplier corresponding to the historical item. When the preset reward function and target constraint capacity are obtained, the reward function is used as the reward function of the basic deep neural network, and the target constraint capacity is used as the spatial constraint of the basic deep neural network, and the action space under any state is limited to the range of the target constraint capacity. The basic deep neural network is DNQ (Deep Q Network), and the reward function is used as the reward after the basic deep neural network simulates the execution of the replenishment action, and the target constraint capacity constrains the action of replenishing items; the basic architecture of the basic deep neural network is obtained according to the reward function and the target constraint capacity.

[0142] A training module 303 is used to train the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determine that the training of the basic deep neural network is completed, and obtain a target deep neural network;

[0143] In this embodiment, when the basic deep neural network is obtained, the basic deep neural network is trained according to the historical state variables of historical objects at different stages. Specifically, the historical state variables of different stages are input into the basic deep neural network, and the historical state variables of each stage are calculated according to the basic deep neural network to obtain the target state variables. Afterwards, the loss function of the basic deep neural network is calculated according to the obtained historical state variables and the target state variables; the network parameters of the basic deep neural network are adjusted by the loss function until the loss function calculated according to the basic deep neural network after the network parameters are adjusted converges, and it is determined that the training of the basic deep neural network is completed, and the target deep neural network is obtained.

[0144] A prediction module 304 is used to obtain a current state variable of a target item, input the current state variable into the target deep neural network, and calculate the predicted reward value of the target item under different replenishment quantities according to the target deep neural network;

[0145] In this embodiment, the target item is an item that currently needs to be estimated for replenishment, and the current state variable is the state variable of the target item at the current stage. Specifically, the inventory quantity of the target item at the beginning of the current stage, the demand quantity of the target item in the current stage corresponding to the previous stage, and the number of orders that the target item will receive in the current stage are obtained; the current state variable of the target item is calculated based on the inventory quantity of the target item, the demand quantity of the target item, and the order quantity of the target item. When the current state variable is calculated, the current state variable is input into the target deep neural network; the current state variable is calculated based on the target deep neural network to obtain the predicted reward value of the target item under different replenishment quantities, and the predicted reward value is the Q value of the target item.

[0146] The selection module 305 is used to select the replenishment quantity corresponding to the maximum predicted reward value as the target replenishment quantity of the target item.

[0147] In this embodiment, when the predicted reward values ​​of the target item under different replenishment quantities are calculated, the replenishment quantities are sorted from high to low according to the predicted reward values, and the replenishment quantity corresponding to the maximum predicted reward value is selected as the target replenishment quantity of the target item.

[0148] In some optional implementations of this embodiment, the building module 302 further includes:

[0149] A third calculation unit is used to obtain the observation value and the stage number of the historical item, and calculate the number of nodes of the basic deep neural network according to the observation value and the stage number;

[0150] The second confirmation unit is used to determine the hidden layer parameters of the basic deep neural network according to the target constraint capacity and the number of nodes, and construct the basic deep neural network based on the hidden layer parameters.

[0151] In this embodiment, the observation value is the dimension contained in the historical state variable corresponding to the historical item at the beginning of each stage, and the number of stages is the number of stages obtained by dividing the historical item according to the preset time period. The observation value and the number of stages of the historical item are obtained, and the product of the observation value and the number of stages is calculated to obtain the number of nodes of the basic deep neural network. Since the output of the basic deep neural network is the Q value corresponding to all actions of the historical item in a certain state, the basic deep neural network is constrained by the upper and lower limits according to the target constraint capacity, and it can be determined that the action space of the historical item in any state is within the range of the target constraint capacity. According to the target constraint capacity and the number of nodes, the hidden layer parameters of the basic deep neural network are determined, and the basic deep neural network is constructed based on the hidden layer parameters.

[0152] For example, the historical state variables corresponding to the historical item at the beginning of each stage include variables of three dimensions: inventory quantity, demand quantity, and order quantity. The observed value of the historical item is 3; the number of stages obtained by dividing the historical item according to the preset time period is m, and the number of nodes of the basic deep neural network calculated based on the observed value and the number of stages is 3m. At the same time, the upper limit constraint of the target constraint capacity of the historical item is a u , the lower limit constraint is a l , the target constraint capacity is a∈[a l ,a u ]∪{0}and a∈Z, then the action space of the historical item in any state is [a l ,a u ]∪{0}. l = 0, the number of output nodes of the basic deep neural network is a u -a l +1; in a l ≠0, the number of output nodes of the basic deep neural network is a u -a l +2. The final construction is that the hidden layer includes 3 layers, and the network structure is 3m, 6m+1, 3m+1, 2m, a u -a l +1 or a u -a l +2 for basic deep neural networks.

[0153] The item replenishment device proposed in this embodiment realizes accurate prediction of the replenishment quantity of the target item, improves the replenishment efficiency of the target item, and further enables the target item to be replenished efficiently and accurately according to the replenishment quantity, thereby avoiding waste of item resources.

[0154] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0155] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0156] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0157] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 6. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the item replenishment method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0158] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions or process data stored in the memory 61, such as computer-readable instructions for running the item replenishment method.

[0159] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0160] The computer device proposed in this embodiment realizes accurate prediction of the replenishment quantity of the target item, improves the replenishment efficiency of the target item, and further enables the target item to be replenished efficiently and accurately according to the replenishment quantity, thereby avoiding waste of item resources.

[0161] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the item replenishment method as described above.

[0162] The computer-readable storage medium proposed in this embodiment realizes accurate prediction of the replenishment quantity of the target item, improves the replenishment efficiency of the target item, and further enables the target item to be replenished efficiently and accurately according to the replenishment quantity, thereby avoiding waste of item resources.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0164] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for replenishing goods, characterized in that: The steps include: Obtaining the inventory quantity, demand quantity and order quantity of the historical item at different stages, and calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity; Obtain a preset reward function and target constraint capacity, and construct a basic deep neural network DQN according to the target constraint capacity and the reward function, wherein the target constraint capacity is a preset constraint capacity of the historical item, and the target constraint capacity is within the range of a set upper limit constraint and a set lower limit constraint, wherein the upper limit constraint is the maximum shelf capacity of the historical item, and the lower limit constraint is the minimum order quantity provided by the supplier corresponding to the historical item; Training the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determining that the training of the basic deep neural network is completed, and obtaining a target deep neural network; Acquire the current state variable of the target item, input the current state variable into the target deep neural network, and calculate the predicted reward value of the target item under different replenishment quantities according to the target deep neural network, wherein the current state variable is the state variable of the target item at the current stage, and the current state variable of the target item is calculated by acquiring the inventory quantity of the target item at the beginning of the current stage, the demand quantity of the target item at the current stage corresponding to the previous stage, and the number of orders to be received by the target item at the current stage, and according to the inventory quantity of the target item at the beginning of the current stage, the demand quantity of the target item at the current stage corresponding to the previous stage, and the number of orders to be received by the target item at the current stage; The replenishment quantity corresponding to the predicted reward value with the maximum value is selected as the target replenishment quantity of the target item; The step of obtaining a preset reward function comprises: Obtaining the minimum display quantity, out-of-stock rate index, display relative distance, and display proportion coefficient of the historical item; The reward function is calculated according to the minimum display quantity, the out-of-stock rate index, the display relative distance, the display proportion coefficient and the inventory quantity; The step of obtaining the minimum display quantity of the historical items comprises: Calculating the difference in the placement amount of two adjacent groups of items in the historical items; When the placement quantity difference is less than or equal to a preset placement threshold, the sub-placement quantity corresponding to the placement quantity difference is determined to be the minimum display quantity.

2. The method for replenishing goods according to claim 1, characterized in that: The step of calculating the difference in the placement amount of two adjacent groups of items in the historical items includes: Obtaining the daily placement amount of the historical objects on different dates, calculating the placement significance value of the historical objects according to the daily placement amount, and determining whether the placement significance value is less than a preset significance value; When the placement significance value is less than the preset significance value, the sub-placement quantities of two adjacent groups of items of the historical items on the same shelf are obtained, and the placement quantity difference is calculated according to the sub-placement quantities.

3. The method for replenishing goods according to claim 2, characterized in that: The step of calculating the placement significance value of the historical items according to the daily placement quantity comprises: Obtaining a preset maximum capacity value and a preset time interval of the historical item; Calculating the placement control number of the historical items according to the maximum capacity value and the preset time interval; Randomly generating a placement quantity sequence based on the placement control number, and determining the daily placement quantity of the historical items according to the placement quantity sequence; A one-way analysis of variance is performed on the daily placement quantity to obtain the placement significance value of the historical items.

4. The method for replenishing goods according to claim 1, characterized in that: The step of constructing a basic deep neural network according to the target constraint capacity and the reward function comprises: Obtaining the observation value and the stage number of the historical item, and calculating the number of nodes of the basic deep neural network according to the observation value and the stage number; According to the target constraint capacity and the number of nodes, the hidden layer parameters of the basic deep neural network are determined, and the basic deep neural network is constructed based on the hidden layer parameters.

5. The method for replenishing goods according to claim 1, characterized in that: The step of calculating the historical state variable of the historical item according to the inventory quantity, the demand quantity and the order quantity comprises: Determine the initial state variables of the historical items at different stages according to the inventory quantity, the demand quantity and the order quantity; The observation period number of the historical item is obtained, and the historical state variable is calculated according to the initial state variable and the observation period number.

6. An item replenishment device, characterized in that: The item replenishment device executes the item replenishment method according to claim 1, and the item replenishment device comprises: An acquisition module, used to acquire the inventory quantity, demand quantity and order quantity of historical items at different stages, and calculate the historical state variable of the historical items according to the inventory quantity, demand quantity and order quantity; A construction module, used to obtain a preset reward function and a target constraint capacity, and construct a basic deep neural network according to the target constraint capacity and the reward function; A training module, used to train the basic deep neural network based on the historical state variables, and when the loss function of the basic deep neural network converges, determine that the training of the basic deep neural network is completed, and obtain a target deep neural network; A prediction module, used to obtain a current state variable of a target item, input the current state variable into the target deep neural network, and calculate a predicted reward value of the target item under different replenishment quantities according to the target deep neural network; A selection module is used to select the replenishment quantity corresponding to the maximum predicted reward value as the target replenishment quantity of the target item.

7. A computer device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the item replenishment method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the item replenishment method according to any one of claims 1 to 5 are implemented.

Images