Intelligent storage resource dynamic allocation method and system based on deep reinforcement learning

Abstract

The invention discloses an intelligent storage resource dynamic allocation method and system based on deep reinforcement learning, and relates to the technical field of storage management. The intelligent storage resource dynamic allocation method based on deep reinforcement learning comprises the following steps: environment data judgment; inventory management evaluation; calculating complex evaluation; and effective evaluation. According to the method, whether inventory allocation is carried out or not is judged through the obtained environment data and the preset environment data, then the storage utilization evaluation value is obtained according to the obtained tobacco inventory management data, whether storage optimization is carried out or not is judged, then the calculation complexity evaluation value is obtained according to the obtained calculation resource data, and whether calculation complexity optimization is carried out or not is judged. And finally, obtaining an optimized effective evaluation value according to the storage utilization evaluation value and the calculated complex evaluation value, thereby achieving the effect of improving the storage resource utilization efficiency in the dynamic allocation process of the intelligent storage resources, and solving the problem of low storage resource utilization efficiency in the dynamic allocation process of the intelligent storage resources in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of warehouse management technology, and in particular to a method and system for dynamic allocation of intelligent warehouse resources based on deep reinforcement learning. Background Art

[0002] With the rapid development of e-commerce and the increasing demand for automation of warehousing systems, the warehousing industry is facing unprecedented challenges and opportunities. In order to improve warehousing efficiency, reduce operating costs and meet rapidly changing market demands, the warehousing industry is gradually transforming from traditional manual operations to intelligent operations. In intelligent warehousing systems, dynamic resource allocation is of great significance to improving the overall performance of the system. Deep reinforcement learning combines the advantages of deep learning and reinforcement learning. It models complex environments through deep neural networks and uses reinforcement learning algorithms to optimize strategies. Deep reinforcement learning can effectively solve the problem of dynamic allocation of warehousing resources.

[0003] In the existing technology, data such as the environmental status in the warehousing system is collected and preprocessed, and then a deep reinforcement learning model is built and optimized based on the acquired data. Finally, the model is used to make autonomous decisions and plan paths to achieve dynamic allocation of intelligent warehousing resources.

[0004] For example, the invention patent announcement with announcement number: CN113537890B discloses a method for calculating safety stock in power storage. In step 1, statistics are made on whether the material demand for a certain material is constant within a unit time; if it is constant, the process proceeds to step 2.1; if it is not constant and obeys a normal distribution, the process proceeds to step 2.2; if it is not constant and obeys a Poisson distribution, the service guarantee rate is calculated; if it is not constant and cannot be classified as a normal distribution or a Poisson distribution, the statistical unit time range is expanded until it is classified as a normal distribution, and the process proceeds to step 2.2; in step 2.1, statistics are made on whether the ordering lead time of the material is constant; if it is constant, there is no need to set a safety stock; if it is not constant, the safety stock quantity is calculated; in step 2.2, statistics are made on whether the ordering lead time of the material is constant; if it is constant, there is no need to set a safety stock; if it is not constant, the safety stock quantity is calculated.

[0005] For example, the invention patent with announcement number: CN108876241B announces a vision-based storage space identification management system, including: a warehouse information management center, a camera, a network switch and a base station. The network switch is connected to the camera and the base station, and the base station is connected to the warehouse information management center; the base station is an image processing workstation; four cameras are installed in each storage location; each storage location is a rectangular area; a camera is installed directly above the center point corresponding to the four boundary lines, and the camera is above the center point of each boundary. The area photographed by the camera is divided into a valid area and an invalid area. After the base station collects the image, the image collected by each camera is a group, a total of 4 groups, to identify the remaining ground area in the corresponding warehouse area that can be stacked, and calculate the area.

[0006] However, in the process of implementing the technical solution of the invention in the embodiment of the present application, it is found that the above technology has at least the following technical problems:

[0007] In the existing technology, the intelligent warehousing environment is usually highly complex and uncertain, and the requirements of warehousing tasks are often changeable and uncertain. If these changes cannot be responded to in a timely manner, the flexibility and accuracy of resource allocation will be affected, and there will be a problem of low efficiency in the utilization of warehousing resources during the dynamic allocation of intelligent warehousing resources. Summary of the invention

[0008] The embodiments of the present application solve the problem of low efficiency of warehouse resource utilization in the process of dynamic allocation of intelligent warehouse resources in the prior art by providing a method and system for dynamic allocation of intelligent warehouse resources based on deep reinforcement learning, and achieve improved efficiency of warehouse resource utilization in the process of dynamic allocation of intelligent warehouse resources.

[0009] The embodiment of the present application provides a method for dynamic allocation of intelligent storage resources based on deep reinforcement learning, comprising the following steps: S1, obtaining environmental data of a tobacco storage area through a sensor, and determining whether to perform inventory allocation based on the environmental data and preset environmental data; S2, performing a storage space evaluation based on the acquired tobacco inventory management data to obtain a storage utilization evaluation value, and determining whether to optimize the storage based on the storage utilization evaluation value and a preset storage utilization threshold, wherein the storage utilization evaluation value is used to quantify the space utilization rate of the tobacco storage area; S3, performing a computational complexity evaluation based on the acquired computing resource data to obtain a computational complexity evaluation value, and determining whether to optimize the computational complexity based on the computational complexity evaluation value and a preset computational complexity threshold, wherein the computational complexity evaluation value is used to quantify the computational complexity of the computing task; S4, performing an evaluation based on the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity to obtain an optimized effective evaluation value, and determining whether to provide feedback based on the optimized effective evaluation value and a preset optimized effective threshold, wherein the optimized effective evaluation value is used to evaluate the effectiveness of optimization measures, wherein the optimization measures include optimizing the storage and optimizing the computational complexity.

[0010] Furthermore, the specific process of acquiring the environmental data is as follows: obtaining corresponding temperature data through temperature sensors deployed at preset monitoring points in the tobacco storage area; obtaining corresponding humidity data through humidity sensors deployed at preset monitoring points in the tobacco storage area; the sensors include temperature sensors and humidity sensors; the environmental data include temperature data and humidity data.

[0011] Furthermore, the specific method for evaluating the storage space according to the acquired tobacco inventory management data to obtain the storage utilization evaluation value is as follows: obtaining the preset safety inventory and the preset inventory from the preset database, wherein the preset safety inventory represents the minimum inventory set according to the enterprise strategy; obtaining the inventory turnover rate according to the relative relationship analysis between the tobacco outbound volume and the total inventory of the tobacco storage area; obtaining the inventory backlog rate according to the relative relationship analysis between the average tobacco inventory, the preset safety inventory and the total inventory of the tobacco storage area; obtaining the proportion of inventory to be adjusted according to the relative relationship analysis between the preset tobacco inventory, the average tobacco inventory and the total inventory of the tobacco storage area, wherein the proportion of inventory to be adjusted represents the proportion of the adjustment amount set to improve the storage utilization rate; processing the inventory turnover rate, the inventory backlog rate and the proportion of inventory to be adjusted to obtain the storage utilization evaluation value; the specific restricted expression of the storage utilization evaluation value is: ; ; ; ;

[0012] In the formula, i represents the number of the preset time period, , T represents the total number of preset time periods, represents the storage utilization evaluation value of the i-th preset time period, represents the inventory turnover rate in the i-th preset time period, represents the inventory backlog rate in the i-th preset time period, Indicates the proportion of inventory to be adjusted in the i-th preset time period, represents the average tobacco inventory in the i-th preset time period, ZRL represents the total inventory in the tobacco storage area, represents the tobacco delivery volume in the i-th preset time period, Indicates the preset safety stock quantity. Indicates the preset tobacco stock level.

[0013] Furthermore, the specific method for performing computational complexity evaluation based on the acquired computing resource data to obtain a computational complexity evaluation value is as follows: obtaining preset computing resource data and preset tobacco types from a preset database, wherein the preset computing resource data includes preset CPU utilization, preset memory utilization, preset I / O read / write rate, and preset algorithm running time; obtaining a CPU utilization deviation based on a relative relationship analysis between the CPU utilization and the preset CPU utilization, wherein the CPU utilization deviation represents the deviation between the CPU utilization and the preset CPU utilization; obtaining a memory utilization deviation based on a relative relationship analysis between the memory utilization and the preset memory utilization, wherein the memory utilization deviation represents the deviation between the memory utilization and the preset memory utilization; obtaining an I / O read / write rate deviation based on a relative relationship analysis between the I / O read / write rate and the preset I / O read / write rate, wherein the I / O The read / write rate deviation indicates the deviation between the I / O read / write rate and the preset I / O read / write rate; the algorithm running time deviation is obtained based on the relative relationship analysis between the algorithm running time and the preset algorithm running time, and the algorithm running time deviation indicates the deviation between the algorithm running time and the preset algorithm running time; the space utilization rate of the tobacco storage area is obtained based on the relative relationship analysis between the average tobacco inventory and the total inventory of the tobacco storage area, and the space utilization rate indicates the proportion of the average tobacco inventory in the total inventory of the tobacco storage area; the tobacco type deviation is obtained based on the relative relationship analysis between the average tobacco type and the preset tobacco type, and the tobacco type deviation indicates the deviation between the average tobacco type and the preset tobacco type; the CPU utilization deviation, memory utilization deviation, I / O read / write rate deviation, algorithm running time deviation, space utilization and tobacco type deviation are processed to obtain a computational complexity evaluation value.

[0014] Furthermore, the specific method for evaluating and obtaining the optimized effective evaluation value based on the storage utilization evaluation value obtained after optimizing the warehousing and the computational complexity evaluation value obtained after optimizing the computational complexity is as follows: obtaining the storage utilization evaluation value deviation based on the relative relationship analysis between the storage utilization evaluation value obtained after optimizing the warehousing and the storage utilization evaluation value obtained before optimizing the warehousing, wherein the storage utilization evaluation value deviation represents the difference between the storage utilization evaluation value obtained after optimizing the warehousing and the storage utilization evaluation value obtained before optimizing the warehousing; obtaining the computational complexity evaluation value deviation based on the relative relationship analysis between the computational complexity evaluation value obtained after optimizing the computational complexity and the computational complexity evaluation value obtained before optimizing the computational complexity; processing the storage utilization evaluation value deviation and the computational complexity evaluation value deviation to obtain the optimized effective evaluation value.

[0015] The embodiment of the present application provides an intelligent storage resource dynamic allocation system based on deep reinforcement learning, including: an environmental data judgment module, an inventory management evaluation module, a computational complexity evaluation module and an effective evaluation module; wherein the environmental data judgment module is used to obtain environmental data of a tobacco storage area through a sensor, and judge whether to perform inventory allocation based on the environmental data and preset environmental data; the inventory management evaluation module is used to evaluate the storage space according to the acquired tobacco inventory management data to obtain a storage utilization evaluation value, and judge whether to optimize the storage based on the storage utilization evaluation value and a preset storage utilization threshold, and the storage utilization evaluation value is used to quantify the space utilization rate of the tobacco storage area; The computational complexity assessment module is used to perform computational complexity assessment based on the acquired computing resource data to obtain a computational complexity assessment value, and determine whether to optimize the computational complexity based on the computational complexity assessment value and a preset computational complexity threshold. The computational complexity assessment value is used to quantify the computational complexity of the computing task; the effectiveness assessment module is used to perform assessment based on the storage utilization assessment value obtained after optimizing the storage and the computational complexity assessment value obtained after optimizing the computational complexity to obtain an optimized effective assessment value, and determine whether to provide feedback based on the optimized effective assessment value and a preset optimized effective threshold. The optimized effective assessment value is used to evaluate the effectiveness of optimization measures, and the optimization measures include optimizing the storage and optimizing the computational complexity.

[0016] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0017] 1. Determine whether to carry out inventory allocation through the acquired environmental data and preset environmental data, then obtain the warehouse utilization evaluation value according to the acquired tobacco inventory management data, then obtain the calculation complexity evaluation value according to the acquired calculation resource data, and finally obtain the optimized effective evaluation value according to the warehouse utilization evaluation value and the calculation complexity evaluation value, thereby realizing the effectiveness of quantitative evaluation optimization measures, and then realizing the improvement of warehouse resource utilization efficiency in the process of dynamic allocation of intelligent warehouse resources, and effectively solving the problem of low warehouse resource utilization efficiency in the process of dynamic allocation of intelligent warehouse resources in the prior art.

[0018] 2. The inventory turnover rate is obtained through the tobacco outbound volume and the total inventory of the tobacco storage area. Then the inventory backlog rate is obtained according to the average tobacco inventory, the preset safety inventory and the total inventory of the tobacco storage area. Then the proportion of inventory to be adjusted is obtained according to the preset tobacco inventory, the average tobacco inventory and the total inventory of the tobacco storage area. Finally, the inventory turnover rate, the inventory backlog rate and the proportion of inventory to be adjusted are used to obtain the storage utilization evaluation value, thereby realizing the quantitative evaluation of the space utilization rate of the tobacco storage area, and then realizing the improvement of the space utilization efficiency of the tobacco storage area.

[0019] 3. The storage utilization evaluation value deviation is obtained by optimizing the storage utilization evaluation value after storage and the storage utilization evaluation value before storage. Then, the computational complexity evaluation value deviation is obtained according to the computational complexity evaluation value obtained after optimizing the computational complexity and the computational complexity evaluation value obtained before optimizing the computational complexity. Finally, the storage utilization evaluation value deviation and the computational complexity evaluation value deviation are used to obtain the optimized effective evaluation value, thereby realizing the quantitative evaluation of the effectiveness of storage optimization and computational complexity optimization measures, and then realizing the improvement of storage space utilization efficiency and computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram showing changes in the warehouse utilization evaluation value provided in the embodiment of the present application as the inventory turnover rate changes;

[0022] Figure 3 A schematic diagram showing changes in the storage utilization evaluation value provided in the embodiment of the present application as the inventory backlog rate changes;

[0023] Figure 4 A schematic diagram showing changes in the storage utilization evaluation value provided in the embodiment of the present application as the proportion of the inventory to be adjusted changes;

[0024] Figure 5 A schematic diagram of the structure of the intelligent warehousing resource dynamic allocation system based on deep reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiment of the present application solves the problem of low efficiency of warehouse resource utilization in the process of dynamic allocation of intelligent warehouse resources in the prior art by providing a method and system for dynamic allocation of intelligent warehouse resources based on deep reinforcement learning. It determines whether to perform inventory allocation by obtaining environmental data and preset environmental data, and then obtains a warehouse utilization evaluation value based on the obtained tobacco inventory management data, and determines whether to optimize the warehouse. Then, it obtains a computational complexity evaluation value based on the obtained computing resource data, and determines whether to optimize the computational complexity. Finally, it obtains an optimized effective evaluation value based on the warehouse utilization evaluation value and the computational complexity evaluation value, and determines whether to provide feedback, thereby improving the efficiency of warehouse resource utilization in the process of dynamic allocation of intelligent warehouse resources.

[0026] The technical solution in the embodiment of the present application is to solve the problem of low efficiency of warehouse resource utilization in the above-mentioned intelligent warehouse resource dynamic allocation process. The overall idea is as follows:

[0027] Whether to carry out inventory allocation is determined by the acquired environmental data and the preset environmental data, and then the warehouse utilization evaluation value is obtained according to the acquired tobacco inventory management data. Then, the computational complexity evaluation value is obtained according to the acquired computing resource data. Finally, the optimized effective evaluation value is obtained according to the warehouse utilization evaluation value and the computational complexity evaluation value, thereby achieving the effect of improving the utilization efficiency of warehouse resources in the process of dynamic allocation of intelligent warehouse resources.

[0028] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] like Figure 1 As shown, it is a flow chart of the intelligent storage resource dynamic allocation method based on deep reinforcement learning provided by an embodiment of the present application, and the method includes the following steps: S1, environmental data judgment: obtaining environmental data of the tobacco storage area through sensors, and judging whether to perform inventory allocation based on the environmental data and preset environmental data; S2, inventory management evaluation: performing storage space evaluation according to the obtained tobacco inventory management data to obtain a storage utilization evaluation value, judging whether to optimize the storage based on the storage utilization evaluation value and the preset storage utilization threshold, and the storage utilization evaluation value is used to quantify the space utilization rate of the tobacco storage area; S3, computational complexity evaluation: performing computational complexity evaluation according to the obtained computing resource data to obtain a computational complexity evaluation value, judging whether to optimize the computational complexity based on the computational complexity evaluation value and the preset computational complexity threshold, and the computational complexity evaluation value is used to quantify the computational complexity of the computing task; S4, effectiveness evaluation: evaluating according to the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity to obtain an optimized effective evaluation value, judging whether to feedback based on the optimized effective evaluation value and the preset optimized effective threshold, and the optimized effective evaluation value is used to evaluate the effectiveness of the optimization measures, and the optimization measures include optimizing the storage and optimizing the computational complexity.

[0030] It should be added that the specific process of obtaining environmental data is as follows: the corresponding temperature data is obtained by using temperature sensors deployed at preset monitoring points in the tobacco storage area; the corresponding humidity data is obtained by using humidity sensors deployed at preset monitoring points in the tobacco storage area; the sensors include temperature sensors and humidity sensors; the environmental data include temperature data and humidity data.

[0031] It should be understood that the warehouse is divided into multiple storage areas according to the needs of tobacco storage and the layout of the storage area. For example, if the total area of ​​the warehouse is 200 square meters, then Area A is 50 square meters for storing strong-flavor tobacco, Area B is 50 square meters for storing light-flavor tobacco, Area C is 50 square meters for storing moist tobacco, and Area D is 50 square meters for storing dry tobacco. The preset monitoring points are usually set according to the needs of tobacco storage and the layout of the storage area to ensure that the environmental conditions of the storage area can be fully reflected. For example, temperature sensors and humidity sensors are set at different locations inside the storage area, including corners, under the roof, near the shelves, and near doors, windows and vents. For a tobacco stack with a height of 3 meters and a width of 2 meters, 6 groups of sensors can be evenly distributed inside it, located at the top, middle, bottom, sides and middle of the tobacco stack respectively. The temperature data is the average value of all temperature sensors in the storage area; the humidity data is the average value of all humidity sensors in the storage area. By acquiring environmental data, subsequent environmental management of the tobacco storage area is achieved.

[0032] In this embodiment, tobacco inventory management data includes the average tobacco inventory, the total inventory of the tobacco storage area, the tobacco outbound quantity and the average tobacco type; the tobacco inventory, tobacco outbound quantity and inventory quantity information of each type of tobacco are recorded through the warehouse management system, and the tobacco types are obtained by statistics; the average tobacco inventory represents the average value of the tobacco inventory within a preset time period; the average tobacco type represents the average value of the tobacco types within a preset time period.

[0033] Computing resource data includes CPU utilization, memory utilization, I / O read and write rates, and algorithm running time.

[0034] In the Linux system, the CPU utilization is obtained by reading the / proc / stat file, the memory utilization is obtained by reading the / proc / meminfo file, and the I / O read and write rates are obtained by using the iostat tool. In the Windows system, the CPU utilization is obtained by calling the GetSystemTimes function, the memory utilization is obtained by calling the GlobalMemoryStatusEx function, and the I / O read and write rates are obtained by using the Performance Monitor. The algorithm running time is obtained by using the time measurement function in the programming language (such as the clock() function in the C language, the time.time() or time.perf_counter() function in Python, etc.).

[0035] Whether to provide feedback is determined based on the optimization effective evaluation value and the preset optimization effective threshold. Feedback means sending a feedback report to the preset personnel. The feedback report includes the storage utilization evaluation value obtained before storage optimization, the storage utilization evaluation value obtained after storage optimization, the computational complexity evaluation value obtained before computational complexity optimization, and the computational complexity evaluation value obtained after computational complexity optimization. By acquiring and analyzing environmental data, inventory management data, and computing resource data in real time, and judging whether optimization measures need to be taken, and evaluating the effectiveness of the optimization measures, it is ensured that tobacco is stored under optimal conditions, while improving the utilization efficiency of computing resources and the overall management efficiency, thereby realizing the improvement of the utilization efficiency of storage resources in the process of dynamic allocation of intelligent storage resources.

[0036] Furthermore, the specific process of determining whether to perform inventory allocation based on environmental data and preset environmental data is as follows: A1, obtaining preset environmental data from a preset database, the preset environmental data including a first preset temperature range, a second preset temperature range, a first preset humidity range, and a second preset humidity range; A2, determining whether the temperature data is within the first preset temperature range, if the temperature data is within the first preset temperature range, executing A4, otherwise executing A3; A3, determining whether the temperature data is within the second preset temperature range, if the temperature data is within the second preset temperature range, triggering high-frequency monitoring and notifying the preset personnel, otherwise performing inventory allocation; A4, determining whether the humidity data is within the first preset humidity range, if the humidity data is within the first preset humidity range, ending the inventory allocation, otherwise executing A5; A5, determining whether the humidity data is within the second preset humidity range, if the humidity data is within the second preset humidity range, triggering high-frequency monitoring and notifying the preset personnel, otherwise performing inventory allocation.

[0037] In this embodiment, the first preset temperature range is set to 20℃-25℃, the second preset temperature range is set to 15℃-30℃, the first preset humidity range is set to 60%-70%RH, and the second preset humidity range is set to 55%-75%RH; the environmental data represents the temperature data and humidity data in the preset storage area; triggering high-frequency monitoring means that when the environmental data of the preset storage area is not within the first preset environmental data range but within the second preset environmental data range, the monitoring frequency of the sensor in the preset storage area will be increased, for example, if the monitoring frequency of the temperature sensor is once an hour, it will be adjusted to once every half an hour; when the humidity data is not within the first preset humidity range but within the second preset humidity range, the monitoring frequency of the humidity sensor will be increased, for example, if the monitoring frequency of the humidity sensor is once an hour, it will be adjusted to once every half an hour.

[0038] By comparing real-time monitoring environmental data with preset environmental data, it is possible to promptly discover whether the inventory environment is suitable, thereby preventing tobacco products from being damaged by environmental factors; by setting different preset environmental data, the safety of inventory items is guaranteed and unnecessary waste of resources is avoided.

[0039] Furthermore, the specific process of inventory allocation is as follows: B1, adjust the environmental control equipment, and determine whether the environmental data obtained after the adjustment is within the first preset environmental data range within the adjustment time period. If the environmental data obtained after the adjustment is within the first preset environmental data range within the adjustment time period, the inventory allocation is terminated, otherwise B2 is executed. The first preset environmental data range includes a first preset temperature range and a first preset humidity range; B2, the tobacco products are allocated to the storage area that meets the first preset environmental data range.

[0040] In this embodiment, a computer controls an environmental control device to adjust environmental data; the environmental control device includes an air conditioner, a dehumidifier and a humidifier; the adjustment time period is set to 2 hours; if the environmental data obtained after the adjustment is not within the first preset environmental data range within the adjustment time period, firstly, the environmental data of all storage areas in the tobacco storage area are obtained through the warehouse management system, and the storage areas that meet the first preset environmental data range are screened, and then the storage area closest to the current storage area where the tobacco product is located is selected from all the storage areas that meet the first preset environmental data range, and finally, if there is an automated transport robot, the tobacco product is transported to the selected storage area by the automated transport robot, otherwise, the preset personnel are notified to transport the tobacco product to the selected storage area, and the quantity and type of the transported tobacco products are recorded; through the above steps, it is ensured that tobacco products can be stored in an environment that meets the preservation requirements, and the effective utilization of storage resources in the dynamic allocation of intelligent storage resources is realized.

[0041] Furthermore, the specific method for evaluating the storage space based on the acquired tobacco inventory management data to obtain the storage utilization evaluation value is as follows: obtaining the preset safety stock and the preset inventory from the preset database, the preset safety stock represents the minimum inventory set according to the enterprise strategy; analyzing the relative relationship between the tobacco outbound volume and the total inventory of the tobacco storage area to obtain the inventory turnover rate (i.e., the specific restriction expression of the storage utilization evaluation value) ), the inventory turnover rate represents the liquidity of the inventory; the inventory backlog rate (i.e. the specific restriction expression of the storage utilization evaluation value) is obtained by analyzing the relative relationship between the average tobacco inventory, the preset safety inventory and the total inventory of the tobacco storage area. ), the inventory backlog rate represents the proportion of the part exceeding the safety stock; the proportion of inventory to be adjusted (i.e. the specific restriction expression of the storage utilization evaluation value) is obtained by analyzing the relative relationship between the preset tobacco inventory, the average tobacco inventory and the total inventory of the tobacco storage area. ), the proportion of inventory to be adjusted represents the proportion of the adjustment amount set to improve warehouse utilization; the inventory turnover rate, inventory backlog rate and the proportion of inventory to be adjusted are processed to obtain the warehouse utilization evaluation value.

[0042] Specifically, the specific restricted expression of the storage utilization evaluation value is: ; ; ; ;

[0043] In the formula, i represents the number of the preset time period, , T represents the total number of preset time periods, represents the storage utilization evaluation value of the i-th preset time period, represents the inventory turnover rate in the i-th preset time period, represents the inventory backlog rate in the i-th preset time period, Indicates the proportion of inventory to be adjusted in the i-th preset time period, represents the average tobacco inventory in the i-th preset time period, ZRL represents the total inventory in the tobacco storage area, represents the tobacco delivery volume in the i-th preset time period, Indicates the preset safety stock quantity. Indicates the preset tobacco stock level.

[0044] In this embodiment, the preset safety stock represents the minimum stock set according to the enterprise strategy. For example, if the average monthly sales volume of strong-flavor tobacco is 1,000 boxes, the delivery cycle is 14 days, and the demand fluctuation coefficient is 20%, then the preset safety stock is box; the preset tobacco inventory represents the average of the average tobacco inventory in the historical time period; the historical time period is set as the beginning of last month to the beginning of this month through the average historical time period, for example, if it is currently August 11, the historical time period is set as July 1 to August 1; through the above steps, a quantitative evaluation of the space utilization rate of the tobacco storage area is achieved, thereby improving the space utilization efficiency of the tobacco storage area.

[0045] For example, the total inventory of the tobacco storage area is 1,000 cartons, the preset safety stock is 300 cartons, and the preset tobacco inventory is 500 cartons. Figure 2 A schematic diagram of the change of the warehouse utilization evaluation value with the inventory turnover rate provided in the embodiment of the present application. It can be seen from the figure that when the inventory backlog rate is 0.2 and the proportion of inventory to be adjusted is 0, the greater the inventory turnover rate, the greater the warehouse utilization evaluation value; Figure 3A schematic diagram of the change of the warehouse utilization evaluation value with the inventory backlog rate provided in the embodiment of the present application. It can be seen from the figure that when the inventory turnover rate is 0.5 and the proportion of inventory to be adjusted is 0, the greater the inventory backlog rate, the smaller the warehouse utilization evaluation value; Figure 4 A schematic diagram of the change of the warehouse utilization evaluation value with the change of the proportion of inventory to be adjusted provided in the embodiment of the present application. It can be seen from the figure that when the inventory backlog rate is 0.2 and the inventory turnover rate is 0.5, the closer the proportion of inventory to be adjusted is to 0, the greater the warehouse utilization evaluation value.

[0046] This algorithm combines the inventory backlog rate, inventory turnover rate and the proportion of inventory to be adjusted to obtain a comprehensive analysis of the warehouse utilization evaluation value. The various parameters are related and do not exist independently. They are a whole and cannot be obtained by simple combination and addition. For example, an increase in the inventory turnover rate in the formula may lead to an increase in the warehouse utilization evaluation value, which does not necessarily mean an increase in the warehouse utilization evaluation value. At this time, the inventory backlog rate and the inventory to be adjusted need to be considered. These parameters can also determine the warehouse utilization evaluation value at the next moment. Generally speaking, a decrease in inventory backlog indicates a higher inventory turnover rate and that warehouse resources can be used more efficiently, thereby improving the warehouse utilization evaluation value; the higher the inventory turnover rate, the higher the warehouse utilization evaluation value, because the inventory continues to flow; the higher the inventory backlog rate, the lower the warehouse utilization evaluation value; the higher the inventory backlog rate, the lower the warehouse utilization evaluation value; the higher the inventory backlog rate often leads to an increase in the proportion of inventory to be adjusted; an increase in the outbound volume will reduce the inventory, which will help reduce the inventory backlog and bring the inventory closer to the demand level, A higher outbound volume helps to improve inventory turnover and warehouse utilization efficiency; if the inventory to be adjusted is a positive number, it means that the inventory needs to be increased to meet future demand and improve warehouse utilization; if the inventory to be adjusted is a negative number, it means that the inventory is too much and the outbound volume needs to be accelerated to reduce the backlog; the smaller the proportion of inventory to be adjusted, the higher the warehouse utilization evaluation value and inventory turnover efficiency. The algorithm obtained through comprehensive analysis can accurately and quantitatively analyze the space utilization rate of the tobacco storage area, indicating the effect of accurately evaluating the storage utilization evaluation value. Therefore, accurate analysis is carried out by establishing a mathematical form; the effective quantification of the space utilization rate of the tobacco storage area is achieved during the dynamic allocation of intelligent storage resources, avoiding subsequent mistakes caused by incorrect storage utilization evaluation values.

[0047] Furthermore, the specific process of judging whether to optimize the warehouse based on the warehouse utilization evaluation value and the preset warehouse utilization threshold is as follows: C1, judging whether the warehouse utilization evaluation value is less than the preset warehouse utilization threshold. If the warehouse utilization evaluation value is less than the preset warehouse utilization threshold, C2 is executed, otherwise the warehouse optimization is not performed; C2, judging whether the inventory to be adjusted is a positive number. If the inventory to be adjusted is a positive number, it is prompted to increase the inventory. If the inventory to be adjusted is a negative number, it is prompted to increase the outbound volume. If the inventory to be adjusted is zero, C3 is executed; C3, converting the virtual cargo space according to the average tobacco inventory and tobacco outbound volume. The virtual cargo space represents a Dynamic logical storage locations for warehouse management, and virtual storage locations are used to achieve flexible management of storage space; C4, inventory turnover optimization, inventory turnover optimization includes batch inventory management and storage location management. Batch inventory management means managing batch inventory with a first-in-first-out strategy, and storage location management means that the outbound quantity of each tobacco product is arranged in descending order, and the storage location of the tobacco products is adjusted according to the obtained outbound quantity sequence, and the tobacco product with the largest outbound quantity is stored in the first position, and the tobacco product with the smallest outbound quantity is stored in the second position. The first position represents the shelf closest to the outbound port, and the second position represents the shelf farthest from the outbound port.

[0048] In this embodiment, the preset storage utilization threshold is represented by the maximum value of the storage utilization evaluation value in the historical time period; if the storage utilization evaluation value is less than the preset storage utilization threshold, it means that the spatial resource utilization rate of the tobacco storage area is insufficient or the turnover rate is low, and optimization is required.

[0049] The inventory to be adjusted represents the difference between the preset tobacco inventory and the average tobacco inventory (i.e. ); Analyze the shipment volume of each tobacco product through the deep Q network and formulate the optimal inventory increase plan; increase the shipment volume by increasing sales efforts and inventory transfer; inventory transfer means transporting the inventory to be adjusted to the area where the tobacco inventory needs to be increased; increase sales efforts means formulating promotional strategies, such as discounts, for tobacco products with excessive inventory.

[0050] The storage space is dynamically divided into physical storage spaces and virtual storage spaces, and then the deep Q network is used to dynamically adjust the allocation and use strategies of virtual storage spaces. Finally, the virtual storage spaces are converted into physical storage spaces based on the average tobacco inventory and tobacco outbound volume. The virtual storage spaces have the following characteristics: dynamic adjustment of size and position, support for mixed storage of multiple types of goods, priority, guidance of storage and retrieval operations, and conversion between physical storage spaces. Priority determination is achieved through a priority experience playback mechanism.

[0051] The deep Q network has the following characteristics: a multi-layer convolutional neural network structure is used to extract data features. After the convolution layer, batch normalization technology is used to improve the stability of the model, and residual connections are introduced between different layers of the network. Then, a random dropout regularization method (Dropout Regularization) is used after the fully connected layer to randomly discard a part of the neurons to prevent the model from overfitting, and finally a linear rectifier function (Rectified Linear Unit) is used; the input layer of the multi-layer convolutional neural network structure includes parameters such as preset safety stock, preset inventory and outbound quantity; the convolution layer of the multi-layer convolutional neural network structure extracts the features of the input data through convolution operations; the pooling layer of the multi-layer convolutional neural network structure is used to reduce the dimension and calculation amount of the data; the priority experience playback mechanism includes: calculating the temporal difference (TD) error of each experience, for example, calculating the TD error based on the difference between the predicted outbound quantity and the actual outbound quantity, and then assigning a priority to each experience according to the TD error, for example, the experience with a TD error of 10 has a higher priority than the experience with a TD error of 5, then using the priority-based importance sampling method to select the experience, and finally regularly updating the priority in the experience pool.

[0052] Through virtual cargo space conversion and storage location management, the storage space is used more rationally, and the space waste is reduced in the process of dynamic allocation of intelligent storage resources. Through batch inventory management and storage location management, the inventory turnover speed is accelerated and the inventory cost is reduced in the process of dynamic allocation of intelligent storage resources. By optimizing the storage location and introducing advanced warehouse management systems, the outbound time in the process of dynamic allocation of intelligent storage resources is shortened.

[0053] Furthermore, a specific method for performing computational complexity evaluation based on the acquired computational resource data to obtain a computational complexity evaluation value is as follows: obtaining preset computational resource data and preset tobacco types from a preset database, wherein the preset computational resource data includes preset CPU utilization, preset memory utilization, preset I / O read / write rate, and preset algorithm running time; obtaining a CPU utilization deviation (i.e., the CPU utilization deviation in the specific restriction expression of the computational complexity evaluation value) based on a relative relationship analysis between the CPU utilization and the preset CPU utilization; ), CPU utilization deviation indicates the deviation between CPU utilization and preset CPU utilization; memory utilization deviation (i.e., the specific restriction expression for calculating complex evaluation value) is obtained based on the relative relationship between memory utilization and preset memory utilization. ), memory utilization deviation indicates the deviation between memory utilization and preset memory utilization; the I / O read / write rate deviation (i.e., the specific restriction expression for calculating the complex evaluation value) is obtained based on the relative relationship between the I / O read / write rate and the preset I / O read / write rate. ), the I / O read / write rate deviation represents the deviation between the I / O read / write rate and the preset I / O read / write rate; the algorithm running time deviation (i.e., the specific restriction expression for calculating the complex evaluation value) is obtained by analyzing the relative relationship between the algorithm running time and the preset algorithm running time. ), the algorithm running time deviation represents the deviation between the algorithm running time and the preset algorithm running time; the space utilization rate of the tobacco storage area is obtained by analyzing the relative relationship between the average tobacco inventory and the total inventory of the tobacco storage area (i.e., the specific restriction expression for calculating the complex evaluation value). ), the space utilization rate represents the proportion of the average tobacco inventory in the total inventory of the tobacco storage area; the tobacco type deviation (i.e., the specific restriction expression in calculating the complex evaluation value) is obtained based on the relative relationship analysis between the average tobacco type and the preset tobacco type. ), tobacco type deviation represents the deviation between the average tobacco type and the preset tobacco type; CPU utilization deviation, memory utilization deviation, I / O read / write rate deviation, algorithm running time deviation, space utilization and tobacco type deviation are processed to obtain the computational complexity evaluation value.

[0054] Among them, the specific restriction expression for calculating the complex evaluation value is: ;

[0055] In the formula, i represents the number of the preset time period, , T represents the total number of preset time periods, represents the computational complexity evaluation value of the i-th preset time period, Indicates the CPU utilization rate in the i-th preset time period. Indicates the memory utilization rate of the i-th preset time period, Indicates the I / O read and write rate in the i-th preset time period, represents the algorithm running time of the i-th preset time period, Indicates the preset CPU utilization. Indicates the preset memory utilization, Indicates the preset I / O read and write rate. Indicates the preset algorithm running time, represents the average tobacco inventory in the i-th preset time period, ZRL represents the total inventory in the tobacco storage area, represents the average tobacco type in the i-th preset time period, Indicates the preset tobacco type.

[0056] In this embodiment, the preset CPU utilization is represented by the average value of COU utilization in the historical time period; the preset memory utilization is represented by the average value of memory utilization in the historical time period; the preset I / O read and write rate is represented by the average value of I / O read and write rate in the historical time period; the preset algorithm running time is represented by the average value of algorithm running time in the historical time period; the preset tobacco type is represented by the average value of tobacco type in the historical time period; through the above steps, a quantitative evaluation of computational complexity is achieved, thereby achieving improvement in computational efficiency in the dynamic allocation of intelligent warehousing resources.

[0057] The higher the complexity of the algorithm, the more computing tasks there are, and the higher the CPU utilization; the higher the complexity of the algorithm, the higher the memory utilization; the algorithm running time is affected by the I / O speed, the smaller the I / O read and write rate, even if the CPU and memory resources are sufficient, the algorithm will be blocked because of waiting for I / O to complete; the computational complexity evaluation value increases with the increase of the algorithm running time; when the CPU, memory and I / O resources compete for computing tasks (for example, in complex data calculations), it will cause a "bottleneck effect". For example, when the memory utilization rate increases, swap operations will be frequently performed, increasing I / O operations, which will cause the CPU to wait for I / O to complete. The smaller the I / O read and write rate, the larger the computational complexity evaluation value; therefore, the higher the CPU utilization rate, the larger the computational complexity evaluation value; the higher the memory utilization rate, the larger the computational complexity evaluation value; the longer the algorithm running time, the larger the computational complexity evaluation value; the smaller the I / O read and write rate, the larger the computational complexity evaluation value.

[0058] The more average tobacco inventory, the higher the memory resources required to store and process these data. The increase in the average tobacco inventory will increase the complexity of the calculation, thereby increasing the computational complexity assessment value. Different types of tobacco usually have different storage and management requirements, requiring more complex data structures, thus increasing the CPU and memory occupancy rate. The increase in tobacco types increases the complexity of warehouse management, resulting in an increase in algorithm complexity, thereby increasing the computational complexity assessment value. Therefore, the more average tobacco inventory, the greater the computational complexity assessment value. The more tobacco types, the greater the computational complexity assessment value. The greater the computational complexity assessment value, the higher the algorithm complexity.

[0059] Furthermore, the specific process of judging whether to optimize the computational complexity based on the computational complexity evaluation value and the preset computational complexity threshold is as follows: D1, judging whether the computational complexity evaluation value is greater than the preset computational complexity threshold. If the computational complexity evaluation value is greater than the preset computational complexity threshold, D2 is executed, otherwise the computational complexity is not optimized; D2, judging whether the difference between the computational complexity evaluation value and the preset computational complexity threshold is less than the first threshold. If the difference between the computational complexity evaluation value and the preset computational complexity threshold is not less than the first threshold, D3 is executed, otherwise cache optimization is performed, and cache optimization includes adjusting the cache strategy and asynchronous cache update; D3, judging whether the difference between the computational complexity evaluation value and the preset computational complexity threshold is less than the second threshold. If the difference between the computational complexity evaluation value and the preset computational complexity threshold is not less than the second threshold, model pruning is performed, otherwise parallel processing is performed, and the model is an adaptive multi-objective optimization model.

[0060] In this embodiment, the preset calculation complexity threshold is represented by the maximum value of the calculation complexity evaluation value in the historical time period; the first threshold is set to 40% of the calculation complexity evaluation value; the second threshold is set to 60% of the calculation complexity evaluation value; through the above steps, the calculation complexity is reduced, and the calculation resource utilization efficiency is improved during the dynamic allocation of intelligent warehousing resources.

[0061] By using the Least Recently Used (LRU) cache strategy, the cache replacement strategy is dynamically adjusted according to the data access pattern to improve the cache hit rate; the asynchronous cache update selects the write-back strategy; the write-back strategy means that when the data is updated, only the modification is made in the cache, and the changes are regularly written to the back-end storage, thereby reducing the frequency of direct write operations to the back-end.

[0062] Parallel processing is achieved by distributing tasks to multiple nodes in a cluster using a distributed computing framework (such as Apache Spark and Hadoop). Model pruning evaluates the importance of each parameter or structure in the adaptive multi-objective optimization model, using methods such as weight pruning (based on the size of weight values) and channel pruning (based on the importance of feature maps) to identify redundant parts that can be removed.

[0063] The construction of the adaptive multi-objective optimization model includes the following steps: constructing the warehouse environment state space, defining the action space, designing the reward function, building a deep Q network, and adopting a priority experience replay mechanism.

[0064] The storage environment state space is the input of the deep reinforcement learning algorithm; the storage environment state space includes cargo information, cargo location information, equipment information, order information, warehouse layout information, human resource information, real-time inventory level, historical order data and current system load status; cargo information is identified and tracked through RFID readers; warehouse status and personnel activities are monitored through visual sensors; cargo weight is accurately measured through weight sensors; storage environment parameters are monitored through temperature and humidity sensors; cargo information includes type, quantity, volume, weight, storage requirements, etc.; cargo location information includes location, capacity, occupancy status, suitable cargo types, etc.; equipment information includes location, status, load, etc. of AGV, forklift, conveyor belt and other equipment; order information includes order type, quantity, priority, delivery time, etc.; warehouse layout information includes shelf distribution, aisle width, entrance and exit locations, etc.; human resource information includes number of employees, skill level, current tasks, etc.; real-time inventory level indicates the current inventory of various commodities; historical order data is used to predict future demand; current system load status indicates the busyness of each subsystem.

[0065] The action space defines all the decision behaviors that can be executed; the action space includes storage space allocation, route planning, task scheduling, inventory replenishment decisions, personnel scheduling, equipment maintenance arrangements, batch picking strategy adjustments and dynamic pricing; storage space allocation means allocating the optimal storage location for incoming goods; route planning means designing the optimal transportation route for inbound and outbound tasks; task scheduling means reasonably allocating and arranging various warehousing tasks; inventory replenishment decisions means determining when and what kind of goods need to be replenished; personnel scheduling means allocating human resources according to task requirements; equipment maintenance arrangements means formulating preventive maintenance plans; batch picking strategy adjustments means optimizing order batch combinations and picking sequences; dynamic pricing means adjusting commodity prices according to inventory levels and market demand.

[0066] The reward function is the core of the reinforcement learning algorithm and is used to evaluate the quality of each action. In this embodiment, the reward function comprehensively considers the following factors: order completion time (the shorter the better); storage space utilization (the higher the better); equipment utilization (maintained within a reasonable range, set according to industry standards, generally set between 70% and 90%); human resource efficiency (maximizing personnel work efficiency); energy consumption (minimizing energy use); cargo damage rate (minimizing cargo damage); customer satisfaction (based on order accuracy and timeliness); order completion time represents the difference between the start and completion time of each order; equipment utilization represents the ratio of the working time of the equipment to the available time; human resource efficiency represents the working time of employees and the number of tasks completed; energy consumption represents the energy consumption data of all equipment; cargo damage rate represents the proportion of cargo damaged during storage; customer satisfaction is obtained by regularly collecting customer feedback and evaluation data.

[0067] The reward function is expressed as: , where n is the number of the parameter, , is the order completion time weight, is the weight of storage space utilization, is the equipment utilization weight, is the weight of human resource efficiency, is the energy consumption weight, is the cargo damage rate weight, is the customer satisfaction weight, The order completion time, For storage space utilization, is the equipment utilization rate, For human resource efficiency, For energy consumption, is the damage rate of goods, For customer satisfaction.

[0068] The deep Q network includes: using a multi-layer convolutional neural network structure, then using batch normalization technology, introducing residual connections, then using a random inactivation regularization method, and finally using a linear rectification function.

[0069] The priority experience playback mechanism includes: calculating the temporal difference error of each experience, then assigning a priority to each experience based on the temporal difference error, then selecting the experience using a priority-based importance sampling method, and finally regularly updating the priority in the experience pool.

[0070] Furthermore, the specific method for evaluating the optimized effective evaluation value based on the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity is as follows: the storage utilization evaluation value deviation (i.e., the specific restriction expression of the optimized effective evaluation value) is obtained based on the relative relationship analysis between the storage utilization evaluation value obtained after optimizing the storage and the storage utilization evaluation value obtained before optimizing the storage. ), the storage utilization evaluation value deviation represents the difference between the storage utilization evaluation value obtained after optimizing the storage and the storage utilization evaluation value obtained before optimizing the storage; the calculation complexity evaluation value deviation (i.e., the specific restriction expression of the optimization effective evaluation value) is obtained by analyzing the relative relationship between the calculation complexity evaluation value obtained after optimizing the calculation complexity and the calculation complexity evaluation value obtained before optimizing the calculation complexity. ), the calculation complexity evaluation value deviation represents the difference between the calculation complexity evaluation value obtained after optimizing the calculation complexity and the calculation complexity evaluation value obtained before optimizing the calculation complexity; the storage utilization evaluation value deviation and the calculation complexity evaluation value deviation are processed to obtain the optimized effective evaluation value.

[0071] Among them, the specific restriction expression for optimizing the effective evaluation value is: ; ; ;

[0072] In the formula, i represents the number of the preset time period, , T represents the total number of preset time periods, represents the optimized effective evaluation value of the i-th preset time period, represents the storage utilization evaluation value deviation of the i-th preset time period, represents the calculation complexity evaluation value deviation of the i-th preset time period, represents the storage utilization evaluation value obtained after optimizing storage in the i-th preset time period, represents the storage utilization evaluation value of the i-th preset time period, represents the computational complexity evaluation value of the i-th preset time period, It represents the computational complexity evaluation value obtained after optimizing the computational complexity of the i-th preset time period.

[0073] In this embodiment, if data is missing within a preset time period, the data can be merged as needed by dynamically adjusting the length of the time period. For example, if data is missing in a certain hour, the time period is extended to two hours to make up for the missing data. Through the above steps, a quantitative evaluation of the effectiveness of warehouse optimization and computational complexity optimization measures is achieved, thereby achieving improved utilization efficiency of warehouse space and computational efficiency.

[0074] The storage utilization evaluation value after storage optimization is usually higher than the storage profit evaluation value before optimization, reflecting more effective inventory management, increased outbound rate and reduced backlog rate; the greater the deviation of the storage utilization evaluation value, the greater the optimization effective evaluation value; after storage optimization, the improvement of the storage utilization evaluation value directly reflects the improvement of inventory management efficiency; by optimizing the calculation complexity, unnecessary calculation and resource waste are reduced, thereby reducing the calculation complexity evaluation value; the greater the deviation of the calculation complexity evaluation value, the greater the optimization effective evaluation value; the increase in the average tobacco inventory in the storage utilization evaluation value will increase the complexity of the calculation, thereby increasing the calculation complexity evaluation value.

[0075] Taking the storage utilization evaluation value as 0.454, the calculation complexity evaluation value as 1.537, and the preset time period as 1 as an example, the statistical table of the changes in the optimized effective evaluation value is shown in Table 1:

[0076] Table 1 Statistics of changes in optimization effective evaluation values

[0077] It can be seen from the first and second groups of data in the table that when the storage utilization evaluation value obtained after optimizing the storage is greater than the storage utilization evaluation value, the larger the storage utilization evaluation value obtained after optimizing the storage, the larger the optimization effective evaluation value; it can be seen from the second and third groups of data that when the computational complexity evaluation value obtained after optimizing the computational complexity is less than the computational complexity evaluation value, the larger the computational complexity evaluation value obtained after optimizing the computational complexity, the smaller the optimization effective evaluation value; it can be seen from the third and fourth groups of data that when the storage utilization evaluation value obtained after optimizing the storage is less than the storage utilization evaluation value, the storage utilization evaluation value deviation is not calculated; it can be seen from the second and fifth groups of data that when the computational complexity evaluation value obtained after optimizing the computational complexity is greater than the computational complexity evaluation value, the computational complexity evaluation value deviation is not calculated.

[0078] like Figure 5 As shown, it is a structural schematic diagram of the intelligent warehousing resource dynamic allocation system based on deep reinforcement learning provided in an embodiment of the present application. The intelligent warehousing resource dynamic allocation system based on deep reinforcement learning provided in an embodiment of the present application includes: an environmental data judgment module, an inventory management evaluation module, a computational complexity evaluation module and an effective evaluation module; wherein the environmental data judgment module is used to obtain environmental data of the tobacco storage area through sensors, and judge whether to perform inventory allocation based on the environmental data and preset environmental data; the inventory management evaluation module is used to evaluate the storage space according to the acquired tobacco inventory management data to obtain a storage utilization evaluation value, and judge whether to optimize the storage based on the storage utilization evaluation value and the preset storage utilization threshold, and the storage utilization is optimized. The evaluation value is used to quantify the space utilization rate of the tobacco storage area; the computational complexity evaluation module is used to evaluate the computational complexity according to the acquired computing resource data to obtain a computational complexity evaluation value, and judge whether to optimize the computational complexity based on the computational complexity evaluation value and a preset computational complexity threshold. The computational complexity evaluation value is used to quantify the computational complexity of the computing task; the effectiveness evaluation module is used to evaluate according to the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity to obtain an optimized effective evaluation value, and judge whether to provide feedback based on the optimized effective evaluation value and a preset optimized effective threshold. The optimized effective evaluation value is used to evaluate the effectiveness of optimization measures, and the optimization measures include optimizing the storage and optimizing the computational complexity.

[0079] In this embodiment, whether to perform inventory allocation is determined by the tobacco storage area environmental data and the preset environmental data, thereby achieving an improvement in the stability of the tobacco storage environment; optimizing warehousing through the deep Q network in the deep reinforcement learning algorithm, achieving a quantitative evaluation of the space utilization of the storage area, thereby achieving an improvement in the utilization rate of warehousing resources; optimizing the allocation of computing resources by quantifying the complexity of computing tasks, thereby achieving an improvement in the utilization rate of computing resources; through the effective evaluation module, a quantitative evaluation of the effectiveness of the optimization measures is achieved, and the optimization strategy is adjusted in time according to the evaluation results, thereby achieving an improvement in the utilization efficiency of storage resources in the process of dynamic allocation of intelligent warehousing resources.

[0080] To summarize, the embodiment of the present application determines whether to perform inventory adjustment through the acquired environmental data and the preset environmental data, and then obtains the warehouse utilization evaluation value based on the acquired tobacco inventory management data, and then obtains the calculation complexity evaluation value based on the acquired computing resource data, and finally obtains the optimized effective evaluation value based on the warehouse utilization evaluation value and the calculation complexity evaluation value, thereby realizing the effectiveness of quantitative evaluation optimization measures, and further realizing the improvement of warehouse resource utilization efficiency in the process of dynamic allocation of intelligent warehouse resources, effectively solving the problem of low warehouse resource utilization efficiency in the process of dynamic allocation of intelligent warehouse resources in the prior art.

[0081] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0085] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0086] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning, characterized in that: The following steps are involved: S1, obtaining environmental data of the tobacco storage area through sensors, and determining whether to perform inventory allocation based on the environmental data and preset environmental data; S2, evaluating the storage space according to the acquired tobacco inventory management data to obtain a storage utilization evaluation value, and judging whether to optimize the storage based on the storage utilization evaluation value and a preset storage utilization threshold, wherein the storage utilization evaluation value is used to quantify the space utilization rate of the tobacco storage area; S3, performing a computational complexity evaluation according to the acquired computing resource data to obtain a computational complexity evaluation value, and judging whether to optimize the computational complexity based on the computational complexity evaluation value and a preset computational complexity threshold, wherein the computational complexity evaluation value is used to quantify the computational complexity of the computing task; S4, evaluate and obtain an optimization effective evaluation value based on the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity, and determine whether to provide feedback based on the optimization effective evaluation value and the preset optimization effective threshold. The optimization effective evaluation value is used to evaluate the effectiveness of the optimization measures, and the optimization measures include optimizing the storage and optimizing the computational complexity.

2. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 1, characterized in that: The specific process of obtaining the environmental data is as follows: Acquire corresponding temperature data through temperature sensors deployed at preset monitoring points in the tobacco storage area; The corresponding humidity data is obtained through humidity sensors deployed at preset monitoring points in the tobacco storage area; The sensors include a temperature sensor and a humidity sensor; The environmental data includes temperature data and humidity data.

3. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 2, characterized in that: The specific process of determining whether to perform inventory allocation based on environmental data and preset environmental data is as follows: A1, obtaining preset environmental data from a preset database, wherein the preset environmental data includes a first preset temperature range, a second preset temperature range, a first preset humidity range, and a second preset humidity range; A2, determining whether the temperature data is within the first preset temperature range, if the temperature data is within the first preset temperature range, executing A4, otherwise executing A3; A3, judging whether the temperature data is within the second preset temperature range, if the temperature data is within the second preset temperature range, triggering high-frequency monitoring and notifying the preset personnel, otherwise performing inventory allocation; A4, determining whether the humidity data is within the first preset humidity range, if the humidity data is within the first preset humidity range, then ending the inventory allocation, otherwise executing A5; A5, determine whether the humidity data is within the second preset humidity range. If the humidity data is within the second preset humidity range, trigger high-frequency monitoring and notify the preset personnel, otherwise perform inventory allocation.

4. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 3, characterized in that: The specific process of inventory allocation is as follows: B1, adjusting the environmental control device, determining whether the environmental data obtained after the adjustment is within the first preset environmental data range within the adjustment time period, if the environmental data obtained after the adjustment is within the first preset environmental data range within the adjustment time period, then ending the inventory allocation, otherwise executing B2, the first preset environmental data range includes a first preset temperature range and a first preset humidity range; B2, dispensing tobacco products to a storage area that complies with the first preset environmental data range.

5. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 1, characterized in that: The specific method of evaluating the storage space based on the acquired tobacco inventory management data to obtain the storage utilization evaluation value is as follows: Acquire a preset safety stock quantity and a preset inventory quantity from a preset database, wherein the preset safety stock quantity represents a minimum inventory quantity set according to an enterprise policy; The inventory turnover rate is obtained by analyzing the relative relationship between the tobacco outbound volume and the total inventory volume in the tobacco storage area; The inventory backlog rate is obtained by analyzing the relative relationship between the average tobacco inventory, the preset safety inventory and the total inventory in the tobacco storage area; The proportion of inventory to be adjusted is obtained according to the relative relationship analysis of the preset tobacco inventory, the average tobacco inventory and the total inventory of the tobacco storage area, wherein the proportion of inventory to be adjusted represents the proportion of the adjustment amount set to improve the storage utilization rate; The inventory turnover rate, inventory backlog rate and the proportion of inventory to be adjusted are processed to obtain the warehouse utilization evaluation value; The specific limiting expression of the storage utilization evaluation value is: ； ； ； ； In the formula, i represents the number of the preset time period, , T represents the total number of preset time periods, represents the storage utilization evaluation value of the i-th preset time period, represents the inventory turnover rate in the i-th preset time period, represents the inventory backlog rate in the i-th preset time period, Indicates the proportion of inventory to be adjusted in the i-th preset time period, represents the average tobacco inventory in the i-th preset time period, ZRL represents the total inventory in the tobacco storage area, represents the tobacco delivery volume in the i-th preset time period, Indicates the preset safety stock amount. Indicates the preset tobacco stock level.

6. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 1, characterized in that: The specific process of determining whether to optimize storage based on the storage utilization evaluation value and the preset storage utilization threshold is as follows: C1, determine whether the storage utilization evaluation value is less than the preset storage utilization threshold. If the storage utilization evaluation value is less than the preset storage utilization threshold, execute C2, otherwise do not perform storage optimization; C2, judge whether the inventory to be adjusted is a positive number. If the inventory to be adjusted is a positive number, it will prompt to increase the inventory. If the inventory to be adjusted is a negative number, it will prompt to increase the outbound quantity. If the inventory to be adjusted is zero, execute C3; C3, converting virtual cargo spaces according to the average tobacco inventory and tobacco outbound volume, wherein the virtual cargo spaces are used to realize flexible management of storage space; C4, inventory turnover optimization, the inventory turnover optimization includes batch inventory management and storage location management, the batch inventory management means managing the batch inventory by adopting a first-in-first-out strategy, and the storage location management means storing the tobacco product with the largest outgoing volume at the first location, and storing the tobacco product with the smallest outgoing volume at the second location.

7. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 1, characterized in that: The specific method of performing computational complexity evaluation based on the acquired computing resource data to obtain a computational complexity evaluation value is as follows: Acquire preset computing resource data and preset tobacco types from a preset database, wherein the preset computing resource data includes preset CPU utilization, preset memory utilization, preset I / O read / write rate, and preset algorithm running time; The CPU utilization deviation is obtained according to the relative relationship analysis between the CPU utilization and the preset CPU utilization, wherein the CPU utilization deviation indicates the deviation between the CPU utilization and the preset CPU utilization; Analyzing the relative relationship between the memory utilization and the preset memory utilization to obtain a memory utilization deviation, wherein the memory utilization deviation indicates a deviation between the memory utilization and the preset memory utilization; Analyzing the relative relationship between the I / O read / write rate and the preset I / O read / write rate to obtain an I / O read / write rate deviation, wherein the I / O read / write rate deviation indicates a deviation between the I / O read / write rate and the preset I / O read / write rate; The algorithm running time deviation is obtained according to the relative relationship analysis between the algorithm running time and the preset algorithm running time, wherein the algorithm running time deviation represents the deviation between the algorithm running time and the preset algorithm running time; The space utilization rate of the tobacco storage area is obtained by analyzing the relative relationship between the average tobacco inventory and the total inventory of the tobacco storage area, wherein the space utilization rate represents the proportion of the average tobacco inventory to the total inventory of the tobacco storage area; Obtaining a tobacco type deviation according to a relative relationship analysis between an average tobacco type and a preset tobacco type, wherein the tobacco type deviation represents a deviation between the average tobacco type and the preset tobacco type; The CPU utilization deviation, memory utilization deviation, I / O read / write rate deviation, algorithm running time deviation, space utilization and tobacco type deviation are processed to obtain the computational complexity evaluation value.

8. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 7, characterized in that: The specific process of determining whether to optimize the computational complexity based on the computational complexity evaluation value and the preset computational complexity threshold is as follows: D1, determine whether the computational complexity evaluation value is greater than the preset computational complexity threshold. If the computational complexity evaluation value is greater than the preset computational complexity threshold, execute D2, otherwise the computational complexity is not optimized; D2, judging whether the difference between the computational complexity evaluation value and the preset computational complexity threshold is less than a first threshold, if the difference between the computational complexity evaluation value and the preset computational complexity threshold is not less than the first threshold, executing D3, otherwise performing cache optimization, the cache optimization including adjusting the cache strategy and asynchronous cache update; D3, determine whether the difference between the calculation complexity evaluation value and the preset calculation complexity threshold is less than a second threshold. If the difference between the calculation complexity evaluation value and the preset calculation complexity threshold is not less than the second threshold, perform model pruning, otherwise perform parallel processing.

9. The method for dynamic allocation of intelligent warehousing resources based on deep reinforcement learning as claimed in claim 1, characterized in that: The specific method for evaluating the optimized effective evaluation value based on the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity is as follows: The warehouse utilization evaluation value deviation is obtained by analyzing the relative relationship between the warehouse utilization evaluation value obtained after the warehouse optimization and the warehouse utilization evaluation value obtained before the warehouse optimization, and the warehouse utilization evaluation value deviation represents the difference between the warehouse utilization evaluation value obtained after the warehouse optimization and the warehouse utilization evaluation value obtained before the warehouse optimization; Obtaining a computational complexity evaluation value deviation according to a relative relationship analysis between a computational complexity evaluation value obtained after optimizing the computational complexity and a computational complexity evaluation value obtained before optimizing the computational complexity, wherein the computational complexity evaluation value deviation represents a difference between the computational complexity evaluation value obtained after optimizing the computational complexity and the computational complexity evaluation value obtained before optimizing the computational complexity; The warehouse utilization assessment value deviation and the calculation complex assessment value deviation are processed to obtain the optimized effective assessment value.

10. Intelligent warehousing resource dynamic allocation system based on deep reinforcement learning, characterized by: include: Environmental data judgment module, inventory management evaluation module, computational complexity evaluation module and effectiveness evaluation module; The environmental data judgment module is used to obtain environmental data of the tobacco storage area through sensors, and judge whether to perform inventory allocation based on the environmental data and preset environmental data; The inventory management evaluation module is used to evaluate the storage space according to the acquired tobacco inventory management data to obtain a storage utilization evaluation value, and determine whether to optimize the storage based on the storage utilization evaluation value and a preset storage utilization threshold value, and the storage utilization evaluation value is used to quantify the space utilization rate of the tobacco storage area; The computational complexity evaluation module is used to evaluate the computational complexity according to the acquired computing resource data to obtain a computational complexity evaluation value, and to determine whether to optimize the computational complexity based on the computational complexity evaluation value and a preset computational complexity threshold, wherein the computational complexity evaluation value is used to quantify the computational complexity of the computing task; The effective evaluation module is used to evaluate and obtain an optimized effective evaluation value based on the storage utilization evaluation value obtained after optimizing the storage and the computational complexity evaluation value obtained after optimizing the computational complexity, and to determine whether to provide feedback based on the optimized effective evaluation value and a preset optimized effective threshold. The optimized effective evaluation value is used to evaluate the effectiveness of optimization measures, and the optimization measures include optimizing storage and optimizing computational complexity.

Images