A fully automatic storage bin and control method thereof

Abstract

The present invention provides a fully automatic storage warehouse and a control method thereof, which relate to the field of automation technology and include a hardware system and a software system. The hardware system includes an intelligent transport robot and an environmental control module, and the software system includes a central control system, a path planning module, a task scheduling module, and a visual management module. The central control system controls the intelligent transport robot to execute access instructions according to an optimal path and a collaborative control strategy, and monitors the state and position information of the intelligent transport robot in real time, and dynamically adjusts task allocation and path planning. The central control system predicts and adjusts the optimal storage environmental parameters according to historical data and current status through an environmental parameter prediction model, and sends the environmental parameter instructions to the environmental control module.

Description

Technical Field

[0001] The invention relates to automation technology, and in particular to a full-automatic storage bin and a control method thereof. Background Art

[0002] Traditional biological sample storage systems can no longer meet current requirements in terms of space utilization, operational efficiency, and management accuracy. There are many deficiencies in the existing technology.

[0003] Traditional biological sample storage systems rely heavily on manual operations, which is not only inefficient, but also has high labor costs and is prone to human errors. The lack of a real-time and accurate inventory tracking system leads to inefficient inventory management. Existing biological sample storage systems are difficult to provide precise environmental control for the storage needs of different types of samples, which affects the quality of sample preservation. Fixed biological sample storage equipment is difficult to quickly adapt to the changing biological sample storage needs and lacks flexibility and scalability. Many biological sample storage systems have not yet fully utilized advanced technologies such as the Internet of Things and big data, and their informationization and intelligence levels are insufficient.

[0004] Traditional biological sample storage systems consume a lot of energy in terms of lighting and temperature control, and lack intelligent energy-saving management. In terms of sample access and personnel management, the security measures of traditional biological sample storage systems are often not perfect. In addition, the lack of intelligent prediction functions based on historical data and current status makes it difficult to make reasonable controls in advance. Summary of the invention

[0005] The embodiment of the present invention provides a fully automatic storage bin and a control method thereof, which can solve the problems in the prior art. The method includes:

[0006] Hardware system and software system, the hardware system includes an intelligent handling robot and an environmental control module, and the software system includes a central control system, a path planning module, a task scheduling module, and a visual management module; the intelligent handling robot is equipped with a lifting device, a telescopic arm and a grasping mechanism, and is installed with a radio frequency identification reader; the environmental control module includes a temperature and humidity sensor, a light sensor, and an environmental parameter adjustment device;

[0007] The central control system exchanges data with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition; the path planning module generates the optimal path based on the current storage status and the decomposed tasks based on the deep learning algorithm; the task scheduling module adopts the multi-agent reinforcement learning method and generates the collaborative control strategy of the intelligent handling robot based on the distributed decision-making mechanism; the visual management module displays the operating status in real time; the central control system controls the intelligent handling robot to execute access instructions based on the optimal path and collaborative control strategy, and monitors the status and location information of the intelligent handling robot in real time, and dynamically adjusts task allocation and path planning;

[0008] The central control system uses the environmental parameter prediction model to predict and adjust the optimal stored environmental parameters based on historical data and current status, and sends the environmental parameter instructions to the environmental control module; the central control system collects operating data in real time, iteratively updates the collaborative control strategy based on the operating data, and generates a performance report that is transmitted to the visualization management module.

[0009] In an optional embodiment,

[0010] The central control system exchanges data with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition in the following steps:

[0011] The central control system parses the access instruction into basic operation units through a finite state machine, wherein the basic operation units include access operations, move operations, and placement operations;

[0012] The basic operation units are input into a dynamic programming algorithm to generate a subtask sequence. The dynamic programming algorithm aims to minimize the total execution time and calculates the minimum execution time when the first several basic operation units are completed and the intelligent handling robot is located at a specific position through a state transition equation;

[0013] Optimizing the subtask sequence using a heuristic rule set, the heuristic rule set includes proximity principle, load balancing and batch optimization; wherein the proximity principle is implemented by a KD tree data structure, load balancing is implemented by a minimum heap, and batch optimization uses a clustering algorithm to group spatially adjacent tasks into batch tasks;

[0014] According to the optimized sub-task sequence, each intelligent handling robot is assigned a task;

[0015] Through machine learning methods, the weights of each rule in the heuristic rule set are dynamically adjusted based on historical performance data;

[0016] During the task execution process, when a new access instruction is detected, the task decomposition process is re-triggered, and the newly generated sub-task sequence is integrated with the task being executed, and the task is re-assigned to each intelligent handling robot.

[0017] In an optional embodiment,

[0018] The path planning module is based on a deep learning algorithm. The steps to generate the optimal path according to the current storage state and the decomposed tasks include:

[0019] Construct a three-dimensional discrete matrix environment model to represent the matrix IoT fully automatic storage warehouse environment as a state space, including the current three-dimensional coordinates of the intelligent handling robot, the three-dimensional coordinates of the target position, the task type, the load status, the RFID reading status, the height of the lifting device, the telescopic arm status, and the grasping mechanism status;

[0020] Design the action space, including the moving direction, lifting device operation, telescopic arm operation and grasping mechanism operation of the intelligent handling robot;

[0021] Based on the three-dimensional discrete matrix environment model and action space, a deep Q network is constructed, including an input layer, two convolutional layers, two fully connected layers and an output layer. The input layer encodes the state space into a vector, the two convolutional layers use 32 and 64 3x3x3 convolution kernels respectively, the two fully connected layers contain 256 and 128 neurons respectively, and the output layer corresponds to all possible actions;

[0022] Design reward functions, including fixed step penalty, target reaching reward, collision penalty, and reward and penalty based on 3D distance change;

[0023] Implement the experience replay mechanism and store the state, action, reward, next state and termination flag in the experience replay buffer;

[0024] Introduced a priority experience replay mechanism, using the absolute value of the TD error as the priority;

[0025] The ε-greedy strategy is used for exploration, and the ε value decays linearly from 1.0 to 0.1;

[0026] The deep Q network was trained using the Adam optimizer with a learning rate of 0.0001 and a discount factor of 0.99;

[0027] Update the target network regularly, with an update frequency of every 1000 steps;

[0028] Iterative training optimizes the parameters of the deep Q network;

[0029] Use the optimized deep Q network to generate the optimal path based on the current storage status, intelligent handling robot status and task requirements.

[0030] In an optional embodiment,

[0031] The reward function is:

[0032] ;

[0033] in,

[0034] R(s,a,s') represents the reward function, a represents the action performed in the current state, s represents the current state, and s' represents the next state;

[0035] α represents a fixed step penalty;

[0036] β represents the reward for reaching the target, which is a positive reward given when the agent reaches the target position;

[0037] γ represents the collision penalty, which is the negative reward given when the agent collides;

[0038] δ represents the distance change coefficient, which is used to give rewards or penalties according to the change in the distance between the agent and the target;

[0039] I(s'=goal) represents the indicator function. When s' is equal to the target state goal, the function value is 1, otherwise it is 0;

[0040] I (collision) represents the indicator function, the function value is 1 when a collision occurs, otherwise it is 0;

[0041] collision indicates a collision event;

[0042] d(s,goal) represents the Euclidean distance from the current state s to the target state goal;

[0043] d(s',goal) represents the Euclidean distance from the next state s' to the target state goal.

[0044] In an optional embodiment,

[0045] The task scheduling module adopts a multi-agent reinforcement learning method. The steps of generating a collaborative control strategy for an intelligent handling robot based on a distributed decision-making mechanism include:

[0046] The intelligent handling robot is defined as an agent; the improved multi-agent deep deterministic policy gradient algorithm is constructed, including: introducing the attention mechanism, using the graph attention network to encode the observation of each agent into a graph structure, where the nodes represent the robot and the task, and the edges represent the relationship between the robot and the task; designing a hierarchical policy network structure, including a high-level policy network responsible for generating long-term goals and a low-level policy network responsible for generating specific actions; using a TD-error-based priority experience replay mechanism to dynamically adjust the sampling probability according to the importance of experience; using a multi-step bootstrap method to update the Q value through n-step returns to balance the deviation and variance; initializing the actor network, the critic network and its target network, and initializing the experience replay buffer;

[0047] In each training round, the agent selects actions based on the current strategy and exploration noise, executes actions and observes rewards and new states; stores state transitions to the experience replay buffer, and when the buffer size exceeds the batch size, samples a small batch from it for learning; calculates the target Q value, updates the critic network to minimize the loss function, and updates the actor network using policy gradients; updates the target network, and repeats the training process until the termination condition is met.

[0048] In an optional embodiment,

[0049] The method further comprises:

[0050] Use the improved multi-agent deep deterministic policy gradient algorithm to train the agents and generate collaborative control strategies;

[0051] Based on the generated collaborative control strategy, the following steps are performed: each intelligent transport robot observes its own status and information about the surrounding environment, and obtains information including location, power level, load status, and nearby tasks and other robots; uses the trained high-level policy network to process the observed information and generate long-term goals, including selecting the next task to be processed or determining the moving area; based on the generated long-term goals and current status, uses the trained low-level policy network to generate specific action instructions; through the trained attention mechanism, each intelligent transport robot pays attention to important information about other robots and tasks to achieve collaborative decision-making; the intelligent transport robot executes the generated action instructions and obtains the new status and reward information returned by the environment; the execution results are fed back to the collaborative control strategy to continuously improve the decision-making process.

[0052] In an optional embodiment,

[0053] The steps of the central control system predicting and adjusting the optimal stored environmental parameters based on historical data and current status through the environmental parameter prediction model and sending the environmental parameter instructions to the environmental control module include:

[0054] Constructing an environmental parameter prediction model based on a long short-term memory network, wherein the long short-term memory network includes an input layer, a long short-term memory layer, a fully connected layer, and an output layer;

[0055] Real-time collection of environmental parameter data, including temperature, humidity and light data;

[0056] Inputting the collected environmental parameter data into the environmental parameter prediction model to obtain predicted environmental parameter values;

[0057] Adopt online learning strategy, dynamically adjust the updating frequency of environmental parameter model according to prediction error, and use collected data to update environmental parameter prediction model;

[0058] The Bayesian optimization method is introduced, and the Gaussian process regression model is used to establish the relationship between environmental parameters and the performance of the central control system. The optimal environmental parameter configuration is selected by maximizing the acquisition function, and the optimal environmental parameter configuration instruction is sent to the environmental control module for execution.

[0059] In an optional embodiment,

[0060] The hardware system also includes:

[0061] The storage device includes internal storage and sorting equipment; the storage device is a storage box with adjustable temperature, humidity and light intensity, and the internal storage and sorting equipment includes multiple storage trays, a storage tray lifting platform, a lifting screw and a clamp; the storage tray lifting platform adopts a sandwich structure, with a bottom plate, a middle plate and a top plate, and the multifunctional clamp is equipped with a sensor.

[0062] The innovative design of a fully automatic storage warehouse and its control of the present invention brings significant benefits. First of all, its multi-layer storage unit makes full use of the storage space and greatly improves the storage density. The intelligent handling robot equipped with a lifting device, a telescopic arm and a grasping mechanism, combined with radio frequency identification technology, realizes efficient fully automatic access operations. This not only greatly shortens the access time of goods, but also significantly improves the accuracy of inventory management. The combination of the environmental control module and the prediction model can dynamically adjust the optimal storage environment according to the characteristics of the goods, which not only improves the quality of biological sample preservation, but also optimizes energy use. The path planning based on deep learning and the task scheduling of multi-agent reinforcement learning have made a qualitative leap in the operating efficiency of the system, greatly reduced the robot's idle running time, and improved the efficiency of collaborative work.

[0063] The adaptive ability and continuous optimization ability of the present invention are also its outstanding advantages. The central control system can monitor and dynamically adjust task allocation and path planning in real time, so that the system has a strong adaptability and can quickly adapt to emergencies and changes in demand. Through continuous data collection and iterative updates, the system's collaborative control strategy can be continuously optimized, and performance continues to improve over time. The visual management module provides an intuitive real-time operating status display and performance report, which greatly improves management efficiency and decision-making speed. In terms of safety, automated operations reduce human errors, and the intelligent monitoring system detects potential risks in a timely manner. The manpower demand is greatly reduced, the energy utilization efficiency is improved, and the overall operational efficiency is improved, and the long-term operating cost of the system has been significantly reduced.

[0064] The modular design of the present invention gives it good scalability, and the capacity can be adjusted or the function upgraded quickly according to the needs, which greatly shortens the expansion time. In terms of environmental protection, intelligent operation and optimized energy use significantly reduce carbon emissions and make positive contributions to environmental protection. The large amount of operating data generated by the system provides enterprises with valuable decision-making basis. Ultimately, fast and accurate access to samples is achieved, reducing the error rate to an extremely low level. These combined effects not only greatly improve the efficiency, accuracy and reliability of the biological sample storage system, but also significantly reduce operating costs, creating considerable economic benefits for enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of a flow chart of a fully automatic storage bin and a control method thereof according to an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the structure of a storage device according to an embodiment of the present invention;

[0067] Figure 3 It is a schematic diagram of the structure of the internal storage and sorting equipment according to an embodiment of the present invention.

[0068] Among them, 2-1 is the hatch; 3-1 is the X-axis moving platform, 3-2 is the Z-axis moving platform, 3-3 is the Y-axis moving platform, 3-4 is the lifting screw, 3-5 is the storage disk lifting platform, 3-6 is the clamp, 3-7 is the access platform, and 3-8 is the storage disk. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0071] Figure 1 FIG. 1 is a flow chart of a fully automatic storage bin and a control method thereof according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0072] S1. Hardware system and software system. The hardware system includes an intelligent handling robot and an environmental control module. The software system includes a central control system, a path planning module, a task scheduling module, and a visual management module. The intelligent handling robot is equipped with a lifting device, a telescopic arm and a grasping mechanism, and is equipped with a radio frequency identification reader. The environmental control module includes a temperature and humidity sensor, a light sensor, and an environmental parameter adjustment device.

[0073] S2. The central control system exchanges data with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition; the path planning module generates the optimal path based on the current storage status and the decomposed tasks based on the deep learning algorithm; the task scheduling module adopts the multi-agent reinforcement learning method and generates the collaborative control strategy of the intelligent handling robot based on the distributed decision-making mechanism; the visual management module displays the operating status in real time; the central control system controls the intelligent handling robot to execute the access instructions based on the optimal path and collaborative control strategy, and monitors the status and location information of the intelligent handling robot in real time, and dynamically adjusts the task allocation and path planning;

[0074] S3. The central control system predicts and adjusts the optimal stored environmental parameters based on historical data and current status through the environmental parameter prediction model, and sends the environmental parameter instructions to the environmental control module; the central control system collects operating data in real time, iteratively updates the collaborative control strategy based on the operating data, and generates a performance report that is transmitted to the visualization management module.

[0075] Exemplarily, the specific implementation of a fully automatic storage bin and control method thereof of the present invention is as follows:

[0076] First, build a hardware system. The hardware system includes storage equipment, intelligent handling robots and environmental control modules; the storage equipment includes internal storage and sorting equipment, which is a storage box with adjustable temperature, humidity and light intensity. The internal storage and sorting equipment includes multiple storage trays, storage tray lifting platforms, lifting screws and clamps; the storage tray lifting platform adopts a sandwich structure with a bottom plate, a middle plate and a top plate. Each storage tray and biological sample is equipped with an RFID tag for storing biological sample information; the multifunctional clamp is equipped with a sensor. The intelligent handling robot adopts a four-wheel drive structure, equipped with a lifting device that can move up and down 2 meters, the maximum telescopic length of the telescopic arm is 1.5 meters, and the maximum load-bearing capacity of the grasping mechanism is 50kg. The robot is equipped with an RFID reader with a reading distance of 0.5 meters. The environmental control module includes a temperature sensor (measuring range -20~50℃, accuracy ±0.5℃), a humidity sensor (measuring range 0~100%RH, accuracy ±3%RH), and a light sensor (measuring range 0~200000lux, accuracy ±5%). Environmental parameter adjustment equipment includes cooling and heating all-in-one machines, humidifiers, dehumidifiers and LED lighting.

[0077] Then develop the software system. The central control system adopts a distributed architecture, consisting of a master server and several edge computing nodes. The wireless communication network adopts 5G technology with a maximum transmission rate of 1Gbps. The path planning module is based on a deep reinforcement learning algorithm, using a convolutional neural network to extract environmental features, and the policy network and value network to output actions. The task scheduling module adopts a multi-agent reinforcement learning method to generate a collaborative control strategy based on a distributed decision-making mechanism. The visualization management module uses Web technology to develop a 3D visualization interface.

[0078] When the system is running, it first receives access instructions. For example, when receiving the instruction "store 100 samples", the central control system decomposes it into 100 sample storage tasks. The path planning module uses a deep reinforcement learning algorithm to generate the optimal path based on the current storage state (800 units have been stored) and the decomposed tasks. Specifically, the storage rack state is represented as a three-dimensional matrix of 100x100x10, and each element represents the state of the corresponding storage unit (0 means idle, 1 means occupied). It is input into the convolutional neural network to extract features, combined with task information, and the output of the policy network and value network is obtained through the fully connected layer. The policy network outputs the action probability distribution, and the value network evaluates the state value. Through the Monte Carlo tree search algorithm, the optimal action sequence is selected as the path planning result.

[0079] The task scheduling module uses a multi-agent reinforcement learning method to generate a collaborative control strategy based on the path planning results. Five intelligent handling robots are used as agents, and the state of each agent includes information such as its own position, power, and current task. The action space includes basic operations such as moving, grabbing, and placing. The reward function considers factors such as task completion time and energy consumption. Through the distributed Q learning algorithm, each agent independently learns the optimal strategy and shares experience through communication. Finally, a collaborative control strategy is obtained to guide multiple robots to collaborate to complete the task.

[0080] The central control system sends control instructions to the intelligent handling robots based on the optimal path and collaborative control strategy. For example, it instructs robot 1 to move to position (5,6,2) to grab the sample, and then move to position (8,9,5) to place the sample. At the same time, it monitors the robot status and position information in real time. If it is found that robot 1 is low on power, it dynamically adjusts the task allocation and assigns the remaining tasks to other robots.

[0081] For environmental control, the central control system uses LSTM neural network to build an environmental parameter prediction model. The input includes temperature, humidity, and light data for the past 24 hours, as well as weather forecast data for the next 24 hours. The output is the predicted values ​​of temperature, humidity, and light for the next 24 hours. Based on the prediction results and the optimal storage conditions for different samples, environmental parameter adjustment instructions are generated. For example, if it is predicted that the temperature will rise to 28°C in 6 hours, and the optimal storage temperature of a certain biological sample is 0~4°C, the refrigeration system will be started 5 hours in advance to adjust the temperature to 2°C.

[0082] The central control system collects operation data in real time, including indicators such as task completion time, energy consumption, and error rate. Based on these data, the policy gradient algorithm is used to iteratively update the collaborative control strategy. Specifically, the difference between the cumulative reward and the baseline value is calculated as the advantage function, which is used to update the policy network parameters. At the same time, a performance report is generated, including indicators such as the average daily task processing volume, average task completion time, and equipment utilization, which are transmitted to the visualization management module for display.

[0083] Through the above method, the intelligent operation and management of the fully automatic storage warehouse is realized, and the access efficiency and resource utilization are improved. This method can be widely used in various sample storage scenarios and has good practical value and promotion prospects.

[0084] In an optional embodiment,

[0085] The central control system exchanges data with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition in the following steps:

[0086] The central control system parses the access instruction into basic operation units through a finite state machine, wherein the basic operation units include access operations, move operations, and placement operations;

[0087] The basic operation units are input into a dynamic programming algorithm to generate a subtask sequence. The dynamic programming algorithm aims to minimize the total execution time and calculates the minimum execution time when the first several basic operation units are completed and the intelligent handling robot is located at a specific position through a state transition equation;

[0088] Optimizing the subtask sequence using a heuristic rule set, the heuristic rule set includes proximity principle, load balancing and batch optimization; wherein the proximity principle is implemented by a KD tree data structure, load balancing is implemented by a minimum heap, and batch optimization uses a clustering algorithm to group spatially adjacent tasks into batch tasks;

[0089] According to the optimized sub-task sequence, each intelligent handling robot is assigned a task;

[0090] Through machine learning methods, the weights of each rule in the heuristic rule set are dynamically adjusted based on historical performance data;

[0091] During the task execution process, when a new access instruction is detected, the task decomposition process is re-triggered, and the newly generated sub-task sequence is integrated with the task being executed, and the task is re-assigned to each intelligent handling robot.

[0092] For example, the central control system interacts with the hardware system in real time through a wireless communication network, receives access instructions and performs task decomposition, which is a key step in achieving efficient and automated biological sample access. This process involves multiple complex technical steps, each of which is carefully designed to ensure the efficiency and reliability of the system.

[0093] First, the central control system uses a finite state machine model to parse access instructions. This method can convert complex instructions into a series of basic operation units, including access operations, movement operations, and placement operations. For example, an instruction to "take out samples from shelf A3 and send them to exit B7" can be parsed into basic operation units such as moving to shelf A3, taking out samples, moving to exit B7, and placing samples. This parsing method enables the system to flexibly handle various complex access requirements.

[0094] After the analysis is completed, the system inputs these basic operation units into the dynamic programming algorithm. The algorithm aims to minimize the total execution time and calculates the minimum execution time when the first several basic operation units are completed and the intelligent handling robot is located at a specific position through the state transition equation. For example, if there are 10 basic operation units, the algorithm will calculate the minimum execution time when the first 1, first 2, ..., first 10 operation units are completed, and record the optimal choice for each step. This method can effectively reduce the computational complexity while considering the global optimum.

[0095] In order to further optimize the sequence of subtasks, the system introduced a set of heuristic rule sets. This set of rules includes the proximity principle, load balancing, and batch optimization. The proximity principle is implemented through the KD tree data structure, which can quickly find the task closest to the current location. For example, if a robot has just completed the task of shelf A3, the system will give priority to assigning it tasks near shelves A2 or A4, rather than tasks far away. Load balancing is achieved through a minimum heap to ensure that the workload of each robot is relatively balanced. Batch optimization uses a clustering algorithm to combine spatially adjacent tasks into batch tasks. For example, if there are multiple pickup tasks in area A, the system will combine these tasks and have one robot complete them at one time, reducing the time for repeated round trips.

[0096] Based on the optimized subtask sequence, the central control system assigns tasks to each intelligent handling robot. The assignment process takes into account factors such as the robot's current position, power status, and load capacity to ensure the rationality and executability of task assignment.

[0097] The system also introduces machine learning methods to dynamically adjust the weights of each rule in the heuristic rule set based on historical performance data. By analyzing the task execution data over a period of time (such as a week or a month), the system can identify which rules are more effective in actual operation and adjust their weights accordingly. For example, if it is found that batch optimization works particularly well during certain periods, the system will increase the weight of batch optimization rules during these periods.

[0098] During the task execution, when the system detects a new access instruction, it will re-trigger the task decomposition process. The newly generated subtask sequence will be integrated with the task being executed, and the system will re-assign tasks to each intelligent handling robot. This dynamic adjustment mechanism ensures that the system can respond to new requirements in real time while maintaining overall operating efficiency.

[0099] For example, suppose the system is executing a batch of tasks, in which robot A is heading to shelf A5 to pick up goods. At this time, the system receives an urgent order to pick up goods from shelf A4. The system will immediately re-plan the tasks and may assign the urgent task of shelf A4 to robot A because it is closest to shelf A4, while the task originally assigned to shelf A5 may be reassigned to other robots or postponed.

[0100] This method of task decomposition and dynamic adjustment has brought significant beneficial effects. It has greatly improved the operating efficiency of the warehouse system, reduced the idle time of robots, and improved resource utilization. At the same time, the real-time response capability and adaptability of the system have been significantly enhanced, and it can better cope with the complex and changing warehouse environment. In addition, through machine learning to continuously optimize decision rules, the performance of the system will continue to improve as the running time increases, achieving self-evolution and continuous optimization.

[0101] In an optional embodiment,

[0102] The path planning module is based on a deep learning algorithm. The steps to generate the optimal path according to the current storage state and the decomposed tasks include:

[0103] Construct a three-dimensional discrete matrix environment model to represent the matrix IoT fully automatic storage warehouse environment as a state space, including the current three-dimensional coordinates of the intelligent handling robot, the three-dimensional coordinates of the target position, the task type, the load status, the RFID reading status, the height of the lifting device, the telescopic arm status, and the grasping mechanism status;

[0104] Design the action space, including the moving direction, lifting device operation, telescopic arm operation and grasping mechanism operation of the intelligent handling robot;

[0105] Based on the three-dimensional discrete matrix environment model and action space, a deep Q network is constructed, including an input layer, two convolutional layers, two fully connected layers and an output layer. The input layer encodes the state space into a vector, the two convolutional layers use 32 and 64 3x3x3 convolution kernels respectively, the two fully connected layers contain 256 and 128 neurons respectively, and the output layer corresponds to all possible actions;

[0106] Design reward functions, including fixed step penalty, target reaching reward, collision penalty, and reward and penalty based on 3D distance change;

[0107] Implement the experience replay mechanism and store the state, action, reward, next state and termination flag in the experience replay buffer;

[0108] Introduced a priority experience replay mechanism, using the absolute value of the TD error as the priority;

[0109] The ε-greedy strategy is used for exploration, and the ε value decays linearly from 1.0 to 0.1;

[0110] The deep Q network was trained using the Adam optimizer with a learning rate of 0.0001 and a discount factor of 0.99;

[0111] Update the target network regularly, with an update frequency of every 1000 steps;

[0112] Iterative training optimizes the parameters of the deep Q network;

[0113] Use the optimized deep Q network to generate the optimal path based on the current storage status, intelligent handling robot status and task requirements.

[0114] Exemplarily, the specific implementation method of the path planning module based on the deep learning algorithm to generate the optimal path according to the current storage state and the decomposed tasks is as follows:

[0115] First, a three-dimensional discrete matrix environment model is constructed to represent the fully automatic storage warehouse environment as a state space. The state space includes the current three-dimensional coordinates (x, y, z) of the intelligent handling robot, the three-dimensional coordinates of the target position (x_target, y_target, z_target), the task type (sample taking / sample placing), the load state (sample present / sample absent), the RFID reading state (read / unread), the height h of the lifting device, the telescopic arm state (extended / retracted), and the gripping mechanism state (grasped / released). For example, the state at a certain moment can be represented as (3, 5, 2, 8, 9, 4, sample taking, sample absent, unread, 1.5, retracted, released).

[0116] Next, we design the action space, including the moving direction of the intelligent handling robot (forward, backward, left, right, up, down), the operation of the lifting device (up, down), the operation of the telescopic arm (extend, retract), and the operation of the grasping mechanism (grasp, release). For example, an action can be expressed as (forward, up, extend, grasp).

[0117] Based on the above three-dimensional discrete matrix environment model and action space, a deep Q network is constructed. The network includes an input layer, two convolutional layers, two fully connected layers and an output layer. The input layer encodes the state space into a vector, for example, the above state is encoded as (3,5,2,8,9,4,0,0,0,1.5,0,0). The two convolutional layers use 32 and 64 3x3x3 convolution kernels respectively to extract spatial features. The two fully connected layers contain 256 and 128 neurons respectively for further feature extraction. The output layer corresponds to all possible actions, for example, 24 output neurons correspond to 24 possible action combinations.

[0118] Design a reward function, including a fixed step penalty (-0.1), a goal-reaching reward (+10), a collision penalty (-5), and rewards and penalties based on changes in 3D distance. For example, if the 3D distance decreases by 0.5 meters after a certain step, a reward of +0.5 is given; if it increases by 0.3 meters, a penalty of -0.3 is given.

[0119] Implement the experience replay mechanism and store the state, action, reward, next state and termination flag in the experience replay buffer. For example, an experience can be represented as ((3,5,2,8,9,4,0,0,0,1.5,0,0),(forward, rise, reach out, grab),0.5,(4,5,3,8,9,4,0,0,0,2.0,1,1),False).

[0120] Introduce a priority experience playback mechanism, using the absolute value of the TD error as the priority. For example, if the TD error of an experience is 0.8, its priority is 0.8. Experiences with higher priorities are sampled more frequently for training.

[0121] The ε-greedy strategy is used for exploration, and the ε value decays linearly from 1.0 to 0.1. For example, in the early stage of training, ε=0.9, there is a 90% probability of randomly selecting an action and a 10% probability of selecting the action with the largest Q value; in the late stage of training, ε=0.1, there is a 10% probability of randomly selecting an action and a 90% probability of selecting the action with the largest Q value.

[0122] The deep Q network is trained using the Adam optimizer, with a learning rate of 0.0001 and a discount factor of 0.99. Each time 32 experiences are sampled from the experience replay buffer for a parameter update.

[0123] Update the target network regularly, with an update frequency of every 1000 steps. That is, copy the parameters of the current network to the target network every 1000 steps.

[0124] Through the above steps, iterative training is performed to optimize the parameters of the deep Q network. For example, training is performed for 1 million steps, and the performance is evaluated every 1,000 steps. The training is terminated when the average reward of 10 consecutive evaluations exceeds a certain threshold.

[0125] Finally, the optimized deep Q network is used to generate the optimal path according to the current storage state, the state of the intelligent handling robot and the task requirements. Specifically, the current state is input into the network, the action with the largest Q value is selected for execution, and then the state is updated. This process is repeated until the target position is reached, thereby obtaining the optimal path sequence.

[0126] For example, for the task of moving from (3,5,2) to (8,9,4), the optimal path generated may be: ((3,5,2), (4,5,2), (5,6,2), (6,7,3), (7,8,3), (8,9,4)). Each coordinate corresponds to an action, such as (move forward, stay still, stay still), (move forward, turn right, stay still), (move forward, turn right, stay still), (move forward, stay still, rise), etc.

[0127] Through the above method, the path planning module can generate the optimal path for the intelligent handling robot based on the current sample storage status and task requirements using the deep reinforcement learning algorithm, thereby achieving efficient and flexible path planning.

[0128] In an optional embodiment,

[0129] The reward function is:

[0130] ;

[0131] in,

[0132] R(s,a,s') represents the reward function, a represents the action performed in the current state, s represents the current state, and s' represents the next state;

[0133] α represents a fixed step penalty;

[0134] β represents the reward for reaching the target, which is a positive reward given when the agent reaches the target position;

[0135] γ represents the collision penalty, which is the negative reward given when the agent collides;

[0136] δ represents the distance change coefficient, which is used to give rewards or penalties according to the change in the distance between the agent and the target;

[0137] I(s'=goal) represents the indicator function. When s' is equal to the target state goal, the function value is 1, otherwise it is 0;

[0138] I (collision) represents the indicator function, the function value is 1 when a collision occurs, otherwise it is 0;

[0139] collision indicates a collision event;

[0140] d(s,goal) represents the Euclidean distance from the current state s to the target state goal;

[0141] d(s',goal) represents the Euclidean distance from the next state s' to the target state goal.

[0142] For example, in order to achieve effective navigation of the agent, a reward function is designed to guide the learning process of the agent. The design of the reward function is a key link in the reinforcement learning algorithm, which directly affects the learning effect and final performance of the agent. For the robot navigation task in the biological sample storage system, a new reward function is designed, which comprehensively considers multiple factors to guide the robot to complete the navigation task efficiently.

[0143] First, a fixed step penalty is introduced into the reward function. This penalty term is a negative value, and its purpose is to encourage the robot to reach the target position with the least number of steps. Every time an action is performed, whether it is effective or not, it will be subject to this fixed penalty. For example, this penalty value can be set to -0.1. In this way, even if the robot does not collide or is not close to the target, it will be slightly punished for taking action, thereby encouraging it to complete the task as soon as possible.

[0144] Second, the reward function includes a reward for reaching the goal. When the robot successfully reaches the goal position, it will receive a large positive reward. This reward value should be large enough to balance the accumulated step penalty and possible collision penalty. For example, this reward value can be set to 100. In this way, even if the robot takes many steps before reaching the goal or has a small number of collisions, it is still "worth it" to successfully reach the goal in the end.

[0145] In addition, the reward function introduces a collision penalty. When the robot collides with an obstacle, it will receive a large negative reward. This penalty value should be large enough to strongly inhibit collision behavior. For example, the collision penalty can be set to -50. This penalty is large enough that even if the goal is reached immediately after the collision, the total reward is still negative, which effectively inhibits collision behavior.

[0146] Finally, a reward and penalty based on the change in three-dimensional distance is added to the reward function. This term calculates the change in distance between the robot and the target before and after the action is performed. If the distance decreases, the robot will receive a positive reward; if the distance increases, it will be negatively penalized. The size of this reward and penalty value is proportional to the degree of distance change. For example, this coefficient can be set to 0.5. If an action brings the robot closer to the target by 1 unit distance, it will receive a reward of 0.5; conversely, if it moves away by 1 unit distance, it will be penalized by 0.5.

[0147] In practical applications, these parameter values ​​need to be adjusted according to the specific environment and task requirements. For example, in a 10x10x10 3D grid world, suppose the robot starts from (0,0,0) and the goal is (9,9,9). If the robot moves to (1,1,1), it will get a reward of -0.1 (step penalty) + 0.5 * sqrt(3) (distance reward) ≈ 0.77. If it accidentally hits an obstacle, the total reward will become -49.23. If it reaches the goal directly, the total reward will be 100-0.1*27 (assuming the shortest path takes 27 steps) + 0.5*sqrt(243) = 92.1.

[0148] The beneficial effects of this reward function design are mainly reflected in the following aspects: First, it encourages the robot to take the shortest path to reach the target, improving navigation efficiency. Second, by introducing a larger collision penalty, collision behavior is effectively avoided and navigation safety is improved. Third, the reward and punishment mechanism based on distance changes provides fine-grained feedback for the robot, enabling it to gradually approach the target in a complex environment. Finally, this design balances the three aspects of efficiency, safety, and goal orientation, so that the learned strategy can complete the task quickly while ensuring safety, and is suitable for actual biological sample storage environments. Through this carefully designed reward function, the reinforcement learning algorithm can better guide the robot to learn efficient and safe navigation strategies, thereby improving the operating efficiency and reliability of the entire fully automatic warehousing system.

[0149] In an optional embodiment,

[0150] The task scheduling module adopts a multi-agent reinforcement learning method. The steps of generating a collaborative control strategy for an intelligent handling robot based on a distributed decision-making mechanism include:

[0151] The intelligent handling robot is defined as an agent; the improved multi-agent deep deterministic policy gradient algorithm is constructed, including: introducing the attention mechanism, using the graph attention network to encode the observation of each agent into a graph structure, where the nodes represent the robot and the task, and the edges represent the relationship between the robot and the task; designing a hierarchical policy network structure, including a high-level policy network responsible for generating long-term goals and a low-level policy network responsible for generating specific actions; using a TD-error-based priority experience replay mechanism to dynamically adjust the sampling probability according to the importance of experience; using a multi-step bootstrap method to update the Q value through n-step returns to balance the deviation and variance; initializing the actor network, the critic network and its target network, and initializing the experience replay buffer;

[0152] In each training round, the agent selects actions based on the current strategy and exploration noise, executes actions and observes rewards and new states; stores state transitions to the experience replay buffer, and when the buffer size exceeds the batch size, samples a small batch from it for learning; calculates the target Q value, updates the critic network to minimize the loss function, and updates the actor network using policy gradients; updates the target network, and repeats the training process until the termination condition is met.

[0153] Exemplarily, the task scheduling module uses a multi-agent reinforcement learning method to generate a collaborative control strategy for the intelligent handling robot based on a distributed decision-making mechanism. The specific implementation steps are as follows:

[0154] First, the intelligent handling robot is defined as an intelligent agent. Each intelligent handling robot, as an independent intelligent agent, can perceive the environment, make decisions and perform actions.

[0155] Then, an improved multi-agent deep deterministic policy gradient algorithm is constructed. The algorithm contains the following key improvements:

[0156] The attention mechanism is introduced, and the graph attention network is used to encode the observation of each agent into a graph structure. Specifically, robots and tasks are defined as nodes in the graph, and the relationship between robots and tasks is defined as edges. For example, for a scenario with 3 robots and 5 tasks, a graph containing 8 nodes can be constructed, where 3 nodes represent robots and 5 nodes represent tasks. The edges between nodes represent the tasks that the robot can perform. The graph attention network can adaptively learn the importance of the relationship between nodes, thereby extracting more effective feature representations.

[0157] Design a hierarchical policy network structure, including a high-level policy network and a low-level policy network. The high-level policy network is responsible for generating long-term goals, such as task allocation plans; the low-level policy network is responsible for generating specific actions, such as moving, grasping, etc. This hierarchical structure can better handle the trade-off between long-term planning and short-term execution.

[0158] A TD-error-based priority experience playback mechanism is used to dynamically adjust the sampling probability according to the importance of the experience. The specific approach is to calculate the TD-error of each experience and use it as the sampling weight. Experiences with larger TD-errors indicate that the model prediction error is larger and has higher learning value, so they are given a higher sampling probability. For example, the TD-error can be normalized to the [0,1] interval as the sampling probability.

[0159] Use a multi-step bootstrap method to update the Q value through n-step rewards to balance bias and variance. Traditional single-step TD learning may have high bias problems, while Monte Carlo methods may have high variance problems. The multi-step bootstrap method can strike a balance between the two. For example, you can choose n=5, that is, use the cumulative rewards of the next 5 steps plus the estimated Q value of the state in the 5th step to update the Q value of the current state.

[0160] Next, initialize the actor network, the critic network and its target network, and initialize the experience replay buffer. The actor network is used to generate actions, and the critic network is used to evaluate the value of state-action pairs. The target network is a lagged version of the actor and critic networks, used to provide a stable learning target. The experience replay buffer is used to store the experience of the agent's interaction with the environment, and can be set to a fixed size, such as 10,000.

[0161] In each training round, the agent selects actions based on the current strategy and exploration noise. You can use the ε-greedy strategy to randomly select actions with probability ε and select the action that the current strategy considers optimal with probability 1-ε. For example, you can set the initial value of ε to 0.9 and linearly decay it to 0.1 during training.

[0162] The agent performs the selected action and observes the reward and new state returned by the environment. The reward function can be designed based on factors such as task completion, energy consumption, time, etc. For example, completing a task can get a reward of +10, each step of movement will deduct a reward of -0.1, and colliding with an obstacle will deduct a reward of -5.

[0163] Store state transitions (current state, action, reward, next state) in the experience replay buffer. When the buffer size exceeds the batch size, sample a small batch from it for learning. The batch size can be set to 64.

[0164] Calculate the target Q value. Use the target actor network to generate the action for the next state, then use the target critic network to evaluate the Q value of the state-action pair, plus the immediate reward, to get the target Q value.

[0165] Update the critic network to minimize the loss function. The loss function can be defined as the mean squared error between the target Q value and the current Q value prediction. Use an optimization algorithm such as stochastic gradient descent to update the critic network parameters.

[0166] Use policy gradient to update the actor network. Specifically, fix the critic network, calculate the gradient of the Q value of the actor network output action to the actor network parameters, and then update the actor network parameters along the gradient direction to maximize the Q value.

[0167] Update the target network. You can use the soft update method, which is to slowly move the target network parameters toward the current network parameters.

[0168] Repeat the above training process until the termination condition is met. The termination condition can be reaching a preset number of training rounds (such as 10,000 rounds) or the average reward reaches a certain threshold (such as 100).

[0169] Through the above steps, a multi-agent reinforcement learning model that can collaboratively complete intelligent handling tasks can be trained. This model can adaptively handle task allocation and path planning problems in complex dynamic environments and achieve efficient collaboration among multiple robots.

[0170] In an optional embodiment,

[0171] The method further comprises:

[0172] Use the improved multi-agent deep deterministic policy gradient algorithm to train the agents and generate collaborative control strategies;

[0173] Based on the generated collaborative control strategy, the following steps are performed: each intelligent transport robot observes its own status and information about the surrounding environment, and obtains information including location, power level, load status, and nearby tasks and other robots; uses the trained high-level policy network to process the observed information and generate long-term goals, including selecting the next task to be processed or determining the moving area; based on the generated long-term goals and current status, uses the trained low-level policy network to generate specific action instructions; through the trained attention mechanism, each intelligent transport robot pays attention to important information about other robots and tasks to achieve collaborative decision-making; the intelligent transport robot executes the generated action instructions and obtains the new status and reward information returned by the environment; the execution results are fed back to the collaborative control strategy to continuously improve the decision-making process.

[0174] In this implementation, the agent is first trained using an improved multi-agent deep deterministic policy gradient algorithm to generate a collaborative control strategy. The algorithm achieves accelerated training and improved generalization capabilities by setting up multiple parallel agents, each of which learns independently and shares experience. Specifically, a central critic network and multiple independent policy networks are constructed. The central critic network evaluates the value of the global state, and each policy network generates actions based on local observations. By introducing an attention mechanism, the agent can focus on important information. During the training process, the agent interacts with other agents in a simulation environment to continuously optimize the policy network parameters.

[0175] Exemplarily, based on the generated collaborative control strategy, the following specific steps are performed:

[0176] First, each intelligent handling robot observes its own status and surrounding environment information, and obtains information including location, power, load status, and nearby tasks and other robots. For example, the observation information of an intelligent handling robot A may include: current location coordinates (10, 20), power 80%, load status is no load, there are 2 pending tasks T1 and T2 nearby, the distances are 5 meters and 8 meters respectively, and there is another intelligent handling robot B 10 meters away.

[0177] Next, the trained high-level policy network is used to process the observed information and generate long-term goals, including selecting the next task to be processed or determining the moving area. The high-level policy network uses a multi-layer perceptron structure, inputs the observed information, and outputs the value evaluation of each optional task or area. For example, for the above robot A, the high-level policy network may output: the value of processing task T1 is 0.8, the value of processing task T2 is 0.6, and the value of moving to an idle area is 0.3. Based on this, robot A chooses processing task T1 as its long-term goal.

[0178] Then, based on the generated long-term goal and current state, the trained low-level policy network is used to generate specific action instructions. The low-level policy network also uses a multi-layer perceptron structure, with input including long-term goals, current state and other information, and outputs specific action parameters. For example, for robot A, the low-level policy network may generate the following action instructions: move in the direction of task T1, at a speed of 0.5 m / s, and at a turning angle of 30 degrees.

[0179] In the process of generating action instructions, the trained attention mechanism enables each intelligent handling robot to pay attention to important information of other robots and tasks, and achieve collaborative decision-making. The attention mechanism calculates the correlation weight between the current robot and other entities (robots or tasks) and aggregates information according to the weight. For example, robot A may give an attention weight of 0.6 to task T1, which is closer, and an attention weight of 0.2 to robot B, which is farther away, so that it can consider more information about task T1 when making decisions.

[0180] The intelligent transport robot executes the generated action instructions and obtains the new state and reward information returned by the environment. For example, after robot A executes the move instruction, the new state may be: position coordinates (12, 23), power 78%, still unloaded, and 2 meters away from task T1. At the same time, it obtains a reward value of 0.2, indicating that it is close to the target task.

[0181] Finally, the execution results are fed back to the collaborative control strategy to continuously improve the decision-making process. Specifically, the new state, action, reward and other information are stored in the experience replay buffer pool, and batch data is regularly sampled from the buffer pool to update the parameters of the central critic network and each policy network. By continuously accumulating experience and learning, the collaborative control strategy can gradually adapt to the complex and changing warehouse environment and improve the accuracy and efficiency of decision-making.

[0182] Through the above steps, a collaborative control method for intelligent handling robots based on deep reinforcement learning is realized. This method can adaptively handle dynamically changing storage environments, coordinate multiple robots to efficiently complete handling tasks, and improve overall operating efficiency.

[0183] In an optional embodiment,

[0184] The steps of the central control system predicting and adjusting the optimal stored environmental parameters based on historical data and current status through the environmental parameter prediction model and sending the environmental parameter instructions to the environmental control module include:

[0185] Constructing an environmental parameter prediction model based on a long short-term memory network, wherein the long short-term memory network includes an input layer, a long short-term memory layer, a fully connected layer, and an output layer;

[0186] Real-time collection of environmental parameter data, including temperature, humidity and light data;

[0187] Inputting the collected environmental parameter data into the environmental parameter prediction model to obtain predicted environmental parameter values;

[0188] Adopt online learning strategy, dynamically adjust the updating frequency of environmental parameter model according to prediction error, and use collected data to update environmental parameter prediction model;

[0189] The Bayesian optimization method is introduced, and the Gaussian process regression model is used to establish the relationship between environmental parameters and the performance of the central control system. The optimal environmental parameter configuration is selected by maximizing the acquisition function, and the optimal environmental parameter configuration instruction is sent to the environmental control module for execution.

[0190] For example, an environmental parameter prediction model based on a long short-term memory network (LSTM) is first constructed. The model includes an input layer, an LSTM layer, a fully connected layer, and an output layer. The input layer receives real-time collected environmental parameter data, including temperature, humidity, and light intensity. The LSTM layer is used to capture the time series characteristics of environmental parameters, the fully connected layer performs feature fusion, and the output layer generates predicted environmental parameter values.

[0191] The number of neurons in the input layer of the LSTM network is the same as the dimension of the environmental parameters, which is 3 (temperature, humidity, and light); the LSTM layer contains 64 neurons; the fully connected layer contains 32 neurons; the number of neurons in the output layer is also 3, corresponding to the predicted temperature, humidity, and light values. The network uses the Adam optimizer, and the loss function uses the mean square error (MSE).

[0192] When training the model, first collect historical environmental parameter data for a period of time (such as 3 months) as the training set. Divide the data into 80% training set and 20% validation set. During the training process, use 24 hours as a time window and predict the environmental parameters for the next hour every hour. For example, use the data from 0:00-23:00 to predict the parameter value at 1:00, use the data from 1:00-24:00 to predict the parameter value at 2:00, and so on. The training is iterated for 500 rounds, and 32 samples are used for batch training in each round.

[0193] After the model training is completed, it enters the real-time prediction stage. The environmental control system collects environmental parameter data every 5 minutes, including temperature (accurate to 0.1°C), relative humidity (accurate to 1%), and light intensity (accurate to 1lux). For example, the data collected at one time is: temperature 22.5°C, relative humidity 65%, and light intensity 500lux. By inputting these data into the trained LSTM model, the predicted values ​​of environmental parameters for the next hour can be obtained.

[0194] In order to ensure the prediction accuracy of the model, an online learning strategy is used to dynamically update the model. The specific approach is: each time the actual environmental parameters are obtained, the error between the actual and predicted values ​​is calculated. If the average error of 10 consecutive predictions exceeds the preset threshold (such as temperature error>1℃, humidity error>5%, and light error>50lux), the model update is triggered. When updating, the model is retrained using the historical data of the last 7 days, and the update frequency does not exceed once a day. This method allows the model to adapt to dynamic changes in the environment in a timely manner.

[0195] On the basis of the predicted values ​​of environmental parameters, the Bayesian optimization method is further introduced to determine the optimal environmental parameter configuration. First, a Gaussian process regression model is established to describe the relationship between environmental parameters and the performance of the central control system. The input of the model is temperature, humidity and light intensity, and the output is the weighted sum of system performance indicators (such as energy consumption, product quality, etc.).

[0196] Initially, 10 sets of environmental parameter configurations are randomly selected, run in the actual system and record the corresponding performance indicators as the initial training data of the Gaussian process model. Afterwards, the expected improvement (EI) is used as the acquisition function to iteratively select the optimal environmental parameter configuration. In each iteration, based on the current Gaussian process model, the EI values ​​corresponding to all possible environmental parameter configurations are calculated, and the configuration with the largest EI value is selected as the next set of parameters to be tried. This set of parameters is applied to the actual system, and the performance indicators are recorded after running for a period of time (such as 1 hour), and the Gaussian process model is updated with this pair of new input-output data.

[0197] Repeat the above process until the preset number of iterations (such as 50) is reached or the performance improvement is less than the threshold. The optimal environmental parameter configuration finally obtained is the instruction that the central control system should execute. For example, the optimal configuration may be: temperature 23.5℃, relative humidity 60%, and light intensity 450lux.

[0198] The central control system sends this set of optimal environmental parameter configuration instructions to the environmental control module for execution. The environmental control module controls the corresponding equipment (such as air conditioners, humidifiers, lighting, etc.) to adjust according to the current actual environmental status and target parameters, so that the environmental parameters gradually reach the target values. In this process, the environmental control module will continuously monitor the actual parameter changes and fine-tune the control strategy as needed to ensure that the environmental parameters are stable near the optimal configuration.

[0199] Through the above method, intelligent prediction and optimization control of environmental parameters based on historical data and current status are realized, which can effectively improve the operating efficiency and performance of the system. The fully automatic storage warehouse can realize fully automatic sample storage without manual management, and can prevent sample confusion, omission, wrong placement, etc., to ensure sample quality and safety. In addition, its intelligent control system can display storage status information in real time, realize automatic storage and record visual management, simplify manual operation, greatly save operating costs, and record each entry and exit of the warehouse for easy historical tracing.

[0200] Figure 2 This is a schematic diagram of the structure of a storage device according to an embodiment of the present invention;

[0201] Figure 3 It is a schematic diagram of the structure of the internal storage and sorting equipment according to an embodiment of the present invention.

[0202] In an optional embodiment,

[0203] The hardware system also includes:

[0204] The storage device includes internal storage and sorting equipment; the storage device is a storage box with adjustable temperature, humidity and light intensity, and the internal storage and sorting equipment includes multiple storage trays, a storage tray lifting platform, a lifting screw and a clamp; the storage tray lifting platform adopts a sandwich structure, with a bottom plate, a middle plate and a top plate, and the multifunctional clamp is equipped with a sensor.

[0205] An exemplary, fully automatic and intelligently managed matrix IoT biological sample storage system is mainly composed of three core parts: refrigeration equipment, internal storage and sorting equipment, and a control system.

[0206] The refrigeration equipment is mainly a temperature-adjustable refrigerator that can meet the storage needs of different samples. The refrigerator door is equipped with an automatic hatch as a channel for storing biological samples, which can be opened and closed automatically as needed.

[0207] The internal storage and sorting equipment is the key structure of this system, which divides the interior of the refrigerator into a storage area and a sorting area. The storage area has multiple storage trays as storage units. To adapt to different sample models and sizes, the storage units can be customized separately. The sorting area includes storage unit sorting and sample sorting. Storage unit sorting is realized by a two-stage telescopic lifting platform. The platform drives the lifting screw through the motor to realize the up and down lifting of the storage tray. The platform adopts a sandwich structure with three layers of bottom plate, middle plate and top plate. Under the drive of the motor, the bottom plate, middle plate and top plate are linked to each other through the synchronous belt structure. Sample sorting is realized by a multifunctional gripper. The gripper can store or remove samples from the storage platform, and the gripper can be adapted to single clamping or whole box clamping, with wide versatility. The gripper can move accordingly on a set of three-axis (X-axis, Y-axis, Z-axis) mobile platforms, and an RFID sensor or a barcode or QR code recognition sensor is installed on the gripper. When clamping, the RFID information or barcode or QR code on the sample can be read at the same time for recognition.

[0208] The core of the control system is the MCU embedded control system, which connects various hardware components, such as RFID, electric grippers, sensor sensors, electric doors, motor drives, etc. The system also includes auxiliary components such as position sensors and encoders, which are responsible for controlling the motion mechanism and sensors, processing host computer instructions, and realizing the automation of the entire system. When an abnormal situation occurs in the storage compartment and needs to be handled, the cloud platform can issue instructions to each motor to reset the position and the cloud platform can monitor the equipment operation status at any time and display the sample storage status in real time.

[0209] The sample storage process starts with the system sending instructions, and goes through a series of steps such as checking the storage space, opening the cabinet door, putting in the sample, reading the RFID, and allocating the storage location. During the storage process, the sample information is identified through RFID or barcodes and QR codes to ensure storage accuracy.

[0210] The sample retrieval process begins with the system sending sampling instructions and RFID information, and then goes through steps such as finding the storage location, moving the storage platform, and extracting the sample. The retrieval process includes RFID verification and error handling to ensure sampling accuracy. The entire process involves multiple automated steps, such as automatic door opening and closing, mobile platform operation, etc.

[0211] When an abnormality occurs in the storage compartment, the cloud platform can issue instructions to reset the position and monitor the equipment operating status and sample storage status in real time.

[0212] The system is highly automated, and can achieve unattended and intelligent management of samples, which greatly reduces labor costs and improves management efficiency. The gripper design is highly versatile and can adapt to samples of different specifications and types. It can move in three axes and has a wide range of access. The system also has accurate sample identification and access management functions to ensure access accuracy. Flexible temperature control settings meet the storage needs of different samples, and real-time monitoring and exception handling capabilities improve system reliability.

[0213] This system represents an important technological breakthrough in biological sample storage management, realizing the transition from manual management to fully automated processing, and providing an efficient, accurate and reliable solution for the storage of precision biological samples. It realizes the storage of samples in fully automated processing, demonstrating the intelligence and accuracy in the process of sample storage and retrieval.

[0214] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fully automatic storage bin and control method thereof, characterized in that: include: Hardware system and software system, the hardware system includes an intelligent handling robot and an environmental control module, and the software system includes a central control system, a path planning module, a task scheduling module, and a visual management module; the intelligent handling robot is equipped with a lifting device, a telescopic arm and a grasping mechanism, and is installed with a radio frequency identification reader; the environmental control module includes a temperature and humidity sensor, a light sensor, and an environmental parameter adjustment device; The central control system interacts with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition; The path planning module generates the optimal path based on the current storage status and decomposed tasks based on the deep learning algorithm; The task scheduling module adopts multi-agent reinforcement learning method to generate the collaborative control strategy of the intelligent handling robot based on the distributed decision-making mechanism; The visual management module displays the operating status in real time; the central control system controls the intelligent handling robot to execute access instructions according to the optimal path and collaborative control strategy, and monitors the status and location information of the intelligent handling robot in real time, and dynamically adjusts task allocation and path planning; The central control system uses the environmental parameter prediction model to predict and adjust the optimal stored environmental parameters based on historical data and current status, and sends environmental parameter instructions to the environmental control module; the central control system collects operating data in real time, iteratively updates the collaborative control strategy based on the operating data, and generates performance reports that are transmitted to the visual management module; The task scheduling module adopts a multi-agent reinforcement learning method. The steps of generating a collaborative control strategy for an intelligent handling robot based on a distributed decision-making mechanism include: The intelligent handling robot is defined as an agent; the improved multi-agent deep deterministic policy gradient algorithm is constructed, including: introducing the attention mechanism, using the graph attention network to encode the observation of each agent into a graph structure, where the nodes represent the robot and the task, and the edges represent the relationship between the robot and the task; designing a hierarchical policy network structure, including a high-level policy network responsible for generating long-term goals and a low-level policy network responsible for generating specific actions; using a TD-error-based priority experience replay mechanism to dynamically adjust the sampling probability according to the importance of experience; using a multi-step bootstrap method to update the Q value through n-step returns to balance the deviation and variance; initializing the actor network, the critic network and its target network, and initializing the experience replay buffer; In each training round, the agent selects actions based on the current strategy and exploration noise, executes actions and observes rewards and new states; stores state transitions in the experience replay buffer, and when the buffer size exceeds the batch size, samples a small batch from it for learning; calculates the target Q value, updates the critic network to minimize the loss function, and updates the actor network using policy gradients; updates the target network, and repeats the training process until the termination condition is met; The method further comprises: Use the improved multi-agent deep deterministic policy gradient algorithm to train the agents and generate collaborative control strategies; Based on the generated collaborative control strategy, the following steps are performed: each intelligent transport robot observes its own status and information about the surrounding environment, and obtains information including location, power level, load status, and nearby tasks and other robots; uses the trained high-level policy network to process the observed information and generate long-term goals, including selecting the next task to be processed or determining the moving area; based on the generated long-term goals and current status, uses the trained low-level policy network to generate specific action instructions; through the trained attention mechanism, each intelligent transport robot pays attention to important information about other robots and tasks to achieve collaborative decision-making; the intelligent transport robot executes the generated action instructions and obtains the new status and reward information returned by the environment; the execution results are fed back to the collaborative control strategy to continuously improve the decision-making process.

2. The method according to claim 1, characterized in that The central control system exchanges data with the hardware system in real time through the wireless communication network, receives access instructions and performs task decomposition in the following steps: The central control system parses the access instruction into basic operation units through a finite state machine, wherein the basic operation units include access operations, move operations, and placement operations; The basic operation units are input into a dynamic programming algorithm to generate a subtask sequence. The dynamic programming algorithm aims to minimize the total execution time and calculates the minimum execution time when the first several basic operation units are completed and the intelligent handling robot is located at a specific position through a state transition equation; Optimizing the subtask sequence using a heuristic rule set, the heuristic rule set includes proximity principle, load balancing and batch optimization; wherein the proximity principle is implemented by a KD tree data structure, load balancing is implemented by a minimum heap, and batch optimization uses a clustering algorithm to group spatially adjacent tasks into batch tasks; According to the optimized sub-task sequence, each intelligent handling robot is assigned a task; Through machine learning methods, the weights of each rule in the heuristic rule set are dynamically adjusted based on historical performance data; During the task execution process, when a new access instruction is detected, the task decomposition process is re-triggered, and the newly generated sub-task sequence is integrated with the task being executed, and the task is re-assigned to each intelligent handling robot.

3. The method according to claim 1, characterized in that The path planning module is based on a deep learning algorithm. The steps to generate the optimal path according to the current storage state and the decomposed tasks include: Construct a three-dimensional discrete matrix environment model to represent the matrix IoT fully automatic storage warehouse environment as a state space, including the current three-dimensional coordinates of the intelligent handling robot, the three-dimensional coordinates of the target position, the task type, the load status, the RFID reading status, the height of the lifting device, the telescopic arm status, and the grasping mechanism status; Design the action space, including the moving direction, lifting device operation, telescopic arm operation and grasping mechanism operation of the intelligent handling robot; Based on the three-dimensional discrete matrix environment model and action space, a deep Q network is constructed, including an input layer, two convolutional layers, two fully connected layers and an output layer. The input layer encodes the state space into a vector, the two convolutional layers use 32 and 64 3x3x3 convolution kernels respectively, the two fully connected layers contain 256 and 128 neurons respectively, and the output layer corresponds to all possible actions; Design reward functions, including fixed step penalty, target reaching reward, collision penalty, and reward and penalty based on 3D distance change; Implement the experience replay mechanism and store the state, action, reward, next state and termination flag in the experience replay buffer; Introduced a priority experience replay mechanism, using the absolute value of the TD error as the priority; The ε-greedy strategy is used for exploration, and the ε value decays linearly from 1.0 to 0.1; The deep Q network was trained using the Adam optimizer with a learning rate of 0.0001 and a discount factor of 0.99; Update the target network regularly, with an update frequency of every 1000 steps; Iterative training optimizes the parameters of the deep Q network; Use the optimized deep Q network to generate the optimal path based on the current storage status, intelligent handling robot status and task requirements.

4. The method according to claim 3, characterized in that The reward function is: R(s,a,s′)=-α+β·I(s′=goal)-γ·I(collision)+δ·(d(s,goal)-d(s′,goal)); in, R(s,a,s') represents the reward function, a represents the action performed in the current state, s represents the current state, and s' represents the next state; α represents a fixed step penalty; β represents the reward for reaching the target, which is a positive reward given when the agent reaches the target position; γ represents the collision penalty, which is the negative reward given when the agent collides; δ represents the distance change coefficient, which is used to give rewards or penalties according to the change in the distance between the agent and the target; I(s'=goal) represents the indicator function. When s' is equal to the target state goal, the function value is 1, otherwise it is 0; I (collision) represents the indicator function, the function value is 1 when a collision occurs, otherwise it is 0; collision indicates a collision event; d(s,goal) represents the Euclidean distance from the current state s to the target state goal; d(s',goal) represents the Euclidean distance from the next state s' to the target state goal.

5. The method according to claim 1, characterized in that The steps of the central control system predicting and adjusting the optimal stored environmental parameters based on historical data and current status through the environmental parameter prediction model and sending the environmental parameter instructions to the environmental control module include: Constructing an environmental parameter prediction model based on a long short-term memory network, wherein the long short-term memory network includes an input layer, a long short-term memory layer, a fully connected layer, and an output layer; Real-time collection of environmental parameter data, including temperature, humidity and light data; Inputting the collected environmental parameter data into the environmental parameter prediction model to obtain predicted environmental parameter values; Adopt online learning strategy, dynamically adjust the updating frequency of environmental parameter model according to prediction error, and use collected data to update environmental parameter prediction model; The Bayesian optimization method is introduced, and the Gaussian process regression model is used to establish the relationship between environmental parameters and the performance of the central control system. The optimal environmental parameter configuration is selected by maximizing the acquisition function, and the optimal environmental parameter configuration instruction is sent to the environmental control module for execution.

6. The method according to claim 1, characterized in that The hardware system also includes: The storage equipment includes internal storage and sorting equipment; the storage equipment is a storage box with adjustable temperature, humidity and light intensity, and the internal storage and sorting equipment includes multiple storage trays, a storage tray lifting platform, a lifting screw and a clamp; the storage tray lifting platform adopts a sandwich structure, with a bottom plate, a middle plate and a top plate, and the multifunctional clamp is equipped with a sensor.

Images