Intelligent manufacturing shelf storage method and surrounding shelf mechanism

Abstract

The application discloses an intelligent manufacturing goods shelf storage method and a surrounding goods shelf mechanism, integrates a camera and a depth sensor on a mechanical arm, and combines a convolutional neural network and a reinforcement learning algorithm to perform target identification and positioning, the surrounding goods shelf mechanism comprises a goods shelf module and a mechanical arm module; a goods shelf base is arranged at the bottom of the goods shelf module, and the mechanical arm module is placed on the goods shelf base. The application realizes automatic identification and accurate positioning of goods, improves the accuracy and efficiency of grabbing, and reduces the need for human intervention.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent warehousing systems, in particular to an intelligent manufacturing shelf storage method and a ring-shaped shelf mechanism. BACKGROUND

[0002] With the development of intelligent manufacturing industry and the reduction of labor population, more and more unmanned systems appear. Past warehousing equipment is mostly designed for humans, and it is somewhat difficult to face the future of full automation. The present technology proposes a ring-shaped warehousing shelf for a mechanical arm, which greatly reduces the cost while realizing unmanned factory and improves the modularization and standardization of the factory. The technology is mainly used in large-scale warehousing factories, large-scale express stations, etc.

[0003] In order to fully save costs, the traditional logistics company's warehousing shelves are high and prone to tipping, so there is great inconvenience and danger, and this ring-shaped shelf system also has certain improvement. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application provides an intelligent manufacturing shelf storage method and a ring-shaped shelf mechanism, which realizes automatic recognition and accurate positioning of goods, improves the accuracy and efficiency of grabbing, and reduces the need for human intervention.

[0005] The present application realizes the above technical purpose through the following technical means.

[0006] An intelligent manufacturing shelf storage method integrates a camera and a depth sensor on a mechanical arm, combines a convolutional neural network and a reinforcement learning algorithm for target recognition and positioning, and includes the following steps:

[0007] Step one: install a high-resolution camera and a depth sensor on the mechanical arm to capture detailed visual and depth information of the goods. Specifically, collect large-scale data, including pictures of different types, sizes and shapes of goods and their corresponding depth information; each goods should be photographed under different light and background conditions to enhance the generalization ability of the model;

[0008] Step two: standardize the captured images, including scaling, cropping and normalization, to adapt to the input requirements of the convolutional neural network; label the goods in the image to determine the position of the bounding box, which is used for subsequent target detection training;

[0009] Step three: select ResNet architecture for feature extraction: apply DQN method to let the model learn how to effectively recognize and locate targets through interaction with the environment; use GPU-accelerated deep learning framework to train the model, input image data and corresponding label information, and perform end-to-end learning;

[0010] Step four: the trained model is deployed into the robot arm control system to process camera and depth sensor data in real time; test in actual operation environment to evaluate recognition and positioning accuracy of the model under various conditions and optimize parameters to improve efficiency.

[0011] In the above scheme, in step one, hardware configuration, data collection and data storage and processing are included; specifically,

[0012] Hardware configuration includes configuration of high-resolution camera and depth sensor; wherein the high-resolution camera is installed at the end of the robot arm to capture front and side images of the goods to capture high-resolution images of the goods and obtain detailed information; the depth sensor is installed at the end of the robot arm near the camera position to ensure synchronous acquisition of depth information to capture depth information of the goods and provide 3D image data;

[0013] Data collection includes data collection strategy and data collection process;

[0014] Among them, the data collection strategy collects images of goods and their corresponding depth images; covers various types of goods, including objects of different sizes, shapes, colors and materials; takes pictures under various lighting conditions and background conditions;

[0015] The data collection process uses adjustable LED light sources to simulate different lighting conditions; uses the robot arm to drive the camera and depth sensor to automatically capture images and depth information of the goods; takes pictures from different angles and distances to ensure data diversity;

[0016] Data storage and processing use high-capacity NAS storage devices to ensure data security and accessibility; use LabelImg tools to label images to record the bounding box, category and depth information of the goods; apply data augmentation techniques to increase data volume and improve the generalization ability of the model.

[0017] In the above scheme, in step two, image processing and labeled data are included; specifically,

[0018] Image processing includes scaling, cropping and normalization;

[0019] Among them, scaling scales the captured images to a uniform size to adapt to the input requirements of the convolutional neural network; for the interpolation method, it uses bilinear interpolation for image scaling to ensure image quality;

[0020] Cropping crops the goods part in the image to remove redundant background information; its specific process is: according to the actual position of the goods, determine the cropping area; crop through the pre-labeled bounding box; appropriately expand the size of the bounding box during actual cropping to ensure that the complete information of the goods is included;

[0021] Normalization, normalize the image pixel value to facilitate neural network training; the commonly used normalization method for mean and standard deviation is to standardize the image, that is, subtract the mean and divide by the standard deviation;

[0022] The annotation data includes boundary box annotation, boundary box format, annotation tool and annotation data format; specifically,

[0023] Boundary box annotation, the goods in the image are annotated to determine the boundary box position of the goods for subsequent target detection training;

[0024] Boundary box format, using (xmin, ymin, xmax, ymax) annotation format to represent the coordinates of the upper left corner and the lower right corner of the boundary box;

[0025] Annotation tool, using LabelImg image annotation tool to manually annotate the image;

[0026] Annotation data format, save the annotation data in a standard format for subsequent loading and use; use XML format to store the annotation data, each image corresponds to an annotation file; each annotation file contains image file name, image size, boundary box coordinates and class label.

[0027] In the above scheme, in step three, including ResNet architecture feature extraction, DQN target recognition and positioning, end-to-end learning combined with ResNet and DQN, and improving ResNet algorithm; specifically,

[0028] ResNet architecture feature extraction, ResNet network architecture is ResNet-50, using residual block for feature extraction;

[0029] DQN target recognition and positioning;

[0030] End-to-end learning combined with ResNet and DQN, input image through ResNet to extract feature vector and input into DQN network; according to the feature vector extracted by ResNet, the DQN network outputs the Q value of each action, selects the optimal action; the robot arm executes the grabbing operation according to the optimal action output by the DQN network; according to the grabbing result, the corresponding reward is given, and the Q value is updated; adjust the network parameters through back propagation algorithm to optimize the joint model of ResNet and DQN;

[0031] Improving ResNet algorithm, improving ResNet to optimize its feature extraction ability and computing efficiency, specifically including introducing attention mechanism and optimizing residual block structure;

[0032] Among them, the attention mechanism is introduced, which helps the network to focus on important features better and improve the accuracy of feature extraction; the Squeeze-and-Excitation (SE) module is a commonly used attention mechanism that can dynamically adjust the weights of each channel;

[0033] Optimize the structure of the residual block: introduce the bottleneck structure to reduce the number of parameters and improve the calculation efficiency; the bottleneck structure reduces the dimension and increases the dimension through the 1x1 convolution layer to reduce the calculation amount;

[0034] The specific formula is as follows

[0035] The basic convolution layer is as follows

[0036] Y = Conv (X, W) + B

[0037] Where X is the input image; W is the convolution kernel weight; B is the bias;

[0038] The formula of the residual block is as follows

[0039] The output of the standard residual block is as follows

[0040]

[0041] Where x is the input; is a residual function composed of convolution layer, batch normalization and ReLU;

[0042] The formula of the bottleneck residual block is as follows

[0043] The output of the optimized bottleneck residual block is as follows

[0044]

[0045] Where W1 is a 1x1 convolution kernel for dimension reduction; W2 is a 3x3 convolution kernel for feature extraction; W3 is a 1x1 convolution kernel for dimension increase; BN is batch normalization; and σ is ReLU activation function;

[0046] Introduce the SE module

[0047] The calculation process of the SE module includes Squeeze and Excitation:

[0048]

[0049] s = F ex (z, W) = σ (W2δ (W1z))

[0050] where u is the input feature map; H and W are the height and width of the feature map, respectively; z is the output of the Global Average Pooling; W1 and W2 are the weights of the fully connected layer; δ is the ReLU activation function; σ is the Sigmoid activation function; and s is the channel weight.

[0051] The output of the SE module is:

[0052] V = s·u

[0053] where s is the channel weight calculated by the SE module; and u is the input feature map.

[0054] The formula of the improved residual block is:

[0055]

[0056] where s is the channel weight calculated by the SE module.

[0057] The DQN algorithm is improved in order to better adapt to the robotic arm grasping task in patents. The following improvements are made to the DQN (Deep Q-Network) algorithm: the introduction of Double DQN, Prioritized Experience Replay, the fusion of Hindsight Experience Replay (HER) and Meta-Learning to improve learning efficiency and stability.

[0058] Double DQN solves the instability problem caused by overestimation in the DQN algorithm. The formula derivation is as follows.

[0059] The Q value update formula of the traditional DQN is:

[0060]

[0061] The Q value update formula of Double DQN is:

[0062]

[0063] where Q and Q' are two independent Q value networks (one for selecting actions and one for evaluating actions); α is the learning rate; r t+1 is the immediate reward; γ is the discount factor; is the optimal action selected by the target network Q' in state s t+1 .

[0064] Prioritized experience replay, enhance training efficiency, by prioritizing the experience with higher TD error for replay, improve the speed and effect of learning, the formula derivation as follows:

[0065] Priority calculation:

[0066] p i = |δ i | + ∈

[0067] Where p i is the priority of sample i; δ is the TD error of sample i; ∈ is a small constant to prevent the priority from being zero;

[0068] Sampling probability:

[0069]

[0070] Where P(i) is the probability of sample i being selected; α controls the importance of priority;

[0071] Importance sampling weight:

[0072]

[0073] Where w i is the importance sampling weight of sample i; β controls the smoothing degree of importance sampling; N is the size of the experience pool;

[0074] Fusion Hindsight Experience Replay (HER), after introducing HER, by reusing failure experience, make more experience useful to the model, especially in sparse reward environment; At the same time, due to the random replacement of the target and the redefinition of the reward, the model can be generalized in a wider range of scenarios; Finally, significantly improve the efficiency and stability of DQN algorithm in robot grasping task, make it better adapt to the complex environment in practical application;

[0075] Meta-Learning, Meta-Learning through learning how to learn, make the model can quickly adapt and generalize between different tasks, so as to perform well in new tasks or environment.

[0076] The core idea of Meta-Learning is to train a meta-model that can quickly adjust its parameters when facing new tasks, so as to quickly adapt to new tasks; This method is particularly suitable for dynamic environment, can significantly improve the adaptability and efficiency of algorithm;

[0077] Specific steps and formula derivation: define the grasping task as a series of subtasks, each subtask corresponds to different target objects or environment state;

[0078] In the meta-training phase, an initialization parameter is learned by training the model on multiple sub-tasks, so that it can quickly adapt to new tasks through a small number of gradient updates;

[0079]

[0080] where θ represents the parameters of the model; <- represents the update operation, which means assigning the expression on the right side to the variable on the left side; β represents the learning rate, which controls the step size of parameter update; represents the gradient with respect to the parameter θ; represents the sum of all possible tasks T i , where T i is sampled from the task distribution p(T); LTiLTi represents the loss function for task TiTi; f θ represents the prediction function of the model, with parameter θ; α represents the internal update step, which is used for fast adaptation in the meta-training phase;

[0081] In the meta-testing phase, the initialization parameter learned in the meta-training phase is applied to the new task, and a small number of gradient updates are used to quickly adapt to the new task;

[0082]

[0083] where θ' represents the updated model parameter on the new task; θ represents the initialization parameter learned in the meta-training phase; α represents the same as in the meta-training phase, used to control the gradient update step on the new task; represents the loss function of the new task; f θ represents the model prediction function using the parameter θ obtained in the meta-training phase;

[0084] Meta-learning learns the initialization parameter on multiple sub-tasks, so that the model can quickly adjust its parameters in the new task and quickly adapt to the new environment; through multi-task training, the generalization ability of the model is improved, so that it performs more stably when facing different tasks; only a small number of gradient updates are needed on the new task to obtain good performance, significantly improving the learning efficiency;

[0085] By introducing the meta-learning method, combining double DQN, priority experience replay and HER, the adaptability and learning efficiency of the DQN algorithm in the robot grasping task are further improved, so that it can better cope with dynamic changes in the environment and complex tasks;

[0086] In the above scheme, in step four, model deployment and real-time testing are included; specifically,

[0087] Model deployment includes hardware configuration, software configuration, model export and model integration;

[0088] Hardware configuration, using a computing platform with high-performance GPU for model inference; integrating high-resolution cameras and depth sensors to ensure high-quality visual and depth information acquisition;

[0089] Software configuration, PyTorch for model inference; Ubuntu 18.04 version, providing a stable running environment; using ROS (Robot Operating System) for robot arm control and data communication;

[0090] Model export, using ONNX for model optimization to reduce inference time; exporting the trained model into a format suitable for deployment;

[0091] Model integration, integrating the optimized model into the robot arm control system to ensure that the model can process camera and sensor data in real time; designing data interfaces to ensure that the model can receive real-time data from the camera and depth sensor and send the processing results to the robot arm control module;

[0092] Real-time testing includes environment setup, test indicators, test process, and parameter optimization;

[0093] Environment setup, simulating the actual operating environment, setting up test scenes with multiple lighting conditions and backgrounds, placing different types, sizes, and shapes of goods; ensuring that the robot arm can move freely and cover all positions in the test area;

[0094] Test indicators, evaluating the model's recognition accuracy of goods under different lighting conditions and backgrounds; evaluating the model's positioning accuracy of goods to ensure that the robot arm can accurately grasp the goods; recording the time from data reception to output results to ensure real-time requirements; counting the ratio of successful grasping to the number of attempts in different scenarios;

[0095] Test process, the robot arm acquires images and depth information of goods at different positions and angles; multiple data collection is performed under different lighting and background conditions to ensure data diversity;

[0096] Input real-time collected images and depth data into the model; ResNet and DQN end-to-end models perform result inference to infer action instructions; control the robot arm to move and grasp goods according to the model output action instructions; record successful and failed operations according to the grasping results and adjust model parameters to improve success rate;

[0097] Parameter optimization, according to the test results, adjust the learning rate, batch size, convolution kernel size, and other hyperparameters of the model to improve model performance; perform real-time testing multiple times to continuously optimize parameters to ensure the stability and efficiency of the model in various environments.

[0098] The intelligent manufacturing shelf storage method is used for a surrounding type shelf mechanism, and the surrounding type shelf mechanism comprises a shelf module and a mechanical arm module; a shelf base is arranged at the bottom of the shelf module, and the mechanical arm module is placed on the shelf base;

[0099] The shelf module comprises a support vertical plate, an enclosing plate and a placing plate; the enclosing plate is an arc-shaped cavity, a plurality of layers of placing plates are arranged along the longitudinal direction of the enclosing plate, a plurality of support vertical plates are arranged between adjacent placing plates in the longitudinal direction, and the support vertical plates are used to cooperate with the mechanical arm module to lift the height of goods.

[0100] In the above scheme, a large gear rack is arranged on the support vertical plate along the longitudinal direction, wherein the large gear rack and a small gear rack are arranged in parallel, the large gear rack and the small gear rack are both engaged with a gear, the gear is driven by a rudder machine, the small gear rack is driven to move upward or downward by the gear, and a lifting plate is arranged at the upper end of the small gear rack; the lifting plate is an arc-shaped structure and cooperates with the placing plate to realize the upward movement of goods.

[0101] In the above scheme, the rudder machine is arranged on a fixed clamping plate, and an output end of the rudder machine drives the gear through a rudder machine transmission groove; the fixed clamping plate is fixedly connected with the large gear rack.

[0102] In the above scheme, the transverse section of the enclosing plate is a semi-ring shape.

[0103] In the above scheme, the fixed clamping plate is in sliding connection with the small gear rack.

[0104] Beneficial effects:

[0105] 1. The device is different from most of the existing shelves, which are rectangular in plan view, the ring-shaped structure increases the stability of the shelf and reduces the dead zone of the mechanical arm, thereby saving costs.

[0106] 2. The present application does not require human intervention. The circular disc type base is arranged inside the ring-shaped shelf, which facilitates the placement of various mechanical arm models. After the goods are placed on the lifting plate, the lifting mechanism is used to reach the corresponding position;

[0107] 3. The present application realizes automatic recognition and accurate positioning of goods, improves the accuracy and efficiency of grabbing, and reduces the need for human intervention.

[0108] 4. The present application is matched with various mechanical arm modules on the market, is designed for unmanned factories, saves labor costs, and well solves the problems raised in the above background technology. At the same time, the weight of the mechanical arm arranged on the shelf base reduces the center of gravity of the shelf, and the shelf adopts a semi-enclosed structure to increase the stability of the shelf, and a recognition algorithm combining convolutional neural network and reinforcement learning is introduced to realize automation.

[0109] 5. The device comprises a shelf module and a mechanical arm module, realizes automatic taking and placing of goods; at the same time, the semi-ring design of the shelf fully utilizes the contactable position of the mechanical arm, saves the cost and improves the stability of the shelf; at the same time, the camera and the depth sensor are integrated on the mechanical arm, the target recognition and positioning are carried out combining the convolutional neural network and the reinforcement learning algorithm; automatic recognition and accurate positioning of goods are realized, the accuracy and efficiency of grabbing are improved, and the demand for human intervention is reduced; in general, the present application has the advantages of automatic control and stable and safe structure. BRIEF DESCRIPTION OF DRAWINGS

[0110] Figure 1 It is a schematic diagram of the surrounding type shelf mechanism structure involved in the present application.

[0111] Figure 2 It is a perspective view of the shelf module involved in the present application. Figure 1

[0112] It is a partial enlarged sectional view of the rack and pinion lifting module. Figure 3

[0113] It is a flow chart of the intelligent manufacturing shelf storage method involved in the present application. Figure 4 REFERENCE SIGNS:

[0114] 1 - shelf module; 2 - mechanical arm module; 11 - support vertical plate; 12 - surrounding plate; 13 - lifting plate; 14 - lifting mechanism; 15 - multi-layer placing plate; 16 - shelf base; 141 - large gear rack; 142 - gear; 143 - small gear rack; 144 - fixed clamping plate; 145 - steering gear transmission groove.

[0115] DETAILED DESCRIPTION The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0116]

[0117] ​In the description of the present application, it needs to be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "axial", "radial", "vertical", "horizontal", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second" can be explicitly or implicitly included one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly specified and limited.

[0118] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0119] In conjunction with the accompanying Figure 4 As shown, an intelligent manufacturing shelf storage method integrates a camera and a depth sensor on a mechanical arm, combines a convolutional neural network and a reinforcement learning algorithm for target recognition and positioning; realizes automatic recognition and accurate positioning of goods, improves the accuracy and efficiency of grabbing, and reduces the need for human intervention;

[0120] Step 1 Install high-resolution cameras and depth sensors on the mechanical arm module to ensure that detailed visual and depth information of the goods can be captured; collect large amounts of data, including pictures of different types, sizes and shapes of goods and their corresponding depth information; each goods should be taken under different lighting and background conditions to enhance the generalization ability of the model;

[0121] 1.1 Hardware configuration

[0122] High-resolution camera: installed at the end of the mechanical arm to ensure that front and side images of the goods can be captured; used to capture high-resolution images of the goods to obtain detailed information;

[0123] Depth sensor: installed at the end of the mechanical arm, close to the camera position, to ensure synchronous acquisition of depth information; used to capture depth information of the goods to provide 3D image data;

[0124] 1.2 Data Collection

[0125] Data collection strategy: Collect at least 100,000 images of goods and their corresponding depth images; cover various types of goods, including objects of different sizes, shapes, colors and materials; take pictures under various lighting conditions (strong light, weak light, natural light) and background conditions (complex background, simple background);

[0126] Data collection process: Adjustable LED light sources are used to simulate different lighting conditions; a robotic arm drives a camera and depth sensor to automatically capture images and depth information of the cargo; images are taken from different angles (0 degrees, 45 degrees, 90 degrees) and distances (0.5m, 1m, 1.5m) to ensure data diversity;

[0127] 1.3 Data Storage and Processing: High-capacity NAS storage devices are used to ensure data security and accessibility; LabelImg tools are used to annotate images, recording the bounding boxes, categories, and depth information of goods; data augmentation techniques (rotation, scaling, translation) are applied to increase the amount of data and improve the generalization ability of the model;

[0128] Step 2 involves standardizing the captured images, including scaling, cropping, and normalization, to adapt them to the input requirements of the convolutional neural network; and labeling the goods in the images to determine their bounding box locations for subsequent object detection training.

[0129] 2.1 Image Processing

[0130] Scaling: The captured image is scaled to a uniform size to fit the input requirements of a convolutional neural network (CNN); its specific parameters are: target size of 224x224 pixels, and bilinear interpolation is used for image scaling to ensure image quality.

[0131] Cropping: Cropping out the main part of the image (i.e., the cargo part) and removing unnecessary background information; the specific process is as follows: determine the cropping area according to the actual position of the cargo; crop using a pre-marked bounding box; during actual cropping, appropriately expand the bounding box size (here, expand the boundary by 10%) to ensure that complete cargo information is included;

[0132] Normalization: Normalize the image pixel values for neural network training; the specific process is first normalize the image pixel values to the range [0, 1]; the commonly used normalization method for mean and standard deviation is to standardize the image, that is, subtract the mean and divide by the standard deviation (here, the ImageNet dataset is used, whose mean and standard deviation are [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225]);

[0133] 2.2 Labeling data

[0134] Bounding box labeling: Label the goods in the image and determine the bounding box position of the goods for subsequent target detection training;

[0135] Bounding box format: Use the (xmin, ymin, xmax, ymax) labeling format to represent the coordinates of the top-left and bottom-right corners of the bounding box;

[0136] Labeling tool: Use the LabelImg image labeling tool to manually label the image;

[0137] Labeling data format: Save the labeling data in a standard format for subsequent loading and use; use XML format to store the labeling data, with each image corresponding to a labeling file; each labeling file contains the image file name, image size, bounding box coordinates, and class label;

[0138] Step 3: Select ResNet architecture for feature extraction; apply DQN method to let the model learn how to most effectively identify and locate targets through interaction with the environment; use GPU-accelerated deep learning framework for model training, input image data and corresponding labeling information, and perform end-to-end learning;

[0139] 3.1 ResNet architecture feature extraction

[0140] ResNet (Residual Network) is a deep convolutional neural network that addresses the vanishing gradient problem in deep networks by introducing residual connections. Here, the network architecture is ResNet-50, which contains 50 layers and uses residual blocks for feature extraction. The specific process is as follows: standardized cargo images are captured; the input image size is 224x224x3; a convolution kernel (Filter) is used for feature extraction, with a kernel size of 3x3; each residual block contains two convolution layers and a skip connection; the convolution features are mapped to a high-dimensional space for subsequent classification and detection tasks; the kernel size is set to 3x3, the stride is 1, the activation function is ReLU, the batch size is 32, the learning rate is 0.001, and the number of training epochs is 100;

[0141] 3.2DQN target recognition and positioning

[0142] DQN (Deep Q-Network) is a reinforcement learning algorithm that combines Q-Learning and deep neural networks to learn the optimal policy through interaction with the environment. Its network architecture uses a convolutional neural network (CNN) as an approximator of the Q-value function. The implementation process is as follows

[0143] State (S): The current state of the environment, including image features captured by the camera (feature vectors extracted by ResNet);

[0144] Action (A): Manipulator operation, moving direction (up, down, left, right), and grasping action;

[0145] Reward (R): Reward given according to the effect of the manipulator's operation; here, a positive reward is given for successfully grasping the cargo, and a negative reward is given for failure;

[0146] Q-value update formula:

[0147]

[0148] Q(s t , a t ) represents the Q-value of performing action a t in state s t ; α represents the learning rate, controlling the step size of Q-value update; r t+1 represents the immediate reward after performing the action; γ represents the discount factor, controlling the importance of future rewards; representing the maximum Q value in the new state s t+1 ;

[0149] 3.3 End-to-end learning combined with ResNet and DQN

[0150] The input image is extracted by ResNet to obtain a feature vector, which is input into the DQN network. According to the feature vector extracted by ResNet, the DQN network outputs the Q value of each action, and selects the optimal action. The robot arm executes the grabbing operation according to the optimal action output by the DQN network. According to the grabbing result, the corresponding reward is given, and the Q value is updated. The network parameters are adjusted through the back propagation algorithm, and the joint model of ResNet and DQN is optimized.

[0151] 3.4 Improved ResNet algorithm

[0152] In order to better combine with the robot arm grabbing task in the patent, the ResNet is improved to optimize its feature extraction ability and calculation efficiency, which includes introducing attention mechanism and optimizing residual block structure.

[0153] Introducing attention mechanism: Attention mechanism helps the network focus better on important features and improve the accuracy of feature extraction. Squeeze-and-Excitation (SE) module is a commonly used attention mechanism that can dynamically adjust the weights of each channel.

[0154] Optimize residual block structure: Introduce bottleneck structure to reduce the number of parameters and improve calculation efficiency. Bottleneck structure reduces dimension and increases dimension through 1x1 convolution layer to reduce calculation amount.

[0155] The specific formula is as follows

[0156] Basic convolution layer:

[0157] Y = Conv(X, W) + B

[0158] Where X is the input image; W is the convolution kernel weight; B is the bias.

[0159] Residual block formula:

[0160] The output of the standard residual block is:

[0161]

[0162] Where x is the input; is a residual function composed of convolution layer, batch normalization and ReLU;

[0163] Bottleneck residual block formula:

[0164] The output of the optimized Bottleneck residual block is:

[0165]

[0166] wherein W1 is a 1x1 convolution kernel for dimension reduction; W2 is a 3x3 convolution kernel for feature extraction; W3 is a 1x1 convolution kernel for dimension increase; BN is batch normalization; and σ is a ReLU activation function.

[0167] An SE module is introduced:

[0168] The calculation process of the SE module includes two parts: Squeeze and Excitation:

[0169]

[0170] s = F ex (z, W) = σ(W2δ(W1z))

[0171] wherein u is an input feature map; H and W are the height and width of the feature map, respectively; z is the output of global average pooling; W1 and W2 are the weights of the full connection layer; δ is a ReLU activation function; σ is a Sigmoid activation function; and s is a channel weight.

[0172] The output of the SE module is:

[0173] v = s·u

[0174] wherein s is a channel weight calculated by the SE module; and u is an input feature map.

[0175] The formula of the improved residual block is:

[0176]

[0177] wherein s is a channel weight calculated by the SE module.

[0178] 3.5 Improved DQN algorithm

[0179] In order to better adapt to the mechanical arm grasping task in the patent, the following improvements are made to the DQN (DeepQ-Network) algorithm: Double DQN (Double DQN) and Prioritized Experience Replay (Prioritized Experience Replay) are introduced to improve learning efficiency and stability.

[0180] 3.5.1 Double DQN (Double DQN)

[0181] To solve the instability problem caused by overestimation in DQN algorithm, the formula derivation is as follows:

[0182] The Q value update formula of traditional DQN:

[0183]

[0184] The Q value update formula of double DQN:

[0185]

[0186] Where Q and Q' are two independent Q value networks (one for selecting actions and one for evaluating actions); α is the learning rate; r t+1 is the immediate reward; γ is the discount factor; is the optimal action selected by the target network Q' in state s t+1 ;

[0187] 3.5.2 Prioritized Experience Replay

[0188] Enhance training efficiency by prioritizing experience with high TD error for replay, improve learning speed and effect, the formula derivation is as follows:

[0189] Priority calculation:

[0190] p i = |δ i | + ∈

[0191] Where p i is the priority of sample i; δ i is the TD error of sample i; ∈ is a small constant to prevent priority from being zero;

[0192] Sampling probability:

[0193]

[0194] Where P(i) is the probability of sample i being selected; α controls the importance of priority;

[0195] Importance sampling weight:

[0196]

[0197] Where w i is the importance sampling weight of sample i; β controls the smoothing degree of importance sampling; N is the size of experience pool;

[0198] Step 4: The trained model is deployed into the robot arm control system to process camera and sensor data in real-time; it is tested in actual operating environments to evaluate its recognition and positioning accuracy under various conditions, and parameters are optimized to improve efficiency;

[0199] 4.1 Model Deployment

[0200] Hardware Configuration: Use a computing platform with high-performance GPUs for model inference; integrate high-resolution cameras and depth sensors to ensure high-quality visual and depth information acquisition;

[0201] Software Configuration: PyTorch for model inference; Ubuntu 18.04 version for stable running environment; use ROS (Robot Operating System) for robot arm control and data communication;

[0202] Model Export: Use ONNX for model optimization to reduce inference time; export the trained model into a format suitable for deployment;

[0203] Model Integration: Integrate the optimized model into the robot arm control system to ensure that the model can process camera and sensor data in real-time; design data interfaces to ensure that the model can receive real-time data from the camera and depth sensor and send the processing results to the robot arm control module;

[0204] 4.2 Real-time Testing

[0205] Environment Setup: Simulate actual operating environments by setting up test scenarios with various lighting conditions and backgrounds, and placing goods of different types, sizes, and shapes; ensure that the robot arm can move freely to cover all positions in the test area;

[0206] Test Indicators: Evaluate the model's recognition accuracy of goods under different lighting conditions and backgrounds; evaluate the model's positioning accuracy of goods to ensure that the robot arm can accurately grasp the goods; record the time from data reception to output results to ensure real-time requirements; count the ratio of successful grasping to the number of attempts in different scenarios;

[0207] Test Process: The robot arm acquires images and depth information of goods at different positions and angles; multiple data collections are conducted under different lighting and background conditions to ensure data diversity;

[0208] Input real-time collected images and depth data into the model; the ResNet and DQN end-to-end model performs result inference to infer action instructions; control the robot arm to move and grasp goods according to the model output action instructions; record successful and failed operations according to the grasping results, and adjust model parameters to improve success rate;

[0209] Parameter optimization: according to the test results, adjust the learning rate, batch size, convolution kernel size and other hyperparameters of the model to improve the performance of the model; real-time test is carried out for many times, and the parameters are optimized continuously to ensure the stability and high efficiency of the model in various environments.

[0210] In combination with the accompanying Figures 1-3 The intelligent manufacturing shelf storage method is used for a wraparound shelf mechanism, which includes a shelf module 1 and a mechanical arm module 2; the shelf module 1 is provided with a shelf base 16 at the bottom, and the mechanical arm module 2 is placed on the shelf base 16;

[0211] The shelf module 1 includes a support vertical plate 11, an enclosing plate 12 and a placement plate 15; the enclosing plate 12 is an arc-shaped cavity, and a plurality of placement plates 15 are arranged in the longitudinal direction of the enclosing plate 12; in the longitudinal direction, a plurality of support vertical plates 11 are arranged between adjacent placement plates 15, and the support vertical plates 11 are used to cooperate with the mechanical arm module 2 to lift the height of the goods.

[0212] In the longitudinal direction, a large gear rack 141 is arranged on the support vertical plate 11, wherein the large gear rack 141 and a small gear rack 143 are arranged in parallel, and the large gear rack 141 and the small gear rack 143 are engaged with a gear 142, and the gear 142 is driven by a rudder; the small gear rack 142 is driven by the gear 142 to move upwards or downwards, and the upper end of the small gear rack 142 is provided with a lifting plate 13; the lifting plate 13 is an arc-shaped structure and cooperates with the placement plate 15 to realize the upward movement of the goods.

[0213] The rudder is arranged on a fixed clamping plate 144, and the output end of the rudder penetrates through a rudder transmission groove 145 to drive the gear 142; the fixed clamping plate 144 is fixedly connected with the large gear rack 141.

[0214] The transverse section of the enclosing plate 12 is a semi-ring shape.

[0215] The fixed clamping plate 144 is slidably connected with the small gear rack 141.

[0216] In use, the rudder rotates to drive the gear 142 to rotate, and then drive the small gear rack 143 to move, and the small gear rack 143 can only move up and down due to the limitation of the sliding groove; the lifting plate 13 is fixed with the small gear rack 143 through the groove, and the movement of the small gear rack finally drives the lifting plate 13 to move up and down.

[0217] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0218] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary, and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application without departing from the principles and spirit of the present application.

Claims

1. An intelligent manufacturing shelf storage method, integrating a camera and a depth sensor on a mechanical arm, combining a convolutional neural network and a reinforcement learning algorithm for target recognition and positioning, characterized in that, The method comprises the following steps: Step one: install high-resolution cameras and depth sensors on the mechanical arm to capture detailed visual and depth information of the goods, conduct large-scale data collection, including pictures of different types, sizes and shapes of goods and their corresponding depth information; each goods should be taken under different lighting and background conditions to enhance the generalization ability of the model; Step two: standardize the captured images, including scaling, cropping and normalization, to adapt to the input requirements of the convolutional neural network; label the goods in the image to determine the position of the bounding box for subsequent target detection training; Step three: select ResNet architecture for feature extraction: apply DQN method to let the model learn how to effectively identify and locate targets through interaction with the environment; use GPU-accelerated deep learning framework to train the model, input image data and corresponding label information, and conduct end-to-end learning; in step three, ResNet architecture feature extraction, DQN target identification and positioning, end-to-end learning combining ResNet and DQN, and improving ResNet algorithm are included; ResNet architecture feature extraction: ResNet network architecture is ResNet-50, using residual blocks for feature extraction; DQN target identification and positioning; End-to-end learning combining ResNet and DQN: input image is extracted into feature vector by ResNet and input into DQN network; According to the feature vector extracted by ResNet, the DQN network outputs the Q value of each action, and selects the optimal action; the mechanical arm executes the grabbing operation according to the optimal action output by the DQN network; according to the grabbing result, the corresponding reward is given, and the Q value is updated; the network parameters are adjusted through back propagation algorithm to optimize the joint model of ResNet and DQN; Improving ResNet algorithm: improving ResNet, optimizing its feature extraction ability and computational efficiency, including introducing attention mechanism and optimizing residual block structure; Introducing attention mechanism: introducing attention mechanism helps the network focus on important features better and improves the accuracy of feature extraction; SE module is a commonly used attention mechanism that can dynamically adjust the weights of each channel; Optimizing residual block structure: introducing bottleneck structure to reduce parameter quantity and improve computational efficiency; bottleneck structure reduces dimension through 1x1 convolution layer for dimension reduction and dimension increase to reduce calculation amount; Improving DQN algorithm: introducing double DQN, priority experience replay, fusion HER and meta-learning; Step four: deploy the trained model to the mechanical arm control system to process camera and depth sensor data in real time; test in actual operation environment to evaluate the recognition and positioning accuracy of the model under various conditions and optimize parameters to improve efficiency.

2. The intelligent manufacturing pallet storage method of claim 1, wherein, In step one, hardware configuration, data collection, data storage and processing are included; The hardware configuration includes a high-resolution camera and a depth sensor; the high-resolution camera is installed at the end of the mechanical arm to capture front and side images of the goods to capture high-resolution images of the goods and obtain detailed information; the depth sensor is installed at the end of the mechanical arm near the camera position to ensure synchronous acquisition of depth information to capture depth information of the goods and provide 3D image data; Data collection includes data collection strategy and data collection process; Data collection strategy collects images of goods and their corresponding depth images; covers various types of goods, including objects of different sizes, shapes, colors and materials; Shooting under various lighting conditions and background conditions; Data collection process uses adjustable LED light sources to simulate different lighting conditions; uses a mechanical arm to drive the camera and depth sensor to automatically capture images and depth information of the goods; shoots from different angles and distances to ensure data diversity; Data storage and processing use high-capacity NAS storage devices to ensure data security and accessibility; use LabelImg tool to label images to record the bounding box, class and depth information of the goods; Apply data augmentation techniques to increase data volume and improve model generalization.

3. The intelligent manufacturing pallet storage method of claim 1, wherein, Step two includes image processing and labeled data; The image processing includes scaling, cropping and normalization; Scaling, scaling the captured images to a uniform size to adapt to the input requirements of the convolutional neural network; for the interpolation method, it uses bilinear interpolation for image scaling to ensure image quality; Cropping, crop the goods part in the image to remove redundant background information; determine the cropping area according to the actual position of the goods; crop through the pre-labeled bounding box; appropriately expand the size of the bounding box during actual cropping to ensure that the complete information of the goods is included; Normalization, normalize the image pixel values to facilitate neural network training; the commonly used normalization method for mean and standard deviation is to standardize the image, that is, subtract the mean and divide by the standard deviation; Labeling data includes bounding box labeling, bounding box format, labeling tool and labeling data format; bounding box labeling labels the goods in the image to determine the position of the bounding box of the goods for subsequent target detection training; Bounding box format, use (xmin, ymin, xmax, ymax) labeling format to represent the coordinates of the top-left corner and bottom-right corner of the bounding box; Labeling tool, use LabelImg image labeling tool to manually label images; Labeling data format saves the labeling data in a standard format for subsequent loading and use; use XML format to store labeling data, each image corresponds to a labeling file; each labeling file contains image file name, image size, bounding box coordinates and class label.

4. The intelligent manufacturing shelf storage method of claim 1, wherein the improved ResNet algorithm formula is as follows: The basic convolution layer is: Y = Conv (X, W) + B X is the input image; W is the convolution kernel weight; B is the bias; The residual block formula is: The output of the standard residual block is: ​ x is an input; is a residual function consisting of a convolutional layer, batch normalization, and ReLU; Bottleneck residual block formula: The output of the optimized bottleneck residual block is: W1 is a 1x1 convolution kernel for dimension reduction; W2 is a 3x3 convolution kernel for feature extraction; W3 is a 1x1 convolution kernel for dimension increase; BN is batch normalization; and σ is a ReLU activation function. Introducing the SE module: The calculation process of the SE module includes Squeeze and Excitation: s = F ex (z, W) = σ(W2δ(W1z)) u is the input feature map; H and W are the height and width of the feature map, respectively; z is the output of the global average pooling; W1 and W2 are the weights of the fully connected layer; δ is the ReLU activation function; σ is the Sigmoid activation function; s is the channel weight; The output of the SE module is: v ＝ S·u s is the channel weight calculated by the SE module; u is the input feature map; The formula of the improved residual block: s is a channel weight calculated by the SE module; Double DQN, to solve the instability problem caused by overestimation in DQN algorithm, the formula derivation is as follows: The Q value update formula of traditional DQN: The Q value update formula of double DQN: where Q and Q' are two independent Q-value networks; a is the learning rate; r t+1 is the immediate reward; Y is a discount factor; is the optimal action selected by the target network Q' at state s t+1 under the policy p. Prioritized experience replay, to enhance training efficiency, by prioritizing the experience with high TD error for playback, to improve the speed and effect of learning, the formula derivation is as follows: Priority calculation: p i = |δ i |+∈ where p i is the priority of sample i; δ i is the TD error of sample i; ∈ is a small constant to prevent the priority from being zero; Sampling probability: P(i) is the probability of sample i being selected; α controls the importance of priority; Importance sampling weight: where w i is the importance sampling weight for sample i; β controls the smoothing degree of importance sampling; N is the size of the experience pool; The steps and formula derivation of meta learning: define the grasping task as a series of subtasks, each corresponding to a different target object or environmental state; In the meta training phase, by training the model on multiple subtasks, learn an initialization parameter, so that in the new task, a small amount of gradient update can quickly adapt; θ represents the parameters of the model; represents the update operation, which means assigning the expression on the right side to the variable on the left side; β represents the learning rate, which controls the step size of parameter update; represents the gradient with respect to the parameters θ; represents the prediction function of the model, with parameters θ; α represents the internal update step size, which is used for fast adaptation in the meta-training phase; i sum over all possible tasks T i sampled from the task distribution p(T); LTiLTi represents the loss function for task TiTi; f θ represents the prediction function of the model, with parameters θ; α represents the internal update step size, which is used for fast adaptation in the meta-training phase; In the meta testing phase, apply the initialization parameter learned in the meta training phase to the new task, and quickly adapt to the new task through a small amount of gradient update; θ' represents the updated model parameters on the new task; θ represents the initialization parameters learned in the meta-training phase; α represents the same as in the meta-training phase, used to control the gradient update step size on the new task; represents the loss function of the new task; f θ represents the model prediction function using the parameters θ obtained in the meta-training phase.

5. The intelligent manufacturing pallet storage method of claim 1, wherein, Step four includes model deployment and real-time testing; Model deployment includes hardware configuration, software configuration, model export and model integration; Hardware configuration, use a computing platform with high performance GPU for model inference; integrate high resolution camera and depth sensor to ensure high quality visual and depth information; Software configuration, PyTorch, for model inference; Ubuntu18.04 version, provides a stable running environment; use ROS for robot arm control and data communication; Model export, use ONNX for model optimization to reduce inference time; Export the trained model into a format suitable for deployment; Model integration, integrate the optimized model into the robot arm control system to ensure that the model can process camera and sensor data in real time; Design data interface to ensure that the model can receive real-time data from the camera and depth sensor and send the processing results to the robot arm control module; Real-time testing includes environment setup, test indicators, test process and parameter optimization; Environment setup, simulate the actual operation environment, build test scenes with multiple lighting conditions and backgrounds, and place goods of different types, sizes and shapes; Ensure that the robot arm can move freely and cover all positions in the test area; Test indicators, evaluate the model's recognition accuracy of goods under different lighting conditions and backgrounds; evaluate the model's positioning accuracy of goods' location to ensure that the robotic arm can accurately grasp the goods; record the time from data reception to output of the model to ensure real-time requirements; count the ratio of successful grasping of goods to the number of attempts by the robotic arm in different scenarios; Test process, the robotic arm acquires images and depth information of goods at different positions and angles; multiple data collections are conducted under different lighting and background conditions to ensure data diversity; Input real-time collected images and depth data into the model; ResNet and DQN end-to-end model infer results and output action instructions; control the robotic arm to move and grasp goods according to the model's output action instructions; According to the grasping results, record successful and failed operations and adjust model parameters to improve success rate; Parameter optimization, according to the test results, adjust the learning rate, batch size, and convolution kernel size of the model to improve model performance; perform real-time testing multiple times to continuously optimize parameters.

6. A wraparound shelving mechanism employing the intelligent manufacturing shelving storage method according to any one of claims 1 to 5, characterized by, The surrounding type shelf mechanism comprises a shelf module and a robotic arm module; the bottom of the shelf module is provided with a shelf base, and the robotic arm module is placed on the shelf base; The shelf module comprises a support upright plate, an enclosing plate, and a placement plate; the enclosing plate is an arc-shaped cavity, and multiple layers of placement plates are arranged along the longitudinal direction of the enclosing plate; a plurality of support upright plates are arranged between adjacent placement plates in the longitudinal direction, and the support upright plates are used to cooperate with the robotic arm module to lift the height of the goods.

7. The wraparound shelving mechanism of claim 6, wherein, Along the longitudinal direction, a large gear rack is arranged on the support upright plate, and the large gear rack and a small gear rack are arranged in parallel; the large gear rack and the small gear rack are engaged with a gear, and the gear is driven by a rudder; the small gear rack is driven by the gear to move upwards or downwards, and the upper end of the small gear rack is provided with a lifting plate; the lifting plate is an arc-shaped structure and cooperates with the placement plate to realize the upward movement of the goods.

8. The wraparound shelving mechanism of claim 7, wherein, The rudder is arranged on a fixed clamping plate, and the output end of the rudder penetrates through a rudder transmission groove to drive the gear; the fixed clamping plate is fixedly connected with the large gear rack.

9. The wraparound rack mechanism of claim 6, wherein, The transverse cross section of the enclosing plate is a semi-ring shape.

10. The wraparound rack mechanism of claim 8, wherein, The fixed clamping plate is slidably connected with the small gear rack.

Images