Forklift automatic guide vehicle, goods taking and unloading method, electronic equipment and storage medium

Abstract

The invention discloses a forklift automatic guide vehicle, a goods taking and unloading method, electronic equipment and a storage medium, and the method comprises the steps that target pallet information is obtained according to image information of a target goods pile and a taking and unloading task, and the target goods pile is a goods taking pile collected at a goods taking point or an unloading pile collected at an unloading point; the target cargo pile comprises at least one pallet on which cargoes are placed; and obtaining taking and unloading execution information according to the target pallet information so as to execute a goods taking or unloading action. According to the method, automatic goods taking and unloading operation of the multi-layer stacked goods can be achieved, the subsequent carrying task can be continuously executed instead of staying and waiting after each time of carrying, the stagnation time caused by waiting is shortened, and therefore the working efficiency of a logistics system is remarkably improved, and the labor intensity of workers is reduced. And the requirement of modern factories for carrying multi-layer goods piles is met.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular to a method for loading and unloading goods by a forklift automatic guided vehicle, an electronic device, a non-volatile readable storage medium and a forklift automatic guided vehicle. Background Art

[0002] In the current field of logistics automation, forklift automated guided vehicles (AGVs) are widely used in the handling of various types of goods, especially in industrial production and warehouse management. AGVs are mainly used in the scene of single-layer cargo handling, which effectively reduces manpower input. Specifically, AGVs can transport single-layer cargo from one fixed location to another, and then stay at that location, waiting for the cargo to be taken away by humans or other AGVs before continuing the next handling task.

[0003] However, with the continuous growth of modern industrial production needs, goods in factories are usually stored in multi-layer stacks. The single-layer cargo handling method can no longer meet the needs of modern factories for multi-layer stacked cargo handling, which will affect the space storage utilization of the factory. In addition, since the AGV must wait for the goods to be removed after each handling before continuing the next handling operation, the total time of logistics operations is extended, which greatly reduces the working efficiency of the logistics system and increases the overall operating cost. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one purpose of the present invention is to propose a method for picking up and unloading goods by a forklift automatic guided vehicle, which can realize the automatic picking up and unloading operation of multi-layer stacked goods, and no longer needs to stop and wait after each handling, but can continuously perform subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for multi-layer cargo stack handling.

[0005] A second objective of the present invention is to provide an electronic device.

[0006] A third object of the present invention is to provide a non-volatile readable storage medium.

[0007] A fourth object of the present invention is to provide a forklift automatic guided vehicle.

[0008] In order to achieve the above-mentioned purpose, the method for picking up and unloading goods by a forklift automatic guided vehicle in the first aspect of the embodiment of the present invention includes: obtaining target pallet information based on image information of the target cargo pile and the picking and unloading task, the target cargo pile is a picking cargo pile collected at a picking point or an unloading cargo pile collected at an unloading point, and the target cargo pile includes at least one pallet with goods placed thereon; obtaining picking and unloading execution information based on the target pallet information to execute the picking or unloading action.

[0009] According to the method for picking up and unloading goods by a forklift automated guided vehicle according to an embodiment of the present invention, the target pallet information is obtained by identifying the image information of the target cargo pile and combining it with the pre-set picking and unloading tasks including specific rules on how to pick up and unload the goods. That is, no matter it is a picking cargo pile at a picking point or a unloading cargo pile at an unloading point, the AGV can accurately identify the posture information of each pallet in the multi-layer stacked goods, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with goods in the target cargo pile, or neatly stacking the goods together according to the posture information of the target pallet, thereby realizing the automatic picking up and unloading operations of the multi-layer stacked goods, and no longer needs to stop and wait after each handling, but can continuously execute subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0010] In some embodiments, the target pallet information is obtained based on point cloud information of the target pallet; the target pallet is determined from valid pallets contained in the image information based on the picking and unloading task.

[0011] In some embodiments, the image information includes a grayscale image of the target cargo pile; the valid pallet is a complete pallet identified in the grayscale image.

[0012] In some embodiments, the complete pallet is obtained by removing the pallets with incomplete shapes from the annotated pallets contained in the grayscale image.

[0013] In some embodiments, the annotated pallet is obtained by identifying the grayscale image through a pallet detection model, wherein the pallet detection model takes the grayscale image as input and takes the type information and position information of the annotated pallet as output.

[0014] In some embodiments, the pallet detection model is based on YOLOV5, and the convolution module in the basic model is replaced by a pyramid selective kernel deep learning module. By using the pyramid selective kernel deep learning module to replace the ordinary convolution module, cross-dimensional spatial information and important features in the channel can be effectively extracted, and information from different receptive fields can be adaptively fused, thereby improving the model's ability to recognize the target pallet in the case of multi-layer cargo stacking.

[0015] In some embodiments, the pyramid selective kernel deep learning module performs the following processing: the input feature map is subjected to a first convolution processing using four different convolution kernels respectively; the four feature maps after the first convolution processing are superimposed in the channel dimension; the superimposed features are subjected to a second convolution processing using two different convolution kernels respectively; the two feature maps after the second convolution processing are subjected to global average pooling processing; the feature map after the global average pooling processing is subjected to fully connected layer processing; the feature map after the fully connected layer processing is multiplied with the feature map after the global average pooling processing and then output.

[0016] In some embodiments, the feature map after global average pooling processing is subjected to fully connected layer processing, including: the feature map after global average pooling processing is converted into a one-dimensional vector feature map and input into the first fully connected layer to change the size of the one-dimensional vector feature map; the feature map output by the first fully connected layer is input into two second fully connected layers, and the two second fully connected layers respectively input feature maps of the same size as the one-dimensional vector feature map; the feature maps output by the two second fully connected layers are respectively input into two normalization layers, and the feature maps output by the two normalization layers are used as the feature maps after processing by the fully connected layers.

[0017] In some embodiments, the first convolution processing of the four different convolution kernels includes convolution processing with a convolution kernel of 3x3, convolution processing with a convolution kernel of 5x5, convolution processing with a convolution kernel of 7x7, and convolution processing with a convolution kernel of 9x9. By dividing the input feature map into four parts and processing them through convolution operations of different scales, the model can process the input feature map in parallel at multiple scales, and by using convolution kernels of different sizes, multi-scale spatial information can be effectively learned, thereby establishing local cross-channel interactions.

[0018] In some embodiments, the second convolution processing of the two different convolution kernels includes a convolution processing with a convolution kernel of 3x3 and a convolution processing with a convolution kernel of 5x5.

[0019] In some embodiments, the unloading and picking tasks include picking tasks; the target pallet is a valid pallet determined according to a standard rule for picking multi-layer stacked goods corresponding to the picking task.

[0020] In some embodiments, the standard rules for picking up multi-layer stacked goods include: corresponding to the multi-layer stacked goods, selecting pallets in a top-to-bottom order in the vertical direction, and selecting pallets in a left-to-right order in the horizontal direction.

[0021] In some embodiments, the picking and unloading task also includes an unloading task; the target pallet is a valid pallet determined according to a standard rule for unloading multi-layer stacked goods corresponding to the unloading task.

[0022] In some embodiments, the standard rules for unloading multi-layer stacked cargo include: determining a target space in the multi-layer stacked cargo at the unloading point based on position information of the marked pallet, and selecting a pallet on the left, right, or below the target space.

[0023] In some embodiments, the standard rules for unloading multi-layer stacked goods also include: the multi-layer stacked goods at the unloading point are placed in order from bottom to top and from left to right, and the target space is selected according to the placement order of the multi-layer stacked goods at the unloading point.

[0024] In some embodiments, the image information of the target cargo pile includes a depth image; and the point cloud information of the target pallet is extracted from the point cloud information of the depth image.

[0025] In some embodiments, the target pallet information includes the position and angle of the target pallet; the position of the target pallet is determined based on the point cloud center of the target pallet, and the angle of the target pallet is calculated based on the geometric relationship of the point cloud center of the target pallet; wherein the point cloud center of the target pallet is obtained by performing plane segmentation and contour extraction on the point cloud information of the target pallet.

[0026] In some embodiments, when the unloading task is a unloading task, the target pallet information also includes goods on the target pallet, and the goods on the target pallet are extracted from the point cloud information of the depth map through filtering and clustering algorithms.

[0027] In some embodiments, when the picking and unloading task is a picking task, the picking and unloading execution information corresponds to the position and angle of the target pallet in the multi-layer stacked goods at the picking point.

[0028] In some embodiments, when the picking and unloading task is a unloading task, the picking and unloading execution information corresponds to the position and angle at which the cargo pallet needs to be placed; the position and angle at which the cargo pallet needs to be placed is determined by mapping the point cloud center and angle of the target pallet to the target space in the multi-layer stacked cargo at the unloading point.

[0029] In order to achieve the above-mentioned purpose, the electronic device of the second aspect embodiment of the present invention includes: at least one processor; a memory communicatively connected to the at least one processor; the memory stores a computer program executable by the at least one processor, and when the at least one processor executes the computer program, it implements the method of forklift automatic guided vehicle picking up and unloading goods described in the above embodiment.

[0030] According to the electronic device of the embodiment of the present invention, at least one processor can recognize the image information of the target cargo pile by executing the computer program of the method for picking up and unloading cargo by the forklift automatic guided vehicle described in the above embodiment, and obtain the target pallet information in combination with the pre-set picking and unloading tasks containing specific rules on how to pick up and unload cargo, that is, whether it is the picking cargo pile at the picking point or the unloading cargo pile at the unloading point, the AGV can accurately identify the posture information of each pallet in the multi-layer stacked cargo, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with cargo in the target cargo pile, or neatly stacking the cargo together according to the posture information of the target pallet, thereby realizing the automatic picking and unloading operations of the multi-layer stacked cargo, and no longer needs to stop and wait after each handling, but can continuously execute subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0031] In order to achieve the above-mentioned purpose, the non-volatile readable storage medium of the third aspect of the embodiment of the present invention stores a computer program thereon, and when the computer program is executed, the method of picking up and unloading goods by an automatic guided vehicle of a forklift as described in the above embodiment is implemented.

[0032] According to the non-volatile readable storage medium of the embodiment of the present invention, by adopting the method of forklift automatic guided vehicle picking up and unloading goods as described in the above embodiment, it is possible to realize automatic picking up and unloading operations of multi-layer stacked goods. It is no longer necessary to stop and wait after each handling, but it is possible to continuously perform subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the work efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo stacks.

[0033] In order to achieve the above-mentioned object, the forklift automatic guided vehicle according to the fourth aspect of the embodiment of the present invention includes the electronic device described in the above embodiment.

[0034] The forklift automated guided vehicle according to the embodiment of the present invention can recognize the image information of the target cargo pile by adopting the electronic device described in the above embodiment, and obtain the target pallet information in combination with the pre-set picking and unloading tasks including specific rules on how to pick up and unload the cargo, that is, whether it is the picking cargo pile at the picking point or the unloading cargo pile at the unloading point, the AGV can accurately recognize the posture information of each pallet in the multi-layer stacked cargo, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with cargo in the target cargo pile, or neatly stacking the cargo together according to the posture information of the target pallet, thereby realizing the automatic picking and unloading operations of the multi-layer stacked cargo, and no longer needs to stop and wait after each handling, but can continuously execute subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0035] In some embodiments, the forklift automated guided vehicle further includes an image acquisition device, which is connected to the electronic device and is used to acquire image information of the target cargo pile.

[0036] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 is a flow chart of a method for a forklift automated guided vehicle to load and unload cargo according to one embodiment of the present invention; Figure 2 is a schematic diagram of multi-layer cargo stacking according to one embodiment of the present invention; Figure 3 is a structural schematic diagram of a pallet detection model according to an embodiment of the present invention; Figure 4 is a schematic diagram of the structure of a pyramid selective kernel deep learning module according to an embodiment of the present invention; Figure 5 is an overall flow chart of a method for a forklift automated guided vehicle to load and unload cargo according to an embodiment of the present invention Figure 6 is a block diagram of an electronic device according to an embodiment of the present invention; Figure 7 is a block diagram of a forklift automatic guided vehicle according to one embodiment of the present invention; Figure 8 The figure is a schematic diagram of an automatic guided vehicle forklift picking up a pallet according to an embodiment of the present invention.

[0038] Reference numerals: Forklift AGV 100; Electronic device 10; Image acquisition device 20; Processor 11; Memory 12. DETAILED DESCRIPTION

[0039] Embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. Embodiments of the present invention are described in detail below.

[0040] Reference below Figure 1 A method for loading and unloading cargoes by an automatic guided vehicle of a forklift according to an embodiment of the present invention is described.

[0041] Figure 1 FIG. 1 is a flow chart of a method for loading and unloading cargoes by an automatic guided vehicle of a forklift according to an embodiment of the present invention. Figure 1 As shown, the method for loading and unloading cargo by an automatic guided vehicle of a forklift according to an embodiment of the present invention at least includes the following steps S1-S2.

[0042] S1, obtaining target pallet information according to the image information of the target cargo pile and the unloading task.

[0043] In some embodiments, the target cargo pile is a pickup cargo pile collected at a pickup point or a discharge cargo pile collected at a discharge point, and the target cargo pile includes at least one pallet on which cargo is placed.

[0044] Among them, the pickup point can be a specific storage area in the logistics warehouse or an area for temporary storage of goods. The pickup cargo pile can refer to a collection of goods stacked at the pickup point. These goods can be newly produced goods from the production line or goods after certain sorting or packaging operations. Goods of different types, quantities and sizes can be placed on each pallet. The pickup cargo pile can be single-layer or multi-layer stacked, and can be adjusted according to the production or storage needs of the factory. The unloading point can be a specific area in the logistics warehouse for unloading or the terminal location where the goods need to be stored. The unloading cargo pile can refer to a collection of goods stacked at the unloading point. The unloading cargo pile can be single-layer or multi-layer stacked. In order to improve the working efficiency of the logistics system and meet the needs of modern factories for the handling of multi-layer cargo piles, such as Figure 2 As shown, the method for loading and unloading cargoes by an automatic guided vehicle of a forklift of the present invention will be specifically described by taking a target cargo pile in a multi-layer stacked form as an example.

[0045] In some embodiments, the image information of the target cargo pile can be obtained by an image acquisition device (such as a camera, a laser radar, a 3D vision sensor, or an RGB-D camera, etc.). The image information of the target cargo pile can be detailed information about the target pallet extracted by the system from the image processing results, which includes not only the overall appearance of the pallet, but also specific details such as the height, position, and angle of the pallet.

[0046] In some embodiments, the picking and unloading tasks may be task instructions that include specific rules and strategies for picking and unloading. For example, the picking task may specify taking out a specific pallet from the picking pile according to certain picking rules, while the unloading task may be placing the goods and pallets at a specific position in the unloading pile according to certain unloading rules to achieve neat stacking.

[0047] In some embodiments, the picking and unloading tasks can be generated by a warehouse management system (WMS) or a factory logistics management system, and the AGVs can be automatically scheduled based on production or storage needs.

[0048] S2, obtaining the pickup and unloading execution information according to the target pallet information to execute the pickup or unloading action.

[0049] Specifically, the AGV generates an execution plan for picking up or unloading goods based on the target pallet information obtained. For example, when picking up goods, the AGV can drive the fork to move to the target pallet and adjust the angle and height of the fork to align with the pallet. By controlling the fork lifting mechanism, the fork is ensured to be inserted smoothly into the pallet. During this process, the AGV monitors the balance state of the goods in real time through sensors and the center of gravity detection system to ensure that the goods will not tilt or fall when picking up goods.

[0050] When unloading, the AGV can transport the goods and pallets to the unloading point according to the generated unloading execution information, and accurately place them in the designated space according to the preset unloading rules. Through sensor detection, the AGV can ensure that the goods are safely placed, and then the fork will automatically move away. In addition, in the scenario of multi-layer cargo stacking, the AGV can accurately adjust the fork height according to the target pallet information to ensure the stability of the cargo stacking and prevent the cargo from tipping over.

[0051] According to the method for picking up and unloading goods by a forklift automated guided vehicle according to an embodiment of the present invention, the target pallet information is obtained by identifying the image information of the target cargo pile and combining it with the pre-set picking and unloading tasks including specific rules on how to pick up and unload the goods. That is, no matter it is a picking cargo pile at a picking point or a unloading cargo pile at an unloading point, the AGV can accurately identify the posture information of each pallet in the multi-layer stacked goods, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with goods in the target cargo pile, or neatly stacking the goods together according to the posture information of the target pallet, thereby realizing the automatic picking up and unloading operations of the multi-layer stacked goods, and no longer needs to stop and wait after each handling, but can continuously execute subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0052] In some embodiments, the target pallet information is obtained based on the point cloud information of the target pallet. Point cloud information is a three-dimensional spatial data, which is a data set composed of countless points, representing different points on the surface of an object in space. Each point has precise three-dimensional coordinates (x, y, z), thus forming the geometric shape and spatial distribution of the object. Therefore, this three-dimensional spatial data can provide spatial position information and detailed geometric information such as three-dimensional shape, surface contour, and size about the target pallet, which enables the AGV to more accurately obtain the position of the target pallet when picking up goods, or the location area where the target pallet needs to be placed when unloading goods when performing picking and unloading tasks.

[0053] In some embodiments, the target pallet is determined from the valid pallets contained in the image information according to the picking and unloading task. Among them, the valid pallet may refer to all pallets that meet the conditions and can perform the picking and unloading operations. Since there may be multiple pallets in the picking cargo pile and the unloading cargo pile at the same time, the system first determines all the pallets available for operation in the current scene through image processing analysis, and screens them according to their position, size and posture. After all the valid pallets are screened out, the system can also finally determine a valid pallet as the current target pallet according to the specific requirements of the current picking and unloading task.

[0054] In some embodiments, the image information includes a grayscale image of the target cargo pile. A grayscale image is a form of image in which the value of each pixel represents the brightness of the pixel, rather than the color. The pixel values ​​of a grayscale image are usually between 0 and 255, where 0 represents black, 255 represents white, and values ​​in between represent different degrees of gray. By capturing the intensity of light, a grayscale image can effectively express the contour and structural information of an object. For a target cargo pile, a grayscale image can reflect the overall contour, size, shape, position, and other information of the pallet and the cargo on it, providing accurate reference data for subsequent operations.

[0055] In some embodiments, a valid pallet is a complete pallet identified in a grayscale image. Among them, a complete pallet may refer to a pallet with physical integrity, for example, a pallet that is physically free of defects, cracks, or other structural damage. This is because a damaged pallet may not provide sufficient support during transportation, causing the goods to tilt or fall, thereby increasing operational risks. In addition, a complete pallet may also refer to a pallet with spatial integrity, for example, a pallet that is not partially obscured by other goods, obstacles, or other pallets. This is because if the pallet is partially obscured in the image, the system cannot accurately identify the complete outline and posture information of the pallet, and may misestimate its position or angle, which may cause deviations when performing pickup or unloading operations, affecting the accuracy of the operation.

[0056] In some embodiments, a complete pallet is obtained by removing the pallets with incomplete shapes from the annotated pallets contained in the grayscale image. The annotated pallets can be potential pallet positions extracted from the grayscale image by image processing technology or machine learning models. These annotated pallets represent all pallet candidate areas identified in the image processing, and each annotated pallet can be represented by a bounding box to represent its position. However, this does not mean that all identified annotated pallets are suitable for picking up or unloading operations. By screening these annotated pallets and removing pallets with incomplete shapes, that is, pallets that appear missing, damaged, deformed, or obscured in the image, it is ensured that the AGV only focuses on those pallets that are complete physically and spatially, so as to ensure the stability of the goods during the handling process and the safety and accuracy of the operation.

[0057] In some embodiments, the labeled pallet is obtained by recognizing a grayscale image through a pallet detection model, which takes the grayscale image as input and takes the type information and position information of the labeled pallet as output.

[0058] Among them, the stack detection model can identify and annotate stacks from grayscale images through deep learning technology, especially convolutional neural networks (CNN) or other similar computer vision models. The stack detection model takes the grayscale image as input and gradually extracts the features of the annotated stack, such as type information and location information, through multi-layer convolution, pooling, full connection, splicing, feature fusion and other processing steps.

[0059] In some embodiments, the training process of the pallet detection model is based on a large number of annotated data sets. Specifically, by taking images of multi-layer stacked goods in real factories, these images include different types of pallets, goods on top of the pallets, and different lighting conditions and shooting angles, etc., hundreds of pallet detection data sets are constructed. The category information and location information of the pallets in the image are annotated by manual annotation to generate an annotated data set. The pallet detection model is trained using the annotated data set to learn how to automatically detect and identify the type information and location information of the pallets in new images. During the training process of the model, the system gradually optimizes the model parameters by learning the annotated data, thereby improving its detection accuracy and robustness for pallets.

[0060] Figure 3 is a schematic diagram of the structure of a pallet detection model according to an embodiment of the present invention. Figure 3 As shown in the figure, the stack detection model is based on YOLOV5, which is a deep learning model widely used in target detection tasks. It has efficient detection capabilities and fast reasoning speed. Especially when dealing with target detection in complex environments, YOLOV5 effectively extracts features and performs target positioning and classification through convolutional neural networks (CNN).

[0061] However, since there are multiple pallets in the captured image, and the types of pallets may be different, the speed and accuracy of the model's pallet detection have a great impact on whether the AGV can smoothly pick up and unload goods. Figure 3 As shown in the figure, by replacing the convolution module in the YOLOV5 basic model with the pyramid selective kernel (PSK) deep learning module, the cross-dimensional spatial information and important features in the channel can be effectively extracted, and the information of different receptive fields can be adaptively integrated, thereby improving the recognition ability of labeled pallets in the case of stacked goods.

[0062] Figure 4 is a schematic diagram of the structure of a pyramid selective kernel deep learning module according to an embodiment of the present invention. Figure 4 As shown, the pyramid selective kernel deep learning module performs the following processing: First, the input feature map is convolved with four different convolution kernels. Specifically, the input feature map size is , where H represents height, W represents width, and C represents the number of channels. Then, the channel dimension of the feature map is divided into four parts through four different convolution kernels, and the number of channels in each part is compressed to one-fourth of the original, resulting in four dimensions of Each convolution kernel independently performs convolution operations on these four feature maps to extract local feature information at different scales.

[0063] Therefore, by dividing the channel dimension of the input feature map into four parts and processing them through convolution operations of different scales, the model can process the input feature map in parallel at multiple scales and extract features of different scales at the same time. Parallel processing improves efficiency and detection diversity, can better adapt to objects of different sizes, effectively learn multi-scale spatial information, and establish local cross-channel interactions.

[0064] In some embodiments, the first convolution processing of four different convolution kernels includes a convolution processing with a convolution kernel of 3x3, a convolution processing with a convolution kernel of 5x5, a convolution processing with a convolution kernel of 7x7, and a convolution processing with a convolution kernel of 9x9. In a convolutional neural network, the size of the convolution kernel determines the size of the local area perceived by the network, that is, the receptive field. Smaller convolution kernels (such as 3x3) are good at extracting local detail features, while larger convolution kernels (such as 9x9) can cover a larger range of image areas and capture a wider range of contextual information.

[0065] Second, the four feature maps after the first convolution are superimposed on the channel dimension. Specifically, the four feature maps with a size of The feature maps of are concatted in the channel dimension, and the size of the feature map after concatination is consistent with the size of the input feature map. It can be expressed as: ; in, is the superimposed feature map; Concat means superposition in the channel dimension; x is the input feature map; It is a 3x3 convolution kernel; is a 5x5 convolution kernel; is a 7x7 convolution kernel; is a 9x9 convolution kernel.

[0066] Third, the superimposed feature maps are subjected to a second convolution process with two different convolution kernels. The second convolution process with two different convolution kernels includes a convolution process with a convolution kernel of 3x3 and a convolution process with a convolution kernel of 5x5. The second convolution process is a further refinement of the image features after the first convolution process. After the first convolution process and channel superposition, the model has captured preliminary features at different scales, and the second convolution process processes these features more finely to further extract and optimize multi-scale features.

[0067] Fourth, the two feature maps after the second convolution process are processed by global average pooling (AvgPool). Specifically, the feature maps after the second convolution process with two different convolution kernels are added together so that the feature maps contain more receptive field information. Then, the feature maps after the addition operation are processed by global average pooling (AvgPool). Global average pooling is to average all pixel values ​​in each channel. These average values ​​represent the global information on the channel and help the network understand the importance of each channel. The specific formula is as follows: ; in, It is the feature map after global average pooling; is global average pooling; is the feature map after superposition; It is a 3x3 convolution kernel; is a 5x5 convolution kernel.

[0068] Fifth, the feature map after global average pooling is processed by the fully connected layer (FC Layer). Among them, the feature map after global average pooling is processed by the fully connected layer, including: the feature map after global average pooling is converted into a one-dimensional vector feature map and input into the first fully connected layer to change the size of the one-dimensional vector feature map. The feature map output by the first fully connected layer is input into the two second fully connected layers, and the two second fully connected layers respectively input feature maps of the same size as the one-dimensional vector feature map. The feature maps output by the two second fully connected layers are respectively input into two normalization layers (SoftMax), and the feature maps output by the two normalization layers are used as the feature maps after being processed by the fully connected layers.

[0069] Specifically, assuming that the feature map size after global average pooling is In order to enter the first fully connected layer, the feature map needs to be converted into a one-dimensional vector feature map. In this way, the spatial features are effectively compressed into vector features that only retain the channel dimension. This conversion helps in weight calculation and feature mapping in subsequent fully connected layers.

[0070] Furthermore, the converted one-dimensional vector feature map is input into the first fully connected layer. The main function of this layer is to map the input one-dimensional vector feature map from the original C dimension (number of channels) to a new dimension through linear transformation (i.e., matrix multiplication and weighted sum), i.e. Dimension.

[0071] Furthermore, the feature map output by the first fully connected layer is input into two second fully connected layers. The second fully connected layer can convert the output of the first fully connected layer into a dimension of The one-dimensional vector feature map is mapped back to the original C dimension, that is, the dimension of the channel is restored, thus completing the information extraction of the channel dimension. This process is to reconstruct the representation of each channel by learning weights to enhance the features of important channels. Therefore, the role of the fully connected layer is to map the input feature map from a high-dimensional space to a low-dimensional space and then map it back to the original space. The fully connected layer transforms the input data through linear transformation, so that the information of each channel can be expressed more efficiently.

[0072] Furthermore, the feature maps output by the two second fully connected layers are respectively input into two normalization layers. The normalization layer is used to convert the feature value of each channel into a probability score, that is, to map the feature value of each channel to the range of [0,1] and ensure that the sum of the normalized feature values ​​of all channels is 1. Therefore, the normalization layer can give the weight (attention score) of each channel, reflecting the importance of each channel in the entire feature map. In addition, the role of the normalization layer is to rescale the output features to ensure that the differences between different features can be effectively suppressed or amplified, so that the model can learn and optimize parameters more stably. This step is very important when training deep neural networks because it can speed up the training convergence and improve the generalization ability of the model.

[0073] Sixth, the feature maps output by the two normalization layers are used as the feature maps processed by the fully connected layer, and are multiplied (Matmul) with the feature maps processed by the global average pooling before output. This allows the model to focus on the channel features with higher weights when generating the final output, thereby enhancing the representation of important features. The specific formula is as follows: ; in, The feature map output by the entire pyramid selective kernel deep learning module; is the feature map after global average pooling processing; SoftMax is the normalization layer, which normalizes the feature map and converts it into an attention score on each channel; FCLayer is the fully connected layer, the first fully connected layer changes the size of the one-dimensional vector feature map, and the second fully connected layer restores the size of the one-dimensional vector feature map to its original size; Matmul is the multiplication processing, that is, the feature map processed by the fully connected layer is multiplied by the feature map processed by the global average pooling.

[0074] Therefore, this step further integrates multi-scale features and global information, enhances the model's ability to represent pallet features, and thus improves the detection accuracy and robustness of pallets. Especially in complex stacking scenarios, the model can more accurately and quickly identify and locate annotated pallets.

[0075] In some embodiments, the unloading task includes a picking task, and the target pallet is a valid pallet determined according to the standard rules for picking up multi-layer stacked goods corresponding to the picking task. The standard rules for picking up multi-layer stacked goods may refer to a set of predefined rules followed by the system to ensure accurate, safe and efficient selection of target pallets and goods from multi-layer stacked goods when performing a picking operation. The rules can help the system determine how to prioritize the selection of appropriate pallets in a complex stacking environment to avoid the collapse or damage of the picking goods pile.

[0076] In some embodiments, the standard rules for picking up multi-layer stacked goods include: corresponding to the multi-layer stacked goods, selecting pallets in a top-to-bottom order in the vertical direction, and selecting pallets in a left-to-right order in the horizontal direction.

[0077] Among them, in the vertical direction, the pallets are selected in order from top to bottom, which means that the AGV starts from the top layer of the multi-layer stacked goods and picks up the goods layer by layer. This rule can ensure that when performing the picking operation, the pallets that are easiest to take out are processed first, avoiding the risk of imbalance or tipping of the upper layer of goods when the pallets are extracted from the lower layer of the stacked goods, thereby ensuring the stability of the stacking of the picked goods. In the horizontal direction, the pallets are selected in order from left to right. This rule can ensure that the picking order of the pallets on each layer is carried out in an orderly manner, and no valid pallets will be missed or there will be jump selection, thereby improving the orderliness and accuracy of picking.

[0078] In some embodiments, the unloading task also includes an unloading task, and the target pallet is a valid pallet determined according to the standard rules for unloading multi-layer stacked goods corresponding to the unloading task. The standard rules for unloading multi-layer stacked goods may refer to a set of predefined rules followed by the system during the unloading operation to ensure that the pallets and goods grabbed by the AGV are accurately, safely and efficiently placed in the empty space in the unloading cargo pile. The rules can ensure the orderly progress of the unloading process and reduce the risk of unstable stacking or accidents of the unloading cargo pile caused by improper operation.

[0079] In some embodiments, the standard rules for unloading multi-layer stacked goods include: determining the target empty space in the multi-layer stacked goods at the unloading point according to the position information of the marked pallets, and selecting the pallet on the left, right or below the empty space as a reference point. Specifically, in the unloading operation of the multi-layer stacked goods, it is first necessary to obtain the position information of all the marked pallets. This information can be detected and extracted by the pallet detection model described in the above embodiment.

[0080] Furthermore, after the location information of all marked pallets is obtained, the system can obtain and update all vacant space information in the stacked cargo area in real time. Among them, vacant space refers to the space area without pallets, which may be the target vacant space position for the next unloading operation.

[0081] Furthermore, in combination with the standard rules for unloading multi-layer stacked goods, the target space in the multi-layer stacked goods at the unloading point can be determined, and then the pallet on the left, right or below the target space is selected as a reference point. By analyzing the distance between the pallet and the target space, the pallet angle, the pallet position, etc., the system can ensure that the pallet to be unloaded can still keep the entire unloaded cargo pile stable and tidy after being placed in the target space, avoiding collisions between pallets or imbalance of cargo.

[0082] For example, when it is detected that there is a target space that can be placed at the bottom of the unloading cargo pile, the pallet on the left side of the target space can be preferentially selected as the target pallet (reference point). By obtaining the target pallet information, the corresponding unloading execution information can be obtained to perform the unloading action. When the pallet on the left side of the target space does not belong to the complete pallet identified in the grayscale image, the pallet on the right side of the target space can be selected as the target pallet (reference point). When it is detected that there is no target space that can be placed at the bottom of the unloading cargo pile and there is a space above the bottom layer, in order to ensure the stability and neatness of the upper and lower layers of goods and pallets, the system will select the pallet on the bottom layer corresponding to the target space as the target pallet (reference point), accurately control the unloading action of the AGV, so that the pallet to be unloaded can be accurately and safely placed in the target space.

[0083] In some embodiments, the standard rules for unloading multi-layer stacked goods further include: the multi-layer stacked goods at the unloading point are placed in the order from bottom to top and from left to right, and the target space is selected according to the placement order of the multi-layer stacked goods at the unloading point. Therefore, the stability and neatness of the stacking of goods during the unloading process are ensured by placing them in the order from bottom to top and from left to right.

[0084] In some embodiments, the image information of the target cargo pile includes a depth image. The depth image may be an image showing the distance information of each point in the scene relative to the image acquisition device. Different from ordinary RGB images or grayscale images, each pixel value in the depth image represents the depth of the position (the distance from the device). By analyzing these distance data, the system can accurately obtain the three-dimensional spatial layout of the target cargo pile.

[0085] In some embodiments, the depth image can be acquired by an image acquisition device such as an RGB-D camera. After the camera captures the depth image, it can convert the depth image into point cloud information. Specifically, in the depth image, the value of the pixel represents the depth information from the camera. In order to convert it into a coordinate point in three-dimensional space, it is necessary to know the intrinsic parameters of the camera (such as focal length, optical center position, etc.). From the pixel coordinates (u, v) and depth value (d) of the depth image, the intrinsic parameters of the camera can be used to convert it into three-dimensional coordinates (x, y, z), and the formula is as follows: ; ; ; Among them, and is the coordinate of the optical center of the camera, and is the focal length of the camera, and d is the depth value. Through the above formula, the depth data of all pixels can be converted into coordinates in three-dimensional space, and these three-dimensional coordinates can form the point cloud information of the depth image.

[0086] Furthermore, after the depth image is converted into point cloud information, the annotated pallet point cloud selected in the grayscale image can be extracted from the point cloud information. Then, the system can extract a specific pallet part, that is, the point cloud information of the target pallet, from the annotated pallet point cloud through point cloud segmentation technology.

[0087] In some embodiments, the target pallet information includes the position and angle of the target pallet. The position of the target pallet is determined according to the point cloud center of the target pallet. The point cloud center of the target pallet is obtained by performing plane segmentation and contour extraction on the point cloud information of the target pallet.

[0088] Among them, the pallet is an object with a regular planar structure. Using a plane segmentation algorithm (such as the RANSAC algorithm), the plane of the pallet can be extracted from the point cloud information of the target pallet and distinguished from other parts (such as goods or other pallets). The plane segmentation algorithm finds an optimal plane equation so that the plane can best fit the point cloud information of the target pallet surface.

[0089] In some embodiments, contour extraction is to extract the boundary of the pallet from the point cloud information of the surface of the target pallet. Common algorithms for contour extraction include edge detection, curve fitting, etc. By analyzing the geometric features of the contour, the straight line segments of the contour (such as the height of the pallet support column and the length of the lower surface of the pallet) can be extracted. By calculating the intersection of these straight line segments, the three-dimensional coordinates of the point cloud center of the target pallet can be determined, thereby determining the specific position of the target pallet.

[0090] In some embodiments, the angle of the target pallet is calculated based on the geometric relationship of the point cloud center of the target pallet. Specifically, the angle between the normal vector n of the point cloud center of the target pallet and the x-axis is calculated to obtain the angle offset of the pallet. The specific formula is as follows: ; Where angle is the offset angle; arccos is the inverse cosine function; n is the normal vector; .x is the x-axis vector (1, 0, 0); It is the dot multiplication operation of vectors; and is the magnitude of the vector, representing the length of the normal vector and the x-axis vector respectively.

[0091] In some embodiments, when the unloading task is an unloading task, the target pallet information also includes the goods on the target pallet, and the goods on the target pallet are extracted from the point cloud information of the depth map through filtering and clustering algorithms. Among them, the purpose of filtering is to determine whether there are outliers by calculating the distance between each point in the point cloud information and the surrounding points. Outliers can be noise data or points that do not belong to the goods on the target pallet. They may be errors or other objects in the environment. By calculating the distance between each point and the points in its neighborhood, it is determined whether this point is far away from other points, thereby determining whether it is an outlier. If it is an outlier, it is removed from the point cloud to ensure the accuracy of the data.

[0092] In some embodiments, after filtering, the system can use a clustering algorithm to segment the point cloud data and extract the goods from the point cloud area above the pallet. The clustering algorithm aggregates the dense point cloud area into a cluster by judging the distance between points, and then determines the location and outline of the goods. In the clustering algorithm, the dense point cloud can be segmented into clusters, each cluster representing an independent goods. Through clustering, the system can effectively distinguish between the target pallet and the goods above the target pallet.

[0093] In some embodiments, when the unloading task is a picking task, the unloading execution information corresponds to the position and angle of the target pallet in the multi-layer stacked goods at the picking point. The AGV can adjust the position, angle and direction of its fork according to the position and angle of the target pallet to ensure that the target pallet can be accurately grabbed from the picking goods pile.

[0094] In some embodiments, when the unloading task is an unloading task, the unloading execution information corresponds to the position and angle at which the cargo pallet needs to be placed. The position and angle at which the cargo pallet needs to be placed is determined by mapping the point cloud center and angle of the target pallet to the target empty space in the multi-layer stacked cargo at the unloading point.

[0095] Among them, the target pallet in the unloading task is the existing pallet in the multi-layer stacked goods described in the above embodiment. The target pallet may be located on the left, right or below the target space, and serves as a reference point in the unloading process. By mapping the point cloud center and angle of the target pallet with the target space at the unloading point, it can be ensured that the newly placed pallet (i.e., the pallet of the goods to be unloaded) is aligned with the existing pallet in position and angle. This alignment is to ensure that the newly placed pallet is consistent with the existing stacking structure, so that the entire unloading process remains neat and stable, avoiding tilted or unstable stacking.

[0096] Figure 5 FIG. 1 is an overall flow chart of a method for loading and unloading cargoes by an automatic guided vehicle of a forklift according to an embodiment of the present invention. Figure 5 As shown, the overall process of the method for loading and unloading cargo by an automatic guided vehicle for forklift includes at least the following steps S10-S14.

[0097] S10, the AGV arrives at the designated station according to the task, completes the fork lifting at the station, and then sends a signal, which includes the next picking or unloading action. After receiving the signal sent by the controller, the industrial computer will control the RGB-D camera to turn on and take pictures of the scene, capturing grayscale images and depth images.

[0098] S11, using the pallet detection model to detect the grayscale image and remove the pallets with incomplete shapes to obtain all the complete labeled pallets contained in the grayscale image, and then find the required labeled pallets according to the picking and unloading tasks.

[0099] S12, converting the depth image into point cloud information, and extracting the point cloud information of the target pallet from the point cloud information of the depth image, obtaining the point cloud center of the target pallet by performing plane segmentation and contour extraction on the point cloud information of the target pallet, and obtaining its three-dimensional coordinates.

[0100] S13, through the three-dimensional coordinates of the point cloud center of the target pallet, respectively calculate the offset angle and position of the point cloud center relative to the camera, and obtain the picking and unloading execution information according to the picking task to execute the picking operation.

[0101] S14, by mapping the point cloud center and angle of the target pallet to the target empty space in the multi-layer stacked goods at the unloading point, the unloading execution information of the position and angle at which the corresponding cargo pallet needs to be placed can be obtained to perform the unloading operation.

[0102] In general, by adopting the above steps, the automated picking and unloading operations of multi-layer stacked goods can be realized. There is no need to stop and wait after each handling, but the subsequent handling tasks can be performed continuously, reducing the stagnation time caused by waiting, thereby significantly improving the work efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo stacks.

[0103] Reference below Figure 6 An electronic device according to an embodiment of the present invention is described.

[0104] Figure 6 is a block diagram of an electronic device according to an embodiment of the present invention, such as Figure 6 As shown, the electronic device 10 according to the embodiment of the present invention includes at least one processor 11 and a memory 12 .

[0105] In some embodiments, at least one processor 11 may be one processor 11, or may be two processors 11, three processors 11, five processors 11, or multiple processors 11. The processor 11 may be a single-core or multi-core processor 11, responsible for executing the method for picking up and unloading goods by a forklift automatic guided vehicle in the memory 12. The processor 11 may be a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA) in the electronic device 10, etc. It depends on the design and purpose of the electronic device 10.

[0106] In some embodiments, the memory 12 can be divided into multiple types, including RAM (random access memory) and ROM (read-only memory), as well as other high-speed cache storage units. When executing tasks, the processor 11 can read program codes from the memory 12 and access real-time data, such as point cloud information of the target stack, image data, sensor data, etc.

[0107] In some embodiments, the memory 12 is communicatively connected to at least one processor 11, and the memory 12 stores a computer program that can be executed by at least one processor 11. When at least one processor 11 executes the computer program, the method of forklift automatic guided vehicle picking up and unloading goods described in the above embodiment is implemented.

[0108] According to the electronic device 10 of the embodiment of the present invention, at least one processor 11 can recognize the image information of the target cargo pile by executing a computer program that implements the method of picking up and unloading goods by a forklift automatic guided vehicle as described in the above embodiment, and obtain the target pallet information in combination with the pre-set picking and unloading tasks containing specific rules on how to pick up and unload goods, that is, whether it is a picking cargo pile at a picking point or a unloading cargo pile at an unloading point, the AGV can accurately identify the posture information of each pallet in the multi-layer stacked cargo, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with goods in the target cargo pile, or neatly stacking the goods together according to the posture information of the target pallet, thereby realizing the automatic picking and unloading operations of the multi-layer stacked cargo, and no longer needs to stop and wait after each handling, but can continuously execute subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0109] The embodiment of the present invention further provides a non-volatile readable storage medium on which a computer program is stored. When the computer program is executed, the method for picking up and unloading goods by a forklift automatic guided vehicle described in the above embodiment is implemented. The specific implementation process of the method for picking up and unloading goods by a forklift automatic guided vehicle can refer to the description of the above embodiment.

[0110] According to the non-volatile readable storage medium of the embodiment of the present invention, by adopting the method of forklift automatic guided vehicle picking up and unloading goods as described in the above embodiment, it is possible to realize automatic picking up and unloading operations of multi-layer stacked goods. It is no longer necessary to stop and wait after each handling, but it is possible to continuously perform subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the work efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo stacks.

[0111] Reference below Figure 7 An automatic guided vehicle for a forklift according to an embodiment of the present invention is described.

[0112] Figure 7 is a block diagram of a forklift automatic guided vehicle according to an embodiment of the present invention, Figure 7 As shown, the forklift automatic guided vehicle 100 according to the embodiment of the present invention includes the electronic device 10 described in the above embodiment.

[0113] In some embodiments, the forklift automatic guided vehicle (AGV) 100 is an automated transport tool. Its basic task is to automatically transport goods according to a predetermined route. The AGV can move autonomously in a warehouse or factory through various navigation technologies (such as laser, magnetic stripe, vision, inertial sensor, etc.) to automatically complete the picking and unloading operations. Therefore, the forklift automatic guided vehicle 100 is widely used in modern warehousing and logistics, which can improve work efficiency, reduce manual operation costs and ensure operation accuracy.

[0114] According to the forklift automatic guided vehicle 100 of the embodiment of the present invention, by adopting the electronic device 10 described in the above embodiment, it is possible to identify the image information of the target cargo pile, and obtain the target pallet information in combination with the pre-set picking and unloading tasks including specific rules on how to pick up and unload the cargo, that is, whether it is a picking cargo pile at a picking point or a unloading cargo pile at a unloading point, the AGV can accurately identify the posture information of each pallet in the multi-layer stacked cargo, and generate picking and unloading execution information based on this, thereby realizing the acquisition of the target pallet with cargo in the target cargo pile, or neatly stacking the cargo together according to the posture information of the target pallet, thereby realizing the automatic picking and unloading operations of the multi-layer stacked cargo, and no longer needs to stop and wait after each handling, but can continuously perform subsequent handling tasks, reducing the stagnation time caused by waiting, thereby significantly improving the working efficiency of the logistics system and meeting the needs of modern factories for the handling of multi-layer cargo piles.

[0115] In some embodiments, Figure 7 and Figure 8 As shown, the forklift automatic guided vehicle 100 also includes an image acquisition device 20. Among them, the image acquisition device 20 can be an RGB-D camera, which can simultaneously capture grayscale images and depth images. The grayscale image can be detected and identified by a pallet detection model, and the area that meets the characteristics of the pallet is extracted therefrom. After the recognition is completed, the system can filter out pallets with incomplete shapes or areas that do not meet the characteristics of the pallet, ensuring that only complete labeled pallets are taken into account in subsequent processing. The depth image can be converted into point cloud information, and the point cloud information can be used to obtain the geometric center and angle information of the target pallet. This information is very important for the subsequent picking and unloading control of the forklift automatic guided vehicle 100.

[0116] In some embodiments, the image acquisition device 20 is connected to the electronic device 10 to acquire image information of the target cargo pile. The electronic device 10 can process the image information in real time by executing a specific computer program, extract useful target information (such as the position and angle of the target pallet, etc.), and make subsequent decisions on the unloading and picking operations.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0118] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for loading and unloading cargo by a forklift automatic guided vehicle, characterized in that: include: Obtaining target pallet information according to the image information of the target cargo pile and the picking and unloading tasks, wherein the target cargo pile is a pickup cargo pile collected at a pickup point or an unloading cargo pile collected at an unloading point, and the target cargo pile includes at least one pallet on which cargo is placed; The picking and unloading execution information is obtained according to the target pallet information to execute the picking or unloading action.

2. The method according to claim 1, characterized in that The target pallet information is obtained based on the point cloud information of the target pallet; The target pallet is determined from valid pallets contained in the image information according to the picking and unloading task.

3. The method according to claim 2, characterized in that The image information includes a grayscale image of the target cargo pile; The valid pallet is a complete pallet identified in the grayscale image.

4. The method according to claim 3, characterized in that The complete pallet is obtained by removing the pallets with incomplete shapes from the annotated pallets contained in the grayscale image.

5. The method according to claim 4, characterized in that The annotated pallet is obtained by identifying the grayscale image through a pallet detection model, wherein the pallet detection model takes the grayscale image as input and takes the type information and position information of the annotated pallet as output.

6. The method according to claim 5, characterized in that The stack detection model is based on YOLOV5, and the convolution module in the basic model is replaced by a pyramid selective kernel deep learning module.

7. The method according to claim 6, characterized in that The pyramid selective kernel deep learning module performs the following processing: The input feature map is subjected to the first convolution process of four different convolution kernels; The four feature maps after the first convolution process are superimposed in the channel dimension; The superimposed feature maps are subjected to a second convolution process using two different convolution kernels; The two feature maps after the second convolution process are subjected to global average pooling process; The feature map after the global average pooling process is processed by a fully connected layer; The feature map processed by the fully connected layer is multiplied with the feature map processed by the global average pooling and then output.

8. The method according to claim 7, characterized in that The feature map after the global average pooling process is subjected to a fully connected layer process, including: The feature map after the global average pooling process is converted into a one-dimensional vector feature map and input into the first fully connected layer to change the size of the one-dimensional vector feature map; The feature map output by the first fully connected layer is input to two second fully connected layers, and the two second fully connected layers respectively input feature maps of the same size as the one-dimensional vector feature map; The two feature maps output by the second fully connected layers are respectively input to two normalization layers, and the feature maps output by the two normalization layers serve as feature maps processed by the fully connected layers.

9. The method according to claim 7, characterized in that: The first convolution processing of the four different convolution kernels includes convolution processing with a convolution kernel of 3x3, convolution processing with a convolution kernel of 5x5, convolution processing with a convolution kernel of 7x7, and convolution processing with a convolution kernel of 9x9.

10. The method according to claim 7, characterized in that The second convolution processing of the two different convolution kernels includes convolution processing with a convolution kernel of 3x3 and convolution processing with a convolution kernel of 5x5.

11. The method according to claim 2, characterized in that The cargo pickup and unloading tasks include cargo pickup tasks; The target pallet is a valid pallet determined according to a standard rule for picking up multi-layer stacked goods corresponding to the picking task.

12. The method according to claim 11, characterized in that The standard rules for picking up multi-layer stacked goods include: corresponding to the multi-layer stacked goods, selecting pallets in a vertical direction in a top-to-bottom order, and selecting pallets in a horizontal direction in a left-to-right order.

13. The method according to claim 11, characterized in that The picking and unloading tasks also include unloading tasks; The target pallet is a valid pallet determined according to a standard rule for unloading multi-layer stacked goods corresponding to the unloading task.

14. The method according to claim 13, characterized in that The standard rules for unloading multi-layer stacked goods include: determining a target space in the multi-layer stacked goods at the unloading point according to the position information of the marked pallet, and selecting a pallet on the left, right or below the target space.

15. The method according to claim 14, characterized in that The standard rules for unloading multi-layer stacked goods also include: the multi-layer stacked goods at the unloading point are placed in order from bottom to top and from left to right, and the target space is selected according to the placement order of the multi-layer stacked goods at the unloading point.

16. The method according to claim 14, characterized in that The image information of the target cargo pile includes a depth image; The point cloud information of the target pallet is extracted from the point cloud information of the depth image.

17. The method according to claim 16, characterized in that The target pallet information includes the position and angle of the target pallet; The position of the target pallet is determined according to the point cloud center of the target pallet, and the angle of the target pallet is calculated according to the geometric relationship of the point cloud center of the target pallet; The point cloud center of the target pallet is obtained by performing plane segmentation and contour extraction on the point cloud information of the target pallet.

18. The method according to claim 17, characterized in that When the unloading task is an unloading task, the target pallet information also includes goods on the target pallet, and the goods on the target pallet are extracted from the point cloud information of the depth map through filtering and clustering algorithms.

19. The method according to claim 17, characterized in that When the picking and unloading task is a picking task, the picking and unloading execution information corresponds to the position and angle of the target pallet in the multi-layer stacked goods at the picking point.

20. The method according to claim 17, characterized in that When the cargo picking and unloading task is a cargo unloading task, the cargo picking and unloading execution information corresponds to the position and angle at which the cargo pallet needs to be placed; The desired placement position and angle of the cargo pallet are determined by mapping the point cloud center and angle of the target pallet to a target empty space in the multi-layer stacked cargo at the unloading point.

21. An electronic device, characterized in that: include: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and when the at least one processor executes the computer program, the method for loading and unloading cargo by an automatic guided vehicle for a forklift as described in any one of claims 1-20 is implemented.

22. A non-volatile readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method for loading and unloading cargo by an automatic guided vehicle of a forklift as described in any one of claims 1-20 is implemented.

23. A forklift automatic guided vehicle, characterized in that: An electronic device comprising the electronic device described in claim 21.

24. The forklift automated guided vehicle according to claim 23, characterized in that: The forklift automated guided vehicle further comprises an image acquisition device, which is connected to the electronic device and is used to acquire image information of the target cargo pile.

Images