Forklift control method and device, vehicle machine, forklift and storage medium

Abstract

The invention discloses a forklift control method, a forklift control device, a vehicle machine, a forklift and a computer readable storage medium. The control method comprises the steps that an initial area where goods are located and a target area where the goods need to be moved to are obtained; the forklift is controlled to run to the initial area; using the camera to shoot a first image; the first image is input into a preset target detection model to be processed, and the position of a pallet where the goods are placed is determined; and according to the position of the pallet and the target area, the forklift is controlled to move the pallet to the target area. The position of the pallet can be accurately detected through the preset target detection model, and the cargo carrying efficiency is greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field of forklift control, and more specifically, to a forklift control method, a forklift control device, a vehicle computer, a forklift, and a computer-readable storage medium. Background Art

[0002] In the field of warehousing and logistics, unmanned forklifts have higher loading and unloading efficiency than traditional forklifts that rely on manual operation. Unmanned forklifts use their own sensors to navigate routes and automatically avoid obstacles to transport goods. However, goods are usually stacked on pallets, and it is difficult for unmanned forklifts to accurately determine the location of the pallet with the goods to be transported, which reduces the accuracy of unmanned forklifts in transporting goods and is not conducive to the widespread application of unmanned forklifts. Summary of the invention

[0003] Embodiments of the present application provide a forklift control method, a forklift control device, a vehicle computer, a forklift, and a computer-readable storage medium.

[0004] The control method of a forklift in an embodiment of the present application includes: obtaining an initial area where goods are located and a target area to which the goods need to be moved; controlling the forklift to travel to the initial area; using the camera to capture a first image; inputting the first image into a preset target detection model for processing to determine the position of a pallet on which the goods are placed; and controlling the forklift to move the pallet to the target area according to the position of the pallet and the target area.

[0005] The control device of a forklift in an embodiment of the present application comprises an acquisition module, a processing module and a camera. The acquisition module is used to acquire the initial area where the goods are located and the target area where the goods need to be moved. The processing module is used to control the forklift to travel to the initial area. The camera is used to capture a first image. The processing module is also used to: input the first image into a preset target detection model for processing to determine the position of the pallet on which the goods are placed; and control the forklift to move the pallet to the target area according to the position of the pallet and the target area.

[0006] An embodiment of the present application also provides a vehicle computer, which includes a memory and a processor, wherein the memory is configured to store a computer program, and when the processor executes the computer program, it implements the following control method: obtaining an initial area where the goods are located and a target area where the goods need to be moved; controlling the forklift to travel to the initial area; using the camera to capture a first image; inputting the first image into a preset target detection model for processing to determine the position of a pallet on which the goods are placed; and controlling the forklift to move the pallet to the target area according to the position of the pallet and the target area.

[0007] The embodiment of the present application also provides a forklift, which includes a forklift control device and a vehicle computer, and the forklift control device includes an acquisition module, a processing module and a camera. The acquisition module is used to obtain the initial area where the goods are located and the target area where the goods need to be moved. The processing module is used to control the forklift to travel to the initial area. The camera is used to capture a first image. The processing module is also used to: input the first image into a preset target detection model for processing to determine the position of the pallet on which the goods are placed; and according to the position of the pallet and the target area, control the forklift to move the pallet to the target area.

[0008] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the following control method when executed by a processor: obtaining an initial area where the goods are located and a target area to which the goods need to be moved; controlling the forklift to travel to the initial area; using the camera to capture a first image; inputting the first image into a preset target detection model for processing to determine the position of a pallet on which the goods are placed; and controlling the forklift to move the pallet to the target area based on the position of the pallet and the target area.

[0009] In the forklift control method, forklift control device, vehicle computer, forklift and computer-readable storage medium provided by the present application, the forklift is controlled to travel to the initial area according to the acquired initial area, and the first image captured by the camera provided on the forklift is input into the preset target detection model for processing to determine the position of the pallet, thereby ensuring that the forklift can accurately fork the pallet and control the forklift to move the pallet to the target area. The present application achieves the purpose of automatically moving the goods to the target area by the forklift, and can accurately detect the position of the pallet through the preset target detection model, greatly improving the efficiency of cargo handling.

[0010] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0012] Figure 1 is a flow chart of a control method for a forklift in some embodiments of the present application;

[0013] Figure 2 is a schematic structural diagram of a forklift control device in some embodiments of the present application;

[0014] Figure 3 is a schematic structural diagram of a forklift control device according to other embodiments of the present application;

[0015] Figure 4 is a schematic diagram of the positional relationship between a pallet and goods in some embodiments of the present application;

[0016] Figure 5 is a schematic diagram of a forklift according to some embodiments of the present application;

[0017] Figure 6 It is a schematic diagram of the structure of a preset target detection model in some implementation modes of the present application;

[0018] Figure 7 It is a flowchart of inputting a first image into a preset target detection model for processing in a forklift control method of some embodiments of the present application to determine the position of a pallet on which goods are placed;

[0019] Figure 8 It is a flow chart of inputting a first image into a backbone network to obtain a first feature sampling graph, a second feature sampling graph, and a third feature sampling graph in a forklift control method of some embodiments of the present application;

[0020] Fig. 9 It is a flow chart of inputting a first feature sampling graph, a second feature sampling graph, and a third feature sampling graph into a neck network to obtain a first fused feature graph, a second fused feature graph, and a third fused feature graph in a control method of a forklift in some embodiments of the present application;

[0021] Fig.10 It is a schematic diagram of the structure of the preset improvement module of some implementation methods of the present application;

[0022] Fig.11 It is a schematic flow chart of inputting the second transition map into an upsampling module and performing upsampling processing, and then inputting the preset improvement module to obtain a first fusion feature map in a forklift control method of some embodiments of the present application;

[0023] Fig.12 It is a schematic flow chart of inputting the third transition map into the upsampling module and performing upsampling processing, and then inputting the preset improvement module to obtain the second fusion feature map in the forklift control method of some embodiments of the present application;

[0024] Fig.13 A schematic diagram of a process of inputting a fourth transition graph into a feature sampling module and inputting the fourth transition graph into a preset improvement module after feature sampling processing in a forklift control method of some embodiments of the present application to obtain a third fusion feature graph;

[0025] Fig.14It is a schematic flow chart of inputting a first fused feature map, a second fused feature map, and a third fused feature map into a head network to determine the position of a pallet in a forklift control method of some embodiments of the present application;

[0026] Fig.15 It is a schematic diagram of the structure of a pallet in some embodiments of the present application;

[0027] Fig.16 is a flowchart of a forklift control method according to other embodiments of the present application;

[0028] Fig.17 is a schematic structural diagram of a forklift according to certain embodiments of the present application;

[0029] Fig.18 It is a schematic diagram of the connection status of a computer-readable storage medium and a processor in certain embodiments of the present application.

[0030] Description of main component symbols:

[0031] Forklift 100;

[0032] A control device 10 for a forklift;

[0033] Acquisition module 11; processing module 12; camera 13; Sky Eye camera 14; cargo 15; pallet 16; center point 161;

[0034] Processor 20;

[0035] Computer readable storage medium 200; computer program 202;

[0036] Car computer 30. DETAILED DESCRIPTION

[0037] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0038] In the field of warehousing and logistics, unmanned forklifts have higher loading and unloading efficiency than traditional forklifts that rely on manual operation. Unmanned forklifts use the sensors they carry to navigate routes and automatically avoid obstacles, thereby achieving the handling of goods. However, goods are usually stacked on pallets, and it is difficult for unmanned forklifts to accurately determine the position of the pallet on which the goods to be handled are placed. This reduces the accuracy of unmanned forklifts in handling goods and is not conducive to the widespread application of unmanned forklifts. How to solve the problem of low accuracy in handling goods by unmanned forklifts due to the difficulty of unmanned forklifts in accurately identifying the position of pallets has become a difficult problem that technicians in this field urgently need to solve. In order to solve this problem, the present application provides a forklift control method (such as Figure 1 and Fig.16 As shown), forklift control device (such as Figure 2 , Figure 3 and Fig.17 As shown), vehicle machine 30 (as shown Fig.17 As shown), forklift 100 (as Figure 5 and Fig.17 ) and computer readable storage medium 200 (as shown Fig.18 shown).

[0039] See also Figures 1 to 5 In the control method of a forklift according to the embodiment of the present application, a camera 11 is provided on the forklift 100, and the control method includes:

[0040] 02: Get the initial area where the goods 15 is located and the target area where the goods 15 needs to be moved;

[0041] 03: Control the forklift 100 to move to the initial area;

[0042] 04: Use the camera 11 to capture a first image;

[0043] 06: Input the first image into a preset target detection model for processing to determine the position of the pallet 16 on which the goods 15 are placed; and

[0044] 08: According to the position of the pallet 16 and the target area, control the forklift 100 to move the pallet 16 to the target area.

[0045] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100. The control device 10 of the forklift of the embodiment of the present application includes an acquisition module 11, a processing module 12 and a camera 13. Among them, the acquisition module 11 is used to obtain the initial area where the goods 15 are located and the target area to which the goods 15 need to be moved. The processing module 12 is used to control the forklift to travel to the initial area. The camera 13 is used to capture a first image. The processing module 12 is also used to: input the first image into a preset target detection model for processing, determine the position of the pallet 16 on which the goods 15 are placed; and according to the position of the pallet 16 and the target area, control the forklift to move the pallet 16 to the target area.

[0046] The control device 10 of the forklift 100 is one of the core control components installed inside the forklift 100, and is responsible for managing and coordinating the operation of various parts of the forklift 100. The control device 10 of the forklift 100 controls the working state of various components such as the engine, gearbox, brake system, steering system, etc. of the forklift 100 by collecting, processing and analyzing data from various sensors, such as data from sensors such as cameras, ranging radars and infrared ranging devices, so as to ensure the safe and efficient operation of the forklift 100. The control device 10 of the forklift 100 controls the overall performance of the forklift 100 by adjusting the parameters inside the forklift 100. In the present application, the control device 10 of the forklift 100 is used to control the forklift 100 to travel to the initial area where the goods 15 obtained by the acquisition module 11 are located, and the first image captured by the camera 13 is subjected to target detection through a preset target detection model to determine the position of the pallet 16 on which the goods 15 are placed, and the forklift 100 is controlled by the processing module 12 to move the pallet 16 to the target area to which the goods 15 obtained by the acquisition module 11 need to be moved.

[0047] More specifically, the control device 10 of the forklift 100 includes an acquisition module 11, a processing module 12 and a camera 13. The acquisition module 11 is used to execute 02, the processing module 12 is used to execute 03, 06 and 08, the camera 13 is used to execute 04, and the processing module is used to execute 04, 05, 07 and 08. In the present application, the acquisition module 11 is a module in the control device 10 of the forklift 100 for acquiring external information. The camera 13 is provided on the control device 10 of the forklift 100 and is configured to capture a first image around the control device 10 of the forklift 100. The processing module 12 is a core module in the control device 10 of the forklift 100 for processing various types of external information and controlling the operation mode of the forklift 100. The processing module 12 is used to process the image captured by the camera 13 through a preset target detection model to determine the position of the pallet 16 on which the goods 15 are placed in the image captured by the camera 13, so as to control the forklift 100 to accurately fork the pallet 16 according to the position of the pallet 16 to move to the target area.

[0048] More specifically, the control method of the forklift 100 of the present application is mainly used in the scenario where the forklift 100 automatically moves the goods 15 from one area to another area. In method 02, the initial area where the goods 15 are located is the current storage area of ​​the goods 15. The initial area can be a warehouse or an area in the warehouse. The acquisition module 11 can obtain the initial area where the goods 15 are located and the target area where the goods 15 need to be moved to, so as to determine the pickup point and destination of the forklift 100 during this transportation process. The target area where the goods 15 need to be moved can also be a warehouse or an area in the warehouse. Please refer to Figure 3 In the present application, whether it is the initial area where the goods 15 are located or the target area where the goods 15 need to be moved to, a Sky Eye system is provided (in the field of warehousing and logistics, the Sky Eye system refers to the monitoring system in the warehouse), and the Sky Eye camera 14 of the Sky Eye system can process the environmental image to obtain the initial area where the goods 15 are located and the target area where the goods 15 need to be moved to, and send the initial area where the goods 15 are located and the target area where the goods 15 need to be moved to the acquisition module 11 of the forklift 100. In methods 03, 04 and 05, the preset target detection model has no special requirements for the input image. Therefore, the camera 13 can be any type of camera, and the single camera function can be used to capture images to reduce the overall energy consumption of the control device 10 of the forklift 100. Please refer to Figure 4 as well as Figure 5 , the goods 15 are stacked on the pallet 16. After obtaining the position of the pallet 16, the processing module 12 can control the forklift 100 to fork the pallet 16 and move the pallet 16 to the target area, thereby completing the automatic transportation of the goods 15. Compared with the traditional forklift operated by humans, the control method of the forklift 100 in the present application can avoid safety accidents such as collision, overturning, and falling of goods caused by human operation errors during the transportation of the goods 15, and can also greatly improve the efficiency of cargo transportation. In addition, the present application processes the first image captured by the camera 13 through a preset target detection model to accurately determine the position of the pallet 16, thereby avoiding transportation errors caused by the forklift 100.

[0049] In certain embodiments, see Figure 3 , Figure 4 , Figure 6 and Figure 7 ,The preset target detection model includes a backbone network, a neck network and a head network. ,06: Input the first image into the preset target detection model for processing, and determine the position of the pallet on which the goods are placed, including:

[0050] 061: Input the first image into the backbone network to obtain a first feature sampling map P1, a second feature sampling map P2, and a third feature sampling map P3;

[0051] 063: Input the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 into the neck network to obtain the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3; and

[0052] 065: Input the first fused feature map Q2, the second fused feature map Q2 and the third fused feature map Q3 into the head network to determine the position of the stack 16.

[0053] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the first image into the backbone network to obtain the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3; input the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 into the neck network to obtain the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3; and input the first fused feature map Q2, the second fused feature map Q2 and the third fused feature map Q3 into the head network to determine the position of the pallet.

[0054] Specifically, the preset target detection model includes a backbone network, a neck network and a head network. In method 061, the input first image is processed by the backbone network to obtain a first feature sampling map P1, a second feature sampling map P2 and a third feature sampling map P3. The first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 are processed by the neck network to obtain a first fused feature map Q1, a second fused feature map Q2 and a third fused feature map Q3. The first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3 are processed by the head network to determine the position of the pallet 16. The specific processing process of the input image by the backbone network, the neck network and the head network will be described in detail below.

[0055] In certain embodiments, see Figure 3 , Figure 6 and Figure 8 , the backbone network includes a pooling module, multiple feature sampling modules and multiple convolution modules, 061: input the first image into the backbone network to obtain a first feature sampling map P1, a second feature sampling map P2 and a third feature sampling map P3, including:

[0056] 0611: Input the first image into the convolution module, the convolution module, the feature sampling module, the convolution module and the feature sampling module arranged in sequence to obtain a first feature sampling map P1;

[0057] 0613: Input the first feature sampling map P1 into the sequentially arranged convolution module and feature sampling module to obtain a second feature sampling map P2; and

[0058] 0615: Input the second feature sampling map P2 into the sequentially arranged convolution module, feature sampling module and pooling module to obtain a third feature sampling map P3.

[0059] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the first image into the convolution module, the convolution module, the feature sampling module, the convolution module and the feature sampling module arranged in sequence to obtain the first feature sampling map P1; input the first feature sampling map P1 into the convolution module and the feature sampling module arranged in sequence to obtain the second feature sampling map P2; and input the second feature sampling map P2 into the convolution module, the feature sampling module and the pooling module arranged in sequence to obtain the third feature sampling map P3.

[0060] Specifically, in the present application, the pooling module can use the SPPF module, which can effectively avoid the problem of image distortion caused by partial cropping, scaling and other operations on the image, and can also increase the speed of generating candidate frames and save computing costs. The feature sampling module can use the C2f module. Image feature sampling is a process of reducing image resolution, which is usually used to adapt to the size of the display area or generate thumbnails. In the feature sampling process, the pixels of the original image are reduced in a certain proportion to obtain an image with a smaller size and lower resolution. Feature sampling processing of the image can improve the calculation speed of the target detection model, and can also reduce the noise in the image, especially in high-frequency noise, so as to obtain a clearer image. More specifically, in the process of reducing image pixels, the pixel value of a certain point in the image will be averaged with the pixel value of its neighboring points, and the high-frequency noise in the image will gradually become smoother, while the low-frequency information in the image will be retained. Since high-frequency noise is usually redundant information in the image, and low-frequency information is usually objects or large-area backgrounds in the image, each feature sampling process can filter out redundant information in the image and improve the accuracy of target recognition. In method 0613 and method 0615, the first feature sampling map P1 is further processed by convolution and feature sampling to obtain a second feature sampling map P2 of smaller size, and the second feature sampling map P2 is further processed by convolution, feature sampling and pooling to obtain a third feature sampling map P3 of smaller size. Therefore, the sizes of the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 are different. After being processed by the neck network, the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3 of different sizes can also be obtained. This process will be described in detail below.

[0061] In certain embodiments, see Figure 3 , Figure 4 , Figure 6and Fig. 9 The neck network includes multiple feature combination modules, multiple feature sampling modules, multiple convolution modules, multiple up-sampling modules and multiple preset improvement modules. 063: Input the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 into the neck network to obtain the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3, including:

[0062] 0631: input the third feature sampling map P3 into the upsampling module and perform upsampling processing, then input it into the feature combination module, and merge and add it with the second feature sampling map P2 to obtain a first transition map;

[0063] 0632: input the first transition map into the sequentially arranged feature sampling module and up-sampling module, and after feature sampling and up-sampling processing, input the first transition map into the feature combination module, and merge and add the first feature sampling map P1 to obtain the second transition map;

[0064] 0633: After the second transition image is input into the upsampling module and subjected to upsampling processing, it is input into the preset improvement module to obtain the first fusion feature image Q1;

[0065] 0634: input the second transition map into the sequentially arranged feature sampling module and the convolution module, perform feature sampling and convolution processing, input the second transition map into the feature combination module, and merge and add the first transition map processed by the feature sampling module to obtain a third transition map;

[0066] 0635: After the third transition map is input into the upsampling module and subjected to upsampling processing, it is input into the preset improvement module to obtain the second fusion feature map Q2;

[0067] 0636: input the third transition map processed by the feature sampling module into the convolution module for convolution processing, then input it into the feature combination module, and merge and add it with the third feature sampling map P3 to obtain a fourth transition map; and

[0068] 0637: The fourth transition map is input into the feature sampling module and after feature sampling processing, it is input into the preset improvement module to obtain the third fusion feature map Q3.

[0069] The control method of the forklift described above can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the third feature sampling map P3 into the upsampling module and perform upsampling processing, then input the feature combination module, and merge and add it with the second feature sampling map P2 to obtain a first transition map; input the first transition map into the feature sampling module and the upsampling module arranged in sequence, and after feature sampling processing and upsampling processing, input the feature combination module, and merge and add it with the first feature sampling map P1 to obtain a second transition map; input the second transition map into the upsampling module and perform upsampling processing, then input the preset improvement module to obtain a first fused feature map Q1; input the first transition map into the sequenced feature sampling module and the upsampling module ... then input the feature combination module, and merge and add it with the first feature sampling map P1 to obtain a second transition map. The arranged feature sampling module and convolution module are input into the feature combination module after feature sampling and convolution processing, and are fused and added with the first transition map processed by the feature sampling module to obtain a third transition map; the third transition map is input into the upsampling module after upsampling processing, and is input into the preset improvement module to obtain a second fused feature map Q2; the third transition map processed by the feature sampling module is input into the convolution module after convolution processing, and is input into the feature combination module, and is fused and added with the third feature sampling map to obtain a fourth transition map; and the fourth transition map is input into the feature sampling module after feature sampling processing, and is input into the preset improvement module to obtain a third fused feature map Q3.

[0070] Specifically, the upsampling module is used to upsample the input image to improve the resolution of the image, thereby restoring or generating more detailed image details, and can also restore the detailed information of the image that may be lost during the feature sampling process, so that small objects or detailed features in the image are clearer. The feature combination module is used to add the pixel values ​​of corresponding positions of two pictures of the same size and resolution to obtain a new picture. The fusion addition process can increase the original features of the picture. The upsampling module can use the Unsample module, and the feature combination module can use the Concat module. The preset improvement module can use the Channel Spatial Self-attention (CSS) module to improve the recognition effect of the preset target detection model on the stack 16.

[0071] More specifically, the processing module 12 inputs the third feature sampling map P3 into the upsampling module and performs upsampling processing, and then inputs it into the feature combination module, and merges and adds it with the second feature sampling map P2 to obtain a first transition map. The first transition map merges the image information of the second feature sampling map P2 and the third feature sampling map P3. The processing module 12 inputs the first transition map into the feature sampling module and the upsampling module arranged in sequence, and after feature sampling processing and upsampling processing, inputs it into the feature combination module, and merges and adds it with the first feature sampling map P1 to obtain a second transition map. The second transition map merges the image information of the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3. The processing module 12 inputs the second transition map into the upsampling module and performs upsampling processing, and then inputs it into the preset improvement module, thereby obtaining a first fused feature map Q1. The processing module 12 inputs the first transition map into the feature sampling module and the convolution module arranged in sequence, and performs upsampling processing and convolution processing, and then inputs it into the feature combination module, and merges and adds it with the first transition map processed by the feature sampling module to obtain a third transition map. The third transition map combines the image information in the first transition map and the second transition map. The processing module 12 inputs the third transition map into the upsampling module and performs upsampling processing, and then inputs it into the preset improvement module to obtain the second fused feature map Q2. The processing module 12 inputs the third transition map processed by the feature sampling module into the convolution module and performs convolution processing, and then inputs it into the feature combination module, and merges and adds it with the third feature sampling map P3 to obtain the fourth transition map. The fourth transition map combines the image information of the third transition map and the third feature sampling map P3. The processing module 12 then inputs the fourth transition map into the feature sampling module and after feature sampling processing, it inputs it into the preset improvement module to obtain the third fused feature map Q3.

[0072] See also Figure 3 , Figure 4 , Figure 6 , Fig.10 and Fig.11 In some embodiments, the preset improvement module includes multiple dimension change modules, spatial self-attention modules and channel self-attention modules. 0633: After the second transition image is input into the upsampling module and up-sampled, it is input into the preset improvement module to obtain the first fusion feature image Q1, including:

[0073] 06331: After the second transition image is input into the upsampling module and subjected to upsampling processing, the second transition image is input into the dimension changing module for processing to obtain a fifth transition image;

[0074] 06333: Input the fifth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain the sixth transition map;

[0075] 06335: inputting the fifth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a seventh transition map; and

[0076] 06337: After the sixth transition map and the seventh transition map are fused and added, they are fused and added with the second transition map to obtain the first fused feature map Q1.

[0077] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the second transition map into the upsampling module and perform upsampling processing, and then input it into the dimension change module for processing to obtain a fifth transition map; input the fifth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain a sixth transition map; input the fifth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a seventh transition map; and after the sixth transition map and the seventh transition map are fused and added, they are fused and added with the second transition map to obtain a first fused feature map Q1.

[0078] Specifically, the dimension change module can change the shape of the input data without changing the content of the input data, so that the data can flow between the modules of the model in a suitable shape during the process of being processed by the model. Fig.10 , let the input image be X, the dimension change module can use the Reshape module, after the second transition image is up-sampled, it is input into the dimension change module for dimension change processing to obtain the fifth transition image, that is:

[0079] Q, K, V = Reshape(X);

[0080] Wherein, Q, K, V all represent the vector graph of the fifth transition graph, and Reshape (X) is expressed as a dimension change operation. In the present application, the Reshape module changes the dimension of the input image from (B, C, H, W) to (B, C, H*W), where B represents the number of images, C represents the number of channels, H represents the image height, and W represents the image width. The processing module 12 inputs the fifth transition graph into the sequentially arranged spatial self-attention module and the dimension change module and the sequentially arranged channel self-attention module and the dimension change module, respectively, to obtain the sixth transition graph and the seventh transition graph, and then fuses and adds the sixth transition graph and the seventh transition graph, and then fuses and adds them with the second transition graph to obtain the first fused feature graph Q1. The process of obtaining the first fused feature graph Q1 is as follows:

[0081]

[0082] Among them, Sum means addition, w1 and w2 are learnable parameters, X is the input image, and Output is the output first fused feature map Q1. Both the spatial self-attention module and the channel self-attention module include a dimension exchange module. The dimension exchange module can use the Permute module. For example, the dimension exchange module can change the dimension of the image from (B, C, H*W) to (B, H*W, C), thereby ensuring the normal operation of the spatial self-attention module and the channel self-attention module. The process in which the processing module 12 inputs the fifth transition map into the spatial self-attention module three times (divided into Q, K, and V) is as follows:

[0083]

[0084] Among them, BMM represents matrix multiplication; Softmax represents a probability distribution function used to assign weights; Permute represents dimension exchange. The process of the processing module 12 inputting the fifth transition map into the channel self-attention module three times (divided into Q, K, and V) is as follows:

[0085]

[0086] Among them, BMM represents matrix multiplication; Softmax represents the probability distribution function, which is used to assign weights; Permute represents dimension exchange.

[0087] See also Figure 3 , Figure 4 , Figure 6 , Fig.10 and Fig.12 In some embodiments, the preset improvement module includes multiple dimension change modules, spatial self-attention modules and channel self-attention modules. 0635: After the third transition image is input into the upsampling module and up-sampled, it is input into the preset improvement module to obtain the second fused feature image Q2, including:

[0088] 06351: After the third transition image is input into the upsampling module and subjected to upsampling processing, the third transition image is input into the dimension changing module for processing to obtain an eighth transition image;

[0089] 06353: Input the eighth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain the ninth transition map;

[0090] 06355: inputting the eighth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a tenth transition map; and

[0091] 06357: After the ninth transition map and the tenth transition map are fused and added, they are fused and added with the third transition map to obtain a second fused feature map Q2.

[0092] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the third transition map into the upsampling module and perform upsampling processing, and then input it into the dimension change module for processing to obtain the eighth transition map; input the eighth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain the ninth transition map; input the eighth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain the tenth transition map; and after the ninth transition map and the tenth transition map are fused and added, they are fused and added with the third transition map to obtain the second fused feature map Q2.

[0093] Specifically, the principle of this embodiment is similar to the principle of processing the third transition map to obtain the first fusion feature map Q1, see Fig.10 , let the input image be X, the dimension change module can use the Reshape module, after the third transition image is up-sampled, it is input into the dimension change module for dimension change processing to obtain the eighth transition image, that is:

[0094] Q, K, V = Reshape(X);

[0095] Wherein, Q, K, V all represent the vector graph of the eighth transition graph, and Reshape (X) is expressed as a dimension change operation. In the present application, the Reshape module changes the dimension of the input image from (B, C, H, W) to (B, C, H*W), where B represents the number of images, C represents the number of channels, H represents the image height, and W represents the image width. The processing module 12 inputs the eighth transition graph into the sequentially arranged spatial self-attention module and the dimension change module and the sequentially arranged channel self-attention module and the dimension change module, respectively, to obtain the ninth transition graph and the tenth transition graph, and then fuses and adds the ninth transition graph and the tenth transition graph, and then fuses and adds them with the third transition graph to obtain the first fused feature graph Q1. The process of obtaining the first fused feature graph Q1 is as follows:

[0096] Output=Sum(Reshape(Channel Self-attention(Q,K,V))*w1,

[0097] Reshape(Spatial Self-attention(Q,K,V))*w2,X);

[0098] Among them, Sum means addition, w1 and w2 are learnable parameters, X is the input image, and Output is the output first fused feature map Q1. Both the spatial self-attention module and the channel self-attention module include a dimension exchange module. The dimension exchange module can use the Permute module. For example, the dimension exchange module can change the dimension of the image from (B, C, H*W) to (B, H*W, C), thereby ensuring the normal operation of the spatial self-attention module and the channel self-attention module. The process in which the processing module 12 inputs the eighth transition map into the spatial self-attention module three times (divided into Q, K, and V) is as follows:

[0099] Spatial Self-attention(Q,K,V)=BMM(V,

[0100] Softmax(BMM(Permute(Q),K)));

[0101] Among them, BMM represents matrix multiplication; Softmax represents a probability distribution function used to assign weights; Permute represents dimension exchange. The process of the processing module 12 inputting the eighth transition map into the channel self-attention module three times (divided into Q, K, and V) is as follows:

[0102]

[0103] Among them, BMM represents matrix multiplication; Softmax represents the probability distribution function, which is used to assign weights; Permute represents dimension exchange.

[0104] See also Figure 3 , Figure 4 , Figure 6 , Fig.10 and Fig.13 In some embodiments, the preset improvement module includes multiple dimension change modules, spatial self-attention modules and channel self-attention modules. 0637: The fourth transition map is input into the feature sampling module and after feature sampling processing, it is input into the preset improvement module to obtain the third fused feature map Q3, including:

[0105] 06371: After the fourth transition image is input into the upsampling module and subjected to upsampling processing, the fourth transition image is input into the dimension changing module for processing to obtain the eleventh transition image;

[0106] 06373: Input the eleventh transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain the twelfth transition map;

[0107] 06375: inputting the eleventh transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a thirteenth transition map; and

[0108] 06377: After the twelfth transition map and the thirteenth transition map are fused and added, they are fused and added with the fourth transition map to obtain a third fused feature map Q3.

[0109] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: input the fourth transition map into the upsampling module and perform upsampling processing, and then input it into the dimension change module for processing to obtain the eleventh transition map; input the eleventh transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain the twelfth transition map; input the eleventh transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain the thirteenth transition map; and after the twelfth transition map and the thirteenth transition map are fused and added, they are fused and added with the fourth transition map to obtain the third fused feature map Q3.

[0110] Specifically, the principle of this embodiment is similar to the principle of processing the third transition map to obtain the first fusion feature map Q1, see Fig.10 , let the input image be X, the dimension change module can use the Reshape module, after the fourth transition image is up-sampled, it is input into the dimension change module for dimension change processing to obtain the eleventh transition image, that is:

[0111] Q, K, V = Reshape(X);

[0112] Wherein, Q, K, V all represent the vector graph of the eleventh transition graph, and Reshape (X) is expressed as a dimension change operation. In the present application, the Reshape module changes the dimension of the input image from (B, C, H, W) to (B, C, H*W), wherein B represents the number of images, C represents the number of channels, H represents the image height, and W represents the image width. The processing module 12 inputs the eleventh transition graph into the sequentially arranged spatial self-attention module and the dimension change module and the sequentially arranged channel self-attention module and the dimension change module, respectively, to obtain the twelfth transition graph and the thirteenth transition graph, and then fuses and adds the twelfth transition graph and the thirteenth transition graph, and then fuses and adds the twelfth transition graph and the thirteenth transition graph with the fourth transition graph to obtain the first fused feature graph Q1. The process of obtaining the first fused feature graph Q1 is as follows:

[0113]

[0114] Among them, Sum means addition, w1 and w2 are learnable parameters, X is the input image, and Output is the first fused feature map Q1 of the output. Both the spatial self-attention module and the channel self-attention module include a dimension exchange module. The dimension exchange module can use the Permute module. For example, the dimension exchange module can change the dimension of the image from (B, C, H*W) to (B, H*W, C), thereby ensuring the normal operation of the spatial self-attention module and the channel self-attention module. The process in which the processing module 12 inputs the eleventh transition map into the spatial self-attention module three times (divided into Q, K, and V) is as follows:

[0115]

[0116] Among them, BMM represents matrix multiplication; Softmax represents a probability distribution function used to assign weights; Permute represents dimension exchange. The process of the processing module 12 inputting the eleventh transition map into the channel self-attention module three times (divided into Q, K, and V) is as follows:

[0117]

[0118] Among them, BMM represents matrix multiplication; Softmax represents the probability distribution function, which is used to assign weights; Permute represents dimension exchange. More specifically, in order to prove the effectiveness of the improvement of the original target detection model by this application, the experimental results are shown in Table 1:

[0119] Table 1 Pallet detection test results

[0120] Model <![CDATA[Parameter quantity (×10 6 )]]> mAP(0.5) / % YOLOv8n (original target detection model) 3.01 96.7 CSS-YOLOv8n (preset target detection model) 3.01 97.8

[0121] It can be seen from Table 1 above that when the network parameters are the same, the detection accuracy of the pallet position is significantly improved.

[0122] See also Figure 3 , Figure 4 , Figure 6 and Fig.14 In some embodiments, 065: inputting the first fused feature map Q1, the second fused feature map Q2, and the third fused feature map Q3 into the head network to determine the position of the stack 16, including:

[0123] 0651: Obtain the pixel value of each pixel in the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3;

[0124] 0653: Calculate the probability value of each pixel point in the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3 being the position of the stack 16 based on the pixel value of each pixel point in the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3, and use it as the weight value; and

[0125] 0655: Determine the position of the stack 16 according to the weight value of each pixel point in the first fused feature map Q1, the second fused feature map Q2 and the third fused feature map Q3.

[0126] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: obtain the pixel value of each pixel point in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3; calculate the probability value of each pixel point in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3 being the position of the stack 16 based on the pixel value of each pixel point in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3, and use it as a weight value; and determine the position of the stack 16 according to the weight value of each pixel point in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3.

[0127] Specifically, the processing module 12 calculates the probability value of each pixel in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3 being the column 43 according to the pixel value of each pixel in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3, and uses the probability value as the weight value. For example, the processing module 12 stores a preset probability value calculation formula and a preset probability value range to calculate the probability value according to the pixel value. The processing module 12 determines the image area whose probability value in the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3 is within the preset probability value range according to the preset probability value range, and generates a target frame in the image area whose probability value is within the preset probability value range. The target library is the position of the stack 16. If the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3 do not generate a target frame, the processing module 12 determines that there is no stack 16 in the image.

[0128] In certain embodiments, see Figure 3 , Figure 4 , Fig.14 , Fig.15 and Fig.16 , the camera 13 is a depth camera 13, and the control method further includes:

[0129] 05: Use the depth camera 13 to capture a second image.

[0130] The forklift control method described above may be applied to the control device 10 of the forklift 100 , and the camera 13 is used to capture the second image.

[0131] See also Figure 3 , Figure 4 , Figure 5 , Fig.14 , Fig.15 and Fig.16 In some embodiments, 08: controlling the forklift 100 to move the pallet 16 to the target area according to the position of the pallet 16 and the target area includes:

[0132] 081: Determine the position of the center point 161 of the pallet 16 according to the second image and the position of the pallet 16;

[0133] 083: According to the position of the center point 161 of the pallet 16 and the target area, the forklift 100 is controlled to move the pallet 16 to the target area.

[0134] The forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: determine the position of the center point 161 of the pallet 16 according to the second image and the position of the pallet 16; and control the forklift 100 to move the pallet 16 to the target area according to the position of the center point 161 of the pallet 16 and the target area.

[0135] It is understandable that if the position of the center point 161 of the pallet 16 is to be accurately determined, image processing needs to be performed in combination with the depth map captured by the depth camera. After obtaining the position of the center point 161 of the pallet 16, the processing module 12 can control the position of the camera 13 of the forklift 100 to align with the position of the center point 161 to ensure that the forklift 100 can fork the pallet 16 more accurately.

[0136] See also Figure 3 , Figure 4 , Figure 5 and Fig.16 In some embodiments, the control method further comprises:

[0137] 011: Get the preset cargo 15 handling task;

[0138] 013: According to the preset cargo 15 handling task, determine the preset number of handling times;

[0139] 09: When the current number of transports is less than the preset number of transports, the forklift is controlled to repeatedly execute the steps of obtaining the initial area and the target area and controlling the forklift to move the pallet 16 to the target area according to the position of the pallet 16 and the target area, until the current number of transports is equal to the preset number of transports.

[0140] The above-mentioned forklift control method can be applied to the control device 10 of the forklift 100, and the processing module 12 is used to: obtain a preset cargo 15 handling task; determine a preset number of handlings based on the preset cargo 15 handling task; when the current number of handlings is less than the preset number of handlings, control the forklift to repeatedly execute the steps of obtaining the initial area and the target area until the forklift controls the forklift to move the pallet 16 to the target area according to the position of the pallet 16 and the target area, until the current number of handlings is equal to the preset number of handlings.

[0141] It is understandable that the preset cargo 15 handling task includes the number of cargoes 15 to be handled, that is, the preset number of handling times. When the current number of handling times is less than the preset number of handling times, the processing module 12 needs to continue to control the forklift 100 to handle the cargoes 15 until all the cargoes 15 to be handled are handled, that is, until the current number of handling times is equal to the preset number of handling times.

[0142] In some embodiments, the Sky Eye camera 14 can further divide the target area to determine sub-target areas.

[0143] In summary, the present application provides a control method for a forklift and a control device for a forklift 100, which controls the forklift 100 to travel to the initial area according to the acquired initial area, and inputs the first image captured by the camera 13 provided on the forklift 100 into a preset target detection model for processing to determine the position of the pallet 16, thereby ensuring that the forklift 100 can accurately fork the pallet 16, and controls the forklift 100 to move the pallet 16 to the target area. The present application achieves the purpose of the forklift 100 automatically moving the goods 15 to the target area, and can accurately detect the position of the pallet 16 through the preset target detection model, greatly improving the efficiency of handling the goods 15.

[0144] In certain embodiments, see Fig.17 The present application also provides a vehicle computer 30, which includes a memory and a processor. The memory is configured to store a computer program. When the processor executes the computer program, it implements the control method in any of the above embodiments.

[0145] For example, when the processor of the vehicle computer 30 executes the computer program stored in the memory, the following control method is implemented:

[0146] 02: Get the initial area where the goods 15 is located and the target area where the goods 15 needs to be moved;

[0147] 03: Control the forklift 100 to move to the initial area;

[0148] 04: Use the camera 11 to capture a first image;

[0149] 06: Input the first image into a preset target detection model for processing to determine the position of the pallet 16 on which the goods 15 are placed; and

[0150] 08: According to the position of the pallet 16 and the target area, control the forklift 100 to move the pallet 16 to the target area.

[0151] For another example, when the processor of the vehicle computer 30 executes the computer program stored in the memory, the following control method is implemented:

[0152] 061: Input the first image into the backbone network to obtain a first feature sampling map P1, a second feature sampling map P2, and a third feature sampling map P3;

[0153] 063: Input the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 into the neck network to obtain the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3; and

[0154] 065: Input the first fused feature map Q2, the second fused feature map Q2 and the third fused feature map Q3 into the head network to determine the position of the stack 16.

[0155] For another example, when the processor of the vehicle computer 30 executes the computer program stored in the memory, it can also implement the control methods in 011, 013, 05, 0611, 0613, 0615, 0631, 0632, 0633, 06331, 06333, 06335, 06337, 0634, 0635, 06351, 06353, 06355, 06357, 0636, 0637, 06371, 06373, 06375, 06377, 0651, 0653, 0655, 081, 083 and 09.

[0156] In certain embodiments, see Fig.17 The present application also provides a forklift 100, including the control device 10 of the forklift 100 in any of the above embodiments or the vehicle computer 30 in any of the above embodiments.

[0157] See also Fig.18 In some embodiments, the present application further provides a computer-readable storage medium 200 on which a computer program 202 is stored, and when the program is executed by a processor, the control method in any of the above embodiments is implemented.

[0158] For example, when the computer program 202 is executed by the processor 20, the following control method is implemented:

[0159] 02: Get the initial area where the goods 15 is located and the target area where the goods 15 needs to be moved;

[0160] 03: Control the forklift 100 to move to the initial area;

[0161] 04: Use the camera 11 to capture a first image;

[0162] 06: Input the first image into a preset target detection model for processing to determine the position of the pallet 16 on which the goods 15 are placed; and

[0163] 08: According to the position of the pallet 16 and the target area, control the forklift 100 to move the pallet 16 to the target area.

[0164] For another example, when the computer program 202 is executed by the processor 20, the following control method is implemented:

[0165] 061: Input the first image into the backbone network to obtain a first feature sampling map P1, a second feature sampling map P2, and a third feature sampling map P3;

[0166] 063: Input the first feature sampling map P1, the second feature sampling map P2 and the third feature sampling map P3 into the neck network to obtain the first fusion feature map Q1, the second fusion feature map Q2 and the third fusion feature map Q3; and

[0167] 065: Input the first fused feature map Q2, the second fused feature map Q2 and the third fused feature map Q3 into the head network to determine the position of the stack 16.

[0168] For another example, when the computer program 202 is executed by the processor 20, it can also implement the control methods in 011, 013, 05, 0611, 0613, 0615, 0631, 0632, 0633, 06331, 06333, 06335, 06337, 0634, 0635, 06351, 06353, 06355, 06357, 0636, 0637, 06371, 06373, 06375, 06377, 0651, 0653, 0655, 081, 083 and 09.

[0169] In the computer-readable storage medium 200 of the present application, the forklift 100 is controlled to travel to the initial area according to the acquired initial area, and the first image captured by the camera 13 provided on the forklift 100 is input into the preset target detection model for processing to determine the position of the pallet 16, thereby ensuring that the forklift 100 can accurately fork the pallet 16, and control the forklift 100 to move the pallet 16 to the target area. The present application achieves the purpose of the forklift 100 automatically moving the goods 15 to the target area, and can accurately detect the position of the pallet 16 through the preset target detection model, greatly improving the efficiency of handling the goods 15.

[0170] In the description of this specification, the descriptions with reference to the terms "certain embodiments", "in an example", "exemplarily", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0171] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0172] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A forklift control method, characterized in that: The forklift (100) is provided with a camera (13), and the control method comprises: Obtaining an initial area where the goods (15) are located and a target area to which the goods (15) need to be moved; Controlling the forklift (100) to travel to the initial area; Using the camera (13) to capture a first image; Inputting the first image into a preset target detection model for processing to determine the position of the pallet (16) on which the goods (15) are placed; and According to the position of the pallet (16) and the target area, the forklift (100) is controlled to move the pallet (16) to the target area.

2. The control method according to claim 1, characterized in that: The preset target detection model includes a backbone network, a neck network and a head network, and the first image is input into the preset target detection model for processing to determine the position of the pallet (16) on which the goods (15) are placed, including: Inputting the first image into the backbone network to obtain a first feature sampling map, a second feature sampling map, and a third feature sampling map; Inputting the first feature sampling graph, the second feature sampling graph, and the third feature sampling graph into the neck network to obtain a first fused feature graph, a second fused feature graph, and a third fused feature graph; and The first fused feature map, the second fused feature map and the third fused feature map are input into the head network to determine the position of the stack (16).

3. The control method according to claim 2, characterized in that: The backbone network includes a pooling module, a plurality of feature sampling modules and a plurality of convolution modules. The first image is input into the backbone network to obtain a first feature sampling map, a second feature sampling map and a third feature sampling map, including: Inputting the first image into the convolution module, the convolution module, the feature sampling module, the convolution module and the feature sampling module arranged in sequence to obtain the first feature sampling map; Inputting the first feature sampling graph into the sequentially arranged convolution module and the feature sampling module to obtain the second feature sampling graph; and The second feature sampling map is input into the convolution module, the feature sampling module and the pooling module arranged in sequence to obtain the third feature sampling map.

4. The control method according to claim 2, characterized in that: The neck network includes multiple feature combination modules, multiple feature sampling modules, multiple convolution modules, multiple up-sampling modules and multiple preset improvement modules. The first feature sampling map, the second feature sampling map and the third feature sampling map are input into the neck network to obtain a first fusion feature map, a second fusion feature map and a third fusion feature map, including: After the third feature sampling graph is input into the up-sampling module and subjected to up-sampling processing, the third feature sampling graph is input into the feature combination module and fused and added with the second feature sampling graph to obtain a first transition graph; The first transition map is input into the feature sampling module and the up-sampling module arranged in sequence, and after feature sampling and up-sampling processing, the first transition map is input into the feature combination module, and is fused and added with the first feature sampling map to obtain a second transition map; After the second transition map is input into the upsampling module and subjected to upsampling processing, the second transition map is input into the preset improvement module to obtain the first fusion feature map; The second transition map is input into the feature sampling module and the convolution module arranged in sequence and subjected to feature sampling and convolution processing, and then input into the feature combination module, and fused and added with the first transition map processed by the feature sampling module to obtain a third transition map; After the third transition map is input into the upsampling module and subjected to upsampling processing, the third transition map is input into the preset improvement module to obtain the second fusion feature map; Input the third transition map processed by the feature sampling module into the convolution module for convolution processing, then input it into the feature combination module, and merge and add it with the third feature sampling map to obtain a fourth transition map; and The fourth transition map is input into the feature sampling module and, after being processed by feature sampling, is input into the preset improvement module to obtain the third fusion feature map.

5. The control method according to claim 4, characterized in that: The preset improvement module includes multiple dimension change modules, a spatial self-attention module and a channel self-attention module. After the second transition map is input into the upsampling module and subjected to upsampling processing, it is input into the preset improvement module to obtain the first fusion feature map, including: After the second transition image is input into the upsampling module and subjected to upsampling processing, the second transition image is input into the dimension changing module for processing to obtain a fifth transition image; Inputting the fifth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain a sixth transition map; Inputting the fifth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a seventh transition map; and After the sixth transition map and the seventh transition map are fused and added, they are fused and added with the second transition map to obtain the first fused feature map.

6. The control method according to claim 5, characterized in that: The spatial self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, the channel self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, and the fifth transition map is input into the spatial self-attention module and the dimension change module arranged in sequence to obtain a sixth transition map, including: Input the fifth transition graph into the spatial self-attention module, process the fifth transition graph according to the dimension exchange module, the matrix multiplication module and the probability distribution processing module, and then input the fifth transition graph into the dimension change module to obtain the sixth transition graph; The step of inputting the fifth transition graph into the sequentially arranged channel self-attention module and the dimension change module to obtain a seventh transition graph comprises: The fifth transition graph is input into the spatial self-attention module, and after being processed by the dimension exchange module, the matrix multiplication module and the probability distribution processing module, the fifth transition graph is input into the dimension change module to obtain the seventh transition graph.

7. The control method according to claim 4, characterized in that: The preset improvement module includes multiple dimension change modules, a spatial self-attention module and a channel self-attention module. After the third transition map is input into the upsampling module and subjected to upsampling processing, it is input into the preset improvement module to obtain the second fusion feature map, including: After the third transition image is input into the up-sampling module and subjected to up-sampling processing, the third transition image is input into the dimension changing module for processing to obtain an eighth transition image; Inputting the eighth transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain a ninth transition map; Inputting the eighth transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a tenth transition map; and The ninth transition map and the tenth transition map are fused and added together, and then fused and added together with the third transition map to obtain the second fused feature map.

8. The control method according to claim 7, characterized in that: The spatial self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, the channel self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, and the eighth transition map is input into the spatial self-attention module and the dimension change module arranged in sequence to obtain a ninth transition map, including: Input the eighth transition graph into the spatial self-attention module, process the eighth transition graph according to the dimension exchange module, the matrix multiplication module and the probability distribution processing module, and then input the eighth transition graph into the dimension change module to obtain the ninth transition graph; The step of inputting the eighth transition graph into the sequentially arranged channel self-attention module and the dimension change module to obtain a tenth transition graph comprises: The eighth transition graph is input into the spatial self-attention module, and after being processed by the dimension exchange module, the matrix multiplication module and the probability distribution processing module, the eighth transition graph is input into the dimension change module to obtain the tenth transition graph.

9. The control method according to claim 4, characterized in that: The preset improvement module includes multiple dimension change modules, a spatial self-attention module and a channel self-attention module. The fourth transition map is input into the feature sampling module and after feature sampling processing, it is input into the preset improvement module to obtain the third fusion feature map, including: After inputting the fourth transition image into the upsampling module and performing upsampling processing, the fourth transition image is input into the dimension changing module for processing to obtain an eleventh transition image; Inputting the eleventh transition map into the sequentially arranged spatial self-attention module and the dimension change module to obtain a twelfth transition map; Inputting the eleventh transition map into the sequentially arranged channel self-attention module and the dimension change module to obtain a thirteenth transition map; and The twelfth transition map and the thirteenth transition map are fused and added together, and then fused and added together with the fourth transition map to obtain the third fused feature map.

10. The control method according to claim 9, characterized in that: The spatial self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, the channel self-attention module includes a dimension exchange module, a matrix multiplication module and a probability distribution processing module, and the eleventh transition map is input into the spatial self-attention module and the dimension change module arranged in sequence to obtain a twelfth transition map, including: Input the eleventh transition graph into the spatial self-attention module, process the eleventh transition graph according to the dimension exchange module, the matrix multiplication module and the probability distribution processing module, and then input the eleventh transition graph into the dimension change module to obtain the twelfth transition graph; The step of inputting the eleventh transition graph into the sequentially arranged channel self-attention module and the dimension change module to obtain a thirteenth transition graph comprises: The eleventh transition graph is input into the spatial self-attention module, and after being processed by the dimension exchange module, the matrix multiplication module and the probability distribution processing module, the eleventh transition graph is input into the dimension change module to obtain the thirteenth transition graph.

11. The control method according to claim 2, characterized in that: The step of inputting the first fused feature map, the second fused feature map and the third fused feature map into the head network to determine the position of the stack (16) comprises: Obtaining pixel values ​​of each pixel point in the first fused feature map, the second fused feature map, and the third fused feature map; Based on the pixel values ​​of each pixel in the first fused feature map, the second fused feature map, and the third fused feature map, calculate the probability value that each pixel in the first fused feature map, the second fused feature map, and the third fused feature map is the position of the stack, and use it as a weight value; and The position of the stack (16) is determined according to the weight value of each pixel point in the first fused feature map, the second fused feature map and the third fused feature map.

12. The control method according to claim 1, characterized in that: The camera (13) is a depth camera, and the control method further comprises: The depth camera is used to capture a second image.

13. The control method according to claim 12, characterized in that: The step of controlling the forklift (100) to move the pallet (16) to the target area according to the position of the pallet (16) and the target area comprises: Determining the position of a center point (161) of the pallet (16) based on the second image and the position of the pallet (16); and According to the position of the center point (161) of the pallet (16) and the target area, the forklift (100) is controlled to move the pallet (16) to the target area.

14. The control method according to claim 1, characterized in that: The control method further comprises: Obtaining the preset cargo (15) handling task; Determining a preset number of transport times according to the preset cargo (15) transport task; and When the current number of transports is less than the preset number of transports, the forklift (100) is controlled to repeatedly execute the steps of acquiring the initial area and the target area and controlling the forklift (100) to move the pallet (16) to the target area according to the position of the pallet (16) and the target area, until the current number of transports is equal to the preset number of transports.

15. A control device for a forklift, characterized in that: The control device (10) comprises an acquisition module (11), a processing module (12) and a camera (13), wherein the acquisition module (11) is used to acquire an initial area where the goods (15) are located and a target area to which the goods (15) need to be moved; The processing module (12) is used to control the forklift (100) to travel to the initial area; The camera (13) is used to capture a first image; The processing module (12) is further used to: input the first image into a preset target detection model for processing, and determine the position of the pallet (16) on which the goods (15) are placed; and According to the position of the pallet (16) and the target area, the forklift (100) is controlled to move the pallet (16) to the target area.

16. A vehicle computer, characterized in that: The vehicle computer (30) comprises a memory and a processor, the memory is configured to store a computer program, and the processor implements the control method according to any one of claims 1 to 14 when executing the computer program.

17. A forklift, characterized in that: It comprises the control device (10) of claim 15 and the vehicle computer (30) of claim 16.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor (20), the control method described in any one of claims 1 to 14 is implemented.

Images