Tray pose detection method and device, equipment, storage medium

Abstract

This invention discloses a pallet pose detection method, apparatus, device, and storage medium. The method includes: extracting regions from a point cloud image of the area where the pallet is located to obtain an initial pallet image, which is the image after filtering out background regions from the point cloud image; performing convolution processing on the initial pallet image according to a pre-determined target convolution kernel to obtain a target feature map, where the target convolution kernel is determined based on a pre-determined surface area of ​​each pier hole and the total surface area of ​​the pallet; slicing the initial pallet image according to the target feature map and the pre-determined total surface area of ​​the pallet to obtain point cloud images of multiple sub-candidate pallet layers; and combining the point cloud images of each sub-candidate pallet layer to determine the target pose of the pallet. This method can improve the accuracy and speed of pallet pose detection.

Description

Technical Field

[0001] This application relates to information processing technology, and to, but is not limited to, a pallet position detection method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, with the rise of intelligent logistics and unmanned warehouses, the technology of automated handling and sorting of goods by unmanned forklifts following programmed instructions has become a research and application hotspot. However, in the complex environment of unstructured warehouses, the position and posture of pallets are subject to considerable uncertainty due to factors such as work processes, equipment precision, and manual operations. This makes it difficult for unmanned forklifts to quickly and accurately identify the position and posture of pallets, thus failing to meet the high-efficiency operational needs of the warehousing and logistics industry.

[0003] Furthermore, existing pallet detection technologies often use 3D point cloud features to describe and detect pallet pose. However, methods using geometric features and feature points are usually affected by the quality of the point cloud, making it difficult to accurately and stably extract the physical features of the pallet. Additionally, due to the large volume of point cloud data, processing is typically time-consuming, making it difficult to achieve rapid detection with limited hardware resources.

[0004] Therefore, there is an urgent need to develop a highly accurate and fast method for detecting the pallet position of unmanned forklifts. Summary of the Invention

[0005] In view of this, the pallet pose detection method, apparatus, device, and storage medium provided in the embodiments of this application can improve the accuracy and speed of pallet pose detection. The pallet pose detection method, apparatus, device, and storage medium provided in the embodiments of this application are implemented as follows:

[0006] The pallet pose detection method provided in this application embodiment includes a pallet comprising multiple pallet feet, with foot holes between adjacent pallet feet. The method includes:

[0007] Region extraction is performed on the point cloud image of the area where the tray is located to obtain an initial tray image, which is the image after filtering out the background area in the point cloud image;

[0008] The initial pallet image is convolved according to a predetermined target convolution kernel to obtain a target feature map. The target convolution kernel is determined based on the surface area of ​​each of the pier holes and the total surface area of ​​the pallet.

[0009] Based on the target feature map and the predetermined total surface area of ​​the tray, the initial tray image is sliced ​​to obtain point cloud images of multiple sub-candidate tray layers;

[0010] The target pose of the tray is determined by combining the point cloud images of each of the sub-candidate tray layers.

[0011] In some embodiments, the initial tray image is convolved according to a predetermined target convolution kernel to obtain a target feature map, including:

[0012] The initial tray image is mapped to obtain a projection image of the initial tray image on the first plane;

[0013] The projected image is convolved using the target convolution kernel to obtain the target feature map.

[0014] In some embodiments, determining the target convolution kernel based on the predetermined surface area of ​​each of the pier holes and the total surface area of ​​the tray includes:

[0015] Determine the size of the target convolution kernel corresponding to the projected image based on the resolution of the projected image;

[0016] The sum of the surface areas of multiple pier holes is determined as the first surface area, and the difference between the total surface area of ​​the pallet and the first surface area is determined as the second surface area.

[0017] Given the size of the target convolutional kernel, the ratio of a first value to the first surface area is determined as the weight of each point within the target convolutional kernel corresponding to the hole region, and the ratio of a second value to the first surface area is determined as the weight of each point within the target convolutional kernel corresponding to other regions in the tray besides the hole region, wherein the first value and the second value are opposites of each other; or

[0018] The ratio of the first value to the first surface area is determined as the weight of each point within the target convolution kernel corresponding to the hole region, and the ratio of the second value to the second surface area is determined as the weight of each point within the target convolution kernel corresponding to other regions in the tray besides the hole region; or

[0019] The weights of each point in the target convolution kernel corresponding to the pier surface area of ​​each foot pier are determined to conform to a two-dimensional Gaussian distribution, and the weights of each point in the target convolution kernel corresponding to each pier hole area are opposites of the weights of each point in the target convolution kernel corresponding to other areas in the tray except for the pier hole area.

[0020] In some embodiments, the initial tray image is sliced ​​based on the target feature map and a predetermined total surface area of ​​the tray to obtain point cloud images of multiple sub-candidate tray layers, including:

[0021] Candidate pallet images are extracted from the initial pallet image based on the target feature map and the pre-determined total surface area of ​​the pallet;

[0022] On the second plane of the candidate tray image, the candidate tray image is sliced ​​to obtain multiple sub-candidate tray layers. The second plane is perpendicular to the first plane.

[0023] In some embodiments, determining the target pose of the pallet by combining each sub-candidate pallet layer includes:

[0024] Clustering is performed on each sub-candidate pallet layer to obtain clustering results, which contain multiple point cloud clusters;

[0025] Based on the clustering results of each sub-candidate pallet layer, the sub-candidate pallet layer whose clustering results meet the preset conditions is determined as the target pallet layer from multiple sub-candidate pallet layers;

[0026] Each target pallet layer is fitted to obtain the pallet plane corresponding to each target pallet layer;

[0027] Based on the cluster center position in each pallet plane, the pallet planes are combined to obtain the target pose of the pallet.

[0028] In some embodiments, the preset conditions include that the inter-cluster distance between adjacent point cloud clusters in the sub-candidate tray layer is less than a first threshold, and the number of clusters in the sub-candidate tray layer is greater than a second threshold.

[0029] In some embodiments, region extraction is performed on the point cloud image of the area where the tray is located to obtain an initial tray image, including:

[0030] Obtain the initial pose of the tray;

[0031] Based on the initial pose of the tray and the preset region of interest threshold, the point cloud image is used to extract regions to obtain the initial tray image.

[0032] The pallet pose detection device provided in this application embodiment includes:

[0033] The extraction module is used to extract regions from the point cloud image of the area where the tray is located, and obtain the initial tray image, which is the image after filtering out the background area from the point cloud image;

[0034] A convolution module is used to perform convolution processing on an initial pallet image according to a predetermined target convolution kernel to obtain a target feature map. The target convolution kernel is determined based on the surface area of ​​each of the pier holes and the total surface area of ​​the pallet.

[0035] The processing module is used to slice the initial tray image based on the target feature map and the predetermined total surface area of ​​the tray to obtain point cloud images of multiple sub-candidate tray layers.

[0036] The determination module is used to synthesize the point cloud images of each sub-candidate tray layer to determine the target pose of the tray.

[0037] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0038] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0039] The pallet pose detection method, apparatus, computer device, and computer-readable storage medium provided in this application extract regions from the point cloud image of the area where the pallet is located to obtain an initial pallet image, which is the image after filtering out background regions from the point cloud image. The initial pallet image is then convolved with a pre-determined target convolution kernel to obtain a target feature map. The target convolution kernel is determined based on the pre-determined surface area of ​​each pier hole and the total surface area of ​​the pallet. Based on the target feature map and the pre-determined total surface area of ​​the pallet, the initial pallet image is sliced ​​to obtain multiple sub-candidate pallet layer point cloud images. The target pose of the pallet is determined by combining the point cloud images of each sub-candidate pallet layer. This method filters out interfering point clouds in the point cloud image, improving the accuracy of pallet pose detection. Furthermore, the slicing method reduces the amount of data computation on the image, thereby increasing the detection speed and solving the technical problems mentioned in the background art. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of the pallet pose detection method provided in the embodiments of this application;

[0042] Figure 2 This is a schematic diagram of the structure of the tray provided in an embodiment of this application;

[0043] Figure 3 A schematic diagram illustrating the implementation process of a pallet pose detection method provided in an embodiment of this application;

[0044] Figure 4 A schematic diagram illustrating the implementation process of another pallet pose detection method provided in this application embodiment;

[0045] Figure 5 A schematic diagram illustrating the implementation process of determining the target convolutional kernel provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of the convolution kernel provided in an embodiment of this application;

[0047] Figure 7 A schematic diagram illustrating the construction of a candidate tray image provided in an embodiment of this application;

[0048] Figure 8 A schematic diagram illustrating the implementation process of a pallet pose detection method provided in an embodiment of this application;

[0049] Figure 9 This is a schematic diagram of the pallet pose detection device provided in the embodiments of this application;

[0050] Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0053] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] During warehouse inbound and outbound processes, the position and posture of pallets are subject to significant uncertainty due to factors such as operational procedures, equipment precision, and manual labor. This makes it difficult for automated forklifts to quickly and accurately identify pallet poses, thus failing to meet the high-efficiency operational requirements of the warehousing and logistics industry. Furthermore, existing technologies often use 3D point cloud features to describe and detect pallet poses, employing algorithms such as region growing segmentation, RANSAC model segmentation, corner detection, and clustering. However, in the implementation of these algorithms, the extraction of geometric features and feature points is often affected by the quality of the point cloud, making it difficult to accurately and stably extract the physical features of the pallets. Additionally, point cloud data is large in volume, typically time-consuming to process, making it difficult to achieve rapid detection with limited hardware resources.

[0056] In view of this, embodiments of this application provide a pallet pose detection method, apparatus, device, and storage medium, which can improve the accuracy and speed of pallet pose detection. This method is applied to a detection device, which can be various types of devices with detection functions and information processing capabilities. For example, the detection device may include a depth camera, LiDAR, infrared camera, infrared scanner, etc. The functions implemented by this method can be achieved by a processor in the detection device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the detection device includes at least a processor and a storage medium.

[0057] The following will be described in detail with reference to the accompanying drawings.

[0058] like Figure 1 As shown, Figure 1 This is a schematic diagram of an application scenario for a pallet pose detection method disclosed in an embodiment of this application. In this application scenario, the pallet pose detection system includes an unmanned forklift 11 and a detection device 12 installed on the unmanned forklift.

[0059] The unmanned forklift 11 can be a lurking AGV (Automated Guided Vehicle), a counterbalance AGV, a forklift AGV, etc., and this application embodiment does not limit it to this.

[0060] Among them, the unmanned forklift is an intelligent industrial vehicle robot that integrates forklift technology and AGV technology. Compared with ordinary AGVs, it can not only complete point-to-point material handling, but also realize logistics transportation connecting multiple production links. It is not only good at three major scenarios: high-bay warehouses, off-site receiving areas, and production line transfers, but also plays an irreplaceable role in heavy-duty and special handling scenarios. The application of unmanned forklifts can solve the problems of large material flow and high labor intensity of manual handling in industrial production and warehousing logistics operations.

[0061] Based on product characteristics, forklift-type AGVs are currently widely used in scenarios such as high-bay warehouses, off-site receiving areas, and production line transfers. Especially in the diverse material handling operations of manufacturing enterprises, unmanned forklifts can perform multiple functions according to different needs in stages such as entering and leaving the warehouse, production lines, and storage.

[0062] The detection device 12 can be any device with detection and sensing functions, such as a depth camera, LiDAR, infrared camera, or infrared scanner. Furthermore, in this embodiment, the number of detection devices 12 is not limited; there can be one or more.

[0063] In this embodiment, the installation location of the detection device 12 is not limited. In a preferred embodiment, the detection device 12 may be installed at the midpoint between the fork roots of the unmanned forklift 11 to perform pose detection on pallets in the warehouse.

[0064] Before introducing the technical solution of this application, the structure of the tray will first be explained. Figure 2 A schematic diagram of a tray structure is provided, such as... Figure 2 As shown, the pallet includes multiple pallet feet 201 and an upper pallet plate 202, and there are foot holes 203 between adjacent pallet feet 201. Here, the area where the pallet feet 201 are located is referred to as the foot surface area of ​​the foot foot.

[0065] Figure 3 This is a schematic diagram illustrating the implementation process of the pallet pose detection method provided in this application embodiment, which can improve the accuracy and speed of pallet pose detection. Figure 3 As shown, this method is applied to a detection device installed on an unmanned forklift, and the method may include the following steps 301 to 304:

[0066] Step 301: Extract the region from the point cloud image of the area where the tray is located to obtain the initial tray image, which is the image after filtering out the background region from the point cloud image.

[0067] In this embodiment, the control device can send movement commands to the unmanned forklift to control it to travel to a detection point, which can be any position between 1.5m and 3m away from the pallet. Once the unmanned forklift reaches the detection point, the detection device installed on it can acquire a point cloud image of the area where the pallet is located. After acquiring the point cloud image, region extraction is performed to filter out background areas, retaining only the pallet image, referred to as the initial pallet image. This effectively avoids the influence of interfering point clouds in the background area on the accuracy of pallet pose determination and reduces the amount of point cloud computing required for subsequent pallet pose calculations, thus improving calculation speed.

[0068] Step 302: Perform convolution processing on the initial pallet image according to the predetermined target convolution kernel to obtain the target feature map. The target convolution kernel is determined according to the predetermined surface area of ​​each hole and the total surface area of ​​the pallet.

[0069] In this embodiment of the application, the size information of the pallet is pre-input into the detection device, such as... Figure 2 As shown, the pallet's dimensions include at least the width of the pallet footrest 201, the height of the pallet footrest 201, the height of the upper pallet plate 202, the width of the footrest opening 203, and the overall height of the pallet. In some embodiments, if the pallet has a lower pallet plate, the pallet's dimensions may also include the height of the lower pallet plate.

[0070] After obtaining the pallet's dimensions, the inspection equipment can calculate the surface area of ​​each part of the pallet, such as determining the surface area of ​​each hole and the total surface area of ​​the pallet.

[0071] After extracting the initial pallet image, the target convolution kernel corresponding to the initial pallet image can be determined based on the surface area of ​​each hole and the total surface area of ​​the pallet. Then, the initial pallet image is convolved based on the target convolution kernel to obtain the target feature map.

[0072] In essence, convolution processing of an image is filtering the pixels to extract the relevant and useful information. In practice, image convolution is performed using a template (kernel). For a pixel *i*, the origin of the template coincides with pixel *i*. Then, the weights of multiple points on the template are multiplied sequentially by the pixel value of *i*, resulting in multiple products corresponding to pixel *i*. These products are then summed to obtain the convolution value for pixel *i*. By performing this process on each pixel of the image, the target feature map corresponding to that image can be obtained.

[0073] Depending on the type of convolution kernel, the feature focus of the target feature map obtained by convolution processing the image is also different. Therefore, in order to better meet actual needs and improve detection accuracy, in this embodiment of the application, the target convolution kernel corresponding to the pallet image is determined according to the pre-determined surface area of ​​each hole and the total surface area of ​​the pallet. Then, the initial pallet image is convolved based on the target convolution kernel to obtain the target feature map.

[0074] In some embodiments, step 302 can be implemented by performing steps 403 to 405 in the following embodiments.

[0075] Step 303: Based on the target feature map and the predetermined total surface area of ​​the tray, the initial tray image is sliced ​​to obtain point cloud images of multiple sub-candidate tray layers.

[0076] Understandably, since a pallet mainly consists of multiple pallet legs and upper and lower pallets, and there are multiple support holes between the pallet legs, it is quite difficult to analyze the pallet pose based on the overall pallet. Therefore, in this embodiment, after obtaining the target feature map corresponding to the pallet, the initial pallet image can be sliced ​​according to the target feature map and the predetermined total surface area of ​​the pallet, thereby obtaining point cloud images of multiple sub-candidate pallet layers. In this way, the point cloud image of each sub-candidate pallet layer contains a smaller amount of point cloud data, but includes basic information about the pallet, making processing simpler and faster.

[0077] In some embodiments, step 303 can be achieved by performing steps 406 to 407 as follows.

[0078] Step 304: Combine the point cloud images of each sub-candidate tray layer to determine the target pose of the tray.

[0079] After obtaining the point cloud image of each sub-candidate tray layer, the point cloud image of each sub-candidate tray layer can be analyzed, and the analysis results of each sub-candidate tray layer can be combined to obtain the target pose of the tray.

[0080] In some embodiments, step 304 can be implemented by performing steps 408 to 411 in the following embodiments.

[0081] In this embodiment, an initial pallet image is obtained by extracting regions from the point cloud image of the area where the pallet is located. The initial pallet image is the image after filtering out the background region from the point cloud image. A target convolution kernel is determined based on the pre-determined surface area of ​​each pier hole and the total surface area of ​​the pallet. The initial pallet image is then convolved using the target convolution kernel to obtain a target feature map. Based on the target feature map and the pre-determined total surface area of ​​the pallet, the initial pallet image is sliced ​​to obtain point cloud images of multiple sub-candidate pallet layers. The target pose of the pallet is determined by combining the point cloud images of each sub-candidate pallet layer. This improves the accuracy and speed of pallet pose detection.

[0082] This application provides another method for detecting the position and posture of a tray. Figure 4 This is a schematic diagram illustrating the implementation process of the pallet pose detection method provided in the embodiments of this application, as follows: Figure 4 As shown, the method may include the following steps 401 to 408:

[0083] Step 401: Obtain the initial pose of the tray.

[0084] In this embodiment, the control device can send movement commands to the unmanned forklift to control it to travel to a detection point, which can be any position between 1.5m and 3m away from the pallet. Once the unmanned forklift reaches the detection point, the initial pose of the pallet can be acquired by the detection device installed on it. This initial pose is the pallet's pose relative to the detection device, and can be represented as (x, y, z, theta). Here, x, y, and z refer to the lateral distance, longitudinal distance, and depth distance of the pallet relative to the center point of the detection device, respectively, using the center point of the detection device as the base coordinate system. Theta refers to the angle between the center point of the pallet and the center point of the detection device.

[0085] Step 402: Based on the initial pose of the tray and the preset region of interest threshold, region extraction is performed on the point cloud image to obtain the initial tray image, which is the image after filtering out the background region in the point cloud image.

[0086] The region of interest (ROI) is a region of the image that needs to be processed, delineated using shapes such as rectangles, circles, ellipses, or irregular polygons. This region is the focus of attention when analyzing the image.

[0087] In this embodiment of the application, after obtaining the initial pose (coarse pose) of the tray, the tray area can be extracted as the region of interest from the obtained point cloud image based on the preset region of interest threshold and the initial pose of the tray, thereby filtering out the interference of the background area in the point cloud image.

[0088] This effectively avoids the impact of interfering point clouds in the background area on the accuracy of pallet pose determination, and reduces the amount of point cloud computing required for subsequent pallet pose calculations, thereby improving calculation speed and accuracy.

[0089] Step 403: Map the initial tray image to obtain the projection image of the initial tray image on the first plane.

[0090] Here, the initial tray image is mapped by projecting it along the z-axis onto a plane, resulting in a two-dimensional planar image. This effectively reduces the amount of data and improves processing speed.

[0091] Here, the first plane is the XOY plane.

[0092] Of course, the direction of projecting the initial tray image is not limited in this embodiment. It can also be projected along the x-axis or y-axis, depending on the specific requirements.

[0093] In the embodiments of this application, the specific method of performing planar projection on the initial tray image is not limited. For example, the initial tray image can be orthogonally projected along the z-axis.

[0094] Step 404: Determine the target convolution kernel corresponding to the projected image based on the pre-determined surface area of ​​each hole and the total surface area of ​​the tray.

[0095] Depending on the convolution kernel, the feature emphasis in the target feature map obtained by convolution processing the image will also be different. Therefore, in order to better meet actual needs and improve detection accuracy, in this embodiment of the application, the target convolution kernel to be used is determined based on the pre-determined surface area of ​​each hole and the total surface area of ​​the pallet, and then the initial pallet image is convolved based on the target convolution kernel to obtain the target feature map.

[0096] Furthermore, in convolution operations, the weight distribution of the convolution kernel represents the penalty and gain on the current region. For example, based on the characteristics of the pier hole region, the non-pier hole region, and the pier surface region, different penalties and gains can be set for the pier hole region, the non-pier hole region, and the pier surface region, that is, different convolution kernels can be set for the pier hole region, the non-pier hole region, and the pier surface region.

[0097] Based on this, in some embodiments, to determine the target convolution kernels corresponding to different regions in the tray, the following steps 501 to 503 can be performed:

[0098] Step 501: Determine the size of the target convolution kernel corresponding to the projected image based on the resolution of the projected image.

[0099] The resolution of a projected image refers to the number of pixels per unit surface area. Different resolutions result in different levels of clarity. When determining the specific structure of a convolutional kernel, the size of the target convolutional kernel used in the current convolutional processing must first be determined. To better suit practical needs, the size of the target convolutional kernel corresponding to the projected image can be determined based on the resolution of the projected image.

[0100] In the embodiments of this application, the specific method for determining the size of the target convolution kernel corresponding to the projected image based on the resolution of the projected image is not limited.

[0101] In some embodiments, the size of the target convolution kernel corresponding to the pier surface area in the projected image can be determined to be proportional to the width and height of the pier. Here, the form of the target convolution kernel is not limited; it can be square or rectangular. When the target convolution kernel is square, it can be represented as follows: Figure 6 The 3x3 convolution kernel shown.

[0102] Step 502: Determine the sum of the surface areas of the multiple pier holes as the first surface area, and determine the difference between the total surface area of ​​the pallet and the first surface area as the second surface area.

[0103] like Figure 2 As shown, the tray includes multiple support holes 203. The sum of the surface areas of these multiple support holes 203 is determined as the first surface area, and the surface area of ​​other areas in the tray outside the support holes 203 is taken as the second surface area.

[0104] The sum of the surface areas of multiple holes (i.e., the first surface area) is represented by size_hole, the total surface area of ​​the pallet is represented by size_all, and the second surface area is represented by size_all-size_hole.

[0105] Step 503: Under the size of the target convolution kernel, determine the target convolution kernel corresponding to each region in the tray based on the first surface area and the second surface area.

[0106] In this embodiment of the application, the method for determining the target convolutional kernel may include the following:

[0107] (1) The ratio of the first value to the first surface area is determined to be the weight of each point in the target convolution kernel corresponding to the hole area, and the ratio of the second value to the first surface area is determined to be the weight of each point in the target convolution kernel corresponding to other areas in the tray except the hole area. The first value and the second value are opposites of each other.

[0108] In this embodiment, the specific values ​​of the first and second values ​​are not limited. For example, the first value can be set to 1 and the second value to -1. Then, the weight of each point in the target convolution kernel corresponding to the hole area is 1 / size_hole, and the weight of each point in the target convolution kernel corresponding to other areas in the tray besides the hole area is -1 / size_hole.

[0109] Of course, in other embodiments, based on the same target convolution kernel setting logic, the ratio of the first value to the surface area of ​​a single pier hole region can also be determined as the weight of each point in the target convolution kernel corresponding to each pier hole region, and the ratio of the second value to the surface area of ​​a single pier hole region can be determined as the weight of each point in the target convolution kernel corresponding to other regions in the tray besides the pier hole region, with the first value and the second value being opposites of each other.

[0110] (2) Determine the ratio of the first value to the first surface area as the weight of each point in the target convolution kernel corresponding to the hole area, and determine the ratio of the second value to the second surface area as the weight of each point in the target convolution kernel corresponding to other areas in the tray except the hole area.

[0111] If the first value is set to 1 and the second value is set to -1, then the weight of each point in the target convolution kernel corresponding to the hole area is 1 / size_hole, and the weight of each point in the target convolution kernel corresponding to other areas in the tray, excluding the hole area, is -1 / (size_all-size_hole).

[0112] Of course, in other embodiments, based on the same target convolution kernel setting logic, the ratio of the first value to the surface area of ​​a single pier hole region can also be determined as the weight of each point in the target convolution kernel corresponding to each pier hole region, and the ratio of the second value to the third surface area can be determined as the weight of each point in the target convolution kernel corresponding to other regions in the tray besides the pier hole region, where the third surface area is the difference between the total surface area of ​​the tray and the surface area of ​​a single pier hole region.

[0113] (3) Determine that the weights of each point in the target convolution kernel corresponding to the pier surface area of ​​each foot pier conform to a two-dimensional Gaussian distribution, and that the weights of each point in the target convolution kernel corresponding to each pier hole area are opposites of the weights of each point in the target convolution kernel corresponding to other areas in the tray except for the pier hole area.

[0114] It should be noted that by conforming the weights of each point in the target convolution kernel corresponding to the pier surface area of ​​each pier foot to a two-dimensional Gaussian distribution, the point cloud constituting the pier surface area can have stronger noise resistance in the x and y directions, thus making the method applicable to the detection of pallet pose in the presence of noise.

[0115] The weights of each point in the target convolution kernel corresponding to each pier hole area are set to be opposites of the weights of each point in the target convolution kernel corresponding to other areas in the tray, excluding the pier hole area. That is, the ratio of the weights of each point in the target convolution kernel corresponding to each pier hole area to the weights of each point in the target convolution kernel corresponding to other areas in the tray is 1:-1.

[0116] Based on the various methods for determining the target convolution kernels provided above, one method can be selected according to actual needs during practical use, making the convolution method more flexible and more in line with actual production needs, thereby improving the accuracy of pallet pose detection.

[0117] Step 405: Perform convolution processing on the projected image according to the target convolution kernel to obtain the target feature map.

[0118] It should be noted that when using target convolution kernels to perform convolution processing on the projected image, the process involves performing convolution processing on each of the predetermined target convolution kernels corresponding to the pier hole region, pier surface region, and other regions in the tray besides the aforementioned regions in the projected image, thereby obtaining the target feature map corresponding to the projected image. This target feature map contains the approximate pose of the tray on the XOY plane.

[0119] Step 406: Extract candidate pallet images from the target feature map based on the predetermined total surface area of ​​the pallet.

[0120] Understandably, as described in step 302 above, before performing position detection on the pallet, the pallet's size information will be input into the detection device. For example, the width of the pallet foot block 201, the height of the pallet foot block 201, the height of the upper pallet plate 202, the width of the foot block hole 203, and the overall height of the pallet will be input into the detection device. In this way, the detection device can know the total surface area of ​​the pallet.

[0121] After obtaining the target feature map corresponding to the projected image, candidate pallet images can be extracted again from the target feature map based on the known total surface area of ​​the pallet. These candidate pallet images further filter out interfering point clouds in the target feature map and reduce the amount of point cloud processing data, thereby improving detection accuracy and speed in subsequent pallet pose detection based on the candidate pallet images.

[0122] like Figure 7 As shown, a schematic diagram of the construction of a candidate tray image is given, which is composed of multiple point clouds.

[0123] Step 407: On the second plane of the candidate tray image, the candidate tray image is sliced ​​to obtain point cloud images of multiple sub-candidate tray layers. The second plane is perpendicular to the first plane.

[0124] As analyzed in step 303, since the pallet mainly consists of multiple pallet feet and upper and lower pallet plates, and there are multiple support holes between the pallet feet, it is quite difficult to analyze the pallet pose based on the overall pallet. Therefore, in this embodiment of the application, after obtaining the candidate pallet image corresponding to the pallet, the candidate pallet image can be sliced ​​to obtain the point cloud image of multiple sub-candidate pallet layers corresponding to the pallet on the second plane.

[0125] Here, the second plane is a plane perpendicular to the first plane; that is, if the first plane is the XOY plane, then the second plane is the XOZ plane. It should be noted that in the direction of this second plane (e.g., ... Figure 7When slicing the candidate tray image along the direction indicated by the arrow, the slicing is based on a fixed thickness, thereby obtaining point cloud images of multiple sub-candidate tray layers. Here, the fixed thickness can be set to 0.01m. Of course, this fixed thickness can also be set to other values ​​according to actual needs, and this application embodiment does not limit it in this way.

[0126] Step 408: Perform clustering processing on the point cloud image of each sub-candidate tray layer to obtain clustering results, which contain multiple point cloud clusters.

[0127] Clustering refers to dividing a dataset into different classes or clusters according to a specific criterion (such as distance), so that the similarity of data objects within the same cluster is maximized, while the differences between data objects in different clusters are maximized. In other words, after clustering, data of the same class are grouped together as much as possible, and data of different classes are separated as much as possible.

[0128] In this embodiment, the specific method for clustering the point cloud image of each sub-candidate tray layer is not limited. It can be a partitioning clustering method, a density-based clustering method, a hierarchical clustering method, etc.

[0129] By performing clustering processing on the point cloud images of the sub-candidate pallet layer, the point cloud data constituting a single pallet foot in the point cloud images of the sub-candidate pallet layer can be clustered into a class, thereby obtaining the point cloud cluster corresponding to each pallet foot. In addition, the point cloud data constituting the upper pallet in the point cloud images of the sub-candidate pallet layer can also be clustered into a class, thereby obtaining the point cloud cluster corresponding to the upper pallet.

[0130] Step 409: Based on the clustering results of the point cloud images of each sub-candidate tray layer, determine the point cloud images of the sub-candidate tray layers whose clustering results meet the preset conditions from the point cloud images of multiple sub-candidate tray layers as the point cloud images of the target tray layer.

[0131] Understandably, since noisy point clouds may exist in the point cloud images of each sub-candidate pallet layer, the clustered pallet foot point cloud clusters may not equal the actual number of pallet feet during clustering. For example, during the clustering process, due to noise, adjacent pallet feet may be misidentified as a single cluster, resulting in a final clustered number of pallet feet that is less than the actual number of pallet feet.

[0132] Therefore, in this embodiment, after obtaining the clustering results of the point cloud images for each sub-candidate tray layer, it is necessary to filter the point cloud images of each sub-candidate tray layer based on the clustering results to remove the clustering results of the point cloud images of the sub-candidate tray layers that do not meet the standards, and retain the point cloud images of the sub-candidate tray layers that are correctly clustered. In this way, the influence of erroneous clustering results on the tray pose determination during subsequent processing can be effectively avoided, and the amount of data computation can be reduced, thereby improving the accuracy and speed of detection.

[0133] In this embodiment, the specific conditions for filtering the point cloud images of each sub-candidate pallet layer based on clustering results are not limited. For example, the condition can be set to the inter-cluster distance between adjacent point cloud clusters in the point cloud image of the sub-candidate pallet layer being less than a first threshold, and the number of clusters in the point cloud image of the sub-candidate pallet layer being greater than a second threshold. This ensures that the number of point cloud clusters in the filtered sub-candidate pallet layer point cloud images matches the actual number of pallet feet. It can be seen that the first threshold is the actual width of the foot hole, and the second threshold is the sum of the actual number of pallet feet and the number of upper and lower pallets.

[0134] Step 410: Fit the point cloud image of each target pallet layer to obtain the pallet plane corresponding to the point cloud image of each target pallet layer.

[0135] In this embodiment, the method for fitting the point cloud image of the target tray layer is not limited. For example, the Random Sample Consensus (RANSAC) algorithm can be used to fit a straight line to the point cloud image of the target tray layer to obtain the tray plane corresponding to the point cloud image of the target tray layer. In this way, the influence of other interfering objects and interfering point clouds in the point cloud image of the target tray layer can be filtered out, improving the accuracy and speed of tray pose detection.

[0136] Step 411: Based on the cluster center position in each tray plane, combine the tray planes to obtain the target pose of the tray.

[0137] In some embodiments, to achieve combined processing of each pallet plane to obtain the target pose of the pallet, the point cloud clusters in each pallet plane can be sorted, and the centroid of each point cloud cluster in the pallet plane can be calculated. The centroid is represented by cluster_pij, and based on... Calculate the target pose of the tray, where i represents the point cloud data of the i-th row of foot blocks, j represents the point cloud cluster of the j-th foot block, n is the number of point cloud rows of foot blocks, and m is the number of foot blocks.

[0138] In this embodiment, the initial pose of the pallet is obtained. Based on the initial pose and a preset region of interest threshold, region extraction is performed on the point cloud image to obtain an initial pallet image, which is the image after filtering out the background region from the point cloud image. The initial pallet image is then mapped to obtain a projection image of the initial pallet image on a first plane. Based on the pre-determined surface area of ​​each pier hole and the total surface area of ​​the pallet, a target convolution kernel corresponding to the projection image is determined. The projection image is then convolved using the target convolution kernel to obtain a target feature map. Candidate pallet images are extracted from the target feature map based on the pre-determined total surface area of ​​the pallet. On the second plane of the candidate pallet image, the candidate pallet image is then processed... The tray image is sliced ​​to obtain point cloud images of multiple candidate tray layers, with the second plane perpendicular to the first plane. Each candidate tray layer's point cloud image is then clustered to obtain clustering results containing multiple point cloud clusters. Based on the clustering results of each candidate tray layer, the point cloud images of the target tray layer whose clustering results meet preset conditions are selected. Each target tray layer's point cloud image is then fitted to obtain the corresponding tray plane. Finally, based on the cluster center positions in each tray plane, the tray planes are combined to obtain the target tray pose. This effectively reduces the amount of point cloud data in the tray pose detection process and utilizes convolution kernels more closely aligned with practical needs to perform convolution processing on the acquired images, resulting in more accurate extracted tray features and thus significantly improving the accuracy and speed of tray pose detection.

[0139] The following describes an exemplary application of the embodiments of this application in a real-world application scenario.

[0140] Figure 8 The overall flow of the pallet pose detection method provided in the embodiments of this application is as follows. Figure 8 As shown, the method includes the following steps 801 to 807:

[0141] Step 801: Install a ToF camera (i.e., detection device) at the midpoint between the roots of the forks of the unmanned forklift. Alternatively, it can be installed in other locations or other sensors can be used depending on the actual conditions.

[0142] Step 802: Set a configuration file in the ToF camera. Enter the pallet information in the configuration file, including the width of the pallet feet, the height of the pallet feet, the distance between two pallet feet, the pallet height, and the height of the upper and lower pallets.

[0143] Step 803: Control the unmanned forklift to drive to the detection point (ToF camera 1.5-3.0m away from the pallet), and send the initial pose ideal_pose (i.e. x, y, z, theta) to the ToF camera.

[0144] Step 804: The ToF camera extracts the ROI from the acquired depth image (i.e., point cloud image) based on the initial pose ideal_pose and the preset ROI threshold.

[0145] Step 805: In the camera coordinate system, the ToF camera re-orthogonally projects the point cloud in the ROI region (i.e., the initial tray image) of the depth image along the z-direction according to the input resolution to obtain a new projected image.

[0146] Step 806: Based on the prior information of the tray in step 802, generate a rectangular convolution kernel that is proportional to the front of the tray (the convolution kernel has the same resolution as the projected image).

[0147] In convolution operations, the weight distribution of the convolution kernel represents the penalty and gain for regions with and without convolution holes. Specifically, convolution kernels can be classified into the following categories according to their weight distribution:

[0148] Let the surface area of ​​the front of the tray (XOZ in the camera coordinate system) be size_all, and the surface area of ​​the hole on the front of the tray be size_hole. The convolution kernel is set as follows:

[0149] (1) Ordinary convolution kernel: Based on the distribution of the tray holes, the weight of the corresponding tray hole region in the convolution kernel is set to -1 / size;

[0150] (2) Equal weighted convolution kernel: Based on the distribution of the pallet's foot holes, the weight of the corresponding pallet foot hole region in the convolution kernel is set to -1 / size_hole, and the weight of the other regions (including foot holes and upper and lower pallets) is set to 1 / (size_all-size_hole).

[0151] (3) Equal weight Gaussian convolution kernel: Based on the distribution of the pier holes on the tray, and maintaining the total weight ratio of the pier hole area and the non-pier hole area at -1:1, the weight distribution of the corresponding area of ​​the pier is set to a two-dimensional Gaussian distribution, so that it has stronger noise resistance in the x and y directions.

[0152] Step 807: Convolve the projected image using the convolution kernel (i.e., the target convolution kernel) from step 806 to obtain the pose of the tray in the XOY plane.

[0153] Step 808: Based on the XOY plane pose and prior tray size information, extract candidate tray points (i.e., candidate tray images).

[0154] Step 809: Slice the candidate pallet points into layers. Specifically, slice the points with a fixed thickness parallel to the XOZ plane (camera coordinate system) to obtain horizontal layers of the point cloud (i.e., point cloud images of sub-candidate pallet layers). Cluster the horizontal layer point clouds, retaining layers whose length along the x-direction, inter-cluster distance, and number of clusters meet the thresholds, and consider these layers as candidate pier point clouds. For layers with more clusters than the prior information, perform a combined search to determine the correct candidate pier point cloud (i.e., point cloud image of the target pallet layer).

[0155] Step 810: Use RANSAC linear fitting to search for pier surface point clouds (i.e., the pallet plane corresponding to the point cloud image of the target pallet layer) from the candidate pier point clouds of each layer.

[0156] Step 811: Sort the point cloud clusters on each pier surface, calculate the centroid of each cluster (cluster_pij), and then... Calculate the pallet pose (i.e., the target pose), where i represents the i-th row of pier point cloud, j represents the j-th pier, n is the number of rows of pier point cloud, and m is the number of piers.

[0157] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0158] Based on the foregoing embodiments, this application provides a pallet pose detection device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0159] Figure 9 This is a schematic diagram of the pallet pose detection device provided in the embodiments of this application, as shown below. Figure 9 As shown, the device 900 includes an extraction module 901, a convolution module 902, a processing module 903, and a determination module 904, wherein:

[0160] The extraction module 901 is used to extract the region from the point cloud image of the area where the tray is located to obtain the initial tray image, which is the image after filtering out the background region in the point cloud image.

[0161] The convolution module 902 is used to perform convolution processing on the initial tray image according to a predetermined target convolution kernel to obtain a target feature map. The target convolution kernel is determined according to the surface area of ​​each of the pier holes and the total surface area of ​​the tray.

[0162] The processing module 903 is used to slice the initial tray image according to the target feature map and the predetermined total surface area of ​​the tray to obtain point cloud images of multiple sub-candidate tray layers;

[0163] The determination module 904 is used to synthesize the point cloud images of each sub-candidate tray layer to determine the target pose of the tray.

[0164] In some embodiments, the convolution module 902 is specifically used to perform mapping processing on the initial tray image to obtain a projection image of the initial tray image on a first plane; and to perform convolution processing on the projection image according to the target convolution kernel to obtain the target feature map.

[0165] In some embodiments, the convolution module 902 is further configured to: determine the size of the target convolution kernel corresponding to the projected image based on the resolution of the projected image; determine the sum of the surface areas of the plurality of pier holes as a first surface area, and determine the difference between the total surface area of ​​the tray and the first surface area as a second surface area; and, given the size of the target convolution kernel, determine the ratio of a first value to the first surface area as the weight of each point within the target convolution kernel corresponding to the pier hole region, and determine the ratio of a second value to the first surface area as the weight of each point within the target convolution kernel corresponding to other regions in the tray besides the pier hole region. The first value and the second value are opposites of each other; or the ratio of the first value to the first surface area is determined to be the weight of each point in the target convolution kernel corresponding to the pier hole area, and the ratio of the second value to the second surface area is determined to be the weight of each point in the target convolution kernel corresponding to other areas in the tray besides the pier hole area; or the weight of each point in the target convolution kernel corresponding to the pier surface area of ​​each pier conforms to a two-dimensional Gaussian distribution, and the weight of each point in the target convolution kernel corresponding to each pier hole area is opposite to the weight of each point in the target convolution kernel corresponding to other areas in the tray besides the pier hole area.

[0166] In some embodiments, the processing module 903 is specifically configured to extract a candidate pallet image from the target feature map based on the predetermined total surface area of ​​the pallet; and to slice the candidate pallet image on a second plane to obtain point cloud images of multiple sub-candidate pallet layers, wherein the second plane is perpendicular to the first plane.

[0167] In some embodiments, the determining module 904 is specifically configured to perform clustering processing on the point cloud images of each of the sub-candidate tray layers to obtain clustering results, wherein the clustering results contain multiple point cloud clusters; based on the clustering results of the point cloud images of each of the sub-candidate tray layers, determine the point cloud images of the sub-candidate tray layers whose clustering results meet preset conditions from the point cloud images of the multiple sub-candidate tray layers as the point cloud images of the target tray layer; perform fitting processing on the point cloud images of each target tray layer to obtain the tray plane corresponding to the point cloud image of each target tray layer; and perform combination processing on each tray plane according to the cluster center position in each tray plane to obtain the target pose of the tray.

[0168] In some embodiments, the preset conditions include that the inter-cluster distance between adjacent point cloud clusters in the point cloud image of the sub-candidate tray layer is less than a first threshold, and the number of clusters in the point cloud image of the sub-candidate tray layer is greater than a second threshold.

[0169] In some embodiments, the extraction module 901 is specifically used to obtain the initial pose of the tray; and to extract regions from the point cloud image based on the initial pose of the tray and a preset region of interest threshold to obtain an initial tray image.

[0170] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0171] It should be noted that, in the embodiments of this application... Figure 9 The module division of the pallet pose detection device shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit from two or more other units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0172] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0173] This application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a pallet pose detection method.

[0174] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0175] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0176] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In one embodiment, the pallet pose detection device provided in this application can be implemented as a computer program, and the computer program can be implemented in the form of... Figure 10It runs on the computer device shown. The computer device's memory can store the various program modules that make up the sampling device, for example, Figure 9 The extraction module, convolution module, processing module, and determination module are shown. The computer program, comprised of these modules, causes the processor to execute the steps of the pallet pose detection methods described in the various embodiments of this application.

[0178] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0179] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0180] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0181] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0183] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0185] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0186] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0187] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0188] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0189] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0190] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting the pose of a tray, characterized in that, The pallet includes multiple pallet feet, and there are foot holes between adjacent pallet feet. The method includes: Region extraction is performed on the point cloud image of the area where the tray is located to obtain an initial tray image, which is the image after filtering out the background area in the point cloud image; The initial pallet image is convolved according to a predetermined target convolution kernel to obtain a target feature map. The target convolution kernel is determined based on the surface area of ​​each of the pier holes and the total surface area of ​​the pallet. Based on the target feature map and the predetermined total surface area of ​​the tray, the initial tray image is sliced ​​to obtain point cloud images of multiple sub-candidate tray layers; The target pose of the tray is determined by combining the point cloud images of each of the sub-candidate tray layers.

2. The method according to claim 1, characterized in that, The step of convolving the initial tray image according to a predetermined target convolution kernel to obtain a target feature map includes: The initial tray image is mapped to obtain a projection image of the initial tray image on a first plane; The projected image is convolved using the target convolution kernel to obtain the target feature map.

3. The method according to claim 2, characterized in that, The target convolution kernel is determined based on the pre-determined surface area of ​​each of the aforementioned holes and the total surface area of ​​the tray, including: The size of the target convolutional kernel corresponding to the projected image is determined based on the resolution of the projected image. The sum of the surface areas of the plurality of said holes is determined as the first surface area, and the difference between the total surface area of ​​the pallet and the first surface area is determined as the second surface area; Given the size of the target convolutional kernel, the ratio of a first value to the first surface area is determined as the weight of each point within the target convolutional kernel corresponding to the hole region, and the ratio of a second value to the first surface area is determined as the weight of each point within the target convolutional kernel corresponding to other regions in the tray besides the hole region, wherein the first value and the second value are opposites of each other; or The ratio of the first value to the first surface area is determined as the weight of each point within the target convolution kernel corresponding to the hole region, and the ratio of the second value to the second surface area is determined as the weight of each point within the target convolution kernel corresponding to other regions in the tray besides the hole region; or The weights of each point in the target convolution kernel corresponding to the pier surface area of ​​each foot block conform to a two-dimensional Gaussian distribution, and the weights of each point in the target convolution kernel corresponding to each pier hole area are opposite to the weights of each point in the target convolution kernel corresponding to other areas in the pallet except for the pier hole area. The pier surface area is the area where the pallet foot block is located.

4. The method according to claim 2, characterized in that, The initial tray image is sliced ​​based on the target feature map and the predetermined total surface area of ​​the tray to obtain point cloud images of multiple sub-candidate tray layers, including: Candidate tray images are extracted from the target feature map based on the predetermined total surface area of ​​the tray; On the second plane of the candidate tray image, the candidate tray image is sliced ​​to obtain point cloud images of multiple sub-candidate tray layers, and the second plane is perpendicular to the first plane.

5. The method according to claim 4, characterized in that, The process of synthesizing the point cloud images of each of the sub-candidate tray layers to determine the target pose of the tray includes: Clustering processing is performed on the point cloud image of each of the sub-candidate tray layers to obtain clustering results, which contain multiple point cloud clusters; Based on the clustering results of the point cloud images of each of the sub-candidate tray layers, the point cloud images of the sub-candidate tray layers whose clustering results meet the preset conditions are determined from the point cloud images of the multiple sub-candidate tray layers as the point cloud images of the target tray layer. The point cloud image of each target pallet layer is fitted to obtain the pallet plane corresponding to the point cloud image of each target pallet layer; Based on the cluster center position in each tray plane, the tray planes are combined to obtain the target pose of the tray.

6. The method according to claim 5, characterized in that, The preset conditions include that the inter-cluster distance between adjacent point cloud clusters in the point cloud image of the sub-candidate tray layer is less than a first threshold, and the number of clusters in the point cloud image of the sub-candidate tray layer is greater than a second threshold.

7. The method according to claim 1, characterized in that, The step of extracting regions from the point cloud image of the area where the tray is located to obtain an initial tray image includes: Obtain the initial pose of the tray; Based on the initial pose of the tray and a preset region of interest threshold, the point cloud image is used to extract regions to obtain the initial tray image.

8. A pallet position detection device, characterized in that, include: The extraction module is used to extract regions from the point cloud image of the area where the tray is located to obtain an initial tray image, which is the image after filtering out the background area from the point cloud image. A convolution module is used to perform convolution processing on the initial pallet image according to a predetermined target convolution kernel to obtain a target feature map. The target convolution kernel is determined according to the surface area of ​​each hole and the total surface area of ​​the pallet. The processing module is used to slice the initial tray image according to the target feature map and the predetermined total surface area of ​​the tray to obtain point cloud images of multiple sub-candidate tray layers; The determination module is used to synthesize the point cloud images of each of the sub-candidate tray layers to determine the target pose of the tray.

9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images