A method for surface shape recognition and positioning based on a 3D sensor

Abstract

The present invention provides a surface shape recognition and positioning method based on a 3D sensor, wherein the recognition method comprises the following steps: S1, obtaining target object parameters and constructing a grid template of the end face of the target object; S2, establishing a target point cloud based on sensor data collected by the 3D sensor; S3, removing ground point clouds from the target point cloud to extract surface point cloud blocks perpendicular to the ground for convex hull area calculation; S4, setting a sliding window according to the end face parameters of the target object, sliding the sliding window according to a preset step length, and judging whether the number of point clouds in the sliding window and the convex hull area thereof meet preset conditions each time; S5, when the judgment result meets the preset conditions, extracting the point cloud in the sliding window and matching it with the grid template, wherein the steps comprise: rasterizing the point cloud in the sliding window according to the grid size of the grid template to construct a sliding window grid; sliding the sliding window grid step by step according to a preset step length, calculating the matching degree between the sliding window and the grid template, and stopping when a matching degree threshold is met to obtain a recognition result.

Description

Technical Field

[0001] The present invention relates to robot positioning technology, and in particular to a target object surface shape recognition and positioning method based on data collected by traditional 3D sensors. Background Art

[0002] The robot referred to in the present invention is a mobile robot capable of automatic operation, such as a wheeled robot, which can be divided into: cleaning robot, disinfection robot, inspection robot, handling robot, etc. according to different working attributes.

[0003] In the field of industrial applications, automated robots with mobile and handling functions are gradually replacing human intervention in existing production systems to achieve tasks such as material handling and insertion. However, the reason why these robots can achieve automated control mainly relies on the continuously developing robot recognition and positioning technology.

[0004] For example, an automatic forklift robot transfers goods by inserting and removing pallets. If the specific position of the pallet is not known in advance, insertion failure may easily occur during the robot's insertion process. Therefore, how to quickly identify and locate the position of such pallets is a goal that has been iteratively improved in this field.

[0005] Among the currently available technologies, more mature ones usually use RGB-D cameras to determine the position of the pallet based on features. This technology is also currently a relatively widely used technology, but the problem with this technology is that it is not very universal and relies on specific depth camera equipment.

[0006] On the other hand, sample-based learning technology is also a popular technology at present, also known as deep learning. Deep learning can achieve very high recognition accuracy, but it has high requirements on computing performance. Summary of the Invention

[0007] The main purpose of the present invention is to provide a surface shape recognition and positioning method based on a 3D sensor, so as to be universally applicable to a variety of 3D sensors to improve versatility while reducing computing performance requirements.

[0008] To achieve the above object, according to a first aspect of the present invention, a method for surface shape recognition based on a 3D sensor is provided, the steps of which include:

[0009] S1 obtains the target object parameters and constructs a grid template of the target object end face;

[0010] S2 builds a target point cloud based on the sensor data collected by the 3D sensor;

[0011] S3 removes the ground point cloud from the target point cloud to extract the surface point cloud blocks perpendicular to the ground for convex hull area calculation;

[0012] S4 sets a sliding window according to the end face parameters of the target object, and slides the sliding window according to a preset step size, wherein each sliding step determines whether the number of point clouds and the convex hull area of the surface point cloud block in the sliding window meet the preset conditions;

[0013] S5 When the result of step S4 is judged to meet the preset conditions, the point cloud in the sliding window is extracted and matched with the grid template. The steps include: rasterizing the point cloud in the sliding window according to the grid size of the grid template to construct a sliding window grid; sliding the sliding window grid step by step according to the preset step size, calculating the matching degree between it and the grid template, and stopping when the matching degree threshold is met to obtain the recognition result.

[0014] In order to achieve the above object, according to a second aspect of the present invention, a method for surface shape recognition based on a 3D sensor is provided, the steps of which include:

[0015] S1 obtains the target object parameters and constructs a coarse grid template and a fine grid template of the target object end face;

[0016] S2 builds a target point cloud based on the sensor data collected by the 3D sensor;

[0017] S3 removes the ground point cloud from the target point cloud to extract the surface point cloud blocks perpendicular to the ground for convex hull area calculation;

[0018] S4 sets a sliding window according to the end face parameters of the target object and slides the sliding window according to a preset step size, wherein each sliding step needs to determine whether the number of point clouds and the convex hull area of the surface point cloud block in the sliding window meet the preset conditions;

[0019] S5: When the result of step S4 meets the preset conditions, the point cloud in the sliding window is extracted and matched with the coarse grid template. The steps include: rasterizing the point cloud in the sliding window according to the grid size of the coarse grid template to construct a sliding window grid; sliding the sliding window grid step by step according to a preset step size, and calculating the matching degree between it and the coarse grid template until it meets the matching degree threshold, and recording the matching area;

[0020] S6 establishes a bounding box near the matching area, rasterizes the point cloud in the sliding window grid according to the grid size of the fine grid template to construct a fine sliding window grid, and performs sliding window matching with the fine grid template within the bounding box according to the preset step size until the one with the highest matching degree is obtained as the recognition result.

[0021] In a possible preferred embodiment, step S2 further includes preprocessing the target point cloud, which includes the following steps:

[0022] S21 performs filtering processing on the target point cloud to assign the target point cloud data to point cloud coordinates corresponding to the natural three-axis coordinate system, and filters out unmatched target point clouds according to the height parameter of the target object;

[0023] S22: Statistical filtering is performed on the target point cloud processed in step S21 to remove outliers.

[0024] In a possible preferred embodiment, the step of removing the ground point cloud from the target point cloud in step S3 includes:

[0025] S31 selects the reference plane with the largest number of points within the tolerance distance range from all points in the target point cloud data to the reference plane as the ground, and attributes all points on the plane to the ground component, and the remaining points to the object component.

[0026] In a possible preferred embodiment, the step of extracting planar point cloud blocks perpendicular to the ground in step S3 includes:

[0027] S32 randomly selects a seed point from the object component point cloud data, and determines whether the seed point and the non-seed points surrounding the seed point are in the same plane, wherein the normal vector of the seed point is perpendicular to the ground normal vector, and when it is determined that the seed point and the non-seed point are in the same plane, determines that the non-seed point is a new seed point;

[0028] S33 iteratively determines whether the new seed point and its surrounding non-seed points are in the same plane, and counts all seed points by growing the point cloud area;

[0029] S34 constructs a planar point cloud block perpendicular to the ground based on the counted seed points.

[0030] In a possible preferred embodiment, the step of constructing a planar point cloud block perpendicular to the ground based on the counted seed points in step S34 includes: determining whether the number of counted seed points is within a preset number range, and when the number of seed points is within the number range, constructing the planar point cloud block based on the counted seed points.

[0031] In a possible preferred embodiment, the step of constructing planar point cloud blocks perpendicular to the ground further includes: merging all planar point cloud blocks determined to be on the same surface of the same object.

[0032] In a possible preferred embodiment, the step of determining the same surface of the same object includes: converting each planar point cloud block into a plane equation ax+by+cz=1, and determining whether the three factors a, b, and c in the plane equations of each planar point cloud block are similar and the absolute value of the difference between the factors is less than a preset threshold. If so, it is determined to be the same surface of the same object.

[0033] In a possible preferred embodiment, the matching steps in steps S5 and S6 include:

[0034] Mark the grids in the sliding window grid or fine sliding window grid that contain point clouds and calculate X = (pq) / (1-q);

[0035] Wherein, for step S5, X represents the matching degree between the current sliding window grid and the coarse grid template, p represents the ratio of the number of identical grids between the current sliding window grid and the coarse grid template to the number of grids in the coarse grid template, and q represents the ratio of the number of unmarked grids in the sliding window grid to the total number of grids in the sliding window grid;

[0036] For step S6, X represents the degree of match between the current fine sliding window grid and the fine grid template, p represents the ratio of the number of identical grids between the current fine sliding window grid and the fine grid template to the number of grids in the fine grid template, and q represents the ratio of the number of unmarked grids in the fine sliding window grid to the total number of grids in the fine sliding window grid.

[0037] In order to achieve the above object, according to a third aspect of the present invention, a positioning method based on a 3D sensor is further provided, the steps of which include:

[0038] S1 obtains the center point coordinates of the fine sliding window grid with the highest matching degree as the x, y, and z parameters of 6Dpose according to any of the above-mentioned 3D sensor-based surface shape recognition methods;

[0039] S2 sets the roll and pitch angles to 0;

[0040] S3 establishes the plane equation ax+by+cz=1 based on the surface point cloud block, and calculates the yaw parameter value based on the plane equation and its three factors a, b, and c , to obtain the complete 6D pose.

[0041] The 3D sensor-based surface shape recognition and positioning method provided by the present invention is universally applicable to a variety of 3D sensor technologies and has high versatility, whether it is a depth camera, a multi-line lidar or a solid-state lidar; secondly, the solution of the present invention does not require pre-training samples, and at the same time, it occupies lower computing performance than the deep learning solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 Schematic diagram of the end face of a target object pallet in the surface shape recognition method based on a 3D sensor of the present invention;

[0044] Figure 2 Schematic diagram of a coarse grid template in the surface shape recognition method based on a 3D sensor of the present invention;

[0045] Figure 3 Schematic diagram of a fine grid template in the surface shape recognition method based on a 3D sensor of the present invention;

[0046] Figure 4 Schematic diagram of a sliding window in the surface shape recognition method based on a 3D sensor of the present invention;

[0047] Figure 5 This is a schematic diagram of a sliding window sliding on a planar point cloud block on the end face of a pallet in the surface shape recognition method based on a 3D sensor of the present invention.

[0048] Figure 6 This is a schematic diagram of matching the point cloud in the sliding window with the coarse grid template in the surface shape recognition method based on a 3D sensor of the present invention.

[0049] Figure 7 This is a schematic diagram of matching the point cloud in the sliding window with the fine grid template in the 3D sensor-based surface shape recognition method of the present invention.

[0050] Figure 8 Schematic diagram of the steps of the surface shape recognition and positioning method based on 3D sensor of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the specific technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments to help those skilled in the art further understand the present invention. Obviously, the embodiments described in this case are only embodiments of a part of the present invention, rather than all embodiments. It should be pointed out that, for those of ordinary skill in the art, the embodiments in this application and the features in the embodiments can be combined with each other without departing from the concept of the present invention and without conflicting with each other. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative work should fall within the disclosure and protection scope of the present invention.

[0052] In addition, the terms "first", "second", "S1", "S2", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those described herein. At the same time, the terms "including" and "having" and any variations thereof in the present invention are intended to cover non-exclusive inclusions. Unless otherwise expressly specified and limited, the terms "setting", "layout", "installation", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a communication between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in this case can be understood based on the specific circumstances and in combination with the existing technology.

[0053] It should be noted that in the examples of the present invention, the 3D sensor can be a 3D laser sensor, or a 3D camera sensor that only uses depth information. Specifically, the solution of the present invention is applicable to 3D laser radars with non-repetitive scanning modes, such as the DJI LivoxM70 sensor, but is also applicable to mechanically scanning 3D laser radars and depth cameras. For mechanically scanning 3D laser radars, calculations can be performed directly without stitching the point cloud; for depth cameras, the depth map needs to be converted into a 3D point cloud first, and calculations can be performed without stitching the point cloud.

[0054] Furthermore, while the following examples illustrate the identification and location of a single pallet, those skilled in the art will appreciate that the present invention's approach is also applicable to identifying multiple different types of pallets, and can even be extended to non-standard pallets. Thus, the present invention can directly input CAD drawings to obtain template parameters, rasterize the template and surface point cloud, and rapidly locate and accurately identify objects using coarse and fine grid resolutions.

[0055] The present invention will take a forklift robot inserting and removing a pallet as an example below to illustrate the process of installing a 3D sensor on the forklift robot to identify the pallet and its position.

[0056] To do this, see Figures 1 to 8 As shown, the surface shape recognition method based on a 3D sensor provided by the present invention comprises the following steps:

[0057] Step S1

[0058] Get the target object parameters and build a coarse grid template of the target object end face ( Figure 2shown) and the fine grid template ( Figure 3 Specifically, Figure 1 As shown, in this example, the target object (such as a pallet) can be modeled using software (such as CAD software). For this purpose, the parameters of each end face of the pallet can be obtained in advance. Secondly, after the end face parameters of the pallet are configured, a template of the target object can be formed. Thereafter, the template can be converted into a coarse grid template and a fine grid template for subsequent matching.

[0059] The "coarse / fine" of the coarse grid template and the fine grid template in this example refers to the relative size comparison relationship between the two. Therefore, it can be understood that the coarse / fine grid templates need to be constructed with different resolutions, that is, the resolution of the coarse grid template is smaller than that of the fine grid template, and the specific resolution can be adaptively set according to the actual end face size of the target object. In addition, any rasterized feature in this example can be understood as occupying 1 / 2 of the area of a grid box, that is, the grid is considered to be occupied, which facilitates subsequent template matching judgment.

[0060] Step S2

[0061] The pallet point cloud is created based on the sensor data collected by the 3D sensor and preprocessed. The steps include:

[0062] Step S21 pre-processes the pallet point cloud data by filtering it, then separating the ground component and the object component from the filtered pallet point cloud data. Specifically, the filtering operation assigns the acquired pallet point cloud data to point cloud coordinates corresponding to a natural three-axis coordinate system, then determines the required height of the pallet to be identified. Based on the required height, pallet point cloud data that does not match the required height is filtered out.

[0063] For example, when the height of the pallet is 15 cm, if the pallet is placed on the horizontal ground, a straight-through filter can be used to obtain only the point cloud 0 to 15 cm from the ground. If the pallet is stacked on two layers of pallets of the same height, a straight-through filter can be used to obtain the point cloud 30 to 45 cm from the ground.

[0064] Step S22: A statistical filter is then applied to the point cloud obtained after processing in step S21 to remove outliers.

[0065] Therefore, the noise and redundant data can be effectively reduced through multiple filtering preprocessing methods to improve the accuracy and operation speed of subsequent algorithms, as well as the robustness and real-time performance of the algorithms.

[0066] Step S3

[0067] The ground point cloud in the target point cloud is eliminated to extract the surface point cloud blocks perpendicular to the ground for convex hull area calculation.

[0068] The steps of removing the ground point cloud from the target point cloud include:

[0069] Step S31 selects the reference plane with the largest number of points within the tolerance distance from all points in the target point cloud data to the reference plane as the ground, and attributes all points on the plane to the ground component, while the remaining points are attributed to the object component.

[0070] Specifically, to remove ground point clouds from the point cloud after preprocessing in step S2, it is first necessary to extract the ground component and object component from the pallet point cloud data. To this end, in this example, a random sampling consensus algorithm and a plane model matching method are used to separate the ground component and object component from the point cloud data. Specifically, multiple points can be randomly extracted from the pallet point cloud data multiple times, and multiple reference planes are fitted accordingly; then, the number of corresponding points in the point cloud data whose distances to the multiple reference planes are within the tolerance distance range is counted; and then the number of counted points is compared to determine the ground component from the multiple reference planes.

[0071] For example, the reference plane with the largest number of points within the tolerance distance from all points in the point cloud data to the reference plane can be selected as the ground, and all points on the plane can be attributed to the ground component, thereby separating the ground component and the object component in the pallet point cloud data.

[0072] The steps of extracting planar point cloud blocks perpendicular to the ground include:

[0073] Step S32 randomly selects a seed point from the object component point cloud data, and determines whether the seed point and the non-seed points around the seed point are in the same plane, where the normal vector of the seed point is perpendicular to the ground normal vector. When it is determined that the seed point and the non-seed point are in the same plane, the non-seed point is determined to be a new seed point.

[0074] Step S33 iteratively determines whether the new seed point and the surrounding non-seed points are in the same plane, and counts all the seed points by growing the point cloud area.

[0075] Step S34 constructs a planar point cloud block perpendicular to the ground based on the seed points counted in step S33. Specifically, it determines whether the number of counted seed points is within a preset range. If the number of seed points is within the range, a planar point cloud block perpendicular to the ground can be constructed based on the counted seed points.

[0076] At the same time, if the number of seed points is too high or too low, it can be determined that the surface corresponding to the counted seed points does not belong to the surface point cloud block. This approach can also derive the plane equation parameters.

[0077] After acquiring all planar point cloud blocks, this example further merges all planar point cloud blocks perpendicular to the ground to ensure the integrity of the same surface of the same object. For example, based on the aforementioned plane equation and distance, if the factors of the plane equations are similar and the distance meets the threshold, the two point cloud blocks are merged into the same point cloud block.

[0078] For example, any plane equation can be written as ax+by+cz=1. If the factors a, b, and c in the plane equations of two planar point cloud blocks are similar and the absolute value of the difference between the factors is less than a threshold, the two planar point cloud blocks can be merged into a single point cloud block, thereby ensuring the integrity of the same surface of the same object.

[0079] Since the above steps have already eliminated point cloud blocks with too few points, we can now calculate the area of the convex hull for each point cloud block to eliminate those with too small an area. In this example, calculating the convex hull area actually involves finding a polygon that completely encloses the planar point cloud block and calculating the area of this polygon.

[0080] Step S4

[0081] The sliding window is set according to the end face parameters of the pallet, and the sliding window is slid according to the preset step size. Each time the sliding is performed, it is necessary to determine whether the number of point clouds and the convex hull area of the surface point cloud block in the sliding window meet the preset conditions.

[0082] Specifically, the end face parameters for establishing the sliding window have been obtained in step S1, such as the length and width of the end face of the pallet and the length and width of the middle hole. Figure 4 As shown, a sliding window can be directly set according to the parameter, and then the sliding window is slid according to a preset step size, wherein the preset step size is determined according to the size of the unit grid, for example, an integer multiple of the unit grid length of the coarse grid template.

[0083] Afterwards, if Figure 5 As shown, each time you slide, you need to determine the number of points in the window and the area of the convex hull. If they meet the conditions, you can proceed to the next step. The number of points in the convex hull depends on the resolution of different 3D sensors and the distance of the scanning stack. In the example, it is generally not less than 100 points. In addition, the area can be calculated based on the known length and width of the stack parameters. 2 / 3 of this area is used as the threshold. If it is greater than the threshold, it is considered a valid point cloud block, that is, it meets the conditions. Otherwise, continue sliding the window.

[0084] Step S5

[0085] When the result of step S4 determines that the preset conditions are met, the point cloud in the sliding window is extracted and matched with the coarse grid template. The steps include: rasterizing the point cloud in the sliding window according to the grid size of the coarse grid template to construct a sliding window grid; sliding the sliding window grid step by step according to a preset step size, and calculating the matching degree between the sliding window grid and the coarse grid template until the matching degree threshold is met, and recording the matching area;

[0086] Specifically, if Figure 6 As shown, the point cloud in the sliding window is matched with the constructed coarse grid template. First, the point cloud in the sliding window is rasterized according to the grid size of the coarse grid template to obtain the sliding window grid. The size of the grid determines the accuracy of recognition and positioning and the calculation speed. In this example, the resolution of the coarse grid template is set to 3cm, and the resolution of the fine grid is set to 0.5cm. It should be noted that the specific division of the grid size is related to the resolution of the 3D sensor and the size of the stack. It can be adjusted during implementation. For example, for each planar point cloud block, the largest and smallest points in the point cloud block are first calculated. The height and width of the actual size are obtained by subtracting the x and y coordinates of the largest and smallest points. Then the size of the grid image can be obtained as follows:

[0087]

[0088] Then the sliding window grid and the coarse grid are matched and judged, such as Figure 6 As shown in the figure, the gray grid is the occupied grid and the white grid is the idle grid. The matching calculation formula is:

[0089]

[0090] Where X represents the matching degree between the current sliding window and the coarse grid template, p represents the ratio of the number of identical grids between the current sliding window grid and the coarse grid to the number of grids in the coarse grid, and q represents the ratio of the number of white grids in the window grid to the total number of grids in the coarse grid.

[0091] Afterwards, the sliding window grid is gradually slid according to the preset step size to determine whether the matching degree between the sliding window grid and the coarse grid template is greater than the set threshold. If it is greater than the threshold, it means that the recognition result is obtained, and the sliding window grid is in the matching area. At this time, according to the matched coarse grid template, it can be known which end face of the pallet the current detection input point cloud belongs to.

[0092] Furthermore, after the rough recognition is completed, the positioning calculation after recognition can be directly performed, that is, the position of the pallet point cloud obtained by the current 3D sensor in space, namely 6Dpose, is calculated.

[0093] As another aspect of the present invention, a pallet positioning method based on a 3D sensor is also provided, wherein the pallet has 6 degrees of freedom in space, namely, translation x, y, z and rotation roll, pitch, yaw, i.e., 6D pose. Figure 8 As shown, the steps of the pallet positioning method include:

[0094] Step S1

[0095] According to the surface shape recognition method based on the 3D sensor in the above embodiment, the coordinates of the center point of the sliding window grid with the highest matching degree are obtained as the x, y, and z parameters of 6Dpose.

[0096] Step S2

[0097] Since the pallet and the robot are in the same space by default and the ground is horizontal, it can be assumed that the pallet is parallel to the forklift tines. That is, there are no roll and pitch angles. Therefore, the roll and pitch angles can be set to 0.

[0098] Step S3

[0099] At this time, 6Dpose also needs to calculate the yaw angle. For this purpose, in this example, it is preferred to calculate the yaw parameter value based on the plane equation ax+by+cz=1 of the stack surface point cloud block established in step S34 in the above embodiment, based on the plane equation and its three factors a, b, and c. , you can get the complete 6D pose [x, y, z, roll, pitch, yaw] parameters of the pallet end face, thus completing its positioning.

[0100] On the other hand, based on step S5, in order to obtain more accurate 6D pose data, a fine matching step may be performed.

[0101] Step S6

[0102] A bounding box is established near the matching area in step S5, and the point cloud in the sliding window grid is rasterized according to the grid size of the fine grid template to construct a fine sliding window grid. Sliding window matching is performed with the fine grid template within the bounding box according to the preset step size until the one with the highest matching degree is obtained as the recognition result.

[0103] Specifically, if Figure 7As shown, in order to obtain a more accurate 6D pose of the pallet point cloud, the point cloud in the sliding window grid needs to be more finely rasterized. At this time, a bounding box is established around the matching area in step S5, such as the area surrounded by one or more unit grids that are diffused by the current sliding window. Then, the sliding window grid is transformed according to the grid size of the fine grid template to obtain a fine sliding window grid (for specific steps, please refer to step S5), and the step size is set.

[0104] The fine sliding window grid then slides from the upper left corner of the bounding box, calculating the matching degree with each slide until the matching degree is satisfied. The sliding window result with the highest matching degree (see step S5 for details) is selected as the recognition result. Of course, this result can be further accurately determined based on the matching fine grid template to determine which end face of the pallet the input point cloud represents.

[0105] On the other hand, based on the recognition result, more accurate 6D pose data can be obtained at the end face of the pallet. Specifically, since the pallet's posture has 6 degrees of freedom in space, namely the translation x, y, z and the rotation roll, pitch, yaw, that is, 6D pose. For this reason, the present invention also provides a pallet positioning method based on a 3D sensor, such as Figure 8 As shown, the steps include:

[0106] Step S1

[0107] According to the surface shape recognition method based on the 3D sensor in the above embodiment, the center coordinates of the fine sliding window grid with the highest matching degree are obtained as the x, y, and z parameters of 6Dpose.

[0108] Step S2

[0109] Since the pallet and the robot are in the same space by default and the ground is horizontal, it can be assumed that the pallet is parallel to the forklift tines. That is, there are no roll and pitch angles. Therefore, the roll and pitch angles can be set to 0.

[0110] Step S3

[0111] At this time, 6Dpose also needs to calculate the yaw angle. For this purpose, in this example, it is preferred to calculate the yaw parameter value based on the plane equation ax+by+cz=1 of the stack surface point cloud block established in step S34 in the above embodiment. , you can get the complete 6D pose [x, y, z, roll, pitch, yaw] parameters of the pallet end face, thus completing its positioning.

[0112] In summary, the surface shape recognition and positioning method based on 3D sensors of the present invention can accurately and quickly identify objects and their positions that need to interact with robots, as long as the objects have certain continuous surface features at the angles that the sensor can scan. It is also very extensible. For example, the pallets in this case, whether they are standard pallets or non-standard pallets, only the various structural parameters of the pallets need to be provided to construct a template. In addition, the present invention can be universally applied to a variety of 3D sensors and has high versatility. It can be applied to depth cameras, multi-line laser radars, and solid-state laser radars. The solution of the present invention does not require pre-training samples, so it occupies less computing power than deep learning solutions.

[0113] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made based on the contents of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0114] Those skilled in the art will understand that, in addition to implementing the system, device, and various modules provided by the present invention in pure computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0115] In addition, all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program. The program is stored in a storage medium and includes a number of instructions for causing a single-chip microcomputer, chip, or processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0116] In addition, various implementations of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A surface shape recognition method based on a 3D sensor, characterized in that The steps include: S1 obtains the target object parameters and constructs a grid template of the target object end face; S2 builds a target point cloud based on the sensor data collected by the 3D sensor; S3 removes the ground point cloud from the target point cloud to extract the surface point cloud blocks perpendicular to the ground for convex hull area calculation; S4 sets a sliding window according to the end face parameters of the target object, and slides the sliding window according to a preset step size, wherein each sliding step determines whether the number of point clouds and the convex hull area of the surface point cloud block in the sliding window meet the preset conditions; S5: When the result of step S4 meets the preset conditions, extract the point cloud in the sliding window and match it with the grid template, the steps including: rasterizing the point cloud in the sliding window according to the grid size of the grid template to construct a sliding window grid; The sliding window grid is gradually slid according to the preset step size, and the matching degree between it and the grid template is calculated until the matching degree threshold is met to obtain the recognition result.

2. The surface shape recognition method based on a 3D sensor according to claim 1, characterized in that the steps include: S1 obtains the target object parameters and constructs a coarse grid template and a fine grid template of the target object end face; S2 builds a target point cloud based on the sensor data collected by the 3D sensor; S3 removes the ground point cloud from the target point cloud to extract the surface point cloud blocks perpendicular to the ground for convex hull area calculation; S4 sets a sliding window according to the end face parameters of the target object and slides the sliding window according to a preset step size, wherein each sliding step needs to determine whether the number of point clouds and the convex hull area of the surface point cloud block in the sliding window meet the preset conditions; S5: When the result of step S4 meets the preset conditions, extract the point cloud in the sliding window and match it with the coarse grid template, the steps including: rasterizing the point cloud in the sliding window according to the grid size of the coarse grid template to construct a sliding window grid; Slide the sliding window grid step by step according to the preset step size, and calculate the matching degree between it and the coarse grid template until it meets the matching degree threshold, and record the matching area; S6 establishes a bounding box near the matching area, rasterizes the point cloud in the sliding window grid according to the grid size of the fine grid template to construct a fine sliding window grid, and performs sliding window matching with the fine grid template within the bounding box according to the preset step size until the one with the highest matching degree is obtained as the recognition result.

3. The surface shape recognition method based on a 3D sensor according to claim 2, characterized in that: Step S2 also includes preprocessing the target point cloud, which includes the following steps: S21 performs filtering processing on the target point cloud to assign the target point cloud data to point cloud coordinates corresponding to the natural three-axis coordinate system, and filters out unmatched target point clouds according to the height parameter of the target object; S22: Statistical filtering is performed on the target point cloud processed in step S21 to remove outliers.

4. The surface shape recognition method based on a 3D sensor according to claim 2, characterized in that: The step of removing the ground point cloud from the target point cloud in step S3 includes: S31 selects the reference plane with the largest number of points within the tolerance distance range from all points in the target point cloud data to the reference plane as the ground, and attributes all points on the plane to the ground component, and the remaining points to the object component.

5. The surface shape recognition method based on a 3D sensor according to claim 4, characterized in that: The step of extracting the planar point cloud blocks perpendicular to the ground in step S3 includes: S32 randomly selects a seed point from the object component point cloud data, and determines whether the seed point and the non-seed points surrounding the seed point are in the same plane, wherein the normal vector of the seed point is perpendicular to the ground normal vector, and when it is determined that the seed point and the non-seed point are in the same plane, determines that the non-seed point is a new seed point; S33 iteratively determines whether the new seed point and its surrounding non-seed points are in the same plane, and counts all seed points by growing the point cloud area; S34 constructs a planar point cloud block perpendicular to the ground based on the counted seed points.

6. The surface shape recognition method based on a 3D sensor according to claim 5, characterized in that: The step of constructing a planar point cloud block perpendicular to the ground based on the counted seed points in step S34 includes: determining whether the number of counted seed points is within a preset number range, and when the number of seed points is within the number range, constructing the planar point cloud block based on the counted seed points.

7. The surface shape recognition method based on a 3D sensor according to claim 6, characterized in that: The step of constructing planar point cloud blocks perpendicular to the ground further includes: merging all planar point cloud blocks determined to be on the same surface of the same object.

8. The surface shape recognition method based on a 3D sensor according to claim 2, characterized in that: The matching steps in steps S5 and S6 include: Mark the grids in the sliding window grid or fine sliding window grid that contain point clouds and calculate X = (pq) / (1-q); Wherein, for step S5, X represents the matching degree between the current sliding window grid and the coarse grid template, p represents the ratio of the number of identical grids between the current sliding window grid and the coarse grid template to the number of grids in the coarse grid template, and q represents the ratio of the number of unmarked grids in the sliding window grid to the total number of grids in the sliding window grid; For step S6, X represents the degree of match between the current fine sliding window grid and the fine grid template, p represents the ratio of the number of identical grids between the current fine sliding window grid and the fine grid template to the number of grids in the fine grid template, and q represents the ratio of the number of unmarked grids in the fine sliding window grid to the total number of grids in the fine sliding window grid.

9. A positioning method based on a 3D sensor, characterized in that the steps include: S1, according to the surface shape recognition method based on a 3D sensor as described in any one of claims 1 to 8, obtaining the center point coordinates of the fine sliding window grid with the highest matching degree as the x, y, and z parameters of 6Dpose; S2 sets the roll and pitch angles to 0; S3 establishes the plane equation ax+by+cz=1 based on the surface point cloud block, and calculates the yaw parameter value based on the plane equation and its three factors a, b, and c , to obtain the complete 6D pose.

Images