A circular target positioning method and system based on ToF camera three-dimensional point cloud projection

Abstract

This application relates to a method for locating a circular target based on the projection of a 3D point cloud from a ToF camera. The method includes acquiring a 3D point cloud from the ToF camera; transforming the 3D point cloud from the camera coordinate system to the mobile robot's base coordinate system to obtain a base coordinate system point cloud; cropping the base coordinate system point cloud to obtain a region of interest (ROI) point cloud; fitting the ROI point cloud to obtain a vertical plane; projecting the ROI point cloud onto the vertical plane to obtain a projected point cloud; and rasterizing the projected point cloud into a 2D image; performing preprocessing, contour filtering, and circle fitting on the 2D image to obtain target circle parameters; and calculating the 3D pose of the circular target based on the target circle parameters and the normal vector of the vertical plane through inversion. This method addresses the problem in related technologies of how to reduce computational complexity while ensuring positioning accuracy and meeting industrial real-time requirements.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for locating circular targets based on ToF camera 3D point cloud projection. Background Technology

[0002] In industrial automated warehousing and logistics scenarios, the automated handling and stacking of cylindrical targets (such as paper rolls, steel coils, and plastic rolls) is frequently required. Traditional manual operations are inefficient and labor-intensive, and can no longer meet the needs of modern smart factories. Automated operations using mobile robots (such as AGVs and forklifts) require precise target identification and positioning as a prerequisite. Accurately obtaining the position and orientation of the cylindrical target in space (especially along the roller axis) is essential for completing tasks such as insertion and alignment.

[0003] Currently, commonly used target perception solutions in industrial scenarios include RGB vision, 3D LiDAR, and ToF depth cameras. RGB vision is low in cost but easily affected by lighting, occlusion, and surface texture differences, and cannot directly acquire 3D information; 3D LiDAR has a large range but low point cloud density and high cost; ToF depth cameras use infrared active imaging, are not affected by ambient light, can output high-density point clouds within a working range of 0.5m to 2m, and have a moderate cost, making them the preferred sensor for close-range perception of cylindrical targets.

[0004] However, existing methods for fitting circles or cylinders based on 3D point clouds still have significant shortcomings: on the one hand, the computational complexity of directly fitting point clouds in 3D space is high, making it difficult to meet the real-time requirements of industry; on the other hand, due to the limitations of the camera installation angle, circular targets observed from non-parallel perspectives will undergo perspective distortion (ellipticization), resulting in a decrease in positioning accuracy.

[0005] Therefore, for positioning methods based on ToF camera 3D point cloud projection, no effective solution has yet been proposed to reduce computational complexity while ensuring positioning accuracy and meeting industrial real-time requirements. Summary of the Invention

[0006] This application provides a method and system for locating circular targets based on ToF camera 3D point cloud projection, which at least solves the problem in related technologies of how to reduce computational complexity and meet industrial real-time requirements while ensuring positioning accuracy.

[0007] In a first aspect, embodiments of this application provide a method for locating a circular target based on a ToF camera's 3D point cloud projection, the method comprising: Acquire a 3D point cloud captured by a ToF camera, and transform the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain a base coordinate system point cloud; The region of interest (ROI) is cropped from the point cloud in the base coordinate system to obtain the ROI point cloud. The point cloud of the region of interest is fitted to obtain a vertical plane. The point cloud of the region of interest is then projected onto the vertical plane to obtain a projected point cloud. The projected point cloud is then rasterized into a two-dimensional image. The two-dimensional image is preprocessed, contour filtered, and circle fitted sequentially to obtain the target circle parameters; Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation.

[0008] In one embodiment, the step of transforming the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain a base coordinate system point cloud includes: The height of the cargo-carrying component of the mobile robot is obtained, and the sensor height is adaptively compensated based on the height of the cargo-carrying component to obtain the target sensor height. Based on the target sensor height, X-axis translation vector, Y-axis translation vector, and rotation matrix, construct a transformation matrix from the camera coordinate system to the mobile robot base coordinate system; The coordinates of each point in the three-dimensional point cloud are transformed using the transformation matrix to obtain the point cloud in the base coordinate system.

[0009] In one embodiment, the step of adaptively compensating the sensor height based on the height of the cargo-carrying component to obtain the target sensor height includes: The height of the target sensor is obtained by adding the height of the cargo-carrying component to the fixed initial Z-axis translation of the mobile robot's base coordinate system.

[0010] In one embodiment, cropping the region of interest (ROI) of the base coordinate system point cloud to obtain the ROI point cloud includes: Obtain the preset polygonal region and the preset height search range; Based on the target sensor height and the height search range, determine the lower and upper limits of the stereo region of interest. Based on the polygonal region, the lower height limit, and the upper height limit, a three-dimensional region of interest is constructed. The stereo region of interest is transformed from the map coordinate system to the robot base coordinate system to obtain the stereo region of interest in the base coordinate system. The point cloud in the base coordinate system is cropped by the three-dimensional region of interest in the base coordinate system, and the points located within the polygonal region are retained to obtain the region of interest point cloud.

[0011] In one embodiment, fitting the point cloud of the region of interest to obtain a vertical plane includes: Projecting the point cloud of the region of interest onto a horizontal plane yields a two-dimensional point set; The two-dimensional point set is fitted with a straight line using a random sampling consensus algorithm to obtain a fitted straight line; the normal vector of the fitted straight line is perpendicular to the vertical Z-axis. Extend the fitted line along the vertical direction to obtain a vertical plane.

[0012] In one embodiment, the step of projecting the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterizing the projected point cloud into a two-dimensional image, includes: Based on the normal vector and plane offset of the vertical plane, the distance from each point in the point cloud of the region of interest to the vertical plane is calculated, and each point is projected onto the vertical plane to obtain the projected point cloud. The minimum and maximum values ​​of the projected point cloud along the horizontal direction of the vertical plane, and the minimum and maximum values ​​along the vertical direction of the vertical plane are obtained, and the boundary range of the projected point cloud is determined according to the preset boundary parameters. Based on the preset rasterization resolution, the projected point cloud within the boundary range is mapped to the pixel coordinate system to generate a two-dimensional image, and the grayscale value of the pixel mapped by the projected point cloud is set to a fixed preset value.

[0013] In one embodiment, the step of sequentially preprocessing, contour filtering, and circle fitting the two-dimensional image to obtain target circle parameters includes: The two-dimensional image is subjected to morphological and Gaussian blur processing to obtain a preprocessed image; The contours in the preprocessed image are extracted, and the extracted contours are filtered in multiple stages according to preset geometric features to obtain candidate contours. The candidate contour is fitted with a circle to obtain circle parameters, and the circle parameters are verified according to a preset radius prior. The circle parameters that pass the verification are used as the target circle parameters.

[0014] In one embodiment, performing morphological processing and Gaussian blurring on the two-dimensional image to obtain a preprocessed image includes: sequentially performing a first morphological opening operation, a first morphological closing operation, Gaussian blurring, a second morphological opening operation, and a second morphological closing operation on the two-dimensional image to obtain the preprocessed image; The step of extracting contours from the preprocessed image and performing multi-level filtering on the extracted contours according to preset geometric features to obtain candidate contours includes: extracting contours from the preprocessed image and sequentially performing first-level filtering of contour area, second-level filtering of contour roundness, and third-level filtering of contour inertia comparison to obtain the candidate contours. The candidate contours are fitted with circles to obtain circle parameters, and the circle parameters are verified according to a preset radius prior. The circle parameters that pass the verification are used as the target circle parameters. This includes: performing circle fitting on each contour in the candidate contours using the minimum circumcircle algorithm to obtain circle parameters with center coordinates and radius. The process of performing circle fitting on the candidate contours to obtain circle parameters, and verifying the circle parameters according to a preset radius prior, and using the verified circle parameters as target circle parameters, includes: performing circle fitting on each contour in the candidate contours using the minimum circumcircle algorithm to obtain the center coordinates and radius as circle parameters, calculating the error between the radius and the preset radius prior value, and using the circle parameters with the error less than a preset threshold as target circle parameters.

[0015] In one embodiment, the step of obtaining the three-dimensional pose of the cylindrical target through inversion calculation based on the circle parameters and the normal vector of the vertical plane includes: Based on the center coordinates in the circle parameters, as well as the boundary parameters and rasterization resolution during rasterization, the center coordinates are inversely calculated into Y and Z coordinates in the world coordinate system. Based on the equation coefficients of the vertical plane, the Y coordinate, and the Z coordinate, calculate the X coordinate in the world coordinate system, and use the Y coordinate, the Z coordinate, and the X coordinate as the three-dimensional coordinates of the center of the circle in the world coordinate system; A target coordinate system is constructed based on the normal vector of the vertical plane, wherein the X-axis direction of the target coordinate system is determined by the direction of the normal vector, the Y-axis direction is obtained by performing a vector product operation between the Z-axis of the world coordinate system and the X-axis, and the Z-axis direction is obtained by performing a vector product operation between the X-axis and the Y-axis. Based on the three-dimensional coordinates of the center of the circle and the directions of each axis of the target coordinate system, a pose transformation matrix of the circular target with respect to the base coordinate system of the mobile robot is constructed, and the three-dimensional pose of the circular target is output.

[0016] Secondly, embodiments of this application provide a circular target localization system based on ToF camera 3D point cloud projection, characterized in that the system is used to execute the above method, and the system includes a base coordinate system point cloud acquisition module, a cropping module, a 2D image module, a target circle parameter module, and a 3D pose module; wherein: The module for obtaining the base coordinate system point cloud is used to transform the 3D point cloud acquired by the ToF camera from the camera coordinate system to the mobile robot base coordinate system to obtain the base coordinate system point cloud. The cropping module is used to crop the region of interest from the base coordinate system point cloud to obtain the region of interest point cloud; The two-dimensional image module is used to fit the point cloud of the region of interest to obtain a vertical plane, project the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterize the projected point cloud into a two-dimensional image. The target circle parameter module is used to perform preprocessing, contour filtering and circle fitting on the two-dimensional image in sequence to obtain the target circle parameters. The three-dimensional pose module is used to obtain the three-dimensional pose of the circular target by inversion calculation based on the target circle parameters and the normal vector of the vertical plane.

[0017] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a circular target localization method based on ToF camera 3D point cloud projection as described in the first aspect above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements a circular target localization method based on ToF camera 3D point cloud projection as described in the first aspect above.

[0019] The circular target localization method and system based on ToF camera 3D point cloud projection provided in this application embodiment has at least the following technical effects.

[0020] The algorithm acquires 3D point clouds from a ToF camera and transforms them from the camera coordinate system to the mobile robot's base coordinate system, obtaining a base coordinate system point cloud. The region of interest (ROI) is then cropped from the base coordinate system point cloud to obtain the region of interest (ROI) point cloud. Cropping removes a large amount of irrelevant point cloud data, reducing the amount of data required for subsequent processing and improving the algorithm's real-time performance. The ROI point cloud is then fitted to a vertical plane, which is projected onto this plane to obtain a projected point cloud. This projected point cloud is then rasterized into a 2D image. By fitting the vertical plane and projecting the 3D point cloud onto it, perspective distortion (ellipticization) caused by the camera's non-parallel viewpoint is eliminated, restoring the projected target geometry to a standard circle and increasing detection robustness. Simultaneously, the 3D problem is reduced to 2D image processing, significantly reducing computational complexity and meeting the real-time requirements of industrial applications. The 2D image is then preprocessed, contour filtered, and fitted to obtain the target circle parameters, improving recognition accuracy and anti-interference capabilities. Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation, realizing the precise three-dimensional positioning of the circular target and meeting the requirements of industrial material handling operations for insertion point and centering accuracy. This addresses the problem in related technologies of how to reduce computational complexity while ensuring positioning accuracy and meeting industrial real-time requirements.

[0021] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a circular target localization method based on ToF camera 3D point cloud projection; Figure 2 This is a schematic diagram illustrating three-dimensional point cloud data and various coordinate systems according to an exemplary embodiment; Figure 3 This is a schematic diagram of a region of interest point cloud according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating a fitted straight line according to an exemplary embodiment; Figure 5 This is a schematic diagram of a two-dimensional grayscale image according to an exemplary embodiment; Figure 6 This is a schematic diagram illustrating circle fitting according to an exemplary embodiment; Figure 7 This is a system structure block diagram of a circular target localization system based on ToF camera 3D point cloud projection, according to an exemplary embodiment. Figure 8 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0024] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0025] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0026] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0027] In this document, it should be understood that the terms used may be technical means used to implement part of the present invention or other summary technical terms. For example, the terms may include: Mobile robot base coordinate system: A three-dimensional Cartesian coordinate system fixed on the mobile robot body. It is usually based on a reference point of the robot (such as the geometric center of the chassis, the center of the rear axle, or the center of the forklift body) as the origin. It is used to describe the robot's own position and the spatial relationship between the surrounding environment and the robot.

[0028] Base coordinate system point cloud: The original 3D point cloud acquired by the ToF camera is transformed to obtain point cloud data in the mobile robot's base coordinate system. The coordinates of each point in this point cloud directly reflect the target's 3D position in the mobile robot's base coordinate system.

[0029] Vertical plane: A plane perpendicular to the horizontal plane (ground). In this application, the plane is obtained by fitting a straight line to the clipped point cloud (the point cloud is projected onto the horizontal plane and a straight line is fitted) and extending it along the vertical direction. This plane is used for subsequent projection to eliminate perspective distortion.

[0030] Cargo-carrying components: Liftable mechanisms on mobile robots used to carry, pick up, or transport goods. Examples include forklift forks, AGV carrying platforms, or stacker crane picking devices. Their height information (such as fork height) is used for adaptive compensation of sensor height.

[0031] Height search range: A preset height range parameter, used to expand vertically and horizontally to form a height range [hmin, hmax] centered on the height of the cargo-bearing component (or the height after sensor compensation). Point clouds within this height range are retained, while point clouds outside this range are cropped.

[0032] In a first aspect, embodiments of this application provide a method for locating circular targets based on ToF camera 3D point cloud projection. Figure 1 This is a flowchart of a circular target localization method based on ToF camera 3D point cloud projection, such as... Figure 1 As shown, the method includes: Step S101: Obtain the 3D point cloud captured by the ToF camera, and transform the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain the base coordinate system point cloud.

[0033] Step S102: Crop the region of interest (ROI) of the point cloud in the base coordinate system to obtain the ROI point cloud.

[0034] Step S103: Fit the point cloud of the region of interest to obtain a vertical plane, project the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterize the projected point cloud into a two-dimensional image.

[0035] Step S104: Perform preprocessing, contour filtering, and circle fitting on the two-dimensional image in sequence to obtain the target circle parameters.

[0036] Step S105: Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation.

[0037] In summary, this application provides a method for circular target localization based on ToF camera 3D point cloud projection. It acquires a 3D point cloud from a ToF camera, transforms the 3D point cloud from the camera coordinate system to the mobile robot's base coordinate system, obtaining a base coordinate system point cloud. The base coordinate system point cloud is then cropped to obtain a region of interest (ROI) point cloud. Cropping removes a large amount of irrelevant point cloud data, reducing the amount of data processed subsequently and improving the algorithm's real-time performance. The ROI point cloud is fitted to obtain a vertical plane, which is then projected onto the vertical plane to obtain a projected point cloud. This projected point cloud is then rasterized into a 2D image. By fitting the vertical plane and projecting the 3D point cloud onto it, perspective distortion (ellipticization) caused by the camera's viewpoint is eliminated, restoring the projected target geometry to a standard circle and increasing detection robustness. Simultaneously, the 3D problem is reduced to 2D image processing, significantly reducing computational complexity and meeting the real-time requirements of industrial applications. The 2D image is then preprocessed, contour filtered, and fitted to obtain the target circle parameters, improving the accuracy of recognition and its anti-interference capability. Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation, realizing the precise three-dimensional positioning of the circular target and meeting the requirements of industrial material handling operations for insertion point and centering accuracy. This addresses the problem in related technologies of how to reduce computational complexity while ensuring positioning accuracy and meeting industrial real-time requirements.

[0038] In one embodiment, step S101 involves acquiring a 3D point cloud from a ToF camera and transforming the 3D point cloud from the camera coordinate system to the mobile robot's base coordinate system to obtain a base coordinate system point cloud. Specifically, this includes the following steps: Step S1011: Obtain the height of the cargo-carrying component of the mobile robot, and adaptively compensate the sensor height based on the cargo-carrying component height to obtain the target sensor height. Specifically, this includes adding the cargo-carrying component height to the fixed initial Z-axis translation of the mobile robot's base coordinate system to obtain the target sensor height.

[0039] Step S1012: Based on the target sensor height, X-axis translation vector, Y-axis translation vector, and rotation matrix, construct the transformation matrix from the camera coordinate system to the mobile robot base coordinate system.

[0040] Step S1013: Perform coordinate transformation on each point in the 3D point cloud using a transformation matrix to obtain the point cloud in the base coordinate system.

[0041] Optionally, Figure 2 This is a schematic diagram illustrating three-dimensional point cloud data and various coordinate systems according to an exemplary embodiment, such as... Figure 2 As shown, firstly, the three-dimensional point cloud data from the ToF camera is received, and then the point cloud is transformed from the camera coordinate system to the robot base coordinate system through coordinate transformation.

[0042] Let the original point cloud be ,in A 3D point in the camera coordinate system.

[0043] Subsequently, the height information of the cargo-bearing components (forklift forks) is obtained, and the sensor height parameters are dynamically adjusted, which can be expressed by the following formula:

[0044] in, This refers to the height of the forklift forks. This represents the fixed initial Z-axis translation of the sensor relative to the base coordinate system.

[0045] Next, the highly compensated transformation matrix is ​​constructed. It can be expressed by the following formula:

[0046] in, For rotation matrix, The translation vector after height compensation. Represents the translation vector along the X-axis. This represents the translation vector along the Y-axis.

[0047] Base coordinate system point cloud It can be determined using the following formula:

[0048] Step S101 introduces adaptive compensation based on the height of the cargo-bearing component, dynamically eliminating the impact of sensor height changes caused by the lifting and lowering of the cargo-bearing component on the point cloud coordinates. This ensures that regardless of the fork's height, the transformed base coordinate system point cloud maintains an accurate spatial mapping, significantly improving the accuracy of subsequent positioning. Simultaneously, using pre-calibrated translation vectors (x-axis and y-axis) and rotation matrices for coordinate transformation eliminates the need for repeated calculations of the relative pose between the sensor and the robot, reducing the computational burden of real-time processing and ensuring the system's real-time performance.

[0049] In one embodiment, step S102 involves cropping the region of interest (ROI) of the base coordinate system point cloud to obtain the ROI point cloud. This specifically includes the following steps: Step S1021: Obtain the preset polygonal region and the preset height search range; Step S1022: Determine the lower and upper limits of the stereo region of interest based on the target sensor height and the height search range; Step S1023: Construct the 3D region of interest based on the polygonal region, the lower height limit, and the upper height limit; Step S1024: Transform the stereo region of interest from the map coordinate system to the mobile robot base coordinate system to obtain the stereo region of interest in the base coordinate system; Step S1025: Crop the point cloud of the base coordinate system using the 3D region of interest in the base coordinate system, retaining the points located within the polygonal region, to obtain the point cloud of the region of interest.

[0050] Optionally, Figure 3 This is a schematic diagram of a region of interest point cloud according to an exemplary embodiment, such as... Figure 3 As shown, firstly, based on the target sensor height obtained in step S101 and the preset height search range, the lower and upper limits of the stereo region of interest are determined, which can be expressed by the following formula:

[0051]

[0052] In the formula, Indicates sensor height, This indicates the height search range parameter.

[0053] Next, a three-dimensional region of interest is constructed by combining the horizontal boundary of the polygonal region with its lower and upper height limits. This region of interest is then transformed from the map coordinate system to the mobile robot's base coordinate system to obtain the region of interest in the base coordinate system. Finally, the point cloud in the base coordinate system is cropped using the region of interest in the base coordinate system, retaining only the points located within the polygonal region and whose height is between the upper and lower limits, thus forming the region of interest point cloud.

[0054] Specifically, let the ROI polygon region be... Cropped point clouds Defined as:

[0055] in, Point Located within a polygonal area, Here is the Z-coordinate (height) of the point. The cropping process removes irrelevant point cloud information and extracts point cloud data from the region of interest.

[0056] Step S102 constructs a three-dimensional region of interest by combining polygonal regions and height ranges, which can accurately focus on the spatial range in which circular targets may appear, significantly eliminating background and irrelevant point clouds, effectively reducing the amount of data for subsequent processing, and improving the real-time performance of the algorithm.

[0057] In one embodiment, step S103 involves fitting the point cloud of the region of interest to obtain a vertical plane. Specifically, this includes the following steps: Step S103a1 projects the point cloud of the region of interest onto a horizontal plane to obtain a two-dimensional point set; Step S103a2 uses a random sampling consensus algorithm to fit a straight line to the two-dimensional point set, and obtains a fitted straight line; wherein the normal vector of the fitted straight line is perpendicular to the vertical Z-axis. Step S103a3 extends the fitted line along the vertical direction to obtain a vertical plane.

[0058] Optionally, Figure 4 This is a schematic diagram illustrating a fitted straight line according to an exemplary embodiment, such as... Figure 4 As shown, all points in the point cloud of the region of interest obtained in step S102 are vertically projected onto the horizontal plane, ignoring the height information of each point, to obtain a two-dimensional point set on the horizontal plane. It can be expressed by the following formula:

[0059] Then, the Random Sample Consensus algorithm is used to fit a straight line to the points in the two-dimensional point set, and the fitted line is obtained, which can be expressed by the following formula:

[0060] Where A, B, C, and D are the coefficients of the linear equation (C=0), and the normal vector is... Perpendicular to the Z-axis, it represents a vertical plane.

[0061] The fitted line lies in the horizontal plane, and its normal vector is perpendicular to the vertical Z-axis. Finally, the fitted line is extended along the vertical direction (Z-axis direction) to form a vertical plane perpendicular to the horizontal plane. This vertical plane is the direction information of the plane where the circular target is located, which is used for subsequent point cloud projection.

[0062] Steps S103a1-S103a3 project the 3D point cloud onto a horizontal plane for line fitting, reducing the complex spatial plane fitting problem to a 2D line fitting problem, significantly reducing computational complexity and improving the algorithm's real-time performance. The final constructed vertical plane provides an ideal projection surface for subsequent point cloud projection, eliminating perspective distortion caused by non-parallel camera views, and restoring the projected circular target to a standard circle, thus ensuring the accuracy of subsequent circle detection and pose calculation.

[0063] In one embodiment, step S103 involves fitting the point cloud of the region of interest to obtain a vertical plane, projecting the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and then rasterizing the projected point cloud into a two-dimensional image. Specifically, this includes the following steps: Step S103b1: Based on the normal vector and plane offset of the vertical plane, calculate the distance from each point in the point cloud of the region of interest to the vertical plane, and project each point onto the vertical plane to obtain the projected point cloud.

[0064] Step S103b2: Obtain the minimum and maximum values ​​of the projected point cloud in the horizontal direction along the vertical plane, and the minimum and maximum values ​​in the vertical direction along the vertical plane, and determine the boundary range of the projected point cloud according to the preset boundary parameters. Step S103b3: Based on the preset rasterization resolution, the projected point cloud within the boundary range is mapped to the pixel coordinate system to generate a two-dimensional image, and the grayscale value of the pixel mapped by the projected point cloud is set to a fixed preset value.

[0065] Optionally, firstly, based on the normal vector and plane offset of the fitted vertical plane, the distance from each point in the point cloud of the region of interest to the vertical plane is calculated, which can be expressed by the following formula:

[0066] In the formula, Let be the plane normal vector, and D be the plane offset.

[0067] Then, for Each point in Move the point by the unit normal vector along the plane and project it onto the vertical plane to obtain the projected point. It can be expressed by the following formula:

[0068] In the formula, Let be the unit normal vector, and d be the distance from a point in the point cloud of the region of interest to the vertical plane. Points in the point cloud representing the region of interest. The projected point cloud is... .

[0069] Next, the minimum and maximum values ​​of the projected point cloud in the horizontal direction (i.e., along the transverse direction of the plane) and in the vertical direction (i.e., the height direction) are obtained. Based on the preset boundary expansion margin (0.1 meters horizontally and 0.05 meters vertically), i.e., boundary parameters, the boundary range of the projected point cloud is determined. This can be expressed by the following formula:

[0070] in, , Midpoint of the projected point cloud The Y and Z coordinate components, = 0.1m, = 0.05m is the boundary expansion margin.

[0071] Figure 5 This is a schematic diagram of a two-dimensional grayscale image according to an exemplary embodiment, such as... Figure 5 As shown, finally, according to the preset rasterization resolution (meters / pixel), the projected point cloud within the boundary range is mapped to the pixel coordinate system to generate a two-dimensional grayscale image. The grayscale value of the pixel mapped by the projected point cloud is set to a fixed preset value, which is a fixed foreground value (e.g., 255). Pixels without projection points are set to a background value (0). The cropped point cloud... Project it onto the fitted vertical plane.

[0072] Specifically, mapping the projected point cloud within the boundary range to the pixel coordinate system can be expressed by the following formula:

[0073] Where r is the rasterization resolution (meters per pixel).

[0074] Rasterized image generation: Define a rasterized image I with size I. :

[0075] Image pixel values:

[0076] Steps S103b1-S103b4 project the 3D point cloud onto the fitted vertical plane. Regardless of the camera's viewing angle, the cylindrical target's outline, which was originally elliptical due to perspective distortion, is restored to a standard circle after projection, thus eliminating the impact of perspective distortion on detection accuracy. Discretizing the continuous 3D point cloud into a 2D binary image significantly reduces the data dimensionality, laying the foundation for subsequent fast and stable image processing (such as morphological operations, contour extraction, and circle fitting).

[0077] In one embodiment, step S104 involves preprocessing the two-dimensional image, performing contour filtering, and circle fitting sequentially to obtain the target circle parameters. Specifically, this includes the following steps: Step S1041: Perform morphological and Gaussian blur processing on the two-dimensional image to obtain a preprocessed image. Specifically, this includes performing a first morphological opening operation, a first morphological closing operation, Gaussian blur, a second morphological opening operation, and a second morphological closing operation on the two-dimensional image in sequence to obtain the preprocessed image.

[0078] Optionally, firstly, a first morphological opening operation is performed on the generated two-dimensional grayscale image using an elliptical structuring element of size 3×3 to eliminate minor noise in the image.

[0079] Specifically, morphological processing is performed on the rasterized image I.

[0080] First morphological opening operation: Using elliptical structural elements (Size 3 × 3):

[0081] in, For erosion calculation, This is for the expansion operation. The opening operation eliminates small noise.

[0082] Then, the same structuring element is used to perform the first morphological closing operation to fill the small holes inside the target region; next, a Gaussian blur is applied to the image using a Gaussian kernel with a standard deviation of 1.5 to smooth the edges and reduce noise.

[0083] Specifically, the first morphological closing operation:

[0084] Fill small holes using closing operations.

[0085] Gaussian blur:

[0086] in, N ( p (pixels) pThe neighborhood, For Gaussian kernel function, = 1.5.

[0087] Then, a second morphological opening operation is performed using an elliptical structuring element with a size of 11×11 to further eliminate noise and irrelevant details in larger areas. Specifically, the second morphological opening operation: Using elliptical structural elements (Size 11×11):

[0088] Finally, a second morphological closing operation is performed to enhance the connectivity of the target region and fill residual holes, thereby obtaining the preprocessed image.

[0089] Specifically, the second morphological closing operation:

[0090] Enhance target features and fill holes. Final preprocessed image. = .

[0091] Step S1042: Extract contours from the preprocessed image and perform multi-level filtering on the extracted contours according to preset geometric features to obtain candidate contours. Specifically, this includes: extracting contours from the preprocessed image, and sequentially performing first-level filtering on contour area, second-level filtering on contour roundness, and third-level filtering on contour inertia comparison to obtain candidate contours.

[0092] Optionally, firstly, all contours are extracted from the preprocessed image to obtain an initial contour set; then, each contour is subjected to three levels of geometric feature filtering: the first level filters based on contour area, retaining only contours with areas between a preset minimum and maximum area threshold; the second level filters based on contour roundness, calculating the roundness of each contour and retaining only contours with roundness between a preset minimum and maximum roundness; the third level filters based on contour inertia ratio, calculating the major and minor axes of the contour using the normalized central moments of the contour to obtain the inertia ratio (minor axis / major axis), retaining only contours with inertia ratios between a preset minimum and maximum inertia ratio; after three levels of filtering, a candidate contour set is obtained; finally, each contour in the candidate contours is Gaussian smoothed to reduce the influence of noise points on the contours on subsequent circle fitting, resulting in smoothed candidate contours.

[0093] Specifically, firstly, from Extract contour sets Then, for each contour Three-stage filtration is performed. The specific filtration process is as follows: Primary filtration - area filtration: Retention conditions:

[0094] in, For the outline area, , This is the area threshold.

[0095] Filtered contour set .

[0096] Secondary filtration - circularity filtration: Circularity:

[0097] Retention conditions:

[0098] Where P is the perimeter of the profile and C is the circularity (for an ideal circle, C=1).

[0099] Filtered contour set .

[0100] Three-stage filtration - inertia ratio filtration: Calculate the major axis of the profile based on Hu's moment. and short axis

[0101] Calculate the major and minor axes:

[0102]

[0103]

[0104] in, , , The normalized central moments of the contour Inertia ratio:

[0105] Retention conditions:

[0106] Filtered contour set .

[0107] Finally, Gaussian smoothing is applied to the filtered contours to reduce the impact of contour noise on circle fitting. Perform Gaussian smoothing to obtain the smoothed contour set. .

[0108] Define the outline Smoothed contours :

[0109] Gaussian weights:

[0110] in = K / 2, K =3 is the kernel radius.

[0111] Step S1043: Perform circle fitting on the candidate contours to obtain circle parameters, and verify the circle parameters according to a preset radius prior. The circle parameters that pass the verification are used as the target circle parameters. Specifically, this includes: performing circle fitting on each contour in the candidate contours using the minimum circumcircle algorithm to obtain the center coordinates and radius as circle parameters; calculating the error between the radius and the preset radius prior value; and using the circle parameters with an error less than a preset threshold as the target circle parameters.

[0112] Optionally, firstly, for each contour in the candidate contour set in step S1042, the minimum circumcircle algorithm is used to calculate the smallest circle that can completely contain the contour, thereby obtaining the center pixel coordinates and pixel radius of the corresponding contour as circle parameters; then, the pixel radius is converted into a physical radius according to a preset rasterization resolution, and the physical diameter is calculated; next, the physical diameter is compared with a preset prior diameter value, and the absolute error between the two is calculated; if the error is less than or equal to a preset error threshold, the circle parameter is determined to pass verification and is used as the target circle parameter; if the error is greater than the threshold, the circle parameter is determined to be invalid and discarded. Finally, all verified circle parameters constitute the target circle parameter set, which is used for subsequent 3D pose calculation.

[0113] Specifically, Figure 6 This is a schematic diagram illustrating circle fitting according to an exemplary embodiment, such as... Figure 6 As shown, for the smoothed contour set The minimum circumcircle algorithm is used to fit the circle. For the contour... Calculate its smallest enclosing circle to obtain the center of the circle ( ) and radius r:

[0114] Collection of circle detection results: Diameter verification:

[0115]

[0116] If error e > e threshold If so, the detection fails.

[0117] Step S104 improves the anti-interference capability and accuracy of circular target detection, avoids false detections caused by environmental noise, partial occlusion or similar-shaped objects, and provides reliable target circle information for subsequent three-dimensional pose calculation.

[0118] In one embodiment, step S105 involves obtaining the three-dimensional pose of the circular target through inversion calculation based on the target circle parameters and the normal vector of the vertical plane. Specifically, this includes the following steps: Step S1051: Based on the center coordinates in the circle parameters, the boundary parameters during rasterization, and the rasterization resolution, the center coordinates are inversely calculated into the Y and Z coordinates in the world coordinate system. Step S1052: Calculate the X coordinate in the world coordinate system based on the equation coefficients of the vertical plane and the Y and Z coordinates, and use the Y, Z and X coordinates as the three-dimensional coordinates of the center of the circle in the world coordinate system. Step S1053: Construct a target coordinate system based on the normal vector of the vertical plane. The X-axis direction of the target coordinate system is determined by the direction of the normal vector, the Y-axis direction is obtained by performing a vector product operation between the Z-axis and the X-axis of the world coordinate system, and the Z-axis direction is obtained by performing a vector product operation between the X-axis and the Y-axis. Step S1054: Based on the three-dimensional coordinates of the center of the circle and the directions of each axis of the target coordinate system, construct the pose transformation matrix of the circular target with respect to the base coordinate system of the mobile robot, and output the three-dimensional pose of the circular target.

[0119] Optionally, the detected circular center is back-projected from pixel coordinates to three-dimensional space, and a circular coordinate system is modeled by combining the planar normal vectors to calculate the pose (x, y, z, yaw) of the circular target relative to the robot's base coordinate system.

[0120] First, the transformation from pixel coordinates to world coordinates:

[0121]

[0122] Calculate the x-coordinate based on the plane equation:

[0123] Secondly, the circular coordinate system is constructed: Construct the target coordinate system based on the plane normal vector:

[0124]

[0125]

[0126] in, It is the unit vector of the Z-axis in the world coordinate system.

[0127] Pose transformation matrix:

[0128] in, The coordinates of the center of the circle are 3D coordinates.

[0129] Output the pose of the circular target relative to the robot's base coordinate system. .

[0130] Step S105 achieves high-precision inverse mapping from a two-dimensional image to three-dimensional space by inverting the coordinates of the detected center pixel in the image to the true coordinates in three-dimensional space and combining this with the vertical plane equation to accurately solve for the depth direction (X-axis) coordinates. Using the normal vector of the vertical plane to construct the target coordinate system, the radial and axial directions of the circular target can be naturally described, meeting the precise requirements of industrial handling operations for insertion points and centering directions.

[0131] In summary, this application provides a method for circular target localization based on ToF camera 3D point cloud projection. It acquires a 3D point cloud from a ToF camera, transforms the 3D point cloud from the camera coordinate system to the mobile robot's base coordinate system, obtaining a base coordinate system point cloud. The base coordinate system point cloud is then cropped to obtain a region of interest (ROI) point cloud. Cropping removes a large amount of irrelevant point cloud data, reducing the amount of data processed subsequently and improving the algorithm's real-time performance. The ROI point cloud is fitted to obtain a vertical plane, which is then projected onto the vertical plane to obtain a projected point cloud. This projected point cloud is then rasterized into a 2D image. By fitting the vertical plane and projecting the 3D point cloud onto it, perspective distortion (ellipticization) caused by the camera's viewpoint is eliminated, restoring the projected target geometry to a standard circle and increasing detection robustness. Simultaneously, the 3D problem is reduced to 2D image processing, significantly reducing computational complexity and meeting the real-time requirements of industrial applications. The 2D image is then preprocessed, contour filtered, and fitted to obtain the target circle parameters, improving the accuracy of recognition and its anti-interference capability. Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation, realizing the precise three-dimensional positioning of the circular target and meeting the requirements of industrial material handling operations for insertion point and centering accuracy. This addresses the problem in related technologies of how to reduce computational complexity while ensuring positioning accuracy and meeting industrial real-time requirements.

[0132] Secondly, embodiments of this application provide a circular target positioning system based on ToF camera 3D point cloud projection. Figure 7 This is a system structure block diagram illustrating a circular target localization system based on ToF camera 3D point cloud projection, according to an exemplary embodiment. For example... Figure 7 As shown, the system includes a point cloud acquisition module 710, a cropping module 720, a 2D image module 730, a target circle parameter module 740, and a 3D pose module 750; wherein: The base coordinate system point cloud acquisition module 710 is used to acquire 3D point clouds from the ToF camera and transform the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain the base coordinate system point cloud. The clipping module 720 is used to clip the region of interest from the point cloud in the base coordinate system to obtain the region of interest point cloud. The two-dimensional image module 730 is used to fit the point cloud of the region of interest to obtain a vertical plane, project the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterize the projected point cloud into a two-dimensional image. The target circle parameter module 740 is used to perform preprocessing, contour filtering and circle fitting on the two-dimensional image in sequence to obtain the target circle parameters. The 3D pose module 750 is used to obtain the 3D pose of a circular target through inversion calculation based on the target circle parameters and the normal vector of the vertical plane.

[0133] In summary, the circular target positioning system based on ToF camera 3D point cloud projection provided in this application solves the problem in related technologies of how to reduce computational complexity and meet industrial real-time requirements while ensuring positioning accuracy through the synergistic effect of the base coordinate system point cloud module 710, the cropping module 720, the 2D image module 730, the target circle parameter module 740, and the 3D pose module 750.

[0134] It should be noted that the circular target positioning system based on ToF camera 3D point cloud projection provided in this embodiment is used to implement the above-described embodiments, and details already described will not be repeated. As used above, the terms "module," "unit," "subunit," etc., can refer to combinations of software and / or hardware that perform predetermined functions. Although the apparatus described in the above embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0135] Thirdly, embodiments of this application provide an electronic device, Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. (e.g.) Figure 8 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0136] Specifically, the processor 81 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0137] The memory 82 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include removable or non-removable (or fixed) media. Where appropriate, the memory 82 may be internal or external to a data processing device. In a particular embodiment, the memory 82 is non-volatile memory. In a particular embodiment, the memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0138] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.

[0139] The processor 81 reads and executes the computer program instructions stored in the memory 82 to implement any of the circular target localization methods based on ToF camera 3D point cloud projection in the above embodiments.

[0140] In one embodiment, a circular target positioning device based on ToF camera 3D point cloud projection may further include a communication interface 83 and a bus 80. Wherein, as Figure 8 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0141] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0142] Bus 80 includes hardware, software, or both, that couples together components of a circular target positioning device based on a ToF camera's 3D point cloud projection. Bus 80 includes, but is not limited to, at least one of the following: Data Bus, Address Bus, Control Bus, Expansion Bus, and Local Bus. For example, and not as a limitation, bus 80 may include Accelerated GraphicsPort (AGP) or other graphics buses, Extended Industry Standard Architecture (EISA) buses, Front Side Bus (FSB), HyperTransport (HT) interconnects, Industry Standard Architecture (ISA) buses, InfiniBand interconnects, Low Pin Count (LPC) buses, memory buses, Micro Channel Architecture (MCA) buses, Peripheral Component Interconnect (PCI) buses, PCI-Express (PCI-X) buses, Serial Advanced Technology Attachment (SATA) buses, Video Electronics Standards Association Local Bus (VLB) buses, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 80 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0143] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the circular target localization method based on ToF camera 3D point cloud projection provided in the first aspect.

[0144] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0145] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps of implementing the circular target localization method based on ToF camera 3D point cloud projection provided in the first aspect.

[0146] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for locating a circular target based on ToF camera 3D point cloud projection, characterized in that, The method includes: Acquire a 3D point cloud captured by a ToF camera, and transform the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain a base coordinate system point cloud; The region of interest is clipped from the point cloud in the base coordinate system to obtain the region of interest points; The point cloud of the region of interest is fitted to obtain a vertical plane. The point cloud of the region of interest is then projected onto the vertical plane to obtain a projected point cloud. The projected point cloud is then rasterized into a two-dimensional image. The two-dimensional image is preprocessed, contour filtered, and circle fitted sequentially to obtain the target circle parameters; Based on the target circle parameters and the normal vector of the vertical plane, the three-dimensional pose of the circular target is obtained through inversion calculation.

2. The circular target localization method based on ToF camera 3D point cloud projection according to claim 1, characterized in that, The step of transforming the 3D point cloud from the camera coordinate system to the mobile robot base coordinate system to obtain the base coordinate system point cloud includes: The height of the cargo-carrying component of the mobile robot is obtained, and the sensor height is adaptively compensated based on the height of the cargo-carrying component to obtain the target sensor height. Based on the target sensor height, X-axis translation vector, Y-axis translation vector, and rotation matrix, construct a transformation matrix from the camera coordinate system to the mobile robot base coordinate system; The coordinates of each point in the three-dimensional point cloud are transformed using the transformation matrix to obtain the point cloud in the base coordinate system.

3. The circular target localization method based on ToF camera 3D point cloud projection according to claim 2, characterized in that, The step of adaptively compensating for the sensor height based on the height of the cargo-bearing component to obtain the target sensor height includes: The height of the target sensor is obtained by adding the height of the cargo-carrying component to the fixed initial Z-axis translation of the mobile robot's base coordinate system.

4. The circular target localization method based on ToF camera 3D point cloud projection according to claim 2, characterized in that, The step of cropping the region of interest (ROI) of the point cloud in the base coordinate system to obtain the ROI point cloud includes: Obtain the preset polygonal region and the preset height search range; Based on the target sensor height and the height search range, determine the lower and upper limits of the stereo region of interest. Based on the polygonal region, the lower height limit, and the upper height limit, a three-dimensional region of interest is constructed. The stereo region of interest is transformed from the map coordinate system to the mobile robot's base coordinate system to obtain the stereo region of interest in the base coordinate system. The point cloud in the base coordinate system is cropped by the three-dimensional region of interest in the base coordinate system, and the points located within the polygonal region are retained to obtain the region of interest point cloud.

5. A method for locating a circular target based on ToF camera 3D point cloud projection according to claim 1, characterized in that, The process of fitting the point cloud of the region of interest to obtain a vertical plane includes: Projecting the point cloud of the region of interest onto a horizontal plane yields a two-dimensional point set; The two-dimensional point set is fitted with a straight line using a random sampling consensus algorithm to obtain a fitted straight line; wherein the normal vector of the fitted straight line is perpendicular to the vertical Z-axis. Extend the fitted line along the vertical direction to obtain a vertical plane.

6. The circular target localization method based on ToF camera 3D point cloud projection according to claim 1, characterized in that, The step of projecting the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterizing the projected point cloud into a two-dimensional image, includes: Based on the normal vector and plane offset of the vertical plane, the distance from each point in the point cloud of the region of interest to the vertical plane is calculated, and each point is projected onto the vertical plane to obtain the projected point cloud. The minimum and maximum values ​​of the projected point cloud along the horizontal direction of the vertical plane, and the minimum and maximum values ​​along the vertical direction of the vertical plane are obtained, and the boundary range of the projected point cloud is determined according to the preset boundary parameters. Based on the preset rasterization resolution, the projected point cloud within the boundary range is mapped to the pixel coordinate system to generate a two-dimensional image, and the grayscale value of the pixel mapped by the projected point cloud is set to a fixed preset value.

7. A method for locating a circular target based on ToF camera 3D point cloud projection according to claim 1, characterized in that, The step of preprocessing, contour filtering, and circle fitting of the two-dimensional image to obtain the target circle parameters includes: The two-dimensional image is subjected to morphological and Gaussian blur processing to obtain a preprocessed image; The contours in the preprocessed image are extracted, and the extracted contours are filtered in multiple stages according to preset geometric features to obtain candidate contours. The candidate contour is fitted with a circle to obtain circle parameters, and the circle parameters are verified according to a preset radius prior. The circle parameters that pass the verification are used as the target circle parameters.

8. A method for locating a circular target based on ToF camera 3D point cloud projection according to claim 7, characterized in that, The step of performing morphological processing and Gaussian blurring on the two-dimensional image to obtain a preprocessed image includes: sequentially performing a first morphological opening operation, a first morphological closing operation, Gaussian blurring, a second morphological opening operation, and a second morphological closing operation on the two-dimensional image to obtain the preprocessed image; The step of extracting contours from the preprocessed image and performing multi-level filtering on the extracted contours according to preset geometric features to obtain candidate contours includes: extracting contours from the preprocessed image and sequentially performing first-level filtering of contour area, second-level filtering of contour roundness, and third-level filtering of contour inertia comparison to obtain the candidate contours. The candidate contours are fitted with circles to obtain circle parameters, and the circle parameters are verified according to a preset radius prior. The circle parameters that pass the verification are used as the target circle parameters. This includes: performing circle fitting on each contour in the candidate contours using the minimum circumcircle algorithm to obtain circle parameters with center coordinates and radius. The process of performing circle fitting on the candidate contours to obtain circle parameters, and verifying the circle parameters according to a preset radius prior, and using the verified circle parameters as target circle parameters, includes: performing circle fitting on each contour in the candidate contours using the minimum circumcircle algorithm to obtain the center coordinates and radius as circle parameters, calculating the error between the radius and the preset radius prior value, and using the circle parameters with the error less than a preset threshold as target circle parameters.

9. A method for locating a circular target based on ToF camera 3D point cloud projection according to claim 1, characterized in that, The process of obtaining the three-dimensional pose of the cylindrical target through inversion calculation based on the circular parameters and the normal vector of the vertical plane includes: Based on the center coordinates in the circle parameters, as well as the boundary parameters and rasterization resolution during rasterization, the center coordinates are inversely calculated into Y and Z coordinates in the world coordinate system. Based on the equation coefficients of the vertical plane, the Y coordinate, and the Z coordinate, calculate the X coordinate in the world coordinate system, and use the Y coordinate, the Z coordinate, and the X coordinate as the three-dimensional coordinates of the center of the circle in the world coordinate system; A target coordinate system is constructed based on the normal vector of the vertical plane, wherein the X-axis direction of the target coordinate system is determined by the direction of the normal vector, the Y-axis direction is obtained by performing a vector product operation between the Z-axis of the world coordinate system and the X-axis, and the Z-axis direction is obtained by performing a vector product operation between the X-axis and the Y-axis. Based on the three-dimensional coordinates of the center of the circle and the directions of each axis of the target coordinate system, a pose transformation matrix of the circular target with respect to the base coordinate system of the mobile robot is constructed, and the three-dimensional pose of the circular target is output.

10. A circular target localization system based on ToF camera 3D point cloud projection, characterized in that, The system is used to perform the method according to any one of claims 1 to 9, and the system includes a base coordinate system point cloud acquisition module, a cropping module, a two-dimensional image module, a target circle parameter module, and a three-dimensional pose module; wherein: The module for obtaining the base coordinate system point cloud is used to transform the 3D point cloud acquired by the ToF camera from the camera coordinate system to the mobile robot base coordinate system to obtain the base coordinate system point cloud. The cropping module is used to crop the region of interest from the base coordinate system point cloud to obtain the region of interest point cloud; The two-dimensional image module is used to fit the point cloud of the region of interest to obtain a vertical plane, project the point cloud of the region of interest onto the vertical plane to obtain a projected point cloud, and rasterize the projected point cloud into a two-dimensional image. The target circle parameter module is used to perform preprocessing, contour filtering and circle fitting on the two-dimensional image in sequence to obtain the target circle parameters. The three-dimensional pose module is used to obtain the three-dimensional pose of the circular target by inversion calculation based on the target circle parameters and the normal vector of the vertical plane.

Images