A point cloud-based object pose recognition method

Abstract

This invention discloses a point cloud-based object pose recognition method, belonging to the technical field of 3D visual recognition. The method involves acquiring multi-view images of a target scene and determining the extrinsic parameters of each image acquisition unit in a unified coordinate system; generating a 3D point cloud of the target scene based on the multi-view images, and performing instance segmentation to obtain target object image instances from each viewpoint; establishing a projection mapping relationship to map the image instance information to the corresponding viewpoints in the 3D point cloud, obtaining subsets of the point cloud from each viewpoint; based on the spatial consistency between point cloud subsets from different viewpoints, performing cross-viewpoint association and fusion on point clouds belonging to the same target object, aggregating them to obtain instance-level point clouds of each target object; and obtaining spatial feature information of the target object based on the instance-level point clouds. This invention, through targeted technical design, achieves high-precision and efficient object pose recognition, adapting to the mechanized retrieval needs of large-scale warehousing scenarios, with significant overall technical effects.

Description

Technical Field

[0001] This invention belongs to the technical field of three-dimensional visual recognition, and in particular relates to a method for object pose recognition based on point clouds. Background Technology

[0002] As a core raw material in industries such as steel and machinery manufacturing, wire rod is often stored in large warehouses in a stacked manner. The mechanized retrieval operations in large-scale wire rod storage require precise acquisition of information such as the spatial pose and stacking level of the wire rod using point cloud technology. However, existing point cloud-based wire rod pose recognition technology has many technical shortcomings in practical applications and is difficult to meet industrial needs. First, the adaptability of multi-camera extrinsic calibration is poor. In installation scenarios at heights of more than ten meters, existing methods cannot achieve high-precision calibration in the same coordinate system over long distances, and there is a lack of effective means to correct dynamic errors caused by camera shaking. At the same time, traditional static cameras do not support dynamic updates of extrinsic parameters after rotation, which limits the acquisition of large-scale point clouds. Secondly, the 3D reconstructed point cloud effect is not good. Due to the influence of the length of the freight car and the stacking shape of the wire rod, the point cloud reconstructed from a single view has problems such as missing information and a lot of noise. Moreover, the point cloud optimization method is limited, and it is difficult to effectively remove non-wire rod point clouds and noise, which affects the efficiency and accuracy of subsequent processing. Third, the stability of point cloud instance segmentation is insufficient. Direct processing of 3D point clouds is prone to missed detection and misjudgment due to uneven density and missing regions. It is impossible to accurately separate the instance-level point cloud of a single bar, which restricts the accurate recognition of pose. Fourth, pose determination is cost-effective and has limited functionality. Traditional point cloud recognition algorithms require a large number of samples and manual annotation, resulting in high costs, low accuracy, and the inability to simultaneously determine the occlusion status and stacking level of the wire rods, making them unsuitable for warehouse retrieval rules. Summary of the Invention

[0003] To address the technical problems existing in the background art described above, the present invention provides a point cloud-based object pose recognition method.

[0004] This invention adopts the following technical solution: a point cloud-based object pose recognition method, comprising the following steps: Acquire multi-view images of the target scene and determine the extrinsic parameters of each image acquisition unit in a unified coordinate system; Generate a 3D point cloud of the target scene based on the multi-view images. The multi-view images are segmented into instances to obtain target object image instances from each viewpoint. Establish a projection mapping relationship between the 3D point cloud and images from various viewpoints, and map the target object image instance to the corresponding viewpoint in the 3D point cloud to obtain a subset of the point cloud under each viewpoint. Based on the spatial consistency between subsets of point clouds from different perspectives, point clouds belonging to the same target object are cross-view correlation and fusion, and aggregated to obtain instance-level point clouds of each target object. Spatial feature information of the target object is obtained based on the instance-level point cloud, and the spatial feature information includes at least: pose parameters.

[0005] In a further embodiment, the method further includes the following steps: determining the occlusion state and stacking level of the target object based on the spatial coordinate distribution characteristics of the instance-level point cloud.

[0006] In a further embodiment, the following steps are also included: The spatial pose change of the image acquisition unit is monitored in real time, and the pose deviation of the image acquisition unit is dynamically calibrated and corrected. When a rotational displacement of the image acquisition unit is detected, the corresponding camera extrinsic parameter matrix is ​​updated synchronously based on the calibration parameters.

[0007] In a further embodiment, the process for determining the extrinsic parameters of each image acquisition unit in a unified coordinate system is as follows: Scene image feature points are extracted from each image acquisition unit, and cross-unit feature matching is performed to obtain the measured coordinates of the scene image feature points. ; Obtain the relative extrinsic parameters between each image acquisition unit ,in, For the first The image acquisition unit is relative to the first Rotation matrix of each image acquisition unit, For the first The image acquisition unit is relative to the first Translation matrix of each image acquisition unit are positive integers and ; By performing on-site 3D coordinate calibration on the feature points of the scene image, the true coordinates of the feature points in the unified world coordinate system are obtained. ; With true coordinates Based on the relative external parameters Optimization and calibration were performed to calculate the absolute extrinsic parameters of each image acquisition unit in the unified world coordinate system. ,in Let be the rotation matrix of the image acquisition unit relative to the world coordinate system. This represents the translation matrix of the image acquisition unit relative to the world coordinate system. In a further embodiment, the projection mapping relationship is derived from the 3D point cloud of the target scene. Intrinsic parameter matrices of cameras at various viewpoints and absolute external parameters It is established through the spatial reprojection formula.

[0008] In a further embodiment, the aggregation process of the instance-level point clouds of each target object is as follows: Calculate the spatial overlap and geometric feature similarity of point cloud subsets under different viewpoints, determine the cross-viewpoint correspondence of target object image instances under different viewpoints, and clarify the instance affiliation of the same physical target object in multi-view images; Based on the cross-view correspondence, image instance information belonging to the same physical target object from each viewpoint is uniformly associated and mapped to the 3D point cloud of the target scene. The corresponding point cloud subsets are aggregated, deduplicated, and optimized.

[0009] In a further embodiment, the process of obtaining the spatial feature information is as follows: Preprocessing of instance-level point clouds yields regularized target point clouds. ; Based on the normalized target point cloud Extract the main orientation features of the point cloud and calculate the coarse pose parameters of the target object to complete the coarse registration; Using the coarse pose parameters as initial values, the target point cloud and the standard point cloud model of the target object are iteratively registered, and the pose parameters are optimized by minimizing the Euclidean distance between the point clouds. When the iteration error converges to a preset threshold, the iteration stops and the spatial pose parameters of the target object are output.

[0010] In a further embodiment, the process for determining the occlusion state is as follows: The maximum and minimum coordinate values ​​and coordinate distribution density of the instance-level point cloud of each target object in the preset direction are statistically obtained to determine the vertical coverage and spatial distribution characteristics of the point cloud of each target object. Based on the vertical coverage and spatial distribution characteristics of the point cloud, determine the occlusion status of the target object: If the point cloud of an instance-level target object has a local area where the point cloud density drops sharply or the point cloud is missing, and the three-dimensional spatial coordinates of the area are included by the vertical coverage of the point cloud of other target objects, then the target object is determined to be occluded, and the spatial coordinates of the occluding source target object and the occluded area are marked. If the point cloud density is uniformly distributed and there are no obvious missing areas, the target object is determined to be in an unobstructed state.

[0011] In a further embodiment, the process for determining the stacking level is as follows: Based on the maximum Z-axis coordinate of all target object instance-level point clouds, hierarchical clustering is performed on each target object according to the preset vertical coordinate interval, and target objects falling into the same coordinate interval are divided into the same stacking level. The stacking level with the largest Z-axis coordinate value is determined as the top level of the target object stacking, and a corresponding level number is generated for each stacking level and its corresponding target object. By combining the occlusion status and stacking level number of each target object, the output is a target object spatial state dataset containing the target object number, occlusion status, stacking level, top-level identifier, and occlusion source information.

[0012] The beneficial effects of this invention are as follows: This invention employs a pre-defined calibration process, combining scene image feature point matching, initial triangulation, and total station on-site true value calibration. This enables long-distance, high-precision unified coordinate system calibration of multiple image acquisition units even in scenarios where the unit is installed at a height of over ten meters, solving the problem of insufficient calibration accuracy in such scenarios using conventional methods. Simultaneously, by monitoring the posture of the image acquisition units in real time and dynamically calibrating their pose deviations, the extrinsic parameter matrix can be updated synchronously when the camera rotates. This compensates for the inability of traditional fixed cameras to dynamically correct extrinsic parameters, suppresses errors caused by camera vibration, and ensures the consistency and reliability of large-scale point cloud acquisition, providing a stable data foundation for subsequent 3D reconstruction and posture recognition.

[0013] This invention employs multi-viewpoint point cloud stitching to unify local point clouds from various regions into the same world coordinate system. After fusion, redundant data is eliminated, improving the point cloud incompleteness caused by factors such as truck length and target object stacking patterns (e.g., coiled wire). Simultaneously, multi-dimensional filtering optimization effectively removes background noise and non-target point clouds, enhancing the purity and integrity of the target object's point cloud. High-quality point clouds reduce the difficulty of instance segmentation and pose calculation, improve processing efficiency and recognition stability, and reduce computational biases caused by point cloud quality issues.

[0014] This invention employs a processing method of first segmenting image instances and then mapping across viewpoints. First, target object instances are segmented on each main viewpoint image to obtain image-level instance information. Then, through projection mapping relationships, the multi-viewpoint instance information is mapped onto a 3D point cloud and aggregated to form a single target object instance-level point cloud. This method avoids problems such as uneven density, local missing values, and false positives that easily occur when segmenting directly on the 3D point cloud, improving the reliability of instance segmentation and stably separating single target object point clouds, providing reliable data for high-precision pose recognition.

[0015] This invention employs a two-step strategy of coarse registration followed by fine registration, combining principal component analysis and generative ICP algorithms. It achieves high-precision target object pose calculation without requiring a large number of samples or manual annotation, reducing implementation costs and improving pose calculation efficiency and accuracy. Simultaneously, based on instance-level point cloud spatial distribution characteristics, it can simultaneously determine the occlusion state and stacking level of target objects, locate occlusion relationships and layer affiliations, expanding the functional boundaries of traditional recognition methods. It better aligns with warehouse automation retrieval rules and can provide complete and reliable spatial information for unmanned overhead cranes and intelligent grasping equipment, improving the intelligence level and efficiency of warehouse operations. Attached Figure Description

[0016] Figure 1 This is a flowchart of a point cloud-based object pose recognition method according to Example 1. Detailed Implementation

[0017] The method of this invention is not only applicable to wire rods, but also to other stacked objects with regular or near-regular geometric structures, including but not limited to pipes, coils, rings, or block-shaped stacked objects. The invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Example 1 This embodiment uses wire rod as a typical example, primarily applied to smart warehousing areas, automated hoisting and transfer stations, and disordered grasping operations of industrial robots in the metallurgical and steel processing industries for hot-rolled wire rod / coil products. Addressing the challenges of high-density stacking, nested overlapping, and disordered stacking of wire rod after mass production, the system achieves precise 3D spatial pose recognition, individual wire rod instance segmentation, occlusion status determination, and automatic stacking hierarchy classification. This provides core visual and 3D data support for automated hoisting by unmanned overhead cranes, intelligent grasping by industrial robots, accurate inventory counting of wire rods, and collision avoidance planning of work paths, meeting the unmanned operation requirements of smart steel plants and intelligent logistics warehousing.

[0019] like Figure 1 As shown, this embodiment discloses a point cloud-based object pose recognition method, including the following steps: Acquire multi-view images of the target scene and determine the extrinsic parameters of each image acquisition unit in a unified coordinate system; Generate a 3D point cloud of the target scene based on the multi-view images. The multi-view images are segmented into instances to obtain target object image instances from each viewpoint; each target object image instance includes pixel region coordinates, contour features, and semantic labels from the corresponding viewpoint.

[0020] Establish a projection mapping relationship between the 3D point cloud and images from various viewpoints, and map the target object image instance to the corresponding viewpoint in the 3D point cloud to obtain a subset of the point cloud under each viewpoint. Based on the spatial consistency between subsets of point clouds from different perspectives, point clouds belonging to the same target object are cross-view correlation and fusion, and aggregated to obtain instance-level point clouds of each target object. Spatial feature information of the target object is obtained based on the instance-level point cloud, and the spatial feature information includes at least: pose parameters.

[0021] Considering the relatively high deployment height of image acquisition units within the workshop (typically 8-15 meters) and the wide coverage area of ​​the wire rod scene (5-8 meters field of view radius for a single camera), long-distance imaging can easily lead to the accumulation of feature point matching errors and a decrease in extrinsic parameter calibration accuracy, thereby affecting the accuracy of subsequent point cloud reconstruction and projection mapping. The process for determining the extrinsic parameters of each image acquisition unit in a unified coordinate system is as follows: The scene image feature points of each image acquisition unit are extracted using the SFM (Structure of Motion) algorithm, and cross-unit feature matching is performed to obtain the measured coordinates of the scene image feature points. ; The relative extrinsic parameters between each image acquisition unit were initially calculated using triangulation. ,in, For the first The image acquisition unit is relative to the first Rotation matrix of each image acquisition unit, For the first The image acquisition unit is relative to the first Translation matrix of each image acquisition unit are positive integers and ; By using a total station to perform on-site 3D coordinate calibration of the scene image feature points, the true coordinates of the scene image feature points in a unified world coordinate system are obtained. ; With true coordinates Based on the relative external parameters Optimization and calibration were performed to calculate the absolute extrinsic parameters of each image acquisition unit in the unified world coordinate system. ,in Let be the rotation matrix of the image acquisition unit relative to the world coordinate system. This is the translation matrix of the image acquisition unit relative to the world coordinate system. The external parameters of all image acquisition units are calibrated in a unified coordinate system in sequence.

[0022] By using millimeter-level true coordinate constraints provided by the total station, the pain point of feature point matching error accumulation under long-distance imaging is solved. Compared with traditional single SFM calibration, the rotation angle error of the external parameter calibration is reduced from ±0.3° to below ±0.08°, the translation error is optimized from ±5mm to within ±1.2mm, and the reprojection error is controlled within ±0.8 pixels, meeting the high-precision requirements of wire rod attitude recognition.

[0023] In this embodiment, the measured coordinates The acquisition process is as follows: Scene image feature points are extracted from images captured by each image acquisition unit. The ORB feature extraction algorithm is used, and cross-unit feature matching between image acquisition units is completed using the SIFT feature matching method. After removing erroneous matching pairs, the pixel coordinates of the feature points of the corresponding images of each image acquisition unit are obtained. At the same time, a one-to-one correspondence between feature points of different image acquisition units is established, thereby obtaining a preliminary correspondence between image feature points and scene 3D points; Based on camera intrinsic parameter matrix Establish feature point pixel coordinates and world coordinates Mapping relationship between : The measured coordinates of scene feature points are initially obtained through this mapping relationship. .

[0024] Based on this, the absolute extrinsic parameter described in this embodiment The calculation process is as follows: obtain the measured coordinates of three or more feature points in the scene. , and the measured coordinates The deviation value was calculated using the following formula for comparison. : ,right Threshold judgment convergence output absolute extrinsic parameters .like At that time, the external parameter calibration is completed, and the final absolute external parameter is output. .

[0025] The above technical solutions solve the technical problems of large 3D coordinate errors, coordinate distortion caused by feature mismatch, lack of true value benchmark and uncontrollable calibration accuracy in traditional algorithms.

[0026] In a further embodiment, the projection mapping relationship is derived from the 3D point cloud of the target scene. Intrinsic parameter matrices of cameras at various viewpoints and absolute external parameters This is established using the spatial reprojection formula. Specifically, the spatial reprojection formula is expressed as follows: ; In the formula, : 3D point cloud of the target scene (The column vector of the three-dimensional point coordinate vector before three-dimensional homogenization); : This is the rotation matrix in the camera's absolute extrinsic parameters, which realizes the rotation transformation from the world coordinate system to the camera coordinate system; : is the translation vector in the absolute extrinsic parameters of the camera, which realizes the translation transformation from the world coordinate system to the camera coordinate system; : This is the camera intrinsic parameter matrix, which includes parameters such as focal length and principal point, to realize the perspective projection from the camera coordinate system to the image pixel coordinate system; : is the homogeneous pixel coordinate vector of a 3D point on the corresponding main viewpoint image, which is then homogeneously normalized (i.e., divided by the third component). ), which can obtain two-dimensional pixel coordinates .

[0027] In a further embodiment, the aggregation process of instance-level point clouds of each target object is as follows: Calculate the spatial overlap and geometric feature similarity of point cloud subsets under different viewpoints, determine the cross-viewpoint correspondence of target object image instances under different viewpoints, and clarify the instance affiliation of the same physical target object in multi-view images; Based on the cross-view correspondence, image instance information belonging to the same physical target object from each viewpoint is uniformly associated and mapped to the 3D point cloud of the target scene. The corresponding point cloud subsets are aggregated, deduplicated, and optimized.

[0028] To address the issues of inconsistent disk bar instance attribution and severe fragmentation of 3D point clouds caused by independent segmentation of multi-view images, making it impossible to obtain complete point clouds for individual disk bars, the above-mentioned technical solution is adopted to achieve accurate matching of disk bar instances across different viewpoints. This allows for the aggregation and optimization to obtain complete, non-redundant point clouds at the individual disk bar instance level, providing a complete data foundation for subsequent pose calculation.

[0029] It should be noted that the 3D point cloud of the target scene described in this embodiment... The acquisition process is as follows: Based on the spatial division of the wire rod scenario, the following were selected: Group main view local 3D point cloud , For the first A local 3D point cloud independently reconstructed from the group's main perspective. It is an integer greater than or equal to 2. Furthermore, each group of local 3D point clouds All meet the following conditions: Spatial coverage conditions: It needs to cover the corresponding divided wire rod scene space area, with no visual blind spots, and ensure that each space partition has corresponding point cloud data support; Valid overlap condition: any two adjacent pairs There must be an effective overlapping area between them, with an overlap rate of not less than 30%, to provide a matching basis for subsequent splicing and ensure splicing accuracy; Data validity conditions: The corresponding point cloud data is complete and without obvious missing parts, and the point cloud density is not less than 100 points / square centimeter to ensure the availability of local point clouds; Target association conditions: Focusing on the wire rod target itself, invalid point clouds containing a large amount of background clutter and excessive noise points are removed to ensure that the core information of the point cloud is wire rod-related features.

[0030] Based on the calibrated absolute external parameters , will the Group main view local 3D point cloud Aligning to the same world coordinate system, fusing the local point clouds and eliminating redundant data in overlapping areas, a global stitched point cloud covering the entire disk strip scene is obtained. ; For the global stitched point cloud Multi-dimensional filtering is performed to obtain the 3D point cloud of the target scene. .

[0031] The multi-dimensional filtering process described in this embodiment can be: processing the global stitched point cloud. The system sequentially performs multi-dimensional optimization processing, including voxelization downsampling, disparity filtering to remove redundancy, and classification filtering to remove background, thereby removing noise points, discrete invalid points, and non-disk target point clouds.

[0032] Taking wire rod as an example, due to occlusion, missing data, and noise interference in the wire rod point cloud, traditional ICP registration is prone to getting trapped in local optima, resulting in low pose calculation accuracy and an inability to adapt to the cylindrical geometry of wire rod. Therefore, the process of acquiring spatial feature information described in this embodiment is as follows: Outlier filtering, point cloud downsampling, and normal vector and curvature feature extraction are performed sequentially on the instance-level point cloud to obtain the normalized target point cloud. ; The filtered PCA algorithm is used to perform principal component analysis on the normalized target point cloud, extract the principal orientation features of the point cloud, and calculate the coarse pose parameters of the wire rod. Complete coarse registration; Using the coarse pose parameters as initial values, the generative ICP algorithm is used for precise registration: Based on the geometric prior model of the wire rod (such as the cylindrical model) and the current coarse pose parameters A virtual standard point cloud matching the physical shape of the wire rod is generated under a unified world coordinate system. : ,in, This represents the total number of points in the virtual standard point cloud. For virtual standard point cloud The coordinates.

[0033] Construct a global error function : ,in, For the first normalization of the target point cloud Point cloud coordinates; The square of the Euclidean distance; The pose transformation matrix is ​​optimized iteratively using the least squares method. Minimize the global error function The virtual standard point cloud is updated in real time during the iteration process. And recalculate the error; When the error difference between two adjacent iterations converges to a preset threshold, the iteration stops and the high-precision spatial pose parameters of the bar are output. The spatial pose parameters include three-dimensional position coordinates and three-dimensional rotation angles.

[0034] By combining PCA coarse registration to quickly locate the initial pose, and using cylindrical priors to generate virtual point clouds, the defect of missing points is effectively overcome through generative ICP iterative optimization, thus realizing the spatial pose calculation of the bar.

[0035] Example 2 Based on the point cloud-based object pose recognition method disclosed in Embodiment 1, this embodiment also discloses a method for determining the occlusion state and stacking level of a target object based on the spatial coordinate distribution characteristics of instance-level point clouds.

[0036] Considering that high-density stacking of wire rods can cause mutual occlusion, making it impossible to automatically identify occlusion relationships, which can easily lead to collisions and failures in automated grasping. This embodiment also includes: determining the occlusion state and stacking level of the target object based on the spatial coordinate distribution characteristics of the instance-level point cloud.

[0037] Taking the wire rod as the target object, the specific occlusion judgment process is as follows: extract the three-dimensional spatial coordinate set of the point cloud of each wire rod instance level, and statistically obtain the maximum and minimum coordinate values ​​and coordinate distribution density of each wire rod instance level point cloud in a preset direction (such as the vertical Z-axis) and other point cloud features to determine the vertical coverage range and spatial distribution characteristics of the point cloud of each wire rod. Based on the vertical coverage and spatial distribution characteristics of the point cloud, determine the occlusion status of the wire rod: If the point cloud of an instance-level coil has a local area where the point cloud density drops sharply or the point cloud is missing, and the three-dimensional spatial coordinates of the area are included by the vertical coverage of the point cloud of other coils, then the coil is determined to be in an occluded state, and the spatial coordinates of the occluded source coil and the occluded area are marked. If the point cloud density distribution is uniform and there are no obvious missing areas, the disk strip is determined to be in an unobstructed state.

[0038] Based on 3D coordinates and point cloud density, the occlusion status is accurately determined, the occlusion source and region are located, and collision avoidance basis is provided for crawling path planning.

[0039] Disorderly stacked wire rods are difficult to classify into layers, making it impossible to quickly locate the top layer of grabbable wire rods, which affects operational efficiency.

[0040] The corresponding process for determining the stacking level of wire rods is as follows: Based on the maximum Z-axis coordinate of all bar instance-level point clouds, each bar is hierarchically clustered according to a preset vertical coordinate interval, and the bars falling into the same coordinate interval are divided into the same stacking level. The stacking level with the largest Z-axis coordinate value is determined as the top level of the wire rod stack, and a corresponding level number is generated for each stacking level and its associated wire rod. By combining the occlusion status of each bar with the stacking level number, the output bar spatial status dataset includes bar number, occlusion status, stacking level, top-level identifier, and occlusion source information.

[0041] The above technical solution enables automatic clustering and division of stacked layers, accurately identifies the top layer of wire rods, outputs complete spatial status data, and supports automated operation logic that prioritizes the capture of the top layer of wire rods.

[0042] Example 3 Considering that vibrations from overhead crane operation, equipment scraping, and support deformation in a workshop environment can easily cause positional shifts in the image acquisition unit, this embodiment also includes the following steps: The spatial pose change of the image acquisition unit is monitored in real time, and the pose deviation of the image acquisition unit is dynamically calibrated and corrected. When a rotational displacement of the image acquisition unit is detected, the corresponding camera extrinsic parameter matrix is ​​updated synchronously based on the calibration parameters.

[0043] For example, in the wire rod stacking workshop, the image acquisition unit (industrial camera) is fixed 12 meters above the storage area by a bracket to capture multi-angle images of the wire rod.

[0044] The continuous vibration caused by the frequent back-and-forth movement of the heavy overhead crane in the workshop caused slight deformation of the No. 2 camera bracket. The camera deflected downward by 1.2° and shifted horizontally by 8mm, which exceeded the system's preset attitude error range.

[0045] Using the above technical solution, correction parameters are determined. Based on the correction parameters obtained from calibration, the extrinsic parameter matrix of camera No. 2 is automatically updated, and the new extrinsic parameter parameters are synchronized to the 3D reconstruction and point cloud projection mapping modules.

[0046] If the external parameters are not updated in time, the mapping relationship between the bar point cloud and the image will shift, resulting in misalignment of bar instance segmentation and pose recognition error exceeding ±12mm, which can easily cause lifting and grasping deviation. After dynamic calibration and external parameter update, the camera attitude is restored to calibration accuracy, and the projection mapping deviation is controlled within ±2mm, ensuring the accuracy of wire rod attitude recognition and stacking determination.

[0047] It should be noted that the algorithm steps involved in the above embodiments, such as feature extraction, 3D reconstruction, point cloud processing, instance association, and pose calculation, are all preferred implementations. Without departing from the technical principles of this invention, those skilled in the art can use other equivalent algorithms or implementations to replace them.

Claims

1. A point cloud-based object pose recognition method, characterized in that, Includes the following steps: Acquire multi-view images of the target scene and determine the extrinsic parameters of each image acquisition unit in a unified coordinate system; A 3D point cloud of the target scene is generated based on the multi-view images. The multi-view images are segmented into instances to obtain target object image instances from each viewpoint. Establish a projection mapping relationship between the 3D point cloud and images from various viewpoints, and map the target object image instance to the corresponding viewpoint in the 3D point cloud to obtain a subset of the point cloud under each viewpoint. Based on the spatial consistency between subsets of point clouds from different perspectives, point clouds belonging to the same target object are cross-view correlation and fusion, and aggregated to obtain instance-level point clouds of each target object. Spatial feature information of the target object is obtained based on the instance-level point cloud, and the spatial feature information includes at least: pose parameters.

2. The object pose recognition method based on point clouds according to claim 1, characterized in that, It also includes the following steps: Based on the spatial coordinate distribution characteristics of instance-level point clouds, the occlusion state and stacking level of target objects are determined.

3. The object pose recognition method based on point clouds according to claim 1, characterized in that, It also includes the following steps: The spatial pose change of the image acquisition unit is monitored in real time, and the pose deviation of the image acquisition unit is dynamically calibrated and corrected. When a rotational displacement of the image acquisition unit is detected, the corresponding camera extrinsic parameter matrix is ​​updated synchronously based on the calibration parameters.

4. The object pose recognition method based on point clouds according to claim 1, characterized in that, The process for determining the extrinsic parameters of each image acquisition unit in a unified coordinate system is as follows: Extract scene image feature points from each image acquisition unit and perform cross-unit feature matching to obtain the measured coordinates of the scene image feature points. ; Obtain the relative extrinsic parameters between each image acquisition unit ,in, For the first The image acquisition unit is relative to the first Rotation matrix of each image acquisition unit, For the first The image acquisition unit is relative to the first Translation matrix of each image acquisition unit are positive integers and ; By performing on-site 3D coordinate calibration on the feature points of the scene image, the true coordinates of the feature points in the unified world coordinate system are obtained. ; With true coordinates Based on the relative external parameters Optimization and calibration were performed to calculate the absolute extrinsic parameters of each image acquisition unit in the unified world coordinate system. ,in Let be the rotation matrix of the image acquisition unit relative to the world coordinate system, and let be the translation matrix of the image acquisition unit relative to the world coordinate system.

5. The object pose recognition method based on point clouds according to claim 4, characterized in that, The projection mapping relationship is based on the 3D point cloud of the target scene. Intrinsic parameter matrices of cameras at various viewpoints and absolute external parameters It is established through the spatial reprojection formula.

6. The object pose recognition method based on point clouds according to claim 1, characterized in that, The aggregation process of the instance-level point clouds of each target object is as follows: Calculate the spatial overlap and geometric feature similarity of point cloud subsets under different viewpoints, determine the cross-viewpoint correspondence of target object image instances under different viewpoints, and clarify the instance affiliation of the same physical target object in multi-view images; Based on the cross-viewpoint correspondence, image instance information belonging to the same physical target object from each viewpoint is uniformly associated and mapped to the 3D point cloud of the target scene. The corresponding point cloud subsets are aggregated, deduplicated, and optimized.

7. The object pose recognition method based on point clouds according to claim 1, characterized in that, The process of obtaining the spatial feature information is as follows: Preprocessing of instance-level point clouds yields regularized target point clouds. ; Based on the normalized target point cloud Extract the main orientation features of the point cloud and calculate the coarse pose parameters of the target object to complete the coarse registration; Using the coarse pose parameters as initial values, the target point cloud and the standard point cloud model of the target object are iteratively registered, and the pose parameters are optimized by minimizing the Euclidean distance between the point clouds. When the iteration error converges to a preset threshold, the iteration stops and the spatial pose parameters of the target object are output.

8. The object pose recognition method based on point clouds according to claim 2, characterized in that, The process for determining the occlusion state is as follows: The maximum and minimum coordinate values ​​and coordinate distribution density of the instance-level point cloud of each target object in the preset direction are statistically obtained to determine the vertical coverage and spatial distribution characteristics of the point cloud of each target object. Based on the vertical coverage and spatial distribution characteristics of the point cloud, determine the occlusion status of the target object: If the point cloud of an instance-level target object has a local area where the point cloud density drops sharply or the point cloud is missing, and the three-dimensional spatial coordinates of the area are included by the vertical coverage of the point cloud of other target objects, then the target object is determined to be occluded, and the spatial coordinates of the occluding source target object and the occluded area are marked. If the point cloud density is uniformly distributed and there are no obvious missing areas, the target object is determined to be in an unobstructed state.

9. The object pose recognition method based on point clouds according to claim 2, characterized in that, The process for determining the stacking level is as follows: Based on the maximum Z-axis coordinate of all target object instance-level point clouds, hierarchical clustering is performed on each target object according to the preset vertical coordinate interval, and target objects falling into the same coordinate interval are divided into the same stacking level. The stacking level with the largest Z-axis coordinate value is determined as the top level of the target object stacking, and a corresponding level number is generated for each stacking level and its corresponding target object. By combining the occlusion status and stacking level number of each target object, the output is a target object spatial state dataset containing the target object number, occlusion status, stacking level, top-level identifier, and occlusion source information.

Images