A robot 3D vision guidance method based on omnidirectional low-rank constraint denoising

Abstract

The application relates to a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, which comprises the steps of point cloud data preprocessing, three-dimensional high-order tensor sequence construction, denoising processing and denoised point cloud generation. By sampling reference points, grouping similarity, constructing a three-order tensor and expanding the three-order tensor into a high-order tensor sequence, low-rank features in different directions are extracted. The core tensor is denoised by threshold shrinkage, and high-quality point cloud data is generated by combining inverse transformation. The method effectively removes point cloud noise, retains key geometric features, significantly improves the vision guidance accuracy and robustness of robots in complex environments, and is suitable for robot operation in industrial automation and dynamic scenes.

Description

Technical Field

[0001] This invention relates to a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, which is particularly suitable for robots to perform high-precision 3D vision perception and navigation in complex environments. Background Technology

[0002] With the continuous development of artificial intelligence, robotics, and automation technologies, robots are increasingly being used in various fields, especially in industrial production, logistics, smart homes, and autonomous driving. Robots have become a key technology for improving production efficiency and workplace safety. In industrial automation, robots need high-precision perception and navigation capabilities to operate stably in complex and dynamic environments. 3D vision technology, as one of the core technologies for robots to acquire environmental information, plays a crucial role. Through accurate 3D perception, robots can identify objects in their surroundings and complete key tasks such as spatial positioning, path planning, and obstacle avoidance.

[0003] Robot 3D vision technology primarily relies on sensors such as LiDAR, stereo cameras, and Time-of-Flight (ToF) cameras to acquire 3D point cloud data of the environment. This point cloud data, through precise descriptions of object surface morphology, helps robots build environmental models, providing them with accurate visual information about the environment. For example, in industrial automation, robots use high-precision point cloud data to identify, grasp, and place objects; while in the field of autonomous driving, robots rely on 3D LiDAR data for environmental mapping and decision support. However, despite the powerful perceptual capabilities that 3D vision technology provides to robots, it still faces many challenges in practical applications, especially regarding the noise problem in point cloud data.

[0004] 3D point cloud data is typically collected by various sensors when a robot perceives its environment. However, due to limitations in sensor accuracy, environmental interference, and surface irregularities, the collected point cloud data often contains noise. This noise can originate from several sources. First, the measurement errors and accuracy limitations of the sensors themselves can lead to inaccurate point cloud data. For example, LiDAR may produce measurement errors when encountering surfaces with low reflectivity (such as black objects or glass surfaces), or sensors may be interfered with under adverse weather conditions (such as rain or snow), resulting in noisy data. Second, in complex working environments (such as industrial workshops or outdoor scenes), the presence of dynamic objects, changes in lighting, and weather conditions can interfere with the sensor's perception capabilities, further increasing noise. Finally, surface irregularities are also a significant factor contributing to noise. In some cases, irregular shapes or occlusions on the object's surface can prevent the sensor from accurately capturing the complete surface of the object, thus generating discrete, inaccurate points and creating noisy data.

[0005] These noises not only affect the quality of point cloud data, but also prevent robots from accurately understanding and recognizing environmental information when performing tasks, thus impacting their localization, planning, and decision-making abilities. For example, when a robot is performing an object grasping task, noise may cause it to misidentify the shape or position of an object, thereby affecting the accuracy and success rate of the task. Therefore, denoising point cloud data has become a key technology in 3D vision.

[0006] In recent years, point cloud denoising technology has become a research hotspot in the field of 3D vision. The goal of point cloud denoising is to improve the quality of point cloud data by removing noise and retaining effective point cloud information, thereby providing more reliable support for robot vision guidance and environmental understanding. Currently, point cloud denoising technology mainly employs several methods. First, filtering-based methods are the most common. Filtering methods smooth point cloud data to remove noise points. Common filtering techniques include Gaussian filtering, K-nearest neighbor method, and mean-shift filtering. These methods reduce the impact of noise by smoothing the point cloud data within its neighborhood. Second, statistical methods utilize the statistical characteristics of point cloud data to identify and remove noise. For example, the RANSAC (Random Sample Consensus) algorithm is widely used for denoising. By constructing a model and comparing it with actual data, it identifies outliers (i.e., noise), thus achieving denoising. In addition, graph and geometry-based methods are also common denoising techniques. These methods represent point cloud data as a graphical structure and use graph theory algorithms to identify noise points and denoise the data. Common techniques include surface reconstruction-based denoising methods and curvature-based denoising methods, which remove noise by fitting surface data to point cloud data.

[0007] With the continuous development of deep learning technology, deep learning-based methods are gradually emerging in the field of point cloud denoising. Deep learning learns the features of point cloud data, enabling it to automatically identify and remove noise. For example, neural network architectures such as PointNet and PointNet++ are widely used in point cloud processing. During training, these neural networks can automatically identify noisy points and use the trained model for denoising. These methods generally achieve excellent denoising results, but they also face some challenges, such as requiring a large amount of labeled data for training and placing high demands on hardware resources.

[0008] Despite significant progress in existing denoising methods, they still have certain limitations in practical applications. For example, while filtering-based denoising methods are simple and effective, they often smooth out the details of point clouds, leading to a decrease in point cloud accuracy. Statistical methods, although capable of removing noise well, often rely on strict assumptions and are limited in their applicability to different noise types. Graphical and geometric methods, while highly accurate, have high computational complexity and poor real-time performance. Deep learning-based methods, while possessing excellent performance, require large amounts of labeled data for training and have high computational resource demands.

[0009] In robotics applications, especially in industrial automation and intelligent manufacturing, point cloud denoising technology is crucial for improving robots' spatial perception capabilities, enhancing operational accuracy, and reliability. Robots need to perform efficient perception and decision-making in dynamic and complex environments, and the quality of point cloud data directly impacts their performance. Therefore, developing an efficient, accurate point cloud denoising technology applicable to various environments has become a key technological requirement in the field of 3D vision-guided robotics. In practical applications, in addition to removing sensor noise and environmental interference, the diversity and complexity of robot tasks must also be considered. For example, in object recognition tasks, robots need to acquire information about object surfaces as accurately as possible; in path planning tasks, robots need to construct accurate environmental maps. Therefore, existing denoising methods need further improvement to enhance denoising effectiveness, ensuring that the denoised point cloud data retains as much effective information as possible while minimizing the loss of detailed structure.

[0010] To address the aforementioned challenges, this invention proposes a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising. This method combines a three-dimensional high-order tensor sequence with low-rank constraints to propose an efficient denoising framework. Through third-order tensor decomposition and enhancement processing of point cloud data, this invention can capture low-rank features in different directions, removing noise while preserving key structural information of the point cloud data. It is applicable to complex and dynamic industrial environments and scenarios, thus providing robots with more accurate and reliable environmental perception. Summary of the Invention

[0011] To address the aforementioned problems in the prior art, this invention proposes a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, comprising the following steps:

[0012] Step 1: Point cloud data preprocessing:

[0013] Reference points are selected by sampling from the point cloud data acquired by the robot;

[0014] For each reference point, select points in its neighborhood to construct a matrix describing the local structure;

[0015] The local matrices are grouped based on similarity analysis to form multiple similar matrix groups;

[0016] Stack similar matrices to form a third-order tensor to describe the local features of point cloud data;

[0017] Step 2: Construction of a three-dimensional high-order tensor sequence:

[0018] The third-order tensor is expanded in multiple directions to generate a sequence of three-dimensional high-order tensors, in order to capture the low-rank features of point cloud data in different directions.

[0019] Step 3: Noise Reduction Processing

[0020] Perform decomposition on a three-dimensional high-order tensor sequence to extract the core tensor and principal components;

[0021] Threshold shrinkage is performed on the core tensor to retain only elements that meet the set conditions in order to remove noise;

[0022] Perform an inverse transformation on the core tensor and principal components after threshold shrinkage to generate a denoised three-dimensional high-order tensor sequence, and generate a denoised block tensor through a mapping operation.

[0023] Step 4: Noise Reduction Point Cloud Generation:

[0024] The points in the denoised block tensor are restored to their original positions to form the final denoised point cloud data, which is used for robot 3D vision guidance.

[0025] In the point cloud data preprocessing step, a seed point set S is randomly selected from the point cloud data V acquired by the robot using a downsampling method to form reference points.

[0026] In the point cloud data preprocessing step, K nearest neighbor points are selected for each seed point s∈S to form a K×3 block matrix A. s ,in

[0027] A s =[as1 ,a s2 ,…,a sK ]

[0028] Among them, a si =[x si ,y si ,z si ] is point s i In three-dimensional space, the coordinate of i ranges from 1 to K;

[0029] A s It is a K×3 block matrix constructed with the seed point s as the center;

[0030] a si Represents the three-dimensional coordinates of the i-th neighboring point around the seed point s;

[0031] x si ,y si ,z si These are the neighboring points s i The x, y, z coordinates in three-dimensional space.

[0032] In the point cloud data preprocessing step, the similarity metric d(A,B) between different block matrices A and B is calculated using the iterative nearest point algorithm, and the similarity metric d(A,B) that meets the conditions is found during grouping. The block matrix B, where δ is a constant.

[0033] In the aforementioned third-order tensor construction step, similar block matrices are stacked to form a third-order tensor. Represented as

[0034] X = [A1, A2, ..., A M ]

[0035] Where M is the number of similar block matrices.

[0036] In the aforementioned three-dimensional higher-order tensor sequence construction step, the third-order tensor is constructed by modifying it along different directions. Tensor augmentation is performed to generate a three-dimensional high-order tensor sequence.

[0037] In the aforementioned denoising process, the three-dimensional high-order tensor sequence is processed. Perform Tucker decomposition.

[0038] In the denoising process described above, threshold shrinkage is performed on the core tensor G. l The elements satisfy

[0039]

[0040] in:

[0041] G l This represents the core tensor obtained by Tucker decomposition in direction l;

[0042] G l (i,j,k) is the core tensor G l The specific element values ​​at the three index positions i, j, and k;

[0043] τ is a preset threshold used for threshold shrinkage processing;

[0044] It is the core tensor after threshold shrinkage.

[0045] In the denoising process described above, the core tensor after threshold shrinkage is... and Perform the inverse transform to obtain the denoised three-dimensional high-order tensor sequence. The denoised block tensor X is generated through inverse enhancement operations. * .

[0046] In the denoised point cloud generation step, the denoised block tensor X is... * The points in the data are restored to their original positions in the point cloud data, forming the final denoised point cloud data V. * .

[0047] Beneficial effects:

[0048] This invention addresses the noise interference problem in point cloud data through an omnidirectional low-rank constraint denoising method, effectively preserving the local geometric features and omnidirectional low-rank characteristics of the data. Through steps such as third-order tensor construction, multi-directional expansion, core tensor threshold shrinkage, and inverse transformation, efficient denoising of point cloud data is achieved, generating smoother and more accurate point cloud data. This method significantly improves the 3D visual guidance capability of robots in complex environments, enhances the accuracy and robustness of task execution, and is suitable for robot operation in industrial automation and complex dynamic scenarios. Attached Figure Description

[0049] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to unduly limit the invention. In the drawings:

[0050] Figure 1 A flowchart of the 3D visual guidance method of the present invention is shown. Detailed Implementation

[0051] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0052] This invention provides a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising. This method improves the quality of 3D point cloud data acquired by the robot by removing noise interference from point cloud data, thereby achieving higher precision 3D vision guidance and control in complex industrial operation scenarios. Based on omnidirectional low-rank constraint denoising technology, this method effectively reduces the impact of multiple factors such as equipment errors, environmental interference, and object surface characteristics on point cloud data, improving data clarity and reliability.

[0053] As attached Figure 1 As shown, the specific implementation of this invention consists of the following four steps: point cloud data preprocessing, construction of a three-dimensional high-order tensor sequence, denoising processing, and denoised point cloud generation. Each step is implemented through different technical means and algorithms, which are interconnected to ensure that the final point cloud data has high quality and high applicability.

[0054] The specific steps are as follows:

[0055] Point cloud data preprocessing:

[0056] In the preprocessing stage of point cloud data, the raw point cloud data V collected by the robot is first processed using a downsampling method to reduce the amount of data and improve computational efficiency. Then, a seed point set S is randomly selected from the downsampled point cloud data as reference points. The selection of the seed point set directly affects the accuracy and stability of subsequent steps. For each seed point s∈S, its K nearest neighbors are selected, thus forming a K×3 block matrix A. s Block matrix A s This is used to describe the geometric information of seed points on their local surfaces, laying the foundation for the construction of subsequent data structures. By constructing this local surface block matrix, the hierarchical nature of data representation and the ability to describe local features can be significantly enhanced.

[0057] Construction of three-dimensional high-order tensor sequences:

[0058] After obtaining the local surface patch matrix, similar patch matrices are stacked to form a third-order tensor. Following the construction of the third-order tensor, to further capture the omnidirectional low-rank features of the point cloud data, directional enhancement processing is applied to the third-order tensor, generating a sequence of three-dimensional high-order tensors. This sequence of three-dimensional high-order tensors is the result of multi-directional enhancement of the point cloud data, thus enabling a more comprehensive description of the geometric features of the point cloud in various directions.

[0059] Noise reduction processing:

[0060] After the 3D high-order tensor sequence is constructed, denoising processing is performed. First, Tucker decomposition is performed on the high-order tensor sequence to extract multi-directional core tensor features. The core tensor represents the most important directional information in the data. Then, threshold shrinkage is performed on the core tensor to remove irrelevant or interfering components. This shrinkage operation retains only elements in the core tensor that meet the set threshold conditions, thereby reducing noise interference. Finally, the thresholded core tensor and principal components are subjected to inverse transform operations to generate the denoised 3D high-order tensor sequence. This denoising process significantly improves the clarity of the tensor data, ensuring that the robot can extract reliable visual guidance information from it.

[0061] Denoising point cloud generation:

[0062] Finally, the points in the denoised 3D high-order tensor sequence generated by the inverse transform are restored to their original point cloud data positions, forming the final denoised point cloud data V. * The denoised point cloud data can minimize the impact of noise caused by the environment and equipment, and has high precision and high stability. It can also effectively support 3D vision guidance and control tasks for robots in complex industrial environments.

[0063] The steps in this invention are interconnected, ensuring both comprehensive denoising and full utilization of the denoising effect of low-rank constraints. This results in denoised point cloud data with high-quality visual information, providing robust data support and technical assurance for robot applications in complex industrial scenarios. The specific implementation of each step is as follows:

[0064] Step 1: Point cloud data preprocessing

[0065] This step aims to select a representative set of points from the 3D point cloud data collected by the robot and describe the local feature structure of the data by constructing a local surface patch matrix, laying the foundation for subsequent denoising and tensor construction. The point cloud data preprocessing process includes the selection of reference points, the construction of the local surface patch matrix, the similarity measurement between patch matrices, and the generation of a third-order tensor.

[0066] The specific steps are as follows:

[0067] From the 3D point cloud dataset V collected by the robot, a seed point set S is randomly selected as reference points using a downsampling method. Downsampling can reduce the amount of data while preserving the structural features of the point cloud, making processing more efficient. The selection of the seed point set S is mainly to extract key features of the point cloud, thereby simplifying the data structure and improving the speed and stability of subsequent calculations. The selected seed point set S serves as a representative sample, reflecting to a certain extent the core geometric features of the overall point cloud, providing a foundation for the next step of constructing the block matrix.

[0068] For each seed point s∈S, select the K nearest neighbors of that point in the point cloud data V, and divide these neighbors into a K×3 block matrix A. s Formal representation. Specifically, block matrix A s It can be represented as:

[0069] A s =[a s1 ,a s2 ,…,a sK ]

[0070] Among them, a si =[x si ,y si ,z si ] is a neighboring point s i Coordinates in three-dimensional space. This block matrix A. s This describes the spatial geometric relationship of a seed point *s* within its local range and can be viewed as a three-dimensional spatial characterization of the local surface features of the seed point. By constructing a local surface block matrix, the local structural characteristics of the point cloud at different locations can be expressed more clearly, providing reliable data support for subsequent steps such as similarity analysis and third-order tensor construction.

[0071] After obtaining multiple block matrices, to identify the similarity of local structures in the point cloud data, this step uses the Iterative Closest Point (ICP) algorithm to calculate the similarity distance between any two block matrices A and B, denoted as d(A,B). The ICP algorithm calculates the similarity between two block matrices by iteratively updating the pose transformation parameters by minimizing the distance between corresponding points. Specifically, for each block matrix A, find all similar block matrices B that satisfy the following conditions:

[0072]

[0073] Here, δ is a preset constant used to control the threshold for similarity measurement. In this way, block matrices can be effectively grouped according to similarity, forming multiple sets of similar block matrices. These sets of similar block matrices represent regions in the point cloud data with similar geometric features, and subsequent steps will utilize these similarity features to construct an efficient tensor representation.

[0074] Based on the similarity measurement results, the block matrices in each similar block matrix set are stacked sequentially to form a third-order tensor. Here, K represents the number of rows in each block matrix, 3 represents the coordinates in 3D space, and M represents the number of similar block matrices. The constructed third-order tensor X aggregates the data features of all similar block matrices, providing structured data support for the subsequent construction of higher-order 3D tensor sequences. This tensor construction method can better capture the local features of point cloud data in different regions, while preserving the geometric structure information between similar block matrices, thus laying the foundation for tensor decomposition and data augmentation operations in the subsequent denoising process.

[0075] Through the above steps, the point cloud data is gradually transformed from the original 3D point cloud data into a third-order tensor with rich local feature information, providing support for subsequent construction of three-dimensional high-order tensor sequences and denoising processing. This point cloud data preprocessing step not only reduces the impact of noise interference, but also greatly improves the accuracy and stability of point cloud data processing in robot 3D vision-guided tasks.

[0076] Step 2: Construction of 3D High-Order Tensor Sequences

[0077] After constructing the third-order tensor X, the next step is to perform directional enhancement processing to form a higher-order tensor sequence, which facilitates the capture of low-rank feature information from multiple directions. This step aims to express the geometric structural features of the point cloud data in a higher dimension through tensor enhancement processing along different directions, thereby laying a richer data foundation for subsequent denoising processing.

[0078] The specific steps are as follows:

[0079] In this step, the third-order tensor Tensor augmentation is performed along different directions to obtain a three-dimensional high-order tensor sequence.

[0080] It is the first tensor in the three-dimensional higher-order tensor sequence, representing the higher-order tensor obtained after tensor augmentation of X along the first direction (dimension K).

[0081] K retains the original dimension, while the other dimensions are expanded through tensor augmentation.

[0082] Capture low-rank features of point cloud data in the first direction and extract global features along the dimension of the number of neighboring points.

[0083] It is the second tensor in the three-dimensional higher-order tensor sequence, representing the higher-order tensor obtained after tensor augmentation of X along the second direction (dimension 3, i.e., three-dimensional coordinates x, y, z).

[0084] Where 3 represents the three-dimensional spatial coordinates, and the other dimensions are extended through tensor augmentation.

[0085] Capture the characteristics of point cloud data in the coordinate dimension and analyze the geometric structure and spatial distribution of the point cloud.

[0086] It is the third tensor in the three-dimensional higher-order tensor sequence, representing the higher-order tensor obtained after tensor augmentation processing of X along the third direction (dimension M, i.e. the number of similar block matrices).

[0087] Where M represents the number of similar block matrices, and other dimensions are expanded through tensor augmentation.

[0088] Extract the feature relationships between similar block matrix groups of point cloud data and capture the global characteristics between different block matrices.

[0089] These three higher-order tensors, through tensor augmentation along different directions, capture the low-rank features of point cloud data in the dimensions of neighborhood points, 3D spatial coordinates, and the number of similar block matrices. Together, they form a sequence of three-dimensional higher-order tensors. Used to describe the omnidirectional characteristics of point cloud data.

[0090] Each higher-order tensor in this tensor sequence contains different directional information, capturing the geometric features of point cloud data in different dimensions and directions.

[0091] Specifically, the constructed three-dimensional high-order tensor sequence The dimensions of each tensor in the equation are as follows:

[0092] tensor

[0093] Defined in dimension On K, and satisfying the condition

[0094]

[0095] That is, the dimensions 3 and M in X are expanded into multiple dimensions through tensor augmentation operations. The structural information on the original dimension K is preserved in this process.

[0096] tensor

[0097] Defined in dimension 3 above, and the conditions are met.

[0098]

[0099] That is, the dimension 3 of the third-order tensor X is preserved, and the dimensions K and M are transformed into a multidimensional structure. This allows for further refinement of directional information.

[0100] tensor

[0101] Defined in dimension On M, and satisfying the condition

[0102]

[0103] That is, the dimension M of the third-order tensor X is retained, and the dimensions K and 3 are transformed into a higher-dimensional structure, further enhancing the directional expressive power of point cloud data.

[0104] This tensor augmentation process expands the third-order tensor X into a higher-dimensional tensor structure along different directions. The resulting higher-order tensor sequence... It can capture the low-rank features of point cloud data from multiple angles and levels, providing richer information support for identifying and processing data characteristics in different directions during the subsequent denoising process.

[0105] The tensor augmentation operation in this step enables each higher-order tensor to carry unique directional feature information, thereby effectively capturing the low-rank features of point cloud data in various directions in three-dimensional space.

[0106] Step 3: Noise Reduction Processing

[0107] Obtaining a three-dimensional high-order tensor sequence The purpose of denoising is to effectively remove noise components from the data while preserving key structural features of the point cloud data through multi-stage mathematical operations. Denoising includes operations such as Tucker decomposition, threshold shrinkage, and inverse transform. The specific steps are as follows:

[0108] This step first involves the three-dimensional high-order tensor sequence. Perform Tucker decomposition, breaking it down into core tensors and principal components in multiple directions. Specifically, Tucker decomposition divides the tensor... It is represented as a set of low-rank approximations, where:

[0109]

[0110] Where, α l The weighting coefficients for each direction l reflect the importance of each direction; G l The core tensor for each direction captures the key features of that direction; In all dimensions The principal component matrix represents the main variations of point cloud data across different dimensions. Tucker decomposition effectively extracts core features in each direction, thereby removing redundant information and noise from the data.

[0111] The main purpose of Tucker decomposition is to reduce the dimensionality of high-dimensional point cloud data and decompose it into low-rank matrices, making the structure of the point cloud data simpler and suppressing noise. This decomposition process can remove redundant information in the point cloud data, improve data quality, and lay the foundation for subsequent threshold shrinkage and inverse transformation processing.

[0112] To further remove noise components, threshold shrinkage was applied to the core tensor G. l Each element in G is filtered and adjusted. Specifically, for each element G... l For the element (i,j,k), if its absolute value is greater than a set threshold τ, then retain the original value of the element; otherwise, set the element to zero. That is, perform the following operation:

[0113]

[0114] in:

[0115] G l Let G represent the core tensor obtained by Tucker decomposition along direction l. l It is the original three-dimensional high-order tensor data The compressed representation contains the main feature information of the point cloud data in that direction, but may also contain noise components.

[0116] G l (i,j,k) is the core tensor G l The specific element values ​​at the three index positions i, j, and k. These represent a specific numerical value in the core tensor, which may reflect the characteristics or noise information of a local region in the point cloud data.

[0117] τ is a preset threshold used for threshold shrinkage processing to distinguish between valid features and noise in the core tensor.

[0118] Its function is to determine if the absolute value of an element is |G l If (i,j,k)| is greater than τ, then the element is considered to be of great significance to the point cloud data and should be retained; otherwise, the element is considered to be noise and should be set to zero.

[0119] This is the core tensor after threshold shrinkage. Compared to the original core tensor G... l , Noise elements have been removed, and only feature information that is important to the point cloud data has been retained.

[0120] It is a cleaner and more reliable core tensor that removes noise and retains the key structural features of the original data, providing higher-quality basic data for subsequent inverse transformation and point cloud reconstruction.

[0121] Thresholding shrinkage preserves features important to point cloud data while effectively suppressing noise. Thresholding shrinkage helps reduce errors and inaccuracies caused by noise, resulting in more accurate and reliable final point cloud data.

[0122] This step is a crucial part of the noise reduction process because it can effectively remove noise components with small fluctuations or abnormalities while ensuring that key information is not lost.

[0123] After completing the threshold shrinkage process, the next step is to process the core tensor in each direction. With the corresponding principal component matrix Perform the inverse transformation of Tucker decomposition. The purpose of this step is to combine the thresholded low-rank core tensor with the principal component matrix to recover its high-dimensional structure, resulting in a denoised three-dimensional high-order tensor sequence. The specific steps are as follows:

[0124]

[0125] in:

[0126] This represents the denoised three-dimensional high-order tensor sequence. It is achieved by modifying the core tensor. and principal component matrix The result obtained by performing the inverse transformation of Tucker decomposition reflects the feature information of the denoised point cloud data in different directions.

[0127] It is a high-quality tensor sequence that has been noise-removed and retains key features, providing a basis for further generation of block tensors X. * And Point Cloud Data V * Provides the foundation.

[0128] I represents the direction index, with a value range of I = 1, 2, 3, corresponding to different directions in a three-dimensional higher-order tensor sequence.

[0129] In the formula, 'l' is used to distinguish the processing of the tensor in different directions, ensuring that the features in each direction are reconstructed. middle.

[0130] α l It is the weight coefficient on each direction l, reflecting the importance of that direction to the overall point cloud data.

[0131] Weighting coefficient α lThis is used to adjust the contribution ratios in different directions so as to integrate information from all directions during the inverse transformation and generate a high-quality denoised tensor.

[0132] It is the core tensor along direction l after threshold shrinkage. It contains the main features of the point cloud data after noise removal. As the low-rank information retained after denoising, It is an important input to the inverse transform, and is related to the principal component matrix. After combining, the denoised high-dimensional structure can be reconstructed.

[0133] The principal component matrix represents the principal component matrix along direction l in dimension i, containing the main variation patterns of the point cloud data in that dimension, where i ranges from 1 to N. Principal Component Matrix It is obtained from the original tensor decomposition and is used to describe the feature information in each dimension. In the inverse transformation, the principal component matrix and the core tensor... Combined to reconstruct the complete tensor.

[0134] × N This represents the tensor-matrix multiplication operation in the Nth dimension. It is used to multiply principal component matrices. Feature information is mapped to the core tensor of the corresponding dimension. In this way, the high-dimensional structure of the tensor can be recovered.

[0135] N is the dimension of the tensor. For a three-dimensional tensor, N = 3, corresponding to the three-dimensional space of point cloud data (e.g., x, y, z). It indicates the dimensional structure of the tensor, and the tensor-matrix multiplication is performed dimension-by-dimensional in the inverse transformation formula.

[0136] X * It is the denoised block tensor after the inverse enhancement operation, representing the point cloud data after noise removal.

[0137] X * By denoising the three-dimensional high-order tensor sequence Convert back to a block matrix structure to generate a denoised block tensor that better matches the original point cloud representation.

[0138] The denoised core tensor after inverse transformation The corresponding principal component matrices are then recombined to recover the denoised three-dimensional tensor sequence.

[0139] Subsequently, after inverse enhancement operations, a denoised block tensor X is generated. * The block tensor contains the noise-removed point cloud data. The inverse enhancement operation transforms the denoised 3D tensor sequence back into the original block matrix structure, eliminating redundant noise components while preserving key information and geometric features in the data.

[0140] The denoised block tensor X is obtained through inverse transformation and aggregation operations. * It more accurately reflects the true structure and characteristics of point cloud data, making the denoised point cloud data smoother and more accurate, suitable for subsequent robot 3D vision guidance tasks.

[0141] In summary, the denoising process effectively removes noise components from point cloud data through a series of mathematical operations, including Tucker decomposition, threshold shrinkage, and inverse transformation, while preserving the key information and structural features of the point cloud data. This process not only improves the quality of point cloud data but also provides more accurate and reliable data support for the visual guidance of robots in complex environments.

[0142] Step 4: Noise Reduction Point Cloud Generation

[0143] After completing the denoising process, the denoised block tensor X is obtained. * The next step is to restore the denoised data to the original spatial coordinate system, thereby generating the final denoised point cloud V. * The main goal of this process is to remap the noise-removed and simplified point cloud data back to its original three-dimensional spatial position, ensuring that the geometric structure of the point cloud is preserved and effectively eliminating the interference of environmental noise and equipment errors. This results in more accurate and reliable point cloud data that can provide clearer visual guidance information for the robot.

[0144] The specific steps are as follows:

[0145] For the denoised block tensor X * For each point in the data, based on its relative position and orientation information recovered during the denoising process, the spatial coordinates of these points are remapped back to their corresponding positions in the original point cloud data. The position of each point is accurately restored to its correct coordinates in three-dimensional space, eliminating deviations or distortions caused by noise. This restoration process ensures that the geometry and spatial features of the point cloud data are preserved after denoising.

[0146] After restoring all the denoised points to their original positions, the resulting point cloud set is the final denoised point cloud V. * Compared to the original point cloud data, this point cloud dataset significantly reduces interference introduced by environmental noise, sensor errors, and other non-ideal factors, preserving a more realistic and accurate three-dimensional spatial structure. This denoising process not only eliminates data errors caused by the external environment or the equipment itself, but also improves the quality of the point cloud data, making subsequent visual analysis, path planning, and decision-making more reliable.

[0147] Point cloud V after noise reduction processing *It offers significant advantages in visually guided tasks. The accuracy of point cloud data is greatly improved, enabling robots to more accurately identify and locate objects in their surroundings, thus enhancing their ability to perform tasks in complex environments. The denoised point cloud reduces inaccuracies caused by errors, allowing robots to work more stably in dynamic or unstable environments and exhibiting greater robustness to external disturbances.

[0148] The final generated denoised point cloud V * This data will serve as input for the robot during 3D vision guidance. The denoised point cloud provides clear environmental structure information, supporting the robot's precise spatial perception, path planning, and motion control. Whether in industrial production, logistics warehousing, or complex operational scenarios, the robot can adjust its behavior and decisions in real time based on this high-quality point cloud data to ensure the smooth execution of tasks.

[0149] The denoised point cloud generation in this step significantly improves the quality of the point cloud data, providing a more accurate and reliable foundation for subsequent visual analysis and robot control.

[0150] The above description is only a preferred embodiment of the present invention. Therefore, all equivalent changes or modifications made to the structure, features and principles described in the claims of this patent application are included in the scope of this patent application.

Claims

1. A robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, characterized in that: Includes the following steps: Step 1: Point cloud data preprocessing: Reference points are selected by sampling from the point cloud data acquired by the robot; For each reference point, select points in its neighborhood to construct a matrix describing the local structure; Based on similarity analysis, the matrices of local structures are grouped to form multiple similar matrix groups; Stack similar matrices to form a third-order tensor to describe the local features of point cloud data; Step 2: Construction of three-dimensional high-order tensor sequences: The third-order tensor is expanded in multiple directions to generate a sequence of three-dimensional high-order tensors, so as to capture the low-rank features of point cloud data in different directions. Step 3: Noise Reduction Processing Perform decomposition on a three-dimensional high-order tensor sequence to extract the core tensor and principal components; Threshold shrinkage is performed on the core tensor to retain only elements that meet the set conditions in order to remove noise; Perform an inverse transformation on the core tensor and principal components after threshold shrinkage to generate a denoised three-dimensional high-order tensor sequence, and generate a denoised block tensor through a mapping operation. Step 4: Noise Reduction Point Cloud Generation: The points in the denoised block tensor are restored to their original positions to form the final denoised point cloud data for use in robot 3D vision guidance. In the three-dimensional high-order tensor sequence construction step, the third-order tensor is constructed by modifying it along different directions. Tensor augmentation is performed to generate a three-dimensional high-order tensor sequence. ; It is the first tensor in a three-dimensional higher-order tensor sequence, representing the first tensor in the sequence of higher-order tensors. The higher-order tensor obtained after tensor augmentation along the first direction; wherein, the first direction is dimension K; Capture low-rank features of point cloud data in the first direction and extract global features along the dimension of the number of neighboring points; It is the second tensor in a three-dimensional higher-order tensor sequence, representing the... The higher-order tensor obtained after tensor augmentation along the second direction; wherein, the second direction is the dimension. That is, three-dimensional coordinates ; Capture the characteristics of point cloud data in the coordinate dimension, and analyze the geometric structure and spatial distribution of point clouds; It is the third tensor in a three-dimensional higher-order tensor sequence, representing the... The higher-order tensor obtained after tensor augmentation along the third direction; where the third direction is the dimension. , that is, the number of similar block matrices; Extract the feature relationships between similar block matrix groups of point cloud data and capture the global characteristics between different block matrices.

2. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the point cloud data preprocessing step, a seed point set S is randomly selected from the point cloud data V acquired by the robot using a downsampling method to form reference points.

3. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the point cloud data preprocessing step, K nearest neighbor points are selected for each seed point s∈S to form a K×3 block matrix. ,in in, For point Coordinates in three-dimensional space The value range is 1- ; Seed point Built around block matrix; Represents seed point Surrounding The three-dimensional coordinates of the neighboring points; These are the neighboring points. In three-dimensional space Coordinate values.

4. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 3, characterized in that: In the point cloud data preprocessing step, the similarity metric d(A, B) between different block matrices A and B is calculated using the iterative nearest point algorithm, and the similarity metric d(A, B) that meets the conditions is found during grouping. The block matrix B, where It is a constant.

5. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 3, characterized in that: In the construction step of the third-order tensor, similar block matrices are stacked to form a third-order tensor. , represented as Where M is the number of similar block matrices; The set of similar block matrices generated after grouping by similarity.

6. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the denoising process, the three-dimensional high-order tensor sequence Perform Tucker decomposition.

7. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the denoising process, threshold shrinkage is performed on the core tensor. The elements satisfy in: Indicates direction The core tensor obtained from Tucker decomposition; It is the core tensor exist The specific element values ​​at the three index positions; It is a preset threshold used for threshold shrinkage processing; It is the core tensor after threshold shrinkage processing.

8. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the denoising process, the core tensor after threshold shrinkage is... and Perform the inverse transform to obtain the denoised three-dimensional high-order tensor sequence. Denoising block tensors are generated through inverse enhancement operations. ,in, In all dimensions Principal component matrix on.

9. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as described in claim 1, characterized in that: In the denoised point cloud generation step, the denoised block tensor The points in the data are restored to their original positions in the point cloud data, forming the final denoised point cloud data. .

Images