Robot 3D visual guidance method based on omnidirectional low-rank constraint denoising

Abstract

The invention relates to a robot 3D visual guidance method based on omni-directional low-rank constraint denoising. The robot 3D visual guidance method comprises the steps of point cloud data preprocessing, three-dimensional high-order tensor sequence construction, denoising processing, denoising point cloud generation and the like. Reference point sampling and similarity grouping are carried out on point cloud data, a third-order tensor is constructed and expanded into a high-order tensor sequence, and low-rank features in different directions are extracted. And de-noising the core tensor by adopting threshold shrinkage, and generating high-quality point cloud data in combination with inverse transformation. According to the method, point cloud noise is effectively removed, key geometric features are reserved, the visual guidance precision and robustness of the robot in a complex environment are remarkably improved, and the method is suitable for robot operation in industrial automation and dynamic scenes.

Description

Technical Field

[0001] The present invention relates to a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, which is particularly suitable for the robot to perform high-precision three-dimensional visual perception and navigation in a complex environment. Background Art

[0002] With the continuous development of artificial intelligence, robotics and automation technology, the application of robots in many fields has gradually increased, especially in industrial production, logistics distribution, smart home and autonomous driving. Robots have become a key technology to improve production efficiency and work safety. In the field of industrial automation, robots need to have high-precision perception and navigation capabilities so that they can work stably in complex and dynamic working environments. As one of the core technologies for robots to obtain environmental information, 3D vision technology plays a vital role. Through precise three-dimensional perception, robots can identify objects in the surrounding environment and complete key tasks such as spatial positioning, path planning and obstacle avoidance.

[0003] Robot 3D vision technology mainly relies on sensors such as laser radar, stereo cameras, and time-of-flight (ToF) cameras to obtain three-dimensional point cloud data in the environment. These point cloud data can help robots build environmental models and provide them with accurate visual information about the environment by accurately describing the surface morphology of objects. For example, in industrial automation, robots use high-precision point cloud data to identify, grasp, and place objects; in the field of autonomous driving, robots rely on three-dimensional laser radar (LiDAR) data for environmental mapping and decision support. However, although 3D vision technology provides robots with powerful perception capabilities, it still faces many challenges in practical applications, especially in the noise problem of point cloud data.

[0004] 3D point cloud data is usually collected by various sensors when the robot perceives the environment, but due to factors such as sensor accuracy limitations, environmental interference, and irregularities on the surface of objects, the collected point cloud data often contains certain noise. These noises may come from many aspects. First, the measurement errors and accuracy limitations of the sensor itself may cause inaccurate point cloud data. For example, LiDAR may produce measurement errors when encountering surfaces with low reflectivity (such as black objects or glass surfaces), or the sensor may be disturbed in severe weather conditions (such as rain and snow), resulting in the generation of noisy data. Secondly, in complex working environments (such as industrial workshops or outdoor scenes), the presence of dynamic objects, changes in lighting, and weather may interfere with the sensor's perception capabilities, further increasing noise. Finally, irregularities on the surface of objects are also an important factor causing noise. In some cases, the surface of an object may have irregular shapes or occlusions, which will cause the sensor to be unable to accurately capture the complete surface of the object, thereby generating discrete inaccurate points and forming noisy data.

[0005] These noises not only affect the quality of point cloud data, but also make it impossible for the robot to accurately understand and identify environmental information when performing tasks, thus affecting its positioning, planning and decision-making capabilities. For example, when the robot is performing an object grasping task, noise may cause the robot to incorrectly identify the shape or position of the object, thereby affecting the execution accuracy and success rate of the task. Therefore, denoising of point cloud data has become a key technology in 3D vision technology.

[0006] In recent years, point cloud denoising technology has become a research hotspot in the field of 3D vision. The goal of point cloud denoising is to improve the quality of point cloud data by removing noise data and retaining effective point cloud information, so as to provide more reliable support for robot visual guidance and environmental understanding. At present, point cloud denoising technology mainly adopts several methods. First, the filtering-based method is the most common type of technology. The filtering method removes noise points by smoothing the point cloud data. Common filtering techniques include Gaussian filtering, K nearest neighbor method and mean shift filtering. These methods reduce the impact of noise by smoothing within the neighborhood of point cloud data. Secondly, statistical methods use the statistical characteristics of point cloud data to identify and remove noise. For example, the RANSAC (random sampling consensus) algorithm is widely used in denoising. By building a model and comparing it with the actual data, outliers (i.e. noise) are identified to achieve denoising. In addition, methods based on graphics and geometry are also common denoising methods. This type of method represents point cloud data as a graph structure, uses graph theory algorithms to identify noise points, and denoises the data. Common techniques include surface reconstruction-based denoising methods and curvature-based denoising methods, which remove noise by performing surface fitting on point cloud data.

[0007] With the continuous development of deep learning technology, deep learning-based methods have gradually emerged in the field of point cloud denoising. Deep learning can automatically identify and remove noise by learning the characteristics of point cloud data. For example, neural network structures such as PointNet and PointNet++ are widely used in the field of point cloud processing. During the training process, neural networks can automatically identify noise points and perform denoising through the trained models. These methods are usually excellent in denoising effects, but they also face some challenges, such as the need for a large amount of labeled data for training and high requirements for hardware resources.

[0008] Although the existing denoising methods have made significant progress, they still have certain limitations in practical applications. For example, although the filtering-based denoising method is simple and effective, it often smoothes the detailed information of the point cloud, resulting in reduced accuracy of the point cloud; while the statistical-based method can remove noise well, it often relies on strict assumptions and has certain limitations on the type of noise; although the graphics and geometry-based methods have high accuracy, they are computationally complex and have poor real-time performance; although the deep learning-based method has excellent performance, it requires a large amount of labeled data for training and has high requirements for computing resources.

[0009] In robotic applications, especially in the fields of industrial automation and intelligent manufacturing, point cloud denoising technology is crucial to improving the robot's spatial perception ability, enhancing operational accuracy and reliability. Robots need to perceive and make decisions efficiently in dynamic and complex environments, and the quality of point cloud data directly affects the robot's behavior. Therefore, developing an efficient, accurate and multi-environmentally applicable point cloud denoising technology has become a key technical requirement in the current field of robot 3D visual guidance. In practical applications, in addition to removing sensor noise and environmental interference, the diversity and complexity of robot tasks also need to be considered. For example, in object recognition tasks, robots need to obtain information on the surface of objects as accurately as possible; in path planning tasks, robots need to build accurate environmental maps. Therefore, the existing denoising methods need to be further improved to improve the denoising effect, ensure that the denoised point cloud data can retain effective information to the greatest extent, and minimize the loss of detailed structures.

[0010] In response to the above challenges, the present invention proposes a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising. This method proposes an efficient denoising framework by combining three-dimensional high-order tensor sequences with low-rank constraints. Through the third-order tensor decomposition and enhancement processing of point cloud data, the present invention can capture low-rank features in different directions, while removing noise, retaining the key structural information of point cloud data, and is suitable for complex and changeable industrial environments and dynamic scenes, thereby providing robots with more accurate and reliable environmental perception. Summary of the invention

[0011] In order to solve the above problems in the prior art, the present invention proposes a robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, comprising the following steps:

[0012] Step 1: Point cloud data preprocessing:

[0013] Select reference points by sampling from the point cloud data acquired by the robot;

[0014] For each reference point, points in the neighborhood are selected to construct a matrix describing the local structure;

[0015] The local matrices are grouped based on similarity analysis to form multiple similar matrix groups;

[0016] Similar matrix groups are stacked to form a third-order tensor to describe the local features of point cloud data;

[0017] Step 2: Three-dimensional high-order tensor sequence construction:

[0018] The third-order tensor is expanded in multiple directions to generate a three-dimensional high-order tensor sequence to capture the low-rank features of the point cloud data in different directions.

[0019] Step 3: De-noising:

[0020] Perform decomposition on a three-dimensional high-order tensor sequence to extract core tensors and principal components;

[0021] Perform threshold shrinkage processing in the core tensor to remove noise by retaining only elements that meet the set conditions;

[0022] Perform inverse transformation on the core tensor and principal component after threshold shrinkage to generate a denoised three-dimensional high-order tensor sequence, and generate a denoised block tensor through a mapping operation;

[0023] Step 4: Denoising cloud generation:

[0024] The points in the denoised block tensor are restored to their original positions to form the final denoised point cloud data for use in robot 3D visual guidance.

[0025] In the point cloud data preprocessing step, a seed point set S is randomly selected from the point cloud data V acquired by the robot through a downsampling method to form reference points.

[0026] In the point cloud data preprocessing step, K nearest neighbor points are selected for each seed point s∈S to form a K×3 block matrix A s ,in

[0027] A s =[as1 ,a s2 ,…,a sK ]

[0028] Among them, a si =[x si ,y si ,z si ] is point s i The coordinates in three-dimensional space, the value range of i is 1-K;

[0029] A s It is a K×3 block matrix built around the seed point s;

[0030] a si Represents the three-dimensional coordinates of the i-th neighborhood point around the seed point s;

[0031] x si ,y si ,z si They are the neighborhood points s i The x, y, z coordinates in three-dimensional space.

[0032] In the point cloud data preprocessing step, the similarity measure d(A, B) between different block matrices A and B is calculated by iterative closest point algorithm, and the similarity measure d(A, B) satisfying the condition is found when grouping. The block matrix B of , where δ is a constant.

[0033] In the third-order tensor construction step, similar block matrices are stacked to form a third-order tensor Expressed as

[0034] X=[A1,A2,…,A M ]

[0035] Where M is the number of similar block matrices.

[0036] In the three-dimensional high-order tensor sequence construction step, the three-order tensor is Perform tensor enhancement to generate a three-dimensional high-order tensor sequence

[0037] In the denoising step, the three-dimensional high-order tensor sequence Perform Tucker decomposition.

[0038] In the denoising step, a threshold shrinkage process is performed, and the core tensor G l The elements satisfy

[0039]

[0040] in:

[0041] G l represents the core tensor obtained by Tucker decomposition in direction l;

[0042] G l (i,j,k) is the core tensor G l The specific element values ​​at the three index positions i, j, and k;

[0043] τ is the preset threshold used for threshold shrinkage processing;

[0044] It is the core tensor after threshold shrinkage.

[0045] In the denoising step, the core tensor after threshold shrinkage is and Perform inverse transform to obtain the denoised three-dimensional high-order tensor sequence And generate the denoised block tensor X through the inverse enhancement operation * .

[0046] In the denoised point cloud generation step, the denoised block tensor X * The points in the point cloud are restored to their original positions to form the final point cloud data V after denoising. * .

[0047] Beneficial effects:

[0048] The present invention solves the noise interference problem in point cloud data through an omnidirectional low-rank constraint denoising method, effectively retaining the local geometric features and omnidirectional low-rank characteristics of the data. Through the steps of third-order tensor construction, multi-directional expansion, core tensor threshold contraction and inverse transformation, efficient denoising of point cloud data is achieved to generate smoother and more accurate point cloud data. This method significantly improves the 3D visual guidance capability of robots in complex environments, improves the accuracy and robustness of task execution, and is suitable for robot operations in industrial automation and complex dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application, but do not constitute an improper limitation of the present invention. In the drawings:

[0050] Figure 1 A flow chart of the 3D vision-guided method of the present invention is shown. DETAILED DESCRIPTION

[0051] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, wherein the illustrative embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0052] The present invention provides a robot 3D visual guidance method based on omnidirectional low-rank constraint denoising. This method improves the quality of 3D point cloud data collected by the robot by removing noise interference in point cloud data, thereby achieving higher-precision 3D visual guidance control in complex industrial operation scenarios. This method is based on omnidirectional low-rank constraint denoising technology, which can effectively reduce the impact of multiple factors such as equipment errors, environmental interference and object surface characteristics on point cloud data, and improve the clarity and reliability of the data.

[0053] As attached Figure 1 As shown in the figure, the specific implementation scheme of the present invention consists of the following four steps: point cloud data preprocessing, three-dimensional high-order tensor sequence construction, denoising and denoising point cloud generation. Each step is implemented through different technical means and algorithms, which are closely linked to each other to ensure that the final point cloud data has high quality and high applicability.

[0054] The specific steps are as follows:

[0055] Point cloud data preprocessing:

[0056] In the preprocessing stage of point cloud data, the original point cloud data V collected by the robot is first processed by the downsampling method to reduce the amount of data and improve the computational efficiency. Then, a set of seed points S is randomly selected from the downsampled point cloud data as reference points. The selection of the seed point set directly affects the accuracy and stability of the subsequent steps. For each seed point s∈S, the K nearest neighbor points around it are selected to form a K×3 block matrix A s , block matrix A s It is used to describe the geometric information of the seed point on its local surface, laying the foundation for the construction of the subsequent data structure. Through the construction of this local surface block matrix, the hierarchical nature of data expression and the ability to describe local features can be significantly enhanced.

[0057] Three-dimensional high-order tensor sequence construction:

[0058] After obtaining the local surface block matrix, similar block matrices are stacked to form a third-order tensor. After constructing the third-order tensor, in order to further capture the omnidirectional low-rank characteristics of the point cloud data, the third-order tensor is directional enhanced to generate a three-dimensional high-order tensor sequence. This three-dimensional high-order tensor sequence is the result of multi-directional enhancement of the point cloud data, so that the geometric characteristics of the point cloud in all directions can be more comprehensively described.

[0059] Denoising:

[0060] After the three-dimensional high-order tensor sequence is constructed, denoising is performed. First, Tucker decomposition is performed on the high-order tensor sequence to extract multi-directional core tensor features. The core tensor represents the most important directional information in the data. Then the core tensor is threshold-shrinked to remove components that are irrelevant or interfering to the data. Through this shrinkage operation, only elements in the core tensor that meet the set threshold conditions are retained to reduce noise interference. Finally, the core tensor after threshold processing is inversely transformed with the principal component to generate a denoised three-dimensional high-order tensor sequence. This denoising step significantly improves the clarity of the tensor data, ensuring that the robot can extract reliable visual guidance information from it.

[0061] Denoised cloud generation:

[0062] Finally, the points in the denoised three-dimensional high-order tensor sequence generated by the inverse transform are restored to the original point cloud data positions to form the final denoised point cloud data V * The denoised point cloud data can minimize the impact of noise caused by the environment and equipment, has high precision and high stability, and can effectively support the 3D vision guidance and control tasks of robots in complex industrial environments.

[0063] The various steps of the method of the present invention are interrelated, which not only ensures the comprehensiveness of the denoising process, but also gives full play to the denoising effect of the low-rank constraint, so that the denoised point cloud data has high-quality visual information, providing solid data support and technical guarantee for the application of robots in complex industrial scenes. The specific implementation of each step is as follows:

[0064] Step 1: Point cloud data preprocessing

[0065] This step aims to select representative point sets from the 3D point cloud data collected by the robot, and describe the local feature structure of the data by constructing a local surface block matrix, laying the foundation for subsequent denoising and tensor construction. The point cloud data preprocessing process includes the selection of reference points, the construction of local surface block matrices, the similarity measurement between block matrices, and the generation of third-order tensors.

[0066] The specific steps are as follows:

[0067] From the 3D point cloud dataset V collected by the robot, a set of seed point sets S is randomly selected as reference points through the downsampling method. Downsampling can reduce the amount of data while maintaining the structural features of the point cloud, making the processing more efficient. The selection of the seed point set S is mainly to extract the key features of the point cloud, thereby simplifying the data structure and improving the speed and stability of subsequent calculations. The selected seed point set S, as a representative sample, reflects the core geometric features of the overall point cloud to a certain extent, and provides a basis for the next step of building the block matrix.

[0068] For each seed point s∈S, select the K nearest neighbors of the point in the point cloud data V and map these neighbors to a K×3 block matrix A. s Specifically, the block matrix A s It can be expressed as:

[0069] A s =[a s1 ,a s2 ,…,a sK ]

[0070] Among them, a si =[x si ,y si ,z si ] is the neighboring point s i Coordinates in three-dimensional space. The block matrix A s It describes the spatial geometric relationship of the seed point s within its local range and can be regarded as a three-dimensional spatial characterization of the local surface features of the seed point. By constructing a local surface block matrix, the local structural characteristics of the point cloud at different positions can be more clearly expressed, providing reliable data support for the similarity analysis and third-order tensor construction in the subsequent steps.

[0071] After obtaining multiple block matrices, in order to identify the similarity of local structures in point cloud data, this step uses the iterative closest point (ICP) algorithm to calculate the similarity distance between any two block matrices A and B, denoted as d(A,B). The ICP algorithm calculates the similarity between two block matrices by minimizing the distance between corresponding points and iteratively updating the pose transformation parameters. The specific conditions are that for each block matrix A, find all similar block matrices B that meet the following conditions:

[0072]

[0073] Where δ is a preset constant used to control the threshold of the similarity measure. In this way, the block matrices can be effectively grouped according to similarity to form multiple similar block matrix sets. These similar block matrix sets represent areas with similar geometric features in the point cloud data. Subsequent steps will use these similarity features to construct efficient tensor representations.

[0074] Based on the similarity measurement results, the block matrices in each similar block matrix set are stacked in order to form a third-order tensor Among them, K represents the number of rows of each block matrix, 3 represents the coordinates in three-dimensional space, and M represents the number of similar block matrices. The constructed third-order tensor X aggregates the data features of all similar block matrices, providing structured data support for the subsequent construction of three-dimensional high-order tensor sequences. This tensor construction method can better capture the local features of point cloud data in different areas, while retaining the geometric structure information between similar block matrices, thus laying the foundation for tensor decomposition and data enhancement operations in the subsequent denoising process.

[0075] Through the above steps, the point cloud data is gradually converted from the original collected 3D point cloud data into a third-order tensor with rich local feature information, which provides support for the subsequent three-dimensional high-order tensor sequence construction and denoising. This point cloud data preprocessing step not only reduces the impact of noise interference, but also greatly improves the accuracy and stability of point cloud data processing in the robot's 3D vision guidance task.

[0076] Step 2: Three-dimensional high-order tensor sequence construction

[0077] After the third-order tensor X is constructed, the next step is to perform directional enhancement processing on it to form a higher-order tensor sequence, which is convenient for further capturing low-rank feature information from multiple directions. This step aims to express the geometric structure features of the point cloud data in a higher dimension by enhancing the tensors along different directions, thereby laying a richer data foundation for subsequent denoising processing.

[0078] The specific steps are as follows:

[0079] In this step, the third-order tensor Perform tensor enhancement processing along different directions to obtain a three-dimensional high-order tensor sequence

[0080] It is the first tensor in the three-dimensional high-order tensor sequence, representing the high-order tensor obtained by tensor enhancement processing of X along the first direction (dimension K).

[0081] Where K retains the original dimension and the other dimensions are expanded through tensor augmentation.

[0082] It captures the low-rank features of point cloud data in the first direction and extracts global features along the dimension of the number of neighborhood points.

[0083] It is the second tensor in the three-dimensional high-order tensor sequence, representing the high-order tensor obtained by tensor enhancement of X along the second direction (dimension 3, that is, the three-dimensional coordinates x, y, z).

[0084] Among them, 3 represents the three-dimensional space coordinates, and the other dimensions are expanded through tensor enhancement.

[0085] Capture the characteristics of point cloud data in the coordinate dimension and analyze the geometric structure and spatial distribution of point clouds.

[0086] It is the third tensor in the three-dimensional high-order tensor sequence, representing the high-order tensor obtained by tensor enhancement processing of X along the third direction (dimension M, that is, the number of similar block matrices).

[0087] Where M represents the number of similar block matrices, and other dimensions are expanded through tensor augmentation.

[0088] The characteristic relationship between similar block matrix groups of point cloud data is extracted to capture the global characteristics between different block matrices.

[0089] These three high-order tensors are enhanced along different directions to capture the low-rank characteristics of point cloud data in the neighborhood point dimension, three-dimensional spatial coordinate dimension, and similar block matrix number dimension. Together, they form a three-dimensional high-order tensor sequence Used to describe the omnidirectional characteristics of point cloud data.

[0090] Each high-order tensor in this tensor sequence contains different directional information, capturing the geometric characteristics of the point cloud data in different dimensions and directions.

[0091] Specifically, the constructed three-dimensional high-order tensor sequence The dimensions of each tensor in are:

[0092] Tensor

[0093] Defined in Dimension K, and the condition is met

[0094]

[0095] That is, the dimensions 3 and M in X are expanded to multiple dimensions through tensor enhancement operations The structural information on the original dimension K is preserved in this process.

[0096] Tensor

[0097] Defined in Dimension 3 and above, and the conditions are met

[0098]

[0099] That is, the dimension 3 of the third-order tensor X is retained, and the dimensions K and M are converted to a multi-dimensional structure Further refinement of directional information is achieved.

[0100] Tensor

[0101] Defined in Dimension M, and satisfy the condition

[0102]

[0103] That is, the dimension M of the third-order tensor X is retained, and the dimensions K and 3 are converted into a higher-dimensional structure to further improve the directional expression ability of the point cloud data.

[0104] Through this tensor enhancement process, the third-order tensor X is expanded into a higher-dimensional tensor structure along different directions. The generated high-order tensor sequence is It can capture the low-rank features of point cloud data from multiple angles and levels, providing richer information support for identifying and processing data characteristics in different directions in the subsequent denoising process.

[0105] The tensor enhancement operation in this step enables each high-order tensor to carry unique directional feature information, thereby effectively capturing the low-rank features of the point cloud data in all directions in three-dimensional space.

[0106] Step 3: Denoising

[0107] In obtaining a three-dimensional high-order tensor sequence Finally, the purpose of denoising is to effectively remove the noise components in the data through multi-stage mathematical operations and retain the key structural features in the point cloud data. Denoising includes operations such as Tucker decomposition, threshold shrinkage and inverse transformation. The specific steps are as follows:

[0108] This step first performs a three-dimensional high-order tensor sequence Perform Tucker decomposition to decompose it into core tensors and principal components in multiple directions. Specifically, Tucker decomposition decomposes the tensor Represented as a set of low-rank approximations, where:

[0109]

[0110] Among them, α l is the weight coefficient in each direction l, reflecting the importance of each direction; G l The core tensor for each direction captures the key features of that direction; For all dimensions The principal component matrix on represents the main changes of point cloud data in different dimensions. Through Tucker decomposition, the core features in each direction can be effectively extracted, thereby removing redundant information and noise in the data.

[0111] The main purpose of Tucker decomposition is to reduce the dimension of high-dimensional point cloud data and decompose it into a low-rank matrix, making the structure of the point cloud data more concise and suppressing noise. This decomposition process can remove redundant information in the point cloud data, improve the quality of the data, and lay the foundation for subsequent threshold shrinkage and inverse transformation processing.

[0112] In order to further remove the noise component, the core tensor G is subjected to threshold shrinkage processing. l Specifically, for each element G l (i,j,k), if its absolute value is greater than the set threshold τ, the original value of the element is retained; otherwise, the element is set to zero. That is, perform the following operations:

[0113]

[0114] in:

[0115] G l Represents the core tensor obtained by Tucker decomposition in direction l. The core tensor G l is the original three-dimensional high-order tensor data The compressed representation of contains the main feature information of the point cloud data in this direction, and may also be mixed with noise components.

[0116] G l (i,j,k) is the core tensor G l The specific element values ​​at the three index positions i, j, and k represent a specific value in the core tensor, which may reflect the characteristics or noise information of a local area of ​​the point cloud data.

[0117] τ is a preset threshold used in the threshold shrinkage process to distinguish valid features from noise in the core tensor.

[0118] The effect is that if the absolute value of an element |G l If (i,j,k)| is greater than τ, the element is considered to be of great significance to the point cloud data and should be retained; otherwise, the element is considered to be noise and should be set to zero.

[0119] is the core tensor after threshold shrinkage. Compared with the original core tensor G l , The noise elements in the image have been removed, and only the feature information that is important to the point cloud data is retained.

[0120] It is a cleaner and more reliable core tensor that removes noise and retains the key structural features of the original data, providing higher quality basic data for subsequent inverse transformation and point cloud reconstruction.

[0121] Through threshold shrinkage processing, the features that are important to the point cloud data can be retained, while the noise components can be effectively suppressed. Threshold shrinkage helps to reduce errors and inaccuracies caused by noise, making the final point cloud data more accurate and reliable.

[0122] This step is a crucial part of the denoising process because it can effectively remove small fluctuations or abnormal noise components while ensuring that key information is not lost.

[0123] After completing the threshold shrinkage process, the core tensor in each direction is With the corresponding principal component matrix Perform the inverse transformation of Tucker decomposition. The purpose of this step is to combine the low-rank core tensor processed by threshold shrinkage with the principal component matrix to restore its high-dimensional structure and obtain a denoised three-dimensional high-order tensor sequence The specific operations are as follows:

[0124]

[0125] in:

[0126] Represents a 3D high-order tensor sequence after denoising. It is obtained by and the principal component matrix The inverse transform of Tucker decomposition is performed, which reflects the characteristic information of the denoised point cloud data in different directions.

[0127] It is a high-quality tensor sequence that removes noise and retains key features, which is used to further generate the block tensor X * and point cloud data V * Provide the foundation.

[0128] I represents the direction index, and its value range is I=1, 2, 3, corresponding to different directions in the three-dimensional high-order tensor sequence.

[0129] In the formula, l is used to distinguish the processing of the tensor in different directions, ensuring that the features in each direction are reconstructed middle.

[0130] α l is the weight coefficient in each direction l, reflecting the importance of this direction to the overall point cloud data.

[0131] Weight coefficient α lIt is used to adjust the contribution ratio of different directions so as to integrate the information of all directions during the inverse transformation process and generate a high-quality denoised tensor.

[0132] is the core tensor after threshold shrinkage in direction l. It contains the main features of the point cloud data after noise removal. As the low-rank information retained after denoising, It is an important input for the inverse transform and is related to the principal component matrix After combination, the denoised high-dimensional structure can be reconstructed.

[0133] Represents the principal component matrix of direction l in dimension i, which contains the main change mode of point cloud data in this dimension, where the value range of i is 1-N. It is decomposed from the original tensor and is used to describe the feature information in each dimension. In the inverse transformation, the principal component matrix and the core tensor Combine to reconstruct the full tensor.

[0134] × N Represents a tensor-matrix multiplication operation on the Nth dimension. It is used to convert the principal component matrix The feature information is mapped to the core tensor of the corresponding dimension , thereby recovering the high-dimensional structure of the tensor.

[0135] N is the number of dimensions of the tensor. For a three-dimensional tensor, N=3, corresponding to the three-dimensional space of the point cloud data (e.g., x, y, z). It indicates the dimensional structure of the tensor, and the tensor-matrix multiplication operation is completed dimension by dimension in the inverse transformation formula.

[0136] X * It is the denoising block tensor after the inverse enhancement operation, representing the point cloud data after the noise is removed.

[0137] X * By transforming the denoised three-dimensional high-order tensor sequence Converting back to a block matrix structure generates a denoised block tensor that is more consistent with the original point cloud representation.

[0138] Through the inverse transformation, the denoised core tensor Recombine the corresponding principal component matrix to restore the denoised three-dimensional tensor sequence

[0139] Then, after the inverse enhancement operation, the denoised block tensor X is generated * , the block tensor contains the point cloud data after noise removal. The inverse enhancement operation converts the denoised three-dimensional tensor sequence back to the original block matrix structure, eliminates the redundant noise components, and retains the key information and geometric features in the data.

[0140] After inverse transformation and aggregation operations, the denoised block tensor X * It more accurately reflects the real structure and characteristics of the point cloud data, making the denoised point cloud data smoother and more accurate, and suitable for subsequent robot 3D vision guidance tasks.

[0141] In summary, the denoising process effectively removes the noise components in the point cloud data through a series of mathematical operations such as Tucker decomposition, threshold shrinkage and inverse transformation, and retains the key information and structural features of the point cloud data. This process not only improves the quality of the point cloud data, but also provides more accurate and reliable data support for the robot's visual guidance in complex environments.

[0142] Step 4: Denoising cloud generation

[0143] After completing the denoising process and obtaining the denoised block tensor X * After that, the next step is to restore the denoised data to the original spatial coordinate system to generate the final denoised point cloud V * The main goal of this process is to remap the denoised and streamlined point cloud data back to its original three-dimensional spatial position, ensuring that the geometric structure of the point cloud is preserved and effectively eliminating the interference of environmental noise and equipment errors, so that the final generated point cloud data is more accurate and reliable, and can provide the robot with clearer visual guidance information.

[0144] The specific steps are as follows:

[0145] For the denoised block tensor X * For each point in the 3D point cloud, the spatial coordinates of these points are remapped back to the corresponding positions in the original point cloud data according to their relative position and orientation information restored during the denoising process. The position of each point is accurately restored to its correct coordinates in 3D space, eliminating the deviation or distortion caused by noise. This restoration process ensures that the geometric shape and spatial features of the point cloud data are maintained after denoising.

[0146] After restoring all denoised points to their original positions, the resulting point cloud set is the final denoised point cloud V * Compared with the original point cloud data, this point cloud collection greatly reduces the interference introduced by environmental noise, sensor errors and other non-ideal factors, and retains a more realistic and accurate three-dimensional spatial structure. This denoising process not only eliminates data errors caused by the external environment or the device itself, but also improves the quality of the point cloud data, making subsequent visual analysis, path planning and decision making more reliable.

[0147] The denoised point cloud V *It has significant advantages in vision-guided tasks. The accuracy of point cloud data is greatly improved, and the robot can more accurately identify and locate objects in the surrounding environment, improving its ability to perform tasks in complex environments. The denoised point cloud reduces the inaccuracy caused by errors, allowing the robot to work more stably in dynamic or unstable environments and be more robust to external interference.

[0148] The final denoised point cloud V * The point cloud will be used as input data for the robot to use when performing 3D visual guidance. The denoised point cloud provides clear environmental structure information, supporting the robot to perform precise spatial perception, path planning and motion control. Whether in industrial production, logistics warehousing, or in complex operation scenarios, the robot can adjust its behavior and decision-making in real time based on this high-quality point cloud data to ensure the smooth execution of the task.

[0149] Through the denoised point cloud generation in this step, the quality of the point cloud data has been significantly improved, providing a more accurate and reliable foundation for subsequent visual analysis and robot control.

[0150] The above description is only a preferred embodiment of the present invention, so all equivalent changes or modifications made according to the structure, characteristics and principles described in the scope of the patent application of the present invention are included in the scope of the patent application of the present invention.

Claims

1. A robot 3D vision guidance method based on omnidirectional low-rank constraint denoising, characterized by: The following steps are involved: Step 1: Point cloud data preprocessing: Select reference points by sampling from the point cloud data acquired by the robot; For each reference point, points in the neighborhood are selected to construct a matrix describing the local structure; The local matrices are grouped based on similarity analysis to form multiple similar matrix groups; Similar matrix groups are stacked to form a third-order tensor to describe the local features of point cloud data; Step 2: Three-dimensional high-order tensor sequence construction: Performing multi-directional expansion processing on the third-order tensor to generate a three-dimensional high-order tensor sequence to capture low-rank features of the point cloud data in different directions; Step 3: De-noising: Perform decomposition on a three-dimensional high-order tensor sequence to extract core tensors and principal components; Perform threshold shrinkage processing in the core tensor to remove noise by retaining only elements that meet the set conditions; Perform inverse transformation on the core tensor and principal component after threshold shrinkage to generate a denoised three-dimensional high-order tensor sequence, and generate a denoised block tensor through a mapping operation; Step 4: Denoising cloud generation: The points in the denoised block tensor are restored to their original positions to form the final denoised point cloud data for use in robot 3D visual guidance.

2. A robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the point cloud data preprocessing step, a seed point set S is randomly selected from the point cloud data V acquired by the robot through a downsampling method to form reference points.

3. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the point cloud data preprocessing step, K nearest neighbor points are selected for each seed point s∈S to form a K×3 block matrix A s ,in A s =[a s1 ,a s2 ,…,a sK ] Among them, a si =[x si ,y si ,z si ] is point s i The coordinates in three-dimensional space, the value range of i is 1-K; A s It is a K×3 block matrix built around the seed point s; a si Represents the three-dimensional coordinates of the i-th neighborhood point around the seed point s; x si ,y si ,z si They are the neighborhood points s i The x, y, z coordinates in three-dimensional space.

4. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 3, characterized in that: In the point cloud data preprocessing step, the similarity measure d(A, B) between different block matrices A and B is calculated by iterative closest point algorithm, and the similarity measure d(A, B) satisfying the condition is found when grouping. The block matrix B of , where δ is a constant.

5. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 3, characterized in that: In the third-order tensor construction step, similar block matrices are stacked to form a third-order tensor It is represented by X = [A1, A2, ..., A M ] Where M is the number of similar block matrices; A1, A2, …, A M A collection of similar block matrices generated after grouping by similarity.

6. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the three-dimensional high-order tensor sequence construction step, the three-order tensor is Perform tensor enhancement to generate a three-dimensional high-order tensor sequence 7. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the denoising step, the three-dimensional high-order tensor sequence Perform Tucker decomposition.

8. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the denoising step, a threshold shrinkage process is performed, and the core tensor G l The elements satisfy in: G l represents the core tensor obtained by Tucker decomposition in direction l; G l (i,j,k) is the core tensor G l The specific element values ​​at the three index positions i, j, and k; τ is the preset threshold used for threshold shrinkage processing; It is the core tensor after threshold shrinkage.

9. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the denoising step, the core tensor after threshold shrinkage is and Perform inverse transform to obtain the denoised three-dimensional high-order tensor sequence And generate the denoised block tensor X through the inverse enhancement operation * .

10. The robot 3D vision guidance method based on omnidirectional low-rank constraint denoising as claimed in claim 1, characterized in that: In the denoised point cloud generation step, the denoised block tensor X * The points in the point cloud are restored to their original positions to form the final point cloud data V after denoising. * .

Images