Three-dimensional recognition vision accuracy matching optimization method and system for industrial robot

Abstract

This invention discloses a method and system for optimizing visual accuracy matching in 3D recognition for industrial robots, relating to the field of visual guidance technology. The method includes the following steps: during the normal movement of the industrial robot, acquiring motor parameters, IMU data, and visual images of the work area; constructing and selecting 3D probability models for various working conditions; predicting the 3D pose distribution for the next sampling period; and outputting a pose prediction sequence composed of multiple candidate pose points. Under the condition of triggering a pose mutation, a preset rule engine is invoked to reconstruct the 3D visual image, plan the motion trajectory of the industrial robot, perform a matching between the motion trajectory and the reconnection and alignment pose, and select the optimal reconnection and alignment pose. This invention improves the visual matching accuracy in weak texture and smooth scenes through texture determination and the construction of full disparity cost analysis, effectively improving the adaptability and efficiency of industrial robot reconnection and alignment scenarios.

Description

Technical Field

[0001] This invention relates to the field of visual guidance technology, specifically to a method and system for optimizing the matching of 3D visual accuracy in industrial robots. Background Technology

[0002] With the development of industrial automation, monocular or binocular stereo vision is the core technology for industrial robots to perform 3D visual recognition. During the operation of industrial robots, abnormal shutdowns and power outages often occur. It is necessary to use binocular vision to collect on-site images, complete 3D reconstruction and pose calculation, select reconnection and alignment poses, and plan motion trajectories to achieve seamless operation.

[0003] Traditional visual guidance technologies suffer from the following drawbacks: On the one hand, they typically use region-based stereo matching algorithms, such as BM and SGM algorithms. While these algorithms are simple and fast, industrial work environments often contain areas with inconsistent textures, such as smooth metal workpiece surfaces, solid-color conveyor belt backgrounds, and low-texture frosted surfaces. This inconsistency in texture types can easily lead to large areas of holes and abrupt changes in the final disparity map, making it impossible to generate dense 3D point clouds. Ultimately, this significantly increases the deviation in pose calculation and reduces the visual matching accuracy in weakly textured and smooth scenes. On the other hand, with the development of deep learning technology, matching schemes based on convolutional neural networks have emerged. These schemes extract depth features from visual images indiscriminately through multi-kernel convolution and apply uniform weights and iterative strategies across the entire disparity range. However, this not only easily leads to insufficient localization details for small disparities and redundancy for large disparities, but also greatly increases computational complexity, reducing matching accuracy and visual recognition efficiency. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a method and system for optimizing 3D visual accuracy matching in industrial robots. During the normal movement of the industrial robot, a 3D probability model for various working conditions is constructed and selected using motor parameters, IMU data, and visual images, outputting a 3D pose distribution. By triggering pose mutations, the matching correlation degree and geometric residual value are combined to reconstruct the 3D of the real-time visual image. Through texture judgment and matching cost analysis, the motion trajectory is obtained. A single matching operation can be performed to select the optimal reconnection and alignment pose, driving the industrial robot to timely position and correct, thus solving the problems mentioned in the background technology.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In the first aspect, this application provides a method for optimizing the matching of visual accuracy for 3D recognition of industrial robots. The method includes: during the normal movement of the industrial robot, acquiring motor parameters, IMU data and visual images of the area to be worked, constructing and selecting a 3D probability model for multiple working conditions, predicting the 3D pose distribution of the next sampling period, and outputting a pose prediction sequence composed of multiple candidate pose points. Under the condition of triggering a pose change, a preset rule engine is invoked to reconstruct the 3D of the visual image, plan the motion trajectory of the industrial robot, perform a matching between the motion trajectory and the reconnection and alignment pose, and select the optimal reconnection and alignment pose. Furthermore, various operating conditions include low-speed operating conditions, high-speed operating conditions, and disturbance operating conditions, including: Simultaneously acquire motor parameters output by the industrial robot controller, IMU data output by the end effector, and visual images of the work area output by the vision acquisition device; The angular velocity and linear velocity arrays of IMU data are identified, a 3D point cloud is generated based on the visual image, and the edge blur array of the 3D point cloud is extracted. Through temporal alignment, the mean and variance of each array are extracted to construct a multi-dimensional working condition vector. The edge blur array is obtained using a 3D LBO operator. Using a multi-dimensional working condition vector with a preset sampling period as a sample set, feature mapping is performed through a preset working condition detection model to output multiple working condition states at N future times; where N is a positive integer greater than 0.

[0006] Furthermore, it also includes: performing a variational mode decomposition algorithm on the motor parameters to obtain motion modes, including a first mode and a second mode.

[0007] Furthermore, three-dimensional probabilistic models for various working conditions are constructed and selected, including: Low-speed operation: Activate the first encoder, extract the number of effective feature points in the visual image, retrieve the geometric features of the current target workpiece, generate a visual mask by combining the first modality, filter the time when the number of effective feature points is less than the standard threshold, and at the corresponding time, use the geometric features as the query vector and the visual mask as the key value, obtain the attention weight through the attention mechanism, and complete the visual feature enhancement. High-speed operation: Activate the second encoder, extract the amplitude distribution of the first mode, establish the mapping relationship between angular velocity / linear velocity and amplitude distribution, determine the motion trend, extract the energy proportion of the second mode, filter out cases where the energy proportion is less than the standard proportion threshold, fuse the features of visual image and IMU data, and automatically adjust the fusion weights according to the temporal proportion of motion trend type to complete motion feature enhancement; among which, motion trends include constant speed, variable speed, and start-stop. Interference condition: Activate the third encoder, retrieve visual images, IMU data and motor parameters of the same sampling period, and complete the multi-source feature fusion through channel cascading and dynamic weighting mechanism to obtain the enhanced features after fusion; Using various enhanced features acquired under different working conditions as input, and minimizing pose deviation as the objective function, a three-dimensional probabilistic model is constructed.

[0008] Furthermore, triggering pose abrupt changes includes: The visual image of the next sampling period is acquired, and each candidate pose point in the pose prediction sequence is spatially projected and matched with the visual image. The matching correlation degree and geometric residual value corresponding to each candidate pose point are calculated. Visual images with matching correlation degree less than the standard matching threshold are selected and determined to have a pose change.

[0009] Furthermore, a preset rule engine is invoked to perform 3D reconstruction of the visual image, including: Based on visual images, texture determination is performed through Fourier transform to identify texture types, and different algorithms are selected to perform feature extraction according to texture types to generate target feature maps; among them, texture types include continuous textures, non-continuous textures, and contour textures; A full disparity cost body is constructed based on the target feature map. By dividing the disparity range of the region to be worked on, differential disparity optimization is performed through a selective fusion iterative network. Adaptive attention weights are selected within different disparity ranges. The cost body corresponding to each disparity range is linearly summed with the attention weights to generate the fusion matching cost. The disparity range includes small disparity range, medium disparity range, and large disparity range. Perform disparity regression on the fusion matching cost volume to generate a disparity map, convert the disparity map into three-dimensional spatial coordinates, and simultaneously map the texture type to the corresponding three-dimensional point coordinates to generate a three-dimensional point cloud with texture type.

[0010] Furthermore, the optimal reconnection alignment pose is sent to the edge layer, driving the industrial robot to follow the changes in the optimal reconnection alignment pose until visual alignment is achieved.

[0011] Furthermore, the process of triggering pose mutation also includes: constructing a comprehensive evaluation function based on the matching correlation degree and geometric residual value through weighted linear summation, and selecting the candidate pose point with the highest comprehensive score from the pose prediction sequence as the reconnection and alignment pose.

[0012] Secondly, this application provides a 3D recognition visual accuracy matching optimization system for industrial robots. The system includes: a first analysis module, which acquires IMU data, visual images and IMU data of the work area during the normal movement of the industrial robot, constructs and selects a 3D probability model of multiple working conditions, predicts the 3D pose distribution of the next sampling period, and outputs a pose prediction sequence composed of multiple candidate pose points. The second analysis module, under the condition of triggering a pose change, calls the preset rule engine to reconstruct the three-dimensional image of the visual image, plans the motion trajectory of the industrial robot, performs a matching between the motion trajectory and the reconnection and alignment pose, and selects the optimal reconnection and alignment pose.

[0013] (III) Beneficial Effects This invention provides a method and system for optimizing the matching of 3D visual accuracy in industrial robots, which has the following beneficial effects: 1. This invention classifies textures into three types: continuous, non-continuous, and contour. It matches corresponding feature extraction algorithms and multi-kernel convolution kernels to different texture types, which not only accurately preserves the high-frequency details of contour textures such as workpiece edges and holes, but also effectively suppresses noise redundancy in smooth backgrounds, improves adaptability to different texture regions, and greatly enhances the matching accuracy and robustness of 3D visual recognition under all working conditions. 2. By dividing the full parallax range into three categories—small, medium, and large—and matching appropriate attention weights to different parallax ranges, weighted fusion is used to generate a fusion matching cost. This improves the accuracy of the parallax map while reducing the computational load, thus meeting the real-time requirements of industrial sites. 3. By constructing a comprehensive evaluation function of matching correlation degree and geometric residual value, and linking it with the motion trajectory of industrial robot, a matching optimization is completed. The optimal reconnection and alignment pose selected not only ensures the reliability of visual calculation, but also takes into account the efficiency and safety of robot motion. 4. This invention achieves low-latency forwarding of optimal pose and incremental visual servo correction through the edge layer, which greatly improves the response speed of reconnection alignment and the final positioning accuracy, meeting the needs of high-speed and high-precision operation of industrial robots. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the present invention; Figure 2 This is a schematic diagram of the modules of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0016] The core of this invention lies in constructing and selecting a three-dimensional probability model for various working conditions during the normal movement of an industrial robot using motor parameters, IMU data, and visual images, and outputting a three-dimensional pose distribution. By triggering pose mutations, the matching correlation degree and geometric residual value are combined to reconstruct the three-dimensional image of the real-time visual image. In this process, a disparity map is obtained through texture judgment and the construction of full disparity cost analysis, forming a motion trajectory to perform a matching, select the optimal reconnection and alignment pose, and drive the industrial robot's positioning correction, thereby improving the positioning accuracy and operational stability of the industrial robot in high-precision operations.

[0017] Example 1: This invention provides a method for optimizing the matching of 3D visual accuracy in industrial robots. Figure 1 This is a flowchart illustrating the invention; please refer to [link / reference]. Figure 1 The method includes the following steps: S1: During the normal movement of the industrial robot, acquire motor parameters, IMU data and visual images of the work area, construct and select three-dimensional probability models for various working conditions, predict the three-dimensional pose distribution of the next sampling period, and output a pose prediction sequence composed of multiple candidate pose points. The acquisition steps for various operating conditions, including low-speed, high-speed, and interference conditions, include: Within a preset sampling period, the system synchronously acquires motor parameters output by the industrial robot controller, IMU data output by the end effector, and visual images of the work area output by the vision acquisition device. It should be noted that the industrial robot main controller acts as a trigger source, sending synchronization pulse signals to the IMU inertial unit of the end effector and the vision acquisition device. A unified global timestamp is assigned to each set of acquired motor parameters, IMU data, and image frames, ensuring sub-millisecond alignment of data from different sampling frequencies on the time axis. Specifically, the industrial robot controller acquires motor parameters, including but not limited to the current components, output torque, and encoder feedback pulses of each drive joint motor. The vision acquisition device can be a monocular or binocular vision camera, acquiring IMU data through the IMU inertial unit built into the industrial robot end effector to determine the robot's motion status. A 3D point cloud is generated based on visual images, and the edge blur array of the 3D point cloud is extracted. The edge blur array is obtained using a 3D LBO operator. It should be noted that if a monocular vision camera is used, the real-time visual image is directly output, and a standard CAD model of the target workpiece is preset. Feature extraction and template matching are performed on a single frame of monocular image. The 6-DOF pose of the target workpiece relative to the camera is calculated using the PnP algorithm. Then, combined with the CAD model of known dimensions, a high-precision 3D point cloud of the target workpiece in the corresponding pose is generated. If a binocular vision camera is used, the binocular vision camera is synchronously triggered by hardware to acquire two 2D visual images of the left and right eyes at the same time with strictly aligned timestamps. After calibration correction and epipolar alignment preprocessing of the binocular images, a disparity map is generated using a stereo matching algorithm. Based on the binocular camera calibration parameters and triangulation principle, the 3D coordinates of the disparity map are calculated to generate a 3D point cloud corresponding to the sampling period, which is used to determine the motion posture of the industrial robot in the working condition. For ease of analysis, only the case of two 2D visual images of the left and right eyes will be analyzed in the following analysis. The edge blur array represents the sequence of edge blur levels of the 3D point cloud in the working area within a preset sampling period. The calculation process is as follows: the 3D point cloud is preprocessed, including filtering to remove outliers and flying points, and voxel filtering, to obtain the preprocessed point cloud of the target workpiece's region of interest; the edge response of the 3D point cloud is extracted using the 3D LBO operator, and the variance of the edge response is calculated. This variance is used as the motion blur level of a single frame. The smaller the variance, the greater the noise in the 3D point cloud and the more blurred the visual image; the larger the variance, the greater the noise in the 3D point cloud and the clearer the image. In addition, the faster the linear velocity and the greater the angular velocity, the more pixel displacement occurs during the camera's exposure time, and the higher the edge blur level calculated by the Laplacian operator. Therefore, in subsequent high-speed conditions, the dominant role of vision needs to be abandoned. Simultaneously, the angular velocity array and linear velocity array of the IMU data are identified; through time alignment, the mean and variance of each array are extracted to construct a multi-dimensional working condition vector; it should be noted that the linear velocity is represented as the real-time linear velocity sequence of the industrial robot end effector in Cartesian space within a preset sampling period, and the angular velocity is represented as the real-time angular velocity sequence of the industrial robot joint space within a preset sampling period. Using a multi-dimensional operating condition vector with a preset sampling period as a sample set, a preset operating condition detection model is used to perform feature mapping, outputting various operating condition states at N future time points, and labeling them; where N is a positive integer greater than 0; it should be noted that the operating condition detection model uses an LSTM architecture, using the multi-dimensional operating condition vector as an input set, and models the evolution trend of operating condition states in historical sampling periods through LSTM, and finally classifies attributes on the predicted sequence at N future time points based on the predicted output of the mapping function, including low-speed operating conditions, high-speed operating conditions, and interference operating conditions.

[0018] A three-dimensional probabilistic model for multiple operating conditions is constructed and selected, including: performing variational mode decomposition algorithm on IMU data to obtain motion modes, including a first mode and a second mode; wherein, the motion modes include a first mode characterizing motion stability and a second mode characterizing noise; Under low-speed conditions: the first encoder is activated, visual images from the same sampling period are retrieved, the number of valid feature points is extracted, and the number of valid feature points is compared with a standard threshold. If the number of valid feature points is less than the standard threshold, the standard CAD model of the target workpiece is retrieved, and the geometric feature library constructed offline is pre-trained to extract the geometric features of the current target workpiece. A visual mask is generated by combining the first modality. At the same time, if the number of valid feature points is less than the standard threshold, the corresponding time is obtained, and the geometric features are used as the query vector, the visual mask is used as the key, and attention weights are obtained through an attention mechanism to obtain enhanced visual features. Conversely, if the number of valid feature points is greater than or equal to the standard threshold, the geometric features of the target workpiece are directly extracted as the enhanced visual features. It should be noted that the effective feature points are image spatial feature vectors whose response values ​​are within a preset saliency range after being extracted by the first encoder. They reflect the texture richness and imaging quality of the area to be worked on, facilitating subsequent 3D visual recognition and matching. The standard number threshold is based on historical statistics. It analyzes the number of image spatial feature vectors whose visual image response values ​​are within the preset saliency range during past operations, performs statistical analysis on the collected data, determines the mean and standard deviation, and sets the standard number threshold based on the statistical results. The standard number threshold is set as the mean number of effective feature points plus twice the standard deviation. Of course, this multiple is just an example, and the specific value can be set and adjusted according to the actual situation, which will not be elaborated here. Under high-speed conditions: the second encoder is activated, IMU data from the same sampling period is retrieved, the amplitude distribution of the first mode is extracted, a mapping relationship between angular velocity / linear velocity and amplitude distribution is established, the motion trend is determined, the energy proportion of the second mode is extracted, and the energy proportion is compared with a standard proportion threshold. If the energy proportion is less than the standard proportion threshold, the features of the visual image and IMU data are fused, and the fusion weight is automatically adjusted according to the temporal proportion of the motion trend type to complete the motion feature enhancement; conversely, if the energy proportion is greater than or equal to the standard proportion threshold, the positioning accuracy is determined to be lost, and the features of the IMU data are directly extracted as the enhanced motion features; among which, the motion trend includes constant speed, variable speed, and start-stop. It should be noted that during the operation of an industrial robot, different motion trends are recorded, along with the energy distribution of the first and second modes generated by the IMU and corresponding encoders. Through historical statistics, the energy distribution of the second mode under a certain motion trend without loss of positioning accuracy in past operations is recorded, and the average value and standard deviation of the energy distribution under this condition are determined. Based on the statistical results, a standard threshold is set, which is the average value of the energy distribution of the second mode without loss of positioning accuracy plus, for example, 2 or 3 times the standard deviation. This is used to calculate the standard percentage threshold of the second mode that can guarantee no loss of positioning accuracy under different motion trends. Of course, this multiplier is just an example, and the specific value can be set and adjusted according to the actual situation, which will not be elaborated here. Interference condition: Activate the third encoder, retrieve visual images, IMU data and IMU data from the same sampling period, and complete multi-source feature fusion through channel cascading and dynamic weighting mechanism to obtain the fused enhanced features.

[0019] For different working conditions, various enhanced features are acquired as inputs. The objective function is to minimize the pose prediction error. The next sampling period is selected as N future time moments. The three-dimensional pose distribution of the next sampling period is predicted and represented by probability values. Multiple candidate pose points are sampled from the three-dimensional pose distribution and sorted in descending order of probability value. The output constitutes the pose prediction sequence. The pose prediction error is the Euclidean distance between the predicted pose and the subsequent measured pose.

[0020] S2: Under the condition of triggering a pose change, the preset rule engine is called to reconstruct the three-dimensional image of the visual image, plan the motion trajectory of the industrial robot, perform a matching between the motion trajectory and the reconnection and alignment pose, and select the optimal reconnection and alignment pose. Triggering a pose mutation includes: acquiring the visual image for the next sampling period, performing spatial projection matching between each candidate pose point in the pose prediction sequence and the visual image, and calculating the matching correlation degree and geometric residual value corresponding to each candidate pose point. The matching correlation degree calculation includes: identifying the camera pose corresponding to the candidate pose point, retrieving the pre-installed standard CAD model of the target workpiece in the system and projecting it onto the visual image to generate a projected contour; extracting the actual contour of the current target workpiece from the visual image, calculating the intersection-union ratio (IU) between the projected contour and the actual contour, and using this IU as the matching correlation degree. The geometric residual value calculation includes: extracting the effective feature points of the current target workpiece from the visual image, back-projecting the effective feature points to three-dimensional space based on the candidate pose points, calculating the Euclidean distance between the back-projected points and the corresponding points of the standard CAD model, and using the average value of this Euclidean distance as the geometric residual value. The matching correlation is compared with a standard matching threshold. If the matching correlation is less than the standard matching threshold, the current visual image is determined to have experienced a pose change; conversely, if the matching correlation is greater than or equal to the standard matching threshold, the current visual image is determined to have a normal pose. It should be noted that the standard matching threshold is based on historical statistics. Historical matching correlation data that has not triggered pose changes are collected, and the collected data is statistically analyzed to determine the average and standard deviation of the matching correlation under the condition that no pose change has been triggered. The standard matching threshold is set according to the statistical results. The standard matching threshold is set as the average matching correlation under the condition that no pose change has been triggered plus three times the standard deviation. Of course, this multiple is just an example, and the specific value can be set and adjusted according to the actual situation, which will not be elaborated here.

[0021] Under conditions that trigger a sudden change in pose, a preset rule engine is invoked to perform 3D reconstruction of the visual image, including: Based on visual images, texture determination is performed using Fourier transform, including: taking the left and right two-dimensional visual images captured by a binocular camera as an example, the visual images are first corrected. For the corrected left visual image, a two-dimensional fast Fourier transform is performed to obtain the frequency domain amplitude spectrum. Using the DC component at the center of the spectrum as the center, the spectrum is divided into low-frequency, mid-frequency, and high-frequency bands, and five types of frequency domain features are extracted: low-frequency energy proportion, mid-frequency energy proportion, high-frequency energy proportion, spectral concentration, and spectral directional entropy. In this embodiment, three types of texture determination can be completed by setting preset thresholds. For example, for continuous textures, a low-frequency energy proportion greater than or equal to 0.85, a spectral concentration greater than or equal to 0.7, and a mid-frequency energy proportion less than or equal to 0.1 can be set, corresponding to smooth gold... For uniform low-frequency regions such as surfaces and solid-color backgrounds, and for contour-type textures, the mid-frequency energy ratio can be set to be greater than or equal to 0.6, the spectral direction entropy to be less than or equal to 0.3, and the low-frequency energy ratio to be less than or equal to 0.3, corresponding to high-frequency regions such as workpiece edges, holes, and slots. For discontinuous textures, the high-frequency energy ratio can be set to be greater than or equal to 0.4, the spectral concentration to be less than or equal to 0.2, and not meeting the above two conditions, corresponding to weak texture regions such as frosted surfaces and low-contrast sprayed surfaces. In this way, a texture label map with the same size as the left visual image is generated and marked as the texture type. The right visual image is similar and will not be described in detail here. It should be noted that the above threshold values ​​are just examples and should be set according to the actual situation. By using Fourier transform for texture determination, the impact of lighting fluctuations and noise interference on texture recognition in industrial scenarios can be effectively avoided. At the same time, it can accurately distinguish between real smooth continuous textures and pseudo-continuous textures caused by motion blur. This provides a highly reliable texture benchmark for subsequent effective feature point hierarchical screening and parallax cost tiered optimization, greatly improving the matching accuracy and robustness of 3D vision recognition under all working conditions. Texture types include continuous textures, non-continuous textures, and outline textures: For continuous textures, the 0th feature channel is selected, and a 3×3 convolution kernel is used in conjunction with the Poincaré exponent algorithm to perform feature extraction. Specifically, this includes: First, identifying all data points in the visual image, since the key attribute points of different target workpiece feature images have different Poincaré exponents. Then, a data point is randomly selected as the initial center point, and its position in high-dimensional space is obtained. The Poincaré distance can be used in high-dimensional space to measure the distance between data points. The geometric center of the dataset is approximated through iteration until the maximum number of iterations is reached, so that the data points better represent the relationship between the original data in low-dimensional space. Finally, more discriminative feature information can be extracted. For discontinuous textures, the first feature channel is selected, and a 5×5 convolution kernel is used in conjunction with the gray-level co-occurrence matrix (GLCM) method to perform feature extraction. Specifically, this involves: first, identifying all pixels in the visual image and defining a gray-level offset direction, typically set to four directions: horizontal, vertical, 45°, and 135°. Then, for each pixel, the frequency of gray-level value pairs occurring in adjacent pixels along the specified offset direction is calculated. Based on the calculation results, a GLCM is constructed, where the rows and columns of the matrix represent gray levels, and each element in the matrix represents the frequency of occurrence of the corresponding gray-level pair. Finally, more discriminative feature information can be extracted. For contour-type textures, the second feature channel is selected, and a 7×7 convolution kernel is used in conjunction with Freeman chain code to perform feature extraction. Specifically, this includes: First, identifying the image contour of the visual image, extracting the boundary of the contour to obtain a sequence of points on the contour, and representing all point sequences with a series of numbers to generate Freeman chain code, where each number represents the direction of movement from the current point to the next point, typically using numbers from 0 to 7 to represent 8 possible directions of movement; then, by performing derivative operations on the Freeman chain code sequence, the difference between two adjacent numbers is calculated to obtain a derivative sequence, which is further processed, such as smoothing and normalization, to finally extract more discriminative feature information; The output channels of 3×3, 5×5, and 7×7 convolutional kernels are allocated in a 7:2:1 ratio. The features extracted from the three feature channels are fused using a multi-scale U-shaped upsampling and downsampling structure, resulting in a three-channel target feature map that is 1 / 4 the size of the visual image, including left and right eye feature maps. A full disparity cost volume is constructed based on the target feature map by dividing the disparity range of the working region into small, medium, and large disparity ranges. Differential disparity optimization is performed through a selective fusion iterative network, selecting appropriate attention weights for different disparity ranges. The cost volume corresponding to each disparity range is weighted and summed with the attention weights to generate a fusion matching cost. It should be noted that the cost volume corresponding to each parallax range is obtained by slicing according to the parallax range to generate three independent sub-cost volumes. The preset maximum parallax range is 768px; the small parallax range is [0, 192px], corresponding to distant outline textures; the medium parallax range is [0, 384px], corresponding to mid-range discontinuous textures; and the large parallax range is [0, 768px], corresponding to near-view continuous textures. For the cost volume of the small parallax range, its dimension is (H / 4) × (W / 4) × 192, and for the cost volume of the medium parallax range, its dimension is (H / 4). ×(W / 4)×384, For the cost volume with a large disparity range, the dimension is (H / 4)×(W / 4)×768; where H is the height and W is the width; These three cost volumes with different disparity ranges are only constructed based on the disparity range, which contain a lot of noise and ambiguity, and the cost volume is lacking in feature capture in the global range and spatial interaction between various feature channels. In the calculation process, the cost volume needs to be regularized. The values ​​mentioned above are for illustrative purposes only. The specific values ​​should be set according to the actual situation and will not be elaborated here. In the weighted summation, the attention weights corresponding to the small, medium, and large parallax ranges are dynamic weights based on adaptive optimization using a genetic algorithm. Specifically, this involves: setting up a high-dimensional space; randomly generating an initial population of m individuals (where m is a positive integer greater than 0); and each individual representing a set of candidate attention weight combinations corresponding to the small, medium, and large parallax ranges, respectively. During initialization, a certain number of individuals are randomly generated to ensure all weight values ​​are within a suitable range, strictly limited to 0 to 1, and the sum of the three weight values ​​is forced to be 1. The fitness of each individual is calculated, and individuals with high fitness are selected as parents. A roulette wheel selection strategy is used to sequentially select particles from the population and place them into the mating pool until the number of particles in the mating pool reaches m. Higher fitness... The higher the probability of an individual being selected as a parent, the more likely it is to be replaced. New individuals are then added to the population through crossover operations to update its composition. The crossover process involves: selecting a crossover point, splitting the parent's genes into two parts, exchanging the two parts, and generating two offspring. Crossover is not performed on every parent pair; a crossover probability is typically set, for example, a 70% probability of crossover and a 30% probability of directly replicating the parent to the next generation. Mutation is then performed on the new individuals after crossover. This selection, crossover, and mutation process is repeated until a preset iteration termination condition is met. After iteration termination, the individual with the highest fitness value in the current population is selected, and its corresponding weight vector becomes the final attention weights for small, medium, and large disparity ranges, used for subsequent weighted summation of fusion matching costs. Based on texture determination, textures are classified into three types: continuous, non-continuous, and contour. Corresponding feature extraction algorithms and multi-kernel convolution kernels are matched for different texture types. This not only accurately preserves the high-frequency details of contour textures such as workpiece edges and holes, but also effectively suppresses noise redundancy in smooth backgrounds, improves adaptability to different texture regions, and significantly reduces the mismatch rate in industrial weak texture and smooth surface scenes.

[0022] Disparity regression is performed on the fusion matching cost volume. For each pixel, the minimum matching cost volume is taken as the target to generate the initial overall disparity. The initial overall disparity and the two disparities to its left and right are taken. By fitting a quadratic curve, the disparity value corresponding to the lowest point of the curve is found to generate the initial disparity map. The initial disparity map is upsampled to the same size as the original visual image by bilinear interpolation to obtain the final disparity map. The final disparity map is converted into three-dimensional spatial coordinates. At the same time, the texture type is mapped to the corresponding three-dimensional point coordinates to generate a three-dimensional point cloud with texture type. The three-dimensional reconstruction is completed to plan the motion trajectory of the industrial robot. By dividing the full parallax range into three categories—small, medium, and large—adaptive attention weights are matched to different parallax ranges, and weighted fusion is used to generate a fusion matching cost. Feature weights are strengthened for key detail areas with small parallax in the distant view, and computational redundancy is reduced for background areas with large parallax in the near view. This solves the problem of insufficient parallax range adaptability in existing technologies, improves the accuracy of the parallax map while reducing the amount of computation, and meets the real-time requirements of industrial sites. In addition, the process of triggering pose mutation also includes: constructing a comprehensive evaluation function based on the matching correlation degree and geometric residual value through weighted linear summation to obtain a comprehensive score. Specifically, this includes: First, in the calculation process, the matching correlation degree and geometric residual value need to be normalized to eliminate the influence of dimensions. The larger the matching correlation degree value, the more reliable the recognized pose; the smaller the matching correlation degree value, the less visually mismatched the recognized pose. The larger the geometric residual value, the lower the accuracy of the recognized pose; the larger the geometric residual value, the higher the accuracy of the recognized pose. Then, the geometric residual value is subtracted from the value of 1. Next, a comprehensive evaluation function is constructed through weighted summation to calculate the comprehensive score of each candidate pose, that is, the value of 1 minus the geometric residual value multiplied by the corresponding weight coefficient, the matching correlation degree multiplied by the corresponding weight coefficient, and the results of the multiplication are added together to obtain the final comprehensive score. The candidate pose point with the highest comprehensive score is selected from the pose prediction sequence as the reconnection and alignment pose. It should be noted that the collection includes all raw data from the historical reconnection and alignment pose operations of the industrial robot, including the raw values ​​of the matching correlation degree and geometric residual for each candidate pose point in each reconnection task, the number of successful reconnection operations after the pose was selected, the total number of reconnection operations, the normalized matching correlation degree, and the normalized geometric residual value. Based on expert experience, each historical candidate pose point is assigned a true reconnection validity label, with the label value set between 0 and 1. 1 represents that the reconnection is completely true and valid, the operation accuracy meets the standard, there are no visual misjudgments, and there are no safety anomalies; 0 represents that the reconnection association is completely invalid, the operation is scrapped, the equipment is collided, or the accuracy is seriously out of tolerance. All historical candidate pose points are sorted from low to high according to the true reconnection validity label to form a historical reconnection association dataset. Every three digits are used to rank the historical reconnection association numbers. The dataset is used to perform one value selection, and the resulting dataset is used as the test set. The remaining dataset is used as the model training set. A lightweight fully connected neural network is preset as the weight learning model. The model input layer has two feature nodes, corresponding to the normalized historical matching correlation degree and the normalized historical geometric precision positive value, respectively. There are two hidden layers, each with 16 neurons, both using the ReLU activation function. The output layer has two weight nodes, corresponding to the first weight of the matching correlation degree and the second weight, which is 1 minus the geometric residual value, and the sum of the first weight and the second weight is 1. At the same time, by learning the mapping relationship between the normalized historical matching correlation degree, the normalized historical geometric residual value and the true reconnection validity label, the first weight and the second weight are iteratively adjusted to minimize the mean square error between the comprehensive score and the true reconnection validity label.

[0023] By constructing a comprehensive evaluation function that combines the matching correlation degree and geometric residual value, and simultaneously linking the motion trajectory of the industrial robot, a matching optimization is completed. The selected optimal reconnection alignment pose not only ensures the reliability of the visual solution but also takes into account the efficiency and safety of the robot's motion. This solves the problem of the disconnect between the existing pose optimization technology and actual operation, and can be directly applied to the reconnection scenario of industrial robots.

[0024] A matching process is performed on the motion trajectory and the reconnection alignment pose. The Euclidean distance between the pose corresponding to the motion trajectory and the reconnection alignment pose is calculated. The reconnection alignment pose with the smallest Euclidean distance is selected as the optimal reconnection alignment pose. The optimal reconnection alignment pose is sent to the edge layer in real time through the industrial Ethernet channel. The edge layer drives the industrial robot to move and triggers the vision acquisition device to take real-time pictures. The high-reliability feature of the contour texture is selected to perform fast pose calculation. The real-time visual pose is sent to the edge layer. The industrial robot pose is dynamically adjusted by incremental visual servo control until the pose deviation in multiple consecutive control cycles meets the pre-planned tolerance. The current pose is locked to complete the visual alignment and improve the visual matching accuracy. By achieving low-latency forwarding of optimal pose and incremental visual servo correction through the edge layer, a millisecond-level visual alignment closed loop is formed, which greatly improves the response speed of reconnection alignment and the final positioning accuracy, meeting the needs of industrial robots for high-speed and high-precision operation.

[0025] Example 2: This invention provides a 3D recognition vision accuracy matching and optimization system for industrial robots; Figure 2 This is a schematic diagram of the module of the present invention; please refer to it. Figure 2 The system includes: a first analysis module and a second analysis module, and the first analysis module and the second analysis module are connected in communication. The first analysis module acquires IMU data, visual images, and IMU data of the work area during the normal movement of the industrial robot, constructs and selects three-dimensional probability models for various working conditions, predicts the three-dimensional pose distribution of the next sampling period, and outputs a pose prediction sequence composed of multiple candidate pose points. The second analysis module, under the condition of triggering a pose change, calls the preset rule engine to reconstruct the three-dimensional image of the visual image, plans the motion trajectory of the industrial robot, performs a matching between the motion trajectory and the reconnection and alignment pose, and selects the optimal reconnection and alignment pose.

[0026] In the application, the various formulas mentioned are all calculated by removing dimensions and taking their numerical values. The formulas are derived from the most recent real-world situation by collecting a large amount of data and conducting software simulations. The formulas are set by those skilled in the art according to the actual situation.

[0027] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0028] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0029] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A three-dimensional recognition visual precision matching optimization method for industrial robots, characterized in that, The method includes: during the normal movement of an industrial robot, acquiring motor parameters, IMU data and visual images of the work area, constructing and selecting a three-dimensional probability model for multiple working conditions, predicting the three-dimensional pose distribution of the next sampling period, and outputting a pose prediction sequence composed of multiple candidate pose points. Under the condition of triggering a pose change, the preset rule engine is invoked to reconstruct the three-dimensional image of the visual image, plan the motion trajectory of the industrial robot, perform a matching between the motion trajectory and the reconnection and alignment pose, and select the optimal reconnection and alignment pose.

2. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 1, characterized in that, Multiple operating conditions include low-speed conditions, high-speed conditions, and interference conditions, including: Simultaneously acquire motor parameters output by the industrial robot controller, IMU data output by the end effector, and visual images of the work area output by the vision acquisition device; The angular velocity and linear velocity arrays of IMU data are identified, a 3D point cloud is generated based on the visual image, and the edge blur array of the 3D point cloud is extracted. Through temporal alignment, the mean and variance of each array are extracted to construct a multi-dimensional working condition vector. The edge blur array is obtained using a 3D LBO operator. Using a multi-dimensional working condition vector with a preset sampling period as a sample set, feature mapping is performed through a preset working condition detection model to output multiple working condition states at N future times; where N is a positive integer greater than 0.

3. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 2, characterized in that, Also includes: The variational mode decomposition algorithm is applied to the motor parameters to obtain the motion modes, including the first mode and the second mode.

4. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 3, characterized in that, Construct and select three-dimensional probabilistic models for various working conditions, including: Low-speed operation: Activate the first encoder, extract the number of effective feature points in the visual image, retrieve the geometric features of the current target workpiece, generate a visual mask by combining the first modality, filter the time when the number of effective feature points is less than the standard threshold, and at the corresponding time, use the geometric features as the query vector and the visual mask as the key value, obtain the attention weight through the attention mechanism, and complete the visual feature enhancement. High-speed operation: Activate the second encoder, extract the amplitude distribution of the first mode, establish the mapping relationship between angular velocity / linear velocity and amplitude distribution, determine the motion trend, extract the energy proportion of the second mode, filter out cases where the energy proportion is less than the standard proportion threshold, fuse the features of visual image and IMU data, and automatically adjust the fusion weights according to the temporal proportion of motion trend type to complete motion feature enhancement; among which, motion trends include constant speed, variable speed, and start-stop. Interference condition: Activate the third encoder, retrieve visual images, IMU data and motor parameters of the same sampling period, and complete the multi-source feature fusion through channel cascading and dynamic weighting mechanism to obtain the enhanced features after fusion; Using various enhanced features acquired under different working conditions as input, and minimizing pose deviation as the objective function, a three-dimensional probabilistic model is constructed.

5. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 1, characterized in that, Triggering pose changes includes: The visual image of the next sampling period is acquired, and each candidate pose point in the pose prediction sequence is spatially projected and matched with the visual image. The matching correlation degree and geometric residual value corresponding to each candidate pose point are calculated. Visual images with matching correlation degree less than the standard matching threshold are selected and determined to have a pose change.

6. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 1, characterized in that, The system invokes a preset rule engine to perform 3D reconstruction of the visual image, including: Based on visual images, texture determination is performed through Fourier transform to identify texture types, and different algorithms are selected to perform feature extraction according to texture types to generate target feature maps; among them, texture types include continuous textures, non-continuous textures, and contour textures; A full disparity cost body is constructed based on the target feature map. By dividing the disparity range of the region to be worked on, differential disparity optimization is performed through a selective fusion iterative network. Adaptive attention weights are selected within different disparity ranges. The cost body corresponding to each disparity range is weighted and summed with the attention weights to generate a fusion matching cost. The disparity range includes small disparity range, medium disparity range, and large disparity range. Perform disparity regression on the fusion matching cost volume to generate a disparity map, convert the disparity map into three-dimensional spatial coordinates, and simultaneously map the texture type to the corresponding three-dimensional point coordinates to generate a three-dimensional point cloud with texture type.

7. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 1, characterized in that, The optimal reconnection alignment pose is sent to the edge layer, driving the industrial robot to follow the changes in the optimal reconnection alignment pose until visual alignment is achieved.

8. The method for optimizing the visual accuracy of 3D recognition in industrial robots according to claim 6, characterized in that, For continuous textures, the Poincaré exponential algorithm is used for feature extraction; for non-continuous textures, the gray-level co-occurrence matrix method is used for feature extraction; and for contour textures, Freeman chain code is used for feature extraction.

9. The method for optimizing the matching of 3D visual accuracy for industrial robots according to claim 5, characterized in that, The process of triggering pose mutation also includes: constructing a comprehensive evaluation function based on the matching correlation degree and geometric residual value through weighted linear summation, and selecting the candidate pose point with the highest comprehensive score from the pose prediction sequence as the reconnection and alignment pose.

10. A system for matching and optimizing the 3D visual accuracy of industrial robots, characterized in that, The system includes: The first analysis module acquires IMU data, visual images, and IMU data of the work area during the normal movement of the industrial robot, constructs and selects three-dimensional probability models for various working conditions, predicts the three-dimensional pose distribution of the next sampling period, and outputs a pose prediction sequence composed of multiple candidate pose points. The second analysis module, under the condition of triggering a pose change, calls the preset rule engine to reconstruct the three-dimensional image of the visual image, plans the motion trajectory of the industrial robot, performs a matching between the motion trajectory and the reconnection and alignment pose, and selects the optimal reconnection and alignment pose.

Images