Control method for intelligent unloading of industrial robot
By generating continuous trajectories through adaptive edge enhancement and texture suppression segmentation algorithms and feature point sets, the problems of insufficient robustness and accuracy of intelligent unloading of industrial robots in complex industrial environments are solved, and the environmental adaptability and real-time performance of unloading are improved.
Patent Information
- Application Number
- CN202510885068.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-10-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In complex industrial environments, existing intelligent unloading methods for industrial robots have problems such as insufficient robustness of image segmentation, insufficient accuracy of cargo boundary detection and positioning, and insufficient environmental adaptability and real-time performance.
An adaptive edge enhancement and texture suppression segmentation algorithm is used, combined with local geometric consistency and texture suppression, to generate a binary image. A continuous trajectory is generated through a set of feature points and 3D world coordinates to control the robot's movement to perform grasping and placement.
It improves the reliability of cargo boundary detection and the accuracy of segmentation results, enhances environmental adaptability and real-time performance, and significantly improves unloading efficiency and safety.
Smart Images
Figure CN120735008A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent unloading, and in particular to a control method for intelligent unloading of an industrial robot. Background Art
[0002] With the rapid development of industrial automation technology, industrial robots are widely used in manufacturing, logistics, warehousing, and other fields, significantly improving production efficiency and operational precision. In modern logistics and warehousing systems, unloading is a critical link in the supply chain. Traditional unloading methods rely primarily on manual operation or semi-automated equipment such as forklifts and conveyor belts. However, manual unloading is not only inefficient but also poses safety risks, such as the risk of occupational injuries caused by heavy lifting. Traditional automated equipment also lacks flexibility and adaptability, making it difficult to cope with complex and changing unloading scenarios. Therefore, developing intelligent control methods that can autonomously perceive the environment, identify cargo, and dynamically plan unloading paths has become a research hotspot in the field of industrial robotics. In the future, with the further development of deep learning, sensor technology, and computing power, intelligent unloading robots will play an even greater role in logistics, manufacturing, and warehousing.
[0003] The above-mentioned control method for intelligent unloading of industrial robots has problems such as insufficient robustness of image segmentation in complex industrial environments, insufficient accuracy of cargo boundary detection and positioning, and insufficient environmental adaptability and real-time performance. Summary of the Invention
[0004] The present invention provides a control method for intelligent unloading of industrial robots to solve the technical problems of insufficient robustness of image segmentation, insufficient accuracy of cargo boundary detection and positioning, and insufficient environmental adaptability and real-time performance in complex industrial environments.
[0005] The present invention provides a control method for intelligent unloading of industrial robots, which specifically includes the following technical solutions: A control method for intelligent unloading of an industrial robot comprises the following steps: S1. Capturing a cargo stack image and performing image preprocessing to generate a preprocessed cargo stack image; segmenting the preprocessed cargo stack image into cargo regions using an adaptive edge enhancement and texture suppression segmentation algorithm to generate a binary image; S2. Extract edge points from the binary image and construct feature vectors for the edge points; based on the feature vectors, construct a set of feature points and obtain the three-dimensional world coordinates of the feature points; based on the three-dimensional world coordinates of the feature points, generate a continuous trajectory and control the robot to move along the trajectory to perform grasping and placement.
[0006] Preferably, the S1 specifically includes: In the implementation process of image preprocessing, the cargo stack image is converted into a grayscale image, and the local contrast is enhanced to obtain an enhanced image; the enhanced image is denoised to obtain a denoised image, and the pixel values of the denoised image are mapped to the range of 0 to 1 to generate a preprocessed cargo stack image.
[0007] Preferably, the S1 specifically includes: In the implementation process of the adaptive edge enhancement and texture suppression segmentation algorithm, the horizontal gradient and vertical gradient of the pixels in the preprocessed cargo stack image are calculated; based on the horizontal gradient and vertical gradient of the pixels, the gradient amplitude and gradient direction of the pixels are calculated.
[0008] Preferably, the S1 specifically includes: A local window is defined for the pixels in the preprocessed cargo stack image, and the average intensity of the pixels in the window is calculated as the local threshold. Based on the local threshold and pixel intensity, the texture complexity is calculated.
[0009] Preferably, the S1 specifically includes: A geometric consistency factor is calculated based on the gradient direction within a local window of each pixel.
[0010] Preferably, the S1 specifically includes: The gradient magnitude, geometric consistency factor and texture complexity are combined, and exponential decay is introduced to calculate the dynamic weight.
[0011] Preferably, the S1 specifically includes: For each pixel of the preprocessed cargo stack image, the product of the local threshold and the dynamic weight is calculated, and the product is compared with the pixel value. When the pixel value is greater than the product, the pixel value of the binary image is set to 1, indicating the cargo area; otherwise, it is set to 0, indicating the background, and finally a binary image is generated.
[0012] Preferably, the S2 specifically includes: Based on the feature vectors of edge points, the edge points are divided into clusters through a clustering algorithm, and the geometric centers of the clusters are used as feature points to form a feature point set; the feature vectors include pixel coordinates and gradient directions.
[0013] Preferably, the S2 specifically includes: For each feature point in the feature point set, a calibration matrix is used to perform a linear transformation, converting the pixel coordinates into three-dimensional world coordinates and outputting a three-dimensional coordinate vector. Based on the three-dimensional coordinate vector, the shortest obstacle avoidance path from the starting point to the target three-dimensional coordinate point is generated, and the path points are smoothed by linear interpolation to generate a continuous trajectory. The joint angles are calculated through inverse kinematics, and the robot is controlled to move along the trajectory to perform grasping and placement.
[0014] The beneficial effects of the technical solution of the present invention are: 1. The adaptive edge enhancement and texture suppression segmentation algorithm analyzes the gradient amplitude and gradient direction of image pixels to accurately distinguish between product edges and background areas. It uses the average intensity of the local window as the threshold to adapt to lighting changes. At the same time, through texture complexity and geometric consistency analysis, it suppresses interference from noise and complex textures to enhance the segmentation effect of real product edges.
[0015] 2. During the segmentation process, by evaluating the consistency of pixel gradient directions within a local window, the real cargo edges with high geometric consistency are preferentially identified, while random interference in noisy areas is reduced, which significantly improves the reliability of edge detection and makes the segmentation results more consistent with the actual cargo boundaries.
[0016] 3. By comprehensively considering gradient amplitude, geometric consistency, and texture complexity, dynamic weights are calculated to enhance the sensitivity of edge areas and suppress misjudgment of high-texture areas. This can effectively cope with lighting changes and surface texture differences in complex industrial environments, improving the adaptability and accuracy of cargo area segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a flow chart of the control method for intelligent unloading of an industrial robot described in the present invention. DETAILED DESCRIPTION
[0018] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0019] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0020] The specific scheme of the control method for intelligent unloading of an industrial robot provided by the present invention is described in detail below with reference to the accompanying drawings.
[0021] Refer to the attached Figure 1 , which shows a flow chart of a control method for intelligent unloading of an industrial robot provided by one embodiment of the present invention, the method comprising the following steps: S1. Capturing a cargo stack image and performing image preprocessing to generate a preprocessed cargo stack image; segmenting the preprocessed cargo stack image into cargo regions using an adaptive edge enhancement and texture suppression segmentation algorithm to generate a binary image; An industrial camera (e.g., Basler acA1280-60gc, 1280×720 resolution) is used to capture dynamic scenes of the cargo stacking area in a logistics warehouse at a rate of 30 frames per second, generating raw image data containing pixel intensities in three channels (red, green, and blue), i.e., the cargo stack image. However, due to limitations in the actual industrial environment, there may be problems such as uneven lighting (e.g., shadows or strong light), noise interference (e.g., sensor noise or environmental noise), and complex cargo surface textures. Therefore, image preprocessing is required to enhance the quality of the cargo stack image. The image preprocessing includes: converting the cargo stack image into a grayscale image and enhancing the local contrast of the grayscale image using adaptive histogram equalization to obtain an enhanced image; denoising the enhanced image using median filtering to obtain a denoised image, and mapping the pixel values of the denoised image to a range of 0 to 1 to generate a preprocessed cargo stack image. The image preprocessing methods are well known to those skilled in the art and are not described in detail here. To achieve accurate cargo area segmentation in complex industrial environments (e.g., where cargo is closely packed with no gaps, with uneven lighting, and with occlusion), an adaptive edge enhancement and texture suppression segmentation algorithm is introduced to segment the cargo area from the preprocessed cargo stack image. The adaptive edge enhancement and texture suppression segmentation algorithm combines local geometric consistency and texture suppression to generate a binary image to distinguish the cargo area from the background. The specific implementation process is as follows: The Sobel operator is applied to the preprocessed cargo stack image, performing convolution in the horizontal and vertical directions respectively to obtain horizontal and vertical gradients. Based on the horizontal and vertical gradients, the gradient amplitude of each pixel in the preprocessed cargo stack image is calculated to reflect the severity of the pixel intensity change. A higher gradient amplitude is more likely to correspond to a cargo edge, while a lower gradient amplitude is more likely to correspond to a smooth area. Based on the horizontal and vertical gradients, the inverse tangent function is used to calculate the gradient direction, which is used to describe the geometric orientation of the edge of the preprocessed cargo stack image. Real cargo edges have high directional consistency, while the directions of noise areas are more random. On the preprocessed cargo stack image, define a local window (recommended 5×5 pixels) for each pixel and calculate the average intensity of the pixels within the window as the local threshold to reflect the brightness characteristics of the local area and adapt to lighting changes and uneven backgrounds. Furthermore, within the local window of each pixel, the deviation of each pixel intensity from the local threshold is calculated, and the deviation is squared. The average value of all square deviations within the local window is calculated as the texture complexity. The larger the square deviation value, the more complex the texture, and the lower the square deviation value, the smoother the area. Within the local window of each pixel, a geometric consistency factor is calculated based on the gradient direction to enhance the segmentation weight of the real product edge. Specifically, the gradient direction of each pixel in the local window is obtained, and the difference between the gradient direction of the central pixel and the gradient directions of other pixels, namely the gradient direction difference, is calculated. The cosine function is applied to calculate the cosine value of the gradient direction difference to quantify the gradient direction similarity. The cosine values of all gradient direction differences in the local window are averaged to obtain the geometric consistency factor. The higher the value of the geometric consistency factor, the stronger the direction consistency (real edge), and the lower the value of the geometric consistency factor, the more random the direction. The calculation formula of the geometric consistency factor is:
[0022] in, Indicates the preprocessed cargo stack image in The geometric consistency factor at ; represents the normalization factor, is the total number of pixels in the window; Expressing The local window at the center Sum the pixels within; The cosine value of the gradient direction difference is used to quantify the similarity of the gradient direction between the central pixel and the neighboring pixels. A cosine value close to 1 indicates consistent direction (real edge), and a cosine value close to -1 or 0 indicates random direction (noise). Represents the preprocessed cargo stack image The gradient direction of the pixel at ; Indicates the preprocessed cargo stack image in The gradient direction of the pixel at ; Based on the complexity requirements of image segmentation in industrial environments, dynamic weights are calculated by combining gradient amplitude, geometric consistency factor, and texture complexity. Gradient amplitude is used to enhance edge sensitivity, geometric consistency factor is used to screen true edges, and texture complexity suppresses high-texture areas through exponential decay. The calculation formula for dynamic weight is:
[0023] in, Indicates the preprocessed cargo stack image in Dynamic weight at represents a constant baseline; Represents the edge enhancement parameter, which is used to control the weight of the gradient amplitude. It is obtained through experiments and has a value range of ; Indicates the preprocessed cargo stack image in The gradient magnitude at , which is used to quantify the pixel intensity change; Indicates the preprocessed cargo stack image in The geometric consistency factor at ; represents the texture suppression factor based on exponential decay; Represents the texture suppression parameter, which is used to control the suppression strength of texture complexity. It is obtained through experiments and has a value range of ; Indicates the preprocessed cargo stack image in The texture complexity at For each pixel in the preprocessed cargo stack image, the product of the local threshold and the dynamic weight is calculated and compared with the pixel value. When the pixel value is greater than the product of the local threshold and the dynamic weight, the pixel value of the binary image is set to 1 to represent the cargo area; otherwise, it is set to 0 to represent the background. Finally, a binary image is generated. The formula is as follows:
[0024] in, Represents a binary image in The pixel value at is used to identify the cargo area; Indicates the preprocessed cargo stack image in The pixel value at ; represents the local threshold; represents the normalization factor, is the total number of pixels in the window; Expressing The local window at the center Sum the pixels within; Indicates the preprocessed cargo stack image in The pixel value at ; Indicates the preprocessed cargo stack image in The dynamic weight of .
[0025] S2. Extract edge points from the binary image and construct feature vectors for the edge points; construct a set of feature points based on the feature vectors and obtain the 3D world coordinates of the feature points; generate a continuous trajectory based on the 3D world coordinates of the feature points and control the robot to move along the trajectory to perform grasping and placement; Extract edge points representing the cargo boundary from the binary image. The edge points must belong to the cargo area and have a gradient amplitude greater than the gradient threshold. The gradient threshold is set based on expert experience, with a recommended value of 0.1, which is used to filter weak edges. Construct feature vectors for the edge points, including pixel coordinates and gradient direction, and apply the K-means clustering algorithm to divide the edge points into multiple clusters. The number of clusters is determined by the estimated number of cargo. For each cluster, calculate the geometric center as a feature point to represent the cargo location, and form all feature points into a feature point set. The K-means clustering algorithm is a well-known technical means for those skilled in the art and will not be described in detail here. For each feature point in the feature point set, a linear transformation is performed using the calibration matrix to convert the pixel coordinates into three-dimensional world coordinates and output a three-dimensional coordinate vector The calibration matrix is a 3×3 matrix based on the intrinsic parameters (related to the lens and sensor) and extrinsic parameters (related to the camera installation position and orientation) of the industrial camera. Based on the three-dimensional coordinate vector, the A* algorithm is used to generate the shortest obstacle avoidance path from the starting point to the target three-dimensional coordinate point, and the path points are smoothed by linear interpolation to generate a continuous trajectory; the joint angles are calculated through inverse kinematics to control the robot to move along the trajectory and perform grasping and placement; the starting point refers to the three-dimensional world coordinates of the initial position of the industrial robot when performing the unloading task; the target three-dimensional coordinate point refers to the three-dimensional world coordinates of the target position to which the robot needs to move, corresponding to the characteristic point of a certain cargo in the cargo stack (that is, the coordinates of the geometric center of the cargo obtained by image processing and clustering algorithms); the A* algorithm, linear interpolation and inverse kinematics are all technical means well known to those skilled in the art and will not be described in detail here.
[0026] In summary, a control method for intelligent unloading of industrial robots has been completed.
[0027] The order in which the embodiments of the invention are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0028] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.
[0029] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.
Claims
1. A control method for intelligent unloading of industrial robots, characterized in that: The following steps are involved: S1. Capturing a cargo stack image and performing image preprocessing to generate a preprocessed cargo stack image; segmenting the preprocessed cargo stack image into cargo regions using an adaptive edge enhancement and texture suppression segmentation algorithm to generate a binary image; S2, extract edge points from the binary image and construct feature vectors for the edge points; Based on the feature vectors, a set of feature points is constructed, and the three-dimensional world coordinates of the feature points are obtained; based on the three-dimensional world coordinates of the feature points, a continuous trajectory is generated, and the robot is controlled to move along the trajectory to perform grasping and placement.
2. The control method for intelligent unloading of an industrial robot according to claim 1, characterized in that: Said S1 specifically includes: In the implementation process of image preprocessing, the cargo stack image is converted into a grayscale image, and the local contrast is enhanced to obtain an enhanced image; the enhanced image is denoised to obtain a denoised image, and the pixel values of the denoised image are mapped to the range of 0 to 1 to generate a preprocessed cargo stack image.
3. The control method for intelligent unloading of an industrial robot according to claim 2, characterized in that: Said S1 specifically includes: In the implementation process of the adaptive edge enhancement and texture suppression segmentation algorithm, the horizontal gradient and vertical gradient of the pixels in the preprocessed cargo stack image are calculated; based on the horizontal gradient and vertical gradient of the pixels, the gradient amplitude and gradient direction of the pixels are calculated.
4. The control method for intelligent unloading of an industrial robot according to claim 3, characterized in that: Said S1 specifically includes: A local window is defined for the pixels in the preprocessed cargo stack image, and the average intensity of the pixels in the window is calculated as the local threshold. Based on the local threshold and pixel intensity, the texture complexity is calculated.
5. The control method for intelligent unloading of an industrial robot according to claim 4, characterized in that: Said S1 specifically includes: A geometric consistency factor is calculated based on the gradient direction within a local window of each pixel.
6. The control method for intelligent unloading of an industrial robot according to claim 5, characterized in that: Said S1 specifically includes: The gradient magnitude, geometric consistency factor and texture complexity are combined, and exponential decay is introduced to calculate the dynamic weight.
7. The control method for intelligent unloading of an industrial robot according to claim 6, characterized in that: Said S1 specifically includes: For each pixel of the preprocessed cargo stack image, the product of the local threshold and the dynamic weight is calculated, and the product is compared with the pixel value. When the pixel value is greater than the product, the pixel value of the binary image is set to 1, indicating the cargo area; otherwise, it is set to 0, indicating the background, and finally a binary image is generated.
8. The control method for intelligent unloading of an industrial robot according to claim 7, characterized in that: Said S2 specifically includes: Based on the feature vectors of edge points, the edge points are divided into clusters through a clustering algorithm, and the geometric centers of the clusters are used as feature points to form a feature point set; the feature vectors include pixel coordinates and gradient directions.
9. The control method for intelligent unloading of an industrial robot according to claim 8, characterized in that: Said S2 specifically includes: For each feature point in the feature point set, a calibration matrix is used to perform a linear transformation, converting the pixel coordinates into three-dimensional world coordinates and outputting a three-dimensional coordinate vector. Based on the three-dimensional coordinate vector, the shortest obstacle avoidance path from the starting point to the target three-dimensional coordinate point is generated, and the path points are smoothed by linear interpolation to generate a continuous trajectory. The joint angles are calculated through inverse kinematics, and the robot is controlled to move along the trajectory to perform grasping and placement.