Multi-color object recognition system

Abstract

This invention is a multi-color object recognition system that simultaneously acquires RGB and depth images through an image sensing module. Combined with HSV conversion, thresholding, and morphological operations by an image processing module, and K-means segmentation by a grouping and color classification module, it identifies multi-color target objects within a single area and obtains their two-dimensional positions. A coordinate transformation module then combines the depth information to generate three-dimensional coordinates and rotational attitude information, which is transmitted via Modbus TCP to a collaborative robotic arm through a communication and control module for automated grasping, rotation, or movement operations. This invention improves the stability of multi-color recognition, 3D positioning accuracy, and the efficiency of control integration with the robotic arm, making it suitable for industrial and logistics automation applications.

Claims

1. A multi-color object recognition system for identifying a target object; the system comprising: an image sensing module for simultaneously acquiring and outputting RGB image data and depth image data, the depth image data including three-dimensional distance information of the target object, wherein the image sensing module includes a Kinect DK camera; an image processing module electrically connected to the image sensing module for converting the RGB image data into an HSV color space, and performing thresholding and morphological operations on the HSV image to generate pre-processed image data; and a clustering and color classification module connected to the image processing module for segmenting the pre-processed image data into target objects based on a K-means clustering algorithm, and generating a color classification result and a two-dimensional position coordinate data of at least one target object; A coordinate conversion module, electrically connected to the image sensing module, is used to establish a conversion matrix between an image coordinate system and a robotic arm coordinate system based on three reference points, and, in conjunction with the depth image data, convert the two-dimensional position coordinate data into three-dimensional coordinate data and rotational attitude information; a communication and control module, electrically connected to the coordinate conversion module, is used to transmit the three-dimensional coordinate data and the rotational attitude information to a collaborative robotic arm according to the Modbus TCP protocol; the collaborative robotic arm, connected to the communication and control module, is used to perform grasping, rotating, or moving operations on target objects based on the three-dimensional coordinate data and the rotational attitude information.

2. The multi-color object recognition system as described in claim 1, wherein, The image processing module is configured to automatically obtain an HSV threshold through percentile calculation in order to exclude outlier color information and improve recognition stability.

3. The multi-color object recognition system as described in claim 1, wherein, The image processing module further performs at least one of closing, dilation, or erosion to remove noise and enhance the outline of the target object.

4. The multi-color object recognition system as described in claim 1, wherein, The grouping and color classification module activates the K-means grouping program when the outline area exceeds a preset threshold, in order to divide adjacent or mixed-color target objects into two or more groups.

5. The multi-color object recognition system as described in claim 1, wherein, The coordinate transformation module includes: an origin reference point, an X-axis reference point, and a Y-axis reference point, and generates the transformation matrix using the pixel spacing and entity distance between the X-axis reference point and the Y-axis reference point.

6. The multi-color object recognition system as described in claim 1, wherein, The communication and control module is configured to split the floating-point data of the three-dimensional coordinates into two 16-bit registers to conform to the Modbus data format of the collaborative robotic arm.

7. The multi-color object recognition system as described in claim 1, wherein, The image sensing module is fixed outside the work area in an eye-to-hand manner and provides a top-down view of the collaborative robotic arm and the target object.

8. The multi-color object recognition system as described in claim 1, wherein, The K-means clustering algorithm executed by this clustering and color classification module uses the cv2.kmeans() function of OpenCV.

9. The multi-color object recognition system as described in claim 1, wherein, It further includes a data recording module for recording the original image data generated by the image sensing module, the clustering result data generated by the clustering and color classification module, and the target object coordinate data generated by the coordinate transformation module.

Images