HSV color parameter online adjustment method for a gluing robot system

Abstract

The application belongs to the technical field of image processing, and particularly relates to a method for online adjustment of HSV color parameters of a glue collecting robot system, which comprises the following steps: a camera collects an original image of a target object and publishes the original image; an image parameter adjustment node subscribes to an original image topic; the image parameter adjustment node performs color segmentation on the original image, the image topic is transmitted to control software and displayed, whether the glue is well segmented is determined by the picture displayed on the software, so that it is determined whether to continue to adjust parameters, an operator manually adjusts HSV parameters on the software; a read-write node encapsulates new HSV parameters into a topic and publishes the topic; the image parameter adjustment node subscribes to the new HSV parameters, and updates the original HSV parameters based on the new HSV parameters, the application adds the step of parameter adjustment before the robot performs glue identification and glue collection, online fine adjustment is performed based on preset parameters in the software end of a tablet computer, and the image after the parameter adjustment can be displayed on the software in real time to help determine the parameter adjustment effect.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a method for online adjustment of HSV color parameters in a glue collection robot system. Background Technology

[0002] HSV (Hue, Saturation, Value) is a color space that describes color and is commonly used in machine vision tasks such as object detection, image segmentation, and color recognition. Adjusting HSV parameters helps to more accurately identify and segment specific colors in an image. HSV color tuning typically involves adjusting the minimum and maximum values ​​of the hue (H), saturation (S), and value (V) of the color to be detected, totaling six parameters to identify colors within the parameter range.

[0003] In a vision system for a glue-collecting robot, color recognition is used to segment the glue. First, the six parameters of the HSV (Highly Safer Color Scale) need to be determined. Currently, most methods for determining HSV parameters involve first acquiring images of the target object, then offline debugging each HSV parameter individually to achieve optimal results. After parameter tuning, these parameters are written into the vision system for storage. Each time the system is needed, the saved, fixed HSV parameters are directly called, and the corresponding colored glue is identified based on its parameter range.

[0004] The traditional method described above requires offline image acquisition, parameter adjustment, writing the determined parameters into the system code, and recompiling for each parameter adjustment. It may be necessary to readjust the parameters under different lighting conditions or when the color of the colloid changes. Therefore, this method is relatively complicated and difficult for workers on site to operate. Summary of the Invention

[0005] The purpose of this invention is to provide a method for online adjustment of color parameters in a vision system for a glue-collecting robot. This method is easy to operate and can improve the work efficiency of workers during the glue-collecting process.

[0006] A method for online adjustment of HSV color parameters in a glue collection robot system, including...

[0007] Step 1: The camera acquires the original image of the target object, encapsulates the original image data into a ROS image topic, and publishes it by the camera node;

[0008] Furthermore, in step one, the image from the camera is read using the OpenCV library, the OpenCV format image is converted to a ROS image using the cv_bridge class, the ROS publishing frequency is set to 30Hz, and the original image topic is published through the publishing object.

[0009] Step 2: The image parameter tuning node subscribes to the original image topic and converts the image topic into Mat format that OpenCV supports;

[0010] Furthermore, in step two, the image parameter tuning node subscribes to the original image topic and uses the cv_bridge class to convert the ROS image format sensor_msgs / Image to the Mat format in OpenCV.

[0011] Step 3: The image parameter tuning node reads the pre-saved HSV parameters, performs color segmentation on the original image according to the HSV parameter range, and encapsulates the segmented image into an image topic, which is then published by the image parameter tuning node.

[0012] Furthermore, in step three, the values ​​of the three RGB channels of the image pixel are red (r), green (g), and blue (b), which range from 0 to 255 in OpenCV's Mat format. Dividing all three values ​​by 255 makes them real numbers between 0 and 1. Let max be the maximum value among r, g, and b, and min be the minimum value among them. The hue (h), saturation (s), and brightness (v) values ​​in the HSV color space can then be calculated using the following formula.

[0013]

[0014]

[0015] v = max

[0016] Read the six HSV threshold parameters (minimum hue value h_min, maximum hue value h_max, minimum saturation value s_min, maximum saturation value s_max, minimum brightness value v_min, maximum brightness value v_max) saved in the default path file, iterate through all pixels of the HSV image, and obtain the image mask using the following formula. The segmentation operation can be completed by performing a bitwise AND operation between the obtained binary mask and the original image.

[0017] Furthermore, the encapsulation in step three refers to using the cv_bridge class to convert an OpenCV format image into a ROS image.

[0018] Furthermore, in step three, "publishing" refers to setting the ROS publishing frequency to 30Hz and publishing the processed image topic through the publishing object.

[0019] Step 4: The image topic is transmitted to the control software and displayed via the RTSP-ROS package;

[0020] Furthermore, in step four, rtsp-ros is a ROS function package used for camera image stream transmission. It reads and parses RTSP data streams into standard ROS image information via the network port, thereby enabling remote image transmission.

[0021] Step 5: The operator observes the image displayed on the software to determine whether the colloid has been well separated, and decides whether to continue parameter adjustment. If the colloid separation effect is good, no further parameter adjustment is needed, and the operator jumps to step 10; otherwise, the operator continues to the next step, step 6.

[0022] Step Six: The operator manually adjusts the HSV parameters on the software;

[0023] Furthermore, in step six, the parameter tuning software includes ROS-based main controller code and Qt-based human-machine interface code. The ROS-based main controller code is developed and runs on a Jetson nano, while the Qt-based human-machine interface code is developed on a Win11 system and runs on an Android tablet. The software has the function of receiving data and instructions from the host computer. The host computer sends data and instruction information, and the Jetson nano receives and executes the corresponding commands. The host computer and Jetson nano also have image transmission functions. The Jetson nano receives image data captured by the camera and transmits the image information to the host computer, allowing the operator to adjust the parameters of the image in real time.

[0024] Step 7: The software sends new HSV parameters to the read / write nodes of the Jetson Nano via the TCP / IP protocol;

[0025] Step 8: The read / write node encapsulates the new HSV parameters into a topic and publishes it;

[0026] Step 9: The image parameter tuning node subscribes to the new HSV parameters and updates the original HSV parameters accordingly. Then, proceed to Step 2.

[0027] Furthermore, in step nine, the image parameter tuning node subscribes to the HSV parameter topic, converts the data therein to int integer type, updates the six variables h_min, h_max, s_min, s_max, v_min, and v_max, and then jumps to step two;

[0028] Step 10: The operator saves the data on the software.

[0029] Step 11: The control software sends a command to save the HSV parameters to the read / write node of the Jetson Nano via the TCP / IP protocol;

[0030] Furthermore, in step eleven, the software and the Jetson Nano main controller connect using a Socket interface at the software abstraction layer.

[0031] Step 12: The image parameter tuning node subscribes to the save parameter command topic and stores the current HSV parameters to the specified path for subsequent colloid recognition and segmentation tasks.

[0032] Furthermore, in step twelve, the image parameter tuning node subscribes to parameter topics as a ROS subscriber and saves the six HSV range parameters to a TXT file according to the default specified path. These six values ​​are then read from the file in this path when performing the colloid recognition and segmentation task later.

[0033] The advantages of this application are:

[0034] 1. This invention adds a parameter adjustment step before the robot performs colloid recognition and colloid collection. The parameters are fine-tuned online based on preset parameters in the software on the tablet computer. The software can display the image after parameter adjustment in real time to help judge the effect of parameter adjustment. The current HSV parameters can be saved with one click to ensure that the appropriate HSV parameters can be obtained quickly before each colloid collection.

[0035] 2. Compared with the prior art, the technical solution proposed in this invention can determine and save the required HSV parameters in a shorter time, avoiding workers from directly reading, writing and recompiling the visual code of the controller, thus improving the efficiency of visual parameter tuning.

[0036] 3. Existing patents convert images to the HSV color space before processing them to segment the target object. However, the method in this application aims to convert to the HSV color space and then observe the segmentation of the colloid and background in real time by adjusting parameters to determine and save the HSV threshold parameters. This method has a different focus and achieves different functions than existing patent methods.

[0037] 4. This application is not simply a matter of changing offline parameter tuning to online parameter tuning, but rather a process that completes the entire process from image acquisition, video transmission, data transmission, parameter tuning by operators, to real-time visualization of parameter tuning results. Attached Figure Description

[0038] Figure 1 This is a system architecture diagram of this method.

[0039] Figure 2 This is a diagram of the software data transmission involved in this method.

[0040] Figure 3 This is a diagram of the software parameter tuning interface involved in this method.

[0041] Figure 4 This is the flowchart for this method. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0043] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0045] In the description of this application, it should be noted that the terms "upper," "vertical," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0046] In the description of this application, it should also be noted that, unless otherwise expressly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0047] Example 1

[0048] A method for online adjustment of HSV color parameters in a glue collection robot system, including...

[0049] Step 1: The camera acquires the original image of the target object, encapsulates the original image data into a ROS image topic, and publishes it by the camera node;

[0050] Step 2: The image parameter tuning node subscribes to the original image topic and converts the image topic into Mat format that OpenCV supports;

[0051] Step 3: The image parameter tuning node reads the pre-saved HSV parameters, performs color segmentation on the original image according to the HSV parameter range, and encapsulates the segmented image into an image topic, which is then published by the image parameter tuning node.

[0052] Step 4: The image topic is transmitted to the control software and displayed via the RTSP-ROS package;

[0053] Step 5: The operator observes the image displayed on the software to determine whether the colloid has been well separated, and decides whether to continue parameter adjustment. If the colloid separation effect is good, no further parameter adjustment is needed, and the operator jumps to step 10; otherwise, the operator continues to the next step, step 6.

[0054] Step Six: The operator manually adjusts the HSV parameters on the software;

[0055] Step 7: The software sends new HSV parameters to the read / write nodes of the Jetson Nano via the TCP / IP protocol;

[0056] Step 8: The read / write node encapsulates the new HSV parameters into a topic and publishes it;

[0057] Step 9: The image parameter tuning node subscribes to the new HSV parameters and updates the original HSV parameters accordingly. Then, proceed to Step 2.

[0058] Step 10: The operator saves the data on the software.

[0059] Step 11: The control software sends a command to save the HSV parameters to the read / write node of the Jetson Nano via the TCP / IP protocol;

[0060] Step 12: The image parameter tuning node subscribes to the save parameter command topic and stores the current HSV parameters to the specified path for subsequent colloid recognition and segmentation tasks.

[0061] Example 2

[0062] Figure 1This is a system architecture diagram of the present invention. The main controller is a Jetson Nano, and the camera is directly connected to the Jetson Nano. They communicate with each other through the robot operating system ROS. The Jetson Nano and the control software are wirelessly connected under the same Wi-Fi network. Control commands and data communicate via the TCP / IP protocol, and image transmission is achieved using the ROS rtsp-ross function package. Operators can directly interact with the control software using the plane, such as reading data, directly viewing images, modifying data, and sending commands.

[0063] like Figure 4 As shown, a method for online adjustment of HSV color parameters in a glue collection robot system includes...

[0064] Step 1: The camera acquires the original image of the target object, encapsulates the original image data into a ROS image topic, and publishes it by the camera node;

[0065] Furthermore, in step one, the image from the camera is read using the OpenCV library, the OpenCV format image is converted to a ROS image using the cv_bridge class, the ROS publishing frequency is set to 30Hz, and the original image topic is published through the publishing object.

[0066] Step 2: The image parameter tuning node subscribes to the original image topic and converts the image topic into Mat format that OpenCV supports;

[0067] Furthermore, in step two, the image parameter tuning node subscribes to the original image topic and uses the cv_bridge class to convert the ROS image format sensor_msgs / Image to the Mat format in OpenCV.

[0068] Step 3: The image parameter tuning node reads the pre-saved HSV parameters, performs color segmentation on the original image according to the HSV parameter range, and encapsulates the segmented image into an image topic, which is then published by the image parameter tuning node.

[0069] Furthermore, in step three, the values ​​of the three RGB channels of the image pixel are red (r), green (g), and blue (b), which range from 0 to 255 in OpenCV's Mat format. Dividing all three values ​​by 255 makes them real numbers between 0 and 1. Let max be the maximum value among r, g, and b, and min be the minimum value among them. The hue (h), saturation (s), and brightness (v) values ​​in the HSV color space can then be calculated using the following formula.

[0070]

[0071]

[0072] v = max

[0073] Read the six HSV threshold parameters (minimum hue value h_min, maximum hue value h_max, minimum saturation value s_min, maximum saturation value s_max, minimum brightness value v_min, maximum brightness value v_max) saved in the default path file, iterate through all pixels of the HSV image, and obtain the image mask using the following formula. The segmentation operation can be completed by performing a bitwise AND operation between the obtained binary mask and the original image.

[0074] Furthermore, the encapsulation in step three refers to using the cv_bridge class to convert an OpenCV format image into a ROS image.

[0075] Furthermore, in step three, "publishing" refers to setting the ROS publishing frequency to 30Hz and publishing the processed image topic through the publishing object.

[0076] Step 4: The image topic is transmitted to the control software and displayed via the RTSP-ROS package;

[0077] Furthermore, in step four, rtsp-ros is a ROS function package used for camera image stream transmission. It reads and parses RTSP data streams into standard ROS image information via the network port, thereby enabling remote image transmission.

[0078] Step 5: The operator observes the image displayed on the software to determine whether the colloid has been well separated, and decides whether to continue parameter adjustment. If the colloid separation effect is good, no further parameter adjustment is needed, and the operator jumps to step 10; otherwise, the operator continues to the next step, step 6.

[0079] Step Six: The operator manually adjusts the HSV parameters on the software;

[0080] Furthermore, in step six, the parameter tuning software includes ROS-based main controller code and Qt-based human-machine interface code. The ROS-based main controller code is developed and runs on a Jetson Nano, while the Qt-based human-machine interface code is developed on a Win11 system and runs on an Android tablet. The software has the function of receiving data and instructions from the host computer. The host computer sends data and instruction information, and the Jetson Nano receives and executes the corresponding commands. The host computer and Jetson Nano also have image transmission capabilities. The Jetson Nano receives image data captured by the camera and transmits the image information to the host computer, allowing the operator to adjust the parameters of the image in real time. A schematic diagram of the software's data transmission is shown below. Figure 2 As shown, the software interface involving parameter tuning is as follows: Figure 3As shown, the right side is the operation interface for operators to adjust HSV transmission, and the left side is the monitoring interface, which outputs the adjusted parameters in real time for operators to observe the results. The Jetson nano and the software are wirelessly connected under the same WIFI network. Control commands and data communicate via TCP / IP protocol. Image transmission is achieved using the ROS rtsp-ross function package. Operators can directly interact with the plane and control software, such as reading data, directly viewing images, modifying data, and sending commands.

[0081] Step 7: The software sends new HSV parameters to the read / write nodes of the Jetson Nano via the TCP / IP protocol;

[0082] Step 8: The read / write node encapsulates the new HSV parameters into a topic and publishes it;

[0083] Step 9: The image parameter tuning node subscribes to the new HSV parameters and updates the original HSV parameters accordingly. Then, proceed to Step 2.

[0084] Furthermore, in step nine, the image parameter tuning node subscribes to the HSV parameter topic, converts the data therein to int integer type, updates the six variables h_min, h_max, s_min, s_max, v_min, and v_max, and then jumps to step two;

[0085] Step 10: The operator saves the data on the software.

[0086] Step 11: The control software sends a command to save the HSV parameters to the read / write node of the Jetson Nano via the TCP / IP protocol;

[0087] Furthermore, in step eleven, the tablet software and the Jetson Nano main controller connect using a Socket interface at the software abstraction layer. A Socket connection is a communication mechanism in computer networks that allows two programs to communicate across different computer networks. In this solution, the tablet software acts as the client, and the Jetson Nano main controller acts as the server; they use socket creation and usage for data transmission.

[0088] Step 12: The image parameter tuning node subscribes to the save parameter command topic and stores the current HSV parameters to the specified path for subsequent colloid recognition and segmentation tasks.

[0089] Furthermore, in step twelve, the image parameter tuning node subscribes to parameter topics as a ROS subscriber and saves the six HSV range parameters to a TXT file according to the default specified path. These six values ​​are then read from the file in this path when performing the colloid recognition and segmentation task later.

Claims

1. A method for online adjustment of HSV color parameters in a glue collection robot system, characterized in that: include Step 1: The camera acquires the original image of the target object, encapsulates the original image data into a ROS image topic, and publishes it by the camera node; Step 2: The image parameter tuning node subscribes to the original image topic and converts the image topic into Mat format that OpenCV supports; Step 3: The image parameter tuning node reads the pre-saved HSV parameters, performs color segmentation on the original image according to the HSV parameter range, and encapsulates the segmented image into an image topic, which is then published by the image parameter tuning node. Step 4: The image topic is transmitted to the control software and displayed via the RTSP-ROS package; Step 5: The operator observes the image displayed on the software to determine whether the colloid has been well separated, and decides whether to continue parameter adjustment. If the colloid separation effect is good, no further parameter adjustment is needed, and the operator jumps to step 10; otherwise, the operator continues to the next step, step 6. Step Six: The operator manually adjusts the HSV parameters on the software; Step 7: The software sends new HSV parameters to the read / write nodes of the Jetson Nano via the TCP / IP protocol; Step 8: The read / write node encapsulates the new HSV parameters into a topic and publishes it; Step 9: The image parameter tuning node subscribes to the new HSV parameters and updates the original HSV parameters accordingly. Then, proceed to Step 2. Step 10: The operator saves the data on the software. Step 11: The control software sends a command to save the HSV parameters to the read / write node of the Jetson Nano via the TCP / IP protocol; Step 12: The image parameter tuning node subscribes to the save parameter command topic and stores the current HSV parameters to the specified path for subsequent colloid recognition and segmentation tasks.

2. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step one, the camera image is read using the OpenCV library, the OpenCV format image is converted to a ROS image using the cv_bridge class, the ROS publishing frequency is set to 30Hz, and the original image topic is published through the publishing object.

3. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step two, the image parameter tuning node subscribes to the original image topic and uses the cv_bridge class to convert the ROS image format sensor_msgs / Image to the Mat format in OpenCV.

4. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step three, the values ​​of the three RGB channels of the image pixels are red (r), green (g), and blue (b), which range from 0 to 255 in OpenCV's Mat format. Dividing all three values ​​by 255 makes them real numbers between 0 and 1. Let max be the maximum value among r, g, and b, and min be the minimum value. The hue (h), saturation (s), and brightness (v) values ​​in the HSV color space can then be calculated using the following formula. v = max Read the six HSV threshold parameters (minimum hue value h_min, maximum hue value h_max, minimum saturation value s_min, maximum saturation value s_max, minimum brightness value v_min, maximum brightness value v_max) saved in the default path file, iterate through all pixels of the HSV image, and obtain the image mask using the following formula. The segmentation operation can be completed by performing a bitwise AND operation between the obtained binary mask and the original image.

5. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: Encapsulation in step three refers to using the cv_bridge class to convert an OpenCV format image into a ROS image; publishing refers to setting the ROS publishing frequency to 30Hz and publishing the processed image topic through the publishing object.

6. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step four, rtsp-ros is a ROS function package used for camera image stream transmission. It reads and parses RTSP data streams into standard ROS image information via the network port, thereby enabling remote image transmission.

7. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step six, the parameter tuning software includes ROS-based main controller code and Qt-based human-machine interface code. The ROS-based main controller code is developed and runs on a Jetson nano, while the Qt-based human-machine interface code is developed on a Win11 system and runs on an Android tablet. The software has the function of receiving data and instructions from the host computer. The host computer sends data and instruction information, and the Jetson nano receives and executes the corresponding commands. The host computer and Jetson nano also have image transmission capabilities. The Jetson nano receives image data captured by the camera and transmits the image information to the host computer, allowing operators to adjust the parameters of the image in real time.

8. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step nine, the image parameter tuning node subscribes to the HSV parameter topic, converts the data therein to int integer type, updates the six variables h_min, h_max, s_min, s_max, v_min, and v_max, and then jumps to step two.

9. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step eleven, the software and the Jetson Nano main controller connect using a Socket interface at the software abstraction layer.

10. The method for online adjustment of HSV color parameters of a glue collection robot system according to claim 1, characterized in that: In step 12, the image parameter tuning node subscribes to parameter topics as a ROS subscriber and saves the six HSV range parameters to a TXT file according to the default specified path. These six values ​​are then read from the file in this path when performing the colloid recognition and segmentation task later.

Images