Robot demonstrator interface information identifying and reading method

Abstract

A robot demonstrator interface information identifying and reading method belongs to the technical field of image data processing, and aims to solve the problem that universality is poor when robot demonstrator interface information is acquired through communication in the prior art, the method comprises the following steps: image acquisition: acquiring a robot demonstrator interface image through an image acquisition tool to obtain an image A1; image preprocessing: carrying out image rotation on the image A1 to enable a text of the image A1 to be horizontal, and then carrying out image clipping on a part needing to be identified to obtain an image A2; image background and target segmentation: carrying out binarization processing on the image A2 to obtain an image A3; image hole repairing: processing the image A3 by adopting morphological closed operation to obtain an image A4; image noise filtering: carrying out image filtering and image sharpening processing on the image A4 to obtain an image A5; the problem of poor universality in the prior art is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image data processing, and in particular relates to a method for identifying and reading interface information of a robot teaching pendant. Background Art

[0002] In the wide application of robots, the teach pendant, as a key device for human-robot interaction, undertakes important functions such as command input, parameter setting, and status monitoring. Operators issue various task instructions through the teach pendant interface to guide the robot to complete complex work processes. However, there are many limitations in the acquisition and use of teach pendant interface information.

[0003] The traditional method relies on manual recording of key information displayed on the teach pendant interface, such as motion trajectory parameters, working mode settings, alarm prompts, etc. This method is not only inefficient, but also easily interfered by human factors, resulting in errors or omissions in information recording. Especially in large-scale industrial production environments, where there are a large number of robots and task switching is frequent, manual recording seriously restricts the improvement of production efficiency.

[0004] Some prior art attempts to obtain data by directly accessing the communication interface of the teach pendant, such as the Chinese patent publication number "CN 107290985 B", the patent name is "Data processing method and device for teach pendant", this method starts the teach pendant through the startup thread, the startup thread is a thread used to start the teach pendant; judge whether the teach pendant passes the inspection after startup; if the teach pendant passes the inspection, enter the main interface of the teach pendant; run the main interface program of the teach pendant and the interface function of the main interface through the main thread to obtain task data, the main thread is a thread used to run the main interface program of the teach pendant and the interface function of the teach pendant, the main interface program is a program used to control the running state of the teach pendant; send task data to the controller system through the communication thread and receive control data from the controller system, the communication thread is a thread used to exchange data between the teach pendant and the controller system, the controller system is used to perform control according to the task data to obtain control data. However, the method of obtaining robot teach pendant interface information through communication has poor versatility. Summary of the invention

[0005] In order to solve the problem of poor universality in the prior art of obtaining robot teaching pendant interface information by communication, the present invention proposes a robot teaching pendant interface information recognition and reading method.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for identifying and reading information on a robot teaching pendant interface, the method comprising:

[0008] Step 1: Image acquisition;

[0009] The robot teaching pendant interface image is acquired by using an image acquisition tool to obtain image A1;

[0010] Step 2: Image preprocessing;

[0011] Image A1 is rotated to make its text horizontal, and then the part to be recognized is cropped to obtain image A2;

[0012] Step 3: Image background and target segmentation;

[0013] Binarize image A2 to obtain image A3;

[0014] Step 4: Image hole repair;

[0015] Image A3 is processed by morphological closing operation to obtain image A4;

[0016] Step 5: Image noise filtering;

[0017] Performing image filtering and image sharpening processing on image A4 to obtain image A5;

[0018] Step 6: Image text recognition;

[0019] The CRNN (Convolutional Recurrent Neural Network) model based on the fusion of convolutional features and sequence features is used to perform text recognition on image A5 to obtain the robot teaching pendant interface information.

[0020] The image rotation uses an affine transformation algorithm, and the image cropping uses a frame buffer access algorithm.

[0021] The binarization process is to use the Otsu threshold method to segment the image A2 to obtain the image A3.

[0022] The Otsu threshold method is calculated based on the image histogram, and the threshold when the inter-class variance of the image histogram is the largest is calculated to segment the foreground and background images.

[0023] The morphological closing operation processes the image by filling small holes between blocks, performing a dilation operation and then an erosion operation.

[0024] The image filtering is processed by a median filtering algorithm, and the image sharpening is processed by a Laplace operator algorithm.

[0025] Beneficial effects of the invention: The invention adopts the technical route of combining image preprocessing with the CRNN model, and uses methods such as affine transformation, Otsu threshold method, morphological closing operation, median filtering and sharpening processing, which effectively solves the problems of poor versatility, low efficiency and susceptibility to interference, and significantly improves the accuracy and robustness of image recognition. It can adapt to teaching pendants of different brands and models and is not affected by ambient light or noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a method for identifying and reading interface information of a robot teaching pendant of the present invention;

[0027] Figure 2 A schematic diagram of image acquisition in an embodiment of the present invention;

[0028] Figure 3 Schematic diagram of image preprocessing in an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the image preprocessing result in an embodiment of the present invention:

[0030] Figure 5 Schematic diagram of image binarization processing in an embodiment of the present invention;

[0031] Figure 6 A schematic diagram of image hole repair in an embodiment of the present invention;

[0032] Figure 7 Schematic diagram of image noise filtering processing in an embodiment of the present invention;

[0033] Figure 8 Schematic diagram of image text recognition results in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, a method for identifying and reading information on a robot teaching pendant interface comprises:

[0036] Step 1: Image acquisition;

[0037] The robot teaching pendant interface image is captured by a camera to obtain image A1;

[0038] Step 2: Image preprocessing;

[0039] Use the affine transformation algorithm to rotate the image A1 so that its text is horizontal, use the framebuffer access algorithm to crop the joint angle range in the image A1 to obtain the image A2, save the position coordinates and size of the cropping, and improve the recognizability and effectiveness of the image content;

[0040] Step 3: Image background and target segmentation;

[0041] The image A2 is binarized, and the binarization process is to segment the image A2 using the Otsu threshold method to obtain the image A3. The Otsu threshold method is calculated based on the image histogram, and the threshold when the inter-class variance of the image histogram is the largest is calculated to segment the foreground and background images, so that the main content of the image is more prominent, which is convenient for subsequent analysis and processing;

[0042] Step 4: Image hole repair;

[0043] The image A3 is processed by a morphological closing operation to obtain an image A4. The morphological closing operation includes closing small holes between image blocks, performing a dilation operation and then an erosion operation on the image, so that the image becomes more coherent and neat, and the small holes in the foreground part of the image are eliminated, and the contour of the foreground object is also made smoother and more complete;

[0044] Step 5: Image noise filtering;

[0045] Image A4 is filtered using a median filter algorithm, and then sharpened using a Laplace operator algorithm to obtain image A5, which effectively removes noise from the image, making the image cleaner and smoother, and enhances the edges and details of the image, making the image clearer and sharper, thereby improving the overall quality of the image;

[0046] Step 6: Image text recognition;

[0047] The CRNN model based on the fusion of convolutional features and sequence features is used to perform text recognition on image A5 to obtain the robot teaching pendant interface information.

[0048] For multiple sets of image data, repeat steps 2 to 6 until all robot teaching pendant interface information is obtained.

[0049] Example:

[0050] In this embodiment, the KUKA robot teaching pendant is taken as the research object.

[0051] Step 1: Image acquisition;

[0052] Use the QCamera class to manage the camera to capture the KUKA robot teaching pendant image A1. During the shooting process, use a bracket to keep the camera and the teaching pendant stable, such as Figure 2 As shown, the joint angles in the photos taken by the camera appear to be fixed in position;

[0053] Step 2: Image preprocessing;

[0054] like Figure 3 As shown, the affine transformation algorithm is used to rotate the center of image A1, and the frame buffer access algorithm is used to clip the joint angle range in image A1 to obtain image A2, as shown in Figure 4 As shown. Since the joint angle position is the same in each shot, only the first picture needs to be processed, the picture is rotated to the text level, and the joint angle part of the picture is cropped. The image rotation angle and the cropping position and size are saved. Then, the other pictures taken by the camera are rotated and cropped in the same angle and position. The joint angle part of all pictures can be cropped and the obtained joint angle pictures are horizontal.

[0055] Step 3: Image background and target segmentation;

[0056] like Figure 5 As shown, the image A2 is binarized and segmented using the Otsu threshold method to obtain a binary image A3. The Otsu threshold method is calculated based on the image histogram, and the threshold when the inter-class variance is the largest is calculated to segment the foreground and background images.

[0057] Step 4: Image hole repair;

[0058] like Figure 6 As shown in the figure, after obtaining the binary image A3, the morphological closing operation is used to process the image to fill the small holes between the blocks. The expansion operation is performed first, and then the erosion operation. The kernels of the operations are all 3×3 matrices. The 3×3 matrix is After the processing is completed, image A4 is obtained. Compared with the image before processing, all the contents in the image can be recognized, and the recognition confidence is improved;

[0059] Step 5: Image noise filtering;

[0060] like Figure 7 As shown, the image A4 is filtered using the median filter algorithm, and then sharpened using the Laplace operator algorithm to obtain the image A5;

[0061] Step 6: Image text recognition;

[0062] like Figure 8 As shown, the CRNN model based on the fusion of convolutional features and sequence features is used to perform text recognition on image A5 to obtain better robot teaching pendant interface information.

[0063] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying and reading information on a robot teaching pendant interface, characterized in that: The method includes: Step 1: Image acquisition; The robot teaching pendant interface image is acquired by using an image acquisition tool to obtain image A1; Step 2: Image preprocessing; Image A1 is rotated to make its text horizontal, and then the part to be recognized is cropped to obtain image A2; Step 3: Image background and target segmentation; Binarize image A2 to obtain image A3; Step 4: Image hole repair; Image A3 is processed by morphological closing operation to obtain image A4; Step 5: Image noise filtering; Performing image filtering and image sharpening processing on image A4 to obtain image A5; Step 6: Image text recognition; The CRNN model based on the fusion of convolutional features and sequence features is used to perform text recognition on image A5 to obtain the robot teaching pendant interface information.

2. A method for identifying and reading interface information of a robot teaching pendant according to claim 1, characterized in that: The image rotation in step 2 uses an affine transformation algorithm, and the image cropping uses a frame buffer access algorithm.

3. A robot teaching pendant interface information recognition and reading method according to claim 1, characterized in that: The binarization process in step 3 is to segment image A2 using the Otsu threshold method to obtain image A3; The Otsu threshold method is calculated based on the image histogram, and the threshold when the inter-class variance of the image histogram is the largest is calculated to segment the foreground and background images.

4. A robot teaching pendant interface information recognition and reading method according to claim 1, characterized in that: The morphological closing operation described in step 4 processes the image by filling small holes between blocks, performing a dilation operation, and then performing an erosion operation.

5. A robot teaching pendant interface information recognition and reading method according to claim 1, characterized in that: The image filtering in step 5 is processed by using a median filtering algorithm, and the image sharpening is processed by using a Laplacian operator algorithm.

Images