A positioning method and system of a robot, an electronic device, and a storage medium

Abstract

The embodiment of the application provides a positioning method and system of a robot, an electronic device and a storage medium, relates to the technical field of image detection, and the method comprises the steps of: controlling the robot to move to a first photographing position, acquiring a first image obtained by a camera shooting a first positioning feature point on a first surface of a workpiece; controlling the robot to move to a second photographing position, acquiring a second image obtained by the camera shooting a second positioning feature point on the first surface; according to a preset distance between the first photographing position and the second photographing position, unifying the image coordinates of different positioning feature points in the same image; according to the first image coordinates and the second image coordinates, calculating a deviation angle between a feature point connecting line and a reference line segment; adjusting the coordinate system of the robot according to the deviation angle, determining that the adjustment operation is completed; controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again. The embodiment of the application can effectively improve the positioning accuracy of the workpiece and meet the demand of high-precision operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to a robot positioning method and system, an electronic device, and a storage medium. BACKGROUND

[0002] At present, in the machining operation of large-size workpieces such as engines, a robot is often required to perform a preset operation. Such workpieces have a large size, and the cross-sectional width often reaches 500 mm or more. When the workpieces are conveyed to the operation station by a tray, the placement position is prone to deviation and poor consistency. In the prior art, a visual positioning guide robot operation is often used, and a 3D vision system is commonly used to obtain spatial position information of the workpiece. However, the 3D vision system has low resolution in a large field of view, and the positioning accuracy is greatly affected by the rotation angle and size of the workpiece, and the error can reach 2 mm or more. This makes it difficult for the robot coordinate system to accurately match the actual position of the workpiece, and it is difficult to meet the requirements of high-precision operation on positioning accuracy, thereby affecting the quality and efficiency of subsequent operation. SUMMARY

[0003] Embodiments of the present application provide a robot positioning method and system, an electronic device, and a storage medium to alleviate or solve the technical problem of insufficient robot positioning accuracy in the prior art, which makes it difficult to meet the requirements of high-precision operation.

[0004] In a first aspect, an embodiment of the present application provides a robot positioning method, which is applied to control a robot carrying a camera. The method comprises performing an adjustment operation on a coordinate system of the robot, and the adjustment operation comprises:

[0005] controlling the robot to move to a first photographing position of the camera, obtaining a first positioning feature point on a first surface of a workpiece photographed by the camera carried on the robot, and obtaining a first image;

[0006] controlling the robot to move to a second photographing position, obtaining a second positioning feature point on the first surface photographed by the camera, and obtaining a second image; the second photographing position is a position obtained by moving the first photographing position by a preset distance in a first direction;

[0007] unifying image coordinates of different positioning feature points in the same image according to the preset distance;

[0008] calculating a deviation angle between a feature point connecting line and a reference line segment according to first image coordinates and second image coordinates, the first image coordinates and the second image coordinates being image coordinates of the first positioning feature point and the second positioning feature point in the same image, and the first positioning feature point and the second positioning feature point forming the feature point connecting line;

[0009] adjusting a coordinate system of the robot according to the deviation angle, determining whether the adjusting operation is completed;

[0010] controlling the robot to move to another side of the workpiece, and performing the adjusting operation on the coordinate system again.

[0011] In some embodiments of the present application, the first positioning feature point is an angle point formed by the intersection of the first edge and the second edge on the first surface. Before the image coordinates of different positioning feature points are unified in the same image according to the preset distance, the method further comprises:

[0012] matching the first edge and the second edge in the first image based on a first image template to obtain a first coarse positioning of the workpiece in the first image;

[0013] placing a preset region of interest template in the first image according to the first coarse positioning to determine a region of interest in the first image;

[0014] extracting edges in the region of interest to obtain the first edge and the second edge;

[0015] obtaining the first image coordinates according to the intersection coordinates of the first edge and the second edge, the first image coordinates being the coordinates of the first positioning feature point in the first image.

[0016] In some embodiments of the present application, the second positioning feature point is an angle point formed by the second edge and a third edge on the first surface. The method of unifying the image coordinates of different positioning feature points in the same image according to the preset distance comprises:

[0017] obtaining the second image coordinates according to the sum of the third image coordinates and the preset distance, the third image coordinates being the coordinates of the second positioning feature point in the second image, and the second image coordinates being the coordinates of the second positioning feature point in the first image.

[0018] In some embodiments of the present application, the robot carries a ranging sensor. After the coordinate system is adjusted again to perform the same adjusting operation as the first surface, the method further comprises:

[0019] controlling the robot to move to a first ranging point and a second ranging point of the ranging sensor opposite the workpiece operation surface according to the adjusted coordinate system;

[0020] obtaining a first distance measured by the ranging sensor at the first ranging point and a second distance measured by the ranging sensor at the second ranging point;

[0021] According to a difference between the first distance and the second distance being less than a preset threshold, it is determined that the adjustment of the coordinate system is valid, otherwise, it is determined that the adjustment of the coordinate system is invalid.

[0022] In some embodiments of the embodiments of the present application, the robot carries a working head, and after it is determined that the adjustment of the coordinate system is valid, the method further comprises:

[0023] controlling the robot to move to a distance between the robot and the working surface satisfying a working distance condition;

[0024] controlling the robot to move along a preset trajectory, and the working head is configured to perform a preset working on the working surface according to movement of the preset trajectory, the preset trajectory being a movement trajectory projected onto the working surface.

[0025] In some embodiments of the embodiments of the present application, the preset working is configured to form a working trajectory on the working surface, and after the robot is controlled to move along the preset trajectory, the method further comprises:

[0026] controlling the robot to move to a target photographing position, and obtaining a third image obtained by the camera photographing the working surface;

[0027] extracting the working trajectory from the third image;

[0028] matching the working trajectory with an image template of the preset trajectory;

[0029] measuring a deviation value of the working trajectory from the preset trajectory according to a matching result.

[0030] In a second aspect, the embodiments of the present application provide a positioning system of a robot, comprising:

[0031] a robot, the robot carrying a camera;

[0032] The control device is configured to perform an adjustment operation on the coordinate system of the robot, and the adjustment operation comprises: controlling the robot to move to a first photographing position of the camera, obtaining a first image by capturing first positioning feature points on a first surface of the workpiece by the camera carried on the robot; controlling the robot to move to a second photographing position, obtaining a second image by capturing second positioning feature points on the first surface by the camera; the second photographing position is a position obtained by moving the first photographing position in a first direction by a preset distance; unifying image coordinates of different positioning feature points in the same image according to the preset distance; calculating a deviation angle between a feature point connecting line and a reference line segment according to first image coordinates and second image coordinates, the first image coordinates and the second image coordinates being image coordinates of the first positioning feature points and the second positioning feature points in the same image, and the first positioning feature points and the second positioning feature points forming the feature point connecting line; adjusting the coordinate system of the robot according to the deviation angle, and determining that the adjustment operation is completed; and controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again.

[0033] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory. The processor implements any of the methods provided in the embodiments of the present application when executing the computer program.

[0034] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement any of the methods provided in the embodiments of the present application.

[0035] In a fifth aspect, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement any of the methods provided in the embodiments of the present application.

[0036] Based on any of the above technical solutions, the present application has at least the following advantages:

[0037] The method comprises performing an adjustment operation on a coordinate system of the robot, the adjustment operation comprising: controlling the robot to move to a first photographing position of the camera, acquiring a first image of a first positioning feature point on a first surface of the workpiece captured by the camera carried on the robot; controlling the robot to move to a second photographing position, acquiring a second image of a second positioning feature point on the first surface captured by the camera; the second photographing position is a position obtained after the first photographing position is moved by a preset distance in a first direction; uniformly arranging image coordinates of different positioning feature points in the same image according to the preset distance; calculating a deviation angle between a feature point connecting line and a reference line segment according to first image coordinates and second image coordinates, the first image coordinates and the second image coordinates being image coordinates of the first positioning feature point and the second positioning feature point in the same image, and the first positioning feature point and the second positioning feature point forming the feature point connecting line; adjusting the coordinate system of the robot according to the deviation angle, and determining that the adjustment operation is completed; controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again. The application adjusts the coordinate system by collecting the positioning feature points through multiple photographing positions and calculating the deviation angle, and combines the multi-plane repeated positioning calibration, so that the positioning error caused by the inclination of the workpiece can be effectively eliminated, the positioning accuracy of the robot is improved, the robot can accurately align the work position in subsequent work, the demand for high-precision work is met, and the work universality can be also enhanced.

[0038] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0039] In the drawings, like reference numerals refer to same or similar components throughout the several views, unless otherwise specified. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments in accordance with the present application and should not be considered as limiting the scope of the present application.

[0040] Figure 1 A schematic block diagram of a positioning system of a robot is shown;

[0041] Figure 2 A three-dimensional schematic diagram of a positioning system of a robot is shown;

[0042] Figure 3 A detailed schematic diagram of the end of the robot of the positioning system of the robot is shown;

[0043] Figure 4A schematic diagram of a shooting position of a positioning system of a robot is shown according to an embodiment of the present application.

[0044] Figure 5 A flowchart of a positioning method of a robot is shown according to an embodiment of the present application Figure 1 .

[0045] Figure 6 An image processing schematic diagram of a positioning method of a robot is shown according to an embodiment of the present application

[0046] Figure 7 A flowchart of a positioning method of a robot is shown according to an embodiment of the present application Figure 2 .

[0047] Figure 8 A flowchart of a positioning method of a robot is shown according to an embodiment of the present application Figure 3 .

[0048] Figure 9 A flowchart of a positioning method of a robot is shown according to an embodiment of the present application Figure 4 .

[0049] Figure 10 A block diagram of an electronic device is shown according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following, only some exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the concept or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature, rather than limiting.

[0051] In order to facilitate understanding of the technical solutions of the embodiments of the present application, the related technologies of the embodiments of the present application are described below. The following related technologies can be combined with the technical solutions of the embodiments of the present application in any way as optional solutions, which all belong to the protection scope of the embodiments of the present application. It should be noted that the application scenarios or application examples provided in the present application are for the convenience of understanding, and the application of the technical solutions of the embodiments of the present application is not limited specifically.

[0052] At present, when the engine body needs to be coated, due to the large size of the engine body, the cross-sectional width of the engine with 12 or more cylinders is 500 mm or more, manual coating is time-consuming and laborious, and the coating is uneven. Therefore, in order to improve the efficiency and coating quality, a visual positioning guided robot is used for automatic coating. Because the placement position of the workpiece has poor consistency, the workpiece product is installed in a tray and transported through a roller line, and whether it is in place is detected through a photoelectric sensor, so the position of the stop is inconsistent, and the angle on the XOY and YOZ planes will also be offset, so positioning in space is needed, and usually such a scene will use a 3D camera, but the 3D camera has a large field of view (500 mmX500 mm) and low resolution, and the accuracy is poor, usually the error can reach 0.5 mm, and considering the angle offset factor, the error can reach more than 1 mm, therefore, the 3D visual guidance cannot meet the requirement of 0.08 mm accuracy of this station. The positioning method and system of the robot provided in the embodiments of the present application solve the above technical problems.

[0053] The technical solutions of the present application and how the technical solutions of the present application solve the foregoing technical problems will be described in detail below with specific embodiments. Several specific embodiments listed can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described in detail below with reference to the drawings.

[0054] Referring to Figures 1-3 The positioning system of the robot provided in the embodiments of the present application. The system comprises:

[0055] The robot 101 and the control device 102, the robot carries a camera 103, which is a 2D camera. Optionally, it can also carry a ranging sensor 104 and a work head 105, wherein the work head 105 can be a glue gun for performing the coating work.

[0056] Exemplarily, the robot can be an industrial robot with multi-degree-of-freedom motion capability, which can be in the form of a mechanical arm as shown in Figures 2-3 The robot's motion trajectory and pose are controlled by the control device. The control device can integrate data processing, instruction sending and logical judgment functions. The control device can also receive data feedback from the camera and the ranging sensor, and output control instructions. When the control device performs the adjustment operation, it first sends instructions to control the robot to move to a first shooting position, the camera shoots the first positioning feature point of the workpiece to obtain a first image and transmits it to the control device. Then the control device instructs the robot to move a preset distance in a first direction to a second shooting position, and the camera shoots the second positioning feature point to obtain a second image. The first direction can be a direction along any axis of the coordinate system, or it can be other directions.

[0057] After the control device receives the two images, the second image coordinates are determined by an internal algorithm according to the first image coordinates and the preset distance. The first image coordinates and the second image coordinates are image coordinates of the first positioning feature point and the second positioning feature point in the same image respectively. Since the moving distance between the first shooting position and the second shooting position is known, the difference between the image coordinate systems of the two images can be directly determined based on the moving distance. Alternatively, the conversion relationship between the image coordinate system and the real coordinate system can be calibrated in advance for conversion.

[0058] The feature point connecting line formed based on the two positioning feature points can represent the position deviation of the workpiece. Specifically, the feature point connecting line is compared with a preset reference line segment, and a deviation angle is calculated. The reference line segment is the reference position of the feature point connecting line, and the angle between the two line segments is the deviation angle. The control device generates a coordinate system adjustment instruction according to the deviation angle and sends it to the robot to complete the coordinate system adjustment. Then the control device instructs the robot to move to the other side of the workpiece again, and repeats the above adjustment process to realize multi-plane accurate positioning.

[0059] In a specific implementation, reference Figure 2 is an automatic rocker cover installation system. An industrial robot carries a 2D camera to take pictures of an engine (workpiece), carries a distance measuring sensor to measure the distance between the engine surface, and carries a glue gun to apply glue on the engine surface. Reference Figure 5 , the control device instructs the robot to take a picture of the first corner point of the first side (XOY plane) of the engine at the first shooting position, and then moves to the second shooting position to take a picture of the second corner point. After calculating the deviation angle to adjust the coordinate system of the robot, the robot is instructed to repeat the adjustment on the other side (YOZ plane) of the engine. Specifically, the robot is instructed to take a picture of the third corner point of the second side (YOZ plane) of the engine at the third shooting position, and then moves to the fourth shooting position to take a picture of the fourth corner point. The deviation angle is calculated and the coordinate system of the robot is adjusted.

[0060] The embodiments of the present application control the movement of the robot through the control device, so that the camera takes pictures at multiple shooting positions. Through point coordinate and angle calculation, the coordinate system of the robot is adjusted, realizing the automation and precision of robot positioning adjustment, reducing manual intervention, and improving positioning efficiency and accuracy.

[0061] The embodiments of the present application also provide a positioning method of a robot, which is applied to control a robot carrying a camera. For example, the positioning method can be executed by the control device in the positioning system of the robot provided by the embodiments of the present application. The positioning system of the robot provided by the embodiments of the present application and the positioning method of the robot provided by the embodiments of the present application can be mutually referenced for the specific implementation and technical effects, and the same parts will not be described again.

[0062] Reference Figure 5A flowchart of a positioning method of the robot is shown, the method comprising a step 51 of performing an adjustment operation on a coordinate system of the robot, the adjustment operation comprising:

[0063] Step 511, controlling the robot to move to a first photographing position of the camera, and acquiring a first image of a first positioning feature point on a first surface of the workpiece captured by the camera carried on the robot;

[0064] Step 512, controlling the robot to move to a second photographing position, and acquiring a second image of a second positioning feature point on the first surface captured by the camera; the second photographing position is a position obtained after the first photographing position is moved by a preset distance in a first direction;

[0065] Step 513, according to the preset distance, unifying the image coordinates of different positioning feature points in the same image;

[0066] Step 514, according to the first image coordinates and the second image coordinates, calculating a deviation angle between a feature point connecting line and a reference line segment, the first image coordinates and the second image coordinates being image coordinates of the first positioning feature point and the second positioning feature point in the same image, and the first positioning feature point and the second positioning feature point forming the feature point connecting line;

[0067] Step 515, according to the deviation angle, adjusting the coordinate system of the robot, and determining that the adjustment operation is completed;

[0068] Step 52, controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again.

[0069] The first positioning feature point and the second positioning feature point are features with clear recognizability on the surface of the workpiece, and can be selected as corner points formed by intersecting edges on the surface of the workpiece. The camera is a 2D industrial camera, and when the robot is controlled to move, the robot end carries the camera to the first photographing position according to a preset program, and captures the first image of the first surface of the workpiece. The first direction is a preset fixed moving direction, such as along the width direction of the workpiece, and the preset distance is set in advance according to the size parameters of the workpiece, so that the second photographing position can capture the second positioning feature point.

[0070] The first image coordinates are image coordinates of the first positioning feature point in the first image or the second image, and the second image coordinates are image coordinates of the second positioning feature point in the same image. It can be understood that the image coordinates of the first positioning feature point can be unified to the second image, or the image coordinates of the second positioning feature point can be unified to the first image, and the latter is taken as an example for description in the embodiments of the application. After acquiring the first image coordinates of the first positioning feature point in the first image and the third image coordinates of the second positioning feature point in the second image, the third image coordinates are converted into the second image coordinates by spatial coordinate conversion logic combined with the preset distance.

[0071] The feature point connecting line is formed by connecting two positioning feature point coordinates, and the position of the reference line segment is a standard position formed by the two positioning feature points when the workpiece is not deviated. The deviation angle is obtained by calculating the included angle between the feature point connecting line and the reference line segment. According to the deviation angle, the robot coordinate system is adjusted to match the coordinate system with the actual position of the workpiece. After completing the adjustment of the first surface, the robot is controlled to move to the other surface of the workpiece, and the above complete adjustment process is repeated to realize multi-surface positioning calibration and ensure the accuracy of subsequent operations.

[0072] By collecting positioning feature points through multiple shooting positions and adjusting the coordinate system by calculating the deviation angle, combined with multi-surface repeated positioning calibration, the positioning error caused by the inclination of the workpiece can be effectively eliminated, the positioning accuracy of the robot can be improved, and the robot can accurately align the operation position in subsequent operations to meet the high-precision operation requirements. At the same time, it is suitable for workpieces in different placement states and enhances the universality of the operation.

[0073] In some embodiments of the embodiments of the present application, the first positioning feature point is an angle point formed by the intersection of the first edge and the second edge on the first surface. Before unifying the image coordinates of different positioning feature points in the same image according to the preset distance, the method further comprises: matching the first edge and the second edge in the first image based on the first image template to obtain the first coarse positioning of the workpiece in the first image; placing a preset region of interest template in the first image according to the first coarse positioning to determine the region of interest in the first image; extracting edges in the region of interest to obtain the first edge and the second edge; and obtaining the first image coordinates according to the intersection coordinates of the first edge and the second edge, the first image coordinates being the coordinates of the first positioning feature point in the first image.

[0074] The first edge and the second edge are two edges intersecting to form the first positioning feature point on the first surface of the workpiece, and have obvious contour features. The first image template is a template established in advance for the standard image of the workpiece, and contains standard contour information of the first edge and the second edge. Optionally, the first image template can be obtained by processing a photo taken when the workpiece is in a reference position, and the specific processing can include image preprocessing such as brightness enhancement, corrosion, etc., and further marking the region of interest of the first edge and the second edge in the preprocessed image to determine the region of interest of edge extraction after matching.

[0075] In the first image, the first coarse positioning is achieved by finding the region matching the first edge and the second edge in the standard template through the image matching algorithm, and the region range of the feature (the first edge and the second edge) is locked. The region of interest template is a region template of a preset specific shape and size, and the template is placed in the corresponding region in the first image according to the first coarse positioning result, so as to frame the effective region containing the first edge and the second edge and reduce irrelevant background interference. In the region of interest, an edge detection algorithm can be used to extract the edge, and the profile lines of the first edge and the second edge are identified through pixel gray value change analysis. According to the pixel coordinates of the two edge lines, the intersection coordinates are obtained through geometric calculation, and the coordinates are the first image coordinates of the first positioning feature point.

[0076] The coarse positioning is achieved through the template matching, the effective range is locked by using the region of interest template, the positioning feature point coordinates are accurately obtained by combining the edge extraction algorithm, the background interference and the recognition error can be reduced, and the accuracy and efficiency of the image coordinate acquisition are improved.

[0077] In some embodiments of the embodiments of the present application, the second positioning feature point is an angle point formed by the second edge and the third edge on the first surface, and the image coordinates of different positioning feature points are unified in the same image according to a preset distance, including: obtaining the second image coordinates according to the sum of the third image coordinates and the preset distance, the third image coordinates being the coordinates of the second positioning feature point in the second image, and the second image coordinates being the coordinates of the second positioning feature point in the first image.

[0078] The second positioning feature point is formed by the intersection of the second edge and the third edge on the first face, belongs to the first face like the first positioning feature point, and has similar corner features, so the same recognition logic can be used to obtain the coordinates. First, the first image template is compared with the second image to find the contour regions corresponding to the second edge and the third edge, and the second coarse positioning is completed. According to the coarse positioning result, a region of interest template with the same specification as that used to obtain the first image coordinates is placed in the second image. Then, edge detection technology is used in the region of interest to accurately extract the contour lines of the second edge and the third edge, and the intersection coordinates of the two edges are obtained through geometric operation. The coordinates are the third image coordinates of the second positioning feature point in the second image. The same acquisition method can ensure the consistency and accuracy of the coordinates of the two positioning feature points, avoid errors caused by differences in recognition methods, and ensure the reliability of subsequent line segment formation and angle calculation. Next, the third image coordinates and the preset distance are summed to obtain the second image coordinates. The preset distance is the image pixel distance corresponding to the actual displacement of the robot from the first shooting position to the second shooting position. Adding the corresponding dimension (consistent with the moving direction) of the third image coordinates to the pixel value of the preset distance can obtain the second image coordinates of the second positioning feature point in the first image. This step unifies the coordinates of the feature points under different shooting positions to the same image coordinate system, providing a unified coordinate reference for subsequent calculation of the deviation angle of the line segment and the reference line segment.

[0079] The embodiments of the present application use the same acquisition method as the first image coordinates to ensure the uniformity and accuracy of the coordinates of the two positioning feature points, reduce errors caused by different recognition methods, make the feature point line formed subsequently more accurately reflect the actual position state of the workpiece, provide accurate basis for deviation angle calculation, and further improve the accuracy of coordinate system adjustment.

[0080] In some embodiments of the embodiments of the present application, the robot carries a distance measuring sensor. After performing the same adjustment operation on the coordinate system as the first face again, the method further comprises: controlling the robot to move to the first distance measuring point and the second distance measuring point of the distance measuring sensor opposite the workpiece working surface according to the adjusted coordinate system; obtaining the first distance measured by the distance measuring sensor at the first distance measuring point and the second distance measured at the second distance measuring point; and determining that the adjustment of the coordinate system is effective if the difference between the first distance and the second distance is less than a preset threshold, otherwise, determining that the adjustment of the coordinate system is invalid.

[0081] The distance measuring sensor is a component with distance measuring function, which can be a laser distance measuring sensor, and can be carried together with the camera at the end of the robot to feed back distance data in real time. The first distance measuring point and the second distance measuring point can be selected as positions near two right angle points on the workpiece work surface, which can reflect the overall position state of the work surface. After the coordinate system adjustment is completed, the robot is controlled to move according to the adjusted coordinate parameters, so that the distance measuring sensor faces the two distance measuring points respectively. At each distance measuring point, the distance measuring sensor emits a measuring signal, and after receiving the reflected signal, the straight-line distance between the sensor and the work surface, i.e., the first distance and the second distance, is calculated. The preset threshold is the maximum allowed distance difference value set according to the work accuracy requirement, such as 0.3 mm. The distance difference value obtained by two measurements is compared with the preset threshold. If the difference value is less than the threshold, it means that the position of the work surface matches the coordinate system adjustment result, the positioning is accurate, and the coordinate system adjustment is effective. If the difference value is greater than or equal to the threshold, it means that there is a deviation in positioning, the coordinate system adjustment is invalid, and the adjustment operation needs to be performed again. For example, the robot carries the distance measuring sensor to measure the distance at two right angle distance measuring points of the engine glue coating work surface to obtain distances L1 and L2, calculate the difference value, and if the difference value is less than 0.3 mm and is between 1 mm and 3 mm, it is determined that the coordinate system adjustment is effective.

[0082] The embodiment of the present application can quickly verify the effectiveness of the coordinate system adjustment result by measuring the distance at the preset distance measuring point by the distance measuring sensor and comparing the difference value, timely find the positioning deviation, avoid the failure of subsequent work caused by improper coordinate system adjustment, ensure the accuracy of positioning before work, and further improve the reliability and success rate of the overall work.

[0083] In some embodiments of the embodiment of the present application, after it is determined that the adjustment of the coordinate system is effective, the method further includes: controlling the robot to move to a distance from the work surface that satisfies a work distance condition; and controlling the robot to move along a preset trajectory, and the work head is used to cooperate with the movement of the preset trajectory to perform a preset work on the work surface. The preset trajectory is a movement trajectory projected onto the work surface.

[0084] The operation head is an execution component such as a glue gun, which is adapted to the preset operation and is carried together with the camera and the distance sensor at the end of the robot and can move accurately with the robot. The operation distance condition is the optimal distance range between the operation head and the operation surface set according to the operation requirement, which ensures the operation effect. After determining that the coordinate system adjustment is effective, the robot is controlled to move according to the operation distance parameter, so that the distance between the operation head and the workpiece operation surface reaches the set range. The preset trajectory is a motion path planned in advance according to the operation requirement and stored in the control device. The trajectory is projected onto the operation surface and consistent with the required trajectory. The robot is controlled to move accurately according to the preset trajectory, and the operation head is started synchronously during the movement to perform the corresponding operation on the operation surface according to the preset operation requirement, such as glue coating, so that the operation trajectory and the preset trajectory are accurately overlapped to ensure the operation quality. For example, the robot carries a glue gun as the operation head, adjusts the distance between the glue gun and the engine glue coating operation surface to the operation range of 1mm to 3mm, controls the robot to move along the preset glue coating trajectory, and the glue gun synchronously coats glue to form a glue coating trajectory.

[0085] After confirming that the coordinate system adjustment is effective, the robot is controlled to move the operation head to the optimal operation distance and along the preset trajectory in the embodiments of the present application, so that the operation head can accurately perform the preset operation, the operation trajectory and the preset trajectory are highly consistent, the operation quality and consistency are improved, the high-precision operation requirement is met, and the operation efficiency is improved.

[0086] In some embodiments of the embodiments of the present application, the preset operation is used to form an operation trajectory on the operation surface. After controlling the robot to move along the preset trajectory, the method further comprises: controlling the robot to move to a target shooting position, obtaining a third image obtained by the camera shooting the operation surface; extracting the operation trajectory in the third image; matching the operation trajectory with an image template of the preset trajectory; and measuring the deviation value of the operation trajectory and the preset trajectory according to the matching result.

[0087] The target photographing position is a preset position that can completely photograph the workpiece work surface, ensuring that the camera can clearly capture the overall track after work. After the preset work is completed, the robot is controlled to move so that the camera reaches the target photographing position, and the work surface is photographed to obtain a third image. The image contains the complete work track. In the third image, a deep learning classification algorithm can be used to extract the work track, and by analyzing the image pixel features, the work track and the background area can be accurately separated. The preset track image template is a standard image template made in advance according to the preset track, which contains features such as the shape and size of the track. The extracted work track is matched with the image template, and the degree of coincidence of the two is analyzed by an image comparison algorithm. According to the matching result, a corresponding measurement tool, such as a caliper tool, is used to measure along the normal direction of the track segment to obtain the deviation value of the work track from the preset track in terms of position, size, etc., providing data support for work quality evaluation. For example, after the glue coating is completed, the robot can carry the camera to the target photographing position to photograph the glue coating work surface, extract the glue coating track, match it with the preset glue coating track template, measure the track width and position deviation value using the image caliper tool, and evaluate the quality of the glue coating track.

[0088] The embodiments of the present application can intuitively and accurately evaluate the work quality by photographing the work surface image, extracting the work track, matching it with the preset track template, and measuring the deviation value. This can timely find the work deviation, provide a basis for subsequent work parameter adjustment, help to continuously optimize the work effect, and improve the work precision and product quality.

[0089] Reference is made below to Figures 6-9 to provide an implementation process of the embodiments of the present application in a specific application scenario.

[0090] Reference is made to Figure 6 and Figure 7 for the purpose of Figure 4The positioning adjustment corresponding to the XOY plane is shown. First, the sensor is determined to stop after detecting the workpiece tray, the code gun scans the code to determine the workpiece type, and the positioning program corresponding to the workpiece is switched, and the robot carrying the camera moves to the shooting position 1 to position. Then, the photograph is taken at the shooting position 1. After that, the right angle edge is positioned by template matching, and the edge finding tool is used to detect and obtain two right angle edge straight lines A1 and B1, respectively, and the intersection point O1(X1, Y1) of the two straight lines is calculated. With the help of template matching to lock the right angle edge area, the edge profile is extracted and fitted into a straight line by using the edge finding tool, and the coordinates of the feature point O1 are obtained through geometric operation, so as to realize the positioning of the first feature point. Then, the robot is translated along the Y axis to the shooting position 2 to take a picture, and the positioning logic is repeated, the right angle edge is positioned by template matching, and the edge finding tool is used to detect and obtain two right angle edge straight lines A2 and B2, respectively, and the intersection point O3(X3, Y3) of the two straight lines is calculated to obtain the coordinates of the second feature point O3. Then, the image coordinates O2(X2, Y2) of the corner point O3 in the shooting position 1 are calculated by combining the coordinate difference of the shooting positions 1 and 2, and the specific formula is as follows: O2(X2, Y2) = O3(X3, Y3) + shooting position 1(X4, Y4) - shooting position 2(X5, Y5). The line segment O1O2 is created, and according to the deviation angle of the line segment O1O2 and the reference line segment, the rotation angle RZ1 of the workpiece around the Z axis is determined. Then, the following is calculated:

[0091] XOY plane X axis offset amount DX1 = X1 - reference X6;

[0092] XOY plane Y axis offset amount DY1 = Y1 - reference Y6;

[0093] XOY plane Z axis rotation angle offset amount DRZ1 = RZ1 - reference RZ2.

[0094] After calculating the X, Y axis offset amount and Z axis rotation angle offset amount of the XOY plane, the data is sent to the robot, and the robot completes the coordinate system offset adjustment.

[0095] Reference Figure 8After the adjustment of the XOY plane, the robot carrying the camera moves to the photographing position 3 to take a picture. After obtaining the image of the workpiece XOY plane, the right-angle edge region is located through template matching, and the straight line A3 and the straight line B3 of the two right-angle edges are detected by the edge finding tool. Then, the intersection O4 (Z1, Y7) of the two straight lines is calculated through geometric operation. Subsequently, the control device instructs the robot to move to the photographing position 4 to take a picture. Similarly, the right-angle edge is located through template matching, and the straight line A4 and the straight line B4 are detected by the edge finding tool. The intersection O5 (Z3, Y8) of the two straight lines is calculated. In order to unify the coordinate reference, the coordinates O6 (Z2, Y9) of the intersection O5 in the O4 image coordinate system are calculated as O6 (Z2, Y9) = O4 (Z1, Y7) + photographing position 3 (Z4, Y9) - photographing position 4 (Z5, Y10). The O5 coordinates collected by the photographing position 4 are converted into the O6 coordinates in the photographing position 3 coordinate system, and the spatial deviation of different photographing positions is eliminated. Then, the line segment O4O6 is created with O4 and O6 as endpoints. The angle between the line segment and the reference direction is calculated to obtain the Z-axis rotation angle RX1 of the XOY plane. Then, the offset is calculated as follows:

[0096] The X-axis offset DZ1 of the XOZ plane is calculated as DZ1 = Z1 - reference Z6.

[0097] The Y-axis offset DY2 of the XOZ plane is calculated as DY2 = Y7 - reference Y11.

[0098] The X-axis rotation angle offset DRX1 of the XOZ plane is calculated as DRX1 = RX1 - reference RX2.

[0099] Meanwhile, the total offset in the Y direction is calculated as DY3 = DY1 + DY2 by combining DY3 with the DY1 obtained from the XOY plane calibration. Then, the offset data DZ1, DY3 and DRX1 are sent to the robot. After receiving the data, the robot drives the joint movement to adjust the coordinate system of itself. Finally, the coordinate system calibration of the XOZ plane is completed, and the precise matching of the spatial position of the robot and the workpiece is realized.

[0100] Reference Figure 9, for positioning verification and glueing quality detection. First, the robot carries a distance sensor to measure the relative distance L1, L2 from the camera to the glue gun at the shooting position 3 and the shooting position 4, respectively. This step preliminarily verifies the rationality of the position of the glue gun and the work surface through double-point distance measurement. Then, it is first determined whether "1mm < L1 < 3mm and 1mm < L2 < 3mm" is true. This interval is the effective distance range for the glue gun operation. If it is not satisfied, the visual positioning fails. If it is satisfied, it is further determined whether "|L1-L2| < 0.3mm" is true. This threshold is used to verify the flatness of the work surface. If it is not satisfied, the visual positioning also fails. When both conditions are met, the robot carries the glue gun to perform the glueing operation, and after completion, the robot carries the camera to take pictures to detect the glueing quality, and the actual trajectory image after operation is collected by the vision equipment. Subsequently, the glueing area is extracted by deep learning, the glueing trajectory and the background are accurately separated by using the algorithm, and the starting position of the glueing is located by template matching to determine the reference endpoint of the trajectory. Then, combined with the preset standard glueing trajectory, the standard trajectory is moved to the actual glueing area using the following function to realize the alignment of the standard and actual trajectories. Then, along the standard trajectory, the glueing width L3 is measured in the normal direction using the caliper tool at an interval of 2mm. This interval can evenly cover the key areas of the trajectory. Then, it is determined whether "1mm < L3 < 3mm" is true. This range is the qualified standard of the glueing width. If it is satisfied, the glueing is qualified, otherwise the glueing is unqualified, and the final process ends.

[0101] Figure 10 A block diagram of an electronic device for implementing the embodiments of the present application is shown. As shown in Figure 10 , the electronic device includes a memory 1001 and a processor 1002, and the memory 1001 stores a computer program that can run on the processor 1002. The processor 1002 implements the method in the above embodiments when executing the computer program. The number of memories 1001 and processors 1002 can be one or more. In specific implementation, the electronic device can further include a communication interface 1003 for communicating with external devices and performing data transmission.

[0102] In a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are independently implemented, the memory 1001, the processor 1002 and the communication interface 1003 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 10 In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0103] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can complete communication between each other through an internal interface.

[0104] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the method provided in the embodiment of the present application.

[0105] The embodiment of the present application provides a computer program product, which includes a computer program, and the program is executed by a processor to implement the method provided in the embodiment of the present application.

[0106] The embodiment of the present application further provides a chip, which includes a processor, is used for calling and running instructions stored in a memory, so that a communication device installed with the chip executes the method provided in the embodiment of the present application.

[0107] The embodiment of the present application further provides a chip, which includes an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used for executing code in the memory, and when the code is executed, the processor is used for executing the method provided in the embodiment of the present application.

[0108] It is to be understood that the above-described processor can be a CPU (Central Processing Unit), and can also be other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or any conventional processor, and the like. It is to be noted that the processor can be a processor supporting an Advanced RISC Machines (ARM) architecture.

[0109] Further, the above-described memory can include a read-only memory and a random access memory. The memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can include a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can include a Random Access Memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available. For example, a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Sync Link DRAM (SLDRAM), and a Direct Rambus RAM (DR RAM) can be used.

[0110] In the above-described embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded on a computer, all or part of the processes or functions according to the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium.

[0111] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, a person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0112] In addition, the terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0113] Any process or method described in the flowchart or otherwise described herein can be understood as a representation of code including one or more executable instructions for implementing specific logical functions or processes. Also, the scope of the preferred embodiments of the present disclosure includes additional implementations, in which the functions can be performed in an order other than that shown or discussed, including in a substantially simultaneous manner or in reverse order, according to the functions involved.

[0114] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing the logic function, which can be specifically implemented in any computer-readable medium for instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can take instructions from an instruction execution system, device or equipment and execute instructions) or in conjunction with these instructions execution system, device or equipment.

[0115] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-mentioned embodiment methods can be completed by a program instructing the relevant hardware, which can be stored in a computer readable storage medium and includes one or a combination of the steps of the embodiment methods when executed.

[0116] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. The above-mentioned integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0117] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A positioning method of a robot, characterized by, The method is applied to control a robot carrying a camera, and the method comprises performing an adjustment operation on a coordinate system of the robot, the adjustment operation comprising: controlling the robot to move to a first photographing position of the camera, and acquiring a first image of a first positioning feature point on a first surface of a workpiece photographed by the camera carried on the robot; controlling the robot to move to a second photographing position, and acquiring a second image of a second positioning feature point on the first surface photographed by the camera; the second photographing position is a position obtained by moving the first photographing position in a first direction by a preset distance; unifying image coordinates of different positioning feature points in the same image according to the preset distance; calculating a deviation angle between a feature point connecting line and a reference line segment according to first image coordinates and second image coordinates; the first image coordinates and the second image coordinates are image coordinates of the first positioning feature point and the second positioning feature point in the same image respectively, and the first positioning feature point and the second positioning feature point form the feature point connecting line; adjusting the coordinate system of the robot according to the deviation angle to determine that the adjustment operation is completed; controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again.

2. The method of claim 1, wherein, The first positioning feature point is a corner point formed by the intersection of a first edge and a second edge on the first surface, and before unifying image coordinates of different positioning feature points in the same image according to the preset distance, the method further comprises: matching the first edge and the second edge in the first image based on a first image template to obtain a first coarse positioning of the workpiece in the first image; placing a preset region of interest template in the first image according to the first coarse positioning to determine a region of interest in the first image; extracting edges in the region of interest to obtain the first edge and the second edge; obtaining the first image coordinates according to the intersection coordinates of the first edge and the second edge; the first image coordinates are coordinates of the first positioning feature point in the first image.

3. The method of claim 2, wherein, The second positioning feature point is a corner point formed by the second edge and a third edge on the first surface, and the unifying image coordinates of different positioning feature points in the same image according to the preset distance comprises: obtaining the second image coordinates according to the sum of third image coordinates and the preset distance; the third image coordinates are coordinates of the second positioning feature point in the second image, and the second image coordinates are coordinates of the second positioning feature point in the first image.

4. The method of claim 1, wherein, The robot carries a distance measuring sensor, and after performing the same adjustment operation on the coordinate system as the first surface again, the method further comprises: controlling the robot to move to a first distance measuring point and a second distance measuring point of the distance measuring sensor opposite to a working surface of the workpiece according to the adjusted coordinate system; acquiring a first distance measured by the distance measuring sensor at the first distance measuring point and a second distance measured by the distance measuring sensor at the second distance measuring point; According to a difference between the first distance and the second distance being less than a preset threshold, it is determined that the adjustment of the coordinate system is valid, otherwise, it is determined that the adjustment of the coordinate system is invalid.

5. The method of claim 4, wherein, The robot carries a work head, and after it is determined that the adjustment of the coordinate system is valid, the method further comprises: Controlling the robot to move to a distance from the work surface satisfying a work distance condition; Controlling the robot to move along a preset track, and the work head is configured to cooperate with movement of the preset track to perform a preset work on the work surface, and the preset track is a movement track projected onto the work surface.

6. The method of claim 5, wherein, The preset work is configured to form a work track on the work surface, and after controlling the robot to move along the preset track, the method further comprises: Controlling the robot to move to a target photographing position, and obtaining a third image obtained by the camera photographing the work surface; Extracting the work track in the third image; Matching the work track with an image template of the preset track; Measuring a deviation value of the work track from the preset track according to a matching result.

7. A positioning system for a robot, characterized in that The robot carries a camera; The control device is configured to perform an adjustment operation on a coordinate system of the robot, and the adjustment operation comprises: controlling the robot to move to a first photographing position of the camera, and obtaining a first image obtained by a camera carried by the robot photographing a first positioning feature point on a first surface of a workpiece; Controlling the robot to move to a second photographing position, and obtaining a second image obtained by the camera photographing a second positioning feature point on the first surface; the second photographing position is a position obtained by moving the first photographing position in a first direction by a preset distance; according to the preset distance, image coordinates of different positioning feature points are unified in the same image; according to first image coordinates and second image coordinates, calculating a deviation angle between a feature point connecting line and a reference line segment, the first image coordinates and the second image coordinates are image coordinates of the first positioning feature point and the second positioning feature point in the same image, and the first positioning feature point and the second positioning feature point form the feature point connecting line; adjusting the coordinate system of the robot according to the deviation angle, and determining that the adjustment operation is completed; controlling the robot to move to another surface of the workpiece, and performing the adjustment operation on the coordinate system again. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

8. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1-6.

10. A computer program product, characterised in that, ​

Images