Method, device and equipment for completing center teaching point location through automatic movement of camera and storage medium

Abstract

The invention discloses a method, device and equipment for completing a central teaching point location through automatic movement of a camera and a storage medium, and the method comprises the steps: controlling the camera to shoot the teaching point location, and recording a first coordinate where the camera is located and a first pixel position of the teaching point location; moving the camera to a second coordinate through the manipulator, controlling the camera to shoot the teaching point, and recording a second pixel position of the teaching point; and according to the first pixel position, the first coordinate, the second coordinate and the second pixel position, the mapping angle and the offset size between the camera and the manipulator are calculated. Controlling the manipulator to move, measuring the coordinates of the manipulator before and after movement and the pixel coordinates of the teaching potential shot by the camera, calculating the mapping angle and the offset size between the camera and the manipulator according to the measured coordinates, and calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point position according to the mapping angle and the offset size; and the camera center is automatically aligned with the teaching point position by moving the manipulator, so that the teaching precision and efficiency are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of automatic control technology, and particularly to a method, device, equipment and storage medium for a camera to automatically move to complete the center teaching point position. Background Art

[0002] In modern industrial automation production, the integration of a manipulator (Selective Compliance Assembly Robot Arm, SCARA) and a vision system has become an important development direction. Installing a camera on the SCARA manipulator and using the camera center as the end effector of the manipulator to perform visual guidance tasks can achieve precise recognition, positioning, and grasping of target objects, significantly improving production efficiency and operation accuracy. However, the SCARA manipulator currently faces many challenges in practical applications. First, when the manipulator leaves the factory, the default is to use the end of the manipulator body as the reference point of the end effector. In the visual application scenario, the camera center needs to be used as the end effector. This difference between the default setting and the actual requirement makes it necessary to redefine and calibrate the end of the manipulator before performing visual guidance operations. Otherwise, it will cause a deviation between the manipulator movement trajectory and the vision system positioning, and the target task cannot be accurately completed. Second, although the SCARA manipulator has a certain repeat positioning accuracy, the absolute accuracy of the manipulator body when it leaves the factory often fails to meet the requirements of high-precision visual guidance operations. During the teaching process, if the factory body accuracy is directly used, when the manipulator performs tasks according to visual information, due to the accumulation of absolute positioning errors, the camera center cannot accurately align with the target point, affecting the operation quality.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for a camera to automatically move to complete the center teaching point position, aiming to solve the technical problem of manually controlling the manipulator to complete the alignment of the camera center with the teaching point position.

[0005] To achieve the above purpose, this application proposes a method for a camera to automatically move to complete the center teaching point position. The method for a camera to automatically move to complete the center teaching point position includes: Controlling the camera to capture the teaching point position, and recording the first coordinate where the camera is located and the first pixel position of the teaching point position; Moving the camera to the second coordinate by the manipulator, controlling the camera to capture the teaching point position, and recording the second pixel position of the teaching point position; Calculate the mapping angle and offset dimension between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position; Calculate the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension; Control the manipulator to move the camera according to the coordinate movement amount.

[0006] In one embodiment, the step of moving the camera to the second coordinate by the manipulator includes: Control the joint angle of the robotic arm to adjust by a first preset angle, and the link rotation angle of the robotic arm to adjust in the reverse direction by the first preset angle, and record the position of the camera as the second coordinate; wherein, when the manipulator moves, ensure that the teaching point is within the shooting range of the camera.

[0007] In one embodiment, the step of calculating the mapping angle and offset dimension between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position includes: Calculate the coordinate movement vector of the camera according to the first coordinate and the second coordinate; Calculate the pixel movement vector of the teaching point according to the first pixel position and the second pixel position; Calculate the mapping angle and the offset dimension according to the coordinate movement vector and the pixel movement vector.

[0008] In one embodiment, the step of calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension further includes: Calculate a second pixel movement vector according to the pixel position of the center point and the second pixel position; Calculate the coordinate movement amount that the manipulator needs to move according to the second pixel movement vector, the mapping angle, and the offset dimension.

[0009] In one embodiment, after the step of controlling the manipulator to move the camera according to the coordinate movement amount, it includes: After the camera movement is completed, obtain the third pixel coordinate of the teaching point by controlling the camera to shoot the teaching point; When the difference between the third pixel coordinate and the center point is greater than a preset difference, execute the step of calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension.

[0010] In one embodiment, after the step of controlling the camera to capture the teaching point position after the movement of the camera is completed and obtaining the third pixel coordinates of the teaching point position, the following steps are included: When the difference between the third pixel coordinates and the center point is less than or equal to the preset difference, it is determined that the automatic teaching of the center point of the camera aligning with the teaching point position is completed.

[0011] In one embodiment, after the step of determining that the automatic teaching of the center point of the camera aligning with the teaching point position is completed, the following steps are included: Re-identify the teaching point position, and execute the step of calculating the coordinate movement amount of the manipulator when the center point of the camera aligns with the teaching point position according to the mapping angle and the offset dimension.

[0012] In addition, to achieve the above object, the present application also proposes a device for automatically moving a camera to complete the center teaching point position. The device includes: A first position acquisition module, configured to control the camera to capture the teaching point position and record the first coordinate where the camera is located and the first pixel position of the teaching point position; A second position acquisition module, configured to move the camera to the second coordinate by the manipulator, control the camera to capture the teaching point position, and record the second pixel position of the teaching point position; A mapping calculation module, configured to calculate the mapping angle and the offset dimension between the camera coordinates and the manipulator coordinates according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position; A third position calculation module, configured to calculate the coordinate movement amount of the manipulator when the center point of the camera aligns with the teaching point position according to the mapping angle and the offset dimension; A teaching completion module, configured to control the manipulator to move the camera according to the coordinate movement amount.

[0013] In addition, to achieve the above object, the present application also proposes a device for automatically moving a camera to complete the center teaching point position. The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the method for automatically moving a camera to complete the center teaching point position as described above.

[0014] In addition, to achieve the above object, the present application also proposes a storage medium. The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for automatically moving a camera to complete the center teaching point position as described above.

[0015] The present application discloses a method, device, equipment and storage medium for a camera to automatically move to complete the center teaching point, which relates to the technical field of automatic control. The method for a camera to automatically move to complete the center teaching point: control the camera to photograph the teaching point, and record the first coordinate where the camera is located and the first pixel position of the teaching point; move the camera to the second coordinate by a manipulator, control the camera to photograph the teaching point, and record the second pixel position of the teaching point; calculate the mapping angle and offset dimension between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate and the second pixel position; calculate the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension; control the manipulator to move the camera according to the coordinate movement amount. Control the manipulator to move, measure the coordinates of the manipulator before and after the movement and the pixel coordinates of the teaching potential photographed by the camera, calculate the mapping angle and offset dimension between the camera and the manipulator according to the measured coordinates, calculate the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension, and complete the automatic alignment of the camera center with the teaching point by moving the manipulator. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flow chart provided for the first embodiment of the method for a camera to automatically move to complete the center teaching point of the present application; Figure 2 It is a teaching point diagram provided for the first embodiment of the method for a camera to automatically move to complete the center teaching point of the present application; Figure 3 It is a schematic flow chart provided for the second embodiment of the method for a camera to automatically move to complete the center teaching point of the present application; Figure 4 It is a structural diagram provided for the second embodiment of the method for a camera to automatically move to complete the center teaching point of the present application; Figure 5 It is a schematic flow chart provided for the third embodiment of the method for a camera to automatically move to complete the center teaching point of the present application; Figure 6Schematic diagram of the module structure of the device for automatically moving the camera in the embodiment of the present application to complete the center teaching point position; Figure 7 Schematic diagram of the device structure of the equipment for automatically moving the camera in the hardware operating environment involved in the method for automatically moving the camera in the embodiment of the present application to complete the center teaching point position.

[0019] The implementation, functional features and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed implementation manners

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0022] The main solution of the embodiment of the present application is: controlling the camera to capture the teaching point position, and recording the first coordinate where the camera is located and the first pixel position of the teaching point position; moving the camera to the second coordinate by a manipulator, controlling the camera to capture the teaching point position, and recording the second pixel position of the teaching point position; calculating the mapping angle and offset dimension between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate and the second pixel position; calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point position according to the mapping angle and the offset dimension; controlling the manipulator to move the camera according to the coordinate movement amount.

[0023] In modern industrial automation production, the integration of a manipulator (Selective Compliance Assembly Robot Arm, SCARA) and a vision system has become an important development direction. Installing a camera on the SCARA manipulator and using the camera center as the end of the manipulator to perform visual guidance tasks can achieve precise recognition, positioning and grasping of target objects, significantly improving production efficiency and operation accuracy. However, at present, SCARA manipulators face many challenges in practical applications. First, when the manipulator leaves the factory, the end of the manipulator body is defaulted as the reference point of the end effector. However, in the visual application scenario, the center of the camera needs to be used as the end effector. This difference between the default setting and the actual requirement makes it necessary to redefine and calibrate the end of the manipulator before performing the visual guidance operation. Otherwise, it will cause a deviation between the manipulator's motion trajectory and the positioning of the visual system, and the target task cannot be accurately completed. Second, although the SCARA manipulator has a certain repeat positioning accuracy, the absolute accuracy of the manipulator body when it leaves the factory often fails to meet the requirements of high-precision visual guidance operations. During the teaching process, if the factory body accuracy is directly used, when the manipulator executes tasks according to visual information, due to the accumulation of absolute positioning errors, the center of the camera cannot accurately align with the target point, affecting the operation quality.

[0024] This application provides a solution. Control the manipulator to move, measure the coordinates of the manipulator before and after the movement and the pixel coordinates of the teaching potential captured by the camera, calculate the mapping angle and offset size between the camera and the manipulator according to the measured coordinates, calculate the coordinate movement amount of the manipulator when the center point of the camera aligns with the teaching point according to the mapping angle and offset size, and complete the automatic alignment of the camera center with the teaching point by moving the manipulator.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a device that can realize the above functions and complete the center teaching point by the automatic movement of the camera. Hereinafter, the controller is taken as an example to illustrate this embodiment and the following embodiments.

[0026] Based on this, the embodiment of this application provides a method for a camera to automatically move to complete the center teaching point. Refer to Figure 1 , Figure 1 It is the flowchart of the first embodiment of the method for a camera to automatically move to complete the center teaching point of this application.

[0027] In this embodiment, the method for a camera to automatically move to complete the center teaching point includes steps S10 to S50: Step S10: Control the camera to capture the teaching point, and record the first coordinate where the camera is located and the first pixel position of the teaching point.

[0028] It should be noted that as Figure 2As shown, the teaching points include point A and point B. Swing points A and B to calibrate the paper. First, make sure the camera can capture points A and B. The camera, with the cooperation of the light source, captures an image of teaching point A and transmits the image signal to the image acquisition card. After converting the image signal into a digital signal, the image acquisition card transmits it to the computer. The vision software preprocesses the captured image, uses image processing algorithms to extract the feature information of the target object. Based on the extracted feature information, the vision software identifies teaching point A through algorithms such as template matching and feature matching, and determines the first coordinate CA1 of teaching point A in the image coordinate system.

[0029] It can be understood that the manipulator coordinate system is a fixed reference coordinate system in which the manipulator is installed in the working space, used to describe the position and posture of the manipulator end in the working space, and is the benchmark for all movements and position descriptions of the manipulator. In this application, the manipulator moves on the same two-dimensional plane, and records the first coordinate PA1 of the camera in the manipulator coordinate system.

[0030] Step S20: Move the camera to the second coordinate through the manipulator, control the camera to capture the teaching point, and record the second pixel position of the teaching point.

[0031] It should be noted that the manipulator drives the camera to move to a predetermined shooting position. When the manipulator reaches the second coordinate, it sends a trigger signal to the camera to make it capture teaching point A in the current field of view. Since the camera has moved, the center point C0 of the image coordinate system has also moved. After moving, the position of the camera in the manipulator coordinate is the second coordinate PA2, and the position of teaching point A in the image coordinate system is the second pixel position CA2.

[0032] Step S30: Calculate the mapping angle and offset size between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position.

[0033] It can be understood that determining the mapping relationship between the camera and the manipulator coordinate system, that is, calculating the mapping angle and offset size between the two, to achieve the precise matching of the camera vision information and the manipulator action, so that the manipulator can accurately execute corresponding operations according to the pixel position information obtained by the camera. The mapping angle angle and offset size pixellen are the conversion quantities between the camera coordinate system and the manipulator coordinate system.

[0034] Step S40: Calculate the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size.

[0035] It is understandable that, based on the known mapping angle and offset dimension between the camera and the manipulator, when the center point of the camera needs to be accurately aligned with the target teaching point, the coordinate movement amount that the manipulator needs to execute is accurately calculated. Provide precise motion instructions for the manipulator to ensure that the center point of the camera at its end effector can accurately align with the teaching point A, and achieve high-precision execution of automated operations.

[0036] It should be noted that according to the calculated mapping angle angle and the offset dimension pixellen, calculate the coordinate movement amount required to directly align the center of the manipulator camera with point A. It is known that the current position of point A in the image coordinate system is the second pixel position CA2, and the center point of the camera in the image coordinate system is the pixel fixed position CC. The center point of the camera is the center of the image, which can be the position of half of the length and width of the image.

[0037] Optionally, a dynamic error compensation mechanism is introduced during the calculation process. By real-time monitoring the influence of factors such as minute vibrations and load changes during the movement of the manipulator on the coordinate accuracy, using machine learning algorithms (such as neural networks) to learn and model historical data, predicting the error compensation value under the current operation, and incorporating it into the calculation of the coordinate movement amount, significantly improving the alignment accuracy.

[0038] Step S50: Control the manipulator to move the camera according to the coordinate movement amount.

[0039] It is understandable that based on the coordinate movement amount of the manipulator calculated in the previous step, precisely control the manipulator to perform corresponding movements, thereby driving the camera to move, so that the center point of the camera accurately aligns with the teaching point. Achieve precise teaching and calibration of the camera center, thereby improving the accuracy and stability of the entire vision guidance system, and enhancing the working efficiency and quality of the manipulator.

[0040] Specifically, according to the calculated coordinate movement amount, control the manipulator to move movex millimeters in the x direction and movey millimeters in the y direction.

[0041] In this embodiment, key data is obtained by taking pictures of the teaching points at different positions with a camera. The mapping parameters between the two are calculated, and then the movement amount of the manipulator is determined to complete precise movement control. The camera is controlled to take pictures of the teaching points, and the first coordinate of the camera at that time (the position in the manipulator coordinate system) and the first pixel position of the teaching point in the image are recorded. The manipulator is used to move the camera to the second coordinate, and the camera is controlled to take pictures of the teaching points again, and the second pixel position of the teaching point at this time is recorded. Based on the first pixel position, the first coordinate, the second coordinate, and the second pixel position, the mapping angle and offset size between the camera and the manipulator are calculated by mathematical methods such as coordinate transformation and geometric relationship analysis. According to the calculated mapping angle and offset size, when the center of the camera needs to be aligned with the teaching point, the coordinate amount that the manipulator needs to move, including the translation amount and the rotation angle adjustment amount of each joint, is calculated through algorithms such as coordinate system transformation and inverse kinematics. The manipulator is controlled to move according to the calculated coordinate movement amount, driving the camera to move so that the center of the camera is accurately aligned with the teaching point.

[0042] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as in the above-mentioned embodiment 1 can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , Figure 3 which is a schematic flowchart provided by the second embodiment of the method for the camera to automatically move to complete the central teaching point of the present application.

[0043] After step S20, the method for the camera to automatically move to complete the central teaching point includes steps S201 to S204.

[0044] Step S201: Control the joint angle of the robotic arm to adjust by a first preset angle, and the link rotation angle of the robotic arm is adjusted in the reverse direction by the first preset angle, and record the position of the camera as the second coordinate.

[0045] It should be noted that the joint angle of the robotic arm refers to the angle between adjacent links, and the link rotation angle refers to the rotation angle of the link around its own axis. Under normal circumstances, the changes in the joint angle and the link rotation angle will jointly affect the position of the end effector (carrying the camera) of the robotic arm. As Figure 4 shown, control the movement of the manipulator, control the manipulator to change from the left figure to the right figure, the first preset angle can be 15 degrees, the link rotation angle movement J1 = J1 - 15, and the joint angle movement J2 = J2 + 15, so that the joint angle and the link rotation angle are adjusted by the same angle in the opposite direction, thereby realizing a specific change in the position of the camera.

[0046] It can be understood that step S201 further includes: when the manipulator moves, ensuring that the taught point is within the shooting range of the camera. After the robotic arm moves, the camera can still capture point A or point B. If not, the angle of the first preset angle can be reduced, for example, changing 15 degrees to 1 degree.

[0047] Step S202: Calculate the coordinate movement vector of the camera according to the first coordinate and the second coordinate.

[0048] It can be understood that according to PA12 = PA1 - PA2, the calculated coordinate movement vector PA12 is the vector by which the manipulator moves in the manipulator coordinate system U0.

[0049] Step S203: Calculate the pixel movement vector of the taught point according to the first pixel position and the second pixel position.

[0050] It can be understood that according to CA12 = CA1 - CA2, the pixel movement vector CA12 is the movement vector of the taught point A in the image coordinate system C0.

[0051] Step S204: Calculate the mapping angle and the offset dimension according to the coordinate movement vector and the pixel movement vector.

[0052] It can be understood that according to these two vectors, the mapping angle between the C0 coordinate system and the U0 coordinate system can be obtained as angle = acos(dot(norm(PA12), norm(CA12))), where the vector normalization (norm) norm(PA12) and norm(CA12) respectively represent the operation of normalizing the vectors PA12 and CA12. Normalization is to normalize the length (modulus) of the vector to 1 while keeping its direction unchanged. The dot product (dot) dot(norm(PA12), norm(CA12)) represents the dot product operation on the two normalized vectors. The calculation method of the dot product is: dot(norm(PA12), norm(CA12)) = norm(PA12) * norm(CA12) * cos(angle) = norm(PA12).x * norm(CA12).x + norm(PA12).y * norm(CA12).y where angle is the included angle between the vectors PA12 and CA12, and also the mapping angle angle. Since norm(PA12) and norm(CA12) are both unit lengths of 1, dot(norm(PA12), norm(CA12)) = cos(angle).

[0053] angle = acos(norm(PA12), norm(CA12)) = acos(cos(angle)) angle = acos(norm(PA12).x * norm(CA12).x + norm(PA12).y * norm(CA12).y) acos is the inverse cosine function, which calculates the mapping angle angle.

[0054] It should be noted that the offset size pixellen corresponding to the pixel is calculated according to CA12 and PA12 at the same time, where norm(PA12) is the modulus of the coordinate movement vector PA12, that is, the actual distance of the camera movement in the physical space, with the unit of millimeter. norm(CA12) is the modulus of the pixel movement vector CA12, that is, the pixel distance of the taught point position movement in the image coordinate system. The offset size is used to calculate the proportional relationship between the pixel and the physical unit (such as millimeter), that is, the physical distance corresponding to each pixel.

[0055] In this embodiment, the position change of the camera in the manipulator coordinate system is determined by using the first coordinate and the second coordinate; the position change of the taught point A in the camera image coordinate system is analyzed in combination with the first pixel position and the second pixel position. The mapping angle angle and the offset size pixellen are determined according to the change amount of the manipulator coordinates and the change amount of the target position in the camera image.

[0056] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment and the second embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 5 , Figure 5 which is the flow schematic diagram provided by the third embodiment of the method for the camera to automatically move to complete the center taught point position of the present application.

[0057] After step S40, the method for the camera to automatically move to complete the center taught point position further includes steps S301 to S302.

[0058] Step S301: Calculate the second pixel movement vector according to the pixel position of the center point and the second pixel position.

[0059] It can be understood that the current pixel of point A is the second pixel position CA2, and the pixel position of the center point is the pixel fixed position CC. From CM = CC - CA2, CM is the pixel distance of point A from the center point in the camera coordinate system C0.

[0060] Step S302: Calculate the coordinate movement amount that the manipulator needs to move according to the second pixel movement vector, the mapping angle, and the offset dimension.

[0061] It can be understood that the second pixel movement vector corresponding to the horizontal pixel of the manipulator is calculated as follows: CM(0) = CM(0) * pixellen CM(1) = CM(1) * pixellen movex = CM(0) * cos(angle) + CM(1) * (-sin(angle)) movey = CM(0) * sin(angle) + CM(1) * cos(angle) movex is the movement amount of the manipulator in the x-axis direction, and movey is the movement amount of the manipulator in the y-axis direction.

[0062] Step S303: After the movement of the camera is completed, obtain the third pixel coordinates of the teaching point by controlling the camera to capture the teaching point.

[0063] It can be understood that after the movement, the pixel coordinates of point A are re-obtained as the third pixel coordinates CA3, and CM2 = CC - CA3. CM2 is the pixel distance of pixel point A from the center point in the camera coordinate system C0.

[0064] Step S304: When the difference between the third pixel coordinates and the center point is less than or equal to the preset difference, it is determined that the automatic teaching of the camera center point aligning with the teaching point is completed.

[0065] It can be understood that the preset difference is 2 pixels. If norm(CM2) ≤ 2 pixels, it means that the automatic teaching of the camera center aligning with point A is completed.

[0066] Step S305: When the difference between the third pixel coordinates and the center point is greater than the preset difference, execute the step of calculating the coordinate movement amount of the manipulator when the center point of the camera aligns with the teaching point according to the mapping angle and the offset dimension.

[0067] It can be understood that if norm(CM2) > 2 pixels, it indicates that the center point of the camera is not aligned with the target teaching point A, and further adjustment is required. According to the previously recorded mapping angle and offset size, calculate the coordinate movement amount of the robot arm required to align the center point of the camera with the teaching point. That is, repeat steps S40 to S50 until norm(CM2) <= 2 pixels is satisfied. Adjust the position of the robot arm according to the calculated coordinate movement amount to align the center point of the camera with the teaching point. This process may require multiple iterations until the difference between the third pixel coordinate and the center point is within the allowable range.

[0068] Step S306: Re-identify the teaching point, and execute the step of calculating the coordinate movement amount of the robot arm when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size.

[0069] It can be understood that after the automatic teaching of aligning the camera center with point A is completed, repeat steps S40 to S50 to complete the automatic teaching of point B. Among them, the teaching points can include 2 to 9. The system traverses all re-identified teaching points and executes the above calculation process for each point in turn.

[0070] In this embodiment, after the camera moves to the specified position with the robot arm, an image containing the teaching point is taken. From the taken image, the third pixel coordinate of the teaching point is obtained through image processing technology. The third pixel coordinate is compared with the pixel coordinate of the camera center point, and the difference between the two is calculated. If the difference is greater than the preset difference: According to the pre-calibrated mapping angle and offset size, calculate the coordinate amount that the robot arm needs to move to align the center point of the camera with the teaching point. If the difference is less than or equal to the preset difference: It is judged that the camera center point has been aligned with the teaching point, and the automatic teaching is completed. Again, according to the mapping angle and offset size, calculate the coordinate movement amount of the robot arm to prepare for aligning with the new teaching point. By comparing the pixel coordinate differences, the system can automatically judge whether the camera is aligned with the teaching point and make corresponding adjustments, and at the same time support the automatic alignment and adjustment of multiple teaching points. The preset difference is used as the threshold for judging the alignment accuracy to ensure that the alignment of the camera center point and the teaching point is within the acceptable error range.

[0071] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation on the method for the camera to automatically move to complete the center teaching point of the present application. Based on this technical concept, more forms of simple transformations are within the protection scope of the present application.

[0072] The present application also provides a device for a camera to automatically move to complete the center teaching point. Please refer to Figure 6 The device for a camera to automatically move to complete the center teaching point includes: The first position acquisition module 10 is configured to control the camera to capture a teaching point, and record the first coordinate where the camera is located and the first pixel position of the teaching point; The second position acquisition module 20 is configured to move the camera to a second coordinate by a manipulator, control the camera to capture a teaching point, and record the second pixel position of the teaching point; The mapping calculation module 30 is configured to calculate a mapping angle and an offset dimension between the camera coordinate and the manipulator coordinate according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position; The third position calculation module 40 is configured to calculate a coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset dimension; The teaching completion module 50 is configured to control the manipulator to move the camera according to the coordinate movement amount.

[0073] The device for automatically moving a camera to complete center teaching points provided by the present application adopts the method for automatically moving a camera to complete center teaching points in the above embodiment, and can solve the technical problem of manually controlling a manipulator to complete the alignment of the camera center with the teaching point. Compared with the prior art, the beneficial effects of the device for automatically moving a camera to complete center teaching points provided by the present application are the same as those of the method for automatically moving a camera to complete center teaching points provided by the above embodiment, and other technical features in the device for automatically moving a camera to complete center teaching points are the same as the features disclosed in the method of the above embodiment, and will not be elaborated herein.

[0074] The present application provides a device for automatically moving a camera to complete center teaching points. The device for automatically moving a camera to complete center teaching points includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for automatically moving a camera to complete center teaching points in the first embodiment above.

[0075] Next, refer to Figure 7, which shows a schematic structural diagram of a device suitable for implementing the camera automatic movement to complete the center teaching point position in the embodiments of the present application. The device for the camera automatic movement to complete the center teaching point position in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Player: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The shown device for the camera automatic movement to complete the center teaching point position is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0076] As Figure 7 shown, the device for the camera automatic movement to complete the center teaching point position may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the xxx device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the device for the camera automatic movement to complete the center teaching point position to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a device for the camera automatic movement to complete the center teaching point position with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0077] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0078] The device for automatically moving the camera to complete the center teaching point provided by the present application adopts the method for automatically moving the camera to complete the center teaching point in the above embodiments, and can solve the technical problem of manually controlling a manipulator to complete the center alignment of the camera to the teaching point. Compared with the prior art, the beneficial effects of the device for automatically moving the camera to complete the center teaching point provided by the present application are the same as those of the method for automatically moving the camera to complete the center teaching point provided by the above embodiments, and other technical features in the device for automatically moving the camera to complete the center teaching point are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0079] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0080] As described above, only the specific embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0081] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for automatically moving the camera to complete the center teaching point in the above embodiments.

[0082] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0083] The above computer-readable storage medium can be included in the device for automatically moving the camera to complete the teaching point positions of the center; or it can exist independently and not be assembled into the device for automatically moving the camera to complete the teaching point positions of the center.

[0084] The computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in a different order than that marked in the accompanying drawings. For example, two consecutive boxes shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as combinations of boxes in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0086] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0087] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the method for automatically moving the camera to complete the center teaching point position, which can solve the technical problems of too long beat time, too low torque control accuracy, and excessive overshoot when increasing the gain of the torque closed-loop controller during the electric batch locking. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the method for automatically moving the camera to complete the center teaching point position provided by the above embodiments, and will not be elaborated here.

[0088] The above are only some embodiments of the present application, and thus do not limit the patent scope of the present application. All equivalent structural transformations made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for automatically moving a camera to complete a center teaching point, characterized in that: The method comprises: Controlling a camera to shoot a teaching point, and recording a first coordinate of the camera and a first pixel position of the teaching point; The camera is moved to a second coordinate by a manipulator, the camera is controlled to shoot a teaching point, and a second pixel position of the teaching point is recorded; Calculate a mapping angle and an offset size between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate, and the second pixel position; Calculating, according to the mapping angle and the offset size, the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point; The robot is controlled to move the camera according to the coordinate movement amount.

2. The method for completing the center teaching point by automatic camera movement as claimed in claim 1, characterized in that: The step of moving the camera to a second coordinate by a manipulator comprises: The joint angle of the robotic arm is controlled to adjust to a first preset angle, the connecting rod angle of the robotic arm is reversely adjusted to the first preset angle, and the position of the camera is recorded as the second coordinate; wherein, when the robotic arm moves, it is ensured that the teaching point is within the shooting range of the camera.

3. The method for completing the center teaching point by automatic camera movement as claimed in claim 1, characterized in that: The step of calculating the mapping angle and offset size between the camera and the manipulator according to the first pixel position, the first coordinate, the second coordinate and the second pixel position comprises: Calculate the coordinate movement phase of the camera according to the first coordinate and the second coordinate; Calculating the pixel shift phase of the teaching point according to the first pixel position and the second pixel position; The mapping angle and the offset size are calculated according to the coordinate movement vector and the pixel movement vector.

4. The method for completing the center teaching point by automatic camera movement as claimed in claim 1, characterized in that: The step of calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size also includes: Calculating a second pixel movement vector according to the pixel position of the center point and the second pixel position; The coordinate movement amount that the robot needs to move is calculated according to the second pixel movement vector, the mapping angle and the offset size.

5. The method for completing the center teaching point by automatic camera movement as claimed in claim 1, characterized in that: After the step of controlling the manipulator to move the camera according to the coordinate movement amount, the method further comprises: After the camera moves, the third pixel coordinates of the teaching point are obtained by controlling the camera to shoot the teaching point; When the difference between the third pixel coordinate and the center point is greater than a preset difference, the step of calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size is performed.

6. The method for completing the center teaching point by automatic camera movement as claimed in claim 5, characterized in that: After the camera moves, the step of controlling the camera to shoot the teaching point and obtaining the third pixel coordinate of the teaching point comprises: When the difference between the third pixel coordinate and the center point is less than or equal to the preset difference, it is determined that the automatic teaching of the camera center point to the teaching point is completed.

7. The method for completing the center teaching point by automatic camera movement as claimed in claim 6, characterized in that: After the step of determining that the automatic teaching of the camera center point aligning with the teaching point is completed, the method further comprises: The teaching point is re-identified, and a step is performed to calculate the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size.

8. A camera automatic movement to complete the center teaching point device, characterized in that: The device comprises: A first position acquisition module, used for controlling a camera to shoot a teaching point, and recording a first coordinate of the camera and a first pixel position of the teaching point; A second position acquisition module, used to move the camera to a second coordinate by a manipulator, control the camera to shoot a teaching point, and record a second pixel position of the teaching point; A mapping calculation module, used for calculating the mapping angle and offset size of the camera coordinates and the manipulator coordinates according to the first pixel position, the first coordinate, the second coordinate and the second pixel position; A third position calculation module, used for calculating the coordinate movement amount of the manipulator when the center point of the camera is aligned with the teaching point according to the mapping angle and the offset size; The teaching completion module is used to control the manipulator to move the camera according to the coordinate movement amount.

9. A camera automatic motion completion center teaching point device, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and running on the processor, wherein the computer program is configured to implement the steps of the method for completing the center teaching point position by automatic movement of a camera as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for automatically moving a camera to complete the center teaching point as described in any one of claims 1 to 7 are implemented.

Images