Position attitude acquisition device and probe
The position and orientation acquisition device addresses camera shake issues by using image acquisition and correction processes to enhance the accuracy of probe positioning and orientation data for robots.
Patent Information
- Application Number
- JP2023077118
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-21
AI Technical Summary
Camera shake during manual movement of a probe by an operator leads to unstable teaching data quality and potential errors in position and orientation acquisition.
A position and orientation acquisition device that includes a photographed image acquisition unit, a position and orientation acquisition section, and correction sections to reduce the influence of camera shake by smoothing and averaging positions and orientations using moving averages.
Reduces the impact of camera shake, enabling accurate and stable acquisition of probe positions and orientations, resulting in improved teaching data for robots.
Smart Images

Figure 2024162001000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a position and orientation acquisition device for acquiring the position and orientation of a probe. [Background technology]
[0002] Conventionally, welding robots have been taught by moving a welding torch attached to the tip of the welding robot. However, welding robots can also be taught by acquiring the position and posture of a probe or the like moved by an operator, instead of using a welding robot (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-194724 Summary of the Invention [Problem to be solved by the invention]
[0004] When an operator holds a probe by hand and moves it, it is convenient that the probe can be easily moved to a desired position in a desired orientation, but the operator may experience camera shake during the movement, which may cause errors in the taught position. In addition, the camera shake may result in unstable quality of the taught data.
[0005] The present invention has been made to solve the above-mentioned problems, and has an object to provide a position and orientation acquisition device, etc., that can reduce the effects of hand shake by an operator moving a probe when acquiring the position of the probe. [Means for solving the problem]
[0006] In order to achieve the above-mentioned object, a position and orientation acquisition device according to one aspect of the present invention includes an image acquisition unit that acquires an image of a probe moved by an operator, a position and orientation acquisition unit that acquires the position and orientation of the probe using the captured image, and a position correction unit that performs correction processing on the position of the probe acquired by the position and orientation acquisition unit to reduce the effects of hand shaking by the operator. Effect of the Invention
[0007] According to the position and orientation acquisition device and the like according to one aspect of the present invention, it is possible to reduce the influence of hand shaking of an operator moving a probe when acquiring the position of the probe. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of an information processing system according to an embodiment of the present invention; [Diagram 2] A flowchart showing the operation of the position and orientation acquisition device according to the embodiment. [Diagram 3] FIG. 13 is a perspective view showing another example of the probe according to the embodiment; [Figure 4] FIG. 2 is a functional block diagram showing an example of a configuration of a probe according to the embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] The position and orientation acquisition device and probe according to the present invention will be described below using embodiments. In the following embodiments, components and steps with the same reference numerals are the same or equivalent, and repeated description may be omitted. The position and orientation acquisition device according to this embodiment performs a correction process for reducing the influence of hand shaking of an operator moving the probe on the position and orientation acquired using a captured image of the probe.
[0010] 1 is a schematic diagram showing the configuration of an information processing system 100 according to this embodiment. The information processing system 100 according to this embodiment includes a robot 1, a camera 2 for photographing a probe 7, a position and orientation acquisition device 3 for acquiring the position and orientation of the probe 7 using an image photographed by the camera 2, and the probe 7. The camera 2 may be attached to the robot 1, as an example.
[0011] As an example, the robot 1 may have a plurality of arms connected by joints driven by a motor. The robot 1 may be, for example, a vertical articulated robot. The number of axes of the robot 1 is not important. In this embodiment, the case where the robot 1 is a welding robot and the welding robot is taught using a probe 7 will be mainly described.
[0012] The camera 2 may be attached to, for example, the tip of the robot 1, or may be attached to a location other than the robot 1. The camera 2 may not be attached to, for example, the robot 1. In this case, the information processing system 100 may not include the robot 1. The camera 2 not attached to the robot 1 may be placed in a location where it is easy to photograph the probe 7, for example, on the ceiling. When the camera 2 is placed other than the robot 1, it is preferable that the relative positional relationship between the camera 2 and the robot 1 is known. This is to enable the position and orientation of the probe 7 based on the robot 1 to be acquired. In this embodiment, the case where the camera 2 is attached near the tip of the robot 1 will be mainly described. Note that, when the probe 7 is photographed by the camera 2, the position and optical axis direction of the camera 2 may or may not be changed by the robot 1 in accordance with the movement of the probe 7. In the former case, for example, when photographing by the camera 2, the position and orientation acquisition device 3 may further include a control unit that controls the robot 1 so that photographing is performed while following the probe 7. If the position and optical axis direction of the camera 2 do not change in accordance with the movement of the probe 7, it is preferable that the position, optical axis direction, angle of view, etc. of the camera 2 are set so that the entire movement of the probe 7 from the start to the end can be captured by the camera 2.
[0013] The camera 2 is for photographing the probe 7, and may be, for example, a monocular camera or a stereo camera. In this embodiment, the case where the camera 2 is a monocular camera will be mainly described. The photographed image acquired by the camera 2 may be, for example, a grayscale image or a color image. For example, when the marker 5 attached to the probe 7 has a plurality of colors, it is preferable that the photographed image is a color image. The camera 2 may, for example, photograph a still image or a video. In the latter case, one frame included in the video may be used as the photographed image by the position and orientation acquisition device 3.
[0014] The probe 7 is moved by an operator. The probe 7 may be moved together with at least a part of the operator's body (e.g., a hand, etc.). The operator usually moves the probe 7 by holding it in his / her hand. The probe 7 is preferably large and heavy enough for the operator to hold it in his / her hand. The probe 7 may have a probe body 70 and a marker 5 attached to at least a part of the probe body 70. The probe body 70 may be, for example, a model of a welding torch of a welding robot. The probe body 70 may have, for example, a plate-like member 7a to which a planar marker 5 is attached, and a rod-like member 7b connected to the plate-like member 7a. The rod-like member 7b may be attached, for example, to the center of gravity of the surface of the plate-like member 7a opposite to the surface to which the marker 5 is attached, so as to be perpendicular to the surface.
[0015] The marker 5 may have, for example, three or more feature points. For example, in the marker 5 shown in FIG. 1, the feature points may be the vertices of a rectangular figure. The planar marker 5 may be, for example, rectangular as shown in FIG. 1, polygonal, or other shapes. As an example, the marker 5 may be one whose orientation can be uniquely specified. Also, for example, the distance between the feature points may be known.
[0016] The position and orientation acquisition device 3 acquires the position and orientation of the probe 7 using a captured image of the probe 7, and includes a captured image acquisition unit 31, a reception unit 32, a position and orientation acquisition unit 33, a storage unit 34, a position correction unit 35, an orientation correction unit 36, and an output unit 37. The position and orientation acquisition device 3 is preferably connected to the camera 2 by wire or wirelessly so as to be able to acquire an image captured by the camera 2. Furthermore, in the case where the robot 1 is also controlled, the position and orientation acquisition device 3 is preferably connected to the robot 1 by wire or wirelessly as well.
[0017] The photographed image acquiring unit 31 acquires a photographed image of the probe 7 moved by the operator. The photographed image may be, for example, one photographed by the camera 2. It is preferable that the photographed image acquiring unit 31 repeatedly acquires the photographed image. For example, the photographed image acquiring unit 31 may acquire the photographed image periodically. For example, the photographed image acquiring unit 31 may acquire the photographed image by transmitting an instruction to acquire the photographed image to the camera 2 and receiving the photographed image transmitted from the camera 2 in response to the instruction. In addition, the photographed image acquiring unit 31 may acquire the photographed image by accepting the photographed image from the camera 2 periodically or irregularly. The acquired photographed image may be stored in a recording medium such as the storage unit 34. It is preferable that the photographed image includes the marker 5 attached to the probe 7.
[0018] The reception unit 32 receives instructions and the like. The reception unit 32 may receive, for example, at least one of an instruction to start acquiring the position and orientation of the probe 7 and an instruction to end the acquisition. The reception unit 32 may also receive, for example, parameters used in the correction process. The reception unit 32 may receive, for example, information input from an input device (for example, a keyboard, a mouse, a touch panel, etc.), or may receive information transmitted via a wired or wireless communication line. The reception unit 32 may or may not include a device for performing the reception, such as an input device or a communication device. The reception unit 32 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device. When the position and orientation acquisition device 3 does not receive instructions and the like, the position and orientation acquisition device 3 may not include the reception unit 32.
[0019] The position and orientation acquisition unit 33 acquires the position and orientation of the probe 7 using the captured image acquired by the captured image acquisition unit 31. This position and orientation are usually the position and orientation in real space. The position and orientation in real space may be, for example, the position and orientation in the coordinate system of the robot 1. In this embodiment, this case will be mainly described. The position of the probe 7 may be, for example, the position of a predetermined location of the probe 7. The position of this predetermined location may be, for example, the position of the tip 7c of the rod-shaped member 7b of the probe 7. In addition, the orientation of the probe 7 may be, for example, the orientation of a predetermined location of the probe 7. In addition, in addition, in addition, the orientation of the predetermined location may be, for example, the orientation of the rod-shaped member 7b of the probe 7. In addition, in ...
[0020] As an example, the position and posture of the welding torch of the robot 1 may be indicated by the position and posture acquired by the position and posture acquisition unit 33. The position of the welding torch may be, for example, the position of the tip of the welding torch. This position may be indicated, for example, by coordinate values in a certain coordinate system. The posture of the welding torch may be, for example, the angle of the tip of the welding torch. The angle of the tip of the welding torch may be indicated, for example, by an angle in a certain coordinate system (e.g., an azimuth angle and an elevation / depression angle, etc.), or may be indicated by an aim angle or an advance / retract angle, which is an angle with respect to the welding path. The aim angle may be an angle between a reference plane including the weld line and a plane including the weld line and the longitudinal axis of the welding torch (for example, it may be the longitudinal axis of the tip of the welding torch or the welding wire). The advance / retract angle may be an angle between a normal to the weld line and the longitudinal axis of the welding torch in a plane including the weld line and the longitudinal axis of the welding torch. The angle may also include, for example, a rotation angle about the longitudinal axis of the welding torch as the central axis. When the posture is indicated by a target angle or an advance / retract angle, for example, the positions of the welding path and the reference plane in the coordinate system in which the posture of the probe 7 is acquired are set in advance, and the position / posture acquisition unit 33 may use them to specify the target angle and the advance / retract angle from the angle of the probe 7.
[0021] The position and orientation acquisition unit 33 may acquire the position and orientation of the probe 7, for example, by using three or more feature points of the marker 5 included in the captured image. More specifically, the position and orientation acquisition unit 33 may acquire the relative positional relationship between the coordinate system of the camera 2 and the coordinate system of the marker 5 (for example, a homogeneous transformation matrix indicating the transformation between both coordinate systems) by identifying the marker 5 in the captured image and using three or more feature points of the identified marker 5. Note that since the acquisition of information indicating the relative positional relationship, such as a homogeneous transformation matrix, using the feature points is publicly known, a detailed description thereof will be omitted. Since the relative positional relationship between the coordinate system of the camera 2 and the coordinate system of the robot 1 is known, and the relative positional relationship between the coordinate system of the marker 5 and the coordinate system of the probe 7 is also known, the position and orientation acquisition unit 33 can acquire, for example, the position and orientation of a predetermined location of the probe 7 in the coordinate system of the robot 1. In addition, when the camera 2 is a stereo camera, for example, a relative positional relationship between the coordinate system of the camera 2 and the coordinate system of the probe 7 may be obtained using a distance to a feature point of the probe 7 acquired using a captured image, and the position and orientation of the probe 7 may be obtained using the relative positional relationship. The position and orientation acquisition unit 33 may store the acquired position and orientation in the storage unit 34, for example.
[0022] The position and orientation acquisition unit 33 may acquire the position and orientation from each of the multiple captured images in a time series acquired by the captured image acquisition unit 31. That is, the acquisition of the position and orientation of the probe 7 may be repeatedly performed, and time series data on the position of the probe 7 and time series data on the orientation of the probe 7 may be acquired. The time series data on the position and the orientation may include multiple positions and multiple orientations of the probe 7 in a time series. It is preferable that the time series data on the position and the time series data on the orientation are synchronized. The synchronization of both time series data may mean that the correspondence between each position included in the time series data on the position and each orientation included in the time series data on the orientation can be understood. The position and orientation acquisition unit 33 may acquire the position and orientation of the captured images acquired during the period from when an instruction to start acquiring the position and orientation of the probe 7 is accepted by the acceptance unit 32 to when an instruction to end the acquisition is accepted by the acceptance unit 32. In addition, when time information is used in a later correction process, the position and orientation acquisition unit 33 may, for example, store the position and orientation in the storage unit 34 in association with the corresponding time. The time corresponding to the position or orientation may be, for example, the shooting time of the captured image used to acquire the position or orientation. The time may be, for example, the time on a clock, or the value of a timer based on a certain point in time.
[0023] The storage unit 34 stores, for example, a plurality of positions and a plurality of orientations acquired by the position and orientation acquisition unit 33. In addition, for example, a plurality of positions after correction by the position correction unit 35 and a plurality of orientations after correction by the orientation correction unit 36 may also be stored in the storage unit 34. The storage unit 34 is preferably realized by a non-volatile recording medium, but may also be realized by a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk.
[0024] The position correction unit 35 performs a correction process for reducing the influence of the operator's shaking on the position of the probe 7 acquired by the position and orientation acquisition unit 33. This correction process is, for example, a process for removing outliers or reducing errors due to shaking, and may be performed using multiple positions. This correction process may be, for example, smoothing by moving average for multiple positions acquired according to the movement of the probe 7. The position correction unit 35 may perform smoothing by repeating a process of calculating an average value for a predetermined number of consecutive position data in the time series data related to the position, or for position data in a predetermined consecutive time range, while shifting the position data to be averaged one by one. By performing this smoothing by moving average, the influence of data shifted from the original position due to the influence of shaking can be reduced, and as a result, errors can be reduced. This moving average may be, for example, a simple moving average or a weighted moving average. In the weighted moving average, as an example, the weight may be set so that the data in the center position of the data in the consecutive positions to be averaged has the largest weight, and the weight is smaller as the data moves away from the center. The number of data items and the time range to be averaged in the moving average may be set as parameters, for example. In this case, the number of data items and the time range, which are parameters input by an operator, may be used in the moving average.
[0025] Furthermore, when the multiple positions acquired by the position and orientation acquisition unit 33 are positions along a route whose shape is known in advance, the position correction unit 35 may, for example, specify a route that fits the multiple acquired positions, and set the position on the specified route as the corrected position. Note that the route whose shape is known in advance may be, for example, a straight line route or a curved route whose curvature is constant. In this case, the position correction unit 35 may, for example, use the least squares method or the like to specify a route that fits the multiple acquired positions so that the error between the multiple acquired positions and the route whose shape is known in advance is reduced.
[0026] The attitude correction unit 36 performs a correction process for reducing the influence of the operator's shaking on the attitude of the probe 7 acquired by the position and attitude acquisition unit 33. This correction process may be performed using, for example, a plurality of attitudes. The attitude of the probe 7 is usually indicated by two or more angles. As described above, the attitude of the probe 7 may be indicated by, for example, an azimuth angle or an elevation and depression angle in a predetermined coordinate system, may be indicated by a target angle or a forward and backward angle, and may further include a rotation angle. The correction process for the attitude may be performed, for example, for at least a part of the angles among the plurality of angles. As an example, the correction process may be performed only on the forward and backward angles, and the correction process may not be performed on the target angle and the rotation angle. Even in this case, since the correction process has been performed on at least a part of the angles, it may be considered that the correction process has been performed on the attitude of the probe 7. For the angles for which the correction process is not performed, the angles acquired by the position and attitude acquisition unit 33 may be used as teaching data as they are. On the other hand, the correction process may be performed on all angles. As an example, the correction process may be performed on each of the forward and backward angles, the target angle, and the rotation angle. This correction process may be, for example, averaging of a plurality of postures acquired in accordance with the movement of the probe 7. That is, this correction process may be calculation of an average value of a plurality of angles along a time series of the probe 7 acquired by the position and posture acquisition unit 33. For example, when downward welding is performed, the angle of the longitudinal axis of the welding torch is usually not changed in many cases. Therefore, by calculating the average posture in this way, it is possible to suppress the fluctuation of the angle and reduce errors due to the influence of camera shake. The averaging of a plurality of postures may be, for example, averaging of a plurality of forward and backward angles along a time series. Also, for example, the average may be calculated for each weld line.
[0027] Similarly to the position of the probe 7, the attitude of the probe 7 may also be subjected to a correction process that is smoothing by a moving average of a plurality of attitudes acquired in accordance with the movement of the probe 7. Note that, for example, different correction processes may be performed for each of a plurality of angles that indicate the attitude of the probe 7. For example, the forward / reverse angle may be subjected to a correction process that is averaging, and the target angle and the rotation angle may be subjected to a correction process that is smoothing by a moving average.
[0028] In addition, when the correction process, which is smoothing by moving average, is performed on the posture, it is preferable that the correspondence relationship between the position after the correction process and the posture after the correction process is known. For example, the correction process on the position and posture may be performed so that the position after the averaging process, which is the correction process, is performed on a plurality of positions, and the posture after the averaging process, which is the correction process, is performed on a plurality of postures corresponding to the plurality of positions, are paired. On the other hand, when the correction process, which is the averaging process, is performed on a plurality of postures, the same posture may be associated with each position after the correction, for example. That is, the correction process on the position and posture may be performed so that each position after the correction is paired with the same posture. In addition, the position after the correction process and the posture after the correction process may be stored in the storage unit 34, for example. The position and posture after the correction process may be used as teaching data. The posture after the correction process may include, for example, an angle on which the correction process has been performed and an angle on which the correction process has not been performed. That is, when the correction process is not performed on at least a portion of the plurality of angles indicating the posture of the probe 7, the posture after the correction process may also include, for example, the angle on which the correction process has not been performed.
[0029] The output unit 37 may output, for example, a plurality of positions after the correction process and a plurality of postures after the correction process. The output by the output unit 37 may be, for example, transmission to a predetermined device via a communication line, storage in a recording medium, or delivery to another component. As an example, the position and posture after the correction process may be transmitted by the output unit 37 to a control device that controls the robot 1. In the case where the position and posture acquisition device 3 is included in the control device that controls the robot 1, the output unit 37 may deliver the position and posture after the correction process to the component that controls the robot 1. In addition, in the case where the position and posture acquisition device 3 includes a control unit that controls the robot 1, the output unit 37 may not output the position and posture. In this case, for example, the control unit may control the robot 1 using the position and posture after the correction process stored in the storage unit 34. In addition, the output unit 37 may display the position and posture after the correction process so that an operator or the like can confirm the position and posture after the correction process. The output unit 37 may or may not include a device that performs output (for example, a communication device, etc.). Moreover, the output unit 37 may be realized by hardware, or may be realized by software such as a driver that drives those devices.
[0030] Furthermore, the position and posture after the correction process are data indicating, for example, the position and posture of a welding torch of the robot 1. Therefore, the position and posture acquisition device 3 may include an inverse kinematics conversion unit that calculates the angle of each joint of the robot 1 by performing an inverse kinematics calculation on the robot 1 for that position and posture. In this case, for example, the angle of each joint of the robot 1 calculated by the inverse kinematics conversion unit may be output by the output unit 37.
[0031] Next, the operation of the position and orientation acquisition device 3 will be described with reference to the flowchart of FIG. (Step S101) The reception unit 32 judges whether or not it has received an instruction to start acquiring the position and orientation of the probe 7. If the instruction to start has been received, the process proceeds to step S102, and if not, the process of step S101 is repeated until the instruction to start is received. (Step S102) The photographed image acquisition unit 31 judges whether or not to acquire a photographed image. If the photographed image is to be acquired, the process proceeds to step S103, and if not, the process proceeds to step S105. Note that the photographed image acquisition unit 31 may periodically judge whether or not to acquire a photographed image, for example.
[0032] (Step S 103 ) The photographed image acquisition unit 31 acquires a photographed image from the camera 2 .
[0033] (Step S104) The position and orientation acquisition unit 33 identifies the marker 5 included in the acquired photographed image, and acquires the position and orientation of the probe 7 to which the marker 5 is attached, using the feature points of the identified marker 5. The acquired position and orientation may be stored in, for example, the storage unit 34. Then, the process returns to step S102.
[0034] (Step S105) The reception unit 32 determines whether or not it has received an instruction to end the acquisition of the position and orientation of the probe 7. If an instruction to end the acquisition has been received, the process proceeds to step S106, and if not, the process returns to step S102.
[0035] (Step S106) The position correction unit 35 performs a correction process on the acquired multiple positions. The positions after the correction process may be stored in the storage unit .
[0036] (Step S107) The attitude correction unit 36 performs a correction process on the acquired attitudes. The attitudes after the correction process may be stored in the storage unit 34. Then, the correction process for acquiring the position and attitude and reducing the influence of camera shake performed on them is completed.
[0037] In addition, in the flowchart of Fig. 2, a step of outputting the corrected positions and orientations may be executed after the correction process. Also, the order of the processes in the flowchart of Fig. 2 is an example, and the order of the steps may be changed as long as the same result can be obtained.
[0038] Next, the operation of the position and attitude acquisition device 3 according to this embodiment will be described using a specific example. In this specific example, a case where a welding path is taught using the probe 7 will be described. In this case, the operator of the probe 7 performing the teaching operation may hold the probe 7 by hand and move the probe 7 so that the tip 7c of the probe 7 is located at the position of the weld line and the attitude of the rod-shaped member 7b is the attitude of the longitudinal axis of the welding torch. The orientation and position of the camera 2 may be adjusted so that the marker 5 attached to the probe 7 can be photographed.
[0039] When starting to move the probe 7, the operator inputs an instruction to start acquiring the position and orientation to the position and orientation acquisition device 3. The start instruction is accepted by the acceptance unit 32 and passed to the position and orientation acquisition unit 33 (step S101). Upon receiving the instruction to end, the position and orientation acquisition unit 33 causes the captured image acquisition unit 31 to start acquiring captured images. As a result, the captured images captured by the camera 2 are periodically acquired by the captured image acquisition unit 31 and passed to the position and orientation acquisition unit 33 (steps S102, S103). When the captured image is received, the position and orientation acquisition unit 33 identifies the marker 5 included in the captured image, acquires a homogeneous transformation matrix indicating the transformation between the coordinate system of the camera 2 and the coordinate system of the marker 5 using the feature points of the marker 5, and acquires a set of the position of the tip 7c of the probe 7 in the coordinate system of the robot 1 and the orientation of the rod-shaped member 7b of the probe 7 by using the homogeneous transformation matrix, the homogeneous transformation matrix indicating the transformation between the coordinate system of the camera 2 and the coordinate system of the robot 1, and the homogeneous transformation matrix indicating the transformation between the coordinate system of the marker 5 and the coordinate system of the probe 7, and stores the set in the storage unit 34 (step S104). By repeating the process of acquiring the captured image and acquiring the position and orientation, time-series data of the position and orientation of the probe 7 is stored in the storage unit 34 (steps S102 to S104).
[0040] When the movement of the probe 7 is completed, the operator inputs an instruction to end the acquisition of the position and orientation to the position and orientation acquisition device 3. The instruction to end is received by the reception unit 32 and is passed to the position and orientation acquisition unit 33, the position correction unit 35, and the orientation correction unit 36 (step S105). Upon receiving the instruction to end, the position and orientation acquisition unit 33 ends the acquisition of the captured image by the captured image acquisition unit 31 and ends the process of acquiring the position and orientation using the captured image. Furthermore, upon receiving the instruction to end, the position correction unit 35 and the orientation correction unit 36 perform correction processing on the multiple positions and multiple orientations stored in the storage unit 34, respectively, and accumulate the positions and orientations after the correction processing in the storage unit 34, respectively (steps S106, S107). In this way, the positions and orientations with reduced influence of the operator's hand shake are stored in the storage unit 34. Note that the teaching data, which are the positions and orientations after the correction processing along the time series, may be used for, for example, welding processing by the robot 1.
[0041] As described above, the position and orientation acquisition device 3 according to this embodiment can acquire the position and orientation of the probe 7 moved by the operator, and can perform smooth teaching with reduced influence of camera shake by performing correction processing on the acquired position and orientation. As a result, it becomes possible to acquire teaching data with reduced error. In addition, by performing correction independently on each of the position and orientation of the probe 7, for example, it is possible to smooth the position by moving average, and to average at least some of the angles indicating the orientation, such as forward and backward angles, to a constant value. In this way, it becomes possible to perform correction processing according to the actual teaching of welding, etc.
[0042] In this embodiment, in order to further increase the accuracy of the position and orientation, the position and orientation corresponding to the probe 7 may be acquired only when the probe 7 is in appropriate contact with an object such as a workpiece. In this case, for example, the probe 7 may be as shown in Figs. 3 and 4. Fig. 3 is a perspective view of the probe 7, and Fig. 4 is a functional block diagram showing the functions of the probe 7. The probe 7 may include a probe body 70, a sensor 71 provided at an end of the probe body 70, an output unit 72, and a transmission unit 73. The end of the probe body 70 may be, for example, a tip 7c of a rod-shaped member 7b.
[0043] The sensor 71 is a sensor for detecting the presence or absence of contact with an object such as a workpiece. The sensor 71 may be, for example, a contact sensor, a proximity sensor, or any other sensor capable of detecting the presence or absence of contact with an object. As an example, the sensor 71 may be one that outputs a signal indicating that it is in contact with an object. The sensor 71, which is a proximity sensor, may be one that outputs a signal indicating that it is in contact with an object when the distance to the object is close enough to be considered as being in contact.
[0044] The output unit 72 outputs an alert when the sensor 71 does not detect contact. This alert may be output by light or sound, or may be a signal indicating the alert. In this embodiment, a case will be mainly described in which the output unit 72 outputs an alert by turning on a light-emitting element such as an LED provided on the rod-shaped member 7b when the sensor 71 does not detect contact. That is, when the light-emitting element is turned on, the tip 7c is not in contact with the object, and when the light-emitting element is turned off, the tip 7c is in contact with the object.
[0045] The transmitter 73 transmits a contact state signal indicating whether the end of the probe 7 is in contact with an object such as a workpiece to the position and orientation acquisition device 3 by wired or wireless communication in accordance with the detection result by the sensor 71. The contact state signal may be any signal as long as it ultimately makes it possible to know whether the end of the probe 7 is in contact with the object or not. For example, the transmitter 73 may transmit a contact state signal indicating that the tip 7c is in contact with the object when the tip 7c is in contact with the object, or may transmit a contact state signal indicating that the tip 7c is not in contact with the object when the tip 7c is not in contact with the object, or may transmit both.
[0046] In this case, the receiving unit 32 of the position and orientation acquisition device 3 may receive, for example, a contact state signal indicating whether or not the end of the probe 7 is in contact with the target object. The reception of this contact state signal may be, for example, a wired or wireless reception of the contact state signal.
[0047] The position correction unit 35 may perform a correction process on the position of the probe 7 when the contact state signal received by the reception unit 32 indicates that the end of the probe 7 is in contact with the object. The attitude correction unit 36 may perform a correction process on the attitude of the probe 7 when the contact state signal received by the reception unit 32 indicates that the end of the probe 7 is in contact with the object. That is, the correction process is performed on the position and attitude acquired when the end of the probe 7 is in contact with the object, but the correction process may not be performed on the position and attitude acquired when the end of the probe 7 is not in contact with the object. In this case, only the position and attitude that have been corrected may be used for controlling the robot 1, for example, for controlling playback in teaching playback. It should be noted that any method may be used to perform the correction process only on the position and attitude acquired when the end of the probe 7 is in contact with the object. For example, when the end of the probe 7 is not in contact with the object, an image may not be acquired, and the position and orientation may not be acquired for an image acquired when the end of the probe 7 is not in contact with the object, or correction processing may not be performed for the position and orientation acquired from an image acquired when the end of the probe 7 is not in contact with the object.
[0048] In this way, the teaching data is acquired using only the captured images acquired when the end of the probe 7, such as the tip 7c, is in proper contact with the object, and as a result, the accuracy of the teaching data can be improved. Also, while moving the probe 7, the operator can easily know whether the end of the probe 7 is in proper contact with the object by the output of the output unit 72. Therefore, for example, when it is found by the alert output from the output unit 72 that the end of the probe 7 is not in contact with the object, the operator can press the end of the probe 7 against the object to acquire the proper position and orientation.
[0049] Here, the case where the probe 7 includes both the output unit 72 and the transmission unit 73 has been mainly described, but this is not necessarily the case. The probe 7 may not include, for example, the output unit 72. Even in this case, the position and orientation acquisition device 3 can acquire the position and orientation after highly accurate correction processing by using the contact state signal transmitted from the transmission unit 73. Also, the probe 7 may not include, for example, the transmission unit 73. In this case, the position and orientation acquisition device 3 may not perform processing using the contact state signal. Even in this case, by using the output of the alert by the output unit 72, the operator can appropriately move the probe 7 while keeping it in contact with the object. As a result, the accuracy of the position and orientation acquired by the position and orientation acquisition device 3 can be improved.
[0050] In addition, in the present embodiment, the case where correction processing regarding the attitude of the probe 7 is also performed has been mainly described, but this is not necessarily the case. Correction processing for reducing the influence of camera shake may not be performed on the attitude of the probe 7. In this case, the position and orientation acquisition device 3 may not include the orientation correction unit 36. Also, in this case, the attitude of the probe 7 acquired by the position and orientation acquisition unit 33 may be used as it is as teaching data without being corrected.
[0051] In addition, in the present embodiment, the case where the marker 5 is a two-dimensional predetermined image has been mainly described, but this is not necessarily the case. As in the markerless AR, at least a part of the probe 7 may be used as the marker. In this case, the marker may be three-dimensional. Even when a two-dimensional marker 5 is used, a plurality of two-dimensional markers 5 may be attached to a plurality of surfaces of the probe 7 so that the marker 5 can be photographed even if the attitude of the probe 7 changes. It is preferable that the plurality of two-dimensional markers 5 are distinguishable from each other.
[0052] In the present embodiment, the position and attitude acquisition device 3 and the probe 7 are mainly used to teach a welding robot, but this is not essential. The position and attitude acquisition device 3 and the probe 7 may be used to teach a robot having a sealing gun (caulking gun) or other work tool attached to its end.
[0053] In addition, in the present embodiment, the position and orientation acquisition device 3 is mainly provided with the output unit 37 that outputs the position and orientation after the correction process, but this is not necessarily the case. When no information is output, the position and orientation acquisition device 3 does not need to have the output unit 37.
[0054] In addition, in the above embodiments, each process or function may be realized by centralized processing in a single device or a single system, or may be realized by distributed processing in multiple devices or multiple systems.
[0055] In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component may be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. During execution, the program execution unit may execute the program while accessing a storage unit or a recording medium. The program may be executed by being downloaded from a server or the like, or may be executed by reading a program recorded on a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.). The program may be used as a program constituting a program product. The computer that executes the program may be single or multiple. That is, centralized processing may be performed, or distributed processing may be performed.
[0056] In addition, in the above embodiments, each process or function may be realized by centralized processing in a single device or a single system, or may be realized by distributed processing in multiple devices or multiple systems.
[0057] The above-mentioned embodiments are merely examples for specifically implementing the present invention, and are not intended to limit the technical scope of the present invention. The technical scope of the present invention is indicated by the claims, not by the description of the embodiments, and is intended to include modifications within the literal scope of the claims and the scope of equivalent meanings. [Explanation of symbols]
[0058] 3 Position and orientation acquisition device, 5 Marker, 7 Probe, 31 Photographed image acquisition unit, 32 Reception unit, 33 Position and orientation acquisition unit, 34 Memory unit, 35 Position correction unit, 36 Attitude correction unit, 70 Probe body, 71 Sensor, 72 Output unit, 73 Transmission unit
Claims
1. an image acquisition unit for acquiring an image of a probe moved by an operator; a position and orientation acquisition unit that acquires a position and orientation of the probe using the captured image; a position correction unit that performs a correction process on the position of the probe acquired by the position and orientation acquisition unit to reduce the effect of camera shake by the operator.
2. 2 . The position and orientation acquisition apparatus according to claim 1 , wherein the correction process performed by the position correction unit is smoothing using a moving average of a plurality of positions acquired in accordance with the movement of the probe.
3. 2 . The position and orientation acquisition device according to claim 1 , further comprising an orientation correction unit that performs a correction process for reducing the influence of shaking of the operator on the orientation of the probe acquired by the position and orientation acquisition unit.
4. 4. The position and orientation acquisition device according to claim 3, wherein the correction process performed by the orientation correction unit is an averaging process for a plurality of orientations acquired in response to movement of the probe.
5. a receiving unit that receives a contact state signal indicating whether the end of the probe is in contact with an object, 5. The position and orientation acquisition device according to claim 1, wherein the position correction unit performs the correction process on a position of the probe when the contact state signal indicates that the end of the probe is in contact with an object.
6. A probe that is moved by an operator, and a position and orientation of the probe are acquired using a captured image of the probe, A probe body; a sensor provided at an end of the probe body for detecting whether or not the probe is in contact with an object; and an output unit that outputs an alert when no contact is detected by the sensor.
7. A probe that is moved by an operator, and a position and orientation of the probe are acquired using a captured image of the probe, A probe body; a sensor provided at an end of the probe body for detecting whether or not the probe is in contact with an object; a transmitter for transmitting a contact state signal indicating whether or not the end of the probe is in contact with an object in response to a detection result by the sensor.
Citation Information
Patent Citations
Welding system, welding robot control parameter creation device, welding robot control parameter creation method, welding robot control parameter creation program, and welding robot control program creation program
JP2021194724A
