Object recognition device, control device and object recognition computer program

Abstract

Object recognition device which features: a storage device (33) that stores a template representing a feature of the appearance of a target object (10) when the target object (10) is viewed from a predetermined direction; a robust detection unit (41) which, when an image acquisition unit (4) that captures the target object (10) and produces an image representing the target object (10), and the target object (10) does not satisfy a predefined positional relationship, detects a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to detect the target object (10) from the image, wherein the predefined positional relationship is set such that a distance between the image acquisition unit (4) and the target object (10) is equal to or less than a predefined distance; and a precision recognition unit (42) which, when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, recognizes a position of the target object (10) on the image by comparing the image with the template.

Description

BACKGROUND OF THE INVENTION 1. Field of the invention

[0001] The present invention relates, for example, to an object recognition device and an object recognition computer program that recognize an object depicted in an image, and to a control device that controls an automatic machine including a moving element using a recognition result of the object depicted in the image. 2. Description of the related technology

[0002] In the case of an automated machine, such as a robot or a machine tool, which includes a moving element, such as an arm driven by a servo motor, techniques have been proposed for using an image of an object as the machine's work target, captured by a camera, to control the machine based on the relative positional relationship between the moving element and the work target. One such technique proposed is for recognizing an object from an image using pattern matching (see, for example, Japanese patent application publication (Kokai) No. 2018-158439).Furthermore, another of these techniques proposed is a technique for obtaining control information to induce an automated machine to perform a predetermined operation by inputting an image showing an object as the work target into a classifier obtained through a machine learning method, such as a neural network (see, for example, Japanese Patent No. 6376296, Japanese Patent Application Publication (Kokai) No. 2009-83095, Japanese Patent Application Publication (Kokai) No. 2018-126799 and Japanese Patent Application Publication (Kokai) No. 2018-205929).

[0003] DE 32 34 608 A1 discloses a method and a circuit arrangement for generating a position-independent object signature, wherein a purely image-internal, two-stage recognition is provided, in which the stage change is controlled by image quality metrics and not by real geometric conditions or actuator feedback. OVERVIEW OF THE INVENTION

[0004] For example, in a case where an object is transported as a work target using a conveyor belt or similar device, the object may be moved. In such a case, the position, orientation, and size of the object depicted in an image change depending on the positional relationship between the camera and the object. Thus, a feature of the object used for pattern matching, such as part of a surface or a contour of the object, may be obscured when viewed from a certain direction, making it difficult to accurately determine the object's position and orientation during pattern matching.This leads to a situation where, during object tracking, a control device of an automated machine cannot continue tracking the object based on an object recognition result from a time series of images, and sometimes cannot move a moving element into a position where work can be performed on the object. Furthermore, since the position, orientation, and size of an object depicted in an image as the work target can change, the position of the area in which the object is depicted in the image may be unknown. In such a case, pattern matching must be performed across the entire image to detect the object, and the time required for object detection can become excessive. In such a case, it becomes difficult for the moving element to follow any change in the object's position.

[0005] On the other hand, if the position and orientation of an object depicted in an image are recognized as a work objective using a classifier based on a machine learning algorithm, then a control device of an automated machine can recognize the object from the image, regardless of the object's position and orientation relative to a camera. However, in this case, the recognition accuracy of the object's position and orientation may not necessarily be sufficient, and for this reason, the control device may sometimes be unable to move a moving element precisely into a position where the moving element can perform work on the object based on the object's position recognized by the classifier.

[0006] According to one aspect, the task is to provide an object recognition device that is able to recognize an object as a recognition target, even when a relative positional relationship of the object changes with respect to an image acquisition unit capturing the object.

[0007] According to the present disclosure, an object recognition device, a control device, and an object recognition computer program are provided according to the independent claims. Further developments are described in the dependent claims.

[0008] According to one embodiment, an object recognition device is provided.The object recognition device includes a storage device (33) that stores a template representing a feature of the appearance of a target object (10) when the target object (10) is viewed from a predetermined direction; a robust recognition unit (41) (robust recognition unit) which, when an image acquisition unit (4) that captures the target object (10) and produces an image representing the target object (10), and the target object (10) does not satisfy a predetermined positional relationship, recognizes a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to recognize the target object (10) from the image; and a precision recognition unit (42) (precision recognition unit) which, when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, recognizes a position of the target object (10) on the image by comparing the image with the template.

[0009] According to a further embodiment, a control device of an automatic machine (2) is provided, which includes at least one movable element (12 to 16). The control device has a storage device (33) that stores a template representing a feature of the appearance of a target object (10) as the working target of the automatic machine (2) when the target object (10) is viewed from a predetermined direction;a robust detection unit (41) which, when an image acquisition unit (4) attached to the movable element (12 to 16), which captures the target object (10) and generates an image representing the target object (10), and the target object (10) do not satisfy a predefined positional relationship, detects a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to recognize the target object (10) from the image, and detects a position of the target object (10) in a real space, based on the position of the target object (10) on the image;a precision recognition unit (42) which, when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, recognizes a position of the target object (10) on the image by comparing the image with the template, recognizes a position of the target object (10) in a real space based on the position of the target object (10) on the image, or recognizes a position of the target object (10) in a real space based on positional information obtained from a positional information acquisition unit that obtains positional information indicating a relative position with respect to the target object (10);a proximity control unit (44) that controls the movable element (12 to 16) such that the image acquisition unit and the target object (10) fulfill the specified positional relationship, based on the position of the target object (10) in a real space, which is detected by the robust detection unit (41) when the image acquisition unit (4) and the target object (10) do not fulfill the specified positional relationship; and a work control unit (45) that controls the movable element (12 to 16) such that the movable element (12 to 16) is moved into a position in which the automatic machine (2) can perform specified work on the target object (10), based on the position of the target object (10) in a real space, which is detected by the precision detection unit (42) when the image acquisition unit (4) and the target object (10) fulfill the specified positional relationship.

[0010] According to yet another embodiment, an object recognition computer program is provided. The object recognition computer program causes a computer to perform the following steps: Detect, if an image acquisition unit (4) capturing a target object (10) and generating an image of the target object (10) and the target object (10) do not satisfy a predefined positional relationship, a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to recognize the target object (10) from the image; and Detect, if the image acquisition unit (4) and the target object (10) satisfy the predefined positional relationship, a position of the target object (10) on the image by comparing the image with a template that represents a feature of an appearance of the target object (10) when the target object (10) is viewed from a predefined direction.

[0011] According to one aspect, even if the relative positional relationship of an object as a recognition target changes in relation to an image acquisition unit capturing the object, the object can still be recognized. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a schematic configuration representation of a robot system according to one embodiment. Fig. Figure 2 is a schematic configuration representation of a control device. Fig. Figure 3 is a functional block diagram of a processor contained in the control device with respect to the processing of the control of the moving elements including the processing of object recognition. Fig. Figure 4 is a diagram that provides an overview of the processing of the control of the moving elements, including the processing of object recognition. Fig. 5 is an operational flow diagram of the processing of the control of the moving elements, including the processing of object recognition. Fig. Figure 6 is a schematic configuration drawing of a robot system according to a modified example. Fig. Figure 7 is a schematic configuration drawing of an object recognition device. DESCRIPTION OF THE EXECUTION FORMS

[0012] An object recognition device according to an embodiment of the present invention is described below with reference to the drawings. The object recognition device, for example, recognizes an object as a work target (hereinafter simply referred to as the target object) of an automatic machine having a movable element, such as an arm, from a series of images taken by an image acquisition unit attached to the movable element of the automatic machine, and recognizes a position of the target object in a real space.The object recognition device at this time recognizes a position of the target object on an image in a robust manner by inputting the image into a classifier that has been previously trained to recognize the target object from the image until the image acquisition unit and the target object satisfy a predefined positional relationship, and recognizes a position of the target object in real space based on the recognition result.If, on the other hand, the image acquisition unit and the target object fulfill the predefined positional relationship, the object recognition device detects the position and orientation of the target object on an image with high accuracy by comparing it to a previously created template, which represents a feature of the target object's appearance when viewed from a predefined direction. Based on this detection result, the device then determines the target object's position and orientation in real space. Furthermore, the object recognition device is integrated into a control device of the automatic machine and controls the moving element of the automatic machine in such a way that the automatic machine can perform operations on the target object based on the detected position or the detected position and orientation of the target object.

[0013] Fig. Figure 1 is a schematic configuration representation of a robot system 1 on which an object recognition device according to one embodiment is mounted. The robot system 1 comprises a robot 2, a control device 3 that controls the robot 2, and a camera 4 that is attached to a movable element of the robot 2 and is used to pick up a workpiece 10, which is an example of a target object. The robot system 1 is an example of an automatic machine.

[0014] The robot 2 comprises a base 11, a rotary table 12, a first arm 13, a second arm 14, a wrist 15, and a tool 16. The rotary table 12, the first arm 13, the second arm 14, the wrist 15, and the tool 16 each represent an example of a movable element. The rotary table 12, the first arm 13, the second arm 14, and the wrist 15 are each supported by a shaft located in a joint, with the rotary table 12, the first arm 13, the second arm 14, and the wrist 15 being attached to their respective joints and driven by a servo motor that drives the shaft. Furthermore, the workpiece 10 is transported, for example, by a conveyor belt, and the robot 2 performs predefined tasks on the workpiece 10 while the workpiece 10 remains within a predefined area.

[0015] The base 11 is an element that serves as a stand when the robot 2 is placed on a floor. The rotary table 12 is rotatably attached to a top surface of the base 11 at a joint 22, by means of a shaft (not shown) which serves as the center of rotation and is positioned perpendicular to the top surface of the base 11.

[0016] The first arm 13 is attached at one of its ends to the rotary table 12, specifically to a joint 22 provided on the rotary table 12. In the present embodiment, as shown in Fig. As shown in Figure 1, the first arm 13 is rotatable about a shaft (not shown) which is provided in the joint 22 parallel to the surface of the base 11 on which the rotary table 12 is attached.

[0017] The second arm 14 is attached to the first arm 13 at one of its end faces, specifically at a joint 23 which is provided on the other end face of the first arm 13 opposite the joint 22. In the present embodiment, as shown in Fig. As shown in Figure 1, the second arm 14 is rotatable about a shaft (not shown) which is provided in the joint 23 parallel to the surface of the base 11 on which the rotary table 12 is attached.

[0018] The wrist 15 is attached via a joint 24 to a tip of the second arm 14 opposite the joint 23. The wrist 15 has a joint 25 and can be flexed with respect to a shaft (not shown) serving as a center of rotation, which is provided in the joint 25 parallel to the shaft of the joint 22 and to the shaft of the joint 23. Furthermore, the wrist 15 can be rotatable in a plane orthogonal to a longitudinal direction of the second arm 14, with a shaft (not shown) as its center of rotation extending parallel to the longitudinal direction of the second arm 14.

[0019] The tool 16 is attached to one end of the wrist 15 opposite the joint 24. The tool 16 has a mechanism or device for performing work on the workpiece 10. For example, the tool 16 may have a laser for processing the workpiece 10 or a servo clamp for welding the workpiece 10. Alternatively, the tool 16 may have a hand mechanism for holding the workpiece 10 or a part mounted on the workpiece 10.

[0020] Camera 4 is an example of an image acquisition unit and is attached, for instance, to tool 16. It should be noted that camera 4 can also be attached to another moving element, such as wrist 15 or second arm 14. When robot 2 performs work on workpiece 10, or when tool 16 approaches workpiece 10, camera 4 is aligned so that workpiece 10 is included within its field of view. Camera 4 then generates an image depicting workpiece 10 by capturing the entire field of view, including workpiece 10, during each predefined image acquisition period. Each time camera 4 generates an image, it transmits the generated image to control device 3 via a communication line 5.

[0021] It should be noted that an automatic machine, which is a control target of the control device 3, does not rely on the in Fig. 1 depicted robot 2 is limited and can have at least one movable element.

[0022] The control device 3 is connected to the robot 2 via the communication line 5 and receives information from the robot 2 via the communication line 5 indicating the operating status of the servo motor driving the shaft provided in each of the robot 2's joints, an image from the camera 4, and the like. The control device 3 then controls the servo motor based on the received information, the received image, and the robot 2's operation, which is received from a host control device (not shown) or was previously set, thus controlling the position and orientation of each of the robot 2's moving elements.

[0023] Fig. Figure 2 is a schematic configuration representation of the control device 3. The control device 3 has a communication interface 31, a drive circuit 32, a memory 33, and a processor 34. Furthermore, the control device 3 can have a user interface (not shown), such as a touch panel.

[0024] The communication interface 31, for example, has a communication interface for connecting the control device 3 to the communication line 5, a circuit for performing the processing related to the transmission and reception of a signal via the communication line 5, and similar components. The communication interface 31 then receives information from the robot 2 via the communication line 5 indicating the operating status of a servomotor 35 (an example of a drive unit), such as a measured value of a rotational amplitude from an encoder for detecting the rotational amplitude of the servomotor 35, and forwards the information to the processor 34. It should be noted that Fig. Figure 2 represents a servomotor 35, but the robot 2 can have a servomotor for each joint, which drives a shaft of the joint. Furthermore, the communication interface 31 receives an image from the camera 4 and forwards the received image to the processor 34.

[0025] The drive circuit 32 is connected to the servo motor 35 via a power supply cable and supplies electrical current according to the torque generated by the servo motor 35, a direction of rotation or a speed of the servo motor 35 in accordance with the control by the processor 34.

[0026] Memory 33 is an example of a storage unit and includes, for example, a readable and writable semiconductor memory and a read-only semiconductor memory. Memory 33 can also include a storage medium, such as a semiconductor memory card, a hard drive, or an optical storage medium, as well as a device that accesses the storage medium.

[0027] Memory 33 stores various computer programs for controlling robot 2, which are executed in the processor 34 of the control device 3, and similar information. Memory 33 also stores information for controlling the operation of robot 2 when the robot 2 is running. Furthermore, memory 33 stores information indicating the operating status of the servo motor 35, which is received by robot 2 during its operation. Memory 33 also stores various data used in object recognition processing. This data includes, for example, a parameter set for defining a classifier, a template representing a feature of the appearance of workpiece 10 when viewed from a predefined direction (e.g., vertically from above), which is used to recognize workpiece 10, and a camera parameter specifying information relating to camera 4, such as...a focal length, a mounting position and an orientation of camera 4, and an image obtained from camera 4.

[0028] Processor 34 is an example of a control unit and includes, for example, a central processing unit (CPU) and CPU peripheral circuitry. Processor 34 may also include a processor for arithmetic operations. Processor 34 then controls the robot system 1 as a whole. Furthermore, Processor 34 handles the processing of the control of the moving elements, including object recognition.

[0029] Fig. Figure 3 is a functional block diagram of the processor 34 relating to the processing of the control of the moving elements, including object recognition processing. The processor 34 comprises a robust detection unit 41, a precision detection unit 42, a determination unit 43, a proximity control unit 44, and a working control unit 45. Each of these units of the processor 34 is, for example, a functional module implemented by a computer program executed on the processor 34. Alternatively, each of these units can be implemented as a dedicated computing circuit mounted on a part of the processor 34. Furthermore, the object recognition processing includes processing by the robust detection unit 41, the precision detection unit 42, and the determination unit 43 among the units of the processor 34.

[0030] The robust detection unit 41 detects, by inputting each of a time series of images captured by the camera 4 into the classifier, the position of the workpiece 10 in the image, i.e., the position of a target object, when the camera 4 and the workpiece 10 do not satisfy a predefined positional relationship. Subsequently, the robust detection unit 41 detects the position of the workpiece 10 in real space based on its position detected in the image. Because the robust detection unit 41 detects the position of the workpiece 10 using the classifier, it can detect the position of the workpiece 10 more robustly than the precision detection unit 42. Since the robust detection unit 41 can perform the same processing on each image, the following section describes the processing on a single image.Furthermore, the specified positional relationship with the unit of determination 43 is described in detail.

[0031] The classifier was trained to recognize the workpiece 10 depicted in an image. For example, the robust detection unit 41 can use a convolutional neural network (CNN) as a classifier, which has been previously trained to recognize an object depicted in an image and display an area in which the recognized object is represented, such as a VGG or a single-shot multibox detector (SSD). In this case, the robust detection unit 41 can obtain a circumscribed rectangle (i.e., a bounding box) of the workpiece 10 depicted in the image by inputting an image of interest into the classifier.

[0032] Alternatively, the robust detection unit 41 can use a classifier previously trained using another machine learning method, such as a support vector machine or AdaBoost. In this case, the robust detection unit 41 sets a plurality of distinct windows for an image of interest and extracts a set of features from each window, such as oriented gradient (HOG) histograms or a hair-like feature. It is important to note that if a position of the workpiece 10 is detected in a previously acquired image, the robust detection unit 41 can set a plurality of windows limited to the area near the detected position. The robust detection unit 41 then inputs the set of features extracted from each window into the classifier.When the feature set is entered, the classifier outputs a result indicating whether workpiece 10 is displayed in the window or not. Therefore, the robust detection unit 41 can determine that workpiece 10 is displayed in the window, the result of which, that workpiece 10 is displayed, is output by the classifier.

[0033] As described above, when an area in which the workpiece 10 is shown in the image is detected, the robust detection unit 41 sets a predefined position in the area, e.g. the center of gravity of the area, as the position of the workpiece 10 in the image.

[0034] If the position of the workpiece 10 is obtained from the image, the robust detection unit 41 recognizes a position of the workpiece 10 in a real space based on the position.

[0035] The position of each pixel in the image corresponds one-to-one to a bearing as seen from camera 4. For example, the robust detection unit 41 can determine a bearing associated with the center of gravity of the area where workpiece 10 is depicted in the image, as the bearing from camera 4 to workpiece 10. Furthermore, the robust detection unit 41 can calculate an estimated distance from camera 4 to workpiece 10 by multiplying a predefined reference distance by the ratio of the area of ​​workpiece 10 in the image (where the reference distance is from camera 4 to workpiece 10) to the area of ​​the region where workpiece 10 is depicted. Therefore, the robust detection unit 41 can determine the position of workpiece 10 in a camera coordinate system relative to the position of camera 4 based on the bearing from camera 4 to workpiece 10 and the estimated distance.The robust detection unit 41 then outputs the detected position of the workpiece 10 to the determination unit 43 and the proximity control unit 44.

[0036] According to a modified example, the robust detection unit 41 can detect not only the position of workpiece 10, but also its orientation. In this case, the robust detection unit 41 can use a CNN as a classifier, which has been previously trained to recognize the area in which an object is represented in an image and the object's orientation (see, e.g., B. Tekin et al., "Real-Time Seamless Single Shot 6D Object Pose Prediction," CVPR2018). The robust detection unit 41 can detect not only the position of workpiece 10 in an image, but also its orientation as viewed from camera 4, i.e., as displayed in the camera's coordinate system, by inputting the image into such a classifier. The detected orientation of workpiece 10 is indicated, for example, by a combination of roll, pitch, and yaw angles.

[0037] In this case, the robust detection unit 41 can output the detected position and the detected orientation of the workpiece 10 to the determination unit 43 and the proximity control unit 44.

[0038] If the camera 4 and the workpiece 10 satisfy the predefined positional relationship, the precision detection unit 42, by comparing each frame of a time series of images captured by the camera 4 with a template, detects the position and orientation of the workpiece 10 in the image with higher accuracy than the robust detection unit 41. The precision detection unit 42 then detects the position and orientation of the workpiece 10 in real space based on the detected position and orientation of the workpiece 10 in the image. Since the precision detection unit 42 can perform the same processing for each image, the processing for a single image is described below. For example, the precision detection unit 42 reads a template from memory 33 and compares the template with an image of interest to detect an area in which the workpiece 10 is represented in the image.The template can, for example, represent an image of workpiece 10 viewed from a predefined direction. In this case, the precision recognition unit 42 calculates a value representing a degree of similarity between the template and the image, such as a cross-correlation value between the template and the image, while changing the relative position and orientation of the template with respect to the image within a matching area on the image. It determines that workpiece 10 is represented in an area corresponding to a position of the template when the degree of similarity is equal to or greater than a predefined threshold.Furthermore, the precision recognition unit 42 can calculate the actual rotational amount of the workpiece 10 relative to an orientation of the workpiece 10 depicted on the template when viewed from a predefined direction, based on the orientation of the template, if the degree of similarity is equal to or greater than the predefined threshold. In this way, the precision recognition unit 42 can obtain a position of the workpiece 10 based on the rotational amount. It should be noted that the precision recognition unit 42 can define as the matching area, e.g., a predefined area around a predicted position of the workpiece 10 on an image of interest, which is calculated by applying a prediction filter, such as a Kalman filter, to a position of the workpiece 10 (e.g.,the focal point of an area in which the workpiece 10 is depicted), which is recognized in each of a series of images taken in front of the image of interest.

[0039] Furthermore, the template can represent a multitude of feature points of the workpiece 10 when viewed from the specified direction, e.g., a plurality of points on a contour of the workpiece 10. In this case, the precision recognition unit 42 can recognize the plurality of feature points by applying an edge detection filter, such as a Sobel filter, or a corner detection filter, such as a Harris filter, or another feature point detection filter, such as a SIFT (scale-invariant feature transformation), to the matching area on the image.The precision recognition unit 42 can then calculate the degree of similarity as the ratio of the number of feature points on the template that correspond to feature points on the image to the number of feature points on the template for each position of the template, while changing the relative position and orientation of the template with respect to the image within the matching area. Again, the precision recognition unit 42 can determine that the workpiece 10 is represented in an area corresponding to a position of the template if the degree of similarity is equal to or greater than the specified threshold.

[0040] Alternatively, if there is a large difference between the brightness of the workpiece 10 and the brightness of a background surrounding the workpiece 10, the precision detection unit 42 can detect an area in which the workpiece 10 is represented by binarizing each pixel of an image based on a luminance value of the pixel. If, for example, the workpiece 10 is brighter than the surroundings, the precision detection unit 42 can detect a pixel with a luminance value higher than a predefined luminance threshold and define as the area in which the workpiece 10 is represented a region consisting of a set of the detected pixels.In this case, the precision recognition unit 42 can, for example, compare a template prepared for each position of the workpiece 10 with the area in which the workpiece 10 is represented and determine as the actual position of the workpiece 10 the position that corresponds to the template whose shape most closely resembles the area in which the workpiece 10 is represented. It should be noted that in this case, the template can also be made into a binary image in which the area representing the workpiece and another area have different values.

[0041] If the position of the area in which the workpiece 10 is depicted in the image is determined, the precision detection unit 42 can determine a position of the workpiece 10 specified in the camera coordinate system, similar to the robust detection unit 41. Furthermore, the precision detection unit 42 can determine a position of the workpiece 10 displayed in the camera coordinate system by rotating the position of the workpiece 10 in the image by a difference between a predefined direction defined for the template and a bearing from the camera 4 that corresponds to the center of gravity of the area in which the workpiece 10 is depicted.

[0042] It should be noted that a plurality of templates can be prepared in advance. In this case, each of the templates can vary depending on the combination of the direction in which the workpiece 10 is viewed and the size of the workpiece 10 on the template. In this case, the precision recognition unit 42 calculates a degree of similarity for each position of the template for each of the plurality of templates, while, similarly to what is described above, the relative position and orientation of the template with respect to the image within the area of ​​similarity are changed. The precision recognition unit 42 can then determine that the workpiece 10, viewed from a direction specified on the template with the highest degree of similarity, is represented in an area corresponding to a position of the template.Furthermore, a size of the workpiece 10 depicted on the template corresponds to a distance from the camera 4 to the workpiece 10. Thus, the precision recognition unit 42 can determine a distance from the camera 4 to the workpiece 10 that corresponds to a size of the workpiece 10 depicted on the template with the greatest degree of similarity.

[0043] The precision detection unit 42 can detect the position of the workpiece 10 in real space using a different technique. For example, the precision detection unit 42 can detect an absolute position (i.e., a position specified in a world coordinate system) of each of the moving elements of the robot 2, based on the rotation of the servomotor 35 from a position where the robot 2 assumes a reference position. It can also detect the position and orientation of the camera 4 in the world coordinate system, based on a relationship between the detected absolute position and the mounting position and orientation of the camera 4. Furthermore, the precision detection unit 42 can determine a bearing from the camera 4 towards the workpiece 10 in the world coordinate system, based on the center of gravity of the area where the workpiece 10 is depicted in the image.Furthermore, if the position of a conveyor belt transporting workpiece 10 in the world coordinate system and the thickness of workpiece 10 are known (these values ​​are, for example, previously stored in memory 33), the precision detection unit 42 can determine the position of camera 4 in the world coordinate system and thus calculate the distance from camera 4 to the conveyor belt transporting workpiece 10, using the bearing from camera 4 to workpiece 10. The precision detection unit 42 can then obtain a value for the distance from camera 4 to workpiece 10, calculated by subtracting the thickness of workpiece 10 from this calculated distance. The precision detection unit 42 can detect the position of workpiece 10 in real space using the bearing from camera 4 towards workpiece 10 and the distance to workpiece 10 obtained in this way.

[0044] The precision detection unit 42 outputs the detected position and the detected location of the workpiece 10 to the determination unit 43 and the work control unit 45.

[0045] The determination unit 43 determines, each time an image is received from the camera 4, whether the camera 4 and the workpiece 10 satisfy a predefined positional relationship. The predefined positional relationship can be set such that, for example, a distance between the camera 4 and the workpiece 10 is equal to or less than a specified distance. For example, the specified distance can be a distance between the camera 4 and the workpiece 10 that corresponds to a size of the workpiece 10 represented on a template used by the precision detection unit 42. For example, if the robust detection unit 41 has detected a position of the workpiece 10 for the latest image, the determination unit 43 compares an estimated distance from the camera 4 to the workpiece 10, calculated by the robust detection unit 41, with the specified distance.If the estimated distance is equal to or less than the specified distance, the determination unit 43 determines that the camera 4 and the workpiece 10 fulfill the specified positional relationship. The determination unit 43 then instructs the precision detection unit 42 to detect a position and orientation of the workpiece 10 for the next image to be captured. Conversely, if the estimated distance is greater than the specified distance, the determination unit 43 determines that the camera 4 and the workpiece 10 do not fulfill the specified positional relationship. The determination unit 43 then instructs the robust detection unit 41 to detect a position of the workpiece 10 for the next image to be captured.

[0046] It should be noted that in a case where a coordinate system of the robot 2 is moved on the basis of a movement amount of the workpiece 10 and an operating program of the robot 2 is programmed such that the robot 2 works on the coordinate system, the determination unit 43 can switch from the robust detection unit 41 to the precision detection unit 42 when the robot 2 reaches a predetermined position on the coordinate system.

[0047] Similarly, once the precision recognition unit 42 has detected a position of the workpiece 10 for the latest image, the determination unit 43 compares a distance from the camera 4 to the workpiece 10, calculated by the precision recognition unit 42, with the predefined distance. If the distance is equal to or less than the predefined distance, the determination unit 43 determines that the camera 4 and the workpiece 10 satisfy the predefined positional relationship. The determination unit 43 then instructs the precision recognition unit 42 to detect a position and orientation of the workpiece 10 for the next image to be captured. If, however, the distance is greater than the predefined distance, the determination unit 43 determines that the camera 4 and the workpiece 10 do not satisfy the predefined positional relationship.Then the determination unit 43 instructs the robust detection unit 41 to detect a position of the workpiece 10 for a picture to be taken next.

[0048] Alternatively, the predefined positional relationship can be set such that the distance between the camera 4 and the workpiece 10 is equal to or less than a predefined distance, and the workpiece 10 assumes a predefined position relative to the camera 4. The predefined position can, for example, be a position in which the camera 4 is located within a predefined angular range from a normal to a predefined surface of the workpiece 10. The predefined surface can, for example, be a surface of the workpiece 10 that is represented on a template used by the precision recognition unit 42. In this case, once the robust recognition unit 41 has detected a position and a position of the workpiece 10 for the latest image, the determination unit 43 can determine whether the detected position is a predefined position or not.If the estimated distance from camera 4 to workpiece 10, calculated by the robust detection unit 41, is equal to or less than the specified distance, and the position of workpiece 10 is the specified position, then the determination unit 43 determines that camera 4 and workpiece 10 fulfill the specified positional relationship. The determination unit 43 then instructs the precision detection unit 42 to determine a position and a position of workpiece 10 for the next image to be captured. Conversely, if the estimated distance is greater than the specified distance, or the position of workpiece 10 is not the specified position, the determination unit 43 determines that camera 4 and workpiece 10 do not fulfill the specified positional relationship. In this case, the determination unit 43 instructs the robust detection unit 41 to determine a position of workpiece 10 for the next image to be captured.

[0049] Similarly, once the precision recognition unit 42 has detected a position and orientation of the workpiece 10 for the latest image, the determination unit 43 determines that the camera 4 and the workpiece 10 satisfy the specified positional relationship if the distance from the camera 4 to the workpiece 10, calculated by the precision recognition unit 42, is equal to or less than the specified distance, and the orientation of the workpiece 10 is the specified orientation. Then, the determination unit 43 instructs the precision recognition unit 42 to detect a position and orientation of the workpiece 10 for the next image to be captured. Conversely, if the distance is greater than the specified distance, or if the orientation of the workpiece 10 is not the specified orientation, the determination unit 43 determines that the camera 4 and the workpiece 10 do not satisfy the specified positional relationship.Then the determination unit 43 instructs the robust detection unit 41 to detect a position of the workpiece 10 for a picture to be taken next.

[0050] The proximity control unit 44 controls the moving element of the robot 2 such that the camera 4 and the workpiece 10 fulfill the predefined positional relationship. This is achieved based on the position of the workpiece 10, which is detected by the robust detection unit 41 when the camera 4 and the workpiece 10 do not fulfill the predefined positional relationship. For example, the proximity control unit 44 predicts a position of the workpiece 10 in the world coordinate system at the time the next image is generated by applying a prediction filter, such as a Kalman filter, to a position of the workpiece 10 in real space, which was detected, for example, for each of a series of images in the last period. The proximity control unit 44 then sets a target position at which the camera 4 fulfills the predefined positional relationship from the predicted position of the workpiece 10.The proximity control unit 44 can determine the amount and direction of rotation of the servomotor 35 so that the movable element of the robot 2, to which the camera 4 is attached, is actuated to move the camera 4 to the target position.

[0051] At this point, the proximity control unit 44 calculates a conversion equation from the camera coordinate system to the world coordinate system for each of the images from the last period, if, for example, the image was generated from a position and orientation of the camera 4 in the world coordinate system. Thus, the proximity control unit 44 can, for example, detect an absolute position of each of the moving elements of the robot 2 for each image, based on the amount of rotation of the servomotor 35 at the time of image generation from a position of each of the moving elements when the robot 2 is in a reference position.Furthermore, the proximity control unit 44 can detect the position and orientation of the camera 4 in the world coordinate system for each image, based on a relationship between the detected absolute position and the mounting position and orientation of the camera 4. Based on this detection result, it calculates a conversion equation from the camera coordinate system to the world coordinate system. Then, for each image, the proximity control unit 44 detects a position of the workpiece 10 specified in the world coordinate system by applying the conversion equation to the position of the workpiece 10 specified in the camera coordinate system at the time of image generation.Then, as described above, the proximity control unit 44 can calculate a predetermined position of the workpiece 10 in the world coordinate system at the time of generating the next image by applying a prediction filter to the detected position of the workpiece 10 in the world coordinate system at the time of generating each image.

[0052] If the predicted position of workpiece 10 is calculated in the world coordinate system and the distance between camera 4 and workpiece 10 is equal to or less than a predefined distance in the predefined positional relationship, the proximity control unit 44 can determine a position at the predefined distance from the predicted position as the target position. The proximity control unit 44 can then determine a target rotation amount and direction for the servo motor 35 to move camera 4 to the target position.

[0053] Furthermore, it is assumed that the predefined positional relationship is set such that the distance between the camera 4 and the workpiece 10 is equal to or less than a predefined distance, the workpiece 10 occupies a predefined position relative to the camera 4, and the position and orientation of the workpiece 10 in real space are detected by the robust detection unit 41. In this case, the proximity control unit 44 can predict the orientation of the workpiece 10 at the time of generating the next image, based on the orientation of the workpiece 10 at the time of generating each image in the last period.In this case as well, the proximity control unit 44 can predict the position of the workpiece 10 in the world coordinate system at the time of generating the next image by applying a prediction filter to the position of the workpiece 10 in the world coordinate system. The calculation is performed by applying the conversion equation from the camera coordinate system to the world coordinate system to the position of the workpiece 10 specified in the camera coordinate system at the time of generating each image in the last period, similar to the procedure described above. The proximity control unit 44 can then calculate a target position and a target orientation of the camera 4 such that the distance between the camera 4 and the workpiece 10 is equal to or less than the specified distance, and the workpiece 10 assumes the specified orientation relative to the camera 4, based on the predicted position and orientation of the workpiece 10.The proximity control unit 44 can determine a target rotation amount and a target direction of rotation of the servomotor 35 so that the camera 4 is moved into the target position and the camera 4 assumes the target position.

[0054] It should be noted that the proximity control unit 44 can determine a target position and a target orientation when the workpiece 10 is stationary by defining a detected position and a detected orientation of the workpiece 10 at the time of generating the latest images as the predicted position and orientation described above, and can determine a target rotation amount and a target rotation direction of the servomotor 35.

[0055] If the proximity control unit 44 determines the target rotational speed and direction of rotation of the servomotor 35, then the proximity control unit 44 can control the servomotor 35, as described, for example, in Japanese patent application publication (Kokai) No. 2006-172149. In other words, when controlling the operation of the robot 2, the proximity control unit 44 calculates a speed command that specifies a rotational speed of the servomotor 35, based on the target rotational speed, the target rotational direction, the actual rotational speed of the servomotor 35, and similar parameters. The proximity control unit 44 performs a speed loop control based on the speed command and speed feedback calculated by differentiating the actual rotational speed of the servomotor 35 to calculate a torque command that specifies the amount of current supplied to the servomotor 35.The proximity control unit 44 then performs a loop control for a current supplied to the servomotor 35, based on the torque command and current feedback provided by a current detector (not shown) in the drive circuit 32, to drive the servomotor 35.

[0056] The work control unit 45 controls each of the moving elements of the robot 2 such that the tool 16 of the robot 2 moves into a position in which the tool 16 can perform work on the workpiece 10, based on a position and orientation of the workpiece 10 detected by the precision detection unit 42 when the camera 4 and the workpiece 10 satisfy the predefined positional relationship. At this point, the work control unit 45, similar to the proximity control unit 44, can calculate a predicted position and orientation of the workpiece 10 at the time of generating the next image by applying a prediction filter, such as a Kalman filter, to a position and orientation of the workpiece 10 in the world coordinate system that were detected for each of a series of images in the last period.The work control unit 45 can then determine the amount and direction of rotation of the servomotor 35 by setting a target position and target orientation of the tool 16 based on the predicted position and location.

[0057] Alternatively, the work control unit 45 can control each of the moving elements of the robot 2 by displaying the workpiece 10 in an image generated by the camera 4 at a predetermined position and size, corresponding to a position in which the tool 16 performs work on the workpiece 10. In this case, the work control unit 45 can control each of the moving elements of the robot 2 according to a technique for controlling a robot based on a camera-captured image of a target object, such as a position- or feature-based method (see, for example, Hashimoto, “Vision and Control”, The Society of Instrument and Control Engineers Control Division Convention Workshop, Kyoto, pp. 37–68, 2001).

[0058] As described above, the work control unit 45 controls the moving element of the robot 2 on the basis of a relative position and a relative orientation of the workpiece 10 with respect to the camera 4, which are detected with high accuracy, and therefore the work control unit 45 can control the robot 2 so that work on the workpiece 10 is carried out appropriately.

[0059] Fig. Figure 4 is a diagram that provides an overview of the processing of the control of the moving elements, including the processing of object recognition. In this example, it is assumed that a predefined positional relationship is established, such that the distance between the camera 4 and the workpiece 10 is equal to or less than a predefined distance Lth. As shown in Fig. As shown in Figure 4, when camera 4 is in position P1, the distance d1 between camera 4 and workpiece 10 is greater than the predefined distance Lth. Therefore, the robust detection unit 41 detects a position (or a position and a orientation) of workpiece 10 using a classifier. The proximity control unit 44 then controls the moving element of robot 2 based on the detection result, bringing camera 4 close to workpiece 10. However, when camera 4 is in position P2, the distance d2 between camera 4 and workpiece 10 is less than the predefined distance Lth. Therefore, the precision detection unit 42 detects a position and an orientation of workpiece 10 using a template.The work control unit 45 then controls the moving element of the robot 2 based on the recognition result and moves the tool 16, to which the camera 4 is attached, into a position in which the tool 16 can perform work on the workpiece 10.

[0060] Fig. Figure 5 is a flowchart for processing the control of the moving elements, including object recognition processing. Processor 34 performs the control of the moving elements processing each time an image is received from camera 4, according to the flowchart described below. Note that the processing in steps S101 to S102 and S104 of the following flowchart is included in the object recognition processing.

[0061] The determination unit 43 determines whether the camera 4 and the workpiece 10 fulfill a predefined positional relationship or not, based on a recognition result of a position and orientation of the workpiece 10 at the time of generation of a previous image (step S101). If the camera 4 and the workpiece 10 do not fulfill the predefined positional relationship (No in step S101), the determination unit 43 instructs the robust recognition unit 41 to recognize a position of the workpiece 10. The robust recognition unit 41 then, by inputting the latest image into a classifier, recognizes a position of the workpiece 10 in the image and recognizes a position of the workpiece 10 in real space based on the recognition result (step S102).Furthermore, the proximity control unit 44 controls the moving element of the robot 2 based on the detected position of the workpiece 10 in such a way that the camera 4 and the workpiece 10 fulfill the specified position relationship (step S103).

[0062] If, on the other hand, the camera 4 and the workpiece 10 fulfill the specified positional relationship (Yes in step S101), the determination unit 43 instructs the precision recognition unit 42 to detect a position and orientation of the workpiece 10. The precision recognition unit 42 then detects, by comparing the latest image with a template, a position and orientation of the workpiece 10 in the image and, based on the detection result, detects a position and orientation of the workpiece 10 in real space (step S104). Furthermore, based on the detected position and orientation of the workpiece 10, the work control unit 45 controls the moving element of the robot 2 such that the tool 16 moves into a position in which it can perform work on the workpiece 10 (step S105).

[0063] After step S103 or S105, processor 34 terminates the processing of the control of the moving element.

[0064] If, as described above, an image acquisition unit and a target object do not share a predefined positional relationship, the object recognition device robustly detects the target object's position using a machine learning-trained classifier. Conversely, if the image acquisition unit and the target object do share the predefined positional relationship, the object recognition device accurately detects the target object's position and orientation in an image by comparing the image to a pre-existing template that represents a feature of the target object's appearance when viewed from a predefined direction. Thus, the object recognition device can detect the target object even if the target object's relative position to the image acquisition unit changes.Furthermore, the object recognition device can detect the position of the target object in a real space using only one image acquisition unit, without employing a stereo method for capturing the position of a target object in a real space using multiple cameras. Additionally, if the image acquisition unit and the target object do not meet the predefined positional relationship, a control device of a robot containing the object recognition device controls a movable element of the robot in such a way that the image acquisition unit and the target object meet the predefined positional relationship, based on a position of the target object detected using the classifier.If, on the other hand, the image acquisition unit and the target object fulfill the predefined positional relationship, the control device controls the robot's moving element in such a way that work can be performed on the target object, based on a position and orientation of the target object that are determined as a result of template matching. Thus, even if a relative positional relationship between the image acquisition unit and the target object changes, the control device can continue to track the target object and appropriately control the robot's moving element according to the positional relationship.

[0065] It should be noted that if a shape does not change even when rotated around a central axis, as in a case where the workpiece 10 has a cylindrical shape, the precision recognition unit 42 may not recognize a position of the workpiece 10 on an image and in real space if the workpiece 10 is arranged in a certain position or the like.

[0066] According to a modified example, a position information acquisition unit can be provided separately from the camera 4. This unit acquires position information indicating a relative position with respect to the workpiece 10, which is used by the precision detection unit 42 to detect the position and orientation of the workpiece 10. For example, one or more pressure sensors can be attached to a tip of the tool 16. The pressure sensor is an example of the position information acquisition unit. In this case, the precision detection unit 42 can refer to a reference table that shows a relationship between the pressure detected by the pressure sensor and a distance between a predetermined section of the tool 16 and the workpiece 10, and detect the distance between the predetermined section of the tool 16 and the workpiece 10 according to the detected pressure.It should be noted that such a reference table is stored in advance in memory 33, for example. As in the embodiment described above, the work control unit 45 controls the moving element of the robot 2 based on an image captured by the camera 4 to move the tool 16 into a position where the pressure sensor can come into contact with the workpiece 10. Subsequently, the work control unit 45 can control the moving element of the robot 2 so that the pressure detected by the pressure sensor has a predetermined value. In this case, the pressure detected by the pressure sensor is an example of the position information.

[0067] Alternatively, a separate camera 4 can be attached to the moving element of the robot 2, and the precision detection unit 42 can detect the position and orientation of the workpiece 10 by performing similar processing on an image generated by the separate camera as described above. Alternatively, a distance sensor, such as a depth camera, can be attached to the moving element of the robot 2. Then, the precision detection unit 42 can detect the position and orientation of the workpiece 10 relative to the distance sensor based on a measured distance to each section of the workpiece 10 detected by the distance sensor.It should be noted that the separate camera and the distance sensor are another example of the position information acquisition unit, and that an image produced by the separate camera and a distance measurement taken by the distance sensor are another example of the position information.

[0068] According to another embodiment, the camera 4 can be fixedly mounted separately from the moving element of the robot 2. For example, the camera 4 can be mounted on the ceiling of a room in which the robot 2 is installed, pointing downwards. In this case, a distance table can be pre-stored in memory 33 for each pixel of an image captured by the camera 4. This table specifies the distance between the camera 4 and the workpiece 10 when the workpiece 10, as seen by the camera 4, is at a bearing corresponding to the pixel.When detecting an area in which workpiece 10 is depicted in the image, the robust detection unit 41 and the precision detection unit 42 can determine the position of workpiece 10 in real space by referring to the distance table and estimating a distance corresponding to one pixel, where the center of gravity of the area is located, as the distance from camera 4 to workpiece 10. Since the position of camera 4 does not change, in this case the robust detection unit 41 and the precision detection unit 42 can also easily determine the position and orientation of workpiece 10 in the world coordinate system by applying the conversion equation from the camera coordinate system with respect to the position of camera 4 to the world coordinate system to a position and orientation of workpiece 10 specified in the camera coordinate system.Then the approach control unit 44 and the work control unit 45 can control the moving element of the robot 2 similarly to the embodiment described above, based on the position and orientation of the workpiece 10 specified in the world coordinate system.

[0069] According to yet another embodiment, the object recognition device can be provided separately from the control device of the robot 2.

[0070] Fig. Figure 6 is a schematic configuration drawing of a robot system 100 according to the modified example. The robot system 100 comprises a robot 2, a control device 3 that controls the robot 2, a camera 4 that is attached to a movable element of the robot 2 and is used to capture a workpiece 10 representing an example of a target object, and an object recognition device 6. The robot 2 and the control device 3 are in communication connection via a communication line 5. Furthermore, the camera 4 and the object recognition device 6 are in communication connection via a communication line 7, and the control device 3 and the object recognition device 6 are in communication connection via a communication line 8.It should be noted that the camera 4, the control device 3 and the object recognition device 6 can be in communication connection with each other via a communication network in accordance with a specified communication standard.

[0071] The robot system 100 differs from the one in Fig. The robot system 1 shown in Figure 1 is distinguished by the fact that the object recognition device 6 is provided separately from the control device 3. Therefore, the object recognition device 6 and an associated part are described below.

[0072] Fig. Figure 7 is a schematic configuration drawing of the object recognition device 6. The object recognition device 6 includes a communication interface 61, a memory 62, and a processor 63. Furthermore, the object recognition device 6 may include a user interface (not shown), such as a touch panel.

[0073] The communication interface 61 includes, for example, a communication interface for connecting the object recognition device 6 to the communication line 7 and the communication line 8, a circuit for performing processing related to the transmission and reception of a signal via the communication line 7 and the communication line 8, and the like. The communication interface 61 then receives, for example, an image from the camera 4 via the communication line 7 and transmits the received image to the processor 63. Furthermore, the communication interface 61 receives information from the control device 3 specifying the position and orientation of the camera 4, determined based on the absolute position of each moving element at the time each image is generated, and transmits the received information to the processor 63.Furthermore, the communication interface 61 receives information from the processor 63 indicating a detected position and a detected orientation of the workpiece 10, and outputs the received information to the control device 3 via the communication line 8.

[0074] Memory 62 is another example of a storage unit and contains, for example, a readable and writable semiconductor memory and a read-only semiconductor memory. Memory 62 can also include a storage medium, such as a semiconductor memory card, a hard drive, or an optical storage medium, as well as a device that accesses the storage medium.

[0075] Memory 62 stores various pieces of information used in the object recognition processing performed by the processor 63 of the object recognition device 6, such as a parameter set defining a classifier, a template used to recognize the workpiece 10, a camera parameter, and an image received from the camera 4. Furthermore, memory 62 stores information indicating the position and orientation of the workpiece 10 as determined by the object recognition processing.

[0076] Processor 63 represents an example of a control unit and includes, for example, a central processing unit (CPU) and CPU peripheral circuitry. Processor 63 may also include a processor for arithmetic operations. Processor 63 then performs object recognition processing. In other words, each time Processor 63 receives an image from Camera 4, Processor 63 can initiate the processing of the robust detection unit 41, the precision detection unit 42, and the determination unit 43 among the units of Processor 34. Fig.The processor 63 executes the control device 3 shown in Figure 3. It then outputs information indicating the detected position and orientation of the workpiece 10 to the control device 3 via the communication interface 61 and the communication line 8. It should be noted that the processor 63 can include information indicating whether the camera 4 and the workpiece 10 fulfill a predefined positional relationship or not, in the information indicating the detected position and orientation of the workpiece 10. The processor 34 of the control device 3 can then execute the processing of the proximity control unit 44 and the working control unit 45 based on the received position and orientation of the workpiece 10.

[0077] According to the modified example, object recognition processing and robot control are performed by separate devices, thereby reducing the computational load on the robot's control device processor.

[0078] A robust detection unit 41 of the processor 63 can detect only one position of the workpiece 10 in each image received from the camera 4, or one position and one orientation of the workpiece 10 in each image, and cannot detect one position and one orientation of the workpiece 10 in real space. Likewise, a precision detection unit 42 of the processor 63 can detect only one position of the workpiece 10 in each image received from the camera 4, or one position and one orientation of the workpiece 10 in each image, and cannot detect one position and one orientation of the workpiece 10 in real space. In this case, the processor 63 can output the position and orientation of the workpiece 10 in the image to the control device 3 via the communication interface 61 and the communication line 8.The processor 34 of the control device 3 can then perform the processing of the detection of the position and orientation of the workpiece 10 in real space, based on the position and orientation of the workpiece 10 on the image in the robust detection unit 41 and the precision detection unit 42.

[0079] According to yet another modified example, a plurality of servers, in which the camera 4 and the control unit 3 are in communication with each other, can perform the object recognition processing. In this case, for example, any one of the plurality of servers can perform the processing of the robust detection unit 41, another of the plurality of servers can perform the processing of the precision detection unit 42, and yet another of the plurality of servers can perform the processing of the determination unit 43.

[0080] Furthermore, a computer program for performing the processing of each unit of the processor 34 of the control device 3 may be in the form of a recording on a computer-readable portable recording medium, such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0081] All examples and conditions listed herein serve the purpose of instruction, to make the concepts contributed by the inventor to this invention more easily understandable to the reader and to advance the state of the art, and are to be interpreted as not being limited to these specifically mentioned examples and conditions. Furthermore, the organization of such examples in the description does not relate to a presentation of the superiority and inferiority of the invention. Although the embodiment of the present invention is described in detail, it is understood that various modifications, substitutions, and alterations may be made to it without departing from the spirit and scope of the invention.

Claims

[1] Object recognition device comprising: a storage device (33) that stores a template representing a feature of the appearance of a target object (10) when the target object (10) is viewed from a predetermined direction; a robust detection unit (41) which, when an image acquisition unit (4) that captures the target object (10) and produces an image representing the target object (10), and the target object (10) does not satisfy a predefined positional relationship, detects a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to recognize the target object (10) from the image, wherein the predefined positional relationship is set such that a distance between the image acquisition unit (4) and the target object (10) is equal to or less than a predefined distance; and a precision recognition unit (42) which, when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, recognizes a position of the target object (10) on the image by comparing the image with the template. [2] Object recognition device according to claim 1, wherein The robust detection unit (41) detects the position of the target object (10) in a real space based on the detected position of the target object (10) in the image, and The precision recognition unit (42) recognizes the position of the target object (10) in a real space based on the recognized position of the target object (10) in the image. [3] Object recognition device according to claim 2, wherein the precision recognition unit (42) recognizes a position and a location of the target object (10) on the image according to a relative positional relationship of the template with respect to the image when the image and the template have the highest degree of similarity, and recognizes a position and a location of the target object (10) in a real space, based on the position and location of the target object (10) on the image. [4] Object recognition device according to claim 2 or 3, in which the classifier has further been trained to recognize a position of the target object (10) shown in the image, and the robust recognition unit (41) recognizes a position and a position of the target object (10) on the image by inputting the image into the classifier, and recognizes a position and a position of the target object (10) in a real space, based on the position and the position of the target object (10) on the image. [5] Object recognition device according to one of claims 2 to 4, which further comprises a determination unit (43) which, based on a position of the target object (10) and a position of the image acquisition unit (4) in real space, determines whether the image acquisition unit (4) and the target object (10) satisfy the predetermined position relationship or not, causes the robust detection unit (41) to recognize a position of the target object (10) on the image if the image acquisition unit (4) and the target object (10) do not satisfy the predetermined position relationship, and, on the other hand, causes the precision detection unit (42) to recognize a position of the target object (10) on the image if the image acquisition unit (4) and the target object (10) satisfy the predetermined position relationship. [6] Control device of an automatic machine (2) comprising at least one movable element (12 to 16), wherein the control device comprises: a storage device (33) which stores a template which represents a feature of the appearance of a target object (10) as the working target of the automatic machine (2) when the target object (10) is viewed from a predetermined direction; a robust detection unit (41) which, when an image acquisition unit (4) attached to the movable element (12 to 16), which captures the target object (10) and generates an image representing the target object (10), and the target object (10) do not satisfy a predefined positional relationship, detects a position of the target object (10) on the image by inputting the image into a classifier that has been previously trained to detect the target object (10) from the image, and detects a position of the target object (10) in a real space based on the position of the target object (10) on the image, wherein the predefined positional relationship is set such that a distance between the image acquisition unit (4) and the target object (10) is equal to or less than a predefined distance; a precision recognition unit (42) which, when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, recognizes a position of the target object (10) on the image by comparing the image with the template, recognizes a position of the target object (10) in a real space based on the target object (10) on the image, or recognizes a position of the target object (10) in a real space based on positional information that specifies a relative position with respect to the target object (10) and is obtained from a positional information acquisition unit that acquires the positional information; a proximity control unit (44) that controls the movable element (12 to 16) such that the image acquisition unit and the target object (10) fulfill the specified positional relationship, based on the position of the target object (10) in a real space, which is detected by the robust detection unit (41) when the image acquisition unit (4) and the target object (10) do not fulfill the specified positional relationship; and a work control unit (45) that controls the movable element (12 to 16) so that the movable element (12 to 16) is moved into a position in which the automatic machine (2) can perform predetermined work on the target object (10), based on the position of the target object (10) in a real space, which is detected by the precision recognition unit (42) when the image acquisition unit (4) and the target object (10) satisfy the predetermined positional relationship. [7] Control device according to claim 6, wherein the target object (10) assumes a predetermined position in relation to the image acquisition unit (4), the classifier was further trained to recognize a position of the target object (10) depicted in the image, and the robust detection unit (41) recognizes a position and a position of the target object (10) on the image by inputting the image into the classifier, and recognizes a position and a position of the target object (10) in a real space, based on the position and the position of the target object (10) on the image, and The proximity control unit (44) controls the movable element (12 to 16) such that the distance between the image acquisition unit (4) and the target object (10) is equal to or less than the specified distance, and the target object (10) assumes the specified position relative to the image acquisition unit, based on the position and orientation of the target object (10) in a real space as detected by the robust detection unit (41). [8] Object recognition computer program that causes a computer to perform the following steps: Detect when an image acquisition unit (4) capturing a target object (10) and generating an image representing the target object (10) and the target object (10) do not satisfy a predefined positional relationship, a position of the target object (10) on the image, by inputting the image into a classifier that has been previously trained to recognize the target object (10) from the image, wherein the predefined positional relationship is set such that a distance between the image acquisition unit (4) and the target object (10) is equal to or less than a predefined distance; and Detect when the image acquisition unit (4) and the target object (10) satisfy the specified positional relationship, a position of the target object (10) on the image, by comparing the image with a template that represents a feature of an appearance of the target object (10) when the target object (10) is viewed from a specified direction.

Images