Image processing system

Abstract

Image processing system (1) that recognizes an image of an object (W) from an image taken by an imaging device (4) whose position relative to the object (W) is changed by a robot (2), wherein the image processing system (1) comprises: a controller (52) that acquires a positional relationship between the imaging device (4) and the object (W) based on positional information from the robot (2), which is used to identify the position of the imaging device (4) in a robot coordinate system, and positional information that specifies the position of the object (W) in an image coordinate system; and a memory (51) that stores a model pattern formed from feature points extracted from a training image in the form of three-dimensional position information based on the model pattern and a positional relationship between the imaging device (4) and the object (W) when the image was acquired for teaching, where the controller (52) executes a detection process for detecting the object (W) from a detected image containing the object (W), based on the result of the match between feature points extracted from the detected image and the model pattern, and wherein the detection process involves the repetition of a projection process in which the model pattern is projected onto an imaginary plane to capture the feature points in the image coordinate system, assuming that the feature points extracted from the captured image are detected in a certain part of the imaginary plane, and an adaptation process of comparing the feature points in the image coordinate system extracted from the detected image with the feature points in the image coordinate system based on the model pattern captured in the projection process, whereby the object (W) from the detected image is detected with a high degree of adaptation based on the result of the adaptation process.

Description

[0001] The present invention relates to an image processing system.

[0002] Conventionally, there is a known method for comparing feature values ​​of a two-dimensional input image, captured by an imaging device, with feature values ​​of a model pattern as preset standard information to determine whether an object is recognized if the degree of similarity exceeds a predetermined level. Examples of such a technique include the generalized Hankel-Fourier transform. Patent documents 1 to 3 disclose image processing techniques for recognizing an object from an input image.

[0003] Patent document 1 relates to a technique for the three-dimensional reconstruction of edges that intersect almost parallel to an epipolar line. For example, paragraph 0014 of patent document 1 describes “the first edges e2 and e4 are two of the edges that lie in the same plane and intersect the epipolar line EP within a predetermined angular range relative to 90 degrees. These edges can be three-dimensionally reconstructed with an accuracy using the stereo method.”

[0004] Patent document 2 relates to a technique for measuring a distance to a predetermined area of ​​a measuring object using at least three imaging devices, each of which takes an image of the measuring object via an imaging lens.

[0005] Patent document 3 relates to a technique for reducing the influence of pseudo-image features extracted from an image area corresponding to a shaded area, thereby improving the stability and accuracy of the fitting and matching.

[0006] See also patent documents 4-9. Patent document 1: JP 2013 - 130 508 A Patent document 2: JP 2009 - 002 761 A Patent document 3: JP 2012 - 042 396 A Patent document 4: DE 10 2016 013 274 A1 Patent document 5: EP 3 226 208 A1 Patent document 6: US 2017 / 0 256 046 A1 Patent document 7: US 2016 / 0 005 161 A1 Patent document 8: US 2014 / 0 067 317 A1 Patent document 9: US 2013 / 0 230 235 A1 Problems to be solved by the invention

[0007] When a two-dimensional camera is used as an imaging device, the captured image is represented in an image coordinate system. This requires a transformation of the two-dimensional image, represented in the image coordinate system, into three-dimensional information, represented in a robot coordinate system, so that a robot can work on the detected object. This transformation into three-dimensional information is achieved, for example, by a method of projecting the imaged detection onto an imaginary plane, assuming that the detection lies in a specific plane.

[0008] If the positional relationship between the imaging device and the object differs from the detected positional relationship between the imaging device and the object during model pattern training, the size and shape of the object in the image will deviate from those of the trained model pattern. Object detection in this state may fail or take time. This phenomenon typically occurs when the robot moves the imaging device or the object, changing their positional relationship. Even when the imaging device is fixed, similar problems arise if it is relocated and the relative positions of the object and the imaging device change. Existing techniques have similar issues. Means to solve the problems

[0009] The above problems are solved by an image processing system according to claims 1 and 2. Effects of the invention

[0010] The present disclosure provides an image processing system that is able to detect an object correctly and efficiently, even if a detected positional relationship between an imaging device and the object differs from a learned positional relationship between the imaging device and the object. Fig. 1 is a view showing a configuration of an image processing system; Fig. Figure 2 is a view showing configurations of an optical sensor control device and a robot control device; Fig. Figure 3 is a functional block diagram that schematically represents the functions performed by a control unit in connection with image processing; Fig. Figure 4 is a view showing a model pattern consisting of a large number of feature points; Fig. Figure 5 is a flowchart of a procedure for creating a model pattern; Fig. Figure 6 is a view showing a model pattern area specified in an image; Fig. Figure 7 is a flowchart of the conversion of a model pattern into three-dimensional points; Fig. Figure 8 is a diagram illustrating the relationship between a line of sight of an optical sensor as an imaging device and a compensation plane; Fig. Figure 9 is a flowchart of an adaptation process for a first example; and Fig. Figure 10 is a flowchart of an adaptation process for a second example.

[0011] One embodiment of the present invention is described below. Fig. Figure 1 is a diagram showing a configuration of an image processing system 1 according to an embodiment of the present invention. The diagram in Figure 1 shows a configuration of an image processing system 1 according to an embodiment of the present invention. Fig. The image processing system 1 depicted detects an image of an object W from an image captured by an optical sensor 4, which serves as an imaging device whose position relative to object W is changed by a robot 2. In the following description, an "image coordinate system" refers to a (two-dimensional) coordinate system defined in an image, and a "sensor coordinate system" refers to a (three-dimensional) coordinate system seen by the optical sensor 4. A "robot coordinate system" (mechanical coordinate system) is a (three-dimensional) coordinate system seen by the robot 2.

[0012] The image processing system 1 of the present embodiment comprises the robot 2, an arm 3, the optical sensor 4, an optical sensor control device 5, a robot control device 6 and a console 7. The image processing system 1 detects the position of the object W, for example on the basis of an image of the object W taken by the optical sensor 4, and handles or processes the object W.

[0013] A hand or tool is attached to the tip of the robot 2's arm 3. The robot 2 performs tasks such as handling and processing the object W under the control of the robot control device 6. The optical sensor 4 is also attached to the tip of the robot 2's arm 3.

[0014] The optical sensor 4 is an imaging device that captures an image of the object W under the control of the optical sensor control device 5. The optical sensor 4 can be a two-dimensional camera with an image acquisition device comprising a charge-coupled image sensor (CCD) and a lens, or a stereoscopic camera capable of performing three-dimensional measurements. In the present embodiment, the optical sensor 4 captures an image of the object W, which is mounted on a worktable 8.

[0015] The robot control device 6 executes an operating program of the robot 2 to control the operation of the robot 2. The robot control device 6 operates the robot 2, and the position of the optical sensor 4 relative to the object W changes.

[0016] Console 7 is a receiving unit through which a user operates the image processing system 1. The user inputs various types of commands to the optical sensor control device 5 via console 7.

[0017] Fig. Figure 2 is a view illustrating the configurations of the optical sensor control device 5 and the robot control device 6. Fig. Figure 3 is a functional block diagram that schematically illustrates the functions related to the image processing performed by a control unit. The optical sensor control device 5 of the present embodiment comprises a memory 51 and a controller 52.

[0018] Memory 51 is a storage device, such as a read-only memory (ROM) in which an operating system and application programs are stored, a working memory (RAM), a hard disk drive and a solid-state drive (SSD), which store different types of information.

[0019] Memory 51 comprises a model pattern memory 511 and a calibration data memory 512.

[0020] The model pattern memory 511 is described below. Model pattern memory 511 stores a model pattern that is a modeled representation of the object W. Examples of the model pattern are described later.

[0021] The calibration data storage unit 512 stores calibration data that links a robot coordinate system, which is a standard for controlling the operation of the robot 2, with an image coordinate system, which is a standard for measurement by the optical sensor 4. The calibration data can be in any format and calculated according to any suggested method.

[0022] The controller 52 is a processor, such as a central processing unit (CPU), and functions as an image processing unit that performs various types of control of the image processing system 1.

[0023] Fig. Figure 3 is a functional block diagram that schematically shows the functions related to the image processing performed by the controller 52. As shown in Fig. As shown in Figure 3, the controller 52 comprises, as functional units, a feature point extraction unit 521, a calibration unit 522, a determination unit 523, and a display processing unit 524. The functional units of the controller 52 operate when the program stored in memory 51 is executed.

[0024] The feature point extraction unit 521 extracts feature points from an input image captured by the optical sensor 4. Various methods can be used to extract the feature points. In the present embodiment, edge points are extracted as feature points, as they exhibit a large luminance gradient in the image and can be used to detect the contour shape of the object. An image of the contour line of object W generally has a large luminance gradient. Therefore, the contour shape of object W can be detected by using the edge points as feature points. A Sobel filter or a Canny edge detector can be used for the extraction of the edge points.

[0025] The calibration unit 522 performs a transformation between the positions of the two-dimensional points in the image coordinate system and the positions of the three-dimensional points in the robot coordinate system based on the positional relationship between the optical sensor 4 and the object W and the calibration data stored in the calibration data memory 512. For example, if data of the three-dimensional points in the robot coordinate system are given, the calibration unit 522 calculates the position of an image of the three-dimensional points in the image captured by the optical sensor 4, i.e., the two-dimensional points in the image coordinate system. Given the data of the two-dimensional points in the image coordinate system, the calibration unit 522 calculates a line of sight in the robot coordinate system (world coordinate system). The line of sight is a three-dimensional straight line passing through a viewpoint and a focus of the optical sensor 4.The viewpoint is a three-dimensional point of object W in the robot coordinate system (three-dimensional position information of the feature point). The calibration unit 522 performs a transformation of the two-dimensional point data into three-dimensional point data, i.e., data specifying the three-dimensional position based on the calculated line of sight.

[0026] The determination unit 523 compares a feature point cloud (an edge point cloud) extracted from the input image captured by the imaging device with the model pattern stored in the model pattern memory 511 and recognizes the object based on the degree of similarity between them.

[0027] The display processing unit 524 displays on console 7 the result of the determination by the determination unit 523 and an operating screen for setting a compensation level (imaginary level), which will be described later.

[0028] The robot control device 6 includes an operating controller 61. The operating controller 61 executes an operating program of the robot 2 in accordance with a command from the optical sensor control device 5 in order to control the operation of the robot 2.

[0029] The following describes how the feature point extraction unit 521 creates the model pattern in the image coordinate system. Fig. Figure 4 is a view showing the model pattern formed from a multitude of feature points. As in Fig. As shown in Figure 4, the model pattern used in the present embodiment is a model pattern formed from a plurality of feature points P_i. As shown in Figure 4, the model pattern used in the present embodiment is a model pattern formed from a plurality of feature points P_i. Fig. As shown in Figure 4, the model pattern is formed from a multitude of feature points P_i (i=1 to NP). In this example, the feature points P_i that form the model pattern are stored in the model pattern memory 511.

[0030] The position and orientation of the feature points P_i that form the model pattern can be represented in any form. For example, a coordinate system of the model pattern (hereinafter referred to as the "model pattern coordinate system") is defined, and the position and orientation of each of the feature points P_i forming the model pattern are represented by a position vector or a direction vector as viewed from the model pattern coordinate system.

[0031] The origin O of the model pattern coordinate system can be defined arbitrarily. For example, any point from the feature points P_i that form the model pattern can be defined as the origin, or the centroid of all feature points P_i that form the model pattern can be defined as the origin.

[0032] The position (axis direction) of the model pattern coordinate system can be defined as needed. For example, the position can be defined so that the image coordinate system and the model pattern coordinate system are parallel to each other in an image in which the model pattern is created. Alternatively, any two points can be selected from the feature points that form the model pattern, such that a direction from one of the two points to the other is defined as the X-axis direction and a direction orthogonal to the X-axis direction as the Y-axis direction. The position can be defined so that the image coordinate system and the model pattern coordinate system are parallel to each other in an image in which a model pattern 50 is created. Thus, the model pattern coordinate system and the origin O can be changed as needed.

[0033] The following is an example of how to create the model. Fig. Figure 5 is a flowchart of a procedure for creating the model pattern. Fig. Figure 6 is a view showing a model pattern area specified in an image.

[0034] First, the object W, which the user wants to train as a model pattern, is positioned within the field of view of the optical sensor 4 so that an image of the object W can be captured. In this way, an input image (a training image) is captured with the object W (step S101). In this step, the position of the optical sensor 4 relative to the object W is preferably the same as the position of the optical sensor 4 when the object W is detected in actual use.

[0035] Then, an area in which object W is found in the captured image is defined as the model pattern area (step S102). The area defined in step S102 is subsequently referred to as the model pattern specification area 60. The model pattern specification area 60 of the present embodiment is defined by a rectangle or a circle that surrounds object W. The model pattern specification area 60 can be stored as operator-created information in the model pattern memory 511.

[0036] Next, the feature points are extracted (step S103). As described above, the feature points form the model pattern. A multitude of feature points P_i (i=1 to NP) is extracted from the model pattern specification area 60.

[0037] In step S103, the physical quantities of the edge points are calculated. Examples of the physical quantities of each edge point are the position of the edge point, as well as the direction and magnitude of the edge point's luminance gradient. If the direction of the edge point's luminance gradient is defined as the location of the feature point, then the feature point's location can be defined together with its position. The physical quantities of the edge point—that is, the edge point's position, the location (direction of the luminance gradient) of the edge point, and the magnitude of the edge point's luminance gradient—are stored as the feature point's physical quantities.

[0038] Then, in the model pattern specification region 60, a model pattern coordinate system is defined and represented by position vectors v_Pi and position vectors t_Pi of the feature points P_i on the basis of the model pattern coordinate system and the origin O.

[0039] Then, the model pattern 50 is created based on the extracted physical quantities of the feature points P_i (step S104). In step S104, the extracted physical quantities of the feature points P_i are stored as the feature points P_i that constitute the model pattern. The feature points P_i form the model pattern. In the present embodiment, the model pattern coordinate system is defined in the model pattern specification area 60, and the position and location of the feature points P_i are stored as values ​​that are derived from the values ​​represented in the image coordinate system (see Fig. 6) into the values ​​represented in the model coordinate system (see Fig. 4) be transformed.

[0040] If necessary, the model pattern 50 is corrected (step S105). The correction of the model pattern in step S105 is performed by an operator or the image processing unit 32. Alternatively, the correction can also be performed automatically by machine learning. If the correction of the model pattern is unnecessary, step S105 can be skipped. The model pattern in the image coordinate system is generated by the series of processes described above.

[0041] The conversion of the model pattern into three-dimensional points is described below. These three-dimensional points represent three-dimensional positional information, identifying the three-dimensional positions of the feature points that make up the model pattern. Fig. Figure 7 is a flowchart of the conversion of the model pattern into three-dimensional points. Fig. Figure 8 is a diagram illustrating a relationship between the line of sight of the optical sensor 4 as an imaging device and a compensation plane.

[0042] A plane in which the model pattern exists is defined as the compensation plane (step S201). The compensation plane is an imaginary plane. The compensation plane can be defined by various methods. For example, the user defines the compensation plane via Console 7 by adding enhancements to the compensation plane as seen from the robot coordinate system or the sensor coordinate system using a robot. The compensation plane does not necessarily have to be a single plane and can also include two or more planes or curved surfaces.

[0043] The line of sight to each feature point of the model pattern is determined based on the calibration data of the optical sensor 4 and the position information of the robot 2 (step S202).

[0044] As in Fig. As shown in Figure 8, an intersection point Pw of the reconciliation plane captured in step S201 with the line of sight captured in step S202 is captured, and the three-dimensional points of the feature points are captured based on the captured intersection point Pw (step S203). The three-dimensional points constitute the three-dimensional position information of the feature points. The three-dimensional position information of the feature points, which forms the model pattern, is stored in the model pattern memory 511 as information used for matching.

[0045] As can be seen from the preceding description, the image processing system 1 of the present embodiment comprises: the controller 52, which acquires the positional relationship between the optical sensor 4 and the object W based on the positional information of the robot 2, which is used to identify the position of the optical sensor 4 in the robot coordinate system, and the positional information that specifies the position of the object W in the image coordinate system; and the memory 51, which stores the model pattern formed from the feature points extracted from an image for training in the form of three-dimensional positional information based on the model pattern and the positional relationship between the optical sensor 4 and the object W when the image for training was acquired.The controller 52 performs a recognition process to identify object W from a recognized image containing object W, based on the result of a comparison between the feature points extracted from the recognized image and the model pattern. The comparison is performed based on the model pattern, which is stored in the form of three-dimensional position information. This avoids situations where the learned relative positions of the optical sensor 4 and object W differ from the recognized relative positions. This prevents object W detection from failing or taking time, and allows for more accurate and efficient object W detection than known techniques.

[0046] The image processing system of the present embodiment acquires the three-dimensional position information of the model pattern stored in memory 51 based on the intersection of the compensation plane, which is the imaginary plane assumed to contain the feature points extracted from the image for teaching purposes, with the line of sight of the optical sensor 4 to the feature points of object W. This allows the acquisition of the three-dimensional position information of the model pattern using the compensation plane, which is the imaginary plane, and thus a more accurate acquisition using the compensation plane. In the present embodiment, it is assumed that object W is acquired on a specific plane (compensation plane). This assumption enables the acquisition of the three-dimensional position information of each of the feature points (edge ​​points) that form the model pattern.

[0047] The image processing system 1 of the present embodiment detects the image of object W from the input image containing the object to be detected, based on the three-dimensional position information of each of the feature points forming the model pattern. The detection of object W using the three-dimensional position information of each feature point forming the model pattern can be performed using a number of methods.

[0048] First, a comparison process of a first example is performed using... Fig. 9 described. Fig. Figure 9 is a flowchart of the matching process of the first example.

[0049] In step S301, the controller 52 acquires an input image captured by the optical sensor 4. The input image is a detected image of a matching target containing the object W to be recognized. The image of the matching target differs from the training image used to create the model pattern and is newly acquired by the optical sensor 4.

[0050] In step S302, the controller 52 extracts the edge points as feature points from the input image. The edge points can be extracted using the method described above.

[0051] In step S303, the controller 52 detects the compensation plane in which the image was captured and the position of the optical sensor 4. The compensation plane is an imaginary plane similar to the imaginary plane defined during the creation of the model pattern. The position of the optical sensor 4 mentioned here is its position relative to object W. The position of the optical sensor 4 is determined based on the calibration data, the position information of robot 2, the compensation plane, and the position information of object W in the image coordinate system. For example, if the optical sensor 4 is a two-dimensional camera, the intersection of the line of sight to the edge points with the compensation plane is detected, assuming that the feature points (edge ​​points) are present on the compensation plane.If the optical sensor 4 is a three-dimensional sensor, information about the distance to the position of the edge points is recorded in order to capture the three-dimensional position information.

[0052] In step S304, the edge points, which are the feature points extracted from the input image of the matching target (detected image), are projected onto the fit plane to capture three-dimensional points as three-dimensional position information for the edge points. In this way, the data of a three-dimensional point cloud is captured from the input image of the matching target.

[0053] In step S305, a matching process is performed; that is, the three-dimensional points extracted from the input image of the matching target (detected image) are compared with the three-dimensional points that form the model pattern. This process detects an image of the object from the input image of the matching target.

[0054] In the recognition process of the first example, the feature points in the captured image are detected in the form of three-dimensional position information based on the intersection of the compensation plane (the imaginary plane assumed to contain the feature points extracted from the captured image) with the line of sight of the optical sensor 4 in the direction of the feature points of object W. Then, object W is detected from the captured image based on the result of comparing the feature points in the captured image (captured in the form of three-dimensional position information) with the feature points based on the model pattern (stored in the form of three-dimensional position information).In the first example, the recognition process can accurately reproduce the three-dimensional positional information of the model pattern, enabling the detection of object W from the detected image based on the appropriate positional relationship. If three-dimensional rotation is added to the model pattern during the adaptation process, recognition can occur independently of three-dimensional changes in body posture. Thus, the first example can account for three-dimensional changes in body posture.

[0055] Next, a comparison procedure of a second example will be presented with reference to Fig. 10 described. Fig. Figure 10 is a flowchart of the matching process of the second example.

[0056] In step S401, the controller 52 acquires an input image captured by the optical sensor 4. The input image is an image of a matching target containing the object W to be detected. The image of the matching target differs from the input image used to create the model pattern and is newly acquired by the optical sensor 4.

[0057] In step S402, the feature points are extracted from the input image. Controller 52 extracts the edge points as feature points from the input image. The edge points can be extracted using the method described above.

[0058] In step S403, the controller 52 detects the compensation plane in which the image was captured and the position of the optical sensor 4.

[0059] In step S404, the controller 52 performs a process of projecting the three-dimensional points of the model pattern onto the compensation plane, assuming that the object W is detected at any part of the compensation plane.

[0060] In step S405, the controller 52 performs a comparison between the feature points in the input image in the image coordinate system and the feature points of the projected model pattern. Specifically, the two-dimensional points of the feature points in the input image are compared with the feature points obtained by a two-dimensional transformation of the three-dimensional points of the model pattern to determine the degree of similarity. The degree of similarity can be calculated using any known method, e.g., the Hankel-Fourier transform described above.

[0061] In step S406, it is determined whether a termination condition is met, and the processes of steps S404 and 405 are repeated until the termination condition is met. When the process returns from step S406 to step S404, the three-dimensional points of the model pattern are projected onto the compensation plane, assuming that the object W is detected at a different part than the last assumed part, and then the process of step S405 is executed. In this way, the image of object W is detected based on the matching result with the highest degree of match among the matching results. Various types of conditions can be specified as termination conditions, for example, that the detection result has a high degree of match or that a predetermined time has elapsed.

[0062] Therefore, the recognition process of the second example includes repeating the projection process, in which the model pattern is projected onto the imaginary plane to capture the feature points in the image coordinate system, assuming that the feature points extracted from the recognized image are captured in a certain part of the imaginary plane, and the fitting process, in which the feature points extracted from the recognized image in the image coordinate system are compared with the feature points in the image coordinate system based on the model pattern captured in the projection process, thereby capturing the object from the recognized image with a high degree of fit based on the result of the fitting process.In the second example, the recognition process can correctly reproduce the three-dimensional positional information of the model pattern, thus enabling the recognition of object W from the recognized image based on the appropriate positional relationship. In this example, the different representation of the object depending on the viewing position can be taken into account by repeating the projection process and the alignment. This allows the alignment to be performed in the two-dimensional coordinate system.

[0063] Embodiments of the present invention have just been described above, but the present invention is not limited to these exemplary embodiments. The advantages described in the exemplary embodiments are merely listed as the most suitable advantages of the present invention and do not limit the advantages of the present invention.

[0064] It was described in the embodiment that the position of the optical sensor 4 as an imaging device relative to the object W is changed by the robot 2, but the present invention is not limited to this configuration. The present invention is applicable, for example, to a configuration in which the imaging device is fixed and the object is moved by the operation of the robot. In particular, a stationary camera can capture an image of the object held by the robot. In this case, the object moves when the robot is operated, thereby changing the positional relationship between the imaging device and the object.

[0065] In the present embodiment, the model pattern formed from a plurality of edge points has been described as an example. However, the model pattern is not limited to this example. For instance, pixels can be considered as feature points, and the model pattern can be formed in an image format. EXPLANATION OF REFERENCE SYMBOLS 1 Image processing system 2 robots 4 Optical sensor (imaging device) 51 Storage 52 Controller

Claims

[1] Image processing system (1) that recognizes an image of an object (W) from an image taken by an imaging device (4) whose position relative to the object (W) is changed by a robot (2), wherein the image processing system (1) comprises: a controller (52) that acquires a positional relationship between the imaging device (4) and the object (W) based on positional information from the robot (2), which is used to identify the position of the imaging device (4) in a robot coordinate system, and positional information that specifies the position of the object (W) in an image coordinate system; and a memory (51) that stores a model pattern formed from feature points extracted from a training image in the form of three-dimensional position information based on the model pattern and a positional relationship between the imaging device (4) and the object (W) when the image was acquired for teaching, where the controller (52) executes a detection process for detecting the object (W) from a detected image containing the object (W), based on the result of the match between feature points extracted from the detected image and the model pattern, and wherein the detection process involves the repetition of a projection process in which the model pattern is projected onto an imaginary plane to capture the feature points in the image coordinate system, assuming that the feature points extracted from the captured image are detected in a certain part of the imaginary plane, and an adaptation process of comparing the feature points in the image coordinate system extracted from the detected image with the feature points in the image coordinate system based on the model pattern captured in the projection process, whereby the object (W) from the detected image is detected with a high degree of adaptation based on the result of the adaptation process. [2] Image processing system (1) that recognizes an image of an object (W) from an image taken by an imaging device (4) whose position relative to the object (W) is changed by a robot (2), wherein the image processing system (1) comprises: a controller (52) that acquires a positional relationship between the imaging device (4) and the object (W) based on positional information from the robot (2), which is used to identify the position of the imaging device (4) in a robot coordinate system, and positional information that specifies the position of the object (W) in an image coordinate system; and a memory (51) that stores a model pattern formed from feature points extracted from a training image in the form of three-dimensional position information based on the model pattern and a positional relationship between the imaging device (4) and the object (W) when the image was acquired for teaching, where the controller (52) executes a detection process for detecting the object (W) from a detected image containing the object (W), based on the result of the match between feature points extracted from the detected image and the model pattern, and where in the detection process, The feature points in the detected image are captured in the form of three-dimensional position information based on an intersection of an imaginary plane on which the feature points extracted from the detected image are assumed to be present. a line of sight from the imaging device (4) to the feature points of the object (W), and the object (W) from the captured image is identified based on the result of the comparison process of the feature points in the detected image, which was captured in the form of three-dimensional position information, with the feature points based on the model pattern, which is stored in the form of three-dimensional position information. [3] Image processing system (1) according to claim 1 or 2, wherein the three-dimensional position information of the model pattern stored in the memory is based on an intersection of an imaginary plane on which the features extracted from the image for teaching purposes are assumed to be present, with a line of sight from the imaging device (4) to the feature points of the object (W).

Images