Image processing system

Abstract

The object is to provide an image processing system that can accurately and efficiently detect an object even if the positional relationship between an imaging device and the object is different during teaching and detection. The image processing system (1) comprises: a control unit (52) that acquires the positional relationship between a visual sensor (4) and an object (W) based on positional information of a robot (2) for determining the position of the visual sensor (4) in the robot coordinate system and positional information indicating the position of the object (W) in the image coordinate system; and a storage unit (51) that stores the model pattern in the form of three-dimensional positional information based on a model pattern composed of feature points extracted from a teaching image and the positional relationship between the visual sensor (4) and the object (W) when the teaching image was captured. The control unit (52) performs the following detection processing: based on the result obtained by matching the feature points extracted from a detection image containing the object (W) with the model pattern, the object (W) is detected from the detection image.

Description

Technical Field

[0001] The present invention relates to an image processing system. Background Art

[0002] Conventionally, known techniques match the features of a two-dimensional input image captured by an imaging device with a pre-set model pattern serving as reference information. When the degree of match exceeds a predetermined level, the object is determined to be detectable. Examples of such methods include the generalized Hough transform. Patent documents 1 to 3 disclose image processing techniques for detecting objects from input images.

[0003] Patent Document 1 relates to a technique for three-dimensionally reconstructing edges whose intersection angles with respect to an epipolar line are nearly parallel. For example, paragraph 0014 of Patent Document 1 states, "The first edges e2 and e4 are edges located on the same plane, whose intersection angles with the epipolar line EP are within a predetermined angle range based on 90°, and are capable of high-precision three-dimensional reconstruction using a stereo method."

[0004] Patent Document 2 relates to a technology for measuring the distance of an object to be measured from a predetermined area using at least three imaging units that capture an image of the object to be measured via imaging lenses.

[0005] Patent Document 3 relates to a technology that reduces the influence of false image features extracted from an image region corresponding to a region where shadows fall, thereby improving the stability and accuracy of fitting / matching.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 2013-130508

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2009-002761

[0010] Patent Document 3: Japanese Patent Application Laid-Open No. 2012-042396 Summary of the Invention

[0011] Problems to be solved by the invention

[0012] However, when using a two-dimensional camera as the imaging device, detection results are expressed in an image coordinate system. Therefore, in order for the robot to operate on the detected object, the two-dimensional detection results expressed in the image coordinate system must be transformed into three-dimensional information expressed in the robot coordinate system. As a method for converting this into three-dimensional information, one method is to project the detection results observed in the image onto a virtual plane, based on the premise that the detection results exist on a specific plane.

[0013] However, if the positional relationship between the camera device and the object during teaching the model pattern is different from the positional relationship between the camera device and the object during detection, the size and shape of the object in the image will be different from the taught model pattern. In this state, even if the object is detected, it cannot be detected, or the detection of the object takes time. The reason why this is likely to happen is that the positional relationship changes when the camera device is moved by a robot, or when the object is moved by a robot, etc. Even if the camera device is fixed, the same problem will occur if the relative positional relationship between the object and the camera device changes due to the movement setting of the camera device, etc. The same problem also exists in the previous technology.

[0014] Solutions for solving problems

[0015] The image processing system involved in the present disclosure is used to detect the image of an object from an image captured by a camera device, wherein the positional relationship between the camera device and the object is changed by a robot, and the image processing system comprises: a control unit, which obtains the positional relationship between the camera device and the object based on position information of the robot used to determine the position of the camera device in the robot coordinate system and position information representing the position of the object in the image coordinate system; and a storage unit, which stores the model pattern in the form of three-dimensional position information based on a model pattern composed of feature points extracted from a teaching image and the positional relationship between the camera device and the object when the teaching image was captured, wherein the control unit performs the following detection processing: detecting the object from the detection image based on the result obtained by matching the feature points extracted from the detection image containing the object with the model pattern.

[0016] Effects of the Invention

[0017] According to the present disclosure, it is possible to provide an image processing system that can accurately and efficiently detect an object even if the positional relationship between an imaging device and the object differs between the time of teaching and the time of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram showing the configuration of an image processing system.

[0019] Figure 2 It is a diagram showing the configuration of a visual sensor control device and a robot control device.

[0020] Figure 3 FIG. 2 is a functional block diagram schematically showing functions related to image processing executed by the control unit.

[0021] Figure 4 is a diagram showing a model pattern composed of a plurality of feature points.

[0022] Figure 5 is a flow chart showing the process of making a model pattern.

[0023] Figure 6 FIG. 1 is a diagram showing a state where a model pattern designation area is designated in an image.

[0024] Figure 7 This is a flowchart showing a conversion process of a model pattern into a three-dimensional point.

[0025] Figure 8 Schematic diagram showing the relationship between the line of sight of a visual sensor as an imaging device and a calibration plane.

[0026] Figure 9 This is a flowchart showing the flow of the matching process in the first example.

[0027] Figure 10 This is a flowchart showing the flow of the matching process in the second example. DETAILED DESCRIPTION

[0028] Next, an example of an embodiment of the present invention will be described. Figure 1 1 is a diagram showing a configuration of an image processing system 1 according to an embodiment of the present invention. Figure 1 The illustrated image processing system 1 detects an image of an object W from an image captured by a vision sensor 4, serving as an imaging device. The positional relationship between the vision sensor 4 and the object W is changed by a robot 2. In the following description, the image coordinate system refers to a two-dimensional coordinate system defined on the image, and the sensor coordinate system refers to a three-dimensional coordinate system viewed from the perspective of the vision sensor 4. The robot coordinate system (machine coordinate system) refers to a three-dimensional coordinate system viewed from the perspective of the robot 2.

[0029] An image processing system 1 of this embodiment includes a robot 2, an arm 3, a vision sensor 4, a vision sensor control device 5, a robot control device 6, and an operation panel 7. The image processing system 1 recognizes the position of the object W based on an image of the object W captured by the vision sensor 4, and performs operations such as processing or machining the object W.

[0030] A hand or a tool is attached to the front end of the arm 3 of the robot 2 . The robot 2 performs operations such as handling or processing an object W under the control of a robot control device 6 . A visual sensor 4 is attached to the front end of the arm 3 of the robot 2 .

[0031] The vision sensor 4 is an imaging device that captures an image of the object W under the control of the vision sensor control device 5. The vision sensor 4 can be a two-dimensional camera or a stereo camera capable of three-dimensional measurement. The two-dimensional camera includes an imaging element composed of a CCD (Charge Coupled Device) image sensor and an optical system including lenses. In this embodiment, the vision sensor 4 is used to capture an image of the object W fixed to the workbench 8.

[0032] The robot control device 6 executes the operation program of the robot 2 to control the operation of the robot 2. As the robot 2 operates via the robot control device 6, the positional relationship of the visual sensor 4 with respect to the object W changes.

[0033] The operation panel 7 is a receiving unit for the user to perform various operations on the image processing system 1. The user inputs various commands to the visual sensor control device 5 through the operation panel 7.

[0034] Figure 2 1 is a diagram showing the configuration of the visual sensor control device 5 and the robot control device 6 . Figure 3 1 is a functional block diagram schematically showing functions related to image processing executed by the control unit. The visual sensor control device 5 of this embodiment includes a storage unit 51 and a control unit 52 .

[0035] The storage unit 51 is a storage device such as a ROM (Read Only Memory) that stores an OS (Operating System) and application programs, a RAM (Random Access Memory), a hard disk drive that stores other various information, or an SSD (Solid State Drive).

[0036] The storage unit 51 includes a model pattern storage unit 511 and a calibration data storage unit 512 .

[0037] The model pattern storage unit 511 will be described. The model pattern storage unit 511 stores a model pattern obtained by modeling the image of the object W. An example of the model pattern will be described later.

[0038] The calibration data storage unit 512 stores calibration data that correlates the robot coordinate system, which serves as a reference for motion control of the robot 2, with the image coordinate system, which serves as a reference for measurement processing performed by the vision sensor 4. Various methods have been proposed regarding the format of calibration data and methods for obtaining the calibration data, and any of these methods can be used.

[0039] The control unit 52 is a processor such as a CPU (Central Processing Unit), and is an image processing unit that executes various controls of the image processing system 1 .

[0040] Figure 3 FIG. 5 is a functional block diagram schematically showing functions related to image processing performed by the control unit 52. Figure 3 As shown, the control unit 52 includes a feature point extraction unit 521, a calibration unit 522, a determination unit 523, and a display processing unit 524 as functional units. These functional units of the control unit 52 function by executing programs stored in the storage unit 51.

[0041] The feature point extraction unit 521 extracts feature points from the input image captured by the visual sensor 4. Various methods can be used to extract feature points. In this embodiment, edge points are extracted as feature points. These edge points are points with a large brightness gradient in the image and can be used to obtain the contour shape of the object. Generally speaking, the brightness gradient of the image of the contour line of the object W is large, so the contour line shape of the object W can be obtained by using edge points as feature quantities. In addition, the extraction of edge points can also use a Sobel filter or a Canny edge detector.

[0042] The calibration unit 522 performs the following processing: based on the positional relationship between the vision sensor 4 and the object W and the calibration data stored in the calibration data storage unit 512, the positions of two-dimensional points in the image coordinate system are converted to the positions of three-dimensional points in the robot coordinate system. For example, when provided with data on a three-dimensional point in the robot coordinate system, the calibration unit 522 calculates the position of the image of the three-dimensional point in the image captured by the vision sensor 4, that is, calculates the two-dimensional point in the image coordinate system. Furthermore, when provided with data on a two-dimensional point in the image coordinate system, the calibration unit 522 calculates the line of sight in the robot coordinate system (world coordinate system). The line of sight here refers to a three-dimensional straight line passing through the gaze point and the focal point of the vision sensor 4. The gaze point is the three-dimensional point of the object W in the robot coordinate system (the three-dimensional position information of the feature point). Furthermore, the calibration unit 522 performs the following processing: based on the calculated line of sight, the calibration unit 522 converts the two-dimensional point data into data representing a three-dimensional position, that is, a three-dimensional point.

[0043] The determination unit 523 compares a feature point group (edge ​​point group) extracted from an input image acquired by the imaging device with the model pattern stored in the model pattern storage unit 511 , and detects an object based on the degree of matching.

[0044] The display processing unit 524 executes processing for displaying the determination result of the determination unit 523 and an operation screen for setting a correction plane (virtual plane) described later on the operation panel 7 .

[0045] The robot control device 6 includes a motion control unit 61 . The motion control unit 61 executes a motion program of the robot 2 based on a command from the vision sensor control device 5 , thereby controlling the motion of the robot 2 .

[0046] Next, generation of a model pattern in the image coordinate system by the feature point extraction unit 521 will be described. Figure 4 is a diagram showing a model pattern composed of a plurality of feature points. Figure 4 As shown in FIG. 1 , in this embodiment, a model pattern consisting of a plurality of feature points P_i is used. Figure 4 As shown, the model pattern is composed of a plurality of feature points P_i (i=1 to NP). In this example, the plurality of feature points P_i constituting the model pattern are stored in the model pattern storage unit 511 .

[0047] The position and posture of the feature points P_i that make up the model pattern can be expressed in any form. For example, the following method can be used: a coordinate system is defined for the model pattern (hereinafter referred to as the model pattern coordinate system), and the position and posture of the feature points P_i that make up the model pattern are expressed using position vectors, direction vectors, etc. as viewed from the model pattern coordinate system.

[0048] The origin O of the model pattern coordinate system can be arbitrarily defined. For example, any point can be selected from the feature points P_i constituting the model pattern and defined as the origin, or the centroid of all feature points P_i constituting the model pattern can be defined as the origin.

[0049] In addition, the posture (axis direction) of the model pattern coordinate system can also be defined arbitrarily. For example, it can be defined in such a way that the image coordinate system is parallel to the model pattern coordinate system in the image obtained by making the model pattern, or it can be defined by selecting any two points from the feature points that constitute the model pattern, defining the direction from one point to the other as the X-axis direction, and defining the direction orthogonal to the X-axis direction as the Y-axis direction. In addition, it is also possible to define it in such a way that the image coordinate system is parallel to the model pattern coordinate system in the image obtained by making the model pattern 50. In this way, the settings of the model pattern coordinate system and the origin O can be appropriately changed according to the situation.

[0050] Next, an example of creating a model pattern will be described. Figure 5 is a flow chart showing the process of making a model pattern. Figure 6 FIG. 1 is a diagram showing a state where a model pattern designation area is designated in an image.

[0051] An object W to be taught as a model pattern is placed within the field of view of the visual sensor 4, and an image of the object W is captured to obtain an input image (teaching image) containing the object W (step S101). At this time, it is preferable to process the image so that the positional relationship between the visual sensor 4 and the object W is the same as the positional relationship when the object W is detected during actual use.

[0052] Next, the area in the captured image where the object W is reflected is designated as the model pattern area (step S102). Hereinafter, the area designated in step S102 is referred to as the model pattern designated area 60. In this embodiment, the model pattern designated area 60 is designated using a rectangle or a circle to surround the object W. This model pattern designated area 60 can be stored in the model pattern storage unit 511 as operator-generated information.

[0053] Next, feature points are extracted (step S103 ). Feature points are the feature points that constitute the model pattern as described above. A plurality of feature points P_i (i=1 to NP) are extracted from the model pattern designation area 60 .

[0054] In step S103, the physical quantities of the edge points are calculated. These physical quantities include the position of the edge point, the direction of the brightness gradient, and the magnitude of the brightness gradient. If the direction of the brightness gradient of the edge point is defined as the posture of the feature point, the position and posture of the feature point can be defined in accordance with the position. The physical quantities of the edge point, namely the position, posture (direction of the brightness gradient), and magnitude of the brightness gradient, are stored as the physical quantities of the feature point.

[0055] Then, a model pattern coordinate system is defined within the designated model pattern designation area 60 , and based on the model pattern coordinate system and the origin O, the feature point P_i is expressed using the posture vector v_Pi, the position vector t_Pi, and the like.

[0056] Next, the model pattern 50 is generated based on the physical quantities of the extracted feature points P_i (step S104). In step S104, the physical quantities of the extracted feature points P_i are stored as the feature points P_i constituting the model pattern. These multiple feature points P_i become the model pattern. In this embodiment, a model pattern coordinate system is defined within the model pattern designation area 60, and the position and posture of the feature points P_i are converted from the image coordinate system (reference Figure 6 ) is expressed in the model pattern coordinate system (see Figure 4 ) to store the value represented by .

[0057] If correction of the model pattern 50 is necessary, the model pattern is corrected (step S105). The correction of the model pattern in step S105 is performed by the operator or the image processing unit 32. Alternatively, the correction can be performed automatically using machine learning or the like. Furthermore, if correction of the model pattern is not necessary, step S105 can be omitted. The series of processes described above creates a model pattern in the image coordinate system.

[0058] Next, the process of converting the model pattern into three-dimensional points will be described. A three-dimensional point is three-dimensional position information for specifying the three-dimensional position of a feature constituting the model pattern. Figure 7 This is a flowchart showing a conversion process of a model pattern into a three-dimensional point. Figure 8 Schematic diagram showing the relationship between the line of sight of the visual sensor 4 as an imaging device and the calibration plane.

[0059] The plane on which the model pattern is located is designated as a calibration plane (step S201). The calibration plane is a virtual plane. Various methods can be used to designate the calibration plane. For example, in a method in which a user uses a robot to modify the calibration plane as viewed from the robot coordinate system or the sensor coordinate system, the user sets the calibration plane using an operation panel 7 or the like. Furthermore, the calibration plane need not be a single plane and may be composed of multiple planes or curved surfaces.

[0060] Next, based on the calibration data of the visual sensor 4 and the position information of the robot 2 , a line of sight toward each feature point of the model pattern is acquired (step S202 ).

[0061] like Figure 8 As shown, the intersection point Pw of the calibration plane obtained in step S201 and the line of sight obtained in step S202 is obtained, and a three-dimensional point of the feature point is obtained based on the obtained intersection point Pw (step S203). The three-dimensional point is the three-dimensional position information of the feature point. The three-dimensional position information of the feature points constituting the model pattern is stored in the model pattern storage unit 511 as information for matching.

[0062] As described above, the image processing system 1 of this embodiment includes: a control unit 52 that acquires the positional relationship between the vision sensor 4 and the object W based on the positional information of the robot 2 used to determine the position of the vision sensor 4 in the robot coordinate system and the positional information indicating the position of the object W in the image coordinate system; and a storage unit 51 that stores the model pattern as three-dimensional positional information based on a model pattern composed of feature points extracted from a teaching image and the positional relationship between the vision sensor 4 and the object W when the teaching image was captured. The control unit 52 performs detection processing to detect the object W from the detection image based on the results of matching the feature points extracted from the detection image containing the object W with the model pattern. Thus, the matching processing is performed based on the model pattern stored as three-dimensional positional information. This avoids situations where the relative positional relationship between the vision sensor 4 and the object W differs between the teaching and detection phases. This avoids situations where the object W cannot be detected or detection of the object W takes time, enabling more accurate and efficient detection of the object W compared to conventional techniques.

[0063] Furthermore, in the image processing system of this embodiment, the three-dimensional positional information of the model pattern stored in the storage unit 51 is acquired based on the intersection of a calibration plane (a virtual plane) assumed to be where the feature points extracted from the teaching image lie, and the line of sight of the visual sensor 4 toward the feature points of the object W. Thus, since the three-dimensional positional information of the model pattern is acquired using the calibration plane (a virtual plane), detection processing can be performed more accurately using this calibration plane. Thus, in this embodiment, the three-dimensional positional information of each feature point (edge ​​point) constituting the model pattern is acquired by assuming that the detection portion of the object W lies on a certain plane (calibration plane).

[0064] Image processing system 1 of this embodiment detects an image of an object W from an input image containing the object to be detected based on the three-dimensional positional information of the feature points that constitute the model pattern. Several methods are conceivable as methods for detecting the object W using the three-dimensional positional information of the feature points that constitute the model pattern.

[0065] First, refer to Figure 9 , the matching process in the first example is explained. Figure 9 This is a flowchart showing the flow of the matching process in the first example.

[0066] In step S301, the control unit 52 acquires an input image captured by the visual sensor 4. The input image is a detection target image that includes the object W to be detected. The image to be matched is different from the teaching image used when generating the model pattern and is newly acquired by the visual sensor 4.

[0067] In step S302, the control unit 52 extracts edge points as feature points from the input image. The edge points can be extracted using the same method as described above.

[0068] In step S303, the control unit 52 obtains the positional relationship between the calibration plane and the visual sensor 4 during shooting. The calibration plane is the same virtual plane as the virtual plane set when generating the model pattern mentioned above. The positional relationship of the visual sensor 4 mentioned here is the positional relationship between the visual sensor 4 and the object W. The positional relationship of the visual sensor 4 is obtained based on the calibration data, the position information of the robot 2, the calibration plane, the position information of the object W in the image coordinate system, etc. For example, in the case where the visual sensor 4 is a two-dimensional camera, it is assumed that the feature point (edge ​​point) exists on the calibration plane, and the intersection of the line of sight toward the edge point and the calibration plane is obtained. In the case where the visual sensor 4 is a three-dimensional sensor, the information of the distance to the part where the edge point is located is obtained to obtain the three-dimensional position information.

[0069] In step S304, the edge points extracted from the input image (detection image) as feature points are projected onto the calibration plane, and the three-dimensional position information of the edge points, i.e., the three-dimensional points, is obtained. Thus, the data of the three-dimensional point group extracted from the input image as the matching target is obtained.

[0070] In step S305, a matching process is performed in which the three-dimensional points extracted from the input image (detection image) to be matched are compared with the three-dimensional points of the model pattern, thereby detecting the image of the object from the input image to be matched.

[0071] Thus, in the detection processing of the first example, the feature points of the detection image are acquired as three-dimensional position information based on the intersection of a calibration plane, serving as a virtual plane, where the feature points extracted from the detection image are assumed to lie, and the line of sight from the visual sensor 4 toward the feature points of the object W. Object W is then detected from the detection image based on the matching results of comparing the feature points of the detection image acquired as three-dimensional position information with feature points of a model pattern stored as three-dimensional position information. This first example allows the detection processing to accurately reflect the three-dimensional position information of the model pattern, and allows object W to be detected from the detection image based on an appropriate positional relationship. Furthermore, by incorporating three-dimensional rotation of the model pattern into the matching process, detection is possible even with three-dimensional posture changes, making it possible to cope with these three-dimensional posture changes.

[0072] Next, refer to Figure 10 , the matching process in the second example is explained. Figure 10This is a flowchart showing the flow of the matching process in the second example.

[0073] In step S401, the control unit 52 acquires an input image captured by the visual sensor 4. The input image is an image serving as a matching target and includes the object W to be detected. The image serving as a matching target is different from the input image used when generating the model pattern and is a newly acquired image by the visual sensor 4.

[0074] In step S402, feature points are extracted from the input image. The control unit 52 extracts edge points as feature points from the input image. The edge points can be extracted using the same method as described above.

[0075] In step S403 , the control unit 52 acquires the positional relationship between the calibration plane and the visual sensor 4 during imaging.

[0076] In step S404 , the control unit 52 executes a process of projecting the three-dimensional points of the model pattern onto the calibration plane, assuming that the object W is detected at an arbitrary position on the calibration plane.

[0077] In step S405, the control unit 52 matches the feature points of the input image in the image coordinate system with the feature points of the projected model pattern. Specifically, the two-dimensional feature points of the input image are matched with the feature points obtained by transforming the three-dimensional points of the model pattern into two dimensions to obtain a matching degree. The matching degree can be calculated using well-known techniques such as the aforementioned Hough transform.

[0078] In step S406, it is determined whether the termination condition is met, and the processes of steps S404 and S405 are repeated until the termination condition is met. If the process returns from step S406 to step S404, a different portion from the previous one is assumed to be an arbitrary portion, and the three-dimensional points of the model pattern are projected onto the calibration plane, after which the process of step S405 is executed. In this way, the image of the object W is detected based on the matching result with the highest degree of matching among the multiple matching results. Various termination conditions can be set, such as when a matching result with a high degree of matching is detected or when a predetermined time has elapsed.

[0079] Thus, in the detection process of the second example, projection and matching processes are repeatedly performed, and object W is detected from the detection image based on matching results with a high degree of matching. In the projection process, feature points extracted from the detection image are assumed to be detected at a certain location on a virtual plane. The model pattern is projected onto this virtual plane, and feature points in the image coordinate system are acquired. In the matching process, the feature points in the image coordinate system extracted from the detection image are compared with the feature points in the image coordinate system based on the model pattern acquired in the projection process. In this second example, the detection process can also accurately reflect the three-dimensional positional information of the model pattern, and object W can be detected from the detection image based on an appropriate positional relationship. Furthermore, by repeating the projection and matching processes, changes in appearance due to positional differences can be addressed, allowing matching to be performed in a two-dimensional image coordinate system.

[0080] While the embodiments of the present invention have been described above, the present invention is not limited to the aforementioned embodiments. In addition, the effects described in the present embodiments are merely examples of the most preferred effects produced by the present invention, and the effects of the present invention are not limited to those described in the present embodiments.

[0081] In the above embodiment, the positional relationship between the visual sensor 4, which serves as an imaging device, and the object W changes with the movement of the robot 2. However, this is not limiting. For example, the present invention can also be applied to a configuration in which the imaging device is fixed and the object moves with the movement of the robot. Specifically, a configuration in which the robot grasps the object and captures it with a fixed camera is possible. In this case, the object moves with the movement of the robot, causing the positional relationship between the imaging device and the object to change.

[0082] In the above embodiment, a model pattern composed of a plurality of edge points is described as an example, but the model pattern is not limited to this form. For example, feature points may be formed in units of pixels, and the model pattern may be formed in the form of an image.

[0083] Description of Reference Numerals

[0084] 1: Image processing system; 2: Robot; 4: Visual sensor (camera); 51: Storage unit; 52: Control unit.

Claims

1. An image processing system for detecting an image of an object from an image captured by an imaging device, wherein: The positional relationship between the imaging device and the object is changed by a robot, and the image processing system includes: a control unit configured to acquire a positional relationship between the imaging device and the object based on position information of the robot for determining a position of the imaging device in a robot coordinate system and position information indicating a position of the object in an image coordinate system; as well as a storage unit that stores the model pattern in the form of three-dimensional position information based on the model pattern composed of feature points extracted from the teaching image and the positional relationship between the camera and the object when the teaching image was captured, The control unit performs the following detection processing: detecting the object from the detection image based on a result obtained by matching feature points extracted from the detection image containing the object with the model pattern, In the detection process, projection processing and matching processing are repeatedly performed, and the object is detected from the detection image based on the matching process result with a high matching degree, In the projection process, it is assumed that the feature point extracted from the detection image is detected at a certain position on a predetermined virtual plane, the model pattern is projected onto the virtual plane, and the feature point in the image coordinate system is obtained. In the matching process, the feature points in the image coordinate system extracted from the detection image are compared with the feature points in the image coordinate system based on the model pattern acquired in the projection process.

2. An image processing system for detecting an image of an object from an image captured by an imaging device, wherein: The positional relationship between the imaging device and the object is changed by a robot, and the image processing system includes: a control unit configured to acquire a positional relationship between the imaging device and the object based on position information of the robot for determining a position of the imaging device in a robot coordinate system and position information indicating a position of the object in an image coordinate system; as well as a storage unit that stores the model pattern in the form of three-dimensional position information based on the model pattern composed of feature points extracted from the teaching image and the positional relationship between the camera and the object when the teaching image was captured, The control unit performs the following detection processing: detecting the object from the detection image based on a result obtained by matching feature points extracted from the detection image containing the object with the model pattern, In the detection process, The feature points of the detection image are acquired in the form of three-dimensional position information based on the intersection of a virtual plane assumed to be where the feature points extracted from the detection image are located and a line of sight from the camera device toward the feature points of the object, The object is detected from the detection image based on a matching processing result comparing the feature points of the detection image acquired in the form of the three-dimensional position information with feature points based on the model pattern stored in the form of the three-dimensional position information.

3. The image processing system according to claim 1 or 2, wherein: The three-dimensional position information of the model pattern stored in the storage unit is acquired based on an intersection of a virtual plane on which the feature points extracted from the teaching image are assumed to lie and a line of sight from the imaging device toward the feature points of the object.

Images