Information processing method, information processing device, image processing method, image processing device, robot, article manufacturing method, program, and recording medium

Abstract

To improve the efficiency of matching processing.SOLUTION: An information processing apparatus comprises a processing unit. The processing unit acquires at least two pieces of first image data obtained by picking up an image of an object, performs first matching processing at least once on each of the pieces of first image data to acquire information on at least two first detection positions related to the position of the object, and determines whether each of the first detection positions is included in a predetermined range.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a matching processing technique. [Background technology]

[0002] A matching process such as that disclosed in Patent Document 1 is known as a method for recognizing the position of an object from a captured image of the object. Another known matching process of this type is a pattern matching process that searches the captured image for a contour that matches contour data. The contour data is generated from CAD data, for example. This pattern matching process detects the position of the object, and information on the detected position is output.

[0003] The appearance of the contour of an object in a captured image varies depending on the color or posture of the object. Therefore, the optimal setting values ​​for recognizing the object through matching processing vary depending on the object. The setting values ​​include, for example, the camera exposure time and matching parameters used in the matching processing. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-015895 Summary of the Invention [Problem to be solved by the invention]

[0005] Conventionally, when starting up a device that performs pattern matching processing, a user adjusts the settings of the device depending on the object. For example, the user would display both a captured image and a contour image matched to the captured image superimposed on a monitor or the like, determine whether the pattern matching processing was correct, and adjust the settings appropriately. Therefore, there has been a demand for automation of the adjustment of the settings.

[0006] On the other hand, Patent Document 1 discloses a method for determining the validity of matching, but the settings related to the matching process are required to be efficient, that is, to reduce the time required, in addition to accuracy.

[0007] An object of the present invention is to improve the efficiency of the matching process. [Means for solving the problem]

[0008] According to a first aspect of the present invention, there is provided an information processing device having a processing unit, characterized in that the processing unit acquires at least two first image data images of an object, performs at least one first matching process on each of the first image data, acquires information on at least two first detection positions related to the position of the object, and determines whether each of the first detection positions is within a predetermined range.

[0009] According to a second aspect of the present invention, there is provided an information processing method, characterized in that at least two image data images of an object are obtained, at least one matching process is performed on each of the image data to obtain information on at least two detection positions related to the position of the object, and it is determined whether each of the detection positions is included in a predetermined range.

[0010] According to a third aspect of the present invention, there is provided an image processing device having a processing unit, characterized in that the processing unit acquires at least two image data of an object, performs at least one matching process on each of the image data to acquire information on at least two detection positions related to the position of the object, and determines whether each of the detection positions is included in a predetermined range.

[0011] According to a fourth aspect of the present invention, there is provided an image processing method, characterized in that at least two sets of image data of an object are obtained, at least one matching process is performed on each of the image data to obtain information on at least two detection positions related to the position of the object, and it is determined whether each of the detection positions is within a predetermined range. [Effects of the Invention]

[0012] According to the present invention, the matching process can be made more efficient. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram of a robot system according to an embodiment. [Figure 2] FIG. 1 is a block diagram of a robot system according to an embodiment. [Figure 3] FIG. 2 is a block diagram of an image processing controller according to the embodiment. [Figure 4] FIG. 2 is a schematic diagram for explaining the operation of the robot system according to the embodiment. [Figure 5] FIG. 2 is an explanatory diagram showing an example of a user interface (UI) image according to the embodiment. [Figure 6] 10 is a flowchart of a setting process in a setting mode according to the embodiment. [Figure 7] 10 is a graph of a frequency distribution according to an embodiment. [Figure 8] 1A is a distribution diagram of detection positions according to an embodiment, and FIG. 1B is a frequency distribution diagram of residuals according to an embodiment. [Figure 9] 10A is a graph showing the relationship between exposure time and degree of match according to an embodiment, and FIG. 10B is a graph showing the relationship between the lower limit contrast and degree of match according to an embodiment. [Figure 10] 10(a) and 10(b) are graphs showing an example of a frequency distribution of the degree of match according to the embodiment. [Figure 11] 10A and 10B are explanatory diagrams showing an example of a UI image according to the embodiment. [Figure 12] 10 is a flowchart of an operation process in an operation mode according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described in detail with reference to the accompanying drawings. Fig. 1 is a schematic diagram of a robot system 1 according to an embodiment.

[0015] The robot system 1 includes a robot 30, a camera unit 41 which is an example of an imaging unit, a control panel 21 which is an example of a display unit, a main controller 20, an image processing controller 40, and a robot controller 50.

[0016] The robot 30 is a manipulator. The robot 30 of this embodiment is an industrial robot that can be used to hold an object. The robot 30 is installed on a production line and used to manufacture articles. A camera unit 41 is attached to the robot 30. A take-out box 10 and a put-in box 12 are arranged around the robot 30.

[0017] The robot system 1 of this embodiment is a picking device that picks out multiple parts 11 contained in a picking box 10 one by one and transfers them to a storage box 12. That is, in this embodiment, the robot 30 holds the parts 11 in the picking box 10 and operates the robot 30 to perform a transport task of transporting the parts 11 to the storage box 12. The parts 11 are an example of an object. Note that multiple picking boxes 10 may be prepared for different types of parts 11. In this case, the multiple picking boxes 10 may be sequentially replaced by a conveyor, an AGV, or the like. The storage box 12 has a partition plate provided inside and is configured to allow the placement position of the parts 11 to be changed depending on the type of part 11. As a result, the storage box 12 can be used to collect parts necessary for assembling a product.

[0018] The work of manufacturing an article is not limited to a transporting work, but may be, for example, an assembly work in which a first workpiece held by the robot 30 is assembled to a second workpiece, a processing work, or a coating work. The processing work may be, for example, a cutting work, a grinding work, a polishing work, or a sealing work.

[0019] The robot 30 has a robot arm 31 and a holding unit 32, which is an example of an end effector. An end effector according to the task is attached to the robot arm 31, and in this embodiment, the holding unit 32 is attached.

[0020] In this embodiment, the robot arm 31 is a vertically articulated robot arm. The base end (fixed end) of the robot arm 31 is installed on a pedestal (not shown). A holding unit 32 and a camera unit 41 are attached to the tip (free end), which is a predetermined location of the robot arm 31. The robot arm 31 is configured to represent the holding unit 32, the object held by the holding unit 32, and the camera unit 41 in a world coordinate system Σ W In this case, it can be adjusted to any position (including posture).

[0021] In this way, the holding unit 32, the object held by the holding unit 32, and the camera unit 41 are in the world coordinate system Σ W In other words, the holding unit 32, the object held by the holding unit 32, and the camera unit 41 are capable of movement with three degrees of freedom linearly along the X, Y, and Z axes in the world coordinate system, and three degrees of freedom rotationally along the A axis around the X axis, the B axis around the Y axis, and the C axis around the Z axis. The X, Y, and Z axes are perpendicular to each other.

[0022] The camera unit 41 is, for example, a digital camera, and includes a lens (not shown) and an image sensor (not shown). The image sensor is, for example, a CMOS image sensor or a CCD image sensor. The camera unit 41 may also include an illumination device, if necessary.

[0023] The camera unit 41 is configured to capture an image of an object present in the field of view, i.e., the imaging area, in accordance with setting information, and transmit image data obtained by this imaging, i.e., captured image data, to the image processing controller 40. The camera unit 41 of this embodiment is a two-dimensional camera, and is capable of capturing an image of an object and acquiring two-dimensional image data. Note that the camera unit 41 is not limited to a two-dimensional camera, and may be, for example, a three-dimensional camera.

[0024] The main controller 20 is a device that comprehensively controls the entire robot system 1. The image processing controller 40, the robot controller 50, and the operation panel 21 are connected to the main controller 20.

[0025] The operation panel 21 has a touch panel display 250 and functions as a display unit capable of displaying images and an input unit that the user can operate to input various pieces of information. In this embodiment, the operation panel 21 displays a user interface (UI) image UI1 as a graphical user interface (GUI) on the touch panel display 250 in accordance with a command from the main controller 20. That is, the main controller 20 accepts input operations from the user via the UI image UI1 displayed on the operation panel 21. Note that the operation panel 21 has a touch panel display 250 in which the display unit and the input unit are integrated, but this is not limiting. For example, the display unit and the input unit may be configured separately.

[0026] The holding unit 32 is a holding mechanism capable of holding the component 11, and for example, a suction hand, a vacuum suction hand, a clamping hand with fingers, or the like can be applied.

[0027] The robot controller 50 is capable of controlling the operation of the robot 30 , that is, the operation of the robot arm 31 and the holding unit 32 , in accordance with commands from the main controller 20 .

[0028] The image processing controller 40 can control the camera unit 41 according to commands from the main controller 20 and can perform image processing on image data obtained from the camera unit 41. Image processing includes processing of two-dimensional image data, synthesis processing, matching processing, and the like.

[0029] The matching process is a process of detecting the position (including the orientation) of the part 11 by matching image data (captured image data) obtained by capturing an image of the part 11 with reference image data. The reference image data is two-dimensional image data created based on three-dimensional shape data, for example, three-dimensional CAD data.

[0030] The position of the component 11 can be detected by searching the captured image data for a contour that matches a contour included in the reference image data. A plurality of matching parameters are used in the matching process. That is, the matching parameters are parameters used in the matching process. One of the plurality of matching parameters is the lower limit contrast. Another parameter is the lower limit degree of agreement.

[0031] The lower limit contrast is the minimum threshold for the difference in contrast between two adjacent pixels (picture elements) that are searched for as contours in the captured image data. The lower the lower limit contrast, the more pixels in the captured image data are searched for. This makes matching easier, but also increases the search time.

[0032] The lower limit of the matching degree is the lower limit threshold for determining whether a match has occurred between the captured image data and the reference image data. The matching degree is expressed as a value between 0 and 1, with 1 being reached when the contours contained in the reference image data and the contours found in the captured image data perfectly match. The lower the lower limit of the matching degree, the more likely there will be a match, but matching errors will also increase.

[0033] For the above reasons, it is desirable that the matching parameters used in the matching process be set appropriately according to the situation. Also, in the camera unit 41, the accuracy of extracting contours from captured image data changes depending on the exposure time. Therefore, it is desirable that the exposure time in the camera unit 41 be set appropriately according to the situation.

[0034] Fig. 2 is a block diagram of the robot system 1 according to the embodiment. Fig. 3 is a block diagram of the image processing controller 40 according to the embodiment.

[0035] 2, the main controller 20 is configured as a computer and has a CPU (Central Processing Unit) 201 as a processor.

[0036] The main controller 20 also has, as examples of storage units, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an HDD (Hard Disk Drive) 204. The main controller 20 also has a recording disk drive 205 and an input / output interface (I / O) 206. The ROM 202, RAM 203, HDD 204, recording disk drive 205, and I / O 206 are connected to the CPU 201 via a bus 210.

[0037] Basic programs such as BIOS are stored in the ROM 202. The RAM 203 is a storage device that temporarily stores various data such as the results of calculations performed by the CPU 201.

[0038] The HDD 204 is a storage device that stores the results of arithmetic processing by the CPU 201 and various data acquired from the outside. A program 230 for causing the CPU 201 to execute arithmetic processing is recorded on this HDD 204. The program 230 is application software. The CPU 201 executes each process of a part of an information processing method, i.e., a part of an image processing method, which will be described later, based on the program 230 recorded (stored) on the HDD 204. The recording disk drive 205 can read various data, programs, etc. recorded on the recording disk 220.

[0039] The I / O 206 is connected to the robot controller 50, the image processing controller 40, and the operation panel 21. The CPU 201 sends commands to each of the robot controller 50, the image processing controller 40, and the operation panel 21 via the I / O 206. The CPU 201 also acquires information from each of the robot controller 50, the image processing controller 40, and the operation panel 21 via the I / O 206. Note that a storage device such as a rewritable nonvolatile memory or an external HDD may be connectable to the I / O 206. Also, a network may be connectable to the I / O 206.

[0040] The HDD 204 is also a non-transitory recording medium readable by a computer. In this embodiment, the program 230 is stored in the HDD 204, but this is not a limitation. The program 230 may be recorded on any non-transitory recording medium readable by a computer. For example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, or a non-volatile memory can be used as a recording medium for providing the program 230. Examples of optical disks include disk media such as Blu-ray disks, DVDs, and CDs. Examples of non-volatile memory include storage devices such as USB memory, memory cards, ROMs, and SSDs. The program 230 may also be downloaded from a network.

[0041] 3, the image processing controller 40 is configured as a computer and has a CPU 401 as a processor.

[0042] The image processing controller 40 also has a ROM 402, a RAM 403, and a HDD 404 as examples of storage units. The image processing controller 40 also has a recording disk drive 405 and an input / output interface (I / O) 406. The ROM 402, RAM 403, HDD 404, recording disk drive 405, and I / O 406 are connected to the CPU 401 via a bus 410.

[0043] Basic programs such as BIOS are stored in the ROM 402. The RAM 403 is a storage device that temporarily stores various data such as the results of calculations performed by the CPU 401.

[0044] The HDD 404 is a storage device that stores the results of arithmetic processing by the CPU 401 and various data acquired from the outside. A program 430 for causing the CPU 401 to execute arithmetic processing is recorded on this HDD 404. The program 430 is application software. The CPU 401 executes each process of part of an information processing method, i.e., part of an image processing method, which will be described later, based on the program 430 recorded (stored) on the HDD 404. The recording disk drive 405 can read various data, programs, etc. recorded on the recording disk 420.

[0045] The main controller 20 and the camera unit 41 are connected to the I / O 406. The CPU 401 receives commands from the main controller 20 via the I / O 406. The CPU 401 then controls the camera unit 41 in accordance with the commands. The CPU 401 also acquires captured image data from the camera unit 41 via the I / O 406. The CPU 401 then executes image processing and sends information indicating the results of the image processing to the main controller 20. Note that a storage device such as a rewritable nonvolatile memory or an external HDD may be connectable to the I / O 406. A network may also be connectable to the I / O 406.

[0046] The HDD 404 is also a non-transitory recording medium readable by a computer. In this embodiment, the program 430 is stored in the HDD 404, but this is not a limitation. The program 430 may be recorded on any non-transitory recording medium readable by a computer. For example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, or a non-volatile memory can be used as a recording medium for providing the program 430. Examples of optical disks include disk media such as Blu-ray disks, DVDs, and CDs. Examples of non-volatile memory include storage devices such as USB memory, memory cards, ROMs, and SSDs. The program 430 may also be downloaded from a network.

[0047] The configurations of the main controller 20 and the image processing controller 40 have been described in detail above, but the robot controller 50 is also configured as a similar computer, so a detailed description will be omitted. That is, the robot controller 50 also has a processor such as a CPU and a storage device such as an HDD. The processor of the robot controller 50 operates based on a program to control the robot 30.

[0048] In this embodiment, the robot system 1 includes a control system 100 that functions as an information processing device (i.e., an image processing device). The control system 100 includes a main controller 20, an image processing controller 40, and a robot controller 50. That is, the control system 100 is configured with multiple computers. Therefore, in this embodiment, the CPU 201 executes the program 230, and the CPU 401 executes the program 430, so that the CPUs 201 and 401 function as processing units.

[0049] Although the main controller 20 and the image processing controller 40 are configured as separate computers, this is not limiting, and the main controller 20 and the image processing controller 40 may be configured as a single computer. Also, the main controller 20, the image processing controller 40, and the robot controller 50 may be configured as a single computer.

[0050] That is, the control system 100 may be configured with one or more computers. In this embodiment, the control system 100 is configured with three computers.

[0051] 1, a UI image UI1 is displayed on the touch panel display 250 in response to a command from the main controller 20. The CPU 201 of the main controller 20 is configured to be able to selectively execute a first mode or a second mode. The first mode is a setting mode in which setting information is set in the image processing controller 40. The second mode is an operation mode in which the CPU 401 of the image processing controller 40 executes image processing based on the setting information, and the robot 30 performs a manufacturing task, such as a transport task of transporting a component 11, based on the results of the image processing. The setting mode is executed when the robot system 1 is started up, for example.

[0052] The UI image UI1 includes a button B1 that causes the main controller 20 to execute the setting mode, and a button B2 that causes the main controller 20 to execute the operation mode. When the user operates button B1, the CPU 201 of the main controller 20 executes the setting mode, and when the user operates button B2, the CPU 201 of the main controller 20 executes the operation mode. This allows the user to select which mode the control system 100 (main controller 20) should execute by operating the UI image UI1.

[0053] First, a case where the setting mode is selected by the user will be described. FIG. 4 is a schematic diagram for explaining the operation of the robot system 1 according to the embodiment. Before executing the setting mode, only one component 11 is placed in the pick-up box 10. When the robot 30 moves the camera unit 41 and causes the camera unit 41 to capture images of the component 11 from various angles, it is sufficient that the component 11 is positioned within the imaging area of ​​the camera unit 41. For example, the component 11 may be placed in the center of the pick-up box 10.

[0054] When button B1 of UI image UI1 is operated, CPU 201 and CPU 401 execute processing corresponding to the setting mode. First, CPU 201 switches the display on the operation panel 21 from UI image UI1 shown in Fig. 4 to UI image UI2 shown in Fig. 5. Fig. 5 is an explanatory diagram showing an example of a UI image U12 according to the embodiment.

[0055] The CPU 201 loads the CAD data into the CPU 401 of the image processing controller 40. The CAD data is data necessary to determine the contour of the component 11 when performing matching processing, and is assumed to be stored in a storage device, such as the HDD 404, inside or outside the image processing controller 40. The user selects the CAD data of the component 11 in a file format using a dialog box or the like, and operates the load button B3 in FIG. 5 to load the CAD data into the CPU 401 of the image processing controller 40. The CAD data loaded into the CPU 401 is temporarily stored in the RAM 403. After the user loads the CAD data into the CPU 401, when the user operates the start button B4, the CPU 201 and CPU 401 execute the following setting process.

[0056] 6 is a flowchart of the setting process in the setting mode according to the embodiment. The setting process is roughly divided into an imaging step, a matching step, and a setting step, and is executed in this order. The order of the processes is not limited to this. It is also possible to improve efficiency by intermixing the processes and reducing the number of matching setting values ​​by determining a portion of the multiple setting values.

[0057] First, in step S21, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to cause the robot controller 50 to operate the robot 30 and change the imaging angle of the camera unit 41 relative to the component 11. At this time, it is preferable to move the camera unit 41 so that the component 11 is positioned at the center of the imaging area of ​​the camera unit 41.

[0058] The process of step S21 is repeatedly executed in a loop LA, and each time, the camera unit 41 is adjusted to a different imaging angle. It is preferable that the direction in which the camera unit 41 is tilted is a combination of the two directions indicated by the arrows in FIG. 4.

[0059] In step S22, the CPU 401 of the image processing controller 40 changes the exposure time T1 of the camera unit 41 when the camera unit 41 captures an image of the component 11. The processing of step S22 is repeatedly executed by a loop LB, and each time the camera unit 41 is adjusted to a different exposure time T1. The range over which the exposure time T1 is changed preferably includes a range from a short exposure time that causes crushed blacks in the captured image even when the component 11 is white to a long exposure time that causes blown-out highlights in the captured image even when the component 11 is black.

[0060] Next, in step S23, the CPU 401 of the image processing controller 40 causes the camera unit 41 to capture an image of the component 11 in the box 10. The camera unit 41 captures an image of the component 11 to generate captured image data I1 in which the component 11 is captured as an image. The captured image data I1 is transferred to the image processing controller 40 via wiring and stored in a storage device of the image processing controller 40, for example, the HDD 404. The multiple (at least two) captured image data I1 acquired in step S23 are defined as first image data. Furthermore, the exposure time T1 of the camera unit 41 in capturing the image in step S23 is defined as a first exposure time.

[0061] The imaging operation of step S23 is executed several hundred times in a double loop of loop LA and loop LB. As a result, a plurality of captured image data I1 obtained by the camera unit 41 capturing images of one and the same component 11 at various imaging angles and various exposure times T1 are acquired by the CPU 401 of the image processing controller 40 and stored in the HDD 404.

[0062] In this way, in steps S21 to S23, the CPU 401 causes the camera unit 41 to capture images of the component 11 at different exposure times T1 and different imaging angles, thereby acquiring a plurality of captured image data I1.

[0063] Note that when acquiring the plurality of captured image data I1, the CPU 401 may keep one of the exposure time T1 and the imaging angle constant. For example, the imaging angle may be kept constant. In this case, the exposure time T1 is changed in step S22. In this manner, the CPU 401 acquires the plurality of captured image data I1 by having the camera unit 41 capture images of the component 11 while varying the exposure time T1 and / or the imaging angle of the camera unit 41. In this case, it is preferable that the CPU 401 acquires the plurality of captured image data I1 by having the camera unit 41 capture images of the component 11 while varying at least the exposure time T1 of the exposure time T1 and the imaging angle of the camera unit 41.

[0064] Next, in step S31, the CPU 401 of the image processing controller 40 creates a matching model for performing matching processing. The matching model includes multiple reference image data. The reference image data is two-dimensional shape data (contour data) obtained by capturing images of the CAD data loaded in the UI image UI2 of FIG. 5 from various angles. The multiple reference image data (contour data) are stored as a matching model in, for example, the HDD 404.

[0065] Next, in step S32, the CPU 401 of the image processing controller 40 sequentially reads out the plurality of pieces of captured image data I1 stored in the HDD 404 from the HDD 404, and temporarily stores them in the RAM 403 or the like.

[0066] The process of step S32 is repeatedly executed by a loop LC, and each time, the captured image data I1 is read out sequentially from the plurality of captured image data I1 stored in the HDD 404. The process of step S32 is repeatedly executed until all of the plurality of captured image data I1 have been read out.

[0067] Next, in step S33, the CPU 401 of the image processing controller 40 changes the value of the lower limit contrast used when performing the matching process in the next step S34. The process of step S33 is repeatedly executed by a loop LD, and the lower limit contrast is adjusted to a different value each time.

[0068] Next, in step S34, the CPU 401 of the image processing controller 40 performs matching processing on the captured image data I1 read out in step S32 based on matching parameters PA1 including the lower limit contrast adjusted in step S33, to detect the position (including orientation) of the component 11. The matching parameters PA1 used in the matching processing in step S34 are an example of first matching parameters. That is, the lower limit contrast adjusted in step S33 is included in the first matching parameters. The CPU 401 determines coordinate values ​​in the camera coordinate system as information on the detected position of the component 11. The matching processing in this embodiment is pattern matching processing.

[0069] The matching process in step S34 is repeated hundreds to thousands of times by a double loop consisting of loop LC and loop LD. The matching process in step S34 uses the same algorithm as the matching process executed in the operation mode after setting the matching parameter PA2, but the matching process executed in step S34 in the setting mode is referred to as the first matching process to distinguish it from the matching process executed in the operation mode. The matching process executed in the operation mode is referred to as the second matching process.

[0070] The lower limit degree of matching used in the matching process in step S34 is set to a value equal to or greater than 0 and lower than a predetermined value. The multiple pieces of captured image data I1 may include captured image data in which matching errors may occur, such as captured image data in which black crush or white blowout has occurred, but even for such captured image data, a degree of matching can be output by setting the lower limit degree of matching to a low value.

[0071] Next, in step S35, the CPU 401 of the image processing controller 40 converts the coordinate values ​​of the part 11 defined in the camera coordinate system output in step S34 into coordinates in the world coordinate system Σ W Convert to coordinate values.

[0072] Here, in step S21, the component 11 is imaged while changing the imaging angle, so the position of the image of the component 11 captured in the captured image data I1 differs for each imaging angle. Therefore, the coordinate values ​​of the component 11 in the camera coordinate system also differ. Therefore, the CPU 401 of the image processing controller 40 converts the information on the detected position P1 of the component 11 into the world coordinate system Σ W The information on the detected position P1 of the component 11 includes information on the orientation. Therefore, the coordinate values ​​of the world coordinate system Σ calculated as the information on the detected position P1 of the component 11 are used. W The coordinate values ​​of are six-dimensional information. The multiple detection positions P1 detected in this way will have a dispersion of about several millimeters due to a mismatch of several pixels, the repeatability of the robot 30, an error in the calibration of the camera unit 41, and the like.

[0073] The multiple (at least two) detection positions P1 thus obtained in step S35 are defined as first detection positions. The CPU 401 performs matching processing at least once for each of the multiple captured image data I1. In this embodiment, the CPU 401 performs matching processing two or more times for each of the multiple captured image data I1 by using different matching parameters in step S33 in a loop LD. In this way, the CPU 401 acquires information on the multiple detection positions P1 related to the position of the target object.

[0074] Next, in step S41, the CPU 401 of the image processing controller 40 calculates the coordinates of the world coordinate system Σ obtained in step S35. W The position of the part 11 is estimated using the coordinate values ​​of

[0075] Fig. 7 is a graph of a frequency distribution according to an embodiment. The frequency distribution illustrated in Fig. 7 is a frequency distribution of only the X-axis coordinate values ​​among the coordinate values ​​of a part that are configured with six degrees of freedom, namely, the X-axis, Y-axis, Z-axis, A-axis, B-axis, and C-axis. To estimate the X-axis coordinate value of part 11, it is preferable to perform an arithmetic average excluding outliers in the frequency distribution.

[0076] Although the frequency distribution has a shape close to a normal distribution, it is a result obtained by varying conditions such as the imaging angle and exposure time T1. For this reason, a statistical method for removing outliers, such as the Smirnoff-Grubbs test, is not suitable. Therefore, in step S41, it is preferable for the CPU 101 to remove outliers from the multiple coordinate values ​​by calculating the interquartile range (IQR) from the multiple coordinate values.

[0077] In Figure 7, the median of the interquartile range is Q2. In Figure 7, one of the two black bars represents an outlier that exceeds the third quartile Q3 of the interquartile range plus 1.5 times the interquartile range (Q3 + 1.5 × IQR). The other black bar represents an outlier that falls below the first quartile Q1 of the interquartile range minus 1.5 times the interquartile range (Q1 - 1.5 × IQR). Each black bar represents a value that is removed as an outlier. The remaining values ​​can be arithmetically averaged to obtain an estimated value for the X-axis of the part 11. Note that while the case of obtaining an estimated value for the X-axis has been described, similar estimates can also be obtained for the Y-axis, Z-axis, A-axis, B-axis, and C-axis. This results in an estimated position P0 of the part 11 on six coordinates.

[0078] Next, in step S42, the CPU 401 of the image processing controller 40 determines a range R1 (that is, a predetermined range) that defines that the matching process has been performed correctly, based on the distribution of the detected positions P1 of the components 11 obtained in step S35.

[0079] Fig. 8(a) is a distribution diagram of detection positions P1 according to the embodiment. The coordinates shown in the distribution diagram in Fig. 8(a) are plotted with information about the detection positions P1 obtained in step S35. For convenience of explanation, Fig. 8(a) shows a distribution in which the detection positions P1 of the component 11 are plotted using coordinate values ​​on the X and Y axes on an XY graph using the X and Y axes. Note that the information about the detection positions P1 obtained in step S35 includes coordinate values ​​on the X, Y, Z, A, B, and C axes.

[0080] Each point plotted on the XY graph is the detected position P1 of the component 11 detected by one matching process from one captured image data I1. That is, by performing the matching process two or more times with different matching parameters PA1 for each captured image data I1 using the loop LD shown in Fig. 6, information on two or more detection positions P1 is obtained.

[0081] For example, let ND be the number of loops LD and NC be the number of captured image data I1. The number NC of captured image data I1 is determined by the number of loops LA and LB. Note that a loop count of 1 means that the process does not return, but in this embodiment, the number of loops is greater than 1. If NA is the number of loops LA and NB is the number of loops LB, then the number NC of captured image data I1 is NA×NB. Since ND matching processes are performed for one captured image data I1, NC×ND (=NA×NB×ND) matching processes are performed for NC captured image data I1. As a result, the number of pieces of information about the detected position P1 of the component 11 is NC×ND (=NA×NB×ND). In this way, information about the detected position P1 of the component 11 is obtained the same number of times as the matching process is performed, resulting in information about multiple detected positions P1 being obtained. The information about the multiple detected positions P1 for the number of times the matching process is performed is plotted as circles, as shown in FIG. 8(a).

[0082] The white circle plotted in the center of the XY graph in Fig. 8(a) is the estimated position P0 of the part 11 estimated in step S41. A detected position P1 that is far from the estimated position P0 is determined to be an incorrect match.

[0083] In this embodiment, the distance between each detected position P1 and the estimated position P0 is expressed as a residual r i The residual r i is calculated using the following formula (1):

number

[0084] FIG. 8(b) shows the residual r i The frequency distribution of residual r i Since is a value greater than or equal to 0, the distribution has a tail on one side, as shown in Figure 8(b). In this case, the method of removing outliers using the interquartile range is not suitable. However, if the idea of ​​the interquartile range is applied to a distribution with a tail on one side, the residual r exceeds four times the median value M of the residual. i can be judged as an outlier.

[0085] In Fig. 8(a), the black circles plotted on the XY graph represent the residual r i The detected position P1 is outside the range R1, exceeding 4M. The detected position P1 indicated by the black circle is an outlier far from the estimated position P0 even when variance is taken into consideration, and can therefore be determined to be an incorrect match.

[0086] On the other hand, the gray circles plotted on the XY graph in Figure 8(a) represent the residual r i The detected position P1 is within the range R1 of 4M. The detected position P1 indicated by the gray circle is close to the estimated position P0 even when variance is taken into account, so it can be determined to be a correct match. i A set of detection positions P1 included in a range R1 within 4M is called a matching set.

[0087] As described above, in step S42, the CPU 401 determines a circular range R1 as shown in FIG. 8A as a predetermined range based on the distribution of the plurality of detection positions P1. The CPU 401 then determines whether each of the plurality of detection positions P1 is included in the range R1. As a result, the information on the plurality of detection positions P1 is classified into information on detection positions P1 that are included in the range R1, indicated by gray circles, and information on detection positions P1 that are outside the range R1, indicated by black circles.

[0088] The accuracy of the above matching may be determined using only two-dimensional information on the X and Y axes, or six-dimensional information on the X, Y, Z, A, B, and C axes, or only one-dimensional information on the X axis. That is, the accuracy of the matching can be determined using at least one-dimensional information from the six-dimensional information. However, since the matching process in this embodiment uses two-dimensional image data, it is preferable to use two-dimensional information on two axes corresponding to the image data, i.e., the X and Y axes.

[0089] Next, in step S43, the CPU 401 of the image processing controller 40 determines the exposure time T2 to be set using the matching set determined in step S42. The exposure time T2 is the exposure time when the camera unit 41 captures an image in the operation mode, and is referred to as the second exposure time to distinguish it from the exposure time T1 of the camera unit 41 in step S23.

[0090] Here, in step S22, the component 11 is imaged by changing the exposure time T1 to obtain multiple captured image data I1, so the matching set contains information on the detection position P1 corresponding to the captured image data I1 captured at various exposure times T1.

[0091] 9A is a graph showing the relationship between the exposure time T1 and the degree of match according to the embodiment. The degree of match is the degree of match between the captured image data I1 and the reference image data calculated when determining the detection position P1 included in the matching set.

[0092] If the exposure time T1 is short, the captured image data I1 will be overexposed, significantly reducing the calculated degree of match. If the exposure time T1 is long, the captured image data I1 will be overexposed, significantly reducing the calculated degree of match. Therefore, in FIG. 9(a), it is preferable to determine the exposure time T1 that maximizes the degree of match as the exposure time T2 to be used in the operation mode. In this way, the exposure time T2 to be used in the operation mode is determined using only the exposure time T1 corresponding to the detection position P1 included in the matching set determined to be a correct match. By determining the exposure time T2 in this manner, the camera unit 41 captures an image of the component 11 at the exposure time T2 in the operation mode, thereby improving the accuracy of the matching process in the operation mode.

[0093] Next, in steps S44 and S45, the CPU 401 of the image processing controller 40 determines a matching parameter PA2 using the matching set determined in step S42. The matching parameter PA2 is a matching parameter used in the matching process in the operating mode, and is designated as a second matching parameter to distinguish it from the matching parameter PA1 used in the matching process in step S34. In this embodiment, the matching parameter PA2 is determined to be a lower limit contrast PA21 shown in the example of FIG. 9(b) and a lower limit agreement PA22 shown in the examples of FIGS. 10(a) and 10(b).

[0094] First, in step S44, the CPU 401 of the image processing controller 40 determines the lower limit contrast PA21 using the matching set determined in step S42.

[0095] Since the matching process is performed by changing the lower limit contrast PA11 in step S33, the matching set includes information on the detection position P1 obtained by the matching process with various lower limit contrasts PA11.

[0096] 9B is a graph showing the relationship between the lower limit contrast PA11 and the degree of match according to the embodiment. The degree of match is the degree of match between the captured image data I1 and the reference image data calculated when determining the detection position P1 included in the matching set.

[0097] 9(b), it is preferable to determine the lower limit contrast PA11 with the highest degree of match as the lower limit contrast PA21 to be used in the operation mode. In this way, the lower limit contrast PA21 to be used in the operation mode is determined using only the lower limit contrast PA11 corresponding to the detection position P11 included in the matching set determined to be a correct match. By determining the lower limit contrast PA21 in this way, the CPU 401 performs the matching process in the operation mode using this lower limit contrast PA21, thereby improving the accuracy of the matching process in the operation mode.

[0098] Next, in step S45, the CPU 401 of the image processing controller 40 determines the lower limit matching score PA22 shown in the example of Fig. 10(a) or 10(b) using at least the matching set determined in step S42. The lower limit matching score PA22 is a threshold value for determining that a match has been made correctly.

[0099] Therefore, it is preferable that the lower limit matching score PA22 be determined based on the relationship between the matching score PA12 corresponding to the information on the detection position P1 included in the matching set and the matching score PA12 corresponding to the information on the detection position P1 not included in the matching set. Because the matching set is obtained in step S42, the lower limit matching score PA22 can be easily determined.

[0100] 10(a) and 10(b) are graphs showing an example of a frequency distribution of matching degrees according to an embodiment. In FIGS. 10(a) and 10(b), the frequency of matching degrees PA12 corresponding to detection positions P1 included in the matching set and the frequency of matching degrees PA12 corresponding to detection positions P1 not included in the matching set are separately shown. In FIGS. 10(a) and 10(b), "correct" refers to the frequency of matching degrees PA12 corresponding to detection positions P1 included in the matching set, and "incorrect" refers to the frequency of matching degrees PA12 corresponding to detection positions P1 not included in the matching set.

[0101] As shown in FIG. 10(a), when the maximum value of the degree of agreement for "incorrect" is smaller than the minimum value of the degree of agreement for "correct," the intermediate value between the minimum value of the degree of agreement for "correct" and the maximum value of the degree of agreement for "incorrect" is determined as the lower limit degree of agreement PA22. Also, as shown in FIG. 10(b), when the maximum value of the degree of agreement for "incorrect" is greater than the minimum value of the degree of agreement for "correct," a value slightly greater than the maximum value of the degree of agreement for "incorrect" is determined as the lower limit degree of agreement PA22. That is, in either example shown in FIG. 10(a) or FIG. 10(b), the lower limit degree of agreement PA22 is determined to be a value greater than the maximum value of the degree of agreement for "incorrect." This allows for more accurate matching processing in the operating mode.

[0102] While the determination of the lower limit contrast PA21 and the lower limit matching degree PA22 has been described above, other matching parameters may also be determined in a similar manner to the lower limit contrast. For example, this can be applied when determining the pyramid level at which matching processing is performed at each stage by changing the resolution of the captured image data, or the matching algorithm that affects the matching accuracy and processing time.

[0103] As described above, in step S43, the CPU 401 determines the exposure time T2 of the camera unit 41 to be used in the operation mode based on the results of the matching process performed multiple times by the loops LC and LD. Specifically, the CPU 401 determines the exposure time T2 of the camera unit 41 to be used in the operation mode based on the exposure time T1 of the camera unit 41 when capturing the captured image data I1 used in the matching process performed when acquiring information on the detection position P1 included in the range R1.

[0104] Furthermore, in steps S44 and S45, the CPU 401 determines a matching parameter PA2 to be used in the matching process in the operation mode based on the results of the matching process performed multiple times by the loops LC and LD. Specifically, in step S44, the CPU 401 determines a lower-limit contrast PA21 to be used in the matching process in the operation mode based on the lower-limit contrast PA11 used in the matching process performed when information on the detection position P1 included in the range R1 was acquired. Furthermore, in step S45, the CPU 401 determines a lower-limit matching degree PA22 to be used in the matching process in the operation mode based on the matching degree PA12 calculated in the matching process performed when information on the detection position P1 included in the range R1 was acquired and the matching degree PA12 calculated in the matching process performed when information on the detection position P1 outside the range R1 was acquired.

[0105] The information on the exposure time T2, the information on the matching parameters PA2 (PA21, PA22), and the distribution information are transmitted to the CPU 201 of the main controller 20.

[0106] Next, in step S46, the CPU 201 of the main controller 20 displays a UI image UI3 shown in Fig. 11(a) on the operation panel 21. Fig. 11(a) is an explanatory diagram showing an example of a UI image UI3 according to the embodiment.

[0107] The CPU 201 of the main controller 20 displays the setting values, i.e., the numerical value of the exposure time T2 and the numerical values ​​of the matching parameters PA2 including the lower limit contrast PA21 and the lower limit coincidence PA22, on the UI image UI3. In this way, the CPU 201 displays information about the exposure time T2 and information about the matching parameters PA2 on the UI image UI3.

[0108] The UI image UI3 includes a save button B5 and a distribution display button B6. Each of the buttons B5 and B6 is a button that accepts user operations. The save button B5 is a button that sends a command to the CPU 201 and the CPU 401 to save information about the exposure time T2 and information about the matching parameter PA2 in the HDD 404.

[0109] When the user operates the save button B5, the CPU 201 of the main controller 20 displays a dialog box (not shown) on the operation panel 21 for selecting a file stored in, for example, the HDD 404. When the user selects a file in which setting values ​​corresponding to the component 11 are recorded, the CPU 201 sends a command to the CPU 401 of the image processing controller 40 to cause the CPU 401 to overwrite and save the determined setting values ​​in the selected file. In this way, the CPU 401 sets the exposure time T2 and the matching parameter PA2 by saving information about the determined exposure time T2 and information about the matching parameter PA2 in the HDD 404.

[0110] When the distribution display button B6 is operated by the user, the CPU 201 of the main controller 20 displays the UI image UI4 shown in FIG. 11(b). FIG. 11(b) is an explanatory diagram showing an example of the UI image UI4 according to the embodiment. In the UI image UI4, a distribution map B7 corresponding to the distribution map of FIG. 8(a) is displayed. The distribution map B7 is an example of distribution information. By displaying the distribution map B7, the user can confirm whether the setting mode has been executed normally. In addition, the user can change the range R within which a match is determined to have occurred by inputting a value into the box B8. If the range R is changed, the process is repeated from step S41.

[0111] Next, a case where an operation mode is selected by the user will be described. When button B2 in Fig. 1 is operated, CPU201 and CPU401 execute processing corresponding to the operation mode. Fig. 12 is a flowchart of the operation processing in the operation mode according to the embodiment.

[0112] In step S11, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to operate the robot 30, and moves the camera unit 41 attached to the robot 30 above the pick-up box 10.

[0113] It is preferable that the CPU 201 of the main controller 20 acquires information on the posture of the robot 30 before the robot 30 operates, and sets waypoints depending on the situation so as to prevent interference with the storage box 12 or the pick-up box 10.

[0114] Next, in step S12, the CPU 401 of the image processing controller 40 causes the camera unit 41 to capture an image of the parts 11 in the box 10. The captured image data I2 generated in the camera unit 41 is transferred to the image processing controller 40 via wiring, as shown in Fig. 1. The captured image data I2 acquired in step S12 is referred to as second image data to distinguish it from the captured image data I1 acquired in step S23.

[0115] If the camera unit 41 is equipped with a lighting device, the camera unit 41 turns on the lighting device before capturing an image and turns off the lighting device after capturing an image. The exposure time when the camera unit 41 captures an image is the exposure time T2 set in the setting process described above.

[0116] Next, in step S13, the CPU 401 of the image processing controller 40 performs a matching process on the captured image data I2 obtained by capturing the image in step S12 to detect the position (including the orientation) of the component 11, and outputs information on the detected position P2 to the main controller 20. The matching parameters including the lower limit contrast and the lower limit degree of agreement used in the matching process in step S13 are matching parameters PA2 including the lower limit contrast PA21 and the lower limit degree of agreement PA22 set in the setting process described above.

[0117] The matching process executed in step S13 is referred to as the second matching process, which is distinguished from the matching process executed in step S34. Furthermore, the information on the detected position P2 acquired in step S13 is referred to as the second detected position information, which is distinguished from the information on the detected position P1 acquired in step S35.

[0118] When there are multiple components 11 in the container 10, the matching process is performed to obtain multiple pieces of detection position information and the degree of match corresponding to each piece of detection position information from the captured image data. Therefore, it is preferable that the CPU 401 of the image processing controller 40 sorts the information on the detection positions of the components 11 in descending order of degree of match and outputs the information on the detection positions of the components 11.

[0119] Next, in step S14, the CPU 201 of the main controller 20 calculates the posture of the robot 30 for orienting the holding unit 32 directly toward the component 11 to be picked up, based on the information on the detected position P2 of the component 11 acquired from the CPU 401. The positional relationship between the camera unit 41 and the holding unit 32 in the flange coordinate system is determined in advance by calibration or the like. If there are multiple components 11 in the pick-up box 10, the CPU 201 acquires information on multiple detection positions, and therefore it is preferable to select the component 11 to be picked up by prioritizing the components so as not to interfere with the pick-up box 10 or surrounding components 11.

[0120] Next, in step S15, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to operate the robot 30 to the posture calculated in step S14, and move the holding unit 32 to a position where the holding unit 32 can hold the component 11. At this time, it is preferable to operate the robot 30 by setting appropriate waypoints so as not to interfere with the delivery box 10 or surrounding components 11.

[0121] Next, in step S16, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to make the holding unit 32 hold the component 11.

[0122] Next, in step S17, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to operate the robot 30 and move the component 11 to the storage box 12. If the storage box 12 has a partition, the position where the component 11 is placed may be changed depending on the type of component 11, the number of times it has been taken out, etc. When moving the component 11, it is preferable to set appropriate waypoints so that the robot 30 and the component 11 do not interfere with the pick-up box 10 or the storage box 12.

[0123] Next, in step S18, the CPU 201 of the main controller 20 sends a command to the robot controller 50 to cause the holding unit 32 to release the part 11 and place the part 11 in the placement box 12.

[0124] As described above, according to this embodiment, the setting work corresponding to the object that was previously performed manually and took time can be performed automatically and efficiently, thereby reducing the time required for setting up the matching process. The greater the number of types of parts to be handled, the greater the effect, making it possible to significantly reduce the number of steps required by the user and the time the user occupies the device.

[0125] Furthermore, according to this embodiment, the coordinate values ​​of correctly matched parts 11 are densely packed in the distribution and fall within range R1. The coordinate values ​​of incorrectly matched parts 11 are scattered in the distribution and fall outside range R1. Therefore, by determining range R1 based on the distribution of multiple detection positions P1, it is possible to determine whether the matching is correct. Furthermore, when the robot system 1 is started up, optimal setting values ​​can be automatically determined for each type of part 11, which reduces the user's workload.

[0126] Furthermore, according to this embodiment, a plurality of captured image data I1 are obtained by capturing images of the component 11 with different exposure times and / or different imaging angles. Then, information on a plurality of detection positions P1 is obtained from the plurality of captured image data I1. The information on the plurality of detection positions P1 is calculated in the world coordinate system Σ W Since the coordinate system is unified, the distribution can be obtained with high accuracy, and the range R1 can be set appropriately.

[0127] Furthermore, according to this embodiment, the robot 30 can rotate the camera unit 41 to capture images of the component 11 from multiple imaging angles. This eliminates the need to prepare multiple camera units, which simplifies the robot system 1. Furthermore, by using the camera unit 41 in common in both the setting mode and the operation mode, the robot system 1 can be simplified.

[0128] Furthermore, according to this embodiment, the exposure time T2 and the matching parameter PA2 are determined based on the results of correct matching, and therefore the accuracy of the matching process is improved in the operation mode.

[0129] The present invention is not limited to the above-described embodiments, and many modifications are possible within the technical concept of the present invention. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0130] In the above embodiment, a case has been described in which the matching parameter PA1 is changed for each piece of captured image data I1 in the setting mode and the matching process is performed multiple times, but this is not limited to this. For example, there may be cases in which the matching parameter PA2 has already been set and does not need to be changed, but the exposure time T2 needs to be set. In such a case, since there is no need to change the matching parameter PA1, it is sufficient to perform the matching process once for each piece of captured image data I1.

[0131] There may also be cases where the exposure time T2 has already been set and it is necessary to set the matching parameter PA2. In such cases, there is no need to vary the exposure time T2, so when obtaining multiple captured image data I1, it is sufficient to keep the exposure time constant and change the imaging angle.

[0132] In the above embodiment, the robot arm 31 is a vertically articulated robot arm, but the present invention is not limited to this. The robot arm may be, for example, a horizontally articulated robot arm, a parallel link robot arm, an orthogonal robot arm, or the like.

[0133] Furthermore, although the above embodiment has been described using a robot, the present invention is not limited to this. For example, the present invention can also be applied to a machine that can automatically perform movements such as extension and contraction, bending and stretching, vertical movement, horizontal movement, or rotation, or a combination of these movements, based on information stored in a storage device provided in a control device.

[0134] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0135] The disclosure of the above embodiments includes the following configurations and methods.

[0136] (Configuration 1) An information processing device including a processing unit, The processing unit acquiring at least two first image data images of an object; performing at least one first matching process on each of the first image data to obtain information on at least two first detection positions related to the position of the object; determining whether each of the first detection positions is within a predetermined range; 1. An information processing device comprising:

[0137] (Configuration 2) the processing unit determines the predetermined range based on a distribution of the first detection positions. 2. The information processing device according to configuration 1,

[0138] (Configuration 3) The processing unit performs the first matching process based on a first matching parameter. 3. The information processing device according to configuration 1 or 2.

[0139] (Configuration 4) the processing unit performs the first matching process two or more times with different first matching parameters for each of the first image data to acquire information on the first detection position; 4. The information processing device according to configuration 3.

[0140] (Configuration 5) the processing unit is capable of executing a first mode in which the determination is made, and a second mode in which second image data of an image of an object is acquired, and second matching processing is performed on the second image data to acquire information on a second detection position of the object, determining second matching parameters to be used in the second matching process based on results of the first matching process performed multiple times during execution of the first mode; 4. The information processing device according to configuration 3.

[0141] (Configuration 6) the processing unit displays information about the determined second matching parameters on a display unit when the first mode is executed. 6. The information processing device according to configuration 5.

[0142] (Configuration 7) the processing unit displays, on a display unit, a button for storing the determined second matching parameters in a storage unit when the first mode is executed. 7. The information processing device according to configuration 5 or 6.

[0143] (Configuration 8) the processing unit acquires the first image data by causing the imaging unit to capture an image of the object while varying a first exposure time and / or an imaging angle of the imaging unit. 8. The information processing device according to any one of configurations 1 to 7.

[0144] (Configuration 9) the processing unit acquires the first image data by causing the imaging unit to capture an image of the object while varying at least the first exposure time of the imaging unit and the imaging angle. 8. The information processing device according to any one of configurations 1 to 7.

[0145] (Configuration 10) the processing unit is capable of executing a first mode in which the determination is made, and a second mode in which second image data of an image of an object is acquired, and second matching processing is performed on the second image data to acquire information on a second detection position of the object, determining a second exposure time of the imaging unit to be used in the second mode based on results of the first matching process that has been performed a plurality of times during execution of the first mode; 10. The information processing device according to configuration 9.

[0146] (Configuration 11) the processing unit displays information about the determined second exposure time on a display unit when the first mode is executed. 11. The information processing device according to configuration 10.

[0147] (Configuration 12) the processing unit displays, on a display unit, a button for storing the determined second exposure time in a storage unit when the first mode is executed. 12. The information processing device according to configuration 10 or 11.

[0148] (Configuration 13) the processing unit displays information about the distribution of the first detection positions on a display unit. 13. The information processing device according to any one of configurations 1 to 12.

[0149] (Method 14) An information processing method, comprising: acquiring at least two images of the object; performing at least one matching process on each of the image data to obtain at least two pieces of detection position information relating to the position of the object; determining whether each of the detected positions is within a predetermined range; 1. An information processing method comprising:

[0150] (Configuration 15) An image processing device including a processing unit, The processing unit acquiring at least two images of the object; performing at least one matching process on each of the image data to obtain at least two pieces of detection position information relating to the position of the object; determining whether each of the detected positions is within a predetermined range; 1. An image processing device comprising:

[0151] (Method 16) 1. An image processing method, comprising: acquiring at least two images of the object; performing at least one matching process on each of the image data to obtain at least two pieces of detection position information relating to the position of the object; determining whether each of the detected positions is within a predetermined range; An image processing method comprising:

[0152] (Configuration 17) Robots and and the information processing device according to configuration 1, which acquires information on a second detection position of the object from second image data obtained by capturing an image of the object; the information processing device controls the robot based on information about the second detected position. A robot system characterized by:

[0153] (Method 18) A method for manufacturing an article, characterized in that the article is manufactured using the robot system according to Configuration 17.

[0154] (Configuration 19) A program for causing a computer to execute the information processing method described in Method 14.

[0155] (Configuration 20) A program for causing a computer to execute the image processing method according to Method 16.

[0156] (Configuration 21) A computer-readable recording medium having the program according to configuration 19 or 20 recorded thereon. [Explanation of symbols]

[0157] 1... robot system, 30... robot, 100... control system (information processing device, image processing device), 201, 401... CPU (processing unit)

Claims

1. An information processing method, Acquire multiple pieces of image data by capturing images of the object with different exposure times, setting an exposure time when capturing an image for which a matching process is performed to acquire the position of the object based on the image data; An information processing method comprising:

2. The image data are obtained by imaging the object at different imaging angles.

2. The information processing method according to claim 1,

3. performing at least one matching process for each of the image data using reference image data relating to the object; obtaining a degree of coincidence between the reference image data and the object in each of the image data; setting an exposure time for capturing the captured image based on the degree of coincidence; 2. The information processing method according to claim 1,

4. An exposure time when capturing the captured image is set to an exposure time when the image data for which the degree of coincidence is within a threshold range is captured.

4. The information processing method according to claim 3.

5. When performing a matching process on the image data using reference image data related to the object, the matching process is performed multiple times while changing the contrast, and a degree of match between the reference image data and the object in the image data is obtained; setting a contrast when performing matching processing on the captured image based on the degree of coincidence; 2. The information processing method according to claim 1,

6. The contrast when performing a matching process on the image data in which the degree of match is within a threshold range is set as the contrast when performing a matching process on the captured image.

6. The information processing method according to claim 5,

7. For each of the image data, at least one matching process is performed using reference image data related to the object; For each of the image data, obtain the position of the object; determining whether each of the acquired positions of the object is within a predetermined range; 2. The information processing method according to claim 1,

8. A matching process using the image data acquired at a position not included in the predetermined range is determined to be incorrect.

8. The information processing method according to claim 7,

9. The predetermined range is set by calculating an interquartile range from a distribution of the positions of the object acquired based on the image data.

8. The information processing method according to claim 7,

10. Information regarding an exposure time when capturing the captured image is displayed on a display unit.

2. The information processing method according to claim 1,

11. Information regarding contrast when performing matching processing on the captured image is displayed on a display unit.

6. The information processing method according to claim 5,

12. The exposure time and / or the imaging angle of the imaging unit are automatically changed to allow the imaging unit to capture an image of the object and acquire the image data.

2. The information processing method according to claim 1,

13. Displaying information on the distribution of the positions of the objects obtained based on the image data on a display unit.

10. The information processing method according to claim 9.

14. An information processing device comprising a processing unit, The processing unit Acquire multiple pieces of image data by capturing images of the object with different exposure times, setting an exposure time when capturing an image for which a matching process is performed to acquire the position of the object based on the image data; 1. An information processing device comprising:

15. An image processing method, comprising: Acquire multiple pieces of image data by capturing images of the object with different exposure times, setting an exposure time when capturing an image for which a matching process is performed to acquire the position of the object based on the image data; An image processing method comprising:

16. An image processing device including a processing unit, The processing unit Acquire multiple pieces of image data by capturing images of the object with different exposure times, setting an exposure time when capturing an image for which a matching process is performed to acquire the position of the object based on the image data; 1. An image processing device comprising:

17. A robot that is controlled based on the captured image of the object captured using an exposure time set by the information processing method described in claim 1.

18. A method for manufacturing an article, characterized in that the article is manufactured using a robot controlled based on the captured image of the object captured using an exposure time set by the information processing method described in claim 1.

19. A program for causing a computer to execute the information processing method according to claim 1.

20. A program for causing a computer to execute the image processing method according to claim 15.

21. A computer-readable recording medium on which the program according to claim 19 or 20 is recorded.

Images