OBJECT RECOGNITION SYSTEM AND OBJECT RECOGNITION METHOD

Abstract

To provide an object recognition system and an object recognition method capable of reducing the time required for recognizing an object.SOLUTION: The object recognition system is provided with: a measurement unit 110 that photographs a photographed object under conditions that the number of data points in parts other than the edges of a recognition object, which is an object to be recognized, decreases by 50% or more due to brightness saturation compared to the condition that most data is generated, and outputs three-dimensional data of the photographed object based on the brightness of the photographed data; and a recognition unit 126 that acquires the three-dimensional data from the measurement unit 110, and determines whether the photographed object is the recognition object by comparing the photographing side data, which is the acquired three-dimensional data or any of edge extraction data, which is generated from the three-dimensional data, with model edge data representing an edge shape of the recognition object.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for photographing an object and recognizing the photographed object.

Background Art

[0002] Patent Document 1 discloses a technique for photographing an object to be recognized with a camera, and recognizing the object by comparing an edge extracted from the photographed image with an object model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When image data of an image photographed by a photographing device such as a camera is subjected to image analysis by software to extract edges, it takes time to extract the edges. As a result, there is a problem that it takes time to recognize an object photographed by the camera (hereinafter, a photographed object).

[0005] The present disclosure has been made based on this situation, and an object thereof is to provide an object recognition system and an object recognition method capable of shortening the time until an object is recognized.

Means for Solving the Problems

[0006] The above object is achieved by a combination of features described in the independent claims, and the dependent claims define further advantageous specific examples. The reference signs in parentheses described in the claims indicate the correspondence relationship with the specific aspects described in the embodiments described later as one aspect, and do not limit the disclosed technical scope.

[0007] One disclosure related to an object recognition system for achieving the above object is a photographing device (110) that photographs a photographing object under a condition where the number of data points in a portion other than an edge in a recognition object, which is an object to be recognized, is reduced by 50% or more due to luminance saturation as compared with a condition where the most data is generated, and outputs three-dimensional data of the photographing object based on the luminance of the photographed data; a recognition unit (126, 226) that acquires three-dimensional data from the photographing device and collates the photographing-side data, which is either the acquired three-dimensional data or edge extraction data generated from the three-dimensional data, with model edge data representing the edge shape of the recognition object to determine whether the photographing object is the recognition object.

[0008] In this object recognition system, the photographing device photographs the photographing object under a photographing condition in which luminance saturation occurs when photographing the recognition object. In a luminance-saturated portion, coordinates cannot be calculated based on the luminance. Therefore, the photographing device generates three-dimensional data with no data in the luminance-saturated portion.

[0009] The photographing device photographs the photographing object under a condition where the number of data points in a portion other than an edge in the recognition object is reduced by 50% or more due to luminance saturation as compared with a condition where the most data is generated. Therefore, the number of data points output by the photographing device is greatly reduced as compared with the case where the most data is generated.

[0010] In the photographing object, a flat portion is likely to be luminance-saturated, and an edge portion is relatively unlikely to be luminance-saturated. This is because light projection is likely to be diffused in various directions in the edge portion, so the luminance tends to be low in the image photographed by the photographing device. Therefore, the three-dimensional data output by the photographing device is data in which the data of the flat portion is reduced and the data of the edge portion remains relatively large.

[0011] The recognition unit acquires three-dimensional data from the imaging device, and collates the imaging-side data, which is either this three-dimensional data or edge extraction data generated from the three-dimensional data, with the model edge data.

[0012] Since the three-dimensional data has relatively more data remaining in the edge portions, it is also possible to collate the three-dimensional data with the model edge data without performing edge extraction processing. Further, even if edge extraction data is generated from the three-dimensional data, since the number of data points in the three-dimensional data is decreasing, the amount of computation for generating the edge extraction data can be reduced. Therefore, regardless of whether the three-dimensional data or the edge extraction data is used as the imaging-side data, the amount of computation can be reduced compared to the case of using three-dimensional data obtained by imaging under normal imaging conditions where intentional luminance saturation is not performed. Since the amount of computation can be reduced, the time until an object is recognized can be shortened.

[0013] Also, one disclosure related to an object recognition method for achieving the above object is a method by which the above-described object recognition system can be implemented. That is, one disclosure related to an object recognition method for achieving the above object is setting the imaging conditions such that, for the recognition object, which is the object to be recognized, the number of data points in the portion other than the edge is reduced by 50% or more due to luminance saturation compared to the condition under which the most data is generated, and imaging the imaging object, outputting three-dimensional data of the imaging object based on the luminance of the captured data, and collating the imaging-side data, which is either the three-dimensional data or the edge extraction data generated from the three-dimensional data, with the model edge data representing the edge shape of the recognition object to determine whether the imaging object is the recognition object, which is an object recognition method.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0015] <First Embodiment> Hereinafter, the embodiments will be described with reference to the drawings. FIG. 1 shows an application example of the object recognition system 100 according to the first embodiment. In FIG. 1, the object recognition system 100 recognizes the component 20 that the robot 10 picks up.

[0016] Based on the component 20 recognized by the object recognition system 100, the robot 10 grasps the shape and position of the component and picks up the component 20. The robot 10 includes a base 11, a robot arm 15, and a robot hand 16. The robot 10 is a vertically articulated robot. However, a horizontally articulated robot or the like may be adopted as the robot 10.

[0017] The base 11 is fixed to the installation surface where the robot 10 is installed using bolts or the like. The robot arm 15 is connected to the base 11. The robot arm 15 is configured by connecting a plurality of shaft components with joint components. The joint components include motors, and the angle of the joint components can be freely controlled by controlling the motors. The plurality of shaft components are connected to be relatively rotatable with respect to each other via the joint components.

[0018] The robot hand 16 is attached to the tip of the robot arm 15. The robot hand 16 includes a motor. The robot hand 16 is configured to be relatively rotatable about the axis of the shaft component provided in the robot arm 15 by controlling the motor.

[0019] The robot hand 16 is provided with claw portions. These claw portions rotate about the rotation axis of the robot hand 16 as the center of rotation. The robot hand 16 picks up the component 20 placed on the workbench 30 by performing an opening and closing operation of expanding and contracting the distance between the claw portions. Further, the robot 10 can move the robot arm 15 and the robot hand 16 with the robot hand 16 gripping the component 20. Thereby, the position and posture of the component 20 can be changed.

[0020] The workbench 30 is a rectangular plate member. However, the type and shape of the workbench 30 are not limited to the above example. A box-shaped parts box may be adopted as the workbench 30. Also, a movable device such as a belt conveyor may be adopted as the workbench 30. The number of components 20 placed on the workbench 30 is not limited to one. For example, a plurality of components 20 may be placed on the workbench 30 in a state where they overlap each other.

[0021] The object recognition system 100 has a configuration including a measurement unit 110 and a computer 120. The measurement unit 110 and the computer 120 are connected by a communication cable 130. Specifically, the communication cable 130 is a USB cable or an Ethernet (registered trademark) cable or the like.

[0022] The measurement unit 110 measures the three-dimensional shape of the component 20. The measurement unit 110 is an imaging device that images the component 20 in order to measure the three-dimensional shape of the component 20, and the component 20 is an object to be imaged, that is, an imaging object. The measurement unit 110 of the present embodiment has a configuration called a three-dimensional scanner, and measures the three-dimensional shape of the component 20 by the phase shift method.

[0023] To perform this measurement, the measurement unit 110 includes a projector 111 as a light source that projects light onto the component 20, and a camera 112 that captures a surface image of the component 20 while the projector 111 is projecting light onto the component 20. The camera 112 includes a two-dimensional image sensor and detects the luminance I for each pixel.

[0024] Furthermore, the measurement unit 110 includes a processing circuit 113 that generates three-dimensional data of the component 20 based on the surface image captured by the camera 112.

[0025] The three-dimensional data is data that represents the three-dimensional shape of the component 20 by a three-dimensional point cloud. The three-dimensional data includes the coordinates of each point that constitutes the three-dimensional point cloud and information on the luminance I of that point.

[0026] The measurement unit 110 is installed above the component 20. When measuring the three-dimensional shape of the component 20, the measurement unit 110 projects a stripe pattern image from the projector 111. The camera 112 captures an image of the component 20 while the stripe pattern image is projected onto the component 20.

[0027] In the phase shift method, it is necessary to calculate three unknowns included in "Equation 1: I = Acosθ + B", namely, the luminance amplitude A, the background luminance B, and the phase θ. Here, I is the luminance, which is expressed as a value from 0 to 255, for example. In order to calculate the unknowns in Equation 1, the measurement unit 110 captures images of the component 20 by changing the phase of the stripe pattern image three or more times. Since the positions of the projector 111 and the camera 112 are known, when the phase θ is known, the three-dimensional coordinates of the points detected by each pixel of the camera 112 can be calculated based on the principle of triangulation.

[0028] The processing circuit 113 has a function of controlling the projector 111 and the camera 112, and calculates three-dimensional coordinates to generate three-dimensional data. The processing circuit 113 may execute processing for generating three-dimensional data by hardware such as an ASIC or an FPGA. Further, it is provided with a processor such as a GPU, and the processor may execute processing for generating three-dimensional data.

[0029] As described above, in order to generate three-dimensional data, it is necessary to calculate the phase θ based on Equation 1. When the luminance I is saturated, even if the phase of the stripe pattern image is changed, the luminance I of a certain pixel does not change, and the phase θ in Equation 1 cannot be calculated. As a result, pixels with saturated luminance I cannot calculate three-dimensional coordinates. Therefore, three-dimensional data is not generated for pixels with saturated luminance I. The measurement unit 110 transfers (i.e., outputs) the generated three-dimensional data to the computer 120 using the communication cable 130.

[0030] The computer 120 determines what the component 20 is based on the three-dimensional data transferred from the measurement unit 110. The configuration of the computer 120 is shown in Figure 2. As shown in Figure 2, the computer 120 includes a storage unit 121, a display unit 122, an operation unit 123, and a processor 124.

[0031] The storage unit 121 includes a writable non-volatile memory, such as a flash memory. In the storage unit 121, model edge data of an object to be recognized (hereinafter referred to as a recognition object) is stored. An example of the model edge data is shown in FIG. 3. The model edge data is data obtained by extracting the edge portion of the recognition object. Therefore, the model edge data represents the edge shape of the recognition object. The number of data points of the model edge data shown in FIG. 3 is approximately 1000. In the present embodiment, the storage unit 121 stores model edge data for one type of recognition object. Further, a program executed by the processor 124 may be stored in the storage unit 121. However, this program may be stored in a memory different from the storage unit 121.

[0032] The display unit 122 is a part that displays various information for the user who operates the computer 120. For example, an image showing the recognition result, an operation screen, etc. are displayed on the display unit 122.

[0033] The operation unit 123 is a part where the user performs various input operations. The user can operate the operation unit 123 to adjust the shooting conditions of the measurement unit 110.

[0034] The processor 124 operates as a measurement unit control unit 125 and a recognition unit 126 by executing a program stored in the storage unit 121 and the like. The execution of these operations means that the method corresponding to the program is executed.

[0035] The measurement unit control unit 125 controls the measurement unit 110. The measurement unit control unit 125 can instruct the measurement unit 110 to generate and output three-dimensional data. Further, the measurement unit control unit 125 can adjust the projection power of the projector 111 and the exposure time of the camera 112.

[0036] The recognition unit 126 acquires three-dimensional data from the measurement unit 110. In the present embodiment, this three-dimensional data is used as the shooting-side data, and by comparing the three-dimensional data with the model edge data stored in the storage unit 121, it is determined whether the photographed object photographed by the camera 112 is the recognition object represented by the model edge data.

[0037] 〔Adjustment of shooting conditions〕 Prior to the collation by the recognition unit 126, the user adjusts the shooting conditions by the measurement unit 110. Fig. 4 shows the procedure for adjusting the shooting conditions. The procedure shown in Fig. 4 is the preparation before implementing Fig. 7. Executing the procedures or processes shown in Fig. 4 and Fig. 7 corresponds to the object recognition method.

[0038] In step S1, model edge data is created. The model edge data can be created based on the three-dimensional CAD data of the recognition object.

[0039] When creating model edge data from three-dimensional CAD data, various edge creation methods can be adopted. An example of a method for creating model edge data is the occlusion edge detection method. In the occlusion edge detection method, the places where the difference in distance values between adjacent distance images is large are extracted as edges. Also, the places where the surfaces intersect with a large angular difference in the three-dimensional CAD data may be extracted as edges. The model edge data may be created by the computer 120 or by another computer.

[0040] In step S2, based on the model edge data created in step S1, the number of edge data points is calculated. The model edge data is represented as a set of points, and the number of edge data points is the number of points constituting the model edge data. In step S3, the model edge data and the number of edge data points are stored in the storage unit 121.

[0041] In step S4, the user controls computer 120 to cause measurement unit 110 to capture a recognized object under the imaging candidate conditions. The imaging instruction from computer 120 to measurement unit 110 is issued by measurement unit control section 125. The imaging candidate conditions are conditions under which the luminance I of the image of the recognized object changes. In the present embodiment, the imaging candidate conditions are conditions under which the projection power projected by projector 111 is different. The user arbitrarily determines the first imaging candidate condition, that is, the first projection power.

[0042] In step S5, the user operates computer 120 to cause computer 120 to calculate the number of data points of the three-dimensional data output by measurement unit 110 in the imaging in S4.

[0043] In step S6, the user determines whether the number of data points calculated in step S5 approximates the number of data points of the model edge data stored in S3. The degree of difference between the number of data points of the model edge data and the number of data points of the three-dimensional data obtained by imaging the recognized object for which approximation is determined can be appropriately determined in consideration of the shape of the object and the like. For example, if the difference between the number of data points of the model edge data and the number of data points of the three-dimensional data obtained by imaging the recognized object is about 10% of either number of data points, it is determined that they are approximate.

[0044] If the determination result in step S6 is NO, the process proceeds to step S7. In step S7, the user operates computer 120 to change the projection power of measurement unit 110. For example, the projection power is increased to a predetermined value. As the projection power is increased, the planar portion of the recognized object reaches luminance saturation, whereby the number of data points of the three-dimensional data decreases. FIG. 5 illustrates three-dimensional data captured with a weak projection power of POWER = 10. FIG. 6 illustrates three-dimensional data captured with a stronger projection power of POWER = 50. In the case of FIG. 5, the number of data points is about 5000, and in the case of FIG. 6, the number of data points is about 1000.

[0045] Referring to FIG. 5, most of the surface portions of the recognized object, including the flat portion of the recognized object, are represented. It can be considered that FIG. 5 is the three-dimensional data obtained by photographing under the condition where the most data is generated.

[0046] As described above, the number of data points of the model edge data is about 1000. Therefore, in the recognized object shown in FIG. 5, the number of data points of the portion other than the edge is about 4000. In the three-dimensional data of FIG. 6, almost only the edges are present. The photographing condition when generating the three-dimensional data of FIG. 6 is a photographing condition in which the number of data points other than the edge is reduced by 50% or more, more precisely, almost 100% reduction due to luminance saturation, compared with the condition where the most data is generated.

[0047] After changing the projection power in step S7, the process returns to S4, and photographing is performed under the changed candidate photographing conditions. As can be seen from the comparison between FIGS. 5 and 6, it can be seen that when the projection power is increased, the data of the flat portion of the recognized object decreases, while the data of the edge portion does not decrease so much. This is because even when the flat portion reaches luminance saturation, the edge portion often does not reach luminance saturation. The reason why the edge portion does not reach luminance saturation even when the flat portion reaches luminance saturation is that since the projection light is easily diffused in various directions in the edge portion, the luminance I tends to be low in the image captured by the camera 112.

[0048] When increasing the projection power in S7, the number of data points of the three-dimensional data obtained by photographing the recognized object decreases and approaches the number of data points of the model edge data. As a result, if the determination in step S6 is YES, the process proceeds to step S8. When the number of data points of the three-dimensional data obtained by photographing the recognized object is approximated to the number of data points of the model edge data, as can be seen from the comparison between FIGS. 3 and 6, the image approximates the model edge data.

[0049] In step S8, the candidate photographing conditions including the projection power set at this time are determined as the photographing conditions for photographing the photographed object.

[0050] 〔Recognition process〕 After adjusting the shooting conditions as described above, the user who uses the object recognition system 100 performs a recognition process to determine whether the component 20 is a recognized object, with the component 20 as the shooting object. Fig. 7 shows the flow of this recognition process. In Fig. 7, steps S11 to S13 are executed by the measurement unit 110, and steps S14 and S15 are executed by the recognition unit 126 of the computer 120.

[0051] In step S11, the measurement unit 110 shoots the component 20 under the shooting conditions determined in Fig. 4. In step S12, the measurement unit 110 generates three-dimensional data of the component 20 from the image data of the component 20 shot in step S11. In step S13, the measurement unit 110 transfers the three-dimensional data generated in step S12 to the computer 120, and the computer 120 acquires the three-dimensional data.

[0052] In step S14, the recognition unit 126 of the computer 120 collates the three-dimensional data with the model edge data stored in the storage unit 121. When the three-dimensional data is data in which only almost edges are represented as shown in Fig. 6, it can be collated with the model edge data without performing edge extraction processing on the three-dimensional data.

[0053] In step S15, the recognition unit 126 outputs the collation result. Specific examples of the content of the collation result are whether the component 20 is a recognized object or not. The output destination of the collation result is, for example, the display unit 122. The output destination of the collation result may also be a robot control device that controls the robot 10. Note that the computer 120 may have the function of a robot control device. When it can be determined based on the collation result that the component 20 is a recognized object, the robot control device can determine an operation to pick up the component 20 based on the shape data of the recognized object stored in advance.

[0054] Robot 10 sequentially picks up a plurality of components 20. Therefore, since the object recognition system 100 also needs to sequentially recognize a plurality of components 20, after executing S15, it returns to S11.

[0055] 〔Summary of the First Embodiment〕 In the object recognition system 100 of the first embodiment described above, the measurement unit 110 sets the shooting conditions to those in which the luminance saturation is reduced by nearly 100% compared with the conditions under which the number of data points in the portion other than the edge in the recognized object is the largest and the most data is generated. When the component 20 is the recognized object, the number of data points of the three-dimensional data obtained by shooting under this shooting condition becomes about 1000, which is greatly reduced compared with the case where the most data is generated.

[0056] The recognition unit 126 acquires three-dimensional data from the measurement unit 110 (S13) and collates this three-dimensional data with the model edge data (S14).

[0057] Since most of the three-dimensional data represents the edge portion, the three-dimensional data is collated with the model edge data without performing edge extraction processing. If three-dimensional data obtained by shooting under normal shooting conditions where the luminance saturation is not intentionally increased is used, edge extraction processing is required when collating with the model edge data. On the other hand, in this embodiment, edge extraction processing is not required, so the amount of calculation can be reduced. Since the amount of calculation can be reduced, the time until the object is recognized can be shortened.

[0058] The object recognition system 100 of this embodiment sets the shooting conditions to those with a stronger light projection power than the conditions under which the most data is generated. Since the light projection power is adjusted, the time until the object is recognized can be shortened compared with the case of adjusting the exposure time.

[0059] The object recognition system 100 includes a measurement unit 110 and a computer 120 that is separate from the measurement unit 110. The measurement unit 110 and the computer 120 are connected by a communication cable 130. When the measurement unit 110 and the computer 120 are connected by the communication cable 130, the data transfer speed from the measurement unit 110 to the computer 120 is slower than when they are connected by an address bus. However, in the object recognition system 100, the number of data points of the three-dimensional data is decreasing. Therefore, even if the data transfer speed from the measurement unit 110 to the computer 120 is slow, an increase in the data transfer time can be suppressed. And when the measurement unit 110 and the computer 120 are configured separately, they can be procured separately, so the measurement unit 110 can be made low-cost.

[0060] The object recognition method described in the first embodiment includes a procedure for adjusting shooting conditions (FIG. 4). In this procedure, the recognition object is photographed while changing the shooting candidate conditions (S4). Then, the number of data points of the three-dimensional data is calculated (S5), and when this number of data points approximates the model edge data (S6: YES), the shooting candidate conditions at that time are set as the shooting conditions (S8). By adjusting the shooting conditions in this way, the recognition unit 126 can compare the three-dimensional data transferred from the measurement unit 110 with the model edge data as it is without performing edge extraction processing.

[0061] <Second Embodiment> Next, the second embodiment will be described. In the following description of the second embodiment and below, elements having the same reference numerals as those used so far are the same as the elements with the same reference numerals in the previous embodiments, unless otherwise specified. Also, when only a part of the configuration is described, the previously described embodiments can be applied to the other parts of the configuration.

[0062] Figure 8 shows the object recognition system 200 of the second embodiment. Similar to the object recognition system 100, the object recognition system 200 also has a measurement unit 110 and a computer 120 connected by a communication cable 130. The computer 120 includes a storage unit 121, a display unit 122, an operation unit 123, and a processor 124, similar to the first embodiment.

[0063] However, the number of types of model edge data stored in the storage unit 121 is different from that of the first embodiment. Also, the operation of the recognition unit 226 that operates when the processor 124 executes a program is different from that of the recognition unit 126 of the first embodiment.

[0064] In the second embodiment, a plurality of types of model edge data are stored as a database in the storage unit 121. Figure 9 is a diagram for explaining this database. As shown in Figure 9, the database has one set of model edge data, the projection power which is the imaging condition to be changed, and the number of data points of the model edge data associated with each other.

[0065] Each set of data is determined by executing the procedure for adjusting the imaging conditions described with reference to Figure 4 in the first embodiment. Assume that the component 20A model shown in Figure 9 is the model shown in Figure 3. Assume that the component 20B model is the model shown in Figure 10.

[0066] Returning to the explanation of Figure 8, the recognition unit 226 includes a model edge data determination unit 227, a shooting condition instruction unit 228, and a collation unit 229.

[0067] When sequentially collating a plurality of components 20, the three-dimensional data of the component 20 is sequentially transferred from the measurement unit 110 to the computer 120. The model edge data determination unit 227 determines whether one piece of model edge data selected from the plurality of pieces of model edge data stored in the storage unit 121 is correct. For this determination, the number of data points that can be determined from the three-dimensional data sequentially acquired from the measurement unit 110 is used.

[0068] For example, the model edge data determination unit 227 compares the number of data points of the three-dimensional data sequentially acquired from the measurement unit 110 with the number of data points of the currently selected model edge data. For the comparison, the difference or ratio of the number of data points can be used. If the difference or ratio of the number of data points is equal to or less than a preset threshold for these, it is determined that the currently selected model edge data is correct. If the difference or ratio of the number of data points exceeds the threshold, it is determined that the currently selected model edge data is incorrect.

[0069] As an example where it is determined that the currently selected model edge data is incorrect, it is conceivable that the part 20 to be picked up by the robot 10 is flowing on a belt conveyor, and the part 20 flowing on the belt conveyor has been switched to another part.

[0070] Another method for the model edge data determination unit 227 to perform the above determination using the number of data points is a method that uses the time-series change of the number of data points of the three-dimensional data sequentially acquired. If the number of data points changes by a threshold or more with respect to the number of data points in the previous time or the average of the number of data points for a plurality of past times, it is determined that the currently selected model edge data is incorrect. This is because when the part 20 is switched to another part, the number of data points is likely to change by a threshold or more with respect to the number of data points in the previous time or the average of the number of data points for a plurality of past times.

[0071] When the model edge data determination unit 227 determines that the currently selected model edge data is incorrect, the imaging condition instruction unit 228 changes the model edge data used for collation. The changed model edge data can be, for example, the model edge data stored in the database next to the model edge data before the change.

[0072] However, in the present embodiment, when there are a plurality of candidates for the changed model edge data, the two-dimensional shape of the model edge data is used to determine the changed model edge data. The candidates for the changed model edge data mean the model edge data included in the database excluding the currently selected model edge data.

[0073] Specifically, the imaging condition instruction unit 228 determines a two-dimensional shape from the three-dimensional data acquired from the measurement unit 110. Further, the imaging condition instruction unit 228 also determines a two-dimensional shape from the candidates of the model edge data after the change. Note that the two-dimensional shape determined from the model edge data may be determined in advance. Then, the imaging condition instruction unit 228 sets the model edge data whose two-dimensional shape determined from the model edge data is close to the two-dimensional shape determined from the three-dimensional data acquired from the measurement unit 110 as the model edge data after the change. The model edge data after the change becomes the model edge data used for collation.

[0074] As a method for determining whether two two-dimensional shapes are close, the similarity calculated by various methods used for two-dimensional shape matching can be adopted. If this similarity is equal to or greater than a predetermined value, it is determined that the two two-dimensional shapes are close. Note that, regardless of whether the similarity is equal to or greater than the predetermined value, the candidate of the model edge data with the highest similarity may be set as the model edge data after the change.

[0075] The imaging condition instruction unit 228 instructs the measurement unit 110 via the measurement unit control unit 125 of the light projection power associated with the model edge data after the change, that is, the imaging condition.

[0076] After the imaging condition instruction unit 228 instructs the measurement unit 110 of the imaging condition, that is, after the imaging condition of the measurement unit 110 is changed, the collation unit 229 collates the three-dimensional data acquired from the measurement unit 110 with the model edge data after the change.

[0077] 〔Recognition process〕 FIG. 11 shows the flow of the recognition process executed in the second embodiment. In FIG. 11, steps S21, S22, and S23 are executed by the measurement unit 110. S21, S22, and S23 are the same as S11, S12, and S13 in FIG. 7, respectively.

[0078] The following steps after step S24 are executed by the computer 120. In step S24, the model edge data determination unit 227 determines whether the selected model edge data is correct. As described above, the number of data points of the three-dimensional data acquired from the measurement unit 110 is used for this determination. For example, the number of data points of the three-dimensional data acquired from the measurement unit 110 is compared with the number of data points of the selected model edge data to determine whether the selected model edge data is correct.

[0079] If the determination result in step S24 is YES, that is, if it is determined that the selected model edge data is correct, the process proceeds to step S25. Steps S25 and S26 are the same as S14 and S15 in FIG. 7, respectively. In step S25, the collation unit 229 collates the three-dimensional data with the selected model edge data. In step S26, the collation unit 229 outputs the collation result in step S25.

[0080] If the determination result in step S24 is NO, that is, if it is determined that the selected model edge data is incorrect, the process proceeds to step S27. Here, assume that component 20B is photographed while the model edge data of component 20A is selected. As shown in FIG. 9, the projection power set for component 20B is stronger than the projection power set for component 20A. Therefore, if component 20B is photographed with the projection power set for component 20A, luminance saturation does not occur as intended. As a result, when component 20B is photographed with the projection power set for component 20A to generate three-dimensional data, the number of data points increases compared to the case where component 20B is photographed with the projection power set for component 20B.

[0081] FIG. 12 illustrates the three-dimensional data generated by photographing component 20B with the projection power set for component 20A. Comparing the image of the three-dimensional data shown in FIG. 12 with the image of the model edge data shown in FIG. 10, it can be seen that the image of the three-dimensional data shown in FIG. 12 has more planar portions represented and has a larger number of data points.

[0082] Steps S27 and the next step S28 are executed by the shooting condition instruction unit 228. In step S27, a shooting condition selection process for selecting the changed shooting conditions is executed. The details of the shooting condition selection process are shown in FIG. 13.

[0083] In FIG. 13, in step S271, it is determined whether there are a plurality of candidates. For example, if there are only two sets of data in the database, the determination result in step S271 is YES. If the determination result in step S271 is NO, that is, if there is only one candidate for the shooting conditions, the process proceeds to step S272.

[0084] In step S272, the shooting condition that has been narrowed down to one is determined as the changed shooting condition. Also, the model edge data used for collation is changed to the model edge data corresponding to that shooting condition.

[0085] If the determination result in step S271 is YES, the process proceeds to step S273. In step S273, it is determined whether the shooting conditions can be narrowed down based on the two-dimensional shape. In this step S273, specifically, the two-dimensional shape is determined from the three-dimensional data acquired by the measurement unit 110. Also, the two-dimensional shape is determined from the candidates of the changed model edge data. Then, if there is a candidate for the model edge data whose two-dimensional shape similarity is equal to or greater than a predetermined value, the determination result in step S273 is set to YES. If the determination result in step S273 is YES, the process proceeds to step S274.

[0086] In step S274, it is determined whether there are a plurality of candidates for the model edge data whose two-dimensional shape similarity is equal to or greater than a predetermined value. If the determination result in step S274 is NO, that is, if there is only one candidate, the process proceeds to step S272, and the model edge data that has been narrowed down to one is used as the model edge data for collation, and the shooting condition corresponding to that model edge data is determined as the changed shooting condition.

[0087] If the determination result in step S274 is YES, the process proceeds to step S275. In step S275, one candidate is selected from a plurality of candidates. The method of selecting one candidate is, for example, the method of selecting the one with the highest similarity. Alternatively, among the plurality of candidates, the one with the smallest or largest number in the database may be selected as the candidate after narrowing down. After executing step S275, the process proceeds to step S272, and the model edge data selected as the model edge data to be used for collation is set, and the imaging condition corresponding to the model edge data is determined as the imaging condition after the change.

[0088] If the determination result in step S273 is NO, the process proceeds to step S276. Even if the candidate of the model edge data is correct, it may be determined that the two-dimensional shapes are not close due to the difference in posture, that is, the determination result in step S273 may be NO. In step S276, in the database, the model edge data with the next number of the currently selected model edge data is selected. Then, the process proceeds to step S272, and the newly selected model edge data is set as the model edge data to be used for collation, and the imaging condition corresponding to the model edge data is determined as the imaging condition after the change.

[0089] When the process of FIG. 13 is completed, the process proceeds to step S28 in FIG. 11. In step S28, the imaging condition instruction unit 228 instructs the measurement unit 110 of the imaging conditions selected in step S27 via the measurement unit control unit 125. As a result, the measurement unit 110 captures an image of the component 20 under the changed imaging conditions, and generates three-dimensional data from the captured image data. FIG. 14 shows the three-dimensional data obtained after changing the imaging conditions. The three-dimensional data shown in FIG. 14 is three-dimensional data approximated to the model edge data shown in FIG. 10. Therefore, the determination result in the subsequent step S24 becomes YES, and in step S25, the component 20 can be correctly recognized.

[0090] 〔Summary of the Second Embodiment〕 In the object recognition system 200 of this second embodiment, the storage unit 121 stores a database including a plurality of sets of data in which model edge data and shooting conditions are associated with each other. When the model edge data determination unit 227 determines that the currently selected model edge data is incorrect, the shooting condition instruction unit 228 changes the model edge data to be used for collation and instructs the measurement unit 110 of the shooting conditions associated with the changed model edge data. Therefore, even when the object to be photographed is switched, the shooting conditions can be automatically changed to those suitable for the object to be photographed after the switch, shortening the time required to recognize the object while continuing accurate object recognition.

[0091] In addition, when the model edge data determination unit 227 determines that the currently selected model edge data is incorrect, the shooting condition instruction unit 228 sets the model edge data whose two-dimensional shape is close to the two-dimensional shape determined from the three-dimensional data as the model edge data to be used for collation. The process of handling two-dimensional shapes can be performed more quickly than the process of handling three-dimensional data. Therefore, by doing so, it is often possible to quickly switch to appropriate model edge data compared to simply determining the changed model edge data in order.

[0092] Although the embodiments have been described above, the disclosed technology is not limited to the above-described embodiments, and the following modification examples are also included in the disclosed scope, and various other modifications can be made without departing from the gist thereof.

[0093] <Modification Example 1> In the embodiment, by adjusting the projection power among the imaging conditions, the number of data points of the three-dimensional data generated by the measurement unit 110 was reduced. However, the imaging condition to be adjusted may be the exposure time. Alternatively, the projection power and the exposure time may be adjusted. By increasing the exposure time, intentional luminance saturation can also be caused. When causing luminance saturation by increasing the exposure time, the projection power can be made weaker than when only the projection power is adjusted. Therefore, problems caused by the projection being too strong, such as suppressing the discomfort given to people existing around, can be reduced.

[0094] <Modification Example 2> In the embodiment, the recognition units 126 and 226 did not perform edge extraction processing on the three-dimensional data transferred from the measurement unit 110, and directly compared the three-dimensional data with the model edge data as it was.

[0095] However, the recognition units 126 and 226 may perform edge extraction processing on the three-dimensional data acquired from the measurement unit 110 to acquire edge extraction data. When the edge extraction data is acquired, the edge extraction data is collated with the model edge data to determine whether the photographed object is the recognized object.

[0096] When performing edge extraction processing, it takes more time to recognize the object than the recognition processing described in the embodiment by the time required for the processing. However, the number of data points of the three-dimensional data transferred from the measurement unit 110 is reduced compared to the case where the most data is generated. Therefore, the transfer time can be shortened compared to the case where the most data is generated. Also, the edge extraction processing time is shortened by the amount of reduction in the number of data points. Therefore, the time until the object is recognized can be shortened compared to the case where the most data is generated.

[0097] In addition, when performing edge extraction processing, when adjusting the shooting conditions according to the procedure shown in FIG. 4, there is an advantage that it is not necessary to adjust the shooting conditions until the number of data points of the three-dimensional data generated by shooting the recognition object approximates the model edge data.

[0098] Therefore, the shooting conditions do not need to be such that the number of data points other than the edges is reduced to nearly 100% due to luminance saturation. The shooting conditions may be such that the number of data points other than the edges is reduced by 50% or more due to luminance saturation compared to the conditions under which the most data is generated. Of course, shooting conditions in which the number of data points other than the edges is reduced more, such as 60% or more, 70% or more, 80% or more, etc., are more preferable than 50% or more.

[0099] <Modification Example 3> The measurement unit 110 measured the three-dimensional shape of the object by the phase shift method. However, the method for measuring the three-dimensional shape of the object is not limited to the phase shift method. For example, the three-dimensional shape of the object may be measured by the TOF method.

[0100] <Modification Example 4> The measurement unit 110 and the computer 120 may be integrated, and the measurement unit 110 and the computer 120 may be connected by an address bus.

[0101] <Modification Example 5> In the embodiment, it was determined whether or not the component 20 was a recognition object. Even if the component 20 is a recognition object, if there are scratches or damages, the degree of coincidence between the three-dimensional data transferred by the measurement unit 110 and the model edge data will decrease. Therefore, the object recognition systems 100 and 200 and the object recognition method disclosed in the embodiment can also be applied to the appearance inspection of objects.

Explanation of Reference Numerals

[0102] 10: Robot 11: Base 15: Robot Arm 16: Robot Hand 20: Component (Photographing Object) 30: Workbench 100: Object Recognition System 110: Measurement Unit (Photographing Device) 111: Projector 112: Camera 113: Processing Circuit 120: Computer 121: Memory Unit 122: Display Unit 123: Operation Unit 124: Processor 125: Measurement Unit Control Unit 126: Recognition Unit 130: Communication Cable 200: Object Recognition System 226: Recognition Unit 227: Model Edge Data Judgment Unit 228: Photographing Condition Instruction Unit 229: Matching Unit

Claims

1. An imaging device (110) that captures an imaging object under a condition where the number of data points in a portion other than an edge in a recognition object, which is an object to be recognized, is reduced by 50% or more due to luminance saturation as compared with a condition under which the most data is generated, and outputs three-dimensional data of the imaging object based on the luminance of the captured data, and a recognition unit (126, 226) that collates imaging-side data, which is either the three-dimensional data acquired from the imaging device or edge extraction data generated from the three-dimensional data, with model edge data representing an edge shape of the recognition object to determine whether the imaging object is the recognition object, An object recognition system comprising the above.

2. The object recognition system according to claim 1, wherein the imaging condition is a condition in which the projection power is stronger than the condition under which the most data is generated.

3. The object recognition system according to claim 1 or 2, wherein the imaging condition is a condition in which the exposure time is longer than the condition under which the most data is generated.

4. The object recognition system according to any one of claims 1 to 3, comprising a computer (120) that operates as the recognition unit, wherein the imaging device and the computer are separate bodies, and the imaging device and the computer are connected by a communication cable (130).

5. The object recognition system according to any one of claims 1 to 4, wherein the recognition unit collates the three-dimensional data acquired from the imaging device with the model edge data to determine whether the imaging object is the recognition object.

6. The object recognition system according to any one of claims 1 to 4, wherein the recognition unit executes edge extraction processing on the three-dimensional data acquired from the imaging device to obtain the edge extraction data, and collates the obtained edge extraction data with the model edge data to determine whether the imaging object is the recognition object.

7. The object recognition system according to any one of claims 1 to 6, comprising a storage unit (121) that stores in association a plurality of the model edge data respectively representing edge shapes of a plurality of the recognition objects and the imaging conditions when the model edge data is selected, and the recognition unit (226) A model edge data determination unit (227) that determines whether the model edge data selected from the plurality of model edge data stored in the storage unit is correct based on the number of data points of the photographed-side data; A photographing condition instruction unit (228) that, when the model edge data determination unit determines that the selected model edge data is incorrect, changes the model edge data used for collation and instructs the photographing device of the photographing conditions determined by the changed model edge data; An object recognition system comprising a collation unit (229) that collates the photographed-side data acquired from the photographing device after the photographing condition instruction unit instructs the photographing conditions to the photographing device with the changed model edge data.

8. The object recognition system according to claim 7, wherein when the model edge data determination unit determines that the selected model edge data is incorrect, the photographing condition instruction unit uses, as the model edge data used for collation, the model edge data whose two-dimensional shape is close to the two-dimensional shape of the photographed-side data.

9. The photographed object is photographed under a condition that the luminance saturation is reduced by 50% or more compared to a condition under which the most data is generated with the number of data points of the portion other than the edge in the recognition object, which is the object to be recognized, as the photographing condition, the three-dimensional data of the photographed object is output based on the luminance of the photographed data, and an object recognition method for collating either the photographed-side data, which is either the three-dimensional data or the edge extraction data generated from the three-dimensional data, with the model edge data representing the edge shape of the recognition object to determine whether the photographed object is the recognition object.

10. The object recognition method according to claim 9, wherein the recognition object is photographed while changing the photographing candidate conditions that are candidates for the photographing conditions, and the photographing candidate conditions when the number of data points of the three-dimensional data approximates the model edge data are set as the photographing conditions.

Images