Three-dimensional measurement system, three-dimensional measurement method, and program

Abstract

Even in environments where production methods are consolidated and mixed, the shape measurement of target workpieces can be performed optimally. [Solution] The three-dimensional measurement system 100 is a three-dimensional measurement system that measures the three-dimensional shape of a target workpiece by capturing an illumination area of ​​a predetermined area that is irradiated in a predetermined pattern with a two-dimensional light receiving sensor 21. The three-dimensional measurement system 100 includes an illumination area division unit 32 that divides the illumination area into two or more parts and irradiates each part with illumination light in a different pattern, an imaging area division unit 33 that divides the imaging area into two or more parts, an area association unit 34 that associates the divided areas of the illumination area with the corresponding divided areas of the imaging area, an assignment unit 35 that assigns one of several measurement methods to each associated divided area, and a shape measurement calculation unit 36 ​​that measures the shape of the target workpiece according to the measurement method assigned to each divided area.

Description

Technical Field

[0001] The present invention relates to a three-dimensional measurement system, a three-dimensional measurement method, and a program.

Background Art

[0002] In the production sites of various products, various articles (such as parts) are in a static state such as being stacked or laid flat, or are in a state of being conveyed by a belt conveyor or the like. In such a production site, the shape of an article is measured by three-dimensional measurement based on the principle of triangulation. In such a production site, the optimal method for recognizing the shape differs between measuring the shape of an article in a static state and measuring the shape of an article in a conveyed state. For example, the phase shift method or the like is suitable for measuring the shape of an article in a static state. On the other hand, the optical cutting method or the like is suitable for measuring the shape of an article in a conveyed state. From such a viewpoint, it has been proposed to apply different measurement methods for measuring the shape of an article in a static state and measuring the shape of an article in a conveyed state (see, for example, Patent Document 1). Patent Document 1 describes switching between a first camera and a second camera according to the distance of a target work and applying one of a plurality of shape measurement methods.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, as will be described below, it is desired that the prior art disclosed in Patent Document 1 suitably measure the shape of a target work even in an environment where production forms are aggregated and mixed.

[0005] In recent years, with the transformation of production methods, it has become necessary to construct the optimal production system each time to cope with small-batch, high-mix production and variable-volume production. Furthermore, while automation using robots is progressing in the transportation and inspection of intermediate work-in-progress parts, the target workpieces may be placed in a bulk state in transport containers such as buckets, or they may be continuously transported using belt conveyors. Therefore, the production system changes in various ways.

[0006] Several known measurement methods exist, including the phase shift method, the light section method, and the spatial coding method. These measurement methods have various characteristics, and it is preferable to use them appropriately depending on the situation. However, the conventional technology disclosed in Patent Document 1 applies only one measurement method to an image of one camera field of view captured by one camera. Therefore, the conventional technology could not use arbitrary measurement methods depending on the situation, that is, to respond to various changing production systems, and in particular, it could not switch between multiple types of measurement methods for each region. Consequently, in the conventional technology, when attempting to measure the shape of a target workpiece using multiple different measurement methods, it was necessary to prepare images of multiple camera fields of view captured by multiple cameras.

[0007] Furthermore, from the perspective of production site space, there is a need to consolidate production processes and optimize the occupied area. Under these circumstances, it is not practical from the standpoint of efficiency and cost to have separate equipment to measure the shape of each workpiece. Ideally, a system that can be used universally is needed.

[0008] The present invention has been made in view of the problems of the prior art described above, and the object of the present invention is to provide a three-dimensional measurement system, a three-dimensional measurement method, and a program that can suitably measure the shape of a target workpiece even in an environment where production methods are consolidated and mixed. [Means for solving the problem]

[0009] The above-mentioned problems of the present invention are solved by the following means.

[0010] (1) A three-dimensional measurement system for measuring the three-dimensional shape of a target workpiece by photographing an irradiated area of ​​a predetermined area irradiated in a predetermined pattern with a two-dimensional light receiving sensor, comprising: an irradiated area division unit that divides the irradiated area into two or more parts and irradiates each part with irradiated light in a different pattern; an imaging area division unit that divides the imaging area into two or more parts; an area association unit that associates the divided areas of the imaging area with the divided areas of the irradiated area; an assignment unit that assigns one of several measurement methods to each of the associated divided areas; and a shape measurement calculation unit that measures the shape of the target workpiece according to the measurement method assigned to each of the divided areas.

[0011] (2) The three-dimensional measurement system according to (1) above, wherein the assignment unit assigns an arbitrary measurement method to each of the divided regions, thereby enabling the simultaneous use of different measurement methods for each divided region according to the state of the target workpiece.

[0012] (3) The three-dimensional measurement system according to (1) above, wherein the measurement method associated with each of the divided regions within the imaging area is two or more different measurement methods.

[0013] (4) The three-dimensional measurement system according to (1) above, comprising: an image capture unit that independently operates a two-dimensional light-receiving sensor to capture an image; an image display unit that displays the image captured by the image capture unit; and a reception unit that receives user instructions, wherein the illumination area division unit and the shooting area division unit determine the division area according to instructions from the user.

[0014] (5) The three-dimensional measurement system according to (1) above, further comprising an object recognition unit that recognizes target objects in the captured image, wherein the irradiation area division unit and the shooting area division unit identify the division areas of the irradiation area and the shooting area according to the recognition result.

[0015] (6) The three-dimensional measurement system according to (1) above, comprising: an object height identification unit that identifies the heights of multiple target objects within the same field of view; a position identification unit that identifies the relative position of the boundary of the divided region and the irradiated light; and a shadow region identification unit that identifies the shadow region in the divided region that is further away in height from the divided region, wherein the shape measurement calculation unit performs shape measurement at a relative position that satisfies the first condition that the target workpiece is within the camera field of view.

[0016] (7) The shape measurement calculation unit performs shape measurement at a relative position that satisfies the first condition and the second condition that the area within the field of view where the irradiated light reaches is 90% or more of the maximum value, or the third condition that the shooting area is 90% or more of the maximum value, as described in (6) above.

[0017] (8) The three-dimensional measurement system according to (6) above, further comprising a distance determination unit that determines the physical distance between multiple objects located within the same field of view, wherein the shape measurement calculation unit, based on the first condition, performs shape measurement within a range in which the shadow of one divided region does not affect the other divided region, such that the length of the shadow of the object on the side where the work distance from the two-dimensional light receiving sensor to the object is farther is less than or equal to a predetermined margin, thus satisfying the fourth condition.

[0018] (9) The three-dimensional measurement system according to (1) above, wherein the irradiation area division unit recognizes the work area and does not form an irradiation pattern in a certain area at the boundary of the work area.

[0019] (10) The three-dimensional measurement system according to (1) above, comprising a division region control display unit that displays the division region, and a method candidate display control unit that displays candidate shape measurement methods, wherein the assignment unit assigns a shape measurement method to the division region in accordance with instructions from the user.

[0020] (11) The object recognition unit estimates the object attributes of the divided region, and the assignment unit assigns a measurement method to the divided region according to the estimation result of the object attributes. The three-dimensional measurement system according to (5) above.

[0021] (12) It has an imaging control unit that switches a plurality of two-dimensional light-receiving sensors. The object recognition unit identifies the height difference between the divided regions, and the imaging control unit selects a sensor that focuses on the object for each divided region. The three-dimensional measurement system according to (5) above.

[0022] (13) It has an imaging control unit that switches a plurality of two-dimensional light-receiving sensors. The object height identification unit detects fluctuations in the height of the object, and the imaging control unit switches to a sensor that focuses on the object when the height becomes a certain distance or more or less. The three-dimensional measurement system according to (6) above.

[0023] (14) It has a storage unit that stores the association information between the divided region and the measurement method, and a content display control unit that displays the content of the stored association information. The region association unit updates the content of the stored association information according to the user's instruction. The three-dimensional measurement system according to (1) above.

[0024] (15) The object recognition unit detects that the target object has moved during the system operation, and the region association unit sets a new divided region by identifying the moved region. The three-dimensional measurement system according to (5) above.

[0025] (16) A three-dimensional measurement method, comprising the steps of: dividing each irradiation region into two or more, and irradiating the irradiation light with different patterns; dividing the imaging region into two or more; associating the corresponding divided regions of the imaging region with the divided regions of the irradiated region; for each of the associated divided regions, assigning one of a plurality of measurement methods; and performing shape measurement of the target workpiece according to the measurement method assigned to each divided region.

[0026] (17) A program that causes a computer to perform procedures of dividing an irradiation area into two or more parts and irradiating irradiation light with different patterns for each part, dividing a shooting area into two or more parts, associating corresponding areas between the divided areas of the irradiation area and the divided areas of the light receiving area, for each of the associated divided areas, allocating one of a plurality of measurement methods, and performing shape measurement of a target work according to the measurement method allocated for each divided area.

Advantages of the Invention

[0027] According to the present invention, even in an environment where production forms are concentrated and mixed, shape measurement of a target work can be suitably performed.

Brief Description of the Drawings

[0028] [Figure 1] It is a configuration block diagram of a three-dimensional measurement system according to an embodiment. [Figure 2] It is a schematic explanatory diagram of the operation of a three-dimensional measurement system according to an embodiment. [Figure 3A] It is an explanatory diagram of the division of the irradiation area. [Figure 3B] It is an explanatory diagram of the division of the irradiation area. [Figure 4A] It is an explanatory diagram (1) of the association of divided areas. [Figure 4B] It is an explanatory diagram (2) of the association of divided areas. s [Figure 4C] It is an explanatory diagram (3) of the association of divided areas. [Figure 5] It is an explanatory diagram of the measurement method. [Figure 6A] It is an operation explanatory diagram (1) when one camera can handle it. [Figure 6B] It is an operation explanatory diagram (2) when one camera can handle it. [Figure 7A] It is an operation explanatory diagram (1) when one camera cannot handle it. [Figure 7B] It is an operation explanatory diagram (2) when one camera cannot handle it. [Figure 8A] This is an explanatory diagram of the irradiation area when no interference prevention area is provided. [Figure 8B] This is an explanatory diagram of the irradiation area when an interference prevention area is provided. [Figure 9A] This is a diagram (1) illustrating the positions of the projector (light source) and camera relative to the target workpiece. [Figure 9B] This is a diagram (2) illustrating the positions of the projector (light source) and camera relative to the target workpiece. [Figure 10A] This is a diagram (1) illustrating the orientation of the projector (light source). [Figure 10B] This is a diagram (2) illustrating the orientation of the projector (light source). [Figure 11A] This is a diagram (1) illustrating the shadowed area. [Figure 11B] This is a diagram (2) illustrating the shadowed area. [Figure 12] This is an explanatory diagram illustrating the relationship between the shadow area and the margin. [Figure 13A] This is diagram (1) illustrating the relationship between the direction of the projector (light source) and the shadow area. [Figure 13B] This is diagram (2) illustrating the relationship between the direction of the projector (light source) and the shadow area. [Figure 14] This flowchart shows the overall schematic operation of the three-dimensional measurement system. [Figure 15] This flowchart shows the process for determining the camera field of view of a three-dimensional measurement system. [Figure 16] This flowchart shows the process of determining the division region of a three-dimensional measurement system. [Figure 17] This is a flowchart showing the assignment of measurement methods for a three-dimensional measurement system. [Figure 18] This is a flowchart showing the shape measurement operation of a three-dimensional measurement system. [Figure 19] This flowchart shows the post-processing operations of a three-dimensional measurement system. [Modes for carrying out the invention]

[0029] Embodiments of the present invention will be described in detail below with reference to the drawings. Note that the drawings are merely schematic representations to allow for a thorough understanding of the present invention. Therefore, the present invention is not limited to the illustrated examples. Furthermore, in each drawing, common or similar components are denoted by the same reference numerals, and their redundant descriptions are omitted.

[0030] <Configuration of the 3D measurement system> The configuration of the three-dimensional measurement system 100 according to this embodiment will be described below with reference to Figure 1. Figure 1 is a block diagram of the configuration of the three-dimensional measurement system 100 according to this embodiment.

[0031] The three-dimensional measurement system 100 is an active 3D vision system equipped with a projector and a two-dimensional light receiving sensor. The three-dimensional measurement system 100 divides the two-dimensional illumination area and the imaging area into separate areas. Furthermore, the three-dimensional measurement system 100 allows for the simultaneous use of different measurement methods, such as the phase shift method and the light section method, depending on the state of the target workpiece (e.g., stationary or moving), and variations or differences in the workpiece distance. In this explanation, the camera height is assumed to be the workpiece distance between the camera and the target workpiece.

[0032] As shown in Figure 1, the three-dimensional measurement system 100 according to this embodiment comprises a projector 10, a camera 20, and a three-dimensional measurement device 30 consisting of a computer.

[0033] Projector 10 is a light source that emits projector light. The light source may also be a laser oscillator that emits laser light instead of projector 10.

[0034] Camera 20 is a means for capturing images of a target workpiece. Camera 20 incorporates a two-dimensional light receiving sensor 21 that emits an electrical signal upon receiving light. Camera 20 captures light projected from the projector 10, which is the light source, with the two-dimensional light receiving sensor 21 to acquire brightness information of the target workpiece and its surroundings. In this embodiment, the three-dimensional measurement system 100 is described as comprising a first camera 20a for short distances and a second camera 20b for long distances. However, the three-dimensional measurement system 100 may have other cameras 20. Each camera 20 is supported by a robot arm 26. The robot arm 26 functions as a movable part that changes the shooting position by directing the optical axis of each camera 20 in any direction.

[0035] The three-dimensional measuring device 30 is a device for measuring the three-dimensional shape and dimensions of a target workpiece. The three-dimensional measuring device 30 is composed of a computer. The three-dimensional measuring device 30 incorporates a control unit 31 and a storage unit 70, and is connected to a display unit 18, which is a display unit, and a reception unit 19, which is an input device such as a keyboard or mouse. The reception unit 19 is a component that receives instructions from the user. The irradiation area division unit 32 and the shooting area division unit 33 determine the division areas according to the instructions from the user via the reception unit 19. The irradiation area division unit 32 and the shooting area division unit 33 can also identify the division areas of the irradiation area and the shooting area according to the recognition results from the object recognition unit 40.

[0036] The control unit 31 is a component that controls the operation of the entire three-dimensional measuring device 30. The control unit 31 constructs each of the functional means shown in Figure 1 as an example by executing a control program 30pr stored in the memory unit 70. In the example shown in Figure 1, the control unit 31 has an illumination area division unit 32, an imaging area division unit 33, an area association unit 34, an assignment unit 35, and a shape measurement calculation unit 36. The control unit 31 also has an object recognition unit 40, an object height identification unit 41, a position identification unit 42, a shadow area identification unit 43, a distance identification unit 44, a divided area display control unit 46, a method candidate display control unit 47, and a content display control unit 48. The control unit 31 also has a post-processing unit 50, a projection control unit 51, and an imaging control unit 52. However, some of these components can be deleted.

[0037] The irradiation area division unit 32 is a component that generates a projection pattern to be projected onto the irradiation area. The irradiation area division unit 32 functions as an irradiation area division unit that divides the irradiation area into two or more parts. The imaging area division unit 33 is a component that selects a method for measuring the shape. The imaging area division unit 33 divides the imaging area into two or more parts.

[0038] The region association unit 34 is a component that associates corresponding divided regions of the irradiation region and the imaging region, that is, a component that associates a divided region of the irradiation region with a corresponding divided region of the imaging region. The assignment unit 35 is a component that assigns one of several measurement methods to each associated divided region. The shape measurement calculation unit 36 ​​is a component that measures the shape of the target workpiece according to the measurement method assigned to each divided region.

[0039] The object recognition unit 40 is a component that recognizes target objects in the acquired image. The object height identification unit 41 is a component that identifies the heights of multiple target objects within the same field of view.

[0040] The position identification unit 42 is a component that identifies the relative position of the boundary of the divided region and the irradiated light. The shadow area identification unit 43 is a component that identifies the shadow area in the division area that is farther in height from the division area that is adjacent to the division area. The distance determination unit 44 is a component that determines the physical distance between multiple objects that exist within the same field of view. The divided region display control unit 46 is a component that displays the divided regions on the display 18 in a way that allows them to be identified.

[0041] The method candidate display control unit 47 is a component that displays candidate shape measurement methods on the display 18 in an identifiable manner. The content display control unit 48 is a component that displays the contents of the information stored in the storage unit 70 (for example, association information 74 and assignment information 75, etc.) on the display 18.

[0042] The post-processing unit 50 is a component that performs the post-processing described later, as shown in Figure 19. The projection control unit 51 is a component that controls the operation of the projector 10. The imaging control unit 52 is a component that controls the operation of the camera 20 and the robot arm 26.

[0043] The memory unit 70 stores information such as imaging area information 71, projection area information 72, projection pattern information 73, association information 74, and assignment information 75.

[0044] The shooting area information 71 is information regarding the shooting area of ​​the camera 20. The projection area information 72 is information regarding the imaging area of ​​the projection pattern projected from the projector 10. The projection pattern information 73 is information regarding the projection pattern projected from the projector 10. Association information 74 represents the correspondence between the divided regions of the associated irradiation area and the divided regions of the imaging area. The assignment information 75 represents the type of measurement method assigned to each associated segmented region from among multiple available measurement methods.

[0045] Furthermore, the memory unit 70 stores programs such as the control program 30pr. The control program 30pr is a program that enables the computer to function as a three-dimensional measuring device 30. The control program 30pr is stored in a storage medium 90 or the like, and is installed into the computer directly or indirectly from the storage medium 90 to enable the computer to function as a three-dimensional measuring device 30.

[0046] Figure 2 is a schematic diagram illustrating the operation of the three-dimensional measurement system 100. As shown in Figure 2, the three-dimensional measurement system 100 divides the two-dimensional illumination area and the imaging area into areas and associates the divided areas of the illumination area with the divided areas of the imaging area. Then, the three-dimensional measurement system 100 assigns two or more different measurement methods, such as the phase shift method or the light section method, to each divided area of ​​the imaging area, depending on the state of the target workpiece (stationary or moving), the variation or difference in the workpiece distance σ from the camera to the target workpiece, etc. After this, the three-dimensional measurement system 100 illuminates (projects) the target workpiece with the projection pattern corresponding to each divided area and acquires a two-dimensional image of the target workpiece. Then, using the measurement methods assigned to each divided area of ​​the imaging area, the three-dimensional measurement system 100 measures the shape of the target workpiece based on the two-dimensional image of the target workpiece and acquires a three-dimensional reconstruction image of the target workpiece.

[0047] In this configuration, the illumination area division unit 32 synthesizes and generates the necessary projection patterns for each divided area of ​​the illumination area based on the information held in the memory unit 70. The projection control unit 51 controls optical means such as the projector 10 to illuminate (project) the light of the projection pattern generated by the illumination area division unit 32 onto the target workpiece. The imaging control unit 52 acquires a two-dimensional image formed on the two-dimensional light receiving sensor 21 built into the camera 20. At this time, the projection control unit 51 and the imaging control unit 52 operate in synchronous and coordinated manner in order to synchronize the timing of light irradiation (projection) and light reception.

[0048] The shooting area division unit 33 selects a measurement method assigned to each divided area of ​​the shooting area, which stores the signals and images obtained from the camera 20 in the storage unit 70. The shape measurement calculation unit 36 ​​uses the measurement method assigned to each divided area of ​​the shooting area to measure the shape of the target workpiece based on the two-dimensional image of the target workpiece and obtains a three-dimensional reconstructed image of the target workpiece.

[0049] By the way, the prior art described in Patent Document 1 applies only one measurement method to an image of one camera field of view captured by one camera. Therefore, if one attempts to perform shape measurement of a target workpiece using multiple different measurement methods, the prior art described in Patent Document 1 requires the preparation of multiple images of multiple camera fields of view captured by multiple cameras. In contrast, the three-dimensional measurement system 100 according to this embodiment performs shape measurement of a target workpiece based on multiple images of multiple camera fields of view captured by multiple cameras, by dividing an image of one camera field of view captured by one camera into multiple regions.

[0050] Figures 3A and 3B are explanatory diagrams of the division of the illumination area, respectively. Figure 3A shows an example of the division of the illumination pattern, and Figure 3B shows an example of the processing on the light-receiving side. The example shown in Figure 3A shows that the illumination patterns are different in the first region 101a on the left side of the divided illumination area and the second region 102aa,102ab on the right side. The example shown in Figure 3B shows that shape measurement is performed using the phase shift method in the first region 101b on the left side of the divided imaging area, and shape measurement is performed using the light section method in the second region 102b on the right side. Note that the phase shift method and the light section method are known measurement techniques, so a detailed explanation is omitted here.

[0051] Figures 4A to 4C are explanatory diagrams illustrating the relationships between the divided regions. Figure 4A shows an example where one belt conveyor 106 and one bucket 107 are divided into N sections. The bucket 107 is a container transported by the belt conveyor 106. Figure 4A shows classification patterns based on differences in size between the belt conveyor 106 and the bucket 107, and differences in the state of movement or stationary state of the belt conveyor 106 and the bucket 107. In Figure 4A, the white arrows indicate the transport direction of the belt conveyor 106. The three-dimensional measurement system 100 changes the allocation of the field of view of the vision according to the equipment configuration. At that time, the three-dimensional measurement system 100 determines the angle of view when photographed from above. Figure 4B shows an example where the illumination area and the shooting area are divided into N sections. In the example shown in Figure 4B, the irradiation area and the imaging area are divided into one first area 101a on the left side corresponding to one belt conveyor 106, and two second areas 102aa and 102ab on the right side corresponding to two buckets 107a and 107b. Note that the division method is not limited to the example shown in Figure 4B, and any N divisions may be used. The three-dimensional measurement system 100 should be divided according to the measurement method best suited to each area. Figure 4C shows an example in which a measurement method is assigned to each divided area. In the example shown in Figure 4C, the light section method is assigned to the first area 101a, and the phase shift method and spatial coding method are assigned to the second areas 102aa and 102ab.

[0052] Figure 5 is an explanatory diagram of the measurement method. Figure 5 shows examples of the assignment of the measurement method in the case where the target workpieces in the first region are stacked loosely and on a conveyor belt, as well as in each combination of the cases where the target workpieces in the first region are stacked loosely and on a conveyor belt.

[0053] The three-dimensional measurement system 100 can perform region division manually or automatically. In manual mode, the user specifies a predetermined number of divisions and division size to the three-dimensional measurement device 30. In automatic mode, the three-dimensional measurement device 30 recognizes objects in the image captured by the camera 20 and automatically distinguishes between the belt conveyor 106 region and the bucket 107 region.

[0054] For each divided region, the three-dimensional measurement system 100 assigns a measurement method. This assignment can also be done manually or automatically. In the manual case, the user specifies the measurement method to the three-dimensional measurement device 30 for each divided region. In the automatic case, the three-dimensional measurement device 30 has pre-associated measurement methods with each divided region, and automatically assigns each divided region to a measurement method according to the attributes of each divided region.

[0055] Figures 2 to 5 show an example where the three-dimensional measurement system 100 has one camera system 20, but the present invention can also be applied when the three-dimensional measurement system 100 has two cameras system 20. When the three-dimensional measurement system 100 has two cameras system 20, in addition to dividing the region, the first camera 20a and the second camera 20b are switched depending on the difference or change in workpiece distance that changes with the height of the camera 20, as well as the accuracy of the measurement and whether or not there is a change in workpiece distance. In this case, the three-dimensional measurement system 100 should define the relationship between each divided region and the corresponding first camera 20a or second camera 20b according to the position of the target workpiece, etc.

[0056] Figures 6A and 6B are explanatory diagrams illustrating the operation when one camera 20 can handle the task. Figures 6A and 6B show examples where the difference between the workpiece distance between the target workpiece 111 on the right and the camera 20, and the workpiece distance between the target workpiece 111a on the left and the camera 20, is small, and one camera 20 can handle the task. Specifically, Figure 6A shows the target workpiece 111a placed on the belt conveyor 106 on the left, and the target workpiece 111b placed in a bucket 107 whose top surface is at the same height as the top surface of the belt conveyor 106 on the right. Figure 6B shows the division of the shooting area, which is an image taken by the first camera 20a for short distances, into a first area 101 and a second area 102. The first area 101 corresponds to the left side of Figure 6A, and the second area 102 corresponds to the right side of Figure 6A. Furthermore, Figure 6B shows that the light section method is assigned to the first region 101 and the phase shift method is assigned to the second region 102 as measurement methods. In the example shown in Figure 6B, since the workpiece distance of the target workpiece 111a in the first region 101 is constant, the light section method, which is suitable for measuring the shape of an object at a constant distance, is assigned to the first region 101 as the measurement method. Also, since the workpiece distance of the target workpiece 111b in the second region 102 changes, the phase shift method, which is suitable for measuring the shape of an object at a changing distance, is assigned to the second region 102 as the measurement method. Assume that the target workpiece 111a placed on the belt conveyor 106 and the target workpiece 111b placed in the bucket 107 are positioned, for example, as shown in Figure 6A. In this case, the three-dimensional measurement system 100 measures the shape of each target workpiece 111a and 111b based on the image captured by the first camera 20a for short distances. At that time, the three-dimensional measurement system 100 measures the shape of each target workpiece 111a, 111b using the assigned measurement method as shown in Figure 6B.

[0057] Figures 7A and 7B are explanatory diagrams of the operation when one camera is insufficient. Figures 7A and 7B show examples where the difference between the workpiece distance between the target workpiece 111a on the right and the camera 20 and the workpiece distance between the target workpiece 111b on the left and the camera 20 is large, and one camera 20 is insufficient. In this case, the three-dimensional measurement system 100 photographs the target workpieces 111a and 111b in each region using the first camera 20a and the second camera 20b. Specifically, Figure 7A shows the target workpiece 111a placed on the belt conveyor 106 on the left, and the target workpiece 111b placed in a bucket 107 whose top is positioned lower than the top surface of the belt conveyor 106 on the right. Figure 7B shows that the imaging area, which is the image captured by the first camera 20a for short distances, and the imaging area, which is the image captured by the second camera 20b for long distances, are divided into a first region 101 and a second region 102, respectively. The first region 101 corresponds to the left side of Figure 7A, and the second region 102 corresponds to the right side of Figure 7A. Figure 7B also shows that, as a measurement method, the light section method is assigned to the first region 101 of the image captured by the first camera 20a, and the phase shift method is assigned to the second region 102 of the image captured by the second camera 20b. In the example shown in Figure 7B, since the workpiece distance of the target workpiece 111a in the first region 101 of the image captured by the first camera 20a is constant, the light section method, which is suitable for measuring the shape of an object at a constant distance, is assigned to the first region 101 as the measurement method. Furthermore, since the workpiece distance of the target workpiece 111b in the second region 102 of the image captured by the second camera 20b changes, the phase shift method, which is suitable for measuring the shape of objects whose distance changes, is assigned to the second region 102 as a measurement method. Assume that the target workpiece 111a placed on the belt conveyor 106 and the target workpiece 111b placed in the bucket 107 are positioned as shown in Figure 7A, for example. In this case, the three-dimensional measurement system 100 measures the shape of each target workpiece 111a, 111b based on the image captured by the first camera 20a for short distances and the image captured by the second camera 20b for long distances.At that time, the three-dimensional measurement system 100 measures the shape of each target workpiece 111a, 111b using the assigned measurement method as shown in Figure 7B.

[0058] In addition to the patterns shown in Figures 6A and 6B, and Figures 7A and 7B, the three-dimensional measurement system 100 may also be configured to automatically switch the image from the camera 20 used for shape measurement when the workpiece distance changes during shape measurement.

[0059] (Interference prevention area) The three-dimensional measurement system 100 may set or change interference prevention areas 103 at the boundaries of any divided regions. Figure 8A is an explanatory diagram of the irradiation area when no interference prevention area 103 is provided. Figure 8B is an explanatory diagram of the irradiation area when an interference prevention area 103 is provided. The interference prevention area 103 is an area that is not irradiated with light, set at the boundary of any divided region. The irradiation area division unit 32 of the three-dimensional measurement system 100 recognizes the work area and operates so as not to form an irradiation pattern in a certain area at its boundary. By setting or changing the interference prevention area 103, such a three-dimensional measurement system 100 can prevent interference in the three-dimensional measurement calculation of the target workpiece 111 at the boundary of adjacent divided regions. The setting or changing of the interference prevention area 103 may be done manually or automatically. When done automatically, the three-dimensional measurement system 100 recognizes the margins of the belt conveyor 106 and bucket 107 and determines the interference prevention area 103.

[0060] (Positions of the projector and camera, which are the light sources for the target workpiece) The three-dimensional measurement system 100 may have areas where the shape of the target workpiece 111 cannot be measured due to the influence of the workpiece distance. An example of this is shown in Figures 9A and 9B. Figures 9A and 9B are explanatory diagrams of the positions of the projector 10 and camera 20 relative to the target workpiece 111, respectively. Figures 9A and 9B show, on the left, the target workpiece 111a placed on the belt conveyor 106, and on the right, the target workpiece 111b placed in a bucket 107 whose top is positioned lower than the top surface of the belt conveyor 106. Furthermore, Figure 9A shows the projector 10 placed above the target workpiece 111a on the left and the camera 20 placed above the target workpiece 111b on the right. Conversely, Figure 9B shows the camera 20 placed above the target workpiece 111a on the left and the projector 10 placed above the target workpiece 111b on the right.

[0061] In the examples shown in Figures 9A and 9B, the target workpiece 111b is placed in the right-hand bucket 107, and the target workpiece 111a is placed on the left-hand belt conveyor 106, which is positioned higher than the bucket 107. In the example shown in Figure 9A, the projector 10 is positioned above the left-hand belt conveyor 106, and the camera 20 is positioned above the right-hand bucket 107. On the other hand, in the example shown in Figure 9B, the camera 20 is positioned above the left-hand belt conveyor 106, and the projector 10 is positioned above the right-hand bucket 107. In the example shown in Figure 9A, an area is created in the right-hand bucket 107 where the illumination light does not reach, and the camera 20 cannot capture an image of that area, making it impossible to measure the shape of the target workpiece 111b in the area where the illumination light does not reach. Conversely, in the example shown in Figure 9B, although no area is not reached within the right-hand bucket 107 by the irradiated light, an area becomes a blind spot for the camera 20. As a result, the camera 20 cannot capture an image of that area, and therefore cannot measure the shape of the target workpiece 111b in the blind spot of the camera 20.

[0062] In this regard, it is desirable that the irradiated light be received in the entire area that can be captured by the two-dimensional light receiving sensor of the camera 20. However, as shown in Figure 9A, if there are target workpieces 111 at different distances, a shadow will be cast in the area of ​​target workpiece 111a that is illuminated by the irradiated light on the area of ​​target workpiece 111b that is not illuminated by the irradiated light. This results in an area where the shape of the other target workpiece 111b cannot be measured. Conversely, even if the irradiated light reaches the entire area, there may be areas that are blind spots for the camera 20. In this case, it is not possible to measure the shape of the target workpiece 111 in the area that is a blind spot for the camera 20. Due to the blind spots of the camera 20, an area where the shape of the target workpiece 111 cannot be measured will be created. Therefore, the three-dimensional measurement system 100 should be designed to prevent the creation of areas where the shape of the target workpiece 111 cannot be measured by devising the position and orientation of the projector 10 and the camera 20.

[0063] Figures 10A and 10B are explanatory diagrams of the orientation of the projector 10 (light source), respectively. In the example shown in Figure 10A, the projector 10 (light source) is positioned above the boundary line of the left-side belt conveyor 106, and light is irradiated along the longitudinal direction (conveying direction) of the belt conveyor 106. This prevents the three-dimensional measurement system 100 from creating areas where the shape of the target workpiece 111 cannot be measured. In the example shown in Figure 10B, the projector 10 (light source) is positioned above the boundary line of the left-side belt conveyor 106 near the right-side bucket 107, and light is irradiated along a direction perpendicular to the longitudinal direction (conveying direction) of the belt conveyor 106. This prevents the three-dimensional measurement system 100 from creating areas where the shape of the target workpiece 111 cannot be measured.

[0064] In such a three-dimensional measurement system 100, it is preferable to determine the position of the camera 20 so that the light source is located on the boundary line of the divided region. The light source may be positioned in a region further away from the boundary line, as long as it is within the field of view that can capture the target workpiece 111. In this case, since the field of view of the camera 20 usually has different vertical and horizontal dimensions, whether the camera 20 itself is parallel or perpendicular to the boundary line should be determined according to the group of target workpieces.

[0065] Furthermore, in order to prevent the occurrence of areas where the shape of the target workpiece 111 cannot be measured, the shape measurement calculation unit 36 ​​of the three-dimensional measurement system 100 may perform shape measurement at a relative position that satisfies the first condition that the target workpiece is contained within the following camera field of view.

[0066] Furthermore, the shape measurement calculation unit 36 ​​of the three-dimensional measurement system 100 performs shape measurement at a relative position that satisfies the second condition, which is that the area within the field of view where the irradiated light reaches is 90% or more of the maximum value, or the third condition, which is that the shooting area is 90% or more of the maximum value, based on the first condition.

[0067] Furthermore, at a relative position that satisfies either the second or third condition, the area will be maximized in the region satisfying either the second or third condition, while the area will be fixed (limited) in the region satisfying the other condition. Here, a relative position that satisfies either the second or third condition means a relative position that maximizes the area of ​​either the second or third condition, among the candidates that maximize the area of ​​the other condition.

[0068] (Margin of the divided area) When determining the position of camera 20 to satisfy the above conditions, the three-dimensional measurement system 100 is based on dividing the divided region with a boundary line. However, if the target workpiece group is far apart in the region to be divided, the three-dimensional measurement system 100 can ensure a degree of freedom in the optimal position of camera 20 by providing a margin between the divided regions. The margin represents the distance at which the shadow does not fall on the target workpiece 111.

[0069] The three-dimensional measurement system 100 can be configured with a projector 10 and a camera 20, for example, as shown in Figures 11A and 11B. Figures 11A and 11B are explanatory diagrams of shadow regions, respectively. Figures 11A and 11B differ from Figures 9A and 9B in that the right bucket 107 is spaced further apart from the right end of the left belt conveyor 106. In the example shown in Figure 11A, even if there is a region that does not receive light, it does not affect the shape measurement of the target workpiece 111. Therefore, in the example shown in Figure 11A, the degree of freedom of the camera field of view can be increased accordingly. Conversely, in the example shown in Figure 11B, even if there is a blind spot on the observation side, it does not affect the shape measurement of the target workpiece 111. Therefore, in the example shown in Figure 11B, the degree of freedom of the camera field of view can be increased accordingly.

[0070] (Relationship between shadow area and margin) Furthermore, the three-dimensional measurement system 100 can be configured to include a projector 10 and a camera 20, for example, as shown in Figure 12. Figure 12 is an explanatory diagram of the relationship between the shadow area and the margin. In the example shown in Figure 12, the projector 10 is positioned at a distance ξ1 from the right end of the belt conveyor 106 to the left, and at a distance δ1 above the workpiece. The bucket 107 is positioned at a distance ξ2 from the right end of the belt conveyor 106 to the right, and at a distance δ2 below the camera 20.

[0071] In the example shown in Figure 12, the shape measurement calculation unit 36 ​​of the three-dimensional measurement system 100, based on the first condition described above, performs shape measurement within a range where the shadow of one divided region does not affect the other divided region, so as to satisfy the fourth condition that the length ξ2 of the shadow of the object on the farther side is less than or equal to a predetermined margin. Fourth condition: ξ2 = ((δ2-σ1) / δ1) × ξ1 ≤ margin The meanings of each symbol in the fourth condition are as follows: δ1: Workpiece distance from the two-dimensional light receiving sensor 21 to the nearest object δ2: Work distance from the two-dimensional light receiving sensor 21 to the object on the far side ξ1: Amount of shift of the light source from the edge of the nearby object. ξ2: Length of the shadow on the farther object

[0072] The three-dimensional measurement system 100 ensures that even if shadowed or blind spots occur within the margin area, the measurement of the shape of the target workpiece 111 will not be affected. Therefore, the three-dimensional measurement system 100 can improve the degree of freedom of camera positioning.

[0073] Furthermore, the three-dimensional measurement system 100 can arrange the projector 10 and camera 20, for example, as shown in Figures 13A and 13B, so as to ensure a margin area 113. Figures 13A and 13B are explanatory diagrams illustrating the relationship between the orientation of the projector 10 (light source) and the shadow area, respectively.

[0074] As shown in Figures 13A and 13B, a certain margin area 113 is assumed to be present in the work area when viewed from directly above. In this case, the three-dimensional measurement system 100 can set the position of the camera 20 within the range in which the shadow generated falls within this margin area 113, based on the relationship between the position of the projector 10 (light source position) and the work distance between the projector 10 and the target work 111. Furthermore, even if the projector 10 is placed on the boundary line of the belt conveyor 106 and the target work area does not fit within the camera's field of view, the three-dimensional measurement system 100 can accommodate this by changing the position of the projector 10 to the left. In this way, the three-dimensional measurement system 100 can achieve both securing an illumination area unaffected by shadows and securing a camera field of view that includes the target work. Such a three-dimensional measurement system 100 can improve the freedom of the camera field of view because the shadow of the projector 10 (light source) does not affect the area of ​​the target work 111. Note that here, the method for determining the camera position when the target object is fixed has been explained. However, if the physical placement of the target object itself is flexible, the three-dimensional measurement system 100 may be configured to provide guidance to the user so that the target object or camera 20 is in the optimal position.

[0075] It is possible to build a system that performs object recognition with a single vision system, without requiring any additional components. Furthermore, in production sites where various methods of transporting the target workpiece 111 (bulk loading, belt conveyor) coexist, the three-dimensional measurement system 100 can also be configured to recognize objects using a single camera 20, without requiring multiple cameras 20 depending on the application.

[0076] <Operation of the 3D Measurement System> The operation of the three-dimensional measurement system 100 will be described below with reference to Figures 14 to 19.

[0077] (Overall general operation of the 3D measurement system) First, the overall general operation of the three-dimensional measurement system 100 will be explained with reference to Figure 14. Figure 14 is a flowchart illustrating the overall general operation of the three-dimensional measurement system 100.

[0078] As shown in Figure 14, the three-dimensional measurement system 100 determines the position of the camera 20 using the robot arm 26 (movable part) (step S105). In step S105, in order to photograph the target workpiece 111, the three-dimensional measurement system 100 positions the camera body of the camera 20 at the optimal position for illumination and shooting angle. The camera 20 can be used either fixed in place or attached to the robot arm 26. Here, we will explain assuming that the camera 20 is attached to the robot arm 26.

[0079] After step S105, the three-dimensional measurement system 100 determines the division areas of the illumination area and the imaging area using the illumination area division unit 32 and the imaging area division unit 33, and divides each into two or more areas (step S110). In step S110, when the imaging angle is divided into multiple areas, the three-dimensional measurement system 100 manually or automatically determines areas with different shape measurement methods. In the manual case, the user is allowed to specify the division areas through a user interface such as the reception unit 19 (Figure 1). In the automatic case, the system recognizes objects such as buckets and conveyor belts on which the target workpiece 111 is placed, and determines the division area for each object.

[0080] After step S110, the three-dimensional measurement system 100 associates corresponding regions of the divided regions of the irradiation region and the divided regions of the imaging region in the region association unit 34 (step S115).

[0081] After step S115, the three-dimensional measurement system 100 assigns one of several measurement methods to each of the associated divided regions in the assignment unit 35 (step S120).

[0082] After step S120, the three-dimensional measurement system 100 measures the shape of the target workpiece 111 according to the measurement method assigned to each divided region by the shape measurement calculation unit 36 ​​(step S125). In step S125, the three-dimensional measurement system 100 projects light and reconstructs the shape of the target workpiece 111 in three dimensions from the brightness information obtained by receiving the light with the two-dimensional light receiving sensor 21 (Figure 1).

[0083] After step S125, the three-dimensional measurement system 100 performs post-processing in the post-processing unit 50 (step S130). In this embodiment, step S130 (post-processing) will be described as either performing a shape inspection of the target workpiece 111 or performing picking (grasping) of the target workpiece 111. The three-dimensional measurement system 100 changes the information output to the outside depending on the content of the post-processing. When performing a shape inspection of the target workpiece 111, the three-dimensional measurement system 100 outputs shape determination information of the target workpiece 111 to the outside. On the other hand, when performing picking (grasping) of the target workpiece 111, the three-dimensional measurement system 100 outputs gripping information of the target workpiece 111 (position and orientation information, gripping position information, etc.) to the outside.

[0084] (Camera position determination process) Next, referring to Figure 15, the process of step S105 (camera position determination process) shown in Figure 14 will be explained. The overall general operation of the three-dimensional measurement system 100 will be described. Figure 15 is a flowchart showing the camera field of view determination process of the three-dimensional measurement system 100.

[0085] As shown in Figure 15, the three-dimensional measurement system 100 moves the camera 20 to its initial position using the robot arm 26 (step S205). Step S205 assumes that the camera 20 is mounted on the robot arm 26. If the camera 20 is fixed, the relative position of the camera 20 and the target workpiece 111 is changed by manually changing the position of the camera 20 or by manually changing the position of the target workpiece 111.

[0086] After step S205, the three-dimensional measurement system 100 projects light over the entire area of ​​the illumination region (step S210). In step S210, the projector 10 (light source) projects a projection pattern of any shape observable on the camera 20 side (imaging side).

[0087] After step S210, the three-dimensional measurement system 100 acquires images with the camera 20 (step S215).

[0088] After step S215, the three-dimensional measurement system 100 determines, based on the captured image, whether the object to be measured (target workpiece 111) is visible and whether the effects of shadows and blind spots are minimized (step S220). The determination in step S220 is made by detecting, based on the captured image, whether there are any shadows or blind spots, or even if there are no shadows or blind spots, whether there are any areas where light cannot be received. At this time, the user may be made to confirm these through a user interface, or these may be automatically determined based on information such as the placement of structures and the distance to the workpiece that has been determined in advance.

[0089] If the determination in step S220 indicates that the object to be measured (target workpiece 111) is not visible, or that the influence of shadows or blind spots is not minimized ("No"), the three-dimensional measurement system 100 changes the relative position of the camera 20 and the target workpiece 111 (step S225). After this, the process returns to step S210. The three-dimensional measurement system 100 repeats the process from steps S210 to S225, changing the relative position of the camera 20 and the target workpiece 111 until the camera position is appropriate.

[0090] On the other hand, if the determination in step S220 is that the object to be measured (target workpiece 111) is visible and the influence of shadows and blind spots is minimal ("Yes"), the three-dimensional measurement system 100 maintains the shooting position (step S230).

[0091] (Process for determining the division areas between the irradiation area and the imaging area, and the process for associating the areas with each other) Next, referring to Figure 16, the processes of step S110 (determination of the division areas between the irradiation area and the imaging area) and step S115 (association process between areas) shown in Figure 14 will be explained. Figure 16 is a flowchart showing the division area determination operation of the three-dimensional measurement system 100.

[0092] As shown in Figure 16, the three-dimensional measurement system 100 identifies the area of ​​a structure that appears in the captured image (step S305). In step S305, the system identifies the area in which structures such as a conveyor belt or a bucket are visible. Possible methods for identification include having the user specify the area through a user interface such as the reception unit 19 (Figure 1), or the three-dimensional measurement device 30 automatically recognizing the structure and identifying the area using a known method.

[0093] After step S305, the three-dimensional measurement system 100 determines the division areas of the irradiation area and the imaging area, and associates the corresponding areas of the division areas of the irradiation area and the division areas of the imaging area. Then, the three-dimensional measurement system 100 stores the division area information of the imaging area (imaging area information 71 (Figure 1)) in the storage unit 70 (Figure 1) (step S310). The three-dimensional measurement system 100 also stores the division area information of the corresponding irradiation side (irradiation area) (projection area information 72 (Figure 1)) in the storage unit 70 (Figure 1) (step S315). However, the processing in step S310 may be performed before the processing in step S310.

[0094] In step S315, the three-dimensional measurement system 100 identifies areas in which structures such as belt conveyors and buckets are captured, and ultimately divides the captured image into multiple parts according to the structures, storing the divided areas in the storage unit 70. Also in step S315, the system determines the corresponding divided area information on the illumination side from the divided area information on the shooting side and stores it in the storage unit 70. As a result, the three-dimensional measurement system 100 matches the divided areas on the illumination side (irradiation area side) with the divided areas on the shooting side.

[0095] (Assignment process of measurement method) Next, referring to Figure 17, we will explain the process of step S120 (measurement method assignment process) shown in Figure 14. Figure 17 is a flowchart of the measurement method assignment operation of the three-dimensional measurement system 100.

[0096] As shown in Figure 17, the three-dimensional measurement system 100 determines one projection pattern for each divided region on the irradiation side (irradiation area side) and associates the divided regions on the irradiation side (irradiation area side) with the projection pattern (step S405). The three-dimensional measurement system 100 also determines one shape measurement method to assign to the divided regions on the imaging side and assigns the shape measurement method to the divided regions on the imaging side (step S410). This information is stored in the storage unit 70 and referenced by the shape measurement calculation unit 36 ​​when executing the shape measurement process in step S125 shown in Figure 14.

[0097] (Shape measurement processing) Next, referring to Figure 18, the process of step S125 (shape measurement process) shown in Figure 14 will be explained. Figure 18 is a flowchart showing the shape measurement operation of the three-dimensional measurement system 100.

[0098] As shown in Figure 18, the three-dimensional measurement system 100 generates a projection pattern in the illumination area division unit 32 and controls the projector 10 (light source) in the projection control unit 51 to project the projection pattern (step S505). In step S505, the three-dimensional measurement system 100 determines a projection pattern from the information held in the memory unit 70 according to the measurement method assigned to each division area on the imaging side, and integrates each projection pattern to generate a single projection pattern for the entire area. The three-dimensional measurement system 100 controls the projector 10 in the projection control unit 51 to project the projection pattern for the entire area onto the target workpiece 111, etc.

[0099] After step S505, the three-dimensional measurement system 100 controls the camera 20 with the imaging control unit 52 to acquire a captured image, and the object recognition unit 40 reads an image of the projection pattern from the captured image (step S510). In step S510, the three-dimensional measurement system 100 receives light from the camera 20 to acquire brightness information for the projection pattern of the entire area.

[0100] In steps S505 and S510, the three-dimensional measurement system 100 may need to switch projection patterns multiple times depending on the measurement method. Therefore, the projection control unit 51 and the imaging control unit 52 synchronize with each other to acquire the desired brightness information.

[0101] After step S510, the three-dimensional measurement system 100 divides the captured image into processing target areas in the allocation unit 35 (step S515). In step S515, the three-dimensional measurement system 100 determines the shape measurement target area based on the captured area held in the memory unit 70.

[0102] After step S515, the three-dimensional measurement system 100 calculates the shape measurement of the target workpiece 111 for each divided region in the shape measurement calculation unit 36 ​​(step S520). In step S520, the three-dimensional measurement system 100 measures the shape of the target workpiece 111 using different shape measurement methods for each divided region.

[0103] Such a three-dimensional measurement system 100 can accurately measure the shape of the target workpiece 111 using multiple known measurement methods (such as the phase shift method, light section method, and spatial coding method) for each divided region on the imaging side.

[0104] (Post-processing) Next, referring to Figure 19, the post-processing steps (post-processing) shown in Figure 14 will be described. Figure 19 is a flowchart showing the post-processing operation of the three-dimensional measurement system 100.

[0105] As shown in Figure 19, the three-dimensional measurement system 100 acquires three-dimensional reconstruction information as a result of shape measurement of the target workpiece 111 (step S605).

[0106] After step S605, the three-dimensional measurement system 100 identifies the purpose of the three-dimensional reconstruction information acquired in step S605 and determines whether the reconstruction information is for shape inspection (step S610). Here, we will explain assuming that the purpose of the reconstruction information is either shape inspection of the target workpiece 111 or picking (gripping) the target workpiece 111.

[0107] If the determination in step S610 determines that the purpose of the restoration information is shape inspection of the target workpiece 111 ("Yes"), the three-dimensional measurement system 100 performs shape determination of the target workpiece 111 (step S615). In this case, the three-dimensional measurement system 100 outputs the shape determination information of the target workpiece 111 to the outside. On the other hand, if the determination in step S610 determines that the purpose of the restoration information is not shape inspection of the target workpiece 111 ("No"), the three-dimensional measurement system 100 performs picking (grasping) of the target workpiece 111 (step S620). In this case, the three-dimensional measurement system 100 acquires gripping information of the target workpiece 111 (position and orientation information, gripping position information, etc.) using general three-dimensional matching processing, etc., and outputs the gripping information of the target workpiece 111, such as position and orientation information or gripping position information, to the outside.

[0108] In the case of a multi-camera configuration, the camera assignment should be taken into consideration in addition to the division area. For example, groups of workpieces with different workpiece distances can be assigned to the respective near-range and far-range cameras. During the picking operation, if the workpiece distance of a bulk-stacked workpiece becomes too far and exceeds a certain threshold, shape measurement can be performed using the far-range camera. In addition, margins should be provided at the boundaries of multiple projection patterns to prevent one area from interfering with another. Furthermore, the camera position should be determined to minimize the effects of shadows and blind spots by considering the differences in workpiece distances of the target workpieces.

[0109] <Main features of the 3D measurement system, 3D measurement method, and program> The three-dimensional measurement system 100 according to this embodiment can be configured to have the following features. (1) The three-dimensional measurement system 100 according to this embodiment is a system that measures the three-dimensional shape of a target workpiece by capturing an irradiated area of ​​a predetermined area, irradiated in a predetermined pattern, with a two-dimensional light receiving sensor 21. As shown in Figure 1, the three-dimensional measurement system 100 has an irradiated area division unit 32, an imaging area division unit 33, an area association unit 34, an assignment unit 35, and a shape measurement calculation unit 36. The irradiated area division unit 32 is a component that divides the irradiated area into two or more parts and irradiates each part with different patterns. The imaging area division unit 33 is a component that divides the imaging area into two or more parts. The area association unit 34 is a component that associates the divided areas of the irradiated area with the corresponding divided areas of the imaging area. The assignment unit 35 is a component that assigns one of several measurement methods to each associated divided area. The shape measurement calculation unit 36 ​​is a component that measures the shape of the target workpiece according to the measurement method assigned to each divided area.

[0110] In the three-dimensional measurement system 100 according to this embodiment, the shape measurement calculation unit 36 ​​measures the shape of the target workpiece according to the measurement method assigned to each divided region. This three-dimensional measurement system 100 according to this embodiment can suitably measure the shape of the target workpiece 111 even in environments where production methods are consolidated and mixed. Furthermore, the three-dimensional measurement system 100 can perform shape measurement as if it were based on images of multiple angles captured by multiple cameras 20 by dividing the image of one angle captured by one camera 20 into regions. Therefore, when attempting to measure the shape of the target workpiece 111 using multiple different measurement methods, the three-dimensional measurement system 100 does not need to prepare images of multiple camera angles captured by multiple cameras 20. In addition, the three-dimensional measurement system 100 does not need to change the system specifications for each changing production system, and can perform shape measurement of the target workpiece 111 with a single general-purpose system. Furthermore, since the three-dimensional measurement system 100 does not need to prepare separate equipment at the production site, it is possible to consolidate production processes and achieve efficient use of occupied space.

[0111] (2) As shown in Figure 2, in the three-dimensional measurement system 100 of item (1) above, the assignment unit 35 assigns an arbitrary measurement method to each divided region, thereby enabling the simultaneous use of different measurement methods for each divided region according to the state of the target workpiece.

[0112] The three-dimensional measurement system 100 according to this embodiment can simultaneously utilize different measurement methods depending on the state of the target workpiece 111.

[0113] (3) As shown in Figure 5, in the three-dimensional measurement system 100 described in item (1) above, the measurement methods associated with each divided region within the imaging area are two or more different measurement methods.

[0114] The three-dimensional measurement system 100 according to this embodiment can utilize two or more different measurement methods simultaneously.

[0115] (4) As shown in Figure 1, the three-dimensional measurement system 100 described in item (1) above includes an image capture unit (camera 20), an image display unit (display 18), and a reception unit 19. The image capture unit (camera 20) is a component that captures images by independently operating a two-dimensional light receiving sensor 21 that receives light. The image display unit (display 18) is a component that displays the images captured by the image capture unit. The reception unit 19 is a component that receives instructions from the user. The illumination area division unit 32 and the shooting area division unit 33 can determine the division area in response to instructions from the user via the reception unit 19.

[0116] The three-dimensional measurement system 100 according to this embodiment can determine a divided region in response to instructions from the user and perform shape measurement of the target workpiece 111.

[0117] (5) As shown in Figure 1, the three-dimensional measurement system 100 described in item (1) above further includes an object recognition unit 40 that recognizes target objects in the captured image. The illumination area division unit 32 and the shooting area division unit 33 can identify the division areas of the illumination area and the light-receiving area according to the recognition result of the target object.

[0118] The three-dimensional measurement system 100 according to this embodiment can identify the divided regions of the irradiation area and the light receiving area according to the recognition result of the target object, and perform shape measurement of the target workpiece 111.

[0119] (6) As shown in Figure 1, the three-dimensional measurement system 100 described in item (1) above includes an object height identification unit 41, a position identification unit 42, and a shadow area identification unit 43. The object height identification unit 41 is a component that identifies the heights of multiple target objects within the same field of view. The position identification unit 42 is a component that identifies the relative position of the boundary of the divided area and the irradiated light. The shadow area identification unit 43 is a component that identifies the shadow area in the divided area that is further away in height from the divided area. The shape measurement calculation unit 36 ​​performs shape measurement at a relative position that satisfies the following condition 1.

[0120] The three-dimensional measurement system 100 according to this embodiment can perform a suitable shape measurement of the target workpiece 111 by performing shape measurement at a relative position that satisfies the above condition 1.

[0121] (7) In the three-dimensional measurement system 100 described in item (6) above, the shape measurement calculation unit 36 ​​performs shape measurement at a relative position that satisfies the following second condition or the following third condition, with the first condition being a prerequisite. The second condition is that the area within the field of view that is illuminated by light is 90% or more of the maximum value. The third condition is that the shooting area is 90% or more of the maximum value.

[0122] The three-dimensional measurement system 100 according to this embodiment can perform suitable shape measurement of the target workpiece 111 by performing shape measurement at a relative position that satisfies condition 2 or condition 3, provided that condition 1 is met as a prerequisite.

[0123] (8) As shown in Figure 12, the three-dimensional measurement system 100 described in item (6) above further includes a distance determination unit 44 that determines the physical distance between multiple objects located within the same field of view. The shape measurement calculation unit 36, assuming condition 1 above, performs shape measurement within a range in which the shadow of one divided region does not affect the other divided region, so as to satisfy the following fourth condition. The fourth condition is that the length of the shadow of the object on the side where the work distance from the two-dimensional light receiving sensor 21 to the object is farther than a predetermined margin.

[0124] The three-dimensional measurement system 100 according to this embodiment can perform suitable shape measurement of the target workpiece 111 by performing shape measurement at a relative position that satisfies condition 4, provided that condition 1 is a prerequisite.

[0125] (9) As shown in Figure 8B, in the three-dimensional measurement system 100 described in item (1) above, the irradiation area division unit 32 recognizes the work area and does not form an irradiation pattern in a certain area of ​​its boundary.

[0126] The three-dimensional measurement system 100 according to this embodiment can suppress interference between the light-receiving area and the irradiated area by not forming an irradiation pattern in a certain area at the boundary of the work area.

[0127] (10) As shown in Figure 1, the three-dimensional measurement system 100 described in item (1) above includes a divided region display control unit 46 and a method candidate display control unit 47. The divided region display control unit 46 is a component that displays a selectable divided region on the display 18. The method candidate display control unit 47 is a component that displays a selectable shape measurement method candidate on the display 18. The assignment unit 35 assigns a shape measurement method to a divided region in response to instructions from the user.

[0128] In this embodiment, the three-dimensional measurement system 100 can assign a suitable shape measurement method to a divided region by having the user instruct (specify) a suitable shape measurement method from among the candidate shape measurement methods that can be selected on the display 18.

[0129] (11) In the three-dimensional measurement system 100 described in item (5) above, the object recognition unit 40 estimates the object attributes of the divided region. The assignment unit 35 assigns a measurement method to the divided region according to the estimation result of the object attributes.

[0130] The three-dimensional measurement system 100 according to this embodiment can estimate the object attributes of a divided region and assign a measurement method to the divided region according to the estimation result of the object attributes.

[0131] (12) The three-dimensional measurement system 100 described in item (5) above has an imaging control unit 52 that switches between a plurality of two-dimensional light receiving sensors 21. The object recognition unit 40 identifies the height difference between the divided regions. The imaging control unit 52 selects a two-dimensional light receiving sensor 21 that is in focus on the object (target workpiece 111) for each divided region.

[0132] The three-dimensional measurement system 100 according to this embodiment identifies the height difference between divided regions and selects a two-dimensional light receiving sensor 21 that is in focus on the object for each divided region according to the height difference. Since such a three-dimensional measurement system 100 can automatically select a suitable two-dimensional light receiving sensor 21, it can improve the shape measurement accuracy of the target workpiece 111.

[0133] (13) The three-dimensional measurement system 100 described in item (6) above has an imaging control unit 52 that switches between a plurality of two-dimensional light receiving sensors 21. The object height determination unit 41 detects changes in the height of the object. When the height becomes above or below a certain distance, the imaging control unit 52 switches to a two-dimensional light receiving sensor 21 that is in focus on the object (target workpiece 111).

[0134] In this embodiment, the three-dimensional measurement system 100 selects a two-dimensional light receiving sensor 21 that focuses on the object when the height of the object becomes above or below a certain distance. Since such a three-dimensional measurement system 100 can automatically select a suitable two-dimensional light receiving sensor 21, the accuracy of shape measurement of the target workpiece 111 can be improved.

[0135] (14) As shown in Figure 1, the three-dimensional measurement system 100 described in item (1) above includes a storage unit 70 that stores association information 74 between divided regions and measurement methods, and a content display control unit 48 that displays the contents of the stored association information 74. The region association unit 34 updates the contents of the association information 74 stored in the storage unit 70 according to the user's instructions.

[0136] The three-dimensional measurement system 100 according to this embodiment updates the contents of the association information 74 stored in the memory unit 70 in accordance with the user's instructions. Such a three-dimensional measurement system 100 can arbitrarily update the contents of the association information 74 according to its operation.

[0137] (15) In the three-dimensional measurement system 100 described in item (5) above, the object recognition unit 40 detects that the target object has moved during system operation. The region association unit 34 identifies the moved region and sets a new divided region.

[0138] In this embodiment, the three-dimensional measurement system 100 can set up new division regions when the target object (target workpiece 111) moves during system operation. Such a three-dimensional measurement system 100 can accurately measure the shape of the target object even when the target object moves.

[0139] (16) As shown in Figure 14, the three-dimensional measurement method according to this embodiment includes steps S110, S115, S120, and S125. In step S110, the irradiation area is divided into two or more parts, and irradiation light is irradiated to each part in a different pattern (step S505 in Figure 18). Also in step S110, the imaging area is divided into two or more parts (step S515 in Figure 18). In step S115, the divided areas of the irradiation area are associated with the corresponding divided areas of the imaging area. In step S120, one of several measurement methods is assigned to each associated divided area. In step S125, the shape of the target workpiece is measured according to the measurement method assigned to each divided area.

[0140] In the three-dimensional measurement method according to this embodiment, in step S125, the shape of the target workpiece is measured according to the measurement method assigned to each divided region. This three-dimensional measurement method according to this embodiment is suitable for measuring the shape of the target workpiece 111 even in environments where production methods are consolidated and mixed.

[0141] (17) As shown in Figure 14, the control program 30pr according to this embodiment causes the computer to perform the steps S110, S115, S120, and S125. In step S110, the irradiation area is divided into two or more sections, and the irradiation light is irradiated to each section in a different pattern (step S505 in Figure 18). Also in step S110, the imaging area is divided into two or more sections (step S515 in Figure 18). In step S115, the divided sections of the irradiation area and the divided sections of the light-receiving area are associated with each other. In step S120, one of several measurement methods is assigned to each associated divided section. In step S125, the shape of the target workpiece is measured according to the measurement method assigned to each divided section.

[0142] The control program 30pr according to this embodiment can realize the three-dimensional measurement system 100 according to this embodiment.

[0143] As described above, the three-dimensional measurement system 100 according to this embodiment can suitably measure the shape of a target workpiece even in an environment where production methods are consolidated and mixed.

[0144] It should be noted that the present invention is not limited to the embodiments described above, and various modifications and variations can be made without departing from the spirit of the invention.

[0145] For example, the embodiments described above are explained in detail to make the gist of the present invention easier to understand. Therefore, the present invention is not necessarily limited to having all the components described. Furthermore, the present invention can be modified by adding other components to one component, or by changing some components to other components. Furthermore, the present invention can be modified by deleting some components. [Explanation of symbols]

[0146] 10. Projector (light source) 18. Display (image display area) 19. Reception area (keyboard, mouse, etc.) 20. Camera (Image Capture Section) 20a Camera 1 20b Second camera 21 Two-dimensional light receiving sensor 26. Robot arm (movable part) 30. Three-dimensional measuring device (computer) 30pr control program (program) 31 Control Unit 32 Irradiation area division part 33. Shooting area division section 34. Region association section 35 Allocation Section 36 Shape Measurement Calculation Unit 40 Object Recognition Unit 41 Object height determination section 42 Location identification part 43 Shadow area identification part 44 Distance identification part 46-Divided Area Display Control Unit 47 Method Candidate Display Control Unit 48 Content Display Control Unit 50 Post-processing 51 Projection Control Unit 52 Imaging control unit 70 Storage section 71. Shooting Area Information 72 Projection area information 73 Projection Pattern Information 74 Related Information 75. Allocation Information 90 Storage medium 100 Three-Dimensional Measurement Systems 101,101a 1st area 102,102aa,102ab 2nd area 103 Interference Prevention Area 106 Belt conveyor 107 buckets 111 Target Work 113 Margin Area

Claims

1. A three-dimensional measurement system that measures the three-dimensional shape of a target workpiece by capturing an irradiated area of ​​a predetermined size, irradiated in a predetermined pattern, with a two-dimensional light receiving sensor, The irradiation area is divided into two or more sections, and each section is irradiated with light in a different pattern; A shooting area division unit that divides the shooting area into two or more parts, A region association unit that associates the corresponding divided region of the imaging region with the region obtained by dividing the irradiation region, An assignment unit assigns one of several measurement methods to each associated divided region, The system includes a shape measurement calculation unit that performs shape measurement of a target workpiece according to the measurement method assigned to each of the divided regions. Three-dimensional measurement system.

2. The assignment unit allows for the simultaneous use of different measurement methods for each divided region, depending on the state of the target workpiece, by assigning an arbitrary measurement method to each divided region. The three-dimensional measurement system according to claim 1.

3. The measurement method associated with each of the divided regions within the aforementioned imaging area is two or more different measurement methods. The three-dimensional measurement system according to claim 1.

4. An image capture unit that independently operates a two-dimensional light-receiving sensor to capture an image, An image display unit that displays the image captured by the aforementioned image capture unit, It has a reception area that receives user instructions, The irradiation area division unit and the imaging area division unit determine the division area according to instructions from the user. The three-dimensional measurement system according to claim 1.

5. Furthermore, it has an object recognition unit that recognizes target objects in the captured image. The irradiation area division unit and the imaging area division unit identify the division areas of the irradiation area and the imaging area according to the recognition result. The three-dimensional measurement system according to claim 1.

6. An object height identification unit that identifies the heights of multiple target objects within the same field of view, A position identification unit that identifies the relative position of the boundary of the divided region and the irradiated light, It has a shadow region identification unit that identifies the shadow region in the division region that is farther in height from the division region adjacent to the aforementioned division region, The shape measurement calculation unit performs shape measurement at a relative position that satisfies the first condition that the target workpiece is within the camera's field of view. The three-dimensional measurement system according to claim 1.

7. The shape measurement calculation unit, based on the first condition, performs shape measurement at a relative position that satisfies the second condition, which is that the area within the field of view where the irradiated light reaches is 90% or more of the maximum value, or the third condition, which is that the shooting area is 90% or more of the maximum value. The three-dimensional measurement system according to claim 6.

8. Furthermore, it has a distance determination unit that determines the physical distance between multiple objects that exist within the same field of view. The shape measurement calculation unit, based on the first condition, performs shape measurement within a range in which the shadow of one divided region does not affect the other divided region, such that the length of the shadow of the object on the side where the work distance from the two-dimensional light receiving sensor to the object is farther is less than or equal to a predetermined margin, thus satisfying the fourth condition. The three-dimensional measurement system according to claim 6.

9. The irradiation area division unit recognizes the work area and does not form an irradiation pattern in a certain area at the boundary of the work area. The three-dimensional measurement system according to claim 1.

10. A divided region control display unit that displays the divided region, It includes a method candidate display control unit that displays candidate shape measurement methods, The assignment unit assigns a shape measurement method to the divided region in accordance with instructions from the user. The three-dimensional measurement system according to claim 1.

11. The object recognition unit estimates the object attributes of the divided region, The allocation unit assigns a measurement method to the divided region according to the estimation result of the object attributes. The three-dimensional measurement system according to claim 5.

12. It has an imaging control unit that switches between multiple two-dimensional light receiving sensors, The object recognition unit identifies the difference in height between the divided regions, The imaging control unit selects a sensor that focuses on the object for each divided region. The three-dimensional measurement system according to claim 5.

13. It has an imaging control unit that switches between multiple two-dimensional light receiving sensors, The object height determination unit detects changes in the object's height, The imaging control unit switches to a sensor that focuses on the object when the height becomes above or below a certain distance. The three-dimensional measurement system according to claim 6.

14. A storage unit that stores information relating the divided region to the measurement method, It includes a content display control unit that displays the contents of stored association information, The aforementioned region association unit updates the contents of the stored association information in accordance with the user's instructions. The three-dimensional measurement system according to claim 1.

15. The object recognition unit detects that the target object has moved during system operation. The aforementioned region association unit identifies the moved region and sets up a new divided region. The three-dimensional measurement system according to claim 5.

16. The steps include dividing the irradiation area into two or more sections and irradiating each section with different patterns of light, The steps include dividing the shooting area into two or more parts, The steps include associating the corresponding divided region of the imaging region with the region obtained by dividing the irradiation region, For each associated divided region, the step is to assign one of several measurement methods, The process includes the step of measuring the shape of the target workpiece according to the measurement method assigned to each of the divided regions. Three-dimensional measurement method.

17. On the computer, A procedure for dividing the irradiation area into two or more parts and irradiating each part with different patterns of light. Procedure for dividing the shooting area into two or more parts, A procedure for relating corresponding regions of the divided regions of the irradiated area and the divided regions of the light-receiving area. A procedure for assigning one of several measurement methods to each associated segmented region. A procedure for measuring the shape of a target workpiece according to the measurement method assigned to each divided region. A program to execute.

Images