Three-dimensional photographing apparatus and photographing condition adjustment method

Abstract

A three-dimensional photographing device (1) includes: at least one camera (2) that acquires a two-dimensional image and distance information of an object (O); a monitor that displays the two-dimensional image acquired by the camera (2); and one or more processors including hardware. The processor acquires a first region that does not require distance information in the two-dimensional image displayed on the monitor, and sets the following photographing conditions: the amount of distance information acquired by the camera (2) in the acquired first region becomes equal to or less than a predetermined first threshold, and the amount of distance information acquired by the camera (2) in at least a part of a second region outside the first region is more than a predetermined second threshold, where the second threshold is greater than the first threshold.

Description

Technical Field

[0001] The present invention relates to a three-dimensional photographing apparatus and a photographing condition adjustment method. Background Art

[0002] A known object extraction apparatus uses a three-dimensional photographing apparatus to measure the distance to an object and extracts the object using a robot (for example, see Patent Document 1).

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-246631 Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] In order to obtain the three-dimensional shape of an object in detail, the following operation is usually performed: switching photographing conditions such as the exposure time or the amount of light, thereby adjusting to a photographing condition that can obtain more distance information. In this case, for example, in the case of two or more objects such as cardboard boxes having the same shape and being closely arranged, if a three-dimensional camera with low resolution is not used, it may be difficult to identify the boundary of the object in the obtained three-dimensional image. Therefore, it is desired to more clearly identify a difficult-to-identify area such as the boundary of an object even when using a three-dimensional camera with relatively low resolution by simple adjustment.

[0008] Means for Solving the Problems

[0009] One embodiment of the present invention is a three-dimensional photographing apparatus including: at least one camera that acquires a two-dimensional image and distance information of an object; a monitor that displays the two-dimensional image acquired by the camera; and one or more processors including hardware, the processor acquires a first region that does not require the distance information in the two-dimensional image displayed on the monitor, and sets a photographing condition such that the amount of the distance information acquired by the camera in the acquired first region becomes equal to or less than a predetermined first threshold, and the amount of the distance information acquired by the camera in at least a part of a second region outside the first region is more than a predetermined second threshold, the second threshold being greater than the first threshold. Brief Description of the Drawings

[0010] Figure 1 is a perspective view showing an example of a robot system of a three-dimensional photographing apparatus to which one embodiment of the present invention is applied.

[0011] Figure 2 is showing Figure 1Block diagram of the three-dimensional camera of the three-dimensional photographing apparatus.

[0012] Figure 3 It is to explain Figure 1 Block diagram of the control device of the three-dimensional photographing apparatus.

[0013] Figure 4 It is to show the use of Figure 1 Flowchart of the photographing condition adjustment method of the three-dimensional photographing apparatus.

[0014] Figure 5 It is to show in Figure 1 In the three-dimensional photographing apparatus, a diagram showing an example of designating a first area and a second area on the image displayed on the monitor. Detailed implementation mode

[0015] Hereinafter, with reference to the drawings, a three-dimensional photographing apparatus 1 and a photographing condition adjustment method according to an embodiment of the present invention will be described.

[0016] As Figure 1 shown, the three-dimensional photographing apparatus 1 of the present embodiment is, for example, a device that acquires distance information of an object O taken out by a robot 100. The three-dimensional photographing apparatus 1 includes: a three-dimensional camera (camera) 2 that photographs the object O to acquire a two-dimensional image and distance information; and a control device 3 into which the acquired two-dimensional image and distance information are input. In Figure 1 the figure, reference numeral T is a hand that is attached to the front end of the wrist 110 of the robot 100 and can hold the object O.

[0017] As Figure 2 shown, the three-dimensional camera 2 includes, for example: two two-dimensional cameras 4 and 5 that are arranged at intervals in a direction orthogonal to the optical axis A; a projector (lighting device) 6 that irradiates the object O with illumination light of a predetermined pattern; and a control unit (processor) 7. The three-dimensional camera 2 is, for example, downwardly provided above the object O using a bracket (not shown). Instead, the three-dimensional camera 2 may be mounted on the wrist 110 of the robot 100.

[0018] The three-dimensional camera 2 acquires two two-dimensional images obtained by photographing the object O irradiated with illumination light of a predetermined pattern by the projector 6 from different directions using the two two-dimensional cameras 4 and 5, respectively. The control unit 7 is composed of a processor, receives instructions from the control device 3 side to operate the two-dimensional cameras 4 and 5 and the projector 6, and calculates the distance information of the object O based on the two two-dimensional images acquired by the two-dimensional cameras 4 and 5.

[0019] The method for calculating the distance information is performed as follows, for example: small regions with consistent patterns in two two-dimensional images are explored, and the parallax is obtained based on the difference in the positions of the explored small regions on the images. The farther the distance from the three-dimensional camera 2 to the object, the smaller the parallax, and the closer the distance, the larger the parallax. Utilizing this, the parallax at each position on the image is converted into a distance, thereby calculating the distance information of the object O.

[0020] Small regions with consistent patterns in two two-dimensional images are explored. For example, the matching score is calculated using the SAD (Sum of Absolute Difference) method, and the small region with the highest matching score (the sum of the absolute values of the differences in the luminance values of each pixel is the smallest) is explored.

[0021] As Figure 3 As shown, the control device 3 is a computer, which includes: one or more processors 8 including hardware, a memory 9, a monitor 10, and an input device 11 such as a keyboard or a mouse for user input.

[0022] Any one of the two-dimensional images obtained by the three-dimensional camera 2 and the distance information are input to the control device 3.

[0023] The processor 8 causes the input two-dimensional image to be displayed on the monitor 10. In the displayed two-dimensional image, the input device 11 is used to specify a first region X where distance information does not need to be obtained, and a second region Y where distance information is desired to be obtained. The specification of the first region X and the second region Y is performed, for example, by surrounding the region with a mouse or by moving the cursor on the screen using the keyboard.

[0024] As shooting conditions for obtaining distance information, one or more adjustable parameters and thresholds are stored in the memory 9. As parameters, examples include the exposure times of the two-dimensional cameras 4 and 5, the light amount of the projector 6, the matching score calculated in pattern matching, and the size of the small region used in pattern matching, etc.

[0025] The threshold values are a first threshold Th1 representing the upper limit of the amount L1 of the distance information obtained in the first region X, and a second threshold Th2 representing the lower limit of the amount L2 of the distance information obtained in the second region Y. The second threshold Th2 is greater than the first threshold Th1.

[0026] The processor 8 switches the parameters of the shooting conditions stored in the memory 9 and sends an instruction to the three-dimensional camera 2 to cause it to obtain distance information. Then, the processor 8 explores the following shooting conditions: the amount L1 of the distance information obtained in the first region X becomes below the first threshold Th1, and the amount L2 of the distance information obtained in the second region Y is more than the second threshold Th2.

[0027] Hereinafter, a method for adjusting shooting conditions of the three-dimensional shooting device 1 of the present embodiment configured as such will be described.

[0028] In the shooting condition adjustment method of the present embodiment, as a parameter of the shooting condition, for example, the case of adjusting only the exposure time will be described.

[0029] First, as Figure 4 shown, an object O is arranged in the fields of view of two two-dimensional images (step S1), and without operating the projector 6, either one of the two-dimensional cameras 4 and 5 is used to obtain a two-dimensional image (step S2).

[0030] The processor 8 displays the two-dimensional image of the obtained object O on the monitor 10 (step S3), and a state is achieved in which the user can specify a first area X that does not require distance information and a second area Y for which distance information is to be obtained.

[0031] In this state, the user specifies the first area X and the second area Y on the displayed two-dimensional image (step S4).

[0032] As Figure 5 shown, in the case where the object O is two identical-shaped cardboard boxes arranged adjacent to each other, the first area X is the inside of the rectangle specified by surrounding the boundary line of the two cardboard boxes serving as the object O with a rectangle (represented by hatched lines in the figure). In addition, the specification of the second area Y is, for example, by surrounding the entire upper surface area of the two cardboard boxes serving as the object O arranged adjacent to each other with a rectangle and specifying the inside of the rectangle. In the figure, the reference numeral P is the tape for packaging the cardboard box.

[0033] After the first area X and the second area Y are specified, the processor 8 initializes the counter n (step S5) and sets the parameter P(n) of the shooting condition for obtaining distance information (step S6). Here, the parameter P(n) is the exposure time that increases by a predetermined interval Δt in sequence as n increases by 1 in sequence from n = 1.

[0034] Then, the processor 8 causes the three-dimensional camera 2 to obtain two two-dimensional images with parallax in a state where the shooting conditions are set (step S7).

[0035] The two two-dimensional images obtained by the two two-dimensional cameras 4 and 5 are processed in an image processing unit (not shown) of the three-dimensional camera 2. Pattern matching is performed using small areas of a set size, and the parallax of the two two-dimensional images is calculated only at the corresponding points that satisfy the set matching score, and distance information is obtained based on the parallax (step S8).

[0036] Then, the distance information obtained in the first region X and the second region Y is sent to the control device 3, and it is determined whether the amount L1 of the distance information obtained in the first region X is equal to or less than a first threshold Th1 (step S9). If the amount L1 of the distance information is more than the first threshold Th1, it is determined whether the counter n is the maximum value n max (step S10).

[0037] If the counter n is not the maximum value n max , the counter n is incremented (step S11), and the process starting from step S6 is repeated. If it is determined in step S9 that the amount L1 of the distance information obtained within the first region X is equal to or less than the first threshold Th1, it is determined whether the amount L2 of the distance information obtained within the second region Y is equal to or less than a second threshold Th2 (step S12). If the amount L2 of the distance information is equal to or less than the second threshold Th2, the process proceeds to step S10.

[0038] If the amount L2 of the distance information is more than the second threshold Th2 in step S12, it is determined whether the amount L2 of the distance information is equal to or less than the maximum value L2 max (step S13). If it is equal to or less than the second threshold Th2, the process proceeds to step S10. If the amount L2 of the distance information is greater than the maximum value L2 max , the counter n is saved as the counter max of the parameter P(n) that maximizes the amount L2 of the distance information (step S14). Then, the maximum value L2 max is updated to the value of the amount L2 of the distance information (step S15), and the process proceeds to step S10.

[0039] In step S10, if the counter n is the maximum value n max , P(max) is set as the shooting condition (step S16), and the process ends.

[0040] Thus, according to the shooting condition adjustment method of the present embodiment, it is possible to set the parameter P(max) in which the amount L1 of the distance information obtained in the first region X is equal to or less than the first threshold Th1 and the amount L2 of the distance information obtained in the second region Y is maximized as the shooting condition. That is, in the present embodiment, by setting the boundary portion between the two cardboard boxes as the object O as the first region X, it is possible to set the shooting condition that suppresses the amount L1 of the distance information obtained in the first region X to be equal to or less than the first threshold Th1. As a result, it is possible to recognize the boundary between the two cardboard boxes as a region where the object O does not exist, and the robot 100 can individually recognize each cardboard box as a separate object O.

[0041] Moreover, it has the following advantages: Once the shooting conditions are set, as long as the same type of cardboard box is processed, the same distance information can be obtained, and two cardboard boxes arranged closely can be clearly distinguished as separate objects O.

[0042] In addition, in the present embodiment, the exposure time is adopted as the shooting condition, but it is not limited thereto, and other shooting conditions can also be adopted, such as at least one of the illumination light quantity of the projector, the size of the score for pattern recognition, and the size of the small area.

[0043] When the first area X where distance information is not required is darker than other areas, by setting the light quantity of the projector 6 to be smaller, it is difficult to obtain distance information from the first area X. When the first area X is brighter than other areas, by setting the light quantity of the projector 6 to be larger, halation can be caused, making it difficult to obtain distance information from the first area X.

[0044] By strictly setting (setting to a smaller value) the score (evaluation criterion) for pattern recognition, it becomes difficult to find the corresponding positions between the first areas X of the two two-dimensional images, making it difficult to obtain distance information from the first area X.

[0045] In addition, if the size of the small area is reduced, it is easy to find the corresponding positions even in places with large shape changes or fine shapes. If the size of the small area is increased, it is difficult to find the corresponding positions in places with large shape changes or fine shapes. On the other hand, false correspondences in the flat part can be reduced.

[0046] In addition, the case where the three-dimensional camera 2 is provided with a control unit (processor) 7 for calculating distance information is illustrated, but the three-dimensional camera 2 can also output two two-dimensional images, and the processor 8 in the control device 3 is used to calculate the distance information.

[0047] In addition, the first area X where distance information is not required and the second area Y where distance information is required are specified in the two-dimensional image displayed on the monitor 10, but it is also possible to only specify the first area X and set the second area Y as the remaining entire two-dimensional image.

[0048] Description of reference numerals:

[0049] 1: Three-dimensional shooting device

[0050] 4, 5: Two-dimensional cameras (cameras)

[0051] 6: Projector (lighting device)

[0052] 7: Control unit (processor)

[0053] 8: Processor

[0054] 10: Monitor

[0055] A: Optical axis

[0056] O: Object

[0057] X: First region

[0058] Y: Second region

[0059] Th1: First threshold

[0060] Th2: Second threshold

Claims

1. A three-dimensional photographing device, characterized in that, Comprising: At least one camera that acquires a two-dimensional image and distance information of an object; A monitor that displays the two-dimensional image acquired by the camera; and One or more processors that include hardware, The processor is configured to, Acquire a first region that does not require the distance information in the two-dimensional image displayed on the monitor, Set the following shooting conditions: the amount of the distance information acquired by the camera in the acquired first region becomes equal to or less than a predetermined first threshold, and the amount of the distance information acquired by the camera in at least a part of a second region outside the first region is more than a predetermined second threshold, and the second threshold is greater than the first threshold.

2. The three-dimensional shooting device according to claim 1, wherein The processor acquires the second region that requires the distance information in the two-dimensional image displayed on the monitor.

3. The three-dimensional shooting device according to claim 1 or 2, wherein The shooting conditions include the exposure time of the camera.

4. The three-dimensional shooting device according to claim 1 or 2, wherein The three-dimensional shooting device is provided with an illumination device for illuminating the object, The shooting conditions include the amount of illumination light for illuminating by the illumination device.

5. The three-dimensional shooting device according to claim 1 or 2, wherein Two of the cameras are arranged at intervals in a direction intersecting the optical axis, The processor calculates the parallax of each part of the two two-dimensional images by matching small regions in the two two-dimensional images acquired by the two cameras, and calculates the distance information based on the calculated parallax.

6. The three-dimensional shooting device according to claim 5, wherein The shooting conditions include an evaluation criterion for the matching.

7. The three-dimensional shooting device according to claim 5, wherein The shooting conditions include the size of the small region.

8. A shooting condition adjustment method, wherein In the two-dimensional image acquired by at least one camera that acquires a two-dimensional image and distance information of an object, a first region that does not require the distance information is specified, Set the following shooting conditions: the amount of the distance information acquired in the specified first region becomes equal to or less than a predetermined first threshold, and the amount of the distance information acquired in at least a part of a second region outside the first region is more than a predetermined second threshold, and the second threshold is greater than the first threshold.

9. The shooting condition adjustment method according to claim 8, wherein In the two-dimensional image displayed on the monitor, the second region that requires the distance information is specified.

10. The shooting condition adjustment method according to claim 8 or 9, wherein The shooting conditions include the exposure time of the camera.

11. The shooting condition adjustment method according to claim 8 or 9, wherein The shooting conditions include the amount of illumination light for illuminating the object.

12. The shooting condition adjustment method according to claim 8 or 9, wherein In two two-dimensional images of the object obtained by the two cameras arranged at intervals in a direction intersecting the optical axis, by matching small regions within the two-dimensional images, the parallax of each part of the two two-dimensional images is calculated, and based on the calculated parallax, the distance information is calculated.

13. The shooting condition adjustment method according to claim 12, wherein: The shooting condition includes an evaluation criterion for the matching.

14. The shooting condition adjustment method according to claim 12, wherein: The shooting condition includes the size of the small region.

Images